# Perform knowledge distillation using accelerate

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fperform-knowledge-distillation-using-accelerate%2F33078%3E%5C
**Category:** 🤗Accelerate\
**Created:** [March 5, 2023, 12:36pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fperform-knowledge-distillation-using-accelerate%2F33078 "2023-03-05T12:36:42Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![allanjie](https://avatars.discourse-cdn.com/v4/letter/a/e99b99/32.png) [@allanjie](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fallanjie)%5C
**Post date:** [March 5, 2023, 12:36pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fperform-knowledge-distillation-using-accelerate%2F33078%2F1 "2023-03-05T12:36:42Z")

</div>

I’m working on knowledge distillation from a relatively large model 11B to 2B model, for example, under DDP environment.

Usually, we will do the following to `prepare` the model

```python
model, optimizer = accelerator.prepare(model, optimizer)

```

However, I’m wondering how we can fit two model in one code base, as we have `teacher_model` and `student_model`.  
I tried something like

```python
teacher_model, student_model, optimizer = accelerator.prepare(teacher_model, student_model, optimizer)

```

but it’s not working.

---

<div class="post-metadata">

**Author:** ![dignity045](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dignity045/32/58401_2.png) [@dignity045](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdignity045)%5C
**Post date:** [August 14, 2025, 10:05am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fperform-knowledge-distillation-using-accelerate%2F33078%2F2 "2025-08-14T10:05:26Z")

</div>

Hey there, sorry in I am replying bit late, did you solved the issue