# Using inputs\_embeds as input for GPT2 generation\_utils

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-inputs-embeds-as-input-for-gpt2-generation-utils%2F26725%3E%5C
**Category:** 🤗Transformers\
**Created:** [November 26, 2022, 12:20am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-inputs-embeds-as-input-for-gpt2-generation-utils%2F26725 "2022-11-26T00:20:45Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![TheoCoombes](https://avatars.discourse-cdn.com/v4/letter/t/bc8723/32.png) [@TheoCoombes](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FTheoCoombes)%5C
**Post date:** [November 26, 2022, 12:20am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-inputs-embeds-as-input-for-gpt2-generation-utils%2F26725%2F1 "2022-11-26T00:20:45Z")

</div>

Hi everyone 🙂

I am currently working on building a system named ClipCap (CLIP Captions) at LAION, which takes in a CLIP embedding as an input and performs the reverse-dalle task of captioning the image. This system creates inputs\_embeds using the CLIP embedding - so that when it comes to inference, it’s a bit of a pain to handle.

Is it possible to use inputs\_embeds as an input in the generate() method? If not, could someone point me to any resources that could help me to recreate these methods from scratch so that I could use inputs\_embeds as the input?

Many thanks in advance,  
Theo

---

<div class="post-metadata">

**Author:** ![yujianll](https://avatars.discourse-cdn.com/v4/letter/y/cab0a1/32.png) [@yujianll](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fyujianll)%5C
**Post date:** [March 16, 2023, 6:10am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-inputs-embeds-as-input-for-gpt2-generation-utils%2F26725%2F2 "2023-03-16T06:10:09Z")

</div>

Hi, have you found a solution for this? I have the same problem here. Thanks!