VIT-B-16 training embeddings

[Holloworker]

You mentioned that you have tried training DuoduoCLIP on a ViT-B/16-based CLIP model. Would you be able to share the training and evaluation image embeddings and text embeddings generated by the ViT-B/16 CLIP model? If you don’t have the ViT-B/16 embeddings available, could you please let me know how you obtained the text descriptions for all the Zero123 renderings? Thanks!

[hanhung]

Hello,

Unfortunately we no longer have the embeddings for ViT-B/16. However, the text captions are from OpenShape. So you should be able to get the raw text captions from their dataset.

Hope that answers your question.

[Holloworker]

Hello! Are the Zero123-rendered training images simply the extracted contents of the four archives in openshape: Objaverse/000-xxx.tar.gz, ShapeNet.tar.gz, 3D-FUTURE.tar.gz, and ABO.tar.gz? For the text, did you use blip_caption, msft_caption, or the provided text field? Also, is the same text description shared across all renderings/views of the same object, or does it vary per image? Finally, do you have a JSON (or similar) mapping between object IDs (or image paths) and their corresponding captions that you could share? Thanks a lot!