data

SOLID and SOLID-RL data

• Self-seeding data. The self-seeding data is available in self-seef directory: data/self-seeds.

• SFT data. SOLID data to be sued for Supervised Fine-tuning (SFT) data for training SOLID-RL:

  1. SFT data, without length-based mixed quality training: Download
  2. SFT data, with length-based mixed quality training: Download

• DPO data. We provide SOLID data to be used as chosen inspired by original DPO work and also to be used rejected inspired by self-play. Please note that based on our experiments:

  1. With length-based mixed quality training (SOLID used as chosen and SFT used as rejected: Download
  2. With length-based mixed quality training (SOLID used as rejected and SFT used as chosen: Download