WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions
Can Xu,Qingfeng Sun,Kai Zheng,Xiubo Geng,Pu Zhao,Jiazhan Feng,Chongyang Tao,Qingwei Lin,Daxin Jiang
Training large language models (LLMs) with open-domain instruction following data brings colossal success. However, manually creating such instruction data is very time-consuming and labor-intensive. Moreover, humans may struggle to produce high-complexity instructions. In this paper, we show an avenue for creating large amounts of instruction data with varying levels of complexity using LLM instead of humans. Starting with an initial set of instructions, we use our proposed Evol-Instruct to rewrite them step by step into more complex instructions. Then, we mix all generated instruction data to fine-tune LLaMA. We call the resulting model WizardLM. Both automatic and human evaluations consistently indicate that WizardLM outperforms baselines such as Alpaca (trained from Self-Instruct) and Vicuna (trained from human-created instructions). The experimental results demonstrate that the quality of instruction-following dataset crafted by Evol-Instruct can significantly improve the performance of LLMs.
Discussion
-
https://tripleai.id/newsDefinitely imagine that which you said. Your favorite reason seemed to be on the web the easiest thing to be mindful of. I say to you, I definitely get irked at the same time as people think about worries that they just don't know about. You managed to hit the nail upon the top as well as defined out the entire thing without having side effect , other people could take a signal. Will likely be again to get more. Thank you
Reply