Meta Llama 2
開源商用的 Meta Llama 2 (70B / 13B / 7B) 全系列模型
Applicable Scenarios
在推理、編碼能力和知識測試等方面的基準測試中優於其他開源大語言模型
70B: Excellent understanding and reasoning skills in structured data, tables, and Markdown; ability to join data tables and use SQL and JSON; and proficiency in various application scenarios of 13B and 7B.
13B: Suitable for Markdown intent analysis + JSON format output, and various applicable scenarios of 7B.
7B: Can be used for marketing slogans and content generation, invitation and email writing, Chinese-English translation, article summarization, de-identification, and chatbot Q&A.
Parameter adjustment
Standard
Full parameter fine-tuning involves fine-tuning all parameters of a pre-trained model using a task-specific dataset, which requires significant GPU computing and storage resources.
PEFT
参数高效微调(Parameter-Efficient Fine-Tuning)，固定大部分預訓練參數，僅微調少量額外的模型參數，降低了運算和儲存成本，同時趨近與全量參數微調的效果 (可適用的方法為LoRA)。
LoRA (Low-Rank Adaptation of Large Language Models) fixes the pre-trained model parameters and weights, and adds additional network layer modules. By using a high-dimensional to low-dimensional decomposition matrix module, it updates only the parameters of the network layer modules to simulate the process of full parameter fine-tuning. This significantly reduces the number of parameters required for training, reduces computational and storage resources, and achieves indirect training of large models with fewer parameters, while approaching the effect of full model fine-tuning.