Meta Llama 2
The full range of open-source and commercially available Meta Llama 2 (70B / 13B / 7B) models.
Meta Llama 2
★ Available in 70B / 13B / 7B options ★
★ Training with 2 mega-tokensData volume ★
★ Supports context lengths of up to 4,096 tokens ★
Applicable Scenarios
It outperforms other open-source large language models in benchmark tests of reasoning, coding ability, and knowledge testing.
70B: Excellent understanding and reasoning skills in structured data, tables, and Markdown; ability to join data tables and use SQL and JSON; and proficiency in various application scenarios of 13B and 7B.
13B: Suitable for Markdown intent analysis + JSON format output, and various applicable scenarios of 7B.
7B: Can be used for marketing slogans and content generation, invitation and email writing, Chinese-English translation, article summarization, de-identification, and chatbot Q&A.
Parameter adjustment
Standard
Full parameter fine-tuning involves fine-tuning all parameters of a pre-trained model using a task-specific dataset, which requires significant GPU computing and storage resources.
PEFT
Parameter-Efficient Fine-Tuning fixes most of the pre-trained parameters and fine-tunes only a small number of additional model parameters, reducing computational and storage costs while approaching the effect of full parameter fine-tuning (LoRA is an applicable method).
LoRA (Low-Rank Adaptation of Large Language Models) fixes the pre-trained model parameters and weights, and adds additional network layer modules. By using a high-dimensional to low-dimensional decomposition matrix module, it updates only the parameters of the network layer modules to simulate the process of full parameter fine-tuning. This significantly reduces the number of parameters required for training, reduces computational and storage resources, and achieves indirect training of large models with fewer parameters, while approaching the effect of full model fine-tuning.