Llama 2
The next generation of our open source large language model
This release includes model weights and starting code for pretrained and fine-tuned Llama language models — ranging from 7B to 70B parameters.
Llama 2 was trained on 40% more data than Llama 1, and has double the context length.
Training Llama-2-chat: Llama 2 is pretrained using publicly available online data. An initial version of Llama-2-chat is then created through the use of supervised fine-tuning. Next, Llama-2-chat is iteratively refined using Reinforcement Learning from Human Feedback (RLHF), which includes rejection sampling and proximal policy optimization (PPO).
Meta and Microsoft have partnered to unveil Llama 2, the open-source successor to their widely-utilized large language model, Llama. This groundbreaking model is designed to enhance the capabilities of AI, offering it free for both research and commercial use. Recognized as the preferred partner, Microsoft is integrating Llama 2 into its Azure AI model catalog, providing developers with robust cloud-native tools and optimization for Windows platforms.
Llama 2 is also accessible through other major providers like AWS and Hugging Face. Dedicated to responsible AI innovation, Meta and Microsoft emphasize transparency and community-oriented development with resources like red-teaming exercises, a transparency schematic, and a responsible use guide. Collaborative initiatives such as the Open Innovation AI Research Community and the Llama Impact Challenge are also part of the rollout, aiming to spur responsible applications of Llama 2 across various sectors.
KEY FEATURES
Llama 2 models are trained on 2 trillion tokens and have double the context length of Llama 1. Llama-2-chat models have additionally been trained on over 1 million new human annotations.
Llama 2 outperforms other open source language models on many external benchmarks, including reasoning, coding, proficiency, and knowledge tests.
Llama-2-chat uses reinforcement learning from human feedback to ensure safety and helpfulness.
Free Access: Llama 2 is available at no cost for both research and commercial endeavors.
Enhanced Partnership: Meta has selected Microsoft as the preferred partner for the Llama 2 model.
Open Source Innovation: Emphasizing an open-source ethos, Meta and Microsoft back community-driven AI advancements.
Comprehensive Support: Resources such as red-teaming, transparency schematicsand a responsible use guide are provided to promote safe and responsible AI usage.
Community Engagement: Initiatives like the Open Innovation AI Research Community and Llama Impact Challenge to drive collective progress in AI development
FAQ
-
Why do Meta and Microsoft believe in an open-source approach for AI?Meta and Microsoft advocate for an open-source approach to facilitate rapid advancements and collective problem-solving in AI development.
-
What measures are being taken to ensure responsible use of Llama 2?Llama 2 features red-teaming exercises, transparency schematics, and a responsible use guide among other resources to ensure safety and responsibility.
-
Where can developers access Llama 2?Llama 2 is available through Microsoft Azure, AWS, Hugging Face, and other cloud service providers.
