[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"tool-eugeneyan--open-llms":3,"similar-eugeneyan--open-llms":81},{"id":4,"github_repo":5,"name":6,"description_en":7,"description_zh":8,"ai_summary_zh":8,"readme_en":9,"readme_zh":10,"quickstart_zh":11,"use_case_zh":12,"hero_image_url":13,"owner_login":14,"owner_name":15,"owner_avatar_url":16,"owner_bio":17,"owner_company":18,"owner_location":19,"owner_email":17,"owner_twitter":14,"owner_website":20,"owner_url":21,"languages":17,"stars":22,"forks":23,"last_commit_at":24,"license":25,"difficulty_score":26,"env_os":27,"env_gpu":28,"env_ram":29,"env_deps":30,"category_tags":33,"github_topics":36,"view_count":41,"oss_zip_url":17,"oss_zip_packed_at":17,"status":42,"created_at":43,"updated_at":44,"faqs":45,"releases":80},8170,"eugeneyan\u002Fopen-llms","open-llms","📋 A list of open LLMs available for commercial use.","open-llms 是一个精心整理的开源大语言模型清单，专门收录了那些明确允许商业使用的优质模型。在人工智能飞速发展的今天，许多开发者渴望将大模型融入产品，却往往受困于复杂的授权协议，担心面临法律风险。open-llms 正是为了解决这一痛点而生，它清晰地列出了包括 T5、RWKV、Bloom、ChatGLM 等知名模型在内的详细信息，涵盖参数量、上下文长度、发布日期以及具体的开源许可证类型（如 Apache 2.0、MIT 或 OpenRAIL-M）。\n\n这份清单不仅帮助使用者快速甄别哪些模型可以自由用于商业项目，还提供了指向论文、技术博客及在线试用链接的便捷入口。其中特别值得一提的是对 RWKV 等具有独特架构（如无限上下文长度的 RNN 变体）模型的收录，展现了其技术视野的广度。无论是希望降低研发成本的初创企业开发者、需要合规模型进行学术研究的研究人员，还是正在评估技术选型的技术决策者，open-llms 都能提供极具价值的参考。它让获取合法、可靠的开源大模型变得简单透明，助力大家更安心地构建下一代智能应用。","# Open LLMs\n\nThese LLMs (Large Language Models) are all licensed for commercial use (e.g., Apache 2.0, MIT, OpenRAIL-M). Contributions welcome!\n\n| Language Model | Release Date | Checkpoints | Paper\u002FBlog | Params (B) | Context Length | Licence | Try it                                                                                                                |\n| --- | --- | --- | --- | --- | --- | --- |-----------------------------------------------------------------------------------------------------------------------|\n| T5           | 2019\u002F10 |[T5 & Flan-T5](https:\u002F\u002Fgithub.com\u002Fgoogle-research\u002Ft5x\u002Fblob\u002Fmain\u002Fdocs\u002Fmodels.md#flan-t5-checkpoints), [Flan-T5-xxl (HF)](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fflan-t5-xxl)      | [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https:\u002F\u002Fgithub.com\u002Fgoogle-research\u002Ftext-to-text-transfer-transformer#released-model-checkpoints) | 0.06 - 11       | [512](https:\u002F\u002Fdiscuss.huggingface.co\u002Ft\u002Fdoes-t5-truncate-input-longer-than-512-internally\u002F3602) | Apache 2.0         | [T5-Large](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Ft5)                                               |\n| RWKV 4        | 2021\u002F08| [RWKV, ChatRWKV](https:\u002F\u002Fgithub.com\u002FBlinkDL\u002FRWKV-LM#rwkv-parallelizable-rnn-with-transformer-level-llm-performance-pronounced-as-rwakuv-from-4-major-params-r-w-k-v) | [The RWKV Language Model (and my LM tricks)](https:\u002F\u002Fgithub.com\u002FBlinkDL\u002FRWKV-LM)                                           | 0.1 - 14      | [infinity (RNN)](https:\u002F\u002Fgithub.com\u002FBlinkDL\u002FRWKV-LM#rwkv-parallelizable-rnn-with-transformer-level-llm-performance-pronounced-as-rwakuv-from-4-major-params-r-w-k-v) | Apache 2.0         |                                                                                                                       |\n| GPT-NeoX-20B | 2022\u002F04 | [GPT-NEOX-20B](https:\u002F\u002Fhuggingface.co\u002FEleutherAI\u002Fgpt-neox-20b) | [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https:\u002F\u002Farxiv.org\u002Fabs\u002F2204.06745) | 20 | [2048](https:\u002F\u002Fhuggingface.co\u002FEleutherAI\u002Fgpt-neox-20b) | Apache 2.0 |                                                                                                                       |\n| YaLM-100B | 2022\u002F06 | [yalm-100b](https:\u002F\u002Fhuggingface.co\u002Fyandex\u002Fyalm-100b) | [Yandex publishes YaLM 100B, the largest GPT-like neural network in open source](https:\u002F\u002Fyandex.com\u002Fcompany\u002Fpress_center\u002Fpress_releases\u002F2022\u002F2022-23-06) | 100 | [1024](https:\u002F\u002Fgithub.com\u002Fyandex\u002FYaLM-100B\u002Fblob\u002Fmain\u002Fexamples\u002Fgenerate_interactive.sh) | Apache 2.0 | |\n| UL2          | 2022\u002F10 | [UL2 & Flan-UL2](https:\u002F\u002Fgithub.com\u002Fgoogle-research\u002Fgoogle-research\u002Ftree\u002Fmaster\u002Ful2#checkpoints), [Flan-UL2 (HF)](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fflan-ul2)          | [UL2 20B: An Open Source Unified Language Learner](https:\u002F\u002Fai.googleblog.com\u002F2022\u002F10\u002Ful2-20b-open-source-unified-language.html)                                                       | 20             | [512, 2048](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fflan-ul2#tldr) | Apache 2.0         |                                                                                                                       |\n| Bloom | 2022\u002F11 | [Bloom](https:\u002F\u002Fhuggingface.co\u002Fbigscience\u002Fbloom) | [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https:\u002F\u002Farxiv.org\u002Fabs\u002F2211.05100) | 176 | [2048](https:\u002F\u002Fhuggingface.co\u002Fbigscience\u002Fbloom) |  [OpenRAIL-M v1](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fbigcode\u002Fbigcode-model-license-agreement) |                                                                                                                       |\n| ChatGLM | 2023\u002F03\u003C!--13--> | [chatglm-6b](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm-6b) | [ChatGLM](https:\u002F\u002Fzhipuai.cn\u002Fen\u002Fnews\u002F61), [Github](https:\u002F\u002Fgithub.com\u002FTHUDM\u002FChatGLM-6B\u002Fblob\u002Fmain\u002FREADME_en.md) | 6 | [2048](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm-6b\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm-6b\u002Fblob\u002Fmain\u002FMODEL_LICENSE) Free with some usage restriction (might require registration) | |\n| Cerebras-GPT | 2023\u002F03 | [Cerebras-GPT](https:\u002F\u002Fhuggingface.co\u002Fcerebras)                                           | [Cerebras-GPT: A Family of Open, Compute-efficient, Large Language Models](https:\u002F\u002Fwww.cerebras.net\u002Fblog\u002Fcerebras-gpt-a-family-of-open-compute-efficient-large-language-models\u002F) ([Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.03208)) | 0.111 - 13      | [2048](https:\u002F\u002Fhuggingface.co\u002Fcerebras\u002FCerebras-GPT-13B#model-details) | Apache 2.0         | [Cerebras-GPT-1.3B](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Fcerebras-gpt)                            |\n| Open Assistant (Pythia family) | 2023\u002F03 | [OA-Pythia-12B-SFT-8](https:\u002F\u002Fhuggingface.co\u002FOpenAssistant\u002Fpythia-12b-sft-v8-7k-steps), [OA-Pythia-12B-SFT-4](https:\u002F\u002Fhuggingface.co\u002FOpenAssistant\u002Foasst-sft-4-pythia-12b-epoch-3.5), [OA-Pythia-12B-SFT-1](https:\u002F\u002Fhuggingface.co\u002FOpenAssistant\u002Foasst-sft-1-pythia-12b) | [Democratizing Large Language Model Alignment](https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.07327) | 12     | [2048](https:\u002F\u002Fhuggingface.co\u002FOpenAssistant\u002Fpythia-12b-sft-v8-7k-steps\u002Fblob\u002Fmain\u002Fconfig.json)  | Apache 2.0                | [Pythia-2.8B](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Fpythia)                                        |\n| Pythia       | 2023\u002F04 | [pythia 70M - 12B](https:\u002F\u002Fgithub.com\u002FEleutherAI\u002Fpythia)                                   | [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.01373)                                                                    | 0.07 - 12       | [2048](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2304.01373.pdf) | Apache 2.0         |                                                                                                                       |\n| Dolly        | 2023\u002F04 | [dolly-v2-12b](https:\u002F\u002Fhuggingface.co\u002Fdatabricks\u002Fdolly-v2-12b)                            | [Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM](https:\u002F\u002Fwww.databricks.com\u002Fblog\u002F2023\u002F04\u002F12\u002Fdolly-first-open-commercially-viable-instruction-tuned-llm)             | 3, 7, 12     | [2048](https:\u002F\u002Fgithub.com\u002Fdatabrickslabs\u002Fdolly#dolly) | MIT                |                                                                                                                       |\n| StableLM-Alpha | 2023\u002F04 | [StableLM-Alpha](https:\u002F\u002Fgithub.com\u002FStability-AI\u002FStableLM#stablelm-alpha) | [Stability AI Launches the First of its StableLM Suite of Language Models](https:\u002F\u002Fstability.ai\u002Fblog\u002Fstability-ai-launches-the-first-of-its-stablelm-suite-of-language-models) | 3 - 65 | [4096](https:\u002F\u002Fgithub.com\u002FStability-AI\u002FStableLM#stablelm-alpha) | CC BY-SA-4.0 |                                                                                                                       |\n| FastChat-T5 | 2023\u002F04 | [fastchat-t5-3b-v1.0](https:\u002F\u002Fhuggingface.co\u002Flmsys\u002Ffastchat-t5-3b-v1.0) | [We are excited to release FastChat-T5: our compact and commercial-friendly chatbot!](https:\u002F\u002Ftwitter.com\u002Flmsysorg\u002Fstatus\u002F1652037026705985537?s=20) | 3 | [512](https:\u002F\u002Fhuggingface.co\u002Flmsys\u002Ffastchat-t5-3b-v1.0\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 |                                                                                                                       |\n| DLite | 2023\u002F05 | [dlite-v2-1_5b](https:\u002F\u002Fhuggingface.co\u002Faisquared\u002Fdlite-v2-1_5b) | [Announcing DLite V2: Lightweight, Open LLMs That Can Run Anywhere](https:\u002F\u002Fmedium.com\u002Fai-squared\u002Fannouncing-dlite-v2-lightweight-open-llms-that-can-run-anywhere-a852e5978c6e) | 0.124 - 1.5 | [1024](https:\u002F\u002Fhuggingface.co\u002Faisquared\u002Fdlite-v2-1_5b\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0         | [DLite-v2-1.5B](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Fdlite-v2)                                    |\n| h2oGPT | 2023\u002F05 | [h2oGPT](https:\u002F\u002Fgithub.com\u002Fh2oai\u002Fh2ogpt) | [Building the World’s Best Open-Source Large Language Model: H2O.ai’s Journey](https:\u002F\u002Fh2o.ai\u002Fblog\u002Fbuilding-the-worlds-best-open-source-large-language-model-h2o-ais-journey\u002F) | 12 - 20 | [256 - 2048](https:\u002F\u002Fhuggingface.co\u002Fh2oai) | Apache 2.0 |                                                                                                                       |\n| MPT-7B | 2023\u002F05 | [MPT-7B](https:\u002F\u002Fhuggingface.co\u002Fmosaicml\u002Fmpt-7b), [MPT-7B-Instruct](https:\u002F\u002Fhuggingface.co\u002Fmosaicml\u002Fmpt-7b-instruct) | [Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs](https:\u002F\u002Fwww.mosaicml.com\u002Fblog\u002Fmpt-7b) | 7 | [84k (ALiBi)](https:\u002F\u002Fhuggingface.co\u002Fmosaicml\u002Fmpt-7b#how-is-this-model-different) | Apache 2.0, CC BY-SA-3.0 |                                                                                                                       |\n| RedPajama-INCITE | 2023\u002F05 | [RedPajama-INCITE](https:\u002F\u002Fhuggingface.co\u002Ftogethercomputer) | [Releasing 3B and 7B RedPajama-INCITE family of models including base, instruction-tuned & chat models](https:\u002F\u002Fwww.together.xyz\u002Fblog\u002Fredpajama-models-v1) | 3 - 7 | [2048](https:\u002F\u002Fhuggingface.co\u002Ftogethercomputer\u002FRedPajama-INCITE-Instruct-7B-v0.1\u002Fblob\u002F157bf3174feebb67f37e131ea68f84dee007c687\u002Fconfig.json#L13) | Apache 2.0 | [RedPajama-INCITE-Instruct-3B-v1](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Fredpajama-incite-instruct) |\n| OpenLLaMA | 2023\u002F05 | [open_llama_3b](https:\u002F\u002Fhuggingface.co\u002Fopenlm-research\u002Fopen_llama_3b), [open_llama_7b](https:\u002F\u002Fhuggingface.co\u002Fopenlm-research\u002Fopen_llama_7b), [open_llama_13b](https:\u002F\u002Fhuggingface.co\u002Fopenlm-research\u002Fopen_llama_13b) | [OpenLLaMA: An Open Reproduction of LLaMA](https:\u002F\u002Fgithub.com\u002Fopenlm-research\u002Fopen_llama) | 3, 7 | [2048](https:\u002F\u002Fhuggingface.co\u002Fh2oai) | Apache 2.0 | [OpenLLaMA-7B-Preview_200bt](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Fopenllama)                      |\n| Falcon | 2023\u002F05 | [Falcon-180B](https:\u002F\u002Fhuggingface.co\u002Ftiiuae\u002Ffalcon-180B), [Falcon-40B](https:\u002F\u002Fhuggingface.co\u002Ftiiuae\u002Ffalcon-40b), [Falcon-7B](https:\u002F\u002Fhuggingface.co\u002Ftiiuae\u002Ffalcon-7b) | [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.01116) | 180, 40, 7 | [2048](https:\u002F\u002Fhuggingface.co\u002Ftiiuae\u002Ffalcon-7b\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | \n| GPT-J-6B | 2023\u002F06 | [GPT-J-6B](https:\u002F\u002Fgithub.com\u002Fkingoflolz\u002Fmesh-transformer-jax\u002F#gpt-j-6b), [GPT4All-J](https:\u002F\u002Fgithub.com\u002Fnomic-ai\u002Fgpt4all#raw-model) | [GPT-J-6B: 6B JAX-Based Transformer](https:\u002F\u002Farankomatsuzaki.wordpress.com\u002F2021\u002F06\u002F04\u002Fgpt-j\u002F) | 6 | [2048](https:\u002F\u002Fgithub.com\u002Fkingoflolz\u002Fmesh-transformer-jax\u002F#gpt-j-6b) | Apache 2.0 |                                                                                                                       |\n| MPT-30B | 2023\u002F06 | [MPT-30B](https:\u002F\u002Fhuggingface.co\u002Fmosaicml\u002Fmpt-30b), [MPT-30B-instruct](https:\u002F\u002Fhuggingface.co\u002Fmosaicml\u002Fmpt-30b-instruct) | [MPT-30B: Raising the bar for open-source foundation models](https:\u002F\u002Fwww.mosaicml.com\u002Fblog\u002Fmpt-30b) | 30 | [8192](https:\u002F\u002Fhuggingface.co\u002Fmosaicml\u002Fmpt-30b\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0, CC BY-SA-3.0 | [MPT 30B inference code using CPU](https:\u002F\u002Fgithub.com\u002Fabacaj\u002Fmpt-30B-inference) |\n| LLaMA 2  | 2023\u002F06\u003C!--18--> | [LLaMA 2 Weights](https:\u002F\u002Fai.meta.com\u002Fresources\u002Fmodels-and-libraries\u002Fllama-downloads\u002F) | [Llama 2: Open Foundation and Fine-Tuned Chat Models](https:\u002F\u002Fscontent-ham3-1.xx.fbcdn.net\u002Fv\u002Ft39.2365-6\u002F10000000_662098952474184_2584067087619170692_n.pdf?_nc_cat=105&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=qhK-ahCbkBMAX94XV2X&_nc_ht=scontent-ham3-1.xx&oh=00_AfDB7dN8momft9nkv8X0gqrZdEnKltVjPOxhKBm0XLRinA&oe=64BE66FF)      | 7 - 70       | [4096](https:\u002F\u002Fscontent-ham3-1.xx.fbcdn.net\u002Fv\u002Ft39.2365-6\u002F10000000_662098952474184_2584067087619170692_n.pdf?_nc_cat=105&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=qhK-ahCbkBMAX94XV2X&_nc_ht=scontent-ham3-1.xx&oh=00_AfDB7dN8momft9nkv8X0gqrZdEnKltVjPOxhKBm0XLRinA&oe=64BE66FF)  | [Custom](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fllama\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 700M users and you cannot use LLaMA outputs to train other LLMs besides LLaMA and its derivatives   | [HuggingChat](https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fllama2#demo) |  \n| ChatGLM2 | 2023\u002F06\u003C!--25--> | [chatglm2-6b](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm2-6b) | [ChatGLM2-6B](https:\u002F\u002Fzhipuai.cn\u002Fen\u002Fnews\u002F72), [Github](https:\u002F\u002Fgithub.com\u002FTHUDM\u002FChatGLM2-6B) | 6 | [32k](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm2-6b\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm2-6b\u002Fblob\u002Fmain\u002FMODEL_LICENSE) Free with some usage restriction (might require registration) | |\n| XGen-7B | 2023\u002F06\u003C!--28--> | [xgen-7b-4k-base](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fxgen-7b-4k-base), [xgen-7b-8k-base](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fxgen-7b-8k-base)  | [Long Sequence Modeling with XGen](https:\u002F\u002Fblog.salesforceairesearch.com\u002Fxgen\u002F) | 7 | [4096](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fxgen-7b-4k-base), [8192](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fxgen-7b-8k-base) | Apache 2.0 | |\n| Jais-13b | 2023\u002F08\u003C!--17--> | [jais-13b](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-13b), [jais-13b-chat](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-13b-chat) | [Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models](https:\u002F\u002Farxiv.org\u002Fabs\u002F2308.16149) | 13 | [2048](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-13b\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| OpenHermes | 2023\u002F09\u003C!--14--> | [OpenHermes-7B](https:\u002F\u002Fhuggingface.co\u002Fteknium\u002FOpenHermes-7B), [OpenHermes-13B](https:\u002F\u002Fhuggingface.co\u002Fteknium\u002FOpenHermes-13B) | [Nous Research](https:\u002F\u002Fnousresearch.com\u002F) | 7, 13 | [4096](https:\u002F\u002Fhuggingface.co\u002Fteknium\u002FOpenHermes-13B\u002Fblob\u002Fmain\u002Fconfig.json)| MIT | [OpenHermes-V2 Finetuned on Mistral 7B](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fartificialguybr\u002FOPENHERMES-2)\n| OpenLM  | 2023\u002F09\u003C!--26--> | [OpenLM 1B](https:\u002F\u002Fhuggingface.co\u002Fmlfoundations\u002Fopen_lm_1B), [OpenLM 7B](https:\u002F\u002Fhuggingface.co\u002Fmlfoundations\u002Fopen_lm_7B_1.25T) | [Open LM:  a minimal but performative language modeling (LM) repository](https:\u002F\u002Fgithub.com\u002Fmlfoundations\u002Fopen_lm#pretrained-models)      | 1, 7       | [2048](https:\u002F\u002Fgithub.com\u002Fmlfoundations\u002Fopen_lm\u002Fblob\u002Fmain\u002Fopen_lm\u002Fmodel_configs\u002Fopen_lm_7b.json)  | MIT   |      |  \n| Mistral 7B | 2023\u002F09\u003C!--27--> | [Mistral-7B-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-7B-v0.1), [Mistral-7B-Instruct-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-7B-Instruct-v0.1) | [Mistral 7B](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fannouncing-mistral-7b\u002F) | 7 | [4096-16K with Sliding Windows](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-7B-Instruct-v0.1\u002Fblob\u002Fmain\u002Fconfig.json)| Apache 2.0 | [Mistral Transformer](https:\u002F\u002Fgithub.com\u002Fmistralai\u002Fmistral-src)\n| ChatGLM3 | 2023\u002F10\u003C!--27--> | [chatglm3-6b](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b), [chatglm3-6b-base](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b-base), [chatglm3-6b-32k](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b-32k), [chatglm3-6b-128k](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b-128k) | [ChatGLM3](https:\u002F\u002Fgithub.com\u002FTHUDM\u002FChatGLM3\u002Fblob\u002Fmain\u002FREADME_en.md) | 6 | [8192](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b\u002Fblob\u002Fmain\u002Fconfig.json), [32k](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b-32k\u002Fblob\u002Fmain\u002Fconfig.json), [128k](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b-128k\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b\u002Fblob\u002Fmain\u002FMODEL_LICENSE) Free with some usage restriction (might require registration) | |\n| Skywork | 2023\u002F10\u003C!--30--> | [Skywork-13B-Base](https:\u002F\u002Fhuggingface.co\u002FSkywork\u002FSkywork-13B-Base), [Skywork-13B-Math](https:\u002F\u002Fhuggingface.co\u002FSkywork\u002FSkywork-13B-Math) | [Skywork](https:\u002F\u002Fgithub.com\u002FSkyworkAI\u002FSkywork\u002Fblob\u002Fmain\u002FREADME_EN.md) | 13 | [4096](https:\u002F\u002Fhuggingface.co\u002FSkywork\u002FSkywork-13B-base\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fgithub.com\u002FSkyworkAI\u002FSkywork\u002Fblob\u002Fmain\u002FSkywork%20Community%20License.pdf) Free with usage restriction and models trained on Skywork outputs become Skywork derivatives, subject to this license. | |\n| Jais-30b | 2023\u002F11\u003C!--08--> | [jais-30b-v1](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-v1), [jais-30b-chat-v1](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-chat-v1) | [Jais-30B: Expanding the Horizon in Open-Source Arabic NLP](https:\u002F\u002Fg42.ai\u002Fresources\u002Fpublications\u002FJais-30B) | 30 | [2048](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-v1\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| Zephyr  | 2023\u002F11\u003C!--10--> | [Zephyr 7B](https:\u002F\u002Fhuggingface.co\u002FHuggingFaceH4\u002Fzephyr-7b-gemma-v0.1) | [Website](https:\u002F\u002Farxiv.org\u002Fabs\u002F2310.16944) | 7 | [8192](https:\u002F\u002Fhuggingface.co\u002FHuggingFaceH4\u002Fzephyr-7b-gemma-v0.1)| \tApache 2.0 | |\n| DeepSeek | 2023\u002F11\u003C!--30--> | [deepseek-llm-7b-base](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002Fdeepseek-llm-7b-base), [deepseek-llm-7b-chat](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002Fdeepseek-llm-7b-chat), [deepseek-llm-67b-base](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002Fdeepseek-llm-67b-base), [deepseek-llm-67b-chat](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002Fdeepseek-llm-67b-chat) | [Introducing DeepSeek LLM](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002Fdeepseek-LLM),  | 7, 67 | [4096](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002Fdeepseek-LLM) | [Custom](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-LLM\u002Fblob\u002FHEAD\u002FLICENSE-MODEL) Free with usage restriction and models trained on DeepSeek outputs become DeepSeek derivatives, subject to this license.\n| Mistral 7B v0.2 | 2023\u002F12\u003C!--11--> | [Mistral-7B-v0.2](https:\u002F\u002Fhuggingface.co\u002Fmistral-community\u002FMistral-7B-v0.2), [Mistral-7B-Instruct-v0.2](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-7B-Instruct-v0.2) | [La Plateforme](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fla-plateforme\u002F) | 7 | [32k](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-7B-Instruct-v0.2) | Apache 2.0 | |\n| Mixtral 8x7B v0.1 | 2023\u002F12\u003C!--11--> | [Mixtral-8x7B-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMixtral-8x7B-v0.1), [Mixtral-8x7B-Instruct-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMixtral-8x7B-Instruct-v0.1) | [Mixtral of experts](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmixtral-of-experts\u002F) | 46.7 | [32k](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmixtral-of-experts\u002F) | Apache 2.0 | |\n| LLM360 Amber | 2023\u002F12\u003C!--11--> | [Amber](https:\u002F\u002Fhuggingface.co\u002FLLM360\u002FAmber), [AmberChat](https:\u002F\u002Fhuggingface.co\u002FLLM360\u002FAmberChat), [AmberSafe](https:\u002F\u002Fhuggingface.co\u002FLLM360\u002FAmberSafe) | [Introducing LLM360: Fully Transparent Open-Source LLMs](https:\u002F\u002Fwww.llm360.ai\u002Fblog\u002Fintroducing-llm360-fully-transparent-open-source-llms.html) | 6.7 | [2048](https:\u002F\u002Fhuggingface.co\u002FLLM360\u002FAmber#%F0%9F%9F%A0-model-description) | Apache 2.0 | |\n| SOLAR | 2023\u002F12\u003C!--12--> | [Solar-10.7B](https:\u002F\u002Fhuggingface.co\u002Fupstage\u002FSOLAR-10.7B-v1.0) | [Upstage](https:\u002F\u002Farxiv.org\u002Fabs\u002F2312.15166) | 10.7 | [4096](https:\u002F\u002Fhuggingface.co\u002Fupstage\u002FSOLAR-10.7B-v1.0\u002Fblob\u002Fmain\u002Fconfig.json)| apache-2.0 | |\n| phi-2 | 2023\u002F12\u003C!--12--> | [phi-2 2.7B](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002Fphi-2) | [Microsoft](https:\u002F\u002Fwww.microsoft.com\u002Fen-us\u002Fresearch\u002Fblog\u002Fphi-2-the-surprising-power-of-small-language-models\u002F) | 2.7 | [2048](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002Fphi-2\u002Fblob\u002Fmain\u002Fconfig.json)| MIT | |\n| FLOR | 2023\u002F12\u003C!--22--> | [FLOR-760M](https:\u002F\u002Fhuggingface.co\u002Fprojecte-aina\u002FFLOR-760M), [FLOR-1.3B](https:\u002F\u002Fhuggingface.co\u002Fprojecte-aina\u002FFLOR-1.3B), [FLOR-1.3B-Instructed](https:\u002F\u002Fhuggingface.co\u002Fprojecte-aina\u002FFLOR-1.3B-Instructed), [FLOR-6.3B](https:\u002F\u002Fhuggingface.co\u002Fprojecte-aina\u002FFLOR-6.3B), [FLOR-6.3B-Instructed](https:\u002F\u002Fhuggingface.co\u002Fprojecte-aina\u002FFLOR-6.3B-Instructed) |  [FLOR-6.3B: a chinchilla-compliant model for Catalan, Spanish and English](https:\u002F\u002Fmedium.com\u002F@mpamies247\u002Fflor-6-3b-a-chinchilla-compliant-model-for-catalan-spanish-and-english-7cdb389a9aac) | 0.76, 1.3, 6.3 | [2048](https:\u002F\u002Fhuggingface.co\u002Fbigscience\u002Fbloom-1b1#technical-specifications) | Apache 2.0 with usage restriction inherited from BLOOM | |\n| RWKV 5 v2 | 2024\u002F01\u003C!--28--> | [rwkv-5-world-0.4b-2, rwkv-5-world-1.5b-2, rwkv-5-world-3b-2, rwkv-5-world-3b-2(16k), rwkv-5-world-7b-2](https:\u002F\u002Fhuggingface.co\u002FBlinkDL\u002Frwkv-5-world) | [RWKV 5](https:\u002F\u002Fwww.rwkv.com\u002F) | 0.4, 1.5, 3, 7 | [unlimited(RNN), trained on 4096 (and 16k for 3b)](https:\u002F\u002Fhuggingface.co\u002FBlinkDL\u002Frwkv-5-world\u002Ftree\u002Fmain) | Apache 2.0 | | \n| OLMo | 2024\u002F02\u003C!--01--> | [OLMo 1B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-1B), [OLMo 7B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-7B), [OLMo 7B Twin 2T](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-7B-Twin-2T) | [AI2](https:\u002F\u002Fblog.allenai.org\u002Fhello-olmo-a-truly-open-llm-43f7e7359222) | 1,7 | [2048](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-7B-Twin-2T\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| Qwen1.5 | 2024\u002F02\u003C!--04--> | [Qwen1.5-7B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-7B), [Qwen1.5-7B-Chat](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-7B-Chat), [Qwen1.5-14B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-14B), [Qwen1.5-14B-Chat](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-14B-Chat), [Qwen1.5-72B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-72B), [Qwen1.5-72B-Chat](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-72B-Chat) | [Introducing Qwen1.5](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen1.5\u002F) | 7, 14, 72 | [32k](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-7B-Chat\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-7B-Chat\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 100M users and you cannot use Qwen outputs to train other LLMs besides Qwen and its derivatives | |\n| LWM | 2024\u002F02\u003C!--07--> | [LWM-Text-Chat-128K](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-Chat-128K), [LWM-Text-Chat-256K](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-Chat-256K), [LWM-Text-Chat-512K](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-Chat-512K), [LWM-Text-Chat-1M](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-Chat-1M), [LWM-Text-128K](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-128K), [LWM-Text-256K](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-256K), [LWM-Text-512K](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-512K), [LWM-Text-1M](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-1M) | [Large World Model (LWM)](https:\u002F\u002Fgithub.com\u002FLargeWorldModel\u002FLWM) | 7 | [128k, 256k, 512k, 1M](https:\u002F\u002Fgithub.com\u002FLargeWorldModel\u002FLWM#available-models) | LLaMA 2 license | | \n| Jais-30b v3 | 2024\u002F03\u003C!--08--> | [jais-30b-v3](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-v3), [jais-30b-chat-v3](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-chat-v3) | [Jais 30b v3](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-v3) | 30 | [8192](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-v3\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| Gemma | 2024\u002F02\u003C!--21--> | [Gemma 7B](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fgemma-7b), [Gemma 7B it](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fgemma-7b-it), [Gemma 2B](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fgemma-2b), [Gemma 2B it](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fgemma-2b-it) | [Technical report](https:\u002F\u002Fstorage.googleapis.com\u002Fdeepmind-media\u002Fgemma\u002Fgemma-report.pdf) | 2-7 | [8192](https:\u002F\u002Fstorage.googleapis.com\u002Fdeepmind-media\u002Fgemma\u002Fgemma-report.pdf) | [Gemma Terms of Use](https:\u002F\u002Fai.google.dev\u002Fgemma\u002Fterms) Free with usage restriction and models trained on Gemma outputs become Gemma derivatives, subject to this license. | |\n| Grok-1 | 2024\u002F03\u003C!--17--> | [Grok-1](https:\u002F\u002Fhuggingface.co\u002Fxai-org\u002Fgrok-1) | [Open Release of Grok-1](https:\u002F\u002Fx.ai\u002Fblog\u002Fgrok-os) | 314 | [8192](https:\u002F\u002Fgithub.com\u002Fxai-org\u002Fgrok-1\u002Fblob\u002Fmain\u002Frun.py) | Apache 2.0 | |\n| Qwen1.5 MoE | 2024\u002F03\u003C!--28--> | [Qwen1.5-MoE-A2.7B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-MoE-A2.7B), [Qwen1.5-MoE-A2.7B-Chat](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-MoE-A2.7B-Chat) | [Qwen1.5-MoE: Matching 7B Model Performance with 1\u002F3 Activated Parameters](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen-moe\u002F) | 14.3 | [8192](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-MoE-A2.7B\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-MoE-A2.7B\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 100M users and you cannot use Qwen outputs to train other LLMs besides Qwen and its derivatives | |\n| Jamba 0.1 | 2024\u002F03\u003C!--28--> | [Jamba-v0.1](https:\u002F\u002Fhuggingface.co\u002Fai21labs\u002FJamba-v0.1) | [Introducing Jamba: AI21's Groundbreaking SSM-Transformer Model](https:\u002F\u002Fwww.ai21.com\u002Fblog\u002Fannouncing-jamba) | 52 | [256k](https:\u002F\u002Fhuggingface.co\u002Fai21labs\u002FJamba-v0.1\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| Qwen1.5 32B | 2024\u002F04\u003C!--02--> | [Qwen1.5-32B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-32B), [Qwen1.5-32B-Chat](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-32B-Chat) | [Qwen1.5-32B: Fitting the Capstone of the Qwen1.5 Language Model Series](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen1.5-32b\u002F) | 32 | [32k](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-32B-Chat\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-32B-Chat\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 100M users and you cannot use Qwen outputs to train other LLMs besides Qwen and its derivatives | |\n| Mamba-7B | 2024\u002F04\u003C!--15--> | [mamba-7b-rw](https:\u002F\u002Fhuggingface.co\u002FTRI-ML\u002Fmamba-7b-rw) | [Toyota Research Institute](https:\u002F\u002Fhuggingface.co\u002FTRI-ML\u002Fmamba-7b-rw) | 7 | [unlimited(RNN), trained on 2048](https:\u002F\u002Fhuggingface.co\u002FTRI-ML\u002Fmamba-7b-rw) | Apache 2.0 | |\n| Mixtral8x22B v0.1 | 2024\u002F04\u003C!--17--> | [Mixtral-8x22B-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMixtral-8x22B-v0.1), [Mixtral-8x22B-Instruct-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMixtral-8x22B-Instruct-v0.1) | [Cheaper, Better, Faster, Stronger](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmixtral-8x22b\u002F) | 141 | [64k](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmixtral-8x22b\u002F) | Apache 2.0 | |\n| Llama 3 | 2024\u002F04\u003C!--18--> | [Llama-3-8B](https:\u002F\u002Fhuggingface.co\u002Fmeta-llama\u002FMeta-Llama-3-8B), [Llama-3-8B-Instruct](https:\u002F\u002Fhuggingface.co\u002Fmeta-llama\u002FMeta-Llama-3-8B-Instruct), [Llama-3-70B](https:\u002F\u002Fhuggingface.co\u002Fmeta-llama\u002FMeta-Llama-3-70B), [Llama-3-70B-Instruct](https:\u002F\u002Fhuggingface.co\u002Fmeta-llama\u002FMeta-Llama-3-70B-Instruct), [Llama-Guard-2-8B](https:\u002F\u002Fhuggingface.co\u002Fmeta-llama\u002FMeta-Llama-Guard-2-8B) | [Introducing Meta Llama 3](https:\u002F\u002Fai.meta.com\u002Fblog\u002Fmeta-llama-3\u002F), [Meta Llama 3](https:\u002F\u002Fllama.meta.com\u002Fllama3\u002F) | 8, 70 | [8192](https:\u002F\u002Fgithub.com\u002Fmeta-llama\u002Fllama3) | [Meta Llama 3 Community License Agreement](https:\u002F\u002Fgithub.com\u002Fmeta-llama\u002Fllama3\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 700M users and you cannot use LLaMA 3 outputs to train other LLMs besides LLaMA 3 and its derivatives | |\n| Phi-3 Mini | 2024\u002F04\u003C!--23--> | [Phi-3-mini-4k-instruct](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FPhi-3-mini-4k-instruct), [Phi-3-mini-128k-instruct](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FPhi-3-mini-128k-instruct) | [Introducing Phi-3](https:\u002F\u002Fazure.microsoft.com\u002Fen-us\u002Fblog\u002Fintroducing-phi-3-redefining-whats-possible-with-slms\u002F), [Technical Report](https:\u002F\u002Farxiv.org\u002Fabs\u002F2404.14219) | 3.8 | [4096, 128k](https:\u002F\u002Farxiv.org\u002Fabs\u002F2404.14219) | MIT | |\n| OpenELM | 2024\u002F04\u003C!--24--> | [OpenELM-270M](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-270M), [OpenELM-270M-Instruct](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-270M-Instruct), [OpenELM-450M](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-450M), [OpenELM-450M-Instruct](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-450M-Instruct), [OpenELM-1_1B](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-1_1B), [OpenELM-1_1B-Instruct](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-1_1B-Instruct), [OpenELM-3B](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-3B), [OpenELM-3B-Instruct](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-3B-Instruct) | [OpenELM: An Efficient Language Model Family with Open Training and Inference Framework](https:\u002F\u002Fmachinelearning.apple.com\u002Fresearch\u002Fopenelm) | 0.27, 0.45, 1.1, 3 | [2048](https:\u002F\u002Farxiv.org\u002Fhtml\u002F2404.14619v2) | [Custom open license](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-270M\u002Fblob\u002Fmain\u002FLICENSE) No usage or training restrictions\n| Snowflake Arctic | 2024\u002F04\u003C!--24--> | [snowflake-arctic-base](https:\u002F\u002Fhuggingface.co\u002FSnowflake\u002Fsnowflake-arctic-base), [snowflake-arctic-instruct](https:\u002F\u002Fhuggingface.co\u002FSnowflake\u002Fsnowflake-arctic-instruct) | [Snowflake Arctic: The Best LLM for Enterprise AI — Efficiently Intelligent, Truly Open](https:\u002F\u002Fwww.snowflake.com\u002Fblog\u002Farctic-open-efficient-foundation-language-models-snowflake\u002F) | 480 | [4096](https:\u002F\u002Fhuggingface.co\u002FSnowflake\u002Fsnowflake-arctic-base\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| Qwen1.5 110B | 2024\u002F04\u003C!--25--> | [Qwen1.5-110B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-110B), [Qwen1.5-110B-Chat](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-110B-Chat) | [Qwen1.5-110B: The First 100B+ Model of the Qwen1.5 Series](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen1.5-110b\u002F) | 110 | [32k](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-110B-Chat\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-110B-Chat\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 100M users and you cannot use Qwen outputs to train other LLMs besides Qwen and its derivatives | |\n| RWKV 6 v2.1 | 2024\u002F05\u003C!--06--> | [rwkv-6-world-1.6b-2.1, rwkv-6-world-3b-2.1, rwkv-6-world-7b-2.1](https:\u002F\u002Fhuggingface.co\u002FBlinkDL\u002Frwkv-6-world) | [RWKV 6](https:\u002F\u002Fwww.rwkv.com\u002F) | 1.6, 3, 7 | [unlimited(RNN), trained on 4096](https:\u002F\u002Fhuggingface.co\u002FBlinkDL\u002Frwkv-6-world\u002Ftree\u002Fmain) | Apache 2.0 | |\n| DeepSeek-V2 | 2024\u002F05\u003C!--06--> | [DeepSeek-V2](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002FDeepSeek-V2), [DeepSeek-V2-Chat](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002FDeepSeek-V2-Chat) | [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-V2) | 236 | [128k](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-V2) | [Custom](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-V2\u002Fblob\u002Fmain\u002FLICENSE-MODEL) Free with usage restriction and models trained on DeepSeek outputs become DeepSeek derivatives, subject to this license. | |\n| Fugaku-LLM | 2024\u002F05\u003C!--13--> | [Fugaku-LLM-13B](https:\u002F\u002Fhuggingface.co\u002FFugaku-LLM\u002FFugaku-LLM-13B), [Fugaku-LLM-13B-instruct](https:\u002F\u002Fhuggingface.co\u002FFugaku-LLM\u002FFugaku-LLM-13B-instruct) | [Release of \"Fugaku-LLM\" – a large language model trained on the supercomputer \"Fugaku\"](https:\u002F\u002Fwww.titech.ac.jp\u002Fenglish\u002Fnews\u002F2024\u002F069223) | 13 | [2048](https:\u002F\u002Fhuggingface.co\u002FFugaku-LLM\u002FFugaku-LLM-13B\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FFugaku-LLM\u002FFugaku-LLM-13B-instruct\u002Fblob\u002Fmain\u002FLICENSE) Free with usage restrictions | |\n| Falcon 2 | 2024\u002F05\u003C!--13--> | [falcon2-11B](https:\u002F\u002Fhuggingface.co\u002Ftiiuae\u002Ffalcon-11B) | [Meet Falcon 2: TII Releases New AI Model Series, Outperforming Meta’s New Llama 3](https:\u002F\u002Ffalconllm.tii.ae\u002Ffalcon-2.html) | 11 | [8192](https:\u002F\u002Fhuggingface.co\u002Ftiiuae\u002Ffalcon-11B#training-data) | [Custom Apache 2.0](https:\u002F\u002Ffalconllm-staging.tii.ae\u002Ffalcon-2-terms-and-conditions.html) with mild acceptable use policy | |\n| Yi-1.5 | 2024\u002F05\u003C!--15--> | [Yi-1.5-6B](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-6B), [Yi-1.5-6B-Chat](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-6B-Chat), [Yi-1.5-9B](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-9B), [Yi-1.5-9B-Chat](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-9B-Chat), [Yi-1.5-34B](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-34B), [Yi-1.5-34B-Chat](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-34B-Chat) | [Yi-1.5](https:\u002F\u002Fgithub.com\u002F01-ai\u002FYi-1.5) | 6, 9, 34 | [4096](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-6B\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| DeepSeek-V2-Lite | 2024\u002F05\u003C!--16--> | [DeepSeek-V2-Lite](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002FDeepSeek-V2-Lite), [DeepSeek-V2-Lite-Chat](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002FDeepSeek-V2-Lite-Chat) | [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-V2) | 16 | [32k](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-V2) | [Custom](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-V2\u002Fblob\u002Fmain\u002FLICENSE-MODEL) Free with usage restriction and models trained on DeepSeek outputs become DeepSeek derivatives, subject to this license. | |\n| Phi-3 small\u002Fmedium | 2024\u002F05\u003C!--21--> | [Phi-3-mini-4k-instruct](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FPhi-3-mini-4k-instruct), [Phi-3-mini-128k-instruct](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FPhi-3-mini-128k-instruct), [Phi-3-medium-4k-instruct](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FPhi-3-medium-4k-instruct), [Phi-3-medium-128k-instruct](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FPhi-3-medium-128k-instruct) | [New models added to the Phi-3 family, available on Microsoft Azure](https:\u002F\u002Fazure.microsoft.com\u002Fen-us\u002Fblog\u002Fnew-models-added-to-the-phi-3-family-available-on-microsoft-azure\u002F), [Technical Report](https:\u002F\u002Farxiv.org\u002Fabs\u002F2404.14219) | 7, 14 | [4096, 128k](https:\u002F\u002Farxiv.org\u002Fabs\u002F2404.14219) | MIT | |\n| Phi-4 | 2024\u002F12 | [Phi-4](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002Fphi-4)| [Introducing Phi-4: Microsoft’s Newest Small Language Model Specializing in Complex Reasoning](https:\u002F\u002Ftechcommunity.microsoft.com\u002Fblog\u002Faiplatformblog\u002Fintroducing-phi-4-microsoft%E2%80%99s-newest-small-language-model-specializing-in-comple\u002F4357090), [Technical Report](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2412.08905) | 14 | [4096](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2412.08905) | MIT | |\n| YuLan-Mini | 2024\u002F12 | [YuLan-Mini](https:\u002F\u002Fhuggingface.co\u002Fyulan-team\u002FYuLan-Mini) | [YuLan-Mini: An Open Data-efficient Language Model](https:\u002F\u002Farxiv.org\u002Fabs\u002F2412.17743), [GitHub](https:\u002F\u002Fgithub.com\u002FRUC-GSAI\u002FYuLan-Mini) | 14 | [28672](https:\u002F\u002Fgithub.com\u002FRUC-GSAI\u002FYuLan-Mini) | MIT | [YuLan-Mini](https:\u002F\u002Fhuggingface.co\u002Fyulan-team\u002FYuLan-Mini) |\n| Selene Mini | 2025\u002F01 | [Selene Mini](https:\u002F\u002Fhuggingface.co\u002FAtlaAI\u002FSelene-1-Mini-Llama-3.1-8B) | [Atla Selene Mini: A General Purpose Evaluation Model](https:\u002F\u002Farxiv.org\u002Fabs\u002F2501.17195v1), [GitHub](https:\u002F\u002Fgithub.com\u002Fatla-ai\u002Fselene-mini) | 8 | [128K](https:\u002F\u002Fgithub.com\u002Fatla-ai\u002Fselene-mini) | Apache 2.0 | [Hugging Face Space](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FAtlaAI\u002Fselene) |\n\n## Open LLMs for code  \n\n| Language Model | Release Date | Checkpoints | Paper\u002FBlog | Params (B) | Context Length                                                                         | Licence | Try it                                                                                    |\n| --- | --- | --- | --- | --- |----------------------------------------------------------------------------------------| --- |-------------------------------------------------------------------------------------------|\n| SantaCoder | 2023\u002F01 | [santacoder](https:\u002F\u002Fhuggingface.co\u002Fbigcode\u002Fsantacoder) |[SantaCoder: don't reach for the stars!](https:\u002F\u002Farxiv.org\u002Fabs\u002F2301.03988) | 1.1 | [2048](https:\u002F\u002Fhuggingface.co\u002Fbigcode\u002Fsantacoder\u002Fblob\u002Fmain\u002FREADME.md#model-summary)                | [OpenRAIL-M v1](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fbigcode\u002Fbigcode-model-license-agreement) | [SantaCoder](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Fsantacoder)         |\n| CodeGen2 | 2023\u002F04 | [codegen2 1B-16B](https:\u002F\u002Fgithub.com\u002Fsalesforce\u002FCodeGen2) | [CodeGen2: Lessons for Training LLMs on Programming and Natural Languages](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.02309) | 1 - 16 | [2048](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.02309) | [Apache 2.0](https:\u002F\u002Fgithub.com\u002Fsalesforce\u002FCodeGen2\u002Fblob\u002Fmain\u002FLICENSE)|                                                                                           |\n| StarCoder | 2023\u002F05 | [starcoder](https:\u002F\u002Fhuggingface.co\u002Fbigcode\u002Fstarcoder) | [StarCoder: A State-of-the-Art LLM for Code](https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fstarcoder), [StarCoder: May the source be with you!](https:\u002F\u002Fdrive.google.com\u002Ffile\u002Fd\u002F1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs\u002Fview) | 1.1-15 | [8192](https:\u002F\u002Fhuggingface.co\u002Fbigcode\u002Fstarcoder#model-summary)                         | [OpenRAIL-M v1](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fbigcode\u002Fbigcode-model-license-agreement) |                                                                                           |\n| StarChat Alpha | 2023\u002F05 | [starchat-alpha](https:\u002F\u002Fhuggingface.co\u002FHuggingFaceH4\u002Fstarchat-alpha) | [Creating a Coding Assistant with StarCoder](https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fstarchat-alpha) | 16 | [8192](https:\u002F\u002Fhuggingface.co\u002Fbigcode\u002Fstarcoder#model-summary)  | [OpenRAIL-M v1](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fbigcode\u002Fbigcode-model-license-agreement) |                                                                                           |\n| Replit Code | 2023\u002F05 | [replit-code-v1-3b](https:\u002F\u002Fhuggingface.co\u002Freplit\u002Freplit-code-v1-3b) | [Training a SOTA Code LLM in 1 week and Quantifying the Vibes — with Reza Shabani of Replit](https:\u002F\u002Fwww.latent.space\u002Fp\u002Freza-shabani#details) | 2.7 | [infinity? (ALiBi)](https:\u002F\u002Fhuggingface.co\u002Freplit\u002Freplit-code-v1-3b#model-description) | CC BY-SA-4.0 | [Replit-Code-v1-3B](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Freplit-code) |\n| CodeT5+ | 2023\u002F05 | [CodeT5+](https:\u002F\u002Fgithub.com\u002Fsalesforce\u002FCodeT5\u002Ftree\u002Fmain\u002FCodeT5+)     | [CodeT5+: Open Code Large Language Models for Code Understanding and Generation](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.07922) | 0.22 - 16 | [512](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.07922)                                                                                | [BSD-3-Clause](https:\u002F\u002Fgithub.com\u002Fsalesforce\u002FCodeT5\u002Fblob\u002Fmain\u002FLICENSE.txt)                                                           | [Codet5+-6B](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002FcodeT5%2B)          |\n| XGen-7B | 2023\u002F06 | [XGen-7B-8K-Base](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fxgen-7b-8k-base) | [Long Sequence Modeling with XGen: A 7B LLM Trained on 8K Input Sequence Length](https:\u002F\u002Fblog.salesforceairesearch.com\u002Fxgen\u002F) | 7 | [8192](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fxgen-7b-8k-base\u002Fblob\u002Fmain\u002Fconfig.json) | [Apache 2.0](https:\u002F\u002Fgithub.com\u002Fsalesforce\u002Fxgen\u002Fblob\u002Fmain\u002FLICENSE) | \n| CodeGen2.5 | 2023\u002F07 | [CodeGen2.5-7B-multi](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fcodegen25-7b-multi) | [CodeGen2.5: Small, but mighty](https:\u002F\u002Fblog.salesforceairesearch.com\u002Fcodegen25\u002F) | 7 | [2048](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fcodegen25-7b-multi\u002Fblob\u002Fmain\u002Fconfig.json) | [Apache 2.0](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fcodegen25-7b-multi\u002Fblob\u002Fmain\u002FREADME.md) | \n| DeciCoder-1B | 2023\u002F08 | [DeciCoder-1B](https:\u002F\u002Fhuggingface.co\u002FDeci\u002FDeciCoder-1b#how-to-use) | [Introducing DeciCoder: The New Gold Standard in Efficient and Accurate Code Generation](https:\u002F\u002Fdeci.ai\u002Fblog\u002Fdecicoder-efficient-and-accurate-code-generation-llm\u002F) | 1.1 | [2048](https:\u002F\u002Fhuggingface.co\u002FDeci\u002FDeciCoder-1b#model-architecture) | Apache 2.0 |  [DeciCoder Demo](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FDeci\u002FDeciCoder-Demo)|\n| Code Llama  | 2023\u002F08 | [Inference Code for CodeLlama models]([https:\u002F\u002Fai.meta.com\u002Fresources\u002Fmodels-and-libraries\u002Fllama-downloads\u002F](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fcodellama)) | [Code Llama: Open Foundation Models for Code](https:\u002F\u002Fai.meta.com\u002Fresearch\u002Fpublications\u002Fcode-llama-open-foundation-models-for-code\u002F)     | 7 - 34       | [4096](https:\u002F\u002Fscontent-zrh1-1.xx.fbcdn.net\u002Fv\u002Ft39.2365-6\u002F369856151_1754812304950972_1159666448927483931_n.pdf?_nc_cat=107&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=wURKmnWKaloAX-ib5XW&_nc_ht=scontent-zrh1-1.xx&oh=00_AfAN1GB2K_XwIz54PqXTr-dhilI3CfCwdQoaLMyaYEEECg&oe=64F0A68F)  | [Custom](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fllama\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 700M users and you cannot use LLaMA outputs to train other LLMs besides LLaMA and its derivatives   | [HuggingChat](https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fcodellama) |\n\n\n ## Open LLM datasets for pre-training\n\n| Name | Release Date | Paper\u002FBlog | Dataset | Tokens (T) | License |\n| --- | --- | --- | --- | --- | ---- | \n| RedPajama | 2023\u002F04 | [RedPajama, a project to create leading open-source models, starts by reproducing LLaMA training dataset of over 1.2 trillion tokens](https:\u002F\u002Fwww.together.xyz\u002Fblog\u002Fredpajama) | [RedPajama-Data](https:\u002F\u002Fgithub.com\u002Ftogethercomputer\u002FRedPajama-Data) | 1.2 | Apache 2.0 | \n| starcoderdata | 2023\u002F05 | [StarCoder: A State-of-the-Art LLM for Code](https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fstarcoder) | [starcoderdata](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fbigcode\u002Fstarcoderdata) |  0.25 | Apache 2.0 |\n\n## Open LLM datasets for instruction-tuning\n\n| Name | Release Date |  Paper\u002FBlog | Dataset | Samples (K) | License |\n| --- | --- | --- | --- | --- | ---- | \n| OIG (Open Instruction Generalist)   | 2023\u002F03 | [THE OIG DATASET](https:\u002F\u002Flaion.ai\u002Fblog\u002Foig-dataset\u002F) | [OIG](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Flaion\u002FOIG) | 44,000 | Apache 2.0 |\n| databricks-dolly-15k | 2023\u002F04 | [Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM](https:\u002F\u002Fwww.databricks.com\u002Fblog\u002F2023\u002F04\u002F12\u002Fdolly-first-open-commercially-viable-instruction-tuned-llm) |  [databricks-dolly-15k](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fdatabricks\u002Fdatabricks-dolly-15k) | 15 |  CC BY-SA-3.0 |\n| MPT-7B-Instruct | 2023\u002F05 | [Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs](https:\u002F\u002Fwww.mosaicml.com\u002Fblog\u002Fmpt-7b) | [dolly_hhrlhf](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fmosaicml\u002Fdolly_hhrlhf) | 59 | CC BY-SA-3.0 |\n\n## Open LLM datasets for alignment-tuning\n\n| Name | Release Date |  Paper\u002FBlog | Dataset | Samples (K) | License |\n| --- | --- | --- | --- | --- | ---- |\n| OpenAssistant Conversations Dataset | 2023\u002F04 | [OpenAssistant Conversations - Democratizing Large Language Model Alignment](https:\u002F\u002Fdrive.google.com\u002Ffile\u002Fd\u002F10iR5hKwFqAKhL3umx8muOWSRm7hs5FqX\u002Fview) | [oasst1](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FOpenAssistant\u002Foasst1) | 161 | Apache 2.0 |\n\n## Evals on open LLMs\n\n- [Leaderboard by lmsys.org](https:\u002F\u002Fchat.lmsys.org\u002F?leaderboard)\n- [Evals by MosaicML](https:\u002F\u002Ftwitter.com\u002Fjefrankle\u002Fstatus\u002F1654631746506301441)\n- [Holistic Evaluation of Language Models (HELM)](https:\u002F\u002Fcrfm.stanford.edu\u002Fhelm\u002Flatest\u002F?groups=1)\n- [LLM-Leaderboard](https:\u002F\u002Fgithub.com\u002FLudwigStumpp\u002Fllm-leaderboard)\n- [TextSynth Server Benchmarks](https:\u002F\u002Fbellard.org\u002Fts_server\u002F)\n- [Open LLM Leaderboard by Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FHuggingFaceH4\u002Fopen_llm_leaderboard)\n\n---\n\n### What do the licences mean?\n\n- [Apache 2.0](https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FApache_License): Allows users to use the software for any purpose, to distribute it, to modify it, and to distribute modified versions of the software under the terms of the license, without concern for royalties.\n- [MIT](https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMIT_License): Similar to Apache 2.0 but shorter and simpler. Also, in contrast to Apache 2.0, does not require stating any significant changes to the original code.\n- [CC BY-SA-4.0](https:\u002F\u002Fcreativecommons.org\u002Flicenses\u002Fby-sa\u002F4.0\u002F): Allows (i) copying and redistributing the material and (ii) remixing, transforming, and building upon the material\nfor any purpose, even commercially. But if you do the latter, you **must distribute your contributions under the same license as the original.** (Thus, may not be viable for internal teams.)\n- [OpenRAIL-M v1](https:\u002F\u002Fwww.bigcode-project.org\u002Fdocs\u002Fpages\u002Fmodel-license\u002F): Allows royalty-free access and flexible downstream use and sharing of the model and modifications of it, and comes with a set of use restrictions (see [Attachment A](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fbigcode\u002Fbigcode-model-license-agreement))\n- [BSD-3-Clause](https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FBSD_licenses): This version allows unlimited redistribution for any purpose as long as its copyright notices and the license's disclaimers of warranty are maintained. \n\n**Disclaimer:** The information provided in this repo does not, and is not intended to, constitute legal advice. Maintainers of this repo are not responsible for the actions of third parties who use the models. Please consult an attorney before using models for commercial purposes.\n\n---\n\n### Improvements\n\n- [x] Complete entries for context length, and check entries with `?`\n- [ ] ~~Add number of tokens trained?~~ (see [considerations](https:\u002F\u002Fgithub.com\u002Feugeneyan\u002Fopen-llms\u002Fissues\u002F7))\n- [ ] Add (links to) training code?\n- [ ] Add (links to) eval benchmarks?\n","# 开源大语言模型\n\n这些大语言模型均获得商业使用许可（例如 Apache 2.0、MIT、OpenRAIL-M）。欢迎贡献！\n\n| Language Model | Release Date | Checkpoints | Paper\u002FBlog | Params (B) | Context Length | Licence | Try it                                                                                                                |\n| --- | --- | --- | --- | --- | --- | --- |-----------------------------------------------------------------------------------------------------------------------|\n| T5           | 2019\u002F10 |[T5 & Flan-T5](https:\u002F\u002Fgithub.com\u002Fgoogle-research\u002Ft5x\u002Fblob\u002Fmain\u002Fdocs\u002Fmodels.md#flan-t5-checkpoints), [Flan-T5-xxl (HF)](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fflan-t5-xxl)      | [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https:\u002F\u002Fgithub.com\u002Fgoogle-research\u002Ftext-to-text-transfer-transformer#released-model-checkpoints) | 0.06 - 11       | [512](https:\u002F\u002Fdiscuss.huggingface.co\u002Ft\u002Fdoes-t5-truncate-input-longer-than-512-internally\u002F3602) | Apache 2.0         | [T5-Large](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Ft5)                                               |\n| RWKV 4        | 2021\u002F08| [RWKV, ChatRWKV](https:\u002F\u002Fgithub.com\u002FBlinkDL\u002FRWKV-LM#rwkv-parallelizable-rnn-with-transformer-level-llm-performance-pronounced-as-rwakuv-from-4-major-params-r-w-k-v) | [The RWKV Language Model (and my LM tricks)](https:\u002F\u002Fgithub.com\u002FBlinkDL\u002FRWKV-LM)                                           | 0.1 - 14      | [infinity (RNN)](https:\u002F\u002Fgithub.com\u002FBlinkDL\u002FRWKV-LM#rwkv-parallelizable-rnn-with-transformer-level-llm-performance-pronounced-as-rwakuv-from-4-major-params-r-w-k-v) | Apache 2.0         |                                                                                                                       |\n| GPT-NeoX-20B | 2022\u002F04 | [GPT-NEOX-20B](https:\u002F\u002Fhuggingface.co\u002FEleutherAI\u002Fgpt-neox-20b) | [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https:\u002F\u002Farxiv.org\u002Fabs\u002F2204.06745) | 20 | [2048](https:\u002F\u002Fhuggingface.co\u002FEleutherAI\u002Fgpt-neox-20b) | Apache 2.0 |                                                                                                                       |\n| YaLM-100B | 2022\u002F06 | [yalm-100b](https:\u002F\u002Fhuggingface.co\u002Fyandex\u002Fyalm-100b) | [Yandex publishes YaLM 100B, the largest GPT-like neural network in open source](https:\u002F\u002Fyandex.com\u002Fcompany\u002Fpress_center\u002Fpress_releases\u002F2022\u002F2022-23-06) | 100 | [1024](https:\u002F\u002Fgithub.com\u002Fyandex\u002FYaLM-100B\u002Fblob\u002Fmain\u002Fexamples\u002Fgenerate_interactive.sh) | Apache 2.0 | |\n| UL2          | 2022\u002F10 | [UL2 & Flan-UL2](https:\u002F\u002Fgithub.com\u002Fgoogle-research\u002Fgoogle-research\u002Ftree\u002Fmaster\u002Ful2#checkpoints), [Flan-UL2 (HF)](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fflan-ul2)          | [UL2 20B: An Open Source Unified Language Learner](https:\u002F\u002Fai.googleblog.com\u002F2022\u002F10\u002Ful2-20b-open-source-unified-language.html)                                                       | 20             | [512, 2048](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fflan-ul2#tldr) | Apache 2.0         |                                                                                                                       |\n| Bloom | 2022\u002F11 | [Bloom](https:\u002F\u002Fhuggingface.co\u002Fbigscience\u002Fbloom) | [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https:\u002F\u002Farxiv.org\u002Fabs\u002F2211.05100) | 176 | [2048](https:\u002F\u002Fhuggingface.co\u002Fbigscience\u002Fbloom) |  [OpenRAIL-M v1](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fbigcode\u002Fbigcode-model-license-agreement) |                                                                                                                       |\n| ChatGLM | 2023\u002F03\u003C!--13--> | [chatglm-6b](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm-6b) | [ChatGLM](https:\u002F\u002Fzhipuai.cn\u002Fen\u002Fnews\u002F61), [Github](https:\u002F\u002Fgithub.com\u002FTHUDM\u002FChatGLM-6B\u002Fblob\u002Fmain\u002FREADME_en.md) | 6 | [2048](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm-6b\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm-6b\u002Fblob\u002Fmain\u002FMODEL_LICENSE) Free with some usage restriction (might require registration) | |\n| Cerebras-GPT | 2023\u002F03 | [Cerebras-GPT](https:\u002F\u002Fhuggingface.co\u002Fcerebras)                                           | [Cerebras-GPT: A Family of Open, Compute-efficient, Large Language Models](https:\u002F\u002Fwww.cerebras.net\u002Fblog\u002Fcerebras-gpt-a-family-of-open-compute-efficient-large-language-models\u002F) ([Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.03208)) | 0.111 - 13      | [2048](https:\u002F\u002Fhuggingface.co\u002Fcerebras\u002FCerebras-GPT-13B#model-details) | Apache 2.0         | [Cerebras-GPT-1.3B](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Fcerebras-gpt)                            |\n| Open Assistant (Pythia family) | 2023\u002F03 | [OA-Pythia-12B-SFT-8](https:\u002F\u002Fhuggingface.co\u002FOpenAssistant\u002Fpythia-12b-sft-v8-7k-steps), [OA-Pythia-12B-SFT-4](https:\u002F\u002Fhuggingface.co\u002FOpenAssistant\u002Foasst-sft-4-pythia-12b-epoch-3.5), [OA-Pythia-12B-SFT-1](https:\u002F\u002Fhuggingface.co\u002FOpenAssistant\u002Foasst-sft-1-pythia-12b) | [Democratizing Large Language Model Alignment](https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.07327) | 12     | [2048](https:\u002F\u002Fhuggingface.co\u002FOpenAssistant\u002Fpythia-12b-sft-v8-7k-steps\u002Fblob\u002Fmain\u002Fconfig.json)  | Apache 2.0                | [Pythia-2.8B](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Fpythia)                                        |\n| Pythia       | 2023\u002F04 | [pythia 70M - 12B](https:\u002F\u002Fgithub.com\u002FEleutherAI\u002Fpythia)                                   | [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https:\u002F\u002Farxiv.org\u002Fabs\u002F2304.01373)                                                                    | 0.07 - 12       | [2048](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2304.01373.pdf) | Apache 2.0         |                                                                                                                       |\n| Dolly        | 2023\u002F04 | [dolly-v2-12b](https:\u002F\u002Fhuggingface.co\u002Fdatabricks\u002Fdolly-v2-12b)                            | [Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM](https:\u002F\u002Fwww.databricks.com\u002Fblog\u002F2023\u002F04\u002F12\u002Fdolly-first-open-commercially-viable-instruction-tuned-llm)             | 3, 7, 12     | [2048](https:\u002F\u002Fgithub.com\u002Fdatabrickslabs\u002Fdolly#dolly) | MIT                |                                                                                                                       |\n| StableLM-Alpha | 2023\u002F04 | [StableLM-Alpha](https:\u002F\u002Fgithub.com\u002FStability-AI\u002FStableLM#stablelm-alpha) | [Stability AI Launches the First of its StableLM Suite of Language Models](https:\u002F\u002Fstability.ai\u002Fblog\u002Fstability-ai-launches-the-first-of-its-stablelm-suite-of-language-models) | 3 - 65 | [4096](https:\u002F\u002Fgithub.com\u002FStability-AI\u002FStableLM#stablelm-alpha) | CC BY-SA-4.0 |                                                                                                                       |\n| FastChat-T5 | 2023\u002F04 | [fastchat-t5-3b-v1.0](https:\u002F\u002Fhuggingface.co\u002Flmsys\u002Ffastchat-t5-3b-v1.0) | [We are excited to release FastChat-T5: our compact and commercial-friendly chatbot!](https:\u002F\u002Ftwitter.com\u002Flmsysorg\u002Fstatus\u002F1652037026705985537?s=20) | 3 | [512](https:\u002F\u002Fhuggingface.co\u002Flmsys\u002Ffastchat-t5-3b-v1.0\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 |                                                                                                                       |\n| DLite | 2023\u002F05 | [dlite-v2-1_5b](https:\u002F\u002Fhuggingface.co\u002Faisquared\u002Fdlite-v2-1_5b) | [Announcing DLite V2: Lightweight, Open LLMs That Can Run Anywhere](https:\u002F\u002Fmedium.com\u002Fai-squared\u002Fannouncing-dlite-v2-lightweight-open-llms-that-can-run-anywhere-a852e5978c6e) | 0.124 - 1.5 | [1024](https:\u002F\u002Fhuggingface.co\u002Faisquared\u002Fdlite-v2-1_5b\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0         | [DLite-v2-1.5B](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Fdlite-v2)                                    |\n| h2oGPT | 2023\u002F05 | [h2oGPT](https:\u002F\u002Fgithub.com\u002Fh2oai\u002Fh2ogpt) | [Building the World’s Best Open-Source Large Language Model: H2O.ai’s Journey](https:\u002F\u002Fh2o.ai\u002Fblog\u002Fbuilding-the-worlds-best-open-source-large-language-model-h2o-ais-journey\u002F) | 12 - 20 | [256 - 2048](https:\u002F\u002Fhuggingface.co\u002Fh2oai) | Apache 2.0 |                                                                                                                       |\n| MPT-7B | 2023\u002F05 | [MPT-7B](https:\u002F\u002Fhuggingface.co\u002Fmosaicml\u002Fmpt-7b), [MPT-7B-Instruct](https:\u002F\u002Fhuggingface.co\u002Fmosaicml\u002Fmpt-7b-instruct) | [Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs](https:\u002F\u002Fwww.mosaicml.com\u002Fblog\u002Fmpt-7b) | 7 | [84k (ALiBi)](https:\u002F\u002Fhuggingface.co\u002Fmosaicml\u002Fmpt-7b#how-is-this-model-different) | Apache 2.0, CC BY-SA-3.0 |                                                                                                                       |\n| RedPajama-INCITE | 2023\u002F05 | [RedPajama-INCITE](https:\u002F\u002Fhuggingface.co\u002Ftogethercomputer) | [Releasing 3B and 7B RedPajama-INCITE family of models including base, instruction-tuned & chat models](https:\u002F\u002Fwww.together.xyz\u002Fblog\u002Fredpajama-models-v1) | 3 - 7 | [2048](https:\u002F\u002Fhuggingface.co\u002Ftogethercomputer\u002FRedPajama-INCITE-Instruct-7B-v0.1\u002Fblob\u002F157bf3174feebb67f37e131ea68f84dee007c687\u002Fconfig.json#L13) | Apache 2.0 | [RedPajama-INCITE-Instruct-3B-v1](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Fredpajama-incite-instruct) |\n| OpenLLaMA | 2023\u002F05 | [open_llama_3b](https:\u002F\u002Fhuggingface.co\u002Fopenlm-research\u002Fopen_llama_3b), [open_llama_7b](https:\u002F\u002Fhuggingface.co\u002Fopenlm-research\u002Fopen_llama_7b), [open_llama_13b](https:\u002F\u002Fhuggingface.co\u002Fopenlm-research\u002Fopen_llama_13b) | [OpenLLaMA: An Open Reproduction of LLaMA](https:\u002F\u002Fgithub.com\u002Fopenlm-research\u002Fopen_llama) | 3, 7 | [2048](https:\u002F\u002Fhuggingface.co\u002Fh2oai) | Apache 2.0 | [OpenLLaMA-7B-Preview_200bt](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Fopenllama)                      |\n| Falcon | 2023\u002F05 | [Falcon-180B](https:\u002F\u002Fhuggingface.co\u002Ftiiuae\u002Ffalcon-180B), [Falcon-40B](https:\u002F\u002Fhuggingface.co\u002Ftiiuae\u002Ffalcon-40b), [Falcon-7B](https:\u002F\u002Fhuggingface.co\u002Ftiiuae\u002Ffalcon-7b) | [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.01116) | 180, 40, 7 | [2048](https:\u002F\u002Fhuggingface.co\u002Ftiiuae\u002Ffalcon-7b\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | \n| GPT-J-6B | 2023\u002F06 | [GPT-J-6B](https:\u002F\u002Fgithub.com\u002Fkingoflolz\u002Fmesh-transformer-jax\u002F#gpt-j-6b), [GPT4All-J](https:\u002F\u002Fgithub.com\u002Fnomic-ai\u002Fgpt4all#raw-model) | [GPT-J-6B: 6B JAX-Based Transformer](https:\u002F\u002Farankomatsuzaki.wordpress.com\u002F2021\u002F06\u002F04\u002Fgpt-j\u002F) | 6 | [2048](https:\u002F\u002Fgithub.com\u002Fkingoflolz\u002Fmesh-transformer-jax\u002F#gpt-j-6b) | Apache 2.0 |                                                                                                                       |\n| MPT-30B | 2023\u002F06 | [MPT-30B](https:\u002F\u002Fhuggingface.co\u002Fmosaicml\u002Fmpt-30b), [MPT-30B-instruct](https:\u002F\u002Fhuggingface.co\u002Fmosaicml\u002Fmpt-30b-instruct) | [MPT-30B: Raising the bar for open-source foundation models](https:\u002F\u002Fwww.mosaicml.com\u002Fblog\u002Fmpt-30b) | 30 | [8192](https:\u002F\u002Fhuggingface.co\u002Fmosaicml\u002Fmpt-30b\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0, CC BY-SA-3.0 | [MPT 30B inference code using CPU](https:\u002F\u002Fgithub.com\u002Fabacaj\u002Fmpt-30B-inference) |\n| LLaMA 2  | 2023\u002F06\u003C!--18--> | [LLaMA 2 Weights](https:\u002F\u002Fai.meta.com\u002Fresources\u002Fmodels-and-libraries\u002Fllama-downloads\u002F) | [Llama 2: Open Foundation and Fine-Tuned Chat Models](https:\u002F\u002Fscontent-ham3-1.xx.fbcdn.net\u002Fv\u002Ft39.2365-6\u002F10000000_662098952474184_2584067087619170692_n.pdf?_nc_cat=105&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=qhK-ahCbkBMAX94XV2X&_nc_ht=scontent-ham3-1.xx&oh=00_AfDB7dN8momft9nkv8X0gqrZdEnKltVjPOxhKBm0XLRinA&oe=64BE66FF)      | 7 - 70       | [4096](https:\u002F\u002Fscontent-ham3-1.xx.fbcdn.net\u002Fv\u002Ft39.2365-6\u002F10000000_662098952474184_2584067087619170692_n.pdf?_nc_cat=105&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=qhK-ahCbkBMAX94XV2X&_nc_ht=scontent-ham3-1.xx&oh=00_AfDB7dN8momft9nkv8X0gqrZdEnKltVjPOxhKBm0XLRinA&oe=64BE66FF)  | [Custom](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fllama\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 700M users and you cannot use LLaMA outputs to train other LLMs besides LLaMA and its derivatives   | [HuggingChat](https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fllama2#demo) |  \n| ChatGLM2 | 2023\u002F06\u003C!--25--> | [chatglm2-6b](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm2-6b) | [ChatGLM2-6B](https:\u002F\u002Fzhipuai.cn\u002Fen\u002Fnews\u002F72), [Github](https:\u002F\u002Fgithub.com\u002FTHUDM\u002FChatGLM2-6B) | 6 | [32k](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm2-6b\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm2-6b\u002Fblob\u002Fmain\u002FMODEL_LICENSE) Free with some usage restriction (might require registration) | |\n| XGen-7B | 2023\u002F06\u003C!--28--> | [xgen-7b-4k-base](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fxgen-7b-4k-base), [xgen-7b-8k-base](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fxgen-7b-8k-base)  | [Long Sequence Modeling with XGen](https:\u002F\u002Fblog.salesforceairesearch.com\u002Fxgen\u002F) | 7 | [4096](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fxgen-7b-4k-base), [8192](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fxgen-7b-8k-base) | Apache 2.0 | |\n| Jais-13b | 2023\u002F08\u003C!--17--> | [jais-13b](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-13b), [jais-13b-chat](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-13b-chat) | [Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models](https:\u002F\u002Farxiv.org\u002Fabs\u002F2308.16149) | 13 | [2048](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-13b\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| OpenHermes | 2023\u002F09\u003C!--14--> | [OpenHermes-7B](https:\u002F\u002Fhuggingface.co\u002Fteknium\u002FOpenHermes-7B), [OpenHermes-13B](https:\u002F\u002Fhuggingface.co\u002Fteknium\u002FOpenHermes-13B) | [Nous Research](https:\u002F\u002Fnousresearch.com\u002F) | 7, 13 | [4096](https:\u002F\u002Fhuggingface.co\u002Fteknium\u002FOpenHermes-13B\u002Fblob\u002Fmain\u002Fconfig.json)| MIT | [OpenHermes-V2 Finetuned on Mistral 7B](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fartificialguybr\u002FOPENHERMES-2)\n| OpenLM  | 2023\u002F09\u003C!--26--> | [OpenLM 1B](https:\u002F\u002Fhuggingface.co\u002Fmlfoundations\u002Fopen_lm_1B), [OpenLM 7B](https:\u002F\u002Fhuggingface.co\u002Fmlfoundations\u002Fopen_lm_7B_1.25T) | [Open LM:  a minimal but performative language modeling (LM) repository](https:\u002F\u002Fgithub.com\u002Fmlfoundations\u002Fopen_lm#pretrained-models)      | 1, 7       | [2048](https:\u002F\u002Fgithub.com\u002Fmlfoundations\u002Fopen_lm\u002Fblob\u002Fmain\u002Fopen_lm\u002Fmodel_configs\u002Fopen_lm_7b.json)  | MIT   |      |  \n| Mistral 7B | 2023\u002F09\u003C!--27--> | [Mistral-7B-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-7B-v0.1), [Mistral-7B-Instruct-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-7B-Instruct-v0.1) | [Mistral 7B](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fannouncing-mistral-7b\u002F) | 7 | [4096-16K with Sliding Windows](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-7B-Instruct-v0.1\u002Fblob\u002Fmain\u002Fconfig.json)| Apache 2.0 | [Mistral Transformer](https:\u002F\u002Fgithub.com\u002Fmistralai\u002Fmistral-src)\n| ChatGLM3 | 2023\u002F10\u003C!--27--> | [chatglm3-6b](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b), [chatglm3-6b-base](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b-base), [chatglm3-6b-32k](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b-32k), [chatglm3-6b-128k](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b-128k) | [ChatGLM3](https:\u002F\u002Fgithub.com\u002FTHUDM\u002FChatGLM3\u002Fblob\u002Fmain\u002FREADME_en.md) | 6 | [8192](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b\u002Fblob\u002Fmain\u002Fconfig.json), [32k](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b-32k\u002Fblob\u002Fmain\u002Fconfig.json), [128k](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b-128k\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FTHUDM\u002Fchatglm3-6b\u002Fblob\u002Fmain\u002FMODEL_LICENSE) Free with some usage restriction (might require registration) | |\n| Skywork | 2023\u002F10\u003C!--30--> | [Skywork-13B-Base](https:\u002F\u002Fhuggingface.co\u002FSkywork\u002FSkywork-13B-Base), [Skywork-13B-Math](https:\u002F\u002Fhuggingface.co\u002FSkywork\u002FSkywork-13B-Math) | [Skywork](https:\u002F\u002Fgithub.com\u002FSkyworkAI\u002FSkywork\u002Fblob\u002Fmain\u002FREADME_EN.md) | 13 | [4096](https:\u002F\u002Fhuggingface.co\u002FSkywork\u002FSkywork-13B-base\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fgithub.com\u002FSkyworkAI\u002FSkywork\u002Fblob\u002Fmain\u002FSkywork%20Community%20License.pdf) Free with usage restriction and models trained on Skywork outputs become Skywork derivatives, subject to this license. | |\n| Jais-30b | 2023\u002F11\u003C!--08--> | [jais-30b-v1](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-v1), [jais-30b-chat-v1](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-chat-v1) | [Jais-30B: Expanding the Horizon in Open-Source Arabic NLP](https:\u002F\u002Fg42.ai\u002Fresources\u002Fpublications\u002FJais-30B) | 30 | [2048](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-v1\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| Zephyr  | 2023\u002F11\u003C!--10--> | [Zephyr 7B](https:\u002F\u002Fhuggingface.co\u002FHuggingFaceH4\u002Fzephyr-7b-gemma-v0.1) | [Website](https:\u002F\u002Farxiv.org\u002Fabs\u002F2310.16944) | 7 | [8192](https:\u002F\u002Fhuggingface.co\u002FHuggingFaceH4\u002Fzephyr-7b-gemma-v0.1)| \tApache 2.0 | |\n| DeepSeek | 2023\u002F11\u003C!--30--> | [deepseek-llm-7b-base](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002Fdeepseek-llm-7b-base), [deepseek-llm-7b-chat](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002Fdeepseek-llm-7b-chat), [deepseek-llm-67b-base](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002Fdeepseek-llm-67b-base), [deepseek-llm-67b-chat](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002Fdeepseek-llm-67b-chat) | [Introducing DeepSeek LLM](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002Fdeepseek-LLM),  | 7, 67 | [4096](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002Fdeepseek-LLM) | [Custom](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-LLM\u002Fblob\u002FHEAD\u002FLICENSE-MODEL) Free with usage restriction and models trained on DeepSeek outputs become DeepSeek derivatives, subject to this license.\n| Mistral 7B v0.2 | 2023\u002F12\u003C!--11--> | [Mistral-7B-v0.2](https:\u002F\u002Fhuggingface.co\u002Fmistral-community\u002FMistral-7B-v0.2), [Mistral-7B-Instruct-v0.2](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-7B-Instruct-v0.2) | [La Plateforme](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fla-plateforme\u002F) | 7 | [32k](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-7B-Instruct-v0.2) | Apache 2.0 | |\n| Mixtral 8x7B v0.1 | 2023\u002F12\u003C!--11--> | [Mixtral-8x7B-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMixtral-8x7B-v0.1), [Mixtral-8x7B-Instruct-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMixtral-8x7B-Instruct-v0.1) | [Mixtral of experts](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmixtral-of-experts\u002F) | 46.7 | [32k](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmixtral-of-experts\u002F) | Apache 2.0 | |\n| LLM360 Amber | 2023\u002F12\u003C!--11--> | [Amber](https:\u002F\u002Fhuggingface.co\u002FLLM360\u002FAmber), [AmberChat](https:\u002F\u002Fhuggingface.co\u002FLLM360\u002FAmberChat), [AmberSafe](https:\u002F\u002Fhuggingface.co\u002FLLM360\u002FAmberSafe) | [Introducing LLM360: Fully Transparent Open-Source LLMs](https:\u002F\u002Fwww.llm360.ai\u002Fblog\u002Fintroducing-llm360-fully-transparent-open-source-llms.html) | 6.7 | [2048](https:\u002F\u002Fhuggingface.co\u002FLLM360\u002FAmber#%F0%9F%9F%A0-model-description) | Apache 2.0 | |\n| SOLAR | 2023\u002F12\u003C!--12--> | [Solar-10.7B](https:\u002F\u002Fhuggingface.co\u002Fupstage\u002FSOLAR-10.7B-v1.0) | [Upstage](https:\u002F\u002Farxiv.org\u002Fabs\u002F2312.15166) | 10.7 | [4096](https:\u002F\u002Fhuggingface.co\u002Fupstage\u002FSOLAR-10.7B-v1.0\u002Fblob\u002Fmain\u002Fconfig.json)| apache-2.0 | |\n| phi-2 | 2023\u002F12\u003C!--12--> | [phi-2 2.7B](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002Fphi-2) | [Microsoft](https:\u002F\u002Fwww.microsoft.com\u002Fen-us\u002Fresearch\u002Fblog\u002Fphi-2-the-surprising-power-of-small-language-models\u002F) | 2.7 | [2048](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002Fphi-2\u002Fblob\u002Fmain\u002Fconfig.json)| MIT | |\n| FLOR | 2023\u002F12\u003C!--22--> | [FLOR-760M](https:\u002F\u002Fhuggingface.co\u002Fprojecte-aina\u002FFLOR-760M), [FLOR-1.3B](https:\u002F\u002Fhuggingface.co\u002Fprojecte-aina\u002FFLOR-1.3B), [FLOR-1.3B-Instructed](https:\u002F\u002Fhuggingface.co\u002Fprojecte-aina\u002FFLOR-1.3B-Instructed), [FLOR-6.3B](https:\u002F\u002Fhuggingface.co\u002Fprojecte-aina\u002FFLOR-6.3B), [FLOR-6.3B-Instructed](https:\u002F\u002Fhuggingface.co\u002Fprojecte-aina\u002FFLOR-6.3B-Instructed) |  [FLOR-6.3B: a chinchilla-compliant model for Catalan, Spanish and English](https:\u002F\u002Fmedium.com\u002F@mpamies247\u002Fflor-6-3b-a-chinchilla-compliant-model-for-catalan-spanish-and-english-7cdb389a9aac) | 0.76, 1.3, 6.3 | [2048](https:\u002F\u002Fhuggingface.co\u002Fbigscience\u002Fbloom-1b1#technical-specifications) | Apache 2.0 with usage restriction inherited from BLOOM | |\n| RWKV 5 v2 | 2024\u002F01\u003C!--28--> | [rwkv-5-world-0.4b-2, rwkv-5-world-1.5b-2, rwkv-5-world-3b-2, rwkv-5-world-3b-2(16k), rwkv-5-world-7b-2](https:\u002F\u002Fhuggingface.co\u002FBlinkDL\u002Frwkv-5-world) | [RWKV 5](https:\u002F\u002Fwww.rwkv.com\u002F) | 0.4, 1.5, 3, 7 | [unlimited(RNN), trained on 4096 (and 16k for 3b)](https:\u002F\u002Fhuggingface.co\u002FBlinkDL\u002Frwkv-5-world\u002Ftree\u002Fmain) | Apache 2.0 | | \n| OLMo | 2024\u002F02\u003C!--01--> | [OLMo 1B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-1B), [OLMo 7B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-7B), [OLMo 7B Twin 2T](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-7B-Twin-2T) | [AI2](https:\u002F\u002Fblog.allenai.org\u002Fhello-olmo-a-truly-open-llm-43f7e7359222) | 1,7 | [2048](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-7B-Twin-2T\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| Qwen1.5 | 2024\u002F02\u003C!--04--> | [Qwen1.5-7B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-7B), [Qwen1.5-7B-Chat](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-7B-Chat), [Qwen1.5-14B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-14B), [Qwen1.5-14B-Chat](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-14B-Chat), [Qwen1.5-72B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-72B), [Qwen1.5-72B-Chat](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-72B-Chat) | [Introducing Qwen1.5](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen1.5\u002F) | 7, 14, 72 | [32k](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-7B-Chat\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-7B-Chat\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 100M users and you cannot use Qwen outputs to train other LLMs besides Qwen and its derivatives | |\n| LWM | 2024\u002F02\u003C!--07--> | [LWM-Text-Chat-128K](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-Chat-128K), [LWM-Text-Chat-256K](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-Chat-256K), [LWM-Text-Chat-512K](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-Chat-512K), [LWM-Text-Chat-1M](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-Chat-1M), [LWM-Text-128K](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-128K), [LWM-Text-256K](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-256K), [LWM-Text-512K](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-512K), [LWM-Text-1M](https:\u002F\u002Fhuggingface.co\u002FLargeWorldModel\u002FLWM-Text-1M) | [Large World Model (LWM)](https:\u002F\u002Fgithub.com\u002FLargeWorldModel\u002FLWM) | 7 | [128k, 256k, 512k, 1M](https:\u002F\u002Fgithub.com\u002FLargeWorldModel\u002FLWM#available-models) | LLaMA 2 license | | \n| Jais-30b v3 | 2024\u002F03\u003C!--08--> | [jais-30b-v3](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-v3), [jais-30b-chat-v3](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-chat-v3) | [Jais 30b v3](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-v3) | 30 | [8192](https:\u002F\u002Fhuggingface.co\u002Fcore42\u002Fjais-30b-v3\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| Gemma | 2024\u002F02\u003C!--21--> | [Gemma 7B](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fgemma-7b), [Gemma 7B it](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fgemma-7b-it), [Gemma 2B](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fgemma-2b), [Gemma 2B it](https:\u002F\u002Fhuggingface.co\u002Fgoogle\u002Fgemma-2b-it) | [Technical report](https:\u002F\u002Fstorage.googleapis.com\u002Fdeepmind-media\u002Fgemma\u002Fgemma-report.pdf) | 2-7 | [8192](https:\u002F\u002Fstorage.googleapis.com\u002Fdeepmind-media\u002Fgemma\u002Fgemma-report.pdf) | [Gemma Terms of Use](https:\u002F\u002Fai.google.dev\u002Fgemma\u002Fterms) Free with usage restriction and models trained on Gemma outputs become Gemma derivatives, subject to this license. | |\n| Grok-1 | 2024\u002F03\u003C!--17--> | [Grok-1](https:\u002F\u002Fhuggingface.co\u002Fxai-org\u002Fgrok-1) | [Open Release of Grok-1](https:\u002F\u002Fx.ai\u002Fblog\u002Fgrok-os) | 314 | [8192](https:\u002F\u002Fgithub.com\u002Fxai-org\u002Fgrok-1\u002Fblob\u002Fmain\u002Frun.py) | Apache 2.0 | |\n| Qwen1.5 MoE | 2024\u002F03\u003C!--28--> | [Qwen1.5-MoE-A2.7B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-MoE-A2.7B), [Qwen1.5-MoE-A2.7B-Chat](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-MoE-A2.7B-Chat) | [Qwen1.5-MoE: Matching 7B Model Performance with 1\u002F3 Activated Parameters](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen-moe\u002F) | 14.3 | [8192](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-MoE-A2.7B\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-MoE-A2.7B\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 100M users and you cannot use Qwen outputs to train other LLMs besides Qwen and its derivatives | |\n| Jamba 0.1 | 2024\u002F03\u003C!--28--> | [Jamba-v0.1](https:\u002F\u002Fhuggingface.co\u002Fai21labs\u002FJamba-v0.1) | [Introducing Jamba: AI21's Groundbreaking SSM-Transformer Model](https:\u002F\u002Fwww.ai21.com\u002Fblog\u002Fannouncing-jamba) | 52 | [256k](https:\u002F\u002Fhuggingface.co\u002Fai21labs\u002FJamba-v0.1\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| Qwen1.5 32B | 2024\u002F04\u003C!--02--> | [Qwen1.5-32B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-32B), [Qwen1.5-32B-Chat](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-32B-Chat) | [Qwen1.5-32B: Fitting the Capstone of the Qwen1.5 Language Model Series](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen1.5-32b\u002F) | 32 | [32k](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-32B-Chat\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-32B-Chat\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 100M users and you cannot use Qwen outputs to train other LLMs besides Qwen and its derivatives | |\n| Mamba-7B | 2024\u002F04\u003C!--15--> | [mamba-7b-rw](https:\u002F\u002Fhuggingface.co\u002FTRI-ML\u002Fmamba-7b-rw) | [Toyota Research Institute](https:\u002F\u002Fhuggingface.co\u002FTRI-ML\u002Fmamba-7b-rw) | 7 | [unlimited(RNN), trained on 2048](https:\u002F\u002Fhuggingface.co\u002FTRI-ML\u002Fmamba-7b-rw) | Apache 2.0 | |\n| Mixtral8x22B v0.1 | 2024\u002F04\u003C!--17--> | [Mixtral-8x22B-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMixtral-8x22B-v0.1), [Mixtral-8x22B-Instruct-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMixtral-8x22B-Instruct-v0.1) | [Cheaper, Better, Faster, Stronger](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmixtral-8x22b\u002F) | 141 | [64k](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmixtral-8x22b\u002F) | Apache 2.0 | |\n| Llama 3 | 2024\u002F04\u003C!--18--> | [Llama-3-8B](https:\u002F\u002Fhuggingface.co\u002Fmeta-llama\u002FMeta-Llama-3-8B), [Llama-3-8B-Instruct](https:\u002F\u002Fhuggingface.co\u002Fmeta-llama\u002FMeta-Llama-3-8B-Instruct), [Llama-3-70B](https:\u002F\u002Fhuggingface.co\u002Fmeta-llama\u002FMeta-Llama-3-70B), [Llama-3-70B-Instruct](https:\u002F\u002Fhuggingface.co\u002Fmeta-llama\u002FMeta-Llama-3-70B-Instruct), [Llama-Guard-2-8B](https:\u002F\u002Fhuggingface.co\u002Fmeta-llama\u002FMeta-Llama-Guard-2-8B) | [Introducing Meta Llama 3](https:\u002F\u002Fai.meta.com\u002Fblog\u002Fmeta-llama-3\u002F), [Meta Llama 3](https:\u002F\u002Fllama.meta.com\u002Fllama3\u002F) | 8, 70 | [8192](https:\u002F\u002Fgithub.com\u002Fmeta-llama\u002Fllama3) | [Meta Llama 3 Community License Agreement](https:\u002F\u002Fgithub.com\u002Fmeta-llama\u002Fllama3\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 700M users and you cannot use LLaMA 3 outputs to train other LLMs besides LLaMA 3 and its derivatives | |\n| Phi-3 Mini | 2024\u002F04\u003C!--23--> | [Phi-3-mini-4k-instruct](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FPhi-3-mini-4k-instruct), [Phi-3-mini-128k-instruct](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FPhi-3-mini-128k-instruct) | [Introducing Phi-3](https:\u002F\u002Fazure.microsoft.com\u002Fen-us\u002Fblog\u002Fintroducing-phi-3-redefining-whats-possible-with-slms\u002F), [Technical Report](https:\u002F\u002Farxiv.org\u002Fabs\u002F2404.14219) | 3.8 | [4096, 128k](https:\u002F\u002Farxiv.org\u002Fabs\u002F2404.14219) | MIT | |\n| OpenELM | 2024\u002F04\u003C!--24--> | [OpenELM-270M](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-270M), [OpenELM-270M-Instruct](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-270M-Instruct), [OpenELM-450M](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-450M), [OpenELM-450M-Instruct](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-450M-Instruct), [OpenELM-1_1B](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-1_1B), [OpenELM-1_1B-Instruct](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-1_1B-Instruct), [OpenELM-3B](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-3B), [OpenELM-3B-Instruct](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-3B-Instruct) | [OpenELM: An Efficient Language Model Family with Open Training and Inference Framework](https:\u002F\u002Fmachinelearning.apple.com\u002Fresearch\u002Fopenelm) | 0.27, 0.45, 1.1, 3 | [2048](https:\u002F\u002Farxiv.org\u002Fhtml\u002F2404.14619v2) | [Custom open license](https:\u002F\u002Fhuggingface.co\u002Fapple\u002FOpenELM-270M\u002Fblob\u002Fmain\u002FLICENSE) No usage or training restrictions\n| Snowflake Arctic | 2024\u002F04\u003C!--24--> | [snowflake-arctic-base](https:\u002F\u002Fhuggingface.co\u002FSnowflake\u002Fsnowflake-arctic-base), [snowflake-arctic-instruct](https:\u002F\u002Fhuggingface.co\u002FSnowflake\u002Fsnowflake-arctic-instruct) | [Snowflake Arctic: The Best LLM for Enterprise AI — Efficiently Intelligent, Truly Open](https:\u002F\u002Fwww.snowflake.com\u002Fblog\u002Farctic-open-efficient-foundation-language-models-snowflake\u002F) | 480 | [4096](https:\u002F\u002Fhuggingface.co\u002FSnowflake\u002Fsnowflake-arctic-base\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| Qwen1.5 110B | 2024\u002F04\u003C!--25--> | [Qwen1.5-110B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-110B), [Qwen1.5-110B-Chat](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-110B-Chat) | [Qwen1.5-110B: The First 100B+ Model of the Qwen1.5 Series](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen1.5-110b\u002F) | 110 | [32k](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-110B-Chat\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen1.5-110B-Chat\u002Fblob\u002Fmain\u002FLICENSE) Free if you have under 100M users and you cannot use Qwen outputs to train other LLMs besides Qwen and its derivatives | |\n| RWKV 6 v2.1 | 2024\u002F05\u003C!--06--> | [rwkv-6-world-1.6b-2.1, rwkv-6-world-3b-2.1, rwkv-6-world-7b-2.1](https:\u002F\u002Fhuggingface.co\u002FBlinkDL\u002Frwkv-6-world) | [RWKV 6](https:\u002F\u002Fwww.rwkv.com\u002F) | 1.6, 3, 7 | [unlimited(RNN), trained on 4096](https:\u002F\u002Fhuggingface.co\u002FBlinkDL\u002Frwkv-6-world\u002Ftree\u002Fmain) | Apache 2.0 | |\n| DeepSeek-V2 | 2024\u002F05\u003C!--06--> | [DeepSeek-V2](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002FDeepSeek-V2), [DeepSeek-V2-Chat](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002FDeepSeek-V2-Chat) | [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-V2) | 236 | [128k](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-V2) | [Custom](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-V2\u002Fblob\u002Fmain\u002FLICENSE-MODEL) Free with usage restriction and models trained on DeepSeek outputs become DeepSeek derivatives, subject to this license. | |\n| Fugaku-LLM | 2024\u002F05\u003C!--13--> | [Fugaku-LLM-13B](https:\u002F\u002Fhuggingface.co\u002FFugaku-LLM\u002FFugaku-LLM-13B), [Fugaku-LLM-13B-instruct](https:\u002F\u002Fhuggingface.co\u002FFugaku-LLM\u002FFugaku-LLM-13B-instruct) | [Release of \"Fugaku-LLM\" – a large language model trained on the supercomputer \"Fugaku\"](https:\u002F\u002Fwww.titech.ac.jp\u002Fenglish\u002Fnews\u002F2024\u002F069223) | 13 | [2048](https:\u002F\u002Fhuggingface.co\u002FFugaku-LLM\u002FFugaku-LLM-13B\u002Fblob\u002Fmain\u002Fconfig.json) | [Custom](https:\u002F\u002Fhuggingface.co\u002FFugaku-LLM\u002FFugaku-LLM-13B-instruct\u002Fblob\u002Fmain\u002FLICENSE) Free with usage restrictions | |\n| Falcon 2 | 2024\u002F05\u003C!--13--> | [falcon2-11B](https:\u002F\u002Fhuggingface.co\u002Ftiiuae\u002Ffalcon-11B) | [Meet Falcon 2: TII Releases New AI Model Series, Outperforming Meta’s New Llama 3](https:\u002F\u002Ffalconllm.tii.ae\u002Ffalcon-2.html) | 11 | [8192](https:\u002F\u002Fhuggingface.co\u002Ftiiuae\u002Ffalcon-11B#training-data) | [Custom Apache 2.0](https:\u002F\u002Ffalconllm-staging.tii.ae\u002Ffalcon-2-terms-and-conditions.html) with mild acceptable use policy | |\n| Yi-1.5 | 2024\u002F05\u003C!--15--> | [Yi-1.5-6B](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-6B), [Yi-1.5-6B-Chat](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-6B-Chat), [Yi-1.5-9B](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-9B), [Yi-1.5-9B-Chat](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-9B-Chat), [Yi-1.5-34B](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-34B), [Yi-1.5-34B-Chat](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-34B-Chat) | [Yi-1.5](https:\u002F\u002Fgithub.com\u002F01-ai\u002FYi-1.5) | 6, 9, 34 | [4096](https:\u002F\u002Fhuggingface.co\u002F01-ai\u002FYi-1.5-6B\u002Fblob\u002Fmain\u002Fconfig.json) | Apache 2.0 | |\n| DeepSeek-V2-Lite | 2024\u002F05\u003C!--16--> | [DeepSeek-V2-Lite](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002FDeepSeek-V2-Lite), [DeepSeek-V2-Lite-Chat](https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002FDeepSeek-V2-Lite-Chat) | [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-V2) | 16 | [32k](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-V2) | [Custom](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002FDeepSeek-V2\u002Fblob\u002Fmain\u002FLICENSE-MODEL) Free with usage restriction and models trained on DeepSeek outputs become DeepSeek derivatives, subject to this license. | |\n| Phi-3 small\u002Fmedium | 2024\u002F05\u003C!--21--> | [Phi-3-mini-4k-instruct](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FPhi-3-mini-4k-instruct), [Phi-3-mini-128k-instruct](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FPhi-3-mini-128k-instruct), [Phi-3-medium-4k-instruct](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FPhi-3-medium-4k-instruct), [Phi-3-medium-128k-instruct](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002FPhi-3-medium-128k-instruct) | [New models added to the Phi-3 family, available on Microsoft Azure](https:\u002F\u002Fazure.microsoft.com\u002Fen-us\u002Fblog\u002Fnew-models-added-to-the-phi-3-family-available-on-microsoft-azure\u002F), [Technical Report](https:\u002F\u002Farxiv.org\u002Fabs\u002F2404.14219) | 7, 14 | [4096, 128k](https:\u002F\u002Farxiv.org\u002Fabs\u002F2404.14219) | MIT | |\n| Phi-4 | 2024\u002F12 | [Phi-4](https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002Fphi-4)| [Introducing Phi-4: Microsoft’s Newest Small Language Model Specializing in Complex Reasoning](https:\u002F\u002Ftechcommunity.microsoft.com\u002Fblog\u002Faiplatformblog\u002Fintroducing-phi-4-microsoft%E2%80%99s-newest-small-language-model-specializing-in-comple\u002F4357090), [Technical Report](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2412.08905) | 14 | [4096](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2412.08905) | MIT | |\n| YuLan-Mini | 2024\u002F12 | [YuLan-Mini](https:\u002F\u002Fhuggingface.co\u002Fyulan-team\u002FYuLan-Mini) | [YuLan-Mini: An Open Data-efficient Language Model](https:\u002F\u002Farxiv.org\u002Fabs\u002F2412.17743), [GitHub](https:\u002F\u002Fgithub.com\u002FRUC-GSAI\u002FYuLan-Mini) | 14 | [28672](https:\u002F\u002Fgithub.com\u002FRUC-GSAI\u002FYuLan-Mini) | MIT | [YuLan-Mini](https:\u002F\u002Fhuggingface.co\u002Fyulan-team\u002FYuLan-Mini) |\n| Selene Mini | 2025\u002F01 | [Selene Mini](https:\u002F\u002Fhuggingface.co\u002FAtlaAI\u002FSelene-1-Mini-Llama-3.1-8B) | [Atla Selene Mini: A General Purpose Evaluation Model](https:\u002F\u002Farxiv.org\u002Fabs\u002F2501.17195v1), [GitHub](https:\u002F\u002Fgithub.com\u002Fatla-ai\u002Fselene-mini) | 8 | [128K](https:\u002F\u002Fgithub.com\u002Fatla-ai\u002Fselene-mini) | Apache 2.0 | [Hugging Face Space](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FAtlaAI\u002Fselene) |\n\n## 面向代码的开源大语言模型  \n\n| 语言模型 | 发布日期 | 检查点 | 论文\u002F博客 | 参数量 (B) | 上下文长度                                                                         | 许可证 | 试用                                                                                    |\n| --- | --- | --- | --- | --- |----------------------------------------------------------------------------------------| --- |-------------------------------------------------------------------------------------------|\n| SantaCoder | 2023\u002F01 | [santacoder](https:\u002F\u002Fhuggingface.co\u002Fbigcode\u002Fsantacoder) |[SantaCoder: 不要好高骛远！](https:\u002F\u002Farxiv.org\u002Fabs\u002F2301.03988) | 1.1 | [2048](https:\u002F\u002Fhuggingface.co\u002Fbigcode\u002Fsantacoder\u002Fblob\u002Fmain\u002FREADME.md#model-summary)                | [OpenRAIL-M v1](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fbigcode\u002Fbigcode-model-license-agreement) | [SantaCoder](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Fsantacoder)         |\n| CodeGen2 | 2023\u002F04 | [codegen2 1B-16B](https:\u002F\u002Fgithub.com\u002Fsalesforce\u002FCodeGen2) | [CodeGen2：关于在编程和自然语言上训练大语言模型的经验教训](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.02309) | 1 - 16 | [2048](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.02309) | [Apache 2.0](https:\u002F\u002Fgithub.com\u002Fsalesforce\u002FCodeGen2\u002Fblob\u002Fmain\u002FLICENSE)|                                                                                           |\n| StarCoder | 2023\u002F05 | [starcoder](https:\u002F\u002Fhuggingface.co\u002Fbigcode\u002Fstarcoder) | [StarCoder：最先进的代码专用大语言模型](https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fstarcoder), [StarCoder：愿源代码与你同在！](https:\u002F\u002Fdrive.google.com\u002Ffile\u002Fd\u002F1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs\u002Fview) | 1.1-15 | [8192](https:\u002F\u002Fhuggingface.co\u002Fbigcode\u002Fstarcoder#model-summary)                         | [OpenRAIL-M v1](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fbigcode\u002Fbigcode-model-license-agreement) |                                                                                           |\n| StarChat Alpha | 2023\u002F05 | [starchat-alpha](https:\u002F\u002Fhuggingface.co\u002FHuggingFaceH4\u002Fstarchat-alpha) | [使用StarCoder创建编码助手](https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fstarchat-alpha) | 16 | [8192](https:\u002F\u002Fhuggingface.co\u002Fbigcode\u002Fstarcoder#model-summary)  | [OpenRAIL-M v1](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fbigcode\u002Fbigcode-model-license-agreement) |                                                                                           |\n| Replit Code | 2023\u002F05 | [replit-code-v1-3b](https:\u002F\u002Fhuggingface.co\u002Freplit\u002Freplit-code-v1-3b) | [一周内训练出SOTA代码大语言模型并量化其效果——与Replit的Reza Shabani对话](https:\u002F\u002Fwww.latent.space\u002Fp\u002Freza-shabani#details) | 2.7 | [无穷？(ALiBi)](https:\u002F\u002Fhuggingface.co\u002Freplit\u002Freplit-code-v1-3b#model-description) | CC BY-SA-4.0 | [Replit-Code-v1-3B](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002Freplit-code) |\n| CodeT5+ | 2023\u002F05 | [CodeT5+](https:\u002F\u002Fgithub.com\u002Fsalesforce\u002FCodeT5\u002Ftree\u002Fmain\u002FCodeT5+)     | [CodeT5+：面向代码理解与生成的开源大型语言模型](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.07922) | 0.22 - 16 | [512](https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.07922)                                                                                | [BSD-3-Clause](https:\u002F\u002Fgithub.com\u002Fsalesforce\u002FCodeT5\u002Fblob\u002Fmain\u002FLICENSE.txt)                                                           | [Codet5+-6B](https:\u002F\u002Fgithub.com\u002Fslai-labs\u002Fget-beam\u002Ftree\u002Fmain\u002Fexamples\u002FcodeT5%2B)          |\n| XGen-7B | 2023\u002F06 | [XGen-7B-8K-Base](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fxgen-7b-8k-base) | [使用XGen进行长序列建模：一款输入序列长度达8K的7B大语言模型](https:\u002F\u002Fblog.salesforceairesearch.com\u002Fxgen\u002F) | 7 | [8192](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fxgen-7b-8k-base\u002Fblob\u002Fmain\u002Fconfig.json) | [Apache 2.0](https:\u002F\u002Fgithub.com\u002Fsalesforce\u002Fxgen\u002Fblob\u002Fmain\u002FLICENSE) | \n| CodeGen2.5 | 2023\u002F07 | [CodeGen2.5-7B-multi](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fcodegen25-7b-multi) | [CodeGen2.5：虽小但强大](https:\u002F\u002Fblog.salesforceairesearch.com\u002Fcodegen25\u002F) | 7 | [2048](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fcodegen25-7b-multi\u002Fblob\u002Fmain\u002Fconfig.json) | [Apache 2.0](https:\u002F\u002Fhuggingface.co\u002FSalesforce\u002Fcodegen25-7b-multi\u002Fblob\u002Fmain\u002FREADME.md) | \n| DeciCoder-1B | 2023\u002F08 | [DeciCoder-1B](https:\u002F\u002Fhuggingface.co\u002FDeci\u002FDeciCoder-1b#how-to-use) | [隆重推出DeciCoder：高效且准确代码生成的新标杆](https:\u002F\u002Fdeci.ai\u002Fblog\u002Fdecicoder-efficient-and-accurate-code-generation-llm\u002F) | 1.1 | [2048](https:\u002F\u002Fhuggingface.co\u002FDeci\u002FDeciCoder-1b#model-architecture) | Apache 2.0 |  [DeciCoder演示](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FDeci\u002FDeciCoder-Demo)|\n| Code Llama  | 2023\u002F08 | [CodeLlama模型的推理代码]([https:\u002F\u002Fai.meta.com\u002Fresources\u002Fmodels-and-libraries\u002Fllama-downloads\u002F](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fcodellama)) | [Code Llama：面向代码的开源基础模型](https:\u002F\u002Fai.meta.com\u002Fresearch\u002Fpublications\u002Fcode-llama-open-foundation-models-for-code\u002F)     | 7 - 34       | [4096](https:\u002F\u002Fscontent-zrh1-1.xx.fbcdn.net\u002Fv\u002Ft39.2365-6\u002F369856151_1754812304950972_1159666448927483931_n.pdf?_nc_cat=107&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=wURKmnWKaloAX-ib5XW&_nc_ht=scontent-zrh1-1.xx&oh=00_AfAN1GB2K_XwIz54PqXTr-dhilI3CfCwdQoaLMyaYEEECg&oe=64F0A68F)  | [自定义](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fllama\u002Fblob\u002Fmain\u002FLICENSE) 如果用户数少于7亿且不能将LLaMA输出用于训练除LLaMA及其衍生模型之外的其他大语言模型，则免费   | [HuggingChat](https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fcodellama) |\n\n\n ## 面向预训练的开源大语言模型数据集\n\n| 名称 | 发布日期 | 论文\u002F博客 | 数据集 | 词元数量 (T) | 许可证 |\n| --- | --- | --- | --- | --- | ---- | \n| RedPajama | 2023\u002F04 | [RedPajama项目旨在打造领先的开源模型，首先复现了超过1.2万亿词元的LLaMA训练数据集](https:\u002F\u002Fwww.together.xyz\u002Fblog\u002Fredpajama) | [RedPajama-Data](https:\u002F\u002Fgithub.com\u002Ftogethercomputer\u002FRedPajama-Data) | 1.2 | Apache 2.0 | \n| starcoderdata | 2023\u002F05 | [StarCoder：最先进的代码专用大语言模型](https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fstarcoder) | [starcoderdata](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fbigcode\u002Fstarcoderdata) |  0.25 | Apache 2.0 |\n\n## 面向指令微调的开源大语言模型数据集\n\n| 名称 | 发布日期 |  论文\u002F博客 | 数据集 | 样本数量 (K) | 许可证 |\n| --- | --- | --- | --- | --- | ---- | \n| OIG (开放指令通用型)   | 2023\u002F03 | [THE OIG 数据集](https:\u002F\u002Flaion.ai\u002Fblog\u002Foig-dataset\u002F) | [OIG](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Flaion\u002FOIG) | 44,000 | Apache 2.0 |\n| databricks-dolly-15k | 2023\u002F04 | [免费Dolly：介绍全球首个真正开放的指令微调大语言模型](https:\u002F\u002Fwww.databricks.com\u002Fblog\u002F2023\u002F04\u002F12\u002Fdolly-first-open-commercially-viable-instruction-tuned-llm) |  [databricks-dolly-15k](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fdatabricks\u002Fdatabricks-dolly-15k) | 15 |  CC BY-SA-3.0 |\n| MPT-7B-Instruct | 2023\u002F05 | [隆重推出MPT-7B：开源、可商用大语言模型的新标准](https:\u002F\u002Fwww.mosaicml.com\u002Fblog\u002Fmpt-7b) | [dolly_hhrlhf](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fmosaicml\u002Fdolly_hhrlhf) | 59 | CC BY-SA-3.0 |\n\n## 用于对齐微调的开源大模型数据集\n\n| 名称 | 发布日期 | 论文\u002F博客 | 数据集 | 样本数（千） | 许可证 |\n| --- | --- | --- | --- | --- | ---- |\n| OpenAssistant 对话数据集 | 2023年4月 | [OpenAssistant 对话——推动大型语言模型对齐民主化](https:\u002F\u002Fdrive.google.com\u002Ffile\u002Fd\u002F10iR5hKwFqAKhL3umx8muOWSRm7hs5FqX\u002Fview) | [oasst1](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FOpenAssistant\u002Foasst1) | 161 | Apache 2.0 |\n\n## 开源大模型的评估\n\n- [lmsys.org 的排行榜](https:\u002F\u002Fchat.lmsys.org\u002F?leaderboard)\n- [MosaicML 的评估](https:\u002F\u002Ftwitter.com\u002Fjefrankle\u002Fstatus\u002F1654631746506301441)\n- [语言模型整体评估（HELM）](https:\u002F\u002Fcrfm.stanford.edu\u002Fhelm\u002Flatest\u002F?groups=1)\n- [LLM-Leaderboard](https:\u002F\u002Fgithub.com\u002FLudwigStumpp\u002Fllm-leaderboard)\n- [TextSynth Server 基准测试](https:\u002F\u002Fbellard.org\u002Fts_server\u002F)\n- [Hugging Face 的开源大模型排行榜](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FHuggingFaceH4\u002Fopen_llm_leaderboard)\n\n---\n\n### 许可证意味着什么？\n\n- [Apache 2.0](https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FApache_License)：允许用户出于任何目的使用该软件、分发它、对其进行修改，并在许可证条款下分发修改后的版本，无需支付版税。\n- [MIT](https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMIT_License)：与 Apache 2.0 类似，但更简短、更简单。此外，与 Apache 2.0 不同，它不要求声明对原始代码的重大更改。\n- [CC BY-SA-4.0](https:\u002F\u002Fcreativecommons.org\u002Flicenses\u002Fby-sa\u002F4.0\u002F)：允许 (i) 复制和重新分发材料，以及 (ii) 混编、转换并在此基础上创作新作品，无论是否用于商业目的。但如果进行后一种操作，你 **必须以与原作品相同的许可证分发你的贡献。**（因此，可能不适合内部团队使用。）\n- [OpenRAIL-M v1](https:\u002F\u002Fwww.bigcode-project.org\u002Fdocs\u002Fpages\u002Fmodel-license\u002F)：允许免版税地访问和灵活地下游使用及共享模型及其修改版本，并附有一组使用限制（参见 [附件 A](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fbigcode\u002Fbigcode-model-license-agreement)）。\n- [BSD-3-Clause](https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FBSD_licenses)：此版本允许为任何目的无限次再分发，只要保留其版权声明和许可证中的免责声明即可。\n\n**免责声明：** 本仓库提供的信息不构成也不意图构成法律建议。本仓库的维护者不对使用这些模型的第三方的行为负责。如需将模型用于商业目的，请务必咨询律师。\n\n---\n\n### 改进建议\n\n- [x] 补全上下文长度条目，并检查带有 `?` 的条目\n- [ ] ~~添加训练的 token 数量？~~（参见 [注意事项](https:\u002F\u002Fgithub.com\u002Feugeneyan\u002Fopen-llms\u002Fissues\u002F7)）\n- [ ] 添加（指向）训练代码的链接？\n- [ ] 添加（指向）评估基准的链接？","# Open LLMs 快速上手指南\n\n本指南旨在帮助开发者快速了解并运行 `open-llms` 列表中支持的各类开源大语言模型（如 LLaMA 2, ChatGLM, Falcon, MPT 等）。由于该仓库是一个模型清单而非单一软件包，以下步骤以通用的 Python 环境搭建及通过 Hugging Face 加载模型为例。\n\n## 1. 环境准备\n\n在开始之前，请确保您的开发环境满足以下要求：\n\n*   **操作系统**: Linux (推荐 Ubuntu 20.04+), macOS, 或 Windows (WSL2 推荐)。\n*   **Python 版本**: 3.8 - 3.10。\n*   **硬件要求**:\n    *   **GPU**: 推荐 NVIDIA GPU (显存需求视模型参数量而定，7B 模型通常需 16GB+ 显存，量化后可降低)。\n    *   **CPU**: 小参数模型（如 \u003C3B）或量化模型可在纯 CPU 环境下运行，但速度较慢。\n*   **前置依赖**:\n    *   Git\n    *   CUDA Toolkit (如需使用 GPU 加速)\n\n## 2. 安装步骤\n\n### 2.1 创建虚拟环境\n建议使用 `conda` 或 `venv` 隔离环境。\n\n```bash\n# 使用 conda\nconda create -n open-llms python=3.9\nconda activate open-llms\n\n# 或使用 venv\npython -m venv open-llms-env\nsource open-llms-env\u002Fbin\u002Factivate  # Windows: open-llms-env\\Scripts\\activate\n```\n\n### 2.2 安装核心依赖\n安装 PyTorch 和 Hugging Face 生态库。**国内用户推荐使用清华或中科大镜像源加速下载**。\n\n```bash\n# 安装 PyTorch (以 CUDA 11.8 为例，其他版本请访问 pytorch.org 查询)\npip install torch torchvision torchaudio --index-url https:\u002F\u002Fdownload.pytorch.org\u002Fwhl\u002Fcu118\n\n# 安装 Transformers, Accelerate, SentencePiece 等核心库 (使用清华镜像)\npip install transformers accelerate sentencepiece protobuf -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple\n```\n\n### 2.3 配置 Hugging Face 访问 (可选但推荐)\n部分模型（如 LLaMA 2, Falcon-180B）需要登录 Hugging Face 并接受协议。\n```bash\n# 安装 huggingface-cli\npip install huggingface_hub -i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple\n\n# 登录账号 (输入您的 Access Token)\nhuggingface-cli login\n```\n> **提示**: 如果无法连接 Hugging Face 官方源，可在代码中设置 `HF_ENDPOINT` 环境变量使用镜像站：\n> `export HF_ENDPOINT=https:\u002F\u002Fhf-mirror.com`\n\n## 3. 基本使用\n\n以下示例展示如何使用 Python 和 `transformers` 库加载并运行一个典型的开源模型（以 **ChatGLM2-6B** 或 **LLaMA 2** 为例）。\n\n### 3.1 代码示例\n\n创建一个名为 `run_model.py` 的文件：\n\n```python\nimport torch\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\n# 1. 选择模型 ID (从 open-llms 列表中选择，例如 ChatGLM2 或 LLaMA-2-7b-hf)\n# 注意：LLaMA 2 需要先获得 Meta 授权并在 HF 上接受协议\nmodel_id = \"THUDM\u002Fchatglm2-6b\" \n# model_id = \"meta-llama\u002FLlama-2-7b-chat-hf\" \n\nprint(f\"正在加载模型：{model_id} ...\")\n\n# 2. 加载分词器\ntokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)\n\n# 3. 加载模型\n# device_map=\"auto\" 会自动分配 GPU 显存，若无 GPU 可改为 device_map=\"cpu\"\n# torch_dtype=torch.float16 可节省显存\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_id,\n    trust_remote_code=True,\n    torch_dtype=torch.float16,\n    device_map=\"auto\"\n)\n\n# 4. 生成文本\ninput_text = \"你好，请介绍一下你自己。\"\ninputs = tokenizer(input_text, return_tensors=\"pt\").to(model.device)\n\noutputs = model.generate(\n    **inputs,\n    max_new_tokens=512,\n    do_sample=True,\n    temperature=0.8,\n    top_p=0.95\n)\n\nresponse = tokenizer.decode(outputs[0], skip_special_tokens=True)\nprint(\"\\n模型回复:\")\nprint(response)\n```\n\n### 3.2 运行脚本\n\n```bash\npython run_model.py\n```\n\n### 3.3 针对不同模型的特别说明\n\n*   **LLaMA 2**: 确保您已填写 [Meta 申请表](https:\u002F\u002Fai.meta.com\u002Fresources\u002Fmodels-and-libraries\u002Fllama-downloads\u002F) 并在 Hugging Face 页面点击 \"Agree and access repository\"。代码中 `model_id` 需替换为具体的 HF 路径。\n*   **RWKV**: 该模型架构特殊，建议直接使用其官方仓库 `BlinkDL\u002FRWKV-LM` 提供的推理脚本，而非标准 transformers 接口。\n*   **大模型 (如 Falcon-180B, Bloom-176B)**: 单机显存通常不足，需要使用多卡并行 (`device_map=\"auto\"` 配合 `accelerate`) 或加载量化版本 (如 `bitsandbytes` 4-bit 量化)。\n\n### 3.4 使用量化版本 (低显存方案)\n\n如果您的显存有限，推荐加载 4-bit 量化模型：\n\n```python\nfrom transformers import BitsAndBytesConfig\n\nquantization_config = BitsAndBytesConfig(\n    load_in_4bit=True,\n    bnb_4bit_compute_dtype=torch.float16\n)\n\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_id,\n    quantization_config=quantization_config,\n    device_map=\"auto\",\n    trust_remote_code=True\n)\n```","一家初创电商公司希望在其客服系统中集成智能问答功能，以自动处理用户关于订单和退换货的咨询，同时必须确保模型可商用且数据不出境。\n\n### 没有 open-llms 时\n- 团队花费数周时间手动筛选 Hugging Face 上的海量模型，难以快速确认哪些模型明确支持商业授权，面临潜在的法律合规风险。\n- 由于缺乏清晰的参数与上下文长度对比，开发人员误选了显存占用过大的模型，导致服务器成本飙升且推理延迟严重。\n- 在寻找支持长文本或多语言（如中英混合）的开源方案时反复试错，错过了像 RWKV 这样具备无限上下文或 Bloom 这样多语言适配的优质选项。\n- 无法快速获取模型的官方论文或测试链接，技术选型会议因信息碎片化而迟迟无法达成共识，延误产品上线周期。\n\n### 使用 open-llms 后\n- 团队直接利用 open-llms 筛选出 Apache 2.0 和 OpenRAIL-M 等明确可商用的模型列表，瞬间排除了法律隐患，安心推进项目。\n- 通过表格直观对比参数量与上下文长度，迅速锁定适合现有硬件的 ChatGLM-6B 和 T5，将部署成本降低了 60% 并显著提升响应速度。\n- 依据场景需求精准匹配特性，针对长文档分析选用了 RWKV，针对多语言客服选用了 Bloom，实现了技术方案的最优配置。\n- 借助列表中集成的论文链接与在线试玩入口，技术负责人在半天内完成了可行性验证，推动项目提前两周进入开发阶段。\n\nopen-llms 将分散的开源大模型信息转化为结构化的决策资产，帮助企业在合规前提下以最低成本实现高效的商业落地。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Feugeneyan_open-llms_5564c467.png","eugeneyan","Eugene Yan","https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002Feugeneyan_41d20c44.jpg",null,"@anthropics","Seattle","eugeneyan.com","https:\u002F\u002Fgithub.com\u002Feugeneyan",12720,969,"2026-04-16T08:10:32","Apache-2.0",4,"","未说明 (具体需求取决于所选模型参数量，如 7B\u002F13B\u002F70B 等对显存要求差异巨大)","未说明",{"notes":31,"python":29,"dependencies":32},"该 README 仅为开源大语言模型（Open LLMs）的列表汇总，包含模型名称、参数量、上下文长度、许可证及下载链接等信息，并未提供具体的运行环境配置、安装步骤或依赖库版本。实际运行需求需参考各模型对应的官方仓库或文档。",[],[34,35],"语言模型","开发框架",[37,38,39,40],"commercial","large-language-models","llm","llms",2,"ready","2026-03-27T02:49:30.150509","2026-04-17T09:53:25.503987",[46,51,56,61,66,71,76],{"id":47,"question_zh":48,"answer_zh":49,"source_url":50},36547,"OpenLlama 模型可以使用在 Llama 模型上训练的 LoRA 适配器吗？","不可以。尽管 OpenLlama 和原始 Llama 模型的架构及训练过程相似，但它们的权重不同。因此，为原始 Llama 权重训练的适配器无法在 OpenLlama 上正常工作。解决方案是直接在 OpenLlama 模型上训练您自己的 LoRA 适配器。推荐使用 https:\u002F\u002Fgithub.com\u002Ftloen\u002Falpaca-lora 作为起点，该过程可以在 Google Colab Pro 或本地 RTX3090\u002F4090 显卡上完成。","https:\u002F\u002Fgithub.com\u002Feugeneyan\u002Fopen-llms\u002Fissues\u002F18",{"id":52,"question_zh":53,"answer_zh":54,"source_url":55},36548,"在哪里可以找到开源大语言模型的性能基准测试数据？","您可以访问 Hugging Face 的开源 LLM 排行榜查看性能评估结果：https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FHuggingFaceH4\u002Fopen_llm_leaderboard。此外，项目 README 中的“Evals on open LLMs”部分也列出了一些外部资源。由于不同基准测试众多且同一模型的不同变体表现各异，直接将所有评估结果列入表格并不现实，建议参考上述专用排行榜获取详细对比。","https:\u002F\u002Fgithub.com\u002Feugeneyan\u002Fopen-llms\u002Fissues\u002F17",{"id":57,"question_zh":58,"answer_zh":59,"source_url":60},36549,"StableLM 模型是否因为许可证问题应该被移除？","是的，需要注意许可证兼容性。StableVicuna 等模型发布的权重仅是相对于原始 LLaMA 权重的增量（deltas）。由于原始 LLaMA 的非商业许可证限制，合并后的有效模型实际上受 LLaMA 的许可证条款约束（可能涉及 GPLv3 或非商业限制），这与本项目旨在收录可商用开放模型的目标不符。维护者已根据此情况更新了列表以反映这一限制或移除相关条目。","https:\u002F\u002Fgithub.com\u002Feugeneyan\u002Fopen-llms\u002Fissues\u002F4",{"id":62,"question_zh":63,"answer_zh":64,"source_url":65},36550,"Llama 2 是否被包含在列表中，其许可证是否允许商用？","Llama 2 已被纳入考虑。虽然其许可证包含针对月活用户超过 7 亿的大型公司的额外商业条款，且禁止使用其输出改进其他大模型，但对于绝大多数非 FAANG 级别的用户而言，可以直接使用 Llama 2 及其衍生作品。因此，只要不涉及上述特定限制条款，大多数用户可以将 Llama 2 视为可用模型并包含在列表中。","https:\u002F\u002Fgithub.com\u002Feugeneyan\u002Fopen-llms\u002Fissues\u002F63",{"id":67,"question_zh":68,"answer_zh":69,"source_url":70},36551,"如何查找 OpenAlpaca 和 OpenChatKit 的演示或相关信息？","OpenLlama 和 OpenAlpaca 的演示可以在 Hugging Face 上找到（例如 open_llama_7b_400bt_preview 和 openalpaca_7b_preview_3bt）。关于 OpenChatKit，它主要使用了两个模型：GPT-NeoXT-Chat-Base-20B 和 Pythia-Chat-Base-7B。需要注意的是，OpenChatKit 的部分技术后来演变成了 RedPajamas INCITE 模型，这些模型可能已经存在于列表中了。","https:\u002F\u002Fgithub.com\u002Feugeneyan\u002Fopen-llms\u002Fissues\u002F26",{"id":72,"question_zh":73,"answer_zh":74,"source_url":75},36552,"如何在模型列表中体现多语言支持信息？","建议在表格中增加一列语言信息。对于支持多种语言的模型，如果列举所有语言会导致表格混乱，可以标记为“multilingual”（多语言）。通常情况下，数据集会有主导语言（如 90% 是英语或 Python 代码），标明这一主导语言或统称为多语言对用户选择模型具有参考价值。","https:\u002F\u002Fgithub.com\u002Feugeneyan\u002Fopen-llms\u002Fissues\u002F43",{"id":77,"question_zh":78,"answer_zh":79,"source_url":55},36553,"在哪里可以获取关于模型推理所需的 GPU 显存需求数据？","目前表格尚未直接列出每个模型的具体显存需求，因为这取决于模型的具体变体和量化方式。一般经验法则是，加载模型所需的显存大致与模型参数量成正比（例如 7B 模型通常需要 14GB+ 显存用于半精度推理，或使用量化技术降低需求）。建议参考专门的 LLM-Leaderboard 项目或具体模型的 Hugging Face 页面获取详细的硬件要求估算，以便在提出基于 LLM 的解决方案时评估最低技术门槛。",[],[82,94,102,110,118,127],{"id":83,"name":84,"github_repo":85,"description_zh":86,"stars":87,"difficulty_score":88,"last_commit_at":89,"category_tags":90,"status":42},4358,"openclaw","openclaw\u002Fopenclaw","OpenClaw 是一款专为个人打造的本地化 AI 助手，旨在让你在自己的设备上拥有完全可控的智能伙伴。它打破了传统 AI 助手局限于特定网页或应用的束缚，能够直接接入你日常使用的各类通讯渠道，包括微信、WhatsApp、Telegram、Discord、iMessage 等数十种平台。无论你在哪个聊天软件中发送消息，OpenClaw 都能即时响应，甚至支持在 macOS、iOS 和 Android 设备上进行语音交互，并提供实时的画布渲染功能供你操控。\n\n这款工具主要解决了用户对数据隐私、响应速度以及“始终在线”体验的需求。通过将 AI 部署在本地，用户无需依赖云端服务即可享受快速、私密的智能辅助，真正实现了“你的数据，你做主”。其独特的技术亮点在于强大的网关架构，将控制平面与核心助手分离，确保跨平台通信的流畅性与扩展性。\n\nOpenClaw 非常适合希望构建个性化工作流的技术爱好者、开发者，以及注重隐私保护且不愿被单一生态绑定的普通用户。只要具备基础的终端操作能力（支持 macOS、Linux 及 Windows WSL2），即可通过简单的命令行引导完成部署。如果你渴望拥有一个懂你",349277,3,"2026-04-06T06:32:30",[91,35,92,93],"Agent","图像","数据工具",{"id":95,"name":96,"github_repo":97,"description_zh":98,"stars":99,"difficulty_score":88,"last_commit_at":100,"category_tags":101,"status":42},3808,"stable-diffusion-webui","AUTOMATIC1111\u002Fstable-diffusion-webui","stable-diffusion-webui 是一个基于 Gradio 构建的网页版操作界面，旨在让用户能够轻松地在本地运行和使用强大的 Stable Diffusion 图像生成模型。它解决了原始模型依赖命令行、操作门槛高且功能分散的痛点，将复杂的 AI 绘图流程整合进一个直观易用的图形化平台。\n\n无论是希望快速上手的普通创作者、需要精细控制画面细节的设计师，还是想要深入探索模型潜力的开发者与研究人员，都能从中获益。其核心亮点在于极高的功能丰富度：不仅支持文生图、图生图、局部重绘（Inpainting）和外绘（Outpainting）等基础模式，还独创了注意力机制调整、提示词矩阵、负向提示词以及“高清修复”等高级功能。此外，它内置了 GFPGAN 和 CodeFormer 等人脸修复工具，支持多种神经网络放大算法，并允许用户通过插件系统无限扩展能力。即使是显存有限的设备，stable-diffusion-webui 也提供了相应的优化选项，让高质量的 AI 艺术创作变得触手可及。",162132,"2026-04-05T11:01:52",[35,92,91],{"id":103,"name":104,"github_repo":105,"description_zh":106,"stars":107,"difficulty_score":41,"last_commit_at":108,"category_tags":109,"status":42},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",158594,"2026-04-16T23:34:05",[35,91,34],{"id":111,"name":112,"github_repo":113,"description_zh":114,"stars":115,"difficulty_score":41,"last_commit_at":116,"category_tags":117,"status":42},2271,"ComfyUI","Comfy-Org\u002FComfyUI","ComfyUI 是一款功能强大且高度模块化的视觉 AI 引擎，专为设计和执行复杂的 Stable Diffusion 图像生成流程而打造。它摒弃了传统的代码编写模式，采用直观的节点式流程图界面，让用户通过连接不同的功能模块即可构建个性化的生成管线。\n\n这一设计巧妙解决了高级 AI 绘图工作流配置复杂、灵活性不足的痛点。用户无需具备编程背景，也能自由组合模型、调整参数并实时预览效果，轻松实现从基础文生图到多步骤高清修复等各类复杂任务。ComfyUI 拥有极佳的兼容性，不仅支持 Windows、macOS 和 Linux 全平台，还广泛适配 NVIDIA、AMD、Intel 及苹果 Silicon 等多种硬件架构，并率先支持 SDXL、Flux、SD3 等前沿模型。\n\n无论是希望深入探索算法潜力的研究人员和开发者，还是追求极致创作自由度的设计师与资深 AI 绘画爱好者，ComfyUI 都能提供强大的支持。其独特的模块化架构允许社区不断扩展新功能，使其成为当前最灵活、生态最丰富的开源扩散模型工具之一，帮助用户将创意高效转化为现实。",108322,"2026-04-10T11:39:34",[35,92,91],{"id":119,"name":120,"github_repo":121,"description_zh":122,"stars":123,"difficulty_score":41,"last_commit_at":124,"category_tags":125,"status":42},6121,"gemini-cli","google-gemini\u002Fgemini-cli","gemini-cli 是一款由谷歌推出的开源 AI 命令行工具，它将强大的 Gemini 大模型能力直接集成到用户的终端环境中。对于习惯在命令行工作的开发者而言，它提供了一条从输入提示词到获取模型响应的最短路径，无需切换窗口即可享受智能辅助。\n\n这款工具主要解决了开发过程中频繁上下文切换的痛点，让用户能在熟悉的终端界面内直接完成代码理解、生成、调试以及自动化运维任务。无论是查询大型代码库、根据草图生成应用，还是执行复杂的 Git 操作，gemini-cli 都能通过自然语言指令高效处理。\n\n它特别适合广大软件工程师、DevOps 人员及技术研究人员使用。其核心亮点包括支持高达 100 万 token 的超长上下文窗口，具备出色的逻辑推理能力；内置 Google 搜索、文件操作及 Shell 命令执行等实用工具；更独特的是，它支持 MCP（模型上下文协议），允许用户灵活扩展自定义集成，连接如图像生成等外部能力。此外，个人谷歌账号即可享受免费的额度支持，且项目基于 Apache 2.0 协议完全开源，是提升终端工作效率的理想助手。",100752,"2026-04-10T01:20:03",[126,91,92,35],"插件",{"id":128,"name":129,"github_repo":130,"description_zh":131,"stars":132,"difficulty_score":41,"last_commit_at":133,"category_tags":134,"status":42},4721,"markitdown","microsoft\u002Fmarkitdown","MarkItDown 是一款由微软 AutoGen 团队打造的轻量级 Python 工具，专为将各类文件高效转换为 Markdown 格式而设计。它支持 PDF、Word、Excel、PPT、图片（含 OCR）、音频（含语音转录）、HTML 乃至 YouTube 链接等多种格式的解析，能够精准提取文档中的标题、列表、表格和链接等关键结构信息。\n\n在人工智能应用日益普及的今天，大语言模型（LLM）虽擅长处理文本，却难以直接读取复杂的二进制办公文档。MarkItDown 恰好解决了这一痛点，它将非结构化或半结构化的文件转化为模型“原生理解”且 Token 效率极高的 Markdown 格式，成为连接本地文件与 AI 分析 pipeline 的理想桥梁。此外，它还提供了 MCP（模型上下文协议）服务器，可无缝集成到 Claude Desktop 等 LLM 应用中。\n\n这款工具特别适合开发者、数据科学家及 AI 研究人员使用，尤其是那些需要构建文档检索增强生成（RAG）系统、进行批量文本分析或希望让 AI 助手直接“阅读”本地文件的用户。虽然生成的内容也具备一定可读性，但其核心优势在于为机器",93400,"2026-04-06T19:52:38",[126,35]]