açık kaynak LLM'ler

[Muhammed Uludag]

https://github.com/eugeneyan/open-llms?utm_source=bensbites&utm_medium=newsletter&utm_campaign=google-isn-t-ready-to-give-up

Open LLMs

These LLMs are all licensed for commercial use (e.g., Apache 2.0, MIT, OpenRAIL-M). Contributions welcome!

Language Model	Release Date	Checkpoints	Paper/Blog	Params (B)	Context Length	Licence
T5	2019/10	T5 & Flan-T5, Flan-T5-xxl (HF)	Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer	0.06 - 11	512	Apache 2.0
UL2	2022/10	UL2 & Flan-UL2, Flan-UL2 (HF)	UL2 20B: An Open Source Unified Language Learner	20	512, 2048	Apache 2.0
Cerebras-GPT	2023/03	Cerebras-GPT	Cerebras-GPT: A Family of Open, Compute-efficient, Large Language Models(Paper)	0.111 - 13	2048	Apache 2.0
Open Assistant (Pythia family)	2023/03	OA-Pythia-12B-SFT-8, OA-Pythia-12B-SFT-4, OA-Pythia-12B-SFT-1	Democratizing Large Language Model Alignment	12	2048	Apache 2.0
Pythia	2023/04	pythia 70M - 12B	Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling	0.07 - 12	2048	Apache 2.0
Dolly	2023/04	dolly-v2-12b	Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM	3, 7, 12	2048	MIT
DLite	2023/05	dlite-v2-1_5b	Announcing DLite V2: Lightweight, Open LLMs That Can Run Anywhere	0.124 - 1.5	1024	Apache 2.0
RWKV	2021/08	RWKV, ChatRWKV	The RWKV Language Model (and my LM tricks)	0.1 - 14	infinity (RNN)	Apache 2.0
GPT-J-6B	2023/06	GPT-J-6B, GPT4All-J	GPT-J-6B: 6B JAX-Based Transformer	6	2048	Apache 2.0
GPT-NeoX-20B	2022/04	GPT-NEOX-20B	GPT-NeoX-20B: An Open-Source Autoregressive Language Model	20	2048	Apache 2.0
Bloom	2022/11	Bloom	BLOOM: A 176B-Parameter Open-Access Multilingual Language Model	176	2048	OpenRAIL-M v1
StableLM-Alpha	2023/04	StableLM-Alpha	Stability AI Launches the First of its StableLM Suite of Language Models	3 - 65	4096	CC BY-SA-4.0
FastChat-T5	2023/04	fastchat-t5-3b-v1.0	We are excited to release FastChat-T5: our compact and commercial-friendly chatbot!	3	512	Apache 2.0
h2oGPT	2023/05	h2oGPT	Building the World’s Best Open-Source Large Language Model: H2O.ai’s Journey	12 - 20	256 - 2048	Apache 2.0
MPT-7B	2023/05	MPT-7B, MPT-7B-Instruct	Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs	7	84k (ALiBi)	Apache 2.0, CC BY-SA-3.0
RedPajama-INCITE	2023/05	RedPajama-INCITE	Releasing 3B and 7B RedPajama-INCITE family of models including base, instruction-tuned & chat models	3 - 7	2048	Apache 2.0
OpenLLaMA	2023/05	OpenLLaMA-7b-preview-300bt	OpenLLaMA: An Open Reproduction of LLaMA	7	2048	Apache 2.0

Open LLMs for code

Language Model	Release Date	Checkpoints	Paper/Blog	Params (B)	Context Length	Licence
SantaCoder	2023/01	santacoder	SantaCoder: don't reach for the stars!	1.1	2048	OpenRAIL-M v1
StarCoder	2023/05	starcoder	StarCoder: A State-of-the-Art LLM for Code, StarCoder: May the source be with you!	15	8192	OpenRAIL-M v1
StarChat Alpha	2023/05	starchat-alpha	Creating a Coding Assistant with StarCoder	16	8192	OpenRAIL-M v1
Replit Code	2023/05	replit-code-v1-3b	Training a SOTA Code LLM in 1 week and Quantifying the Vibes — with Reza Shabani of Replit	2.7	infinity? (ALiBi)	CC BY-SA-4.0
CodeGen2	2023/04	codegen2 1B-16B	CodeGen2: Lessons for Training LLMs on Programming and Natural Languages	1 - 16	2048	Apache 2.0
CodeT5+	2023/05	CodeT5+	CodeT5+: Open Code Large Language Models for Code Understanding and Generation	0.22 - 16	512	BSD-3-Clause

Open LLM datasets for pre-training

Name	Release Date	Paper/Blog	Dataset	Tokens (T)	License
starcoderdata	2023/05	StarCoder: A State-of-the-Art LLM for Code	starcoderdata	0.25	Apache 2.0
RedPajama	2023/04	RedPajama, a project to create leading open-source models, starts by reproducing LLaMA training dataset of over 1.2 trillion tokens	RedPajama-Data	1.2	Apache 2.0

Open LLM datasets for instruction-tuning

Name	Release Date	Paper/Blog	Dataset	Samples (K)	License
MPT-7B-Instruct	2023/05	Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs	dolly_hhrlhf	59	CC BY-SA-3.0
databricks-dolly-15k	2023/04	Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM	databricks-dolly-15k	15	CC BY-SA-3.0
OIG (Open Instruction Generalist)	2023/03	THE OIG DATASET	OIG	44,000	Apache 2.0

Open LLM datasets for alignment-tuning

Name	Release Date	Paper/Blog	Dataset	Samples (K)	License
OpenAssistant Conversations Dataset	2023/04	OpenAssistant Conversations - Democratizing Large Language Model Alignment	oasst1	161	Apache 2.0

Evals on open LLMs

• Leaderboard by lmsys.org
• Evals by MosaicML
• Holistic Evaluation of Language Models (HELM)
• LLM-Leaderboard
• TextSynth Server Benchmarks
• Open LLM Leaderboard by Hugging Face

--
A. M. Uludag
http://math.gsu.edu.tr/uludag/

twittter: @metamathician

[Akif Eyler]

Hocam, bu listeden ne öğreniyoruz, önemi nedir?

On Mon, May 22, 2023 at 7:03 PM Muhammed Uludag 

[Muhammed Uludag]

Bu listedeki LLM'ler açık kaynak, bilgisayarınıza indirip kendiniz eğitebilirsiniz.

Bir kısmı GPU gerektirebilir.

Ancak ChatGPT tarzı generative kabiliyetler beklememelisiniz. 

Sadece nitelik değil yavaşlık açısından da pek kullanışlı olmayacaklardır.

Türkçe konusundaki yeteneklerini bilmiyorum.

Keşke bizde bu modelleri çalıştırıp Türkçe yeteneklerini mukayese eden birileri olsaydı.

ChatGPT tarzı üretici LLM elde etmek için  büyük emekler ortaya dökmek gerekiyor.

Ancak buradaki açık kaynak modeller daha spesifik NLP tasklar için etkili olabilirler, mesela:

Duygu analizi

Semantik arama

İsim tanıma 

Hürmetler

--
A. M. Uludag
http://math.gsu.edu.tr/uludag/

twittter: @metamathician

--
Gruba davet etmek istediklerinize bu linki ulaştırabilirsiniz:
https://groups.google.com/g/yz-ve-insan/about
---
Bu iletiyi Google Grupları'ndaki "Yapay Zeka ve İnsan" grubuna abone olduğunuz için aldınız.
Bu grubun aboneliğinden çıkmak ve bu gruptan artık e-posta almamak için yz-ve-insan...@googlegroups.com adresine e-posta gönderin.
Bu tartışmayı web'de görüntülemek için https://groups.google.com/d/msgid/yz-ve-insan/CAC5h89xxt7VCZA%3Dh_e7%2BaAESmVmzGLGhUs4Y6qGOmbmYBXdDbQ%40mail.gmail.com adresini ziyaret edin.