meta releases LLM LLama 3.1 open source

[Jim DiNunzio]

big news in the open source LLM arena. meta’s LLama 3.1. This free to use and modify model cost hundreds of millions of dollars to make, and Meta is opening it to the public.

Early benchmarks show it nearly equal and sometimes even better than OpenAIs closed source GPT-4o. The 405 billion parameter model is the flagship which won’t run on your computer, but the 8 billion parameter one which also scores very well in its category will certainly run. 

I’m looking forward to checking it out and hope they use the standard API that OpenAI has given to the industry.

Jim

[Alan Timm]

Aw.... yeah!

The 405B parameter model is cool, but I continue to be impressed with the performance of 7B/8B models.  RLHF fine tuning really takes it to the next level.

In a few days TheBloke should be releasing a 4bit quantized version of it. And it only takes 4GB of graphics card memory when you run it.

Alan

[Chris Albertson]

Yes, this is true.  I get very good results with the reduced-size models.  It can run on my Mac Mini using Apple’s built-in GPU.   For robotics, the smaller models are all that you need.