looking for new 17" 128G 64bit amd laptop... (ultimately ubuntu)

[bruce]

Hey...

Anyone have any pointers on where to get one, and price.

Some of the online places that claim they do, don't.

thanks.

ps
this is/would be to test out reasonable LLM. 32G runs slow...
molasses slow. So, wondering if a larger mem footprint would make a
diff. I'm assuming a larger mem device, would also have a better
cpu/gpu handling..

thanks again,,

[Matt B.]

If you just want to run LLMs, I would suggest putting your money into a desktop and then remoting into it from an inexpensive laptop. 

Depending on what you want to do you will be bottlenecked either by VRAM or RAM (or both). GPUs run parallel tasks like inference (LLMs) and diffusion (image generation) much faster than GPUs, but the model size is generally best limited to the amount of VRAM in said GPU(s). 

On the other hand you can run larger models in RAM, even without any GPU, but the processing per token will be much slower as it will be CPU bound. You can split a model across RAM and VRAM, but it’ll suffer from having to swap data between them. 

My advice: rent some servers from something like Runpod to test out what RAM and GPU you can afford / want / need across different models. It’s cheap and would be essentially the same experience as running it locally on a desktop or laptop. Then if you still want a local device you’ll know the specs you’ll need. 

—Matt 

--
You received this message because you are subscribed to the Google Groups "LVL1 - Louisville's Hackerspace" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lvl1+uns...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/lvl1/CAP16ngqdWbmcqKNz0n2qVASx%2BnTk%2BaToA5Fzipuzh1OE-Y1sLA%40mail.gmail.com.

[bruce]

hey matt!

thanks for replying..

i was starting to test a few things with gemini  and goog changed to antigravity as the idea.  but, in doing that, something on their end brokecm with quotas.  ie, u fire it up, select a folder to open.  it comes back saying ur quotas expired.  

the goog dev forum has plenty of people rather perturbed on this.

so.  I was curious to know what size dev llm could be run on a 128g laptop.  I thought of a desktop  and remoting in, but that wouldn't travel around with me!

thanks

To view this discussion visit https://groups.google.com/d/msgid/lvl1/CAHiMttueJFtyDzdN9X%3DSzzPfACuYw8BnYLtxyzLzDncYFZFY5w%40mail.gmail.com.