I keep looking at mac mini’s and wondering if I should run models locally. But they’re always so expensive I don’t really need to run AI models at all it’s just an idle curiosity.
I think I’m going to ignore AI until it either gets cheaper or builds us a warp drive.
Them being a great deal is gone
Call me when something lets me train and run my own 1.4T model locally, in about 120 years.
Those 120 years went by in a flash.
He is the world’s dumbest man afterall
I feel there is a catch here. I forgot mixture of experts was a thing -_-. They only activate like 32B at a time.
But this can be incredibly good, lets see!
Btw “and train” people missed the very very key word.
EDIT: @_@ training qwen 3.8 on under 6 GB. Yes, this seems VERY acceptable! I can see the endless potential here.
Nah, it seems it can finetune a 128 GB model with 6 GB RAM, I don’t see evidence that it can finetune something of 1.4T or more. I was excited for nothing.
Only 3 years behind. Getting better, Apple
I mean it’s apple and their AI is Siri. They’re lucky if they can even break even.
I give it a week before someone reverse engineers it and uploads a foss option.
Idk if I hate this kind of corpo project if they’re paying people for their work over more FOSS-slop
There are already a lot of FOSS options that run on Apple Silicon.
Oh sure I’d imagine so. What I meant was whatever magic sauce that makes this better will be assimilated.
Sooooo it’s Ollama but not FOSS?
https://sleepingrobots.com/dreams/stop-using-ollama/
Turns out that ollama is kinda garbage, and their desktop app wasn’t FOSS for a while too.
Also, modelfiles suck
Ooh I didn’t know that! Thank you! I like FOSS, so I’ll try out the stuff referenced in your article!
sus
it is called install asahi or buy a proper computer
Harbor is currently in a private preview under Apple’s Claris brand…
Now there’s a name I haven’t heard in a long time.
FileMaker is still going strong
Dynamic pooling could make this more accessible to lower power macs.
Are there other handlers that can bundle computers into a local Ai instance in a similar manner?
I guess vLLM supports clusters so you could just use that? The setup is more difficult than just plugging in a cable though.
Interesting. I kind of want to buy another Mac to dee if it works either way my M4 Max MBP.
