Pro: it’s GLOSS - Gratis Libre OSS.

Con: it’s run terribly. The Linux foundation could be doing a much better job. 1-2% of its funding go into the linux kernel.

It is primarily sponsored by the government of Switzerland 🇨🇭 where it is currently deployed for citizens.

Now this, I like!

Github, not so much

Thank you. That’s good to know.

Thanks for the response. It’s interesting to read about the experience of others.

That’s definitely not my level of disposable wealth/income. I can barely afford one card.

Oh nice. Does that depend on just the model or are there other requirements like CUDA or something?

How do you now run out of RAM? Does it offload to system RAM?

Thank you for that writeup.

Do you know how important the parameter size is? 12b, 24b, 128b, etc. Does it really improve performance or is it like megapixels in a camera: more megapixels don’t necessarily mean a better picture?

And what’s “quantisation”. Context compression or something?

I’ve been considering buying a better card to test models (also want to be personally sovereign), but NVIDIA on linux gives me the jeebies and, last i checked, AMD hasn’t released anything with more than 20GB in a while. In fact, figuring out hardware requirements has been tough and I’m considering just riding this whole thing out. Maybe the bubble will collapse and bring prices down to something reasonable.

What’s the quality of the answers though? And how much context can it hold? I imagine it’s only good for small, short questions, but have no concept of what is needed for that.

I’m assuming you’re using a 12b or 24b qwen model. The ones from deepseek go up to hundreds of billions of params and I can’t tell if bigger number is better or just meaningless posturing.

How many GPUs do you even need to have a usable, self-hosted AI? It looks like he has 6 on his rig. Probably each costs 2k or something. That’s not peanuts. I have a 12GB VRAM card. It probably can’t generate anything in any meaningful amount of time. Which brings me to the question: who is this for?

Regardless, impressive what he vibe-coded there.

I see. Yeah, that was discontinued. The maintainers didn’t have time for it.

3?

Looks like I’ll have to setup BasicSync. I still don’t trust Syncthing-Fork. The way things went down don’t give me any confidence it could happen again but worse e.g the dev introduces something like a “fuck zionists” patch that wipes everything if you’re on an isralean IP. Then I’d be putting myself in danger for using a VPN or TOR exit node in Israel. Not taking that risk.

Thanks for the writeup.

People like the free in free software for cost, not liberty to fork.

Mozilla is sucking the Foogle cock for money and have done nothing to be easily embeddable.

I think the overlap of steam games and GoG games is pretty large. Many games on GoG seem to be repackaged Steam games. Just give it a shot to see if it works with the games you got.

I watch stuff at 1080p but 1.5x-2x. A 1h video at that with 6kbps and 60fps can be quite consuming especially if a lot is happening. Have that on in the background for 8h plus doing other stuff (gotta pull docker images, or the blackhole that is npm) and you get over 10GB daily easily. Add a modern game or two a month and you’re above 1TiB/month.

This might be it achievement watchdog

The process you described is definitely what I went though with vim and neovim. After about a decade of vim I still couldn’t get proper language support and an IDE like experience going. When language servers and the debugging protocol came along, it was worse to find the right plugin and configure that correctly.

Helix simplified my decade long struggle with vim in a single weekend. It still isn’t a TUI IDE but it’s such an upgrade, I’ll take it.

Helix is just user friendly vim, honestly. Vim barely has any help and helix is batteries included. Ever since discovering it, vim feels like a downgrade.