Another good option is Qwen3.5-9B. I’d try a Q4 or Q5.
Whichever model you choose, I’d also suggest a few MCP servers to give it access to reliable information rather than rely entirely on its internal knowledge.
Microsoft Learn MCP: Official PowerShell, Windows and network documentation. Will help with your scripting stuff.
I’d start with Microsoft Learn and Exa, then add the others as needed. You’ll need an MCP-capable client (which pretty much all of them are now - eg: llama.cpp web ui, OpenWebUI, Light Local Chat etc), and I’d keep file access read-only initially.
I’d personally use McProxy instead of adding MCPs singularly, but I fear I’m already over loading you with information.
EDIT: The PolyStrata recommendation assumes you have a discrete GPU with sufficient VRAM (unclear if you do?). If not, disregard Ornith and try Tiny Ling 3 Q6 or Qwen3.5-9B with ik_llama.cpp instead.
Either way, both smaller models should be capable of the tasks you’ve described, particularly with access to the right documentation. Just verify any network or firewall changes before applying them.
With an iGPU results are going to be very slow. Most promising things I have heard of so far is Mixture of Experts type models, if you have an 8GB card you might be able to run Qwen 3.8-35B-A3B-MoE which I think is the most capable for that hardware.
There are websites that can suggest a few for your specific hardware, I assume some are vibecoded and are only occasionally updated to the latest models, so take it as you will. See this Lemming’s comment. And !localLLaMa@sh.itjust.works is where I think most of the discussion around new development with Local LLMs happens here on Lemmy.
Is there a local LLM I can run on an 8-10th gen i5 with 16GB RAM?
Yes, quite a few, but it depends on what you’re trying to do with them. Any ideas?
Assist with writing letters, home assistant code, light powrshell scripting.
Edit: helping segment my network
That should be well within reach of a local LLM.
If you’ve got a discrete GPU with sufficient VRAM, something like Ornith running on PolyStrata could be worth a shot. Try a Q3 size.
https://huggingface.co/bartowski/Ornith-1.5-35B-A3B-GGUF
https://github.com/VecSzn/PolyStrata
Hell, Tiny Ling 3 might be able to do it. Try a Q6.
https://huggingface.co/bartowski/Ling-3.0-tiny-GGUF
Another good option is Qwen3.5-9B. I’d try a Q4 or Q5.
Whichever model you choose, I’d also suggest a few MCP servers to give it access to reliable information rather than rely entirely on its internal knowledge.
https://learn.microsoft.com/en-us/training/support/mcp
https://github.com/exa-labs/exa-mcp-server
https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem
If you’re actually running Home Assistant, its official MCP server is worth a look too.
https://www.home-assistant.io/integrations/mcp_server/
I’d start with Microsoft Learn and Exa, then add the others as needed. You’ll need an MCP-capable client (which pretty much all of them are now - eg: llama.cpp web ui, OpenWebUI, Light Local Chat etc), and I’d keep file access read-only initially.
I’d personally use McProxy instead of adding MCPs singularly, but I fear I’m already over loading you with information.
EDIT: The PolyStrata recommendation assumes you have a discrete GPU with sufficient VRAM (unclear if you do?). If not, disregard Ornith and try Tiny Ling 3 Q6 or Qwen3.5-9B with ik_llama.cpp instead.
https://github.com/ikawrakow/ik_llama.cpp
Either way, both smaller models should be capable of the tasks you’ve described, particularly with access to the right documentation. Just verify any network or firewall changes before applying them.
With an iGPU results are going to be very slow. Most promising things I have heard of so far is Mixture of Experts type models, if you have an 8GB card you might be able to run Qwen 3.8-35B-A3B-MoE which I think is the most capable for that hardware.
There are websites that can suggest a few for your specific hardware, I assume some are vibecoded and are only occasionally updated to the latest models, so take it as you will. See this Lemming’s comment. And !localLLaMa@sh.itjust.works is where I think most of the discussion around new development with Local LLMs happens here on Lemmy.
links