I promise this Ollama replacement will give you the fastest t/s you've ever had

wowitsbase · 2026-09-09T21:52:17

A small developer team and I had worked on this for a while for personal reasons, so I have decided to port it to cpp and release it to the general public. The stats are in the github. I promise if you have a gpu this will improve your speeds by at least 50%, even if you believe your configuration is optimized.

FloatArtifact · 2026-09-09T22:00:59

Does it have Intel Arc support?

wowitsbase · 2026-09-09T22:11:50

Somewhat, but most of the optimizations are cuda based.

I would guess it would be a little faster, like 20-40%, but not 200% like on Nvidia.