Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Unless I'm missing something, the full 1TB bandwidth isn't used because the memory layout is wrong.

But that's fixable.

Since it's memory bound, it might be possible to reach 15 tok/sec with this build.



I'm curious, what's wrong with the memory layout? You mean the ollama settings?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: