llama.cppIn discussion

llama.cpp

Hacker News users discussed the local inference engine llama.cpp and its desktop wrapper llama.app in a thread that reached 364 points and 164 comments. Many said they had moved from Ollama and found it simply works, while debating whether its built-in router mode can replace llama-swap for juggling models, how ROCm compares with Vulkan, and how much speed custom engines still leave on the table. Others questioned curl-pipe-to-shell installers and noted the llama.app page does not make clear whether it is affiliated with the official ggml project.

Hacker NewsAug 12, 202632 min read