From the source
llama.cpp server now has a router mode that allows dynamic loading, unloading, and switching between multiple models without restarting the server.
From the source
llama.cpp server now ships with router mode, which lets you dynamically load, unload, and switch between multiple models without restarting.
huggingface.co
