From the source

Deploying Solar with BentoML

Guide to deploying Solar 10.7B with BentoML

Upstage published a guide on deploying its Solar 10.7B model using BentoML and vLLM.

The post walks through setting up a BentoML service with OpenAI-compatible endpoints and running inference on a GPU with at least 16 GB VRAM.

From the source
In this blog, we will talk about how to deploy an LLM server powered by Solar and BentoML.
upstage.ai

Other stories

Upstage· 6 OctProduct launch

Connects scattered documents into actionable workflows

Upstage· 1 OctNew model

3B-active-parameter MoE model optimized for agent workloads

ModelsSolar 10.7B, upstage/SOLAR-10.7B-Instruct-v1.0

SP4 scores 42 on Artificial Analysis, matching frontier models at lower cost.

Upstage· 20 AugNew model

Solar Pro 4 targets complex agent workloads with 512K context and 128K output tokens.

Upstage· 11 AugNew model

Upstage releases open-weight foundation model for agentic use

Upstage· 22 JulOpen-weight release

Upstage wins two Stevies for leadership and company achievement.

Upstage· 23 AprCompany news