# Upstage — Deploying Solar with BentoML

- Company: Upstage (upstage.ai)
- Announced: 2024-04-18
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://www.upstage.ai/blog/en/solar-with-bentoml
- Record: https://forck.live/items/18484-deploying-solar-with-bentoml
- Subject: Solar Pro / Solar Mini
- Models affected: Solar 10.7B, upstage/SOLAR-10.7B-Instruct-v1.0

Upstage published a guide on deploying its Solar 10.7B model using BentoML and vLLM. The post walks through setting up a BentoML service with OpenAI-compatible endpoints and running inference on a GPU with at least 16 GB VRAM.

## Evidence

Verbatim from https://www.upstage.ai/blog/en/solar-with-bentoml:

> In this blog, we will talk about how to deploy an LLM server powered by Solar and BentoML.

---

Record: https://forck.live/items/18484-deploying-solar-with-bentoml
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md