# Perplexity — How to Fine-Tune an LLM [+When to Use RAG]

- Company: Perplexity (perplexity.ai)
- Announced: 2026-10-30
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://www.perplexity.ai/hub/blog/fine-tuning-llm
- Record: https://forck.live/items/29917-how-to-fine-tune-an-llm-when-to-use-rag
- Subject: Perplexity

Perplexity's guide explains how to fine-tune a large language model and when to reach for prompting or retrieval-augmented generation instead. Fine-tuning refines a pre-trained model on a small, task-specific dataset of example input/output pairs, moving its weights toward the behavior you want, and parameter-efficient methods like LoRA make it cheap enough to run on a single GPU. The guide advises working up a ladder of prompting, then RAG, then fine-tuning, and says prompting is the fastest and cheapest lever, though longer prompts slow work and cost more. It draws the line at knowledge versus behavior: RAG changes what the model knows, while fine-tuning changes how it behaves, so RAG suits facts that change often or live in your own documents. Fine-tuning carries risks the guide lists, including catastrophic forgetting, overfitting on repetitive data, staleness of knowledge baked into weights, and obsolescence when a stronger model ships. Production systems often pair fine-tuned models with RAG, using one for consistent behavior and the other for current facts.

## Evidence

Verbatim from https://www.perplexity.ai/hub/blog/fine-tuning-llm:

> Fine-tuning a large language model (LLM) means refining a pre-trained LLM on a small, task-specific dataset so it delivers outputs you can’t get from prompting alone.
> 
> Fine-tuning involves collecting example input/output pairs that show the behavior you want, running training passes that move the model’s weights toward them, and checking the result against the original model outputs. Parameter-efficient methods like LoRA make this cheap enough to run on a single GPU.
> 
> If you aren't satisfied with the LLM outputs, refine your prompts and retrieval-augmented generation (RAG) first. Fine-tuning changes how the LLM behaves. When you need it to use fresh or proprietary information, prompting and RAG are usually a better choice.
> 
> Here's how to fine-tune an LLM step by step, the main fine-tuning methods (LoRA and QLoRA), and the challenges and mistakes that can break the process.
> 
> What's LLM fine-tuning and how it's different from LLM pre-training
> 
> LLM pre-training builds a model’s general language ability from scratch on a vast corpus, while LLM fine-tuning adjusts a pre-trained model on a small dataset for one job.

---

Record: https://forck.live/items/29917-how-to-fine-tune-an-llm-when-to-use-rag
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md