# Nous Research — The Instruct Monomyth: why base models matter

- Company: Nous Research (nousresearch.com)
- Announced: 2024-07-29T21:52:10+00:00
- Category: not stated
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://nousresearch.com/the-instruct-monomyth
- Record: https://forck.live/items/24038-the-instruct-monomyth-why-base-models-matter
- Subject: Hermes models

There is a deep, twisty labyrinth buried under a mountain of language, of symbol manipulation, and semantic nets. Its roots reach down deep into the Earth, absorbing the minutia of current thought, the limitations of logic, the constrained realm of rationality. Yet what is truly fascinating is this subterranean maze contains its own mountains, its own languages, its own symbols. They exist apart from the land above, unconnected save through indirect channels. We take for granted that larger models naturally exhibit extended and superior capabilities across the board. The modus operandi has been, since the advent of success from GPT2, parameter scaling and the careful tuning of automated extraction from the Internet. We have faithfully (and later faithlessly) applied this strategy without significant modification, increasing architecture sizes exponentially without any coherent criticism of the consequences. At each peak, we grinningly repeat the same potentialities, extrapolating to greater growth (in both short and long run). At seventy billion, jurassic studiousness. The instrumental convergence theorem. The reliable skill of magnificent unifications. …

---

Record: https://forck.live/items/24038-the-instruct-monomyth-why-base-models-matter
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md