# Hugging Face — Evaluating Language Model Bias with 🤗 Evaluate

- Company: Hugging Face (huggingface.co)
- Announced: 2022-10-24
- Category: capability-change
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://huggingface.co/blog/evaluating-llm-bias
- Record: https://forck.live/items/2140-evaluating-language-model-bias-with-evaluate
- Subject: Platform
- Models affected: GPT-2, BLOOM

Hugging Face adds bias metrics and measurements to its Evaluate library, allowing users to evaluate language model bias in areas like toxicity, polarity, and hurtfulness using causal language models such as GPT-2 and BLOOM.

## Evidence

Verbatim from https://huggingface.co/blog/evaluating-llm-bias:

> we have been working on adding bias metrics and measurements to the 🤗 Evaluate library. In this blog post, we will present a few examples of the new additions and how to use them. We will focus on the evaluation of causal language models (CLMs) like GPT-2 and BLOOM

---

Record: https://forck.live/items/2140-evaluating-language-model-bias-with-evaluate
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md