From the source
Structured output remains a common failure mode for language models, especially when schemas become complex, and strings require careful escaping.
Constrained generation can enforce syntactic validity, but it cannot by itself make the model choose the right fields, values, or escaped content.
Even under a schema constraint, the model's logits still need to meaningfully reflect the user's requested structure.
We built IFStruct, a generative benchmark that tests output validity and schema following.
The task is highly learnable for small models: LFM2.5-350M, trained with RL on a dedicated held-out training split, can exceed the performance of far larger models like Qwen3.5-4B and granite-4.0-h-tiny.
Structured output is one of the most common real-world tasks for LLMs.
IFStruct targets something existing benchmarks miss: that organic user requests present schema requirements in a variety of ways, often with additional constraints about formatting requirements.
What IFStruct measures The intent of IFStruct is to give a narrow signal on the question: “Can the model produce valid structured output and follow diverse schema requirements?”
…
