Lead story
Models & availability
Latest
Lead story
Models & availability
Latest
LG AI Research introduces Segment-based Topic Allocation (SBTA), a method that assigns text segments to topics rather than entire documents, reducing topic contamination. They also propose the Segment Intrusion benchmark for evaluating topic coherence. Experiments show that LLMs using SBTA outperform traditional methods like LDA and BERTopic on the SemEval-STM benchmark.
From the source
Segment-based Topic Allocation (SBTA, right) extracts and groups only the text segments that are semantically relevant to the target topic, resulting in more precise topic assignment, higher semantic purity, and greater interpretability.
lgresearch.ai