Lead story
Models & availability
Latest
Lead story
Models & availability
Latest
Google Research presents a novel approach for understanding user intents from UI interaction trajectories using small multimodal LLMs. The method decomposes the task into two stages: first summarizing each screen individually, then extracting intent from the summaries. The approach shows results comparable to larger models and is suitable for on-device applications.
From the source
We present a novel approach to tackle the task of understanding user intents from UI interaction trajectories using small models, which shows better results than significantly larger models.
research.google