Apple Researchers Introduce Ferret-UI Lite, an On-Device AI Model for Seeing and Controlling UIs

Apple’s Ferret-UI Lite is a 3B-parameter model optimized for mobile and desktop screens, designed to interpret screen images, understand UI elements such as icons and text, and interact with apps by, e.g., reading messages, checking health data, and more.

By Sergio De Simone