Bridging the gap between raw visual perception and symbolic reasoning remains a central challenge for general-purpose AI agents. While deep learning models excel at object recognition, they often require task-specific training data and lack the structural guarantees needed by downstream logical solvers. We present an agnostic grid reconstruction framework that transforms 2D images of structured environments, specifically game boards, into logical assertions without pre-trained models or manual labeling. We propose two complementary methods: Grid-First Detection (GFD), based on classical image processing and genetic auto-tuning, and Object-First Detection (OFD), based on zero-shot segmentation for repeated-object discovery. Both produce a lightweight symbolic abstraction of the board that can be directly consumed by Answer Set Programming (ASP) encodings. We further embed this abstraction in a closed-loop sense–abstract–reason–act–verify architecture for dynamic visual environments. After an action is selected and executed, the system monitors the environment until the symbolic state stabilizes, avoiding decisions based on transient visual effects such as animations, cascades, or delayed updates. This supports robust replanning even when exact post-action prediction is unavailable. Overall, the approach enables rapid deployment of symbolic reasoning agents in visually rich, dynamic environments where data-driven training is impractical.
From Pixels to Predicates: A Robust Perception Front-End for ASP-Based Game Reasoning
Angilica, Denise;Ateeq, Tayyab;Bruno, Pierangela;Galardo, Emanuele;Ianni, Giovambattista
2026-01-01
Abstract
Bridging the gap between raw visual perception and symbolic reasoning remains a central challenge for general-purpose AI agents. While deep learning models excel at object recognition, they often require task-specific training data and lack the structural guarantees needed by downstream logical solvers. We present an agnostic grid reconstruction framework that transforms 2D images of structured environments, specifically game boards, into logical assertions without pre-trained models or manual labeling. We propose two complementary methods: Grid-First Detection (GFD), based on classical image processing and genetic auto-tuning, and Object-First Detection (OFD), based on zero-shot segmentation for repeated-object discovery. Both produce a lightweight symbolic abstraction of the board that can be directly consumed by Answer Set Programming (ASP) encodings. We further embed this abstraction in a closed-loop sense–abstract–reason–act–verify architecture for dynamic visual environments. After an action is selected and executed, the system monitors the environment until the symbolic state stabilizes, avoiding decisions based on transient visual effects such as animations, cascades, or delayed updates. This supports robust replanning even when exact post-action prediction is unavailable. Overall, the approach enables rapid deployment of symbolic reasoning agents in visually rich, dynamic environments where data-driven training is impractical.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.


