Bridging the gap between raw visual perception and symbolic reasoning remains a central challenge for general-purpose AI agents. While deep learning models excel at object recognition, they often require task-specific training data and lack the structural guarantees needed by downstream logical solvers. We present an agnostic grid reconstruction framework that transforms 2D images of structured environments, specifically game boards, into logical assertions without pre-trained models or manual labeling. We propose two complementary methods: Grid-First Detection (GFD), based on classical image processing and genetic auto-tuning, and Object-First Detection (OFD), based on zero-shot segmentation for repeated-object discovery. Both produce a lightweight symbolic abstraction of the board that can be directly consumed by Answer Set Programming (ASP) encodings. We further embed this abstraction in a closed-loop sense–abstract–reason–act–verify architecture for dynamic visual environments. After an action is selected and executed, the system monitors the environment until the symbolic state stabilizes, avoiding decisions based on transient visual effects such as animations, cascades, or delayed updates. This supports robust replanning even when exact post-action prediction is unavailable. Overall, the approach enables rapid deployment of symbolic reasoning agents in visually rich, dynamic environments where data-driven training is impractical.

From Pixels to Predicates: A Robust Perception Front-End for ASP-Based Game Reasoning

Angilica, Denise;Ateeq, Tayyab;Bruno, Pierangela;Galardo, Emanuele;Ianni, Giovambattista
2026-01-01

Abstract

Bridging the gap between raw visual perception and symbolic reasoning remains a central challenge for general-purpose AI agents. While deep learning models excel at object recognition, they often require task-specific training data and lack the structural guarantees needed by downstream logical solvers. We present an agnostic grid reconstruction framework that transforms 2D images of structured environments, specifically game boards, into logical assertions without pre-trained models or manual labeling. We propose two complementary methods: Grid-First Detection (GFD), based on classical image processing and genetic auto-tuning, and Object-First Detection (OFD), based on zero-shot segmentation for repeated-object discovery. Both produce a lightweight symbolic abstraction of the board that can be directly consumed by Answer Set Programming (ASP) encodings. We further embed this abstraction in a closed-loop sense–abstract–reason–act–verify architecture for dynamic visual environments. After an action is selected and executed, the system monitors the environment until the symbolic state stabilizes, avoiding decisions based on transient visual effects such as animations, cascades, or delayed updates. This supports robust replanning even when exact post-action prediction is unavailable. Overall, the approach enables rapid deployment of symbolic reasoning agents in visually rich, dynamic environments where data-driven training is impractical.
2026
9783032383198
9783032383204
Agnostic Grid Reconstruction
Answer Set Programming
Genetic Auto-tuning
Knowledge Extraction from Images
State Representation
Symbolic Abstraction
Zero-shot Perception
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/20.500.11770/417879
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact