Neuro-Symbolic Learning of Lifted Action Models from Visual Traces - LAAS-Robotique Access content directly
Conference Papers Year : 2024

Neuro-Symbolic Learning of Lifted Action Models from Visual Traces

Kai Xi
  • Function : Author
  • PersonId : 1378927
Stephen Gould
  • Function : Author
  • PersonId : 1378928

Abstract

Model-based planners rely on action models to describe available actions in terms of their preconditions and effects. Yet, manually encoding such models is challenging, especially in complex domains. Numerous methods have been proposed to learn action models from examples of plan execution traces. However, high-level information, such as state labels within traces, is often unavailable and needs to be inferred indirectly from raw observations. In this paper, we aim to learn lifted action models from visual traces — sequences of image-action pairs depicting discrete successive trace steps. We present ROSAME, a differentiable neuRO-Symbolic Action Model lEarner that infers action models from traces consisting of probabilistic state predictions and actions. By combining ROSAME with a deep learning computer vision model, we create an end-to-end framework that jointly learns state predictions from images and infers symbolic action models. Experimental results demonstrate that our method succeeds in both tasks, using different visual state representations, with the learned action models often matching or even surpassing those created by humans.
Fichier principal
Vignette du fichier
icaps24-rosame.pdf (3.3 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04561851 , version 1 (14-07-2024)

Licence

Public Domain

Identifiers

  • HAL Id : hal-04561851 , version 1

Cite

Kai Xi, Stephen Gould, Sylvie Thiébaux. Neuro-Symbolic Learning of Lifted Action Models from Visual Traces. International Conference on Automated Planning and Scheduling (ICAPS-24), Jun 2024, Banff, Canada. pp.653-662. ⟨hal-04561851⟩
83 View
11 Download

Share

Gmail Mastodon Facebook X LinkedIn More