ECAI-2002 Conference Paper

[full paper]

Natural Language Texts for a Cognitive Vision System

Michael Arens, Artur Ottlik, Hans-Hellmut Nagel

Text-to-logic conversion is studied in a system approach which extracts a conceptual representation of temporal developments within a road traffic scene recorded by a video camera. A Fuzzy Metric Temporal Horn Logic (FMTHL) facilitates a schematic representation of road vehicle behavior at intersections. Geometric results of a model-based vehicle detection and tracking subsystem are used to interpret this generic conceptual FMTHL representation in order to obtain a conceptual description of the specific developments in the recorded traffic scene. One task accepts a natural language (NL) English text formulation of the generic conceptual knowledge and converts this into the internal FMTHL representation. This requires to distinguish algorithmically between discourse-related and scheme-related NL statements. A separate second task creates a synthetic video sequence from a NL text, using as much as possible the same apparatus as the first one. A comparison between genuine road traffic video sequences and those generated synthetically allows to test both the text-to-logic transformation and the use of geometric knowledge represented within the entire system for image sequence evaluation and subsequent interpretation. Results obtained by the execution of both tasks are discussed.

Keywords: Natural Language Processing, Knowledge Representation, Reasoning about Actions and Change, Knowledge-based Systems, Vision, Description Logics, Qualitative Reasoning, Temporal Reasoning

Citation: Michael Arens, Artur Ottlik, Hans-Hellmut Nagel: Natural Language Texts for a Cognitive Vision System. In F. van Harmelen (ed.): ECAI2002, Proceedings of the 15th European Conference on Artificial Intelligence, IOS Press, Amsterdam, 2002, pp.455-459.