ECAI-2000 Logo

ECAI-2000 Conference Paper

[PDF] [full paper] [prev] [tofc] [next]

Q-Surfing: Exploring a World Model by Significance Values in Reinforcement Learning Tasks

Frank Kirchner, Corinna Richter

Reinforcement Learning addresses the problem of learning to select actions in unknown environments. Due to the poor performance of Reinforcement Learning in more complex and thus more realistic tasks with large state spaces and sparse reinforcement, much effort is done to speed up learning as well as on finding structure in problem spaces. Models are introduced in order to improve learning by allowing to plan on the internal world model. This implies that a directed exploration in the model is a very important factor in relation to better learning results. In this paper we present an algorithm which explores the model by computing so-called Significance Values for each state. Using these values for model planning, during early stages knowledge propagation is enhanced, during later stages values in important states retain higher values and might therefor be useful for future decomposition of state spaces. Empirical results in a simple grid navigation task will demonstrate this process.

Keywords: Reinforcement Learning, Reuse of Knowledge

Citation: Frank Kirchner, Corinna Richter: Q-Surfing: Exploring a World Model by Significance Values in Reinforcement Learning Tasks. In W.Horn (ed.): ECAI2000, Proceedings of the 14th European Conference on Artificial Intelligence, IOS Press, Amsterdam, 2000, pp.311-315.


[prev] [tofc] [next]


ECAI-2000 is organised by the European Coordinating Committee for Artificial Intelligence (ECCAI) and hosted by the Humboldt University on behalf of Gesellschaft für Informatik.