XCS with an internal action table for non-Markov environments

To cope with sequential decision problems in non- Markov environments, learning classifier systems using the internal register have been proposed. Since, by utilizing the action part of classifiers, these systems control the internal register in the same way as choosing actions to the environment, they do not always work well. In this paper, we develop an effective learning classifier system with two different rule sets for internal and external actions. The first one is used for determining internal actions, that is, rules for controlling the internal register. It provides stable performance by separating control of the internal register from the action part of classifiers, and it is represented by “If [external state] & [internal state] then [internal action],” and we call a set of the first rules the internal action table. The second one is for selecting external actions as in the classical classifier system, but its structure is slightly different with the classical one; it is represented by “If [external state] & [internal state] & [internal action] then [external action].” In the proposed system, aliased states in the environment are identified by observing payoffs of a classifier and referring to the internal action table. To demonstrate the efficiency and effectiveness of the proposed system, we apply it to woods environments which are used in the related works, and compare the performance of it to those of the existing classifier systems.

@article{Hayashida2014,
  title     = {XCS with an internal action table for non-Markov environments},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {5},
  number    = {6},
  year      = {2014},
  publisher = {The Science and Information Organization},
  author    = {Tomohiro Hayashida and Ichiro Nishizaki and Keita Moriwake},
  doi       = {10.14569/IJACSA.2014.050626},
  url       = {https://doi.org/10.14569/IJACSA.2014.050626}
}