The Role of the Critic in Learning Systems

TLDR: This article analyzes the three tasks of the Critic as threefold: evaluation of the past actions of the performance element of the learning system, localization of credit and blame to particular portions of that performance element, and recommendation of possible improvements and modifications in the performanceelement.

Abstract: Buchanan, Mitchell, Smith, and Johnson (1978) described a general model of learning systems that included a component called the Critic. The task of the Critic was described as threefold: evaluation of the past actions of the performance element of the learning system, localization of credit and blame to particular portions of that performance element, and recommendation of possible improvements and modifications in the performance element. This article analyzes these three tasks in detail and surveys the methods that have been employed in existing learning systems to accomplish them. The principle method used to evaluate the performance element is to develop a global performance standard by (a) consulting an external source of knowledge, (b) consulting an internal source of knowledge, or (c) conducting deep search. Credit and blame have been localized by (a) asking an external knowledge source to do the localization, (b) factoring the global performance standard to produce a local performance standard, and (c) conducting controlled experiments on the performance element. Recommendations have been conmiunicated to the learning element using (a) local training instances, (b) correlation coefficients, and (c) partially-instantiated schemata.