A01Decision making based on multi-dimensional state- and action-space, and the neural coding in the basal ganglia

 Construction of state space and action space by learning from samples in the reinforcement learning is a fundamental and critical problem in artificial intelligence. It has been suggested that the basal ganglia have a pivotal role in the decision making to survive natural environment. The neural representation in the striatum, the input site of the basal ganglia from the cerebral cortex, has been reported as various information, such as motivation, action, a trigger of habit behavior, the action value, the stimulus value, the context-dependent value, the flexible value and the fixed value. Each of reports have discussed different functional role of the striatum, and the unified functional role of the basal ganglia is still controversial. In this research program, I propose novel hypothesis that the neural representation of striatum is learned a middle-layer representation of multi-layer neural network for multiple functions, such as the policy functions, action values, or state values, through the temporal difference error and the salience information which are carried by the mid-brain dopamine signal. To confirm the hypothesis, first, I will construct the network model of the basal ganglia, and formalize the learning rules that enables multiple functions for reinforcement learning. Second, I will predict what representation were learned through the model. Third, I will compare the predicted representation with the actual neural representation, which is recorded from the cerebral cortex and the striatum of macaque monkeys preformed a reinforcement learning task with multiple state and action space.