The problem of automatic selection of subgoals is currently one of the most relevant in adaptive control problems, in particular, in Reinforcement Learning. This paper proposes a logical-probabilistic approach to the construction of adaptive learning control systems capable of detecting deep implicit subgoals. The approach uses the ideas of the neurophysiological Theory of functional systems to organize the control scheme, and logical-probabilistic methods of machine learning to train the rules of the system and identify subgoals. The efficiency of the proposed approach is demonstrated by an example of solving a three-stage foraging problem containing two nested implicit subgoals

Alexander Demin, Cand. Sci. (Phys.–Math.), Ershov Institute of Informatics Systems, 6, Lavrentyev pr., Novosibirsk, 630090, Russian Federation, tel.: +7 (383) 330-66-60, e-mail: alexandredemin@yandex.ru

Demin A.V. Deep Learning of Adaptive Control Systems Based on a Logical-probabilistic Approach. The Bulletin of Irkutsk State University. Series Mathematics, 2021, vol. 38, pp. 65-83. (in Russian) https://doi.org/10.26516/1997-7670.2021.38.65

1. Anohin P.K. Fundamental questions of the general theory of functional systems. The principles of the systemic organization of functions. Moscow, Science Publ., 1973, pp. 5-61. (in Russian)
2. Vityaev E.E. Extracting Knowledge from Data. Computer Cognition. Models of Cognitive Processes. Novosibirsk, Novosibirsk State University Publ., 2006, 293 p. (in Russian)
3. Demin A.V., Vityaev E.E. A Logical Model of an Adaptive Control System. Neiroinformatika, 2008, vol. 3, no. 1, pp. 79-107. (in Russian)
4. Demin A.V., Vityaev E.E. Relational approach to knowledge discovery and its application. Materials of the All-Russian conference with international participation “Knowledge - Ontology – Theories” (KONT-2013). Novosibirsk, 2013, vol. 3, pp. 122–130. (in Russian).
5. Al-Emran Mostafa. Hierarchical Reinforcement Learning: A Survey. IJCDS Journal, 2015, vol. 4, no. 2, pp. 137-142. https://doi.org/10.12785/ijcds/040207
6. Dietterich T.G. Hierarchical reinforcement learning with the MAXQ value function decomposition. Journal of Artificial Intelligence Research, 2000, vol. 13, pp. 227–303. https://doi.org/10.1613/jair.639
7. Goodfellow Ian, Bengio Yoshua, and Courville Aaron. Deep learning. The MIT Press, 2016, 800 p.
8. Mnih V., Kavukcuoglu K., Silver D. et al. Human-level control through deep reinforcement learning. Nature, 2015, vol. 518, pp. 529-533. https://doi.org/10.1038/nature14236
9. Silver D., Huang A., Maddison C. et al. Mastering the game of Go with deep neural networks and tree search. Nature, 2016, vol. 529, pp. 484-489. https://doi.org/10.1038/nature16961
10. Sutton R.S., Barto A.G. Reinforcement Learning. London, MIT Press, 2012, 320 p.
11. Vityaev E.E., Demin A.V., Kolonin Y.A. Logical probabilistic biologically inspired cognitive architecture. Artificial General Intelligence - 13th International Conference, AGI 2020, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Gabler, 2020, vol. 12177 LNAI, pp. 337-346. https://doi.org/10.1007/978-3-030-52152-3_36
12. Vityaev E.E., Demin A.V. Cognitive architecture based on the functional systems theory. Procedia Computer Science, Elsevier, 2018, vol. 145, pp. 623-628. https://doi.org/10.1016/j.procs.2018.11.072
13. Vityaev E.E., Demin A.V. Recursive subgoals discovery based on the Functional Systems Theory. Biologically Inspired Cognitive Architectures, 2011, IOS Press, 2011, pp. 425-430.
14. Zhou Zhi-Hua and Feng Ji. Deep Forest: Towards An Alternative to Deep Neural Networks. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17), 2017, pp. 3553-3559. https://doi.org/10.24963/ijcai.2017/497
15. Zhou Zhi-Hua and Feng Ji. Deep Forest. National Science Review, 2019, vol. 6, no.1, pp. 74-86. https://doi.org/10.1093/nsr/nwy108