рус eng
«THE BULLETIN OF IRKUTSK STATE UNIVERSITY». SERIES «MATHEMATICS»
«IZVESTIYA IRKUTSKOGO GOSUDARSTVENNOGO UNIVERSITETA». SERIYA «MATEMATIKA»
ISSN 1997-7670 (Print)
ISSN 2541-8785 (Online)

List of issues > Series «Mathematics». 2023. Vol 43

Machine Learning with Probabilistic Law Discovery: a Concise Introduction

Author(s)
Alexander V. Demin1, Denis K. Ponomaryov1

1Ershov Institute of Informatics Systems SB RAS, Novosibirsk, Russian Federation

Abstract
Probabilistic Law Discovery (PLD) is a logic based Machine Learning method, which implements a variant of probabilistic rule learning. In several aspects, PLD is close to Decision Tree/Random Forest methods, but it differs significantly in how relevant rules are defined. The learning procedure of PLD solves the optimization problem related to the search for rules (called probabilistic laws), which have a minimal length and relatively high probability. At inference, ensembles of these rules are used for prediction. Probabilistic laws are human-readable and PLD based models are transparent and inherently interpretable. Applications of PLD include classification/clusterization/regression tasks, as well as time series analysis/anomaly detection and adaptive (robotic) control. In this paper, we outline the main principles of PLD, highlight its benefits and limitations and provide some application guidelines.
About the Authors

Alexander V. Demin, Cand. Sci. (Phys.Math.), Ershov Institute of Informatics Systems, Novosibirsk, 630090, Russian Federation, alexandredemin@yandex.ru

Denis K. Ponomaryov, Cand. Sci. (Phys.Math.), Ershov Institute of Informatics Systems, Novosibirsk, 630090, Russian Federation, ponom@iis.nsk.su

For citation
Demin A. V., Ponomaryov D. K. Machine Learning with Probabilistic Law Discovery: a Concise Introduction. The Bulletin of Irkutsk State University. Series Mathematics, 2023, vol. 43, pp. 91–109. https://doi.org/10.26516/1997-7670.2023.43.91
Keywords
probabilistic rule learning, knowledge discovery, interpretable machine learning
UDC
004.85
MSC
68T05
DOI
https://doi.org/10.26516/1997-7670.2023.43.91
References
  1. Demin A.V. Klasterizatsii dannykh s ispol’zovaniem logiko-veroyatnostnogo podkhoda k izvlecheniyu znaniy. [Data Clusterization with the Logic Probabilistic Approach to Knowledge Discovery.] Informatsionnye tekhnologii v gumanitarnykh issledovaniyakh [Information Technology in the Humanities], 2015, vol. 21, pp. 28–33. (in Russian)
  2. Demin A.V. Deep Learning of Adaptive Control Systems Based on a Logicalprobabilistic Approach. Bulletin of Irkutsk State University, Series Mathematics, 2021, vol. 38, pp. 65–83. https://doi.org/10.26516/1997-7670.2021.38.65
  3. Demin A.V. Adaptive Locomotion Control System for Robots with Arbitrarily Modular Design. Procedia Computer Science, 2020, vol. 169, pp. 829–834. https://doi.org/10.1016/j.procs.2020.02.156
  4. Demin A.V. and Ponomaryov D.K. Interpretable Reinforcement Learning with Multilevel Subgoal Discovery. Proc. 21st International Conference on Machine Learning and Applications (ICMLA), Nassau, Bahamas, 2022. DOI 10.1109/ICMLA55696.2022.00043
  5. Demin A.V., Vityaev E.E. Relyatsionnyy podkhod k izvlecheniyu znaniy i ego primeneniya. [Relational Approach to Knowledge Discovery and its Applications]. Materialy Vserossiyskoy konferentsii s mezhdunarodnym uchastiem “Znaniya – Ontologii – Teorii” [Proc. ZONT Conference], Novosibirsk, 2013, vol. 1, pp. 122–130. (in Russian)
  6. Demin A.V., Vityaev E.E. Metod postroeniya “estestvennoy” klassifikatsii. [A Method for Building “Natural” Classification.] Informatsionnye tekhnologii v gumanitarnykh issledovaniyakh [Information Technology in the Humanities], 2010, vol. 15, pp. 16–22. (in Russian)
  7. Demin A.V., Vityaev E.E. Finansovye vremennye ryady: prognozirovanie i raspoznavanie narusheniy dinamiki. [Financial Time Series: Prediction and Detection of Dynamics Change.] Materialy Vserossiyskoy konferentsii s mezhdunarodnym uchastiem “Znaniya – Ontologii – Teorii” [Proc. ZONT conference], 2009, pp. 79–86. (in Russian)
  8. Demin A.V., Vityaev E.E. Tekhnologiya predskazaniya finansovykh vremennykh ryadov. [A Technology for Predicting Financial Time Series.] Informatsionnyy byulleten’ sed’moy mezhdunarodnoy konferentsii pamyati akademika A.P. Ershova “Perspektivy sistem informatiki” [Bulletin of 7th Intl. Ershov conference Perspectives of Informatics Systems], 2009, pp. 114–119. (in Russian)
  9. Demin A.V., Vityaev E.E. Adaptive Control of Multiped Robot. Procedia Computer Science, 2018, vol. 145, pp. 629–634. https://doi.org/10.1016/j.procs.2018.11.071
  10. Demin A.V., Vityaev E.E. Adaptive Control of Modular Robots. Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing, 2018, vol. 636, pp. 204–212. https://doi.org/10.1007/978-3-319-63940-6_29
  11. Demin A.V., Vityaev E.E. Learning in a Virtual Model of the C. elegans Nematode for Locomotion and Chemotaxis. Biologically Inspired Cognitive Architectures, 2014, vol. 7, pp. 9–14. https://doi.org/10.1016/j.bica.2013.11.005
  12. Demin A.V., Vityaev E.E. Razrabotka universal’noy sistemy izvlecheniya znaniy “DISCOVERY’ i ee primenenie. [The Development of a Universal DISCOVERY Knowledge Mining System and its Applications.] Vestnik NGU, seriya: Informatsionnye tekhnologii [Bulletin of Novosibirsk State University: Information Technologies], 2009, vol. 7, no. 1, pp. 73–83. (in Russian)
  13. Demin A.V., Vityaev E.E., Poloz T.L. Realizatsiya universal’noy sistemy izvlecheniya znaniy “Discovery” i ee primenenie v zadachakh meditsinskoy diagnostiki. [The Development of a Universal DISCOVERY Knowledge Mining System and its Applications in Medical Diagnosis.] Materialy Vserossiyskoy konferentsii s mezhdunarodnym uchastiem “Znaniya – Ontologii – Teorii” [Proc. ZONT conference], 2007, vol. 1, Novosibirsk, pp. 63–70. (in Russian)
  14. Gunopulos D., Khardon R., Mannila H., Saluja S., Toivonen H., and Sharma R.S. Discovering All Most Specific Sentences. ACM Transactions on Database Systems, 2003, vol. 28, no. 2. https://doi.org/10.1145/777943.777945
  15. Khomicheva I., Demin A.V., Vityaev E.E. Transcription Factor Binding Site Discovery by the Probabilistic Rules. Proc. 2nd Workshop on Data mining in Functional Genomics and Proteomics. The 18th European Conference on Machine Learning and the 11th European Conference on Principles and Practice of Knowledge Discovery in Databases, 2007, Warsaw, Poland, pp.104–109.
  16. Kuznetsov S.O. On the Intractability of Computing the DuquenneGuigues Base. J. of Universal Computer Science, 2004, vol. 10, no. 8. http://dx.doi.org/10.3217/jucs-010-08-0927
  17. Vityaev E.E., Kovalerchuk B.K., Fedotov A.M., Barahnin V.B., Belov S.D., Durdin D.S., Demin A.V. Obnaruzhenie zakonomernostey i raspoznavanie anomal’nykh sobytiy v potoke dannykh setevogo trafika. [Regularity Discovery and Anomaly Detection in Network Traffic.] Vestnik NGU, seriya: Informatsionnye tekhnologii [Bulletin of Novosibirsk State University: Information Technologies], 2008, vol. 6, no. 2, pp. 57–68. (in Russian)
  18. Vityaev E.E., Demin A.V., Ponomaryov D.K. Probabilistic Generalization of Formal Concepts. Programming and Computer Software, 2012, vol. vol. 38, no. 5, pp. 219–230. https://doi.org/10.1134/S0361768812050076
  19. Vityaev E.E., Demin A.V., Ponomaryov D.K. Probabilistic Concepts in Formal Contexts. Lecture Notes in Computer Science, 2012, vol. 7162, pp. 394–410. https://doi.org/10.1007/978-3-642-29709-0_33

Full text (english)