Despite the high effectiveness of the Deep Learning methods, they remain a ”thing in themselves”, a ”black box” decisions of which can not be trusted. This is critical for such areas as medicine, financial investments, military applications and others, where the price of the error is too high. In this regard, the European Union is going to demand in 2018 from companies that they give users an explanation of the solutions obtained by automatic systems.

In this paper, we offer an alternative, logical-probabilistic method of Deep Learning that can explain its decisions. This is a method of hierarchical clustering, based on the original logical-probabilistic generalization of formal concepts [13]. For comparison with deep learning based on neural networks, the work [12] was chosen, in which the task of processing natural language on a set of data UMLS is solved. To apply the logicalprobabilistic generalization of formal concepts, a classification algorithm based on the energy of the contradictions Energy Learning [10] is defined. Logical-probabilistic formal concepts are defined through fixed points, as well as the formal concepts themselves, only certain rules of probability are used as rules. The energy of contradictions allows us to resolve the contradictions arising at fixed points that form probabilistic formal concepts. It is shown that this clustering algorithm is not inferior in accuracy to the method of Deep Learning [12] nevertheless, the solutions obtained by it are explained by the set of probabilistic fixed-point rules.

For citation:

Martynovich V.V., Vityaev E.E. Explicative Deep Learning with Probabilistic Formal Concepts in a Natural Language Processing Task. The Bulletin of Irkutsk State University. Series Mathematics, 2017, vol. 22, pp. 31-49. (In Russian). https://doi.org/10.26516/1997-7670.2017.22.31

MSC

68T27

DOI

https://doi.org/10.26516/1997-7670.2017.22.31

1. Vityaev E.E. Izvlechenie znaniy iz dannykh. Komp’yuternoe poznanie. Modelirovanie kognitivnykh protsessov [Knowledge discovery. Computational cognition. Cognitive process models]. Novosibirsk, Novosibirsk State UniversityPress, 2006. 293 p. (in Russian)

2. Vorontsov K.V. Combinatorial approach to learning algorithms estimation. Matematicheskie voprosi kibernetiki, eds. Lupanov O.B., Moscow, FizMatLit, 2004, vol. 13, рp. 5-36. (in Russian).

3. Bordes A., Weston J., Collobert R., Bengio Y. Learning structured embeddings of knowledge bases. Proceedings of the 25th Conference on Artificial Intelligence, AAAI-11, USA, San Francisco, 2011.

4. Bordes A., Glorot X., Weston J., Bengio Y. Joint learning of words and meaning representations for open-text semantic parsing. Proc. of the 15th Intern. Conf. on Artif. Intel. and Stat., JMLR, 2012, vol. 22, pp. 127–135.

5. Bordes A. et al. A Semantic Matching Energy Function for Learning with Multirelational Data. Machine Learning: Special Issue on Learning Semantics, 2013.

6. Bengio, Y., Ducharme, R., Vincent, P. and Jauvin, C. A Neural Probabilistic Language Model. Machine Learning Research, 2003, pp. 1137-1155.

7. Ganter B. Formal Concept Analysis: Methods, and Applications in Computer Science. TU Dresden, Springer, 2003.

8. Ganter B., Obiedkov S. Implications in Triadic Formal Contexts. TU Dresden, Springer, 2004.

9. Goodfellow I., Bengio Y. and Courville A. Deep Learning. MIT Press, 2016.

10. LeCun Y. et al. A Tutorial on Energy-Based Learning. Predicting Structured Outputs, Bakir et al. (eds.), MIT Press, 2006.

11. Lobo J.M., Jimenez-Valverde A., Real R. AUC: a misleading measure of the performance of predictive distribution models. Global Ecology and Biogeography, 2008,vol. 17, pp. 145-151. https://doi.org/10.1111/j.1466-8238.2007.00358.x

12. A. T. McCray. An upper level ontology for the biomedical domain. Comparative and Functional Genomics, 2003, vol. 4, P. 80-88. https://doi.org/10.1002/cfg.255

13. Vityaev E.E., Martynovich V.V. Probabilistic Formal Concepts with Negation. Perspectives of System Informatics, A. Voronkov, I. Virbitskaite (eds.), LNCS, 2015, vol. 8974, pp. 385-399.