The last decade has witnessed an experimental revolution in data science and machine learning, essentially based on two ingredients: representation (or feature learning) and backpropagation. Moreover the analysis of the behavior of deep learning is essentially done through the prism of probabilities. As long as artificial neural networks only capture statistical correlations between data and the tasks/questions that have to be performed/answered, this analysis may be enough. Unfortunately, when we aim at designing neural networks that behave more like animal brains or even humans’ ones, statistics is not enough and we need to perform another type of analysis. By introducing languages and theories in this framework, we will show that the problem of learning is, first, a problem of adequacy between data and the theories that are expressed. This adequacy will be rephrased in terms of toposes. We will unveil the relation between the so-called “generalization” and a stack that models this adequacy between data and the tasks. Finally a five level perspective of learning with neural networks will be given that is based on the architecture (base site), a presemantic (fibration), languages, theories and the notion of semantic information (joint work with Daniel Bennequin).