Special for the blog: an interview with Natalia Efremova, Chief Analyst at NtechLab: about neural networks, machine learning, the best Russian programmers and “smart” houses for everyone.
Girls are different, and they can have the most unusual interests, as well as professions. Although today everybody asserts that the boundaries between purely male and female occupations are erased and that there is a desire for gender equality in the professional sphere, society is not yet fully accustomed to seeing female truck drivers or car mechanics. Our heroine does not drive huge trucks, does not lie in oily overalls under the hood of the car, and, in general, her workplace and tools do not exceed the weight of a personal laptop, although her programming profession for a long time was considered a male-only.
Natalia Efremova, an analyst at NtechLab and a lecturer of the Intelligent Information Systems course at the Plekhanov Russian Economic University, finds it difficult to find free time in a busy schedule. At the very beginning of spring, we managed to talk to Natasha and learn some details about neural networks, their past, present and future.
NtechLab: Hi, Natasha! Thank you for agreeing to have a chat. Well, first of all, let me express my admiration for what you do, what you have dedicated your life to — research of neural networks. Not the most typical profession for a girl, although it’s not customary to divide the line of work by gender. And yet, the first question is simple and logical: how did you come to your profession, your kind of activity, when you realized that you want to do it, that it’s going to be your thing?
NATALIA EFREMOVA: I came into this field by accident. I studied at school with a humanitarian bias and was engaged in English literature. After school I got into Russian State University of Humanities, but at that time I had enough literature and went to study linguistics at the Faculty of Artificial Intelligence. Acquaintance with applied linguistics very quickly led to the understanding that it is, well, not my thing at all. That’s when I had the chance to do the recognition of natural language, which eventually led me to neural networks. Since then, I have been programming neural networks for 10 years.
NTL: What does it mean to do neural networks? Is it necessary to have some super programming skills, a very strong knowledge of mathematics, an understanding of brain structure
N.: There are different degrees of immersion, I would say. Each degree allows you to solve a certain problem. I think that with the current development of neural networks it is not necessary to understand exactly what happens in each function. Neural networks are now at the stage where they are more of a production tool. When we use a computer, we do not fully understand how it works, what processes are going on inside, but we do not need to. You can build a neural network (like a Lego) from different components and work on improving it further. Today there are already modern libraries that can be used; specifically customized libraries can be used for special tasks. In addition, there is an opinion of what types of neural networks are used for what tasks, there is an idea of what type of development is needed for this or that task, whether it is step-by-step or end-to-end development. Although neural network specialists are not tired of offering non-standard ways of using “traditional” algorithms, such as, for example, the use of recurrent networks for image recognition (for more details, see here).
The knowledge base about neural networks itself is so wide and so well used by so many that you don’t have to be an expert in this field to use it anymore. For example, to build a simple neural network to recognize handwritten characters with your own hands, it is enough to have basic skills of python programming, the ability to connect libraries for the python and knowledge of what libraries currently exist.
NTL: What’s your workspace setup?
N.: Here’s my laptop, it travels with me around the world, and if necessary I connect to Amazon servers to do some calculations, but for each task I do my own Amazon server configuration.
NTL: Could you explain neural networks in a really basic way?
N.: I teach the course “Neural Networks” at the Plekhanov Academy and usually begin the story of what are neural networks, with a description of a scene from the cartoon Madagascar. There is a moment in the second movie when penguins are assembling a plane. They call in monkeys and say, “Monkeys, we need your thumbs and frontal lobes.” Monkeys’ thumbs are needed to hold a lot of tools, and their frontal lobes are needed to figure out how to collect a working plane from all the garbage they have at hand. Why frontal lobes? Because monkeys are the only creatures on earth that have the same brain structure as humans. Thinking processes such as pattern recognition, natural language and primates are similar in many ways.
The human brain, like all mammals, is made up of grey and white matter. Grey matter consists of small cells called neurons. These cells perform certain calculations. Each cell has only one function: it “decides” whether to transmit the signal further or not, to get agitated or not. And the multitude of stimuli in the multitude of neurons in our brain creates a certain computing network. This computational network decides every second what to do next: whether to turn my head, look at some object, how to interpret what I heard. It’s the same principle as artificial neural networks. For each minimal fragment of the task a small neuron element is responsible, combining a large number of which allows us to make very complex parallel calculations, including image recognition.
NTL: What can you call complicated and what is simple? Say, will the task of scratching require complex calculations?
N.: It’s a rather complex signal.
NTL: Then what is a simple signal?
N.: A simple signal is, for example, to figure out what I see in front of me without doing anything. Processing what we get from sensory organs is probably one of the easiest things to do. The movement already requires coordinating what we see with the activation of our motor functions, it is a more complex task. We train our neurons for the entire life. Our complex network begins to train in the womb, and the training process continues throughout life.
NTL: Does it look like an artificial neural network (ANN) is being trained?
N.: The ANN also has the training, but it’s a bit simpler, and the function can be described by a formula. Everything that can be described by mathematical formulas is simpler than biological processes. The biological neural network (BNN) is complex because there is a lot of redundancy in the BNN, a lot of parallel processes are duplicated, repeated several times, paralleling several tasks. And the ANN is configured to perform one task, such as face recognition, or natural language recognition, or drone control. I would not compare the learning processes taking place in the BNN and the ANN. In many ways, they are different processes. In the case of a person, the whole large neural network is trained at once. The ANN is usually trained in installments. The ANN can be compared to a piece of the human brain, which performs a certain function. It’s as if the human brain was only set up to recognize images, a sort of a human eye.
In my explanation of neural networks on monkeys from Madagascar, I use the image of their frontal lobes as a symbol of thinking as we know it: a monkey is better able than any other mammal to figure out what can be made out of different parts. Each area of our brain has certain functions, for example, the neural network for Vision will be analogous to the visual cortex, the neural network for Natural Language will be analogous to the areas that form and recognize the natural language in the brain, the so-called Wernicke and Brock areas. The frontal lobes in the human brain are involved in logical calculations and decision making.
NTL: How long does it take for students to get it? They’re not programmers.
N.: They are economists but from the department of computer science. It’s different. There are those who are quick to grab. Besides, I don’t go into mathematical details, into the proofs of theorems and other mathematical conclusions, because, as I have already said before, it is not so important for the modern use of neural networks. Recently I read an article online that said that you do not need a PhD to program neural networks. We can say that this is both true and not entirely true. Everything depends on the task and type of neural network. In the work of some neural networks, it is not easy even for me with my 10 years of experience to understand.
By the way, in December I was at a big conference of NIPS (Neural Information Processing Systems) in Barcelona. There were over 6 thousand specialists in machine training there, and they were mostly neural network specialists. There were many speeches and seminars. I must admit, I didn’t know everything. To become a specialist in one field you have to sacrifice many others. You can do everything, of course, but it’s very difficult. If you are involved in the convolutional neural networks, you will be a specialist in that field, and neural networks that use training with reinforcements will no longer be as transparent. I do pattern recognition, computer vision, it’s mostly convolutional neural networks.
NTL: How did you end up at NtechLab?
N.: I first learned about NtechLab while being abroad. I saw the article in The Guardian. And the funny thing is that there the company appeared as a group of Russian hackers, which launched its own facial recognition algorithm and made a lot of fuss in social networks, recognized “who is who”, helped someone, and pissed someone off. I started to wonder, I dug deeper, I found out that this is not just a group of hackers, but a whole laboratory that deals with facial recognition, and therefore neural networks. It was always interesting to me. Then I learned that the company was recruiting a team (I was contacted by “headhunter” Natalia), and soon I was invited to join the company. I came and arranged a meeting with top-management. This meeting was very inspiring and it was very interesting for me to talk to a smart person who speaks the same language as me. What else got me hooked? The fact that the algorithm won on the MegaFace competition is, certainly, impressive. Since I often work with neural network developers, I know that winning some international challenge is very difficult, there are plenty of skilled teams! Such a victory shows that this system is really capable. It’s certainly recognized. I recall how the guys from Imperial College London remembered that Russian programmers beat them on MegaFace when we met with them for the first time. Many of my colleagues at various universities around the world say that Russian programmers are the best. It is the Russians who really top many competitions.
NTL: Interesting. It seemed to be such an outdated myth about the toughest Russian programmers, and you confirm the opposite. Well, that’s nice. Natasha, in addition to being an expert in neural networks and machine learning, and a leading analyst at NtechLab, you are also involved in various scientific projects. Tell us more about it.
N.: Now, as part of my studies at Oxford, I am engaged in research in conjunction with the University of Oxford. It is a project to process satellite maps for monitoring drinking water supplies and for forecasting their depletion. It is more theoretical than applied research at this stage, but many companies are already trying to find practical applications. These are mainly startups located in Silicon Valley. Moreover, you should clearly understand that a startup today is not necessarily a team of three programmers who gather money to launch some applications. For example, I got acquainted with the guys from the startup who launch microsatellites into space. Now there is a big movement to launch microsatellites to get more information from satellite maps for later commercial use. This joint scientific project with Oxford is related to such research and aims to help investment companies calculate the risks associated with the depletion of natural resources.
NTL: So this isn’t strictly a humanitarian story? Finding sources of drinking water, helping poor countries during drought, etc.
N.: Basically, this project can be aimed at anything, but now there are more and more cases when investment companies start to deal with environmental problems. They have to calculate all the risks associated with the insurance of enterprises, individuals and companies, with real estate insurance, with the general market movement. There are, for example, risks associated with possible warming and a 3-degree rise in temperature. There is a whole industry that deals exclusively with this problem and calculates the relevant risks.
NTL: What future do you think we should expect from the development of research in neural networks? What exactly are we going to do with neural network-based technologies in the very near future?
N.: I think that in the next 2−3 years we will automate many processes such as language recognition, natural language production, dialogue in natural language, video recognition, video prediction, etc.
NTL: What does video prediction mean?
N.: When at the beginning of the action you can predict what will happen next. In most of the production of films and cartoons, which involves computer graphics, everything will be as automated as possible. On the other hand, I think that the so-called Internet of Things will develop further. There will be devices in the house that will minimize energy and water consumption. The system will be configured to automatically turn off the heating and so on. Now the so-called smart houses are available mainly to wealthy people. And in the future, these new-fashioned “smart” features will be cheap and accessible to everyone. I think it will happen soon enough. Now many energy companies are already working to ensure that people minimize their energy consumption. The transition to smart consumption of resources is inevitable, as we are increasing in number, and we need to minimize our impact on the environment. This will provoke not only smart houses but also smart cars, the appliances will calculate when it is better to take a shower so that we would have enough hot water. And the more resources are scarce, the better technology will develop to use those resources. We just won’t have another option, because the lack of resources in the future will become more and more severe, and what we now see as the norm in terms of spending resources will be considered wild, ignorant, something completely unreasonable. Ideally, we will also have our sports, diets, etc. calculated. In a restaurant, you will take pictures of your meal and get information about how many miles you need to run to burn these calories. There are already such developments, but it will be massive, casual, it will be an integral part of our lives.
There’ll be devices for some special occasions. There are already glasses for the blind in the form of an application that tells a person what’s in front of them. And that’s just the beginning.
NTL: What about the distant future? Can you imagine something that so far seems to be absolute fiction, but what can be realized one day through the development of our knowledge about neural networks?
N.: As for the distant future and fantasy, I would bet, in terms of gadgets, for example, that everything will be translucent and wearable. Most likely, it will be something built into glasses or worn on your wrist, built into a contact lens, etc. We are already on our way to it, the Advice Systems will be smarter than they are now, all these recommendations on health, reading, leisure, prediction systems, prediction of wishes.
NTL: What must be done in order for this future to come?
N.: It is absolutely necessary to study neural networks, because this discipline is now at the point of development when it passes from research to technology. There is such a notion as technology S-curve [1] [2], which describes the development of innovative technologies very well. Today, they are at the point where rapid development begins, but in a year it will be too late to enter this field, so we have to do it now. If there is enough knowledge, desire and expertise, the neural network research should continue. Whether it’s face recognition, natural language, or satellite image analysis.
NTL: Is it possible to create artificially an absolute likeness of human life, a human being?
N.: No, of course not. It’s impossible now, but I don’t think it’s necessary. If we get to the point where it is possible to create an absolute likeness, surely already then it will be possible to create something better, more perfect, a certain superhuman, a “fifth element” if you will. That is, already now we should think not about copying our own kind, but about creating a creature of, say, a higher order.
NTL: Natasha, you’ve been asked a “question from the street”, let’s call it that, and we can’t help but bring it up: Should we be afraid of singularity and when it will come?
N.: I get that question a lot. In a nutshell, what is singularity? It’s such a hypothetical moment when the accumulated information becomes so complicated that we can’t understand it. Perhaps we will move to the next level of human development after the creation of artificial intelligence, or the creation of a mechanism where we ourselves will digitize and exist in some info-like space when the body is not needed. Scientists believe that this is what the end of the world will look like. There are several theories on the subject (my favorite is the self-developing artificial intelligence from Gibson’s Neuromancer). If you ask me for my opinion, I think it’s a somewhat exaggerated perception of reality. The overpopulation, global warming, the depletion of resources that we talked about before, and so on, are much more pressing. I am more inclined to think that in the future, resources will be so exhausted that we won’t be able even to afford computers and the singularity problem will disappear by itself. We will live, grow carrots, cabbage, and that is if we are lucky and global warming doesn’t happen. Anyway, I’d worry less about singularity than about global environmental disaster, sadly.
NTL: These are not very cheerful prospects. Will we be able to prevent or overcome this global catastrophe?
N.: That will depend on how soon humanity begins to take its problems seriously. Now we have moved into an era where these problems are denied. Hopefully, it will stop soon.
NTL: Interesting. So there is hope, and the future is not predetermined. I’d love to continue our conversation soon. Natasha, thank you so much for the fascinating story, for sharing your opinion with our readers. Wishing you success in all your endeavors!
- Geroski, P.A. (2003). The Evolution of New Markets. Oxford: Oxford University Press
- Ventresca, M. J & Zhao, M. (2012). Course Note: «System-builders and the evolution of large-scale technology: Lessons for ecosystem and infrastructure».