Back

Speaker "Olaf De Leeuw" Details Back

 

Topic

Towards more energy-efficient Neural Networks? Use your brain!

Abstract

In the last decade many different types of neural networks have been developed. They showed us the amazing power and opportunities of machine learning. Everywhere in the world processes are replaced by ML algorithms, people are matched with their dream job, products are recommended and cars are driven automatically. It is truly amazing what we can do with such models. On the other hand, when you take a critical look, the whole training process is not that efficient. We have to feed models with millions of labeled images or text inputs to make sure your algorithm will perform well. And thinks of what happens in this training process. Each input goes through many layers where multiplications and ReLu or sigmoid functions are applied to each item from the input. Forward and backwards! Due to back propagation. Of course, with all the available computer in the form of GPU’s this is not really an issue. However this cost a lot of energy. With that in mind we do know that neural networks are sort of based on the way humans learn. Except that the human brain is much more energy efficient. Could we achieve that same energy-efficient level in artificial neural networks? The answer is yes! In this talk I will show you what is often called the third generation neural networks: Spiking Neural Networks. Based on the biological processes in the brain this kind of neural network uses discrete spikes and sparse communication to learn. I will give short introduction in some biological processes in the human brain and from there we will define spiking neural networks. We will discuss the downsides compared to artificial neural networks due to their discontinuous nature and I will show a resolution to that. The accuracy maybe still falls short of the artificial neural networks but the field is evolving and I will show the great potential of these networks. You will also get an overview of some existing frameworks based on Pytorch. Let’s go for great accuracy and major energy savings
Who is this presentation for?
Data Scientists and ML enthusiasts
Prerequisite knowledge:
Neural Networks, Pytorch
What you'll learn?
A new approach to neural networks. The advantages of the Spiking Neural Networks (SNN's) and the challenges. Some math behind it and some primary steps in using frameworks built for SNN's.

Profile

As a graduated Mathematician I'm particularly interested in the techniques and math behind algorithms. How do they search for the optimal solution and why is one algorithm faster than the other? In my work as a Data Scientist I develop algorithms or adapt existing solutions to customer needs and put them into production such they can get the most value out of it. In my own time I love to read popular scientific articles or books about mathematics, physics or astrophysics. Besides this I love traveling and cycling.