Dharmendra S. Modha
IBM
TrueNorth's neurons to revolutionize system architecture
Six years ago, IBM and our university partners embarked on a quest – to build a brain-inspired machine – that at the time appeared impossible. Today, in an article published in Science, we deliver on the DARPA SyNAPSE metric of a one million neuron brain-inspired processor. The chip consumes merely 70 milliwatts, and is capable of 46 billion synaptic operations per second, per watt–literally a synaptic supercomputer in your palm.
 |
|
| Dharmendra Modha, IBM Fellow. |
Along the way – progressing through Phase 0, Phase 1,Phase 2, and Phase 3 – we have journeyed from neuroscience to supercomputing, to a new computer architecture, to a new programming language, toalgorithms, applications, and now to a new chip – TrueNorth.
Let me take this opportunity to take you through the road untraveled. At this moment, I hope this reflection will incite within you a burning desire to collaborate and partner with us to make the future journey a joint one.
Retrospective
Today’s computers can be traced back at least to Blaise Pascal’s 1642 mechanical calculator. The modern era in computing started with the unveiling of ENIAC on February 15, 1946. The development of the transistor in 1948 enabled the creation of integrated circuits in 1958, which, in turn, enabled the first microprocessor in 1971. Since then the clock frequency of the microprocessors has increased 1,000-fold. As remarkable as this evolution is, it has been headed in a direction diametrically opposite to the computing paradigm of the brain. Consequently, today’s microprocessors are eight orders of magnitude faster (in terms of clock rate) and four orders of magnitude hotter (in terms of power per unit cortical area) than the brain.
 |
|
| TrueNorth Chip Core Array. |
Considering overall energy consumption underscores the divergence between the brain and today’s computers even more starkly. Note that a “human-scale” simulation with 100 trillion synapses (with relatively simple models of neurons and synapses) required 96 Blue Gene/Q racks of the Lawrence Livermore National Lab Sequoia supercomputer – and, yet, the simulation ran 1,500 times slower than real-time. A hypothetical computer to run this simulation in real-time would require 12 GW, whereas the human brain consumes merely 20 W.
What explains this disparity?
There are two factors: technology and architecture. Unlike today’s inorganic silicon technology, the brain uses biophysical, biochemical, organic wetware. While future enabling nanotechnology is underway, we focused on the second factor: architecture innovation – specifically, on minimizing the product of power, area, and delay in a system that could be implemented in today’s state-of-the-art technology.
Perspective
The cerebral cortex is hypothesized to comprise repeating canonical cortical microcircuits. Inspired by this hypothesis, in 2011, we demonstrated an event-driven “worm-scale” neurosynaptic core that integrated computation and memory. Now, we have shrunk the neurosynaptic core by 15-fold in area and 100-fold in power, and have tiled 4,096 cores via an on-chip network to create TrueNorth – with one million neurons and 256 million synapses. It is worth noting that we had only committed to deliver a chip with 1,024 cores, but, in November 2011, as a team, we made a gutsy decision to increase the scale four-fold to 4,096 cores. Fabricated in Samsung’s 28 nm process, with 5.4 billion transistors, TrueNorth is IBM’s largest chip to date in transistor count. While simulating complex recurrent neural networks, TrueNorth consumes < 100 mW of power and has a power density of 20 mW/cm2.
Unlike the prevailing von Neumann architecture – but like the brain – TrueNorth has a parallel, distributed, modular, scalable, fault-tolerant, flexible architecture that integrates computation, communication, and memory and has no clock. It is fair to say that TrueNorth completely redefines what is now possible in the field of brain-inspired computers, in terms of size, architecture, efficiency, scalability, and chip design techniques.
 |
|
| View "Brain Power" Infographic |
Designing and testing TrueNorth was no cakewalk. Its unprecedented size, unconventional architecture, new hybrid synchronous-asynchronous circuit methodology, and a new unfamiliar technology process required custom design, verification, and testing methodologies that demanded innovation, team work, and project management at the highest level. A critical element was one-to-one equivalence – at the functional level of spikes – between TrueNorth and our software simulator, Compass. This equivalence allowed us to begin developing applications long before chips returned from the foundry and to verify correctness of the chip logic.
Having exhausted all available means and tools for verifying the chip before fabrication, to ensure no stone was left unturned I even offered a $1,000 bottle of champagne to anyone who could find a bug. None was found. It was not until a year later – after the chip passed all unit, regression, functional, and multi-chip communication tests – that we were certain no fatal bugs existed. My champagne was safe!
 |
|
| IBM SyNAPSE Team. Photo Credit: Hita Bambhania-Modha |
The project simply could not have succeeded without the innovative spirit and the tremendous dedication of the current core team provided the critically important asynchronous circuit design tools that we jointly refined over the course of the project. Collaboration with Samsung was critical in gaining access to their advanced 28 nm foundry process that allowed balancing the low active power of the architecture with matching low power of the underlying silicon technology. I am immensely grateful to our 200+ collaborators since 2008 – spanning eight IBM labs and fabs, five universities, one start-up, and two Department of Energy laboratories. Finally, DARPA’s mandate, metrics, and investment were absolutely vital.
Prospective
Let’s be clear: we have not built the brain, or any brain. We have built a computer that is inspired by the brain. The inputs to and outputs of this computer are spikes. Functionally, it transforms a spatio-temporal stream of input spikes into a spatio-temporal stream of output spikes.
If one were to measure activities of 1 million neurons in TrueNorth, one would see something akin to a night cityscape with blinking lights. Given this unconventional computing paradigm, compiling C++ to TrueNorth is like using a hammer for a screw. As a result, to harness TrueNorth, we have designed an end-to-end ecosystem complete with a new simulator, a new programming language, an integrated programming environment, new libraries, new (and old) algorithms as well as applications, and a new teaching curriculum (affectionately called, “SyNAPSE University”). The goal of the ecosystem is to dramatically increase programmer productivity. Metaphorically, if TrueNorth is “ENIAC”, then our ecosystem is the corresponding “FORTRAN.”
We are working, at a feverish pace, to make the ecosystem available – as widely as possible – to IBMers, universities, business partners, start-ups, and customers. In collaboration with the international academic community, by leveraging the ecosystem, we foresee being able to map the existing body of neural network algorithms to the architecture in an efficient manner, as well as being able to imagine and invent entirely new algorithms.
 |
|
| Synapse 16 chip board |
To support these algorithms at ever increasing scale, TrueNorth chips can be seamlessly tiled to create vast, scalable neuromorphic systems. In fact, we have already built systems with 16 million neurons and 4 billion synapses. Our sights are now set high on the ambitious goal of integrating 4,096 chips in a single rack with 4 billion neurons and 1 trillion synapses while consuming ~4 kW of power.
The architecture can solve a wide class of problems from vision, audition, and multi-sensory fusion, and has the potential to revolutionize the computer industry by integrating brain-like capability into devices where computation is constrained by power and speed. These systems can efficiently process high-dimensional, noisy sensory data in real time, while consuming orders of magnitude less power than conventional computer architectures.
On one hand, with portable devices: think smart phones, sensor networks, self-driving automobiles, robots, public safety, medical imaging, real-time video analysis, signal processing, olfactory detection, and digital pathology. On the other hand, with synaptic supercomputers: – think multimedia processing on the cloud. In addition, our chip can be used in combination with other cognitive computing technologies to create systems that learn, reason and help humans make better decisions. Over time, our hope is that SyNAPSE will become an integral component of IBM Watson group offerings.
We have been working with iniLabs Ltd., creators of a retinal camera – the DVS – that directly produces spikes, which are the natural inputs for TrueNorth. Integrating the two, we have begun investigating extremely low-power end-to-end vision systems.
If we think of today’s von Neumann computers as akin to the “left-brain” – fast, symbolic, number-crunching calculators, then TrueNorth can be likened to the “right-brain” – slow, sensory, pattern recognizing machines.
We envision augmenting our neurosynaptic cores with synaptic plasticity to create a new generation of field-adaptable neurosynaptic computers capable of online learning.
 |
|
| Left Brain / Right Brain |
I was not there when ENIAC was unveiled, but I have a palpable sense that we are at a similar turning point in the history of computing. The technological and practical possibilities are immense and could touch every sphere of science, technology, business, government, and society. I am optimistic that the enduring value of our work will be the inspiration of a completely different way of thinking about computing. It will, I believe, spawn an outpouring of creativity by universities, startups, established tech companies, and by professionals in countless industries and occupations.
We are not there yet. Indeed, TrueNorth is a direction and not a destination! The end goal is building intelligent business machines that enable a cognitive planet, while transforming industries. Exciting!
