The Myth Of AI. A Conversation with Jaron Lanier

The idea that computers are people has a long and storied history. It goes back to the very origins of computers, and even from before. There's always been a question about whether a program is something alive or not since it intrinsically has some kind of autonomy at the very least, or it wouldn't be a program. There has been a domineering subculture—that's been the most wealthy, prolific, and influential subculture in the technical world—that for a long time has not only promoted the idea that there's an equivalence between algorithms and life, and certain algorithms and people, but a historical determinism that we're inevitably making computers that will be smarter and better than us and will take over from us. ...

That mythology, in turn, has spurred a reactionary, perpetual spasm from people who are horrified by what they hear. You'll have a figure say, "The computers will take over the Earth, but that's a good thing, because people had their chance and now we should give it to the machines." Then you'll have other people say, "Oh, that's horrible, we must stop these computers." Most recently, some of the most beloved and respected figures in the tech and science world, including Stephen Hawking and Elon Musk, have taken that position of: "Oh my God, these things are an existential threat. They must be stopped."

In the history of organized religion, it's often been the case that people have been disempowered precisely to serve what was perceived to be the needs of some deity or another, where in fact what they were doing was supporting an elite class that was the priesthood for that deity. ... That looks an awful lot like the new digital economy to me, where you have (natural language) translators and everybody else who contributes to the corpora that allows the data schemes to operate, contributing to the fortunes of whoever runs the computers. You're saying, "Well, but they're helping the AI, it's not us, they're helping the AI." It reminds me of somebody saying, "Oh, build these pyramids, it's in the service of this deity," and, on the ground, it's in the service of an elite. It's an economic effect of the new idea. The new religious idea of AI is a lot like the economic effect of the old idea, religion.

[39:47]

JARON LANIER is a Computer Scientist; Musician; Author of Who Owns the Future? 

Jaron Lanier's Edge Bio Page

INTRODUCTION 

by John Brockman

This past weekend, during a trip to San Francisco, Jaron Lanier stopped by to talk to me for an Edge feature. He had something on his mind: news reports about comments by Elon Musk and Stephen Hawking, two of the most highly respected and distinguished members of the science and technology communiity, on the dangers of AI. ("Elon Musk, Stephen Hawking and fearing the machine" by Alan Wastler, CNBC 6.21.14). He then talked, uninterrupted, for an hour.

As Lanier was about to depart, John Markoff, the Pulitzer Prize-winning technology correspondent for THE NEW YORK TIMES, arrived. Informed of the topic of the previous hour's conversation, he said, "I have a piece in the paper next week. Read it." A few days later, his article, "Fearing Bombs That Can Pick Whom to Kill" (11.12.14), appeared on the front page. It's one of a continuing series of articles by Markoff pointing to the darker side of the digital revolution.

This is hardly new territory. Cambridge cosmologist Martin Rees, the former Astronomer Royal and President of the Royal Society, addressed similar topics in his 2004 book, Our Final Hour: A Scientist's Warning, as did computer scientist, Bill Joy, co-founder of Sun Microsystems, in his highly influential 2000 article in Wired, "Why The Future Doesn't Need Us: Our most powerful 21st-century technologies — robotics, genetic engineering, and nanotech — are threatening to make humans an endangered species".

But these topics are back on the table again, and informing the conversation in part is Superintelligence: Paths, Dangers, Strategies, the recently published book by Nick Bostrom, founding director of Oxford University’s Institute for the Future of Humanity. In his book, Bostrom asks questions such as "what happens when machines surpass humans in general intelligence? Will artificial agents save or destroy us?"

I am encouraging, and hope to publish, a Reality Club conversation, with comments (up to 500 words) on, but not limited to, Lanier's piece. This is a very broad topic that involves many different scientific fields and I am sure the Edgies will have lots of interesting things to say.

—JB

Related on Edge:

Jaron Lanier: 

• "Digital Maoism: The Hazards of the New Online Collectivism" (2006) ;
• "One Half A Manifesto" (2000)

Kevin Kelly: 

• "The Technium" (2014)

George Dyson: 

• "Turing's Cathedral" (2004)

THE MYTH OF AI (Transcript)

A lot of us were appalled a few years ago when the American Supreme Court decided, out of the blue, to decide a question it hadn't been asked to decide, and declare that corporations are people. That's a cover for making it easier for big money to have an influence in politics. But there's another angle to it, which I don't think has been considered as much: the tech companies, which are becoming the most profitable, the fastest rising, the richest companies, with the most cash on hand, are essentially people for a different reason than that. They might be people because the Supreme Court said so, but they're essentially algorithms.

IBM's Brain-Inspired Computer Chip Comes from the Future

Illustration: IBM

Brain-inspired computers have tickled the public imagination ever since Arnold Schwarzenegger's character in “Terminator 2: Judgment Day” uttered: “My CPU is a neural net processor; a learning computer.” Today, IBM researchers backed by U.S. military funding unveiled a new computer chip that they say could revolutionize everything from smartphones to smart cars—and perhaps pave the way for neural networks to someday approach the computing capabilities of the human brain.

The IBM neurosynaptic computer chip consists of one million programmable neurons and 256 million programmable synapses conveying signals between the digital neurons. Each of chip’s 4,096 neurosynaptic cores includes the entire computing package—memory, computation, and communication. They all operate in parallel based on “event-driven” computing, similar to the signal spikes and cascades of activity when human brain cells work in concert. Such architecture helps to bypass the bottleneck in traditional computing where program instructions and operation data cannot pass through the same route simultaneously.

“We have not built a brain,” says Dharmendra Modha, chief scientist and founder of IBM’s Cognitive Computing group at IBM Research-Almaden. “But we have come the closest to creating learning function and capturing it in silicon in a scalable way to provide new computing capability that was not possible before.”

Such capability could enable new mobile device applications that emulate the human brain’s capability to swiftly process information about new events or other changes in real-world environments, whether that involves recognizing familiar sounds or a certain face in a moving crowd. IBM envisions its new chips working together with traditional computing devices as hybrid machines—providing an added dose of brain-like intelligence for smart car sensors, cloud computing applications or mobile devices such as smartphones. The chip's architecture was detailed in a new paper published in the 7 August online issue of the journal Science.

Add caption

With a total of 5.4 billion transistors the computer chip, named TrueNorth, is one of the largest CMOS chips ever built. Yet the chip uses just 70 milliwatts while running and has a power density of 20 milliwatts per square centimeter— almost 1/10,000th the power of most modern microprocessors. That brings the new chip's efficiency much closer to the human brain’s astounding power consumption of just 20 watts, or less than the average incandescent light bulb.

“This is literally a supercomputer the size of a postage stamp, light like a feather, and low power like a hearing aid,” Modha says.

One reason IBM was able to minimize power usage is that its chip's computation only triggers when needed. Traditional computer chips have a clock that uses power to trigger and coordinate all the computational processes. But the IBM chip's digital neurons can work together asynchronously when triggered by the signal spikes. IBM also designed its chip to have low power consumption by creating an on-chip network to interconnect all the neurosynaptic cores and building the chip with a low-power process technology used for making mobile devices.

It’s also a supercomputer that can easily scale up in size. IBM designed its computer chip architecture so that it could simply add new neurosynatpic cores within the chip. The chips themselves can be arranged in a repeatable 2-D tile pattern to create bigger machines—IBM has already tested that idea with a 16-chip configuration. That’s the “blueprint of a scalable supercomputer,” Modha says.

Past brain-inspired neural networks have used a combination of both analog and digital to represent the individual neurons. IBM chose to represent the neurons in digital form, which provided several advantages. (At least one other project, SpiNNaker also depends on digital.)

First, the choice allowed IBM engineers to avoid the physical problems of dealing with differences in the manufacturing process or temperature fluctuations. Second, it provided a “one to one equivalence with software and hardware” that allowed the IBM software team to build applications on a simulator even before the physical chip had been designed and tested—applications that ran without problems on the finished chip. Third, the lack of analog circuitry allowed the IBM team to dramatically shrink the size of its circuits. (IBM fabricated its chip using Samsung’s 28-nm process technology—typical for manufacturing chips for mobile devices.)

IBM’s new chip represents the culmination of a decade of Modha’s personal research and almost six years of funding from the U.S. Defense Advanced Research Projects Agency (DARPA). Modha currently heads DARPA’s SyNAPSE project, a global effort that has committed US $53 million to making learning computers since 2008.

Now IBM has built an entire ecosystem around its new chip hardware and software, including a new programming language and a curriculum to teach coders everything they need to know. And the company is reaching out to potential customers, universities, government agencies, and IBM employees to fully explore the commercial applications of its chip technology.

“Our long-term end goal is to build a ‘brain in a box’ with 100 billion synapses consuming 1 kilowatt of power,” Modha says. “In the near future, we’ll be looking at multiple things for empowering smartphones, mobile devices and cloud services with this technology.”

Learn More DARPA SyNAPSE IBM cognitive computing brain-inspired computerslearning computers neural networks neuro-inspired computing

ORIGINAL: IEEE Spectrum

By Jeremy Hsu

Posted 7 Aug 2014 | 18:00 GMT

Palm's Jeff Hawkins is building a brain-like AI. He told us why he thinks his life's work is right

Inside a big bet on future machine intelligence

Feature Jeff Hawkins has bet his reputation, fortune, and entire intellectual life on one idea: that he understands the brain well enough to create machines with an intelligence we recognize as our own.

If his bet is correct, the Palm Pilot inventor will father a new technology, one that becomes the crucible in which a general artificial intelligence is one day forged. If his bet is wrong, then Hawkins will have wasted his life. At 56 years old that might sting a little.

"I want to bring about intelligent machines, machine intelligence, accelerated greatly from where it was going to happen and I don't want to be consumed – I want to come out at the other end as a normal person with my sanity," Hawkins told The Register. "My mission, the mission of Numenta, is to be a catalyst for machine intelligence."

A catalyst, he says, staring intently at your correspondent, "is something which accelerates a reaction by a thousand or ten thousand or a million-fold, and doesn't get consumed in the process."

His goal is ambitious, to put it mildly.

Before we dig deep into Hawkins' idiosyncratic approach to artificial intelligence, it's worth outlining the state of current AI research, why his critics have a right to be skeptical of his grandiose claims, and how his approach is different to the one being touted by consumer web giants such as Google.

AI researcher Jeff Hawkins

The road to a successful, widely deployable framework for an artificial mind is littered with failed schemes, dead ends, and traps. No one has come to the end of it, yet. But while major firms like Google and Facebook, and small companies like Vicarious, are striding over well-worn paths, Hawkins believes he is taking a new approach that could take him and his colleagues at his company,Numenta, all the way.

For over a decade, Hawkins has poured his energy into amassing enough knowledge about the brain and about how to program it in software. Now, he believes he is on the cusp of a great period of invention that may yield some very powerful technology.

Some people believe in him, others doubt him, and some academics El Reg has spoken with are suspicious of his ideas.

One thing we have established is that the work to which Hawkins has dedicated his life has become an influential touchstone within the red-hot modern artificial intelligence industry. His 2004 book, On Intelligence, appears to have been read by and inspired many of the most prominent figures in AI, and the tech Numenta is creating may trounce other commercial efforts by much larger companies such as Google, Facebook, and Microsoft.

"I think Jeff is largely right in what he wrote in On Intelligence," explains Hawkins' former colleague Dileep George (now running his own AI startup, Vicarious, which recently received $40m in funding from Mark Zuckerberg, space pioneer Elon Musk, and actor-turned-VC Ashton Kutcher). "Hierarchical systems, associative memory, time and attention – I think all those ideas are correct."

One of Google's most prominent AI experts agrees: "Jeff Hawkins ... has served as inspiration to countless AI researchers, for which I give him a lot of credit," explains former Google brain king and current Stanford Professor Andrew Ng.

Some organizations have taken Hawkins' ideas and stealthily run with them, with schemes already underway at companies like IBM and federal organizations like DARPA to implement his ideas in silicon, paving the way for neuromorphic processors that process information in near–real time, develop representations of patterns, and make predictions. If successful, these chips will make Qualcomm's "neuromorphic" Zeroth processors look like toys.

He has also inspired software adaptations of his work, such as CEPT, which has built an intriguing natural language processing engine partly out of Hawkins' ideas.

How we think: time and hierarchy

Hawkins' idea is that to build systems that behave like the brain, you have to be able to 

• take in a stream of changing information, 
• recognize patterns in it without knowing anything about the input source, 
• make predictions, and 
• react accordingly. 

The only context you have for this analysis is an ability to observe how the stream of data changes over time.

Though this sounds similar to some of the data processing systems being worked on by researchers at Google, Microsoft, and Facebook, it has some subtle differences.

Part of it is heritage – Hawkins traces his ideas back to his own understanding of how our neocortex works based on a synthesis of thousands of academic papers, chats with researchers, and his own work at two of his prior tech companies, Palm and Handspring, whereas the inspiration for most other approaches are neural networks based on technology from the 80s, which itself was refined out of a 1940s paper [PDF], "A Logical Calculus of the Ideas Immanent in Nervous Activity".

"That may be the right thing to do, but it's not the way brains work and it's not the principles of intelligence and it's not going to lead to a system that can explore the world or systems that can have behavior," Hawkins tells us.

So far he has outlined the ideas for this approach in his influential On Intelligence, plus a white paper published in 2011, a set of open source algorithms called NuPIC based on his Hierarchical Temporal Memory approach, and hundreds of talks given at universities and at companies ranging from Google to small startups.

Six easy pieces and the one true algorithm

Hawkins' work has "popularized the hypothesis that much of intelligence might be due to one learning algorithm," explains Ng.

Part of why Hawkins' approach is so controversial is that rather than assembling a set of advanced software components for specific computing functions and lashing them together via ever more complex collections of software, Hawkins has dedicated his research to figuring out an implementation of a single, basic approach.

This approach stems from an observation that our brain doesn't appear to come preloaded with any specific instructions or routines, but rather is an architecture that is able to take in, process, and store an endless stream of information and develop higher-order understandings out of that.

The manifestation of Hawkins' approach is the Cortical Learning Algorithm, or CLA.

"People used to think the neocortex was divided into sensory regions and motor regions," he explains. "We know now that is not true – the whole neocortex is sensory and motor."

Ultimately, the CLA will be a single system that involves both sensory processing and motor control – brain functions that Hawkins believes must be fused together to create the possibility of consciousness. For now, most work has been done on the sensory layer, though he has recently made some breakthroughs on the motor integration as well.

To build his Cortical Learning Algorithm system, Hawkins says, he has developed six principles that define a cortical-like processor. These traits are

• "on-line learning from streaming data", 
• "hierarchy of memory regions", 
• "sequence memory", 
• "sparse distributed representations",
•  "all regions are sensory and motor", and 
• "attention".

These principles are based on his own study of the work being done by neuroscientists around the world.

Now, Hawkins says, Numenta is on the verge of a breakthrough that could see the small company birth a framework for building intelligence machines. And unlike the hysteria that greeted AI in the 70s and 80s as the defense industry pumped money into AI, this time may not be a false dawn.

"I am thrilled at the progress we're making," he told El Reg one sunny afternoon at Numenta's whiteboard-crammed offices in Redwood City, California. "It's accelerating. These things are compounding, and it feels like these things are all coming together very rapidly."

The approach Numenta has been developing is producing better and better results, he says, and the CLA is gaining broader capabilities. In the past months, Hawkins has gone through a period of fecund creativity, and has solved one of the main problems that have bedeviled his system (temporal pooling), he says. He sees 2014 as a critical year for the company.

He is confident that he has bet correctly – but it's been a hard road to get here.

That long, hard road

Hawkins' interest in the brain dates back to his childhood, as does his frustration with how it is studied.

Growing up, Hawkins spent time with his father in an old shipyard on the north shore of Long Island, inventing all manner of boats with his father, an inventor with the enthusiasm for creativity of a Dr. Seuss character. In high school, the young Hawkins developed an interest in biophysics and, as he recounts in his book On Intelligence, tried to find out more about the brain at a local library.

"My search for a satisfying brain book turned up empty. I came to realize that no one had any idea how the brain actually worked. There weren't even any bad or unproven theories; there simply were none," he wrote.

This realization sparked a lifelong passion to try to understand the grand, intricate system that makes people who they are, and to eventually model the brain and create machines built in the same manner.

Hawkins graduated from Cornell in 1979 with a Bachelor of Science in Electronic Engineering. After a stint at Intel, he applied to MIT to study artificial intelligence, but had his application rejected because he wanted to understand how brains work, rather than build artificial intelligence. After this he worked at laptop start-up GRiD Systems, but during this time "could not get my curiosity about the brain and intelligent machines out of my head," so he did a correspondence course in physiology and ultimately applied to and was accepted in the biophysics program at the University of California, Berkeley.

When Hawkins started at Berkeley in 1986, his ambition to study a theory of the brain collided with the university administration, which disagreed with his course of study. Though Berkeley was not able to give him a course of study, Hawkins spent almost two years ensconced in the school's many libraries reading as much of the literature available on neuroscience as possible.

This deep immersion in neuroscience became the lens through which Hawkins viewed the world, with his later business accomplishments – Palm, Handspring – all leading to valuable insights on how the brain works and why the brain behaves as it does.

The way Hawkins recounts his past makes it seem as if the creation of a billion-dollar business in Palm, and arguably the prototype of the modern smartphone in Handspring, was a footnote along his journey to understand the brain.

This makes more sense when viewed against what he did in 2002, when he founded the Redwood Neuroscience Institute (now a part of the University of California at Berkeley and an epicenter of cutting-edge neuroscience research in its own right), and in 2005 founded Numenta with Palm/Handspring collaborator Donna Dublinksy and cofounder Dileep George.

These decades gave Hawkins the business acumen, money, and perspective needed to make a go at crafting his foundation for machine intelligence.

Controversy

His media-savvy, confident approach appears to have stirred up some ill feeling among other academics who point out, correctly, that Hawkins hasn't published widely, nor has he invented many ideas on his own.

Numenta has also had troubles, partly due to Hawkins' idiosyncratic view on how the brain works.

In 2010, for example, Numenta cofounder Dileep George left to found his own company, Vicarious, to pick some of the more low-hanging fruit in the promising field of AI. From what we understand, this amicable separation stemmed from a difference of opinion between George and Hawkins, as George tended towards a more mathematical approach, and Hawkins to a more biological one.

Hawkins has also come in for a bit of a drubbing from the intelligentsia, with NYU psychology professor Gary Marcus dismissing Numenta's approach in a New Yorker article titled "Steamrolled by Big Data".

Other academics El Reg interviewed for this article did not want to be quoted, as they felt Hawkins' lack of peer reviewed papers combined with his entrepreneurial persona reduced the credibility of his entire approach.

Hawkins brushes off these criticisms and believes they come down to a difference of opinion between him and the AI intelligentsia.

"These are complex biological systems that were not designed by mathematical principles [that are] very difficult to formalize completely," he told us.

"This reminds me a bit of the beginning of the computer era," he said. "If you go back to the 1930s and early 40s, when people first started thinking about computers they were really interested in whether an algorithm would complete, and they were looking for mathematical completeness, a mathematical proof, that if you implemented something like an algorithm today when we build a computer, no one sits around saying "Let's look at the mathematical formalism of this computer.' It reminds me a little about that. We still have people saying 'You don't have enough math here!' There's some people that just don't like that."

Hawkins' confidence stems from the way Numenta has built its technology, which far from merely taking inspiration from the brain – as many other startups claim to do – is actively built as a digital implementation of everything Hawkins has learned about how the dense, napkin-sized sheet of cells that is our neocortex works.

"I know of no other cortical theories/models that incorporate any of the following: 

• active dendrites, 
• differences between proximal and distal dendrites, 
• synapse growth and decay, 
• potential synapses, 
• dendrite growth, 
• depolarization as a mode of prediction, 
• mini-columns, 
• multiple types of inhibition and their corresponding inhibitory neurons, 
• etcetera. 

The new temporal pooling mechanism we are working on requires metabotropic receptors in the locations they are, and are not, found. Again, I don't know of any theories that have been reduced to practice that incorporate any, let alone all of these concepts," he wrote in a post to the discussion mailing list for NuPic, an open source implementation of Numenta's CLA, in February.

Deep learning is the new shallow learning

But for all the apparent rigorousness of Hawkins' approach, during the years he has worked on the technology there has been a fundamental change in the landscape of AI development: the rise of the consumer internet giants, and with them the appearance of various cavernous stores of user data on which to train learning algorithms.

Google, for instance, was said in January of 2014 to be assembling the team required for the "Manhattan Project for AI", according to a source who spoke anonymously to online publication Re/code. But Hawkins thinks that for all its grand aims, Google's approach may be based on a flawed presumption.

The collective term for the approach pioneered by companies like Google, Microsoft, and Facebook is "Deep Learning", but Hawkins fears it may be another blind path.

"Deep learning could be the greatest thing in the world, but it's not a brain theory," he says.

Deep learning approaches, Hawkins says, encourage the industry to go about refining methods based on old technology, itself based on an oversimplified version of the neurons in a brain.

Because of the vast stores of user data available, the companies are all compelled to approach the quest of creating artificial intelligence through building machines that compute over certain types of data.

In many cases, much of the development at places like Google, Microsoft, and Facebook has revolved around vision – a dead end, according to Hawkins.

"Where the whole community got tripped up – and I'm talking fifty years tripped up – is vision," Hawkins explains. "They said, 'Your eyes are moving all the time, your head is moving, the world is moving – let us focus on a simpler problem: spatial inference in vision'. This turns out to be a very small subset of what vision is. Vision turns out to be an inference problem. What that did is they threw out the most important part of vision – you must learn first how to do time-based vision."

The acquisitions these companies have made speak to this apparent flaw.

Google, for instance, hired AI luminary and University of Toronto professor Geoff Hinton and his startup DNNresearch last year to have him apply his "Deep Belief Networks" approach to Google's AI efforts.

In a talk given at the University of Toronto last year, Hinton said he believed more advanced AI should be based on existing approaches, rather than a rethought understanding of the brain.

"The kind of neural inspiration I like is when making it more like the brain works better," Hinton said. "There's lots of people who say you ought to make it more like the brain – like Henry Markram [of the European Union's brain simulation project], for example. He says, 'Give me a billion dollars and I'll make something like the brain,' but he doesn't actually know how to make it work – he just knows how to make something more and more like the brain. That seems to me not the right approach. What we should do is stick with things that actually work and make them more like the brain, and notice when making them more like the brain is actually helpful. There's not much point in making things work worse."

Hawkins vehemently disagrees with this point, and believes that basing approaches on existing methods means Hinton and other AI researchers are not going to be able to imbue their systems with the generality needed for true machine intelligence.

Another influential Googler agrees.

"We have neuroscientists in our team so we can be biologically inspired but are not slavish to it," Google Fellow Jeff Dean (creator of MapReduce, the Google File System, and now a figure in Google's own "Brain Project" team, also known as its AI division) told us this year.

"I'm surprised by how few people believe they need to understand how the brain works to build intelligent machines," Hawkins says. "I'm disappointed by this."

Hinton's foundational technologies, for example, are Boltzmann machines - advanced "stochastic recurrent neural network" tools that try to mimic some of the characteristics of the brain, which sit at the heart of Hinton's "Deep Belief Networks" (2006).

"The neurons in a restricted Boltzmann machine are not even close [to the brain] – it's not even an approximation," Hawkins explains.

Even Google is not sure about which way to bet on how to build a mind, as illustrated by its buy of UK company "DeepMind Technologies" earlier this year.

That company's founder, Demis Hassabis, has done detailed work on fundamental neuroscience, and has built technology out of this understanding. In 2010, it was reported that he mentioned both Hawkins' Hierarchical Temporal Memory and Hinton's Deep Belief Nets when giving a talk on viable general artificial intelligence approaches.

Facebook has gone down similar paths by hiring the influential artificial intelligence academic Yann LeCun to help it "predict what a user is going to do next," among other things.

Microsoft has developed significant capabilities as well, with systems like the Siri-beater "Cortana" and various endeavors by the company's research division, MSR.

Though the techniques these various researchers employ differ, they all depend on training a dataset over a large amount of information, and then selectively retraining it as information changes.

These AI efforts are built around dealing with problems backed up by large and relatively predictable datasets. This has yielded some incredible inventions, such as

• reasonable natural language processing, 
• image detection, and 
• video tagging.

It has not and cannot, however, yield a framework for a general intelligence, as it doesn't have the necessary architecture for data 

• apprehension, 
• analysis, 
• retention, and 
• recognition 

that our own brains do, Hawkins claims.

Hawkins' focus on time is why he believes his approach will win – something that the consumer internet giants are slowly waking up to.

It's all about time

"I would say that Hawkins is focusing more on how things unfold over time, which I think is very important," Google's research director Peter Norvig told El Reg via email, "while most of the current deep learning work assumes a static representation, unchanging over time. I suspect that as we scale up the applications (i.e., from still images to video sequences, and from extracting noun-phrase entities in text to dealing with whole sentences denoting actions), that there will be more emphasis on the unfolding of dynamic processes over time."

Another former Googler concurs, with Andrew Ng telling us via email, "Hawkins' work places a huge emphasis on learning from sequences. While most deep learning researchers also think that learning from sequences is important, we just haven't figured out ways to do so that we're happy with yet."

Geoff Hinton echoes this praise. "He has great insights about the types of computation the brain must be doing," he tells us – but argues that Jeff Hawkins' actual algorithmic contributions have been "disappointing" so far.

An absolutely crucial ingredient to AI

Time "is one hundred per cent crucial" to the creation of true artificial intelligence, Hawkins tells us. "If you accept the fact intelligent machines are going to work on the principles of the neocortex, it is the entire thing, basically. The only way."

"The brain does two things: 

• it does inference, which is recognizing patterns, and 
• it does behavior, which is generating patterns or generating motor behavior," 

Hawkins explains. "Ninety-nine percent of inference is time-based – language, audition, touch – it's all time-based. You can't understand touch without moving your hand. The order in which patterns occur is very important."

Numenta's approach relies on time. Its Cortical Learning Algorithm (white paper) amounts to an engine for

• processing streams of information, 
• classifying them, 
• learning to spot differences, and 
• using time-based patterns to make predictions about the future.

As mentioned above, there are several efforts underway at companies like IBM and federal research agencies like DARPA to implement Hawkins' systems in custom processors, and these schemes all recognize the importance of Hawkins' reliance on time.

"What I found intriguing about [his approach] – time is not an afterthought. In all of these [other] things, time has been an afterthought," one source currently working on implementing Hawkins' ideas tells us.

So far, Hawkins has used his system to make predictions of diverse phenomena such as 

• hourly energy use and 
• stock trading volumes, and 
• to detect anomalies in data streams. 

Numenta's commercial product, Grok, detects anomalies in computer servers running on Amazon's cloud service.

Hawkins described to us one way to understand the power of this type of pattern recognition. "Imagine you are listening to a musician," he suggested. "After hearing her play for several days, you learn the kind of music she plays, how talented she is, how much she improvises, and how many mistakes she makes. Your brain learns her style, and then has expectations about what she will play and what it will sound like. As you continue to listen to her play, you will detect if her style changes, if the type of music she plays changes, or if she starts making more errors. The same kind of patterns exist in machine-generated data, and Grok will detect changes."

Here again the wider AI community appears to be dovetailing into Hawkins' ideas, with one of Andrew Ng's former Stanford students Honglak Lee having published a paper called "A classification-based polyphonic piano transcription approach using learned feature representations" in 2011. However, the method if implementation is different.

Obscurity through biology

Part of the reason why Hawkins' technology is not more widely known is because for current uses it is hard for it to demonstrate a vast lead over rival approaches. For all of Hawkins' belief in the tech, it is hard to demonstrate a convincing killer application for it that other approaches can't do. The point, Hawkins says, is that the CLA's internal structure gets rid of some of the stumbling blocks that exist in the future of other approaches.

Hawkins believes the CLA's implicit dependence on time means that eventually it will become the dominant approach.

"At the bottom of the [neocortex's] hierarchy are fast-changing patterns and they form sequences – some of them are predictable and some of them are not – and what the neocortex is doing is trying to understand the set of patterns here and give it a constant representation – a name for the sequence, if you will – and it forms that as the next level of the hierarchy so the next level up is more stable," Hawkins explains.

"Changing patterns lead to changing representations in the hierarchy that are more stable, and then it learns the changes in those patterns, and as you go up the hierarchy it forms more and more stable representations of the world and they also tend to be independent of your body position and your senses."

A comparison between Hawkins' Hierarchical Temporal Memory cells (right), 

a neural network neuron (center), and the brain's own neuron (left)

He believes his technology is more effective than the approaches taken by his rivals due to its use of sparse distributed representations as an input device to a storage system he terms "sequence memory".

Sequence memory refers to how information makes its way into the brain as a stream of information that comes in from both external stimuli and internal stimuli, such as signals from the broader body.

Sparse Distributed Representations (SDRs) are partially based on the work of mathematician Pentti Kanerva on "Sparse Distributed Memory" [PDF].

They refer to how the brain represents and stores information. They are designed to mimic the way our brain is believed to encode memories, which is through neuron firings across a very large area in response to inputs. To achieve this, SDRs are written, roughly, as a 2000-bit string of which perhaps two percent are active. This means that you don't need to read all active bits in an SDR to say that it is similar to another, because it merely needs to share a few of the activated bits to be considered similar, due to the sparsity.

Hawkins believes SDRs give input data inherent meaning through this representation approach.

"This means that if two vectors have 1s in the same position, they are semantically similar. Vectors can therefore be expressed in degrees of similarity rather than simply being identical or different. These large vectors can be stored accurately even using a subsampled index of, say, 10 of 2,000 bits. This makes SDR memory fault tolerant to gaps in data. SDRs also exhibit properties that reliably allow the neocortex to determine if a new input is unexpected," the company's commercial website for Grok says.

But what are the drawbacks?

So if Hawkins thinks he has the theory and is on the way to building the technology, and other companies are implementing it, then why are we even calling what he is doing a "bet"? The answer comes down to credibility.

Hawkins' idiosyncratic nature and decision to synthesize insights from two different fields – neuroscience and computer science – are his strengths, but also his drawbacks.

"No one knows how the cortex works, so there is no way to know if Jeff is on the right track or not," Dr. Terry Sejnowski, the laboratory head of the Computational Neurobiology Laboratory at the SALK Institute for Biological Studies, tells us. "To the extent that [Hawkins] incorporates new data into his models he may have a shot, and there will be a flood of data coming from the BRAIN Initiative that was announced by Obama last April."

Hawkins says that this response is typical of the academic community, and that there is enough data available to learn about the brain. You just have to look for it.

"We're not going to replicate the neocortex, we're not going to simulate the neocortex, we just need to understand how it works in sufficient detail so we can say 'A-ha!' and build things like it," Hawkins says. "There is an incredible amount of unassimilated data that exists. Fifty years of papers. Thousands of papers a year. It's unbelievable, and it's always the next set of papers that people think is going to do it. ... it's not true that you have to wait for that stuff."

The root of the problems Hawkins faces may be his approach, which stems more from biology than from mathematics. His old colleague and cofounder of Numenta, Dileep George, confirms this.

"I think Jeff is largely right in what he wrote in On Intelligence," George told us. "There are different approaches on how to bring those ideas. Jeff has an angle on it; we have a different angle on it; the rest of the community have another perspective on it."

These ideas are echoed by Google's Norvig. "Hawkins, at least in his general-public-facing-persona, seems to be more driven by duplicating what the brain does, while the deep learning researchers take some concepts from the brain, but then mostly are trying to optimize mathematical equations," he told us via email.

"I live in the middle," Hawkins explains. "Where I know the neuroscience details very very well, and I have a theoretical framework, and I bounce back and forth between these over and over again."

The future

Hawkins reckons that what he is doing today "is maybe 5 per cent of how humans learn," Hawkins says.

He believes that during the coming year he will begin work on the next major area of development for his technology: action.

For Hawkins' machines to gain independence – the ability, say, to not only recognize and classify patterns, but actively tune themselves to hunt for specific bits of information – the motor component needs to be integrated, he explains.

"What we've proven so far – I say built and tested and put into a product – is pure sensor. It's like an ear listening to sounds that doesn't have a chance to move," he tells us.

If you can add in the motor component, "an entire world opens up," he says.

"For example, I could have something like a web bot – an internet crawler. Today's web crawlers are really stupid, they're like wall-following rats. They just go up and down the length up and down the length," he says.

"If I wanted to look and understand the web, I could have a virtual system that is basically moving through cyberspace thinking about 'What is the structure here? How do I model this?' And so that's an example of a behavioral system that has no physical presence. It basically says, 'OK, I'm looking at this data, now where do I go next to look? Oh, I'm going to follow this link and do that in an intelligent way'."

By creating this technology, Hawkins hopes to dramatically accelerate the speed with which generally applicable artificial intelligence is developed and integrated into our world.

It's taken a lot to get here, and the older Hawkins gets and the more rival companies spend, the bigger the stakes get. As of 2014, he is still betting his life on the fact that he is right and they are wrong. ®

ORIGINAL: The Register

By Jack Clark,

29 Mar 2014

IBM Research Creates New Foundation to Program SyNAPSE Chips

ORIGINAL: ZeitNews
August 8, 2013

credit: IBM Research

Scientists from IBM (NYSE: IBM) today unveiled a breakthrough software ecosystem designed for programming silicon chips that have an architecture inspired by the function, low power, and compact volume of the brain. The technology could enable a new generation of intelligent sensor networks that mimic the brain’s abilities for perception, action, and cognition.

Dramatically different from traditional software, IBM’s new programming model breaks the mold of sequential operation underlying today's von Neumann architectures and computers. It is instead tailored for a new class of distributed, highly interconnected, asynchronous, parallel, large-scale cognitive computing architectures.

“Architectures and programs are closely intertwined and a new architecture necessitates a new programming paradigm,” said Dr. Dharmendra S. Modha, Principal Investigator and Senior Manager, IBM Research. “We are working to create a FORTRAN for synaptic computing chips. While complementing today’s computers, this will bring forth a fundamentally new technological capability in terms of programming and applying emerging learning systems.”

To advance and enable this new ecosystem, IBM researchers developed the following breakthroughs that support all aspects of the programming cycle from design through development, debugging, and deployment:

• Simulator: A multi-threaded, massively parallel and highly scalable functional software simulator of a cognitive computing architecture comprising a network of neurosynaptic cores.
• Neuron Model: A simple, digital, highly parameterized spiking neuron model that forms a fundamental information processing unit of brain-like computation and supports a wide range of deterministic and stochastic neural computations, codes, and behaviors. A network of such neurons can sense, remember, and act upon a variety of spatio-temporal, multi-modal environmental stimuli.
• Programming Model: A high-level description of a “program” that is based on composable, reusable building blocks called “corelets.” Each corelet represents a complete blueprint of a network of neurosynaptic cores that specifies a based-level function. Inner workings of a corelet are hidden so that only its external inputs and outputs are exposed to other programmers, who can concentrate on what the corelet does rather than how it does it. Corelets can be combined to produce new corelets that are larger, more complex, or have added functionality.
• Library: A cognitive system store containing designs and implementations of consistent, parameterized, large-scale algorithms and applications that link massively parallel, multi-modal, spatio-temporal sensors and actuators together in real-time. In less than a year, the IBM researchers have designed and stored over 150 corelets in the program library.
• Laboratory: A novel teaching curriculum that spans the architecture, neuron specification, chip simulator, programming language, application library and prototype design models. It also includes an end-to-end software environment that can be used to create corelets, access the library, experiment with a variety of programs on the simulator, connect the simulator inputs/outputs to sensors/actuators, build systems, and visualize/debug the results.

These innovations are being presented at The International Joint Conference on Neural Networks in Dallas, TX.

Paving the Path to SyNAPSE

Modern computing systems were designed decades ago for sequential processing according to a pre-defined program. Although they are fast and precise “number crunchers,” computers of traditional design become constrained by power and size while operating at reduced effectiveness when applied to real-time processing of the noisy, analog, voluminous, Big Data produced by the world around us (ie. DNA). In contrast, the brain—which operates comparatively slowly and at low precision—excels at tasks such as recognizing, interpreting, and acting upon patterns, while consuming the same amount of power as a 20 watt light bulb and occupying the volume of a two-liter bottle.

In August 2011, IBM successfully demonstrated a building block of a novel brain-inspired chip architecture based on a scalable, interconnected, configurable network of “neurosynaptic cores.” Each core brings 

• memory (“synapses”), 
• processors (“neurons”), and 
• communication (“axons”) 

in close proximity, executing activity in an event-driven fashion. These chips serve as a platform for emulating and extending the brain’s ability to respond to biological sensors and analyzing vast amounts of data from many sources at once.

Having completed Phase 0, Phase 1, and Phase 2, IBM and its collaborators (Cornell University and iniLabs, Ltd) have recently been awarded approximately $12 million in new funding from the Defense Advanced Research Projects Agency (DARPA) for Phase 3 of the Systems of Neuromorphic Adaptive Plastic Scalable Electronics (SyNAPSE) project, thus bringing the cumulative funding to approximately $53 million.

Smarter Sensors
IBM’s long-term goal is to build a chip system with ten billion neurons and hundred trillion synapses, while consuming merely one kilowatt of power and occupying less than two liters of volume.

Systems built from these chips could bring the real-time capture and analysis of various types of data closer to the point of collection. They would not only gather symbolic data, which is fixed text or digital information, but also gather sub-symbolic data, which is sensory based and whose values change continuously. This raw data reflects activity in the world of every kind ranging from commerce, social, logistics, location, movement, and environmental conditions.

Take the human eyes, for example. They sift through over a Terabyte of data per day. Emulating the visual cortex, low-power, light-weight eye glasses designed to help the visually impaired could be outfitted with multiple video and auditory sensors that capture and analyze this optical flow of data.

These sensors would gather and interpret large-scale volumes of data to signal how many individuals are ahead of the user, distance to an upcoming curb, number of vehicles in a given intersection, height of a ceiling or length of a crosswalk. Like a guide dog, sub-symbolic data perceived by the glasses would allow them to plot the safest pathway through a room or outdoor setting and help the user navigate the environment via embedded speakers or ear buds. This same technology -- at increasing levels of scale -- can form sensory-based data input capabilities and on-board analytics for automobiles, medical imagers, healthcare devices, smartphones, cameras, and robots.

“The views expressed are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government. Approved for Public Release, Distribution Unlimited.”

Portable Brain-Scan Headsets: 4 Incredible Applications

ORIGINAL: NatGeo
Brian Handwerk. for National Geographic
August 1, 2013

New technology is moving brain research outside the lab and into the real world.

The Emotiv Insight brain-scanning headset may help scientists understand the mind on the go. Image courtesy Emotiv Lifesciences

Our brain controls what we think, feel, and do, but scientists have a limited capability to watch it at work outside the lab. National Geographic Emerging Explorer Tan Le hopes to change that while, in the process, fighting neurological disorders, enhancing learning, and even helping the disabled move things in the physical world with the power of the mind.

Emotiv Lifesciences, the company Le co-founded, produces portable, high-resolution EEG (electroencephalogram) brain-scanning headsets that Le hopes will open new windows on the complex functioning of our brain. On August 1, Emotiv unveiled Emotiv Insight, a faster, next-generation wireless brain scanner that collects real-time data on the wearer's thoughts and feelings and delivers it directly to a computer, phone, or other device through Android, iOS, OSX, Linux, and Windows platforms.

Tan Le, Photograph from Australia Unlimited/National Geographic

Le hopes the product, which costs $199, can further democratize brain research and help scientists gather more data. Using the EEG headsets, she says, people around the world can study brains under conditions and stimuli as varied as those we encounter in everyday life—because subjects can wear the headset while doing everyday tasks.

"The idea is to empower us all to understand more about ourselves," Le said. "That's really the mystery of the mind."

She added, "It's all very, very personal. Sure, there are some commonalities in the way the brain functions. But what we know now with epigenetics [the study of how the expression of heritable traits is modified by environmental influences] is that every learning experience, every activity we undertake, actually affects our [neural] networks. Your brain yesterday is different than your brain today."

An EEG records the electrical fluctuations in the brain and tracks changes in activity as neurons fire when you are engaged in a cognitive task. It's a time-tested process, but traditionally its use has been limited to the lab and a subset of people because it's relatively costly and time-consuming.

What's needed instead, Le stressed, is as much data as possible on as many brains as possible.

"Until recently there's not been any sort of concentrated effort to collect EEG recordings on well individuals, and that really is an essential part of any sort of background study into abnormalities," Le said. "It's a classic case of science being very skewed toward studying the problem set."

So if we're interested in epilepsy we study patients with epilepsy. If we're interested in Alzheimer's we study patients with Alzheimer's, Le noted.

But with no idea what the norm is, she added, correlations or features that appear in these brains may be present in non-afflicted people as well.

Emotiv hopes to help create a massive digital repository of brain-scan information, as well as a platform for sharing brain data with interested parties around the world. With that information, researchers could send out experiments online and collect data from a wide range of subjects who wear the headsets while performing all types of directed tasks.


"It used to be that you sat in your lab and worked kind of independently of other people," said Kevin Whittingstall, the Canada Research Chair in Neurovascular Coupling and a professor at the University of Sherbrooke. "What we've realized is that the brain is so complex we need to start grouping together data sets to paint a better picture."

"What I think is very promising with the Emotiv system is that the hardware remains constant," added Whittingstall, who has no connection to the project. "With one hospital using one EEG system and another using another system, the electrode positions might be different, for example, and it's hard to integrate the data." (Read "Beyond the Brain" from National Geographic magazine.)

Here are four applications of the portable brain scanners that Le says are already beginning to take shape:

1. Moving Things With the Mind
The EEG headset has helped paralyzed patients control an electric wheelchair and make music via computer using only the power of their minds, said Le.

"The hope is that it's going to be more of a democratizing force, so that whether you are able-bodied or not you're still going to be able to communicate and interact with your world in a meaningful way," Le said during a recent trip to Washington, D.C., for the National Geographic Explorers Symposium. "And we're starting to see the first signs of that."

When patients think of an action—verbalizing a word, for example—the headset can record the brain patterns for that action into a computer via a wireless connection. Then, when the wearer repeats the action, the computer can perform it—allowing or facilitating communication, for example, among those who had lost some or all of that ability.

"I think it speaks to the power of software and algorithms to decipher and interpret electrical signals from the brain," Le said. "When you can start to interpret what's going on in the brain, you can extract unique features that then can help you use them as a command to trigger different events in a device or an application on some sort of computing platform."

The human-machine interface can be used for things as trivial as playing a video game, or as life changing as operating a prosthetic limb—and today's achievements represent only the tip of the iceberg, said Le.

2. Diagnosing Disease
Le stresses that because we lack a large, easily accessible database of "normal" brain data, we're likely missing opportunities to identify and track the causes of brain ailments from their earliest stages—when intervention might be less dramatic and more successful.

"A lot of these conditions are developmental in nature, meaning that you don't get Alzheimer's [or autism] overnight," she said.

Historically, Whittingstall added, most of our information regarding brain function was obtained by studying how damage to one particular area was linked to a cognitive deficit, such as language or memory impairment.

"With EEG, we can now start to non-invasively map out brain function so it enables you to record data from many subjects and improves the statistical power to detect the tiniest differences between experimental conditions," he added.

Le hopes the headset will not only help identify neurological conditions and study their progressions in the ever-changing brain, but also enable intervention.

"The brain is plastic; it's very capable of change, so if it's going down a route that we don't want it to go down, then let's do something about it and fine-tune how we intervene based on feedback from the brain itself," she said. (Video: Visit the Brain Bank at Harvard.)

3. Making Learning Easier
With EEG technology becoming more affordable, scientists and citizens alike can get a more complete picture of how each individual brain operates in real-world situations, ranging from social interactions to studying to intense physical activity, said Le.

"People are more and more interested in quantifying their physical health, and I think we're going to start seeing people more interested in quantifying their cognitive, behavioral, and mental health," Le said.

This window on how the individual brain works could also inspire personally tailored applications to help it learn better. "The value to the individual is that you can start to put together some sort of productivity profile for yourself," said Le. "When am I optimized to do some sort of creative brainstorm work?" she added, as an example.

Whittingstall agreed that such learning boosts could be part of the near-term future. "If you can monitor the brain as someone learns a language, for example, you might correlate their ability to learn through changes in their brain waves," he said.

"However, being able to actually put your brain into that state that's optimized for learning might be further off," he cautioned. (Related: "How Your Brain Cleans Itself—Mystery Solved?")

4. Organizing Data by Thoughts and Emotions
Le suggests that brain-scan information could one day be used to help people organize the avalanche of data, videos, images, audio, and other media that is steadily mounting.

"The first time you view an image [of something you experienced], you're going to have a very strong visceral response because you are reliving that moment," she explained, pointing to the example of watching a video of your child taking her first steps.

Because such images can have a strong emotional cue, there may be a way to label them with some kind of personalized emotional tag based on a brain scan, Le suggests. The concept could allow each of us to build a library of cherished video or images that are organized not just by date or location, but also by the emotional descriptions of what each of them produce in our own brains.

All these applications, exciting as they seem, represent a technology that's still in its infancy—and no one knows where it may lead. "There's no way to tell what the killer app is," Le said.

"The biggest job is to try to get as many people excited about the technology as possible so they can play with it, work with it, and expand it in their own areas of interest and their own passions," she said. "Hopefully we can find some great innovations as a result."

What you can do with a little Insight…

Optimize your brain fitness and performance.

Measure you or your family's cognitive health and wellbeing

Create amazing applications with our APIs and analysis tools.

Tan Le: A headset that reads your brainwaves

http://www.ted.com Tan Le's astonishing new computer interface reads its user's brainwaves, making it possible to control virtual objects, and even physical electronics, with mere thoughts (and a little concentration). She demos the headset, and talks about its far-reaching applications.