Neuromorphic chips are being designed to specifically mimic the human brain – and they could soon replace CPUs
BRAIN ACTIVITY MAP
Neuroscape Lab
AI services like Apple’s Siri and others operate by sending your queries to faraway data centers, which send back responses. The reason they rely on cloud-based computing is that today’s electronics don’t come with enough computing power to run the processing-heavy algorithms needed for machine learning. The typical CPUs most smartphones use could never handle a system like Siri on the device. But Dr. Chris Eliasmith, a theoretical neuroscientist and co-CEO of Canadian AI startup Applied Brain Research, is confident that a new type of chip is about to change that.
“Many have suggested Moore's law is ending and that means we won't get 'more compute' cheaper using the same methods,” Eliasmith says. He’s betting on the proliferation of ‘neuromorphics’ — a type of computer chip that is not yet widely known but already being developed by several major chip makers.
Traditional CPUs process instructions based on “clocked time” – information is transmitted at regular intervals, as if managed by a metronome. By packing in digital equivalents of neurons, neuromorphics communicate in parallel (and without the rigidity of clocked time) using “spikes” – bursts of electric current that can be sent whenever needed. Just like our own brains, the chip’s neurons communicate by processing incoming flows of electricity - each neuron able to determine from the incoming spike whether to send current out to the next neuron.
What makes this a big deal is that these chips require far less power to process AI algorithms. For example, one neuromorphic chip made by IBM contains five times as many transistors as a standard Intel processor, yet consumes only 70 milliwatts of power. An Intel processor would use anywhere from 35 to 140 watts, or up to 2000 times more power.
Eliasmith points out that neuromorphics aren’t new and that their designs have been around since the 80s. Back then, however, the designs required specific algorithms be baked directly into the chip. That meant you’d need one chip for detecting motion, and a different one for detecting sound. None of the chips acted as a general processor in the way that our own cortex does.
This was partly because there hasn’t been any way for programmers to design algorithms that can do much with a general purpose chip. So even as these brain-like chips were being developed, building algorithms for them has remained a challenge.
Eliasmith and his team are keenly focused on building tools that would allow a community of programmers to deploy AI algorithms on these new cortical chips.
Central to these efforts is Nengo, a compiler that developers can use to build their own algorithms for AI applications that will operate on general purpose neuromorphic hardware. Compilers are a software tool that programmers use to write code, and that translate that code into the complex instructions that get hardware to actually do something. What makes Nengo useful is its use of the familiar Python programming language – known for it’s intuitive syntax – and its ability to put the algorithms on many different hardware platforms, including neuromorphic chips. Pretty soon, anyone with an understanding of Python could be building sophisticated neural nets made for neuromorphic hardware.
“Things like vision systems, speech systems, motion control, and adaptive robotic controllers have already been built with Nengo,” Peter Suma, a trained computer scientist and the other CEO of Applied Brain Research, tells me.
Perhaps the most impressive system built using the compiler is Spaun, a project that in 2012 earned international praise for being the most complex brain model ever simulated on a computer. Spaun demonstrated that computers could be made to interact fluidly with the environment, and perform human-like cognitive tasks like recognizing images and controlling a robot arm that writes down what it’s sees. The machine wasn’t perfect, but it was a stunning demonstration that computers could one day blur the line between human and machine cognition. Recently, by using neuromorphics, most of Spaun has been run 9000x faster, using less energy than it would on conventional CPUs – and by the end of 2017, all of Spaun will be running on Neuromorphic hardware.
Eliasmith won NSERC’s John C. Polyani award for that project — Canada’s highest recognition for a breakthrough scientific achievement – and once Suma came across the research, the pair joined forces to commercialize these tools.
“While Spaun shows us a way towards one day building fluidly intelligent reasoning systems, in the nearer term neuromorphics will enable many types of context aware AIs,” says Suma. Suma points out that while today’s AIs like Siri remain offline until explicitly called into action, we’ll soon have artificial agents that are ‘always on’ and ever-present in our lives.
“Imagine a SIRI that listens and sees all of your conversations and interactions. You’ll be able to ask it for things like - "Who did I have that conversation about doing the launch for our new product in Tokyo?" or "What was that idea for my wife's birthday gift that Melissa suggested?,” he says.
When I raised concerns that some company might then have an uninterrupted window into even the most intimate parts of my life, I’m reminded that because the AI would be processed locally on the device, there’s no need for that information to touch a server owned by a big company. And for Eliasmith, this ‘always on’ component is a necessary step towards true machine cognition. “The most fundamental difference between most available AI systems of today and the biological intelligent systems we are used to, is the fact that the latter always operate in real-time. Bodies and brains are built to work with the physics of the world,” he says.
Already, major efforts across the IT industry are heating up to get their AI services into the hands of users. Companies like Apple, Facebook, Amazon, and even Samsung, are developing conversational assistants they hope will one day become digital helpers.
In a new automotive application, we have used convolutional neural networks (CNNs) to map the raw pixels from a front-facing camera to the steering commands for a self-driving car. This powerful end-to-end approach means that with minimum training data from humans, the system learns to steer, with or without lane markings, on both local roads and highways. The system can also operate in areas with unclear visual guidance such as parking lots or unpaved roads.
Figure 1: NVIDIA’s self-driving car in action.
We designed the end-to-end learning system using an NVIDIA DevBox running Torch 7 for training. An NVIDIA DRIVETM PXself-driving car computer, also with Torch 7, was used to determine where to drive—while operating at 30 frames per second (FPS). The system is trained to automatically learn the internal representations of necessary processing steps, such as detecting useful road features, with only the human steering angle as the training signal. We never explicitly trained it to detect, for example, the outline of roads. In contrast to methods using explicit decomposition of the problem, such as lane marking detection, path planning, and control, our end-to-end system optimizes all processing steps simultaneously.
We believe that end-to-end learning leads to better performance and smaller systems. Better performance results because the internal components self-optimize to maximize overall system performance, instead of optimizing human-selected intermediate criteria, e. g., lane detection. Such criteria understandably are selected for ease of human interpretation which doesn’t automatically guarantee maximum system performance. Smaller networks are possible because the system learns to solve the problem with the minimal number of processing steps.
Convolutional Neural Networks to Process Visual Data
CNNs[1] have revolutionized the computational pattern recognition process[2]. Prior to the widespread adoption of CNNs, most pattern recognition tasks were performed using an initial stage of hand-crafted feature extraction followed by a classifier. The important breakthrough of CNNs is that features are now learned automatically from training examples. The CNN approach is especially powerful when applied to image recognition tasks because the convolution operation captures the 2D nature of images. By using the convolution kernels to scan an entire image, relatively few parameters need to be learned compared to the total number of operations.
While CNNs with learned features have been used commercially for over twenty years [3], their adoption has exploded in recent years because of two important developments.
First, large, labeled data sets such as the ImageNet Large Scale Visual Recognition Challenge (ILSVRC)[4] are now widely available for training and validation.
Second, CNN learning algorithms are now implemented on massively parallel graphics processing units (GPUs), tremendously accelerating learning and inference ability.
The CNNs that we describe here go beyond basic pattern recognition. We developed a system that learns the entire processing pipeline needed to steer an automobile. The groundwork for this project was actually done over 10 years ago in a Defense Advanced Research Projects Agency (DARPA) seedling project known as DARPA Autonomous Vehicle (DAVE)[5], in which a sub-scale radio control (RC) car drove through a junk-filled alley way. DAVE was trained on hours of human driving in similar, but not identical, environments. The training data included video from two cameras and the steering commands sent by a human operator.
In many ways, DAVE was inspired by the pioneering work of Pomerleau[6], who in 1989 built the Autonomous Land Vehicle in a Neural Network (ALVINN)system. ALVINN is a precursor to DAVE, and it provided the initial proof of concept that an end-to-end trained neural network might one day be capable of steering a car on public roads. DAVE demonstrated the potential of end-to-end learning, and indeed was used to justify starting the DARPA Learning Applied to Ground Robots (LAGR) program[7], but DAVE’s performance was not sufficiently reliable to provide a full alternative to the more modular approaches to off-road driving. (DAVE’s mean distance between crashes was about 20 meters in complex environments.)
About a year ago we started a new effort to improve on the original DAVE, and create a robust system for driving on public roads. The primary motivation for this work is to avoid the need to recognize specific human-designated features, such as lane markings, guard rails, or other cars, and to avoid having to create a collection of “if, then, else” rules, based on observation of these features. We are excited to share the preliminary results of this new effort, which is aptly named: DAVE–2.
The DAVE-2 System
Figure 2: High-level view of the data collection system.
Figure 2 shows a simplified block diagram of the collection system for training data of DAVE-2. Three cameras are mounted behind the windshield of the data-acquisition car, and timestamped video from the cameras is captured simultaneously with the steering angle applied by the human driver. The steering command is obtained by tapping into the vehicle’s Controller Area Network (CAN) bus. In order to make our system independent of the car geometry, we represent the steering command as 1/r, where r is the turning radius in meters. We use 1/r instead of r to prevent a singularity when driving straight (the turning radius for driving straight is infinity). 1/r smoothly transitions through zero from left turns (negative values) to right turns (positive values).
Training data contains single images sampled from the video, paired with the corresponding steering command (1/r). Training with data from only the human driver is not sufficient; the network must also learn how to recover from any mistakes, or the car will slowly drift off the road. The training data is therefore augmented with additional images that show the car in different shifts from the center of the lane and rotations from the direction of the road.
The images for two specific off-center shifts can be obtained from the left and the right cameras. Additional shifts between the cameras and all rotations are simulated through viewpoint transformation of the image from the nearest camera. Precise viewpoint transformation requires 3D scene knowledge which we don’t have, so we approximate the transformation by assuming all points below the horizon are on flat ground, and all points above the horizon are infinitely far away. This works fine for flat terrain, but for a more complete rendering it introduces distortions for objects that stick above the ground, such as cars, poles, trees, and buildings. Fortunately these distortions don’t pose a significant problem for network training. The steering label for the transformed images is quickly adjusted to one that correctly steers the vehicle back to the desired location and orientation in two seconds.
Figure 3: Training the neural network.
Figure 3 shows a block diagram of our training system. Images are fed into a CNN that then computes a proposed steering command. The proposed command is compared to the desired command for that image, and the weights of the CNN are adjusted to bring the CNN output closer to the desired output. The weight adjustment is accomplished using back propagation as implemented in the Torch 7 machine learning package.
Once trained, the network is able to generate steering commands from the video images of a single center camera. Figure 4 shows this configuration.
Figure 4: The trained network is used to generate steering commands from a single front-facing center camera.
Data Collection
Training data was collected by driving on a wide variety of roads and in a diverse set of lighting and weather conditions. We gathered surface street data in central New Jersey and highway data from Illinois, Michigan, Pennsylvania, and New York. Other road types include two-lane roads (with and without lane markings), residential roads with parked cars, tunnels, and unpaved roads. Data was collected in clear, cloudy, foggy, snowy, and rainy weather, both day and night. In some instances, the sun was low in the sky, resulting in glare reflecting from the road surface and scattering from the windshield.
The data was acquired using either our drive-by-wire test vehicle, which is a 2016 Lincoln MKZ, or using a 2013 Ford Focus with cameras placed in similar positions to those in the Lincoln. Our system has no dependencies on any particular vehicle make or model. Drivers were encouraged to maintain full attentiveness, but otherwise drive as they usually do. As of March 28, 2016, about 72 hours of driving data was collected.
Network Architecture
Figure 5: CNN architecture. The network has about 27 million connections and 250 thousand parameters.
We train the weights of our network to minimize the mean-squared error between the steering command output by the network, and either the command of the human driver or the adjusted steering command for off-center and rotated images (see “Augmentation”, later). Figure 5 shows the network architecture, which consists of 9 layers, including a normalization layer, 5 convolutional layers, and 3 fully connected layers. The input image is split into YUV planes and passed to the network.
The first layer of the network performs image normalization. The normalizer is hard-coded and is not adjusted in the learning process. Performing normalization in the network allows the normalization scheme to be altered with the network architecture, and to be accelerated via GPU processing.
The convolutional layers are designed to perform feature extraction, and are chosen empirically through a series of experiments that vary layer configurations. We then use strided convolutions in the first three convolutional layers with a 2×2 stride and a 5×5 kernel, and a non-strided convolution with a 3×3 kernel size in the final two convolutional layers.
We follow the five convolutional layers with three fully connected layers, leading to a final output control value which is the inverse-turning-radius. The fully connected layers are designed to function as a controller for steering, but we noted that by training the system end-to-end, it is not possible to make a clean break between which parts of the network function primarily as feature extractor, and which serve as controller.
Training Details
DATA SELECTION
The first step to training a neural network is selecting the frames to use. Our collected data is labeled with road type, weather condition, and the driver’s activity (staying in a lane, switching lanes, turning, and so forth). To train a CNN to do lane following, we simply select data where the driver is staying in a lane, and discard the rest. We then sample that video at 10 FPS because a higher sampling rate would include images that are highly similar, and thus not provide much additional useful information. To remove a bias towards driving straight the training data includes a higher proportion of frames that represent road curves.
AUGMENTATION
After selecting the final set of frames, we augment the data by adding artificial shifts and rotations to teach the network how to recover from a poor position or orientation. The magnitude of these perturbations is chosen randomly from a normal distribution. The distribution has zero mean, and the standard deviation is twice the standard deviation that we measured with human drivers. Artificially augmenting the data does add undesirable artifacts as the magnitude increases (as mentioned previously).
Simulation
Before road-testing a trained CNN, we first evaluate the network’s performance in simulation. Figure 6 shows a simplified block diagram of the simulation system, and Figure 7 shows a screenshot of the simulator in interactive mode.
Figure 6: Block-diagram of the drive simulator.
The simulator takes prerecorded videos from a forward-facing on-board camera connected to a human-driven data-collection vehicle, and generates images that approximate what would appear if the CNN were instead steering the vehicle. These test videos are time-synchronized with the recorded steering commands generated by the human driver.
Since human drivers don’t drive in the center of the lane all the time, we must manually calibrate the lane’s center as it is associated with each frame in the video used by the simulator. We call this position the “ground truth”.
The simulator transforms the original images to account for departures from the ground truth. Note that this transformation also includes any discrepancy between the human driven path and the ground truth. The transformation is accomplished by the same methods as described previously.
The simulator accesses the recorded test video along with the synchronized steering commands that occurred when the video was captured. The simulator sends the first frame of the chosen test video, adjusted for any departures from the ground truth, to the input of the trained CNN, which then returns a steering command for that frame. The CNN steering commands as well as the recorded human-driver commands are fed into the dynamic model [7] of the vehicle to update the position and orientation of the simulated vehicle.
Figure 7: Screenshot of the simulator in interactive mode. See text for explanation of the performance metrics. The green area on the left is unknown because of the viewpoint transformation. The highlighted wide rectangle below the horizon is the area which is sent to the CNN.
The simulator then modifies the next frame in the test video so that the image appears as if the vehicle were at the position that resulted by following steering commands from the CNN. This new image is then fed to the CNN and the process repeats.
The simulator records the off-center distance (distance from the car to the lane center), the yaw, and the distance traveled by the virtual car. When the off-center distance exceeds one meter, a virtual human intervention is triggered, and the virtual vehicle position and orientation is reset to match the ground truth of the corresponding frame of the original test video.
Evaluation
We evaluate our networks in two steps: first in simulation, and then in on-road tests.
In simulation we have the networks provide steering commands in our simulator to an ensemble of prerecorded test routes that correspond to about a total of three hours and 100 miles of driving in Monmouth County, NJ. The test data was taken in diverse lighting and weather conditions and includes highways, local roads, and residential streets.
We estimate what percentage of the time the network could drive the car (autonomy) by counting the simulated human interventions that occur when the simulated vehicle departs from the center line by more than one meter. We assume that in real life an actual intervention would require a total of six seconds: this is the time required for a human to retake control of the vehicle, re-center it, and then restart the self-steering mode. We calculate the percentage autonomy by counting the number of interventions, multiplying by 6 seconds, dividing by the elapsed time of the simulated test, and then subtracting the result from 1:
Thus, if we had 10 interventions in 600 seconds, we would have an autonomy value of
ON-ROAD TESTS
After a trained network has demonstrated good performance in the simulator, the network is loaded on the DRIVE PX in our test car and taken out for a road test. For these tests we measure performance as the fraction of time during which the car performs autonomous steering. This time excludes lane changes and turns from one road to another. For a typical drive in Monmouth County NJ from our office in Holmdel to Atlantic Highlands, we are autonomous approximately 98% of the time. We also drove 10 miles on the Garden State Parkway (a multi-lane divided highway with on and off ramps) with zero intercepts.
Here is a video of our test car driving in diverse conditions.
Visualization of Internal CNN State
Figure 8: How the CNN “sees” an unpaved road. Top: subset of the camera image sent to the CNN. Bottom left: Activation of the first layer feature maps. Bottom right: Activation of the second layer feature maps. This demonstrates that the CNN learned to detect useful road features on its own, i. e., with only the human steering angle as training signal. We never explicitly trained it to detect the outlines of roads.
Figures 8 and 9 show the activations of the first two feature map layers for two different example inputs, an unpaved road and a forest. In case of the unpaved road, the feature map activations clearly show the outline of the road while in case of the forest the feature maps contain mostly noise, i. e., the CNN finds no useful information in this image.
This demonstrates that the CNN learned to detect useful road features on its own, i. e., with only the human steering angle as training signal. We never explicitly trained it to detect the outlines of roads, for example.
Figure 9: Example image with no road. The activations of the first two feature maps appear to contain mostly noise, i. e., the CNN doesn’t recognize any useful features in this image.
Conclusions
We have empirically demonstrated that CNNs are able to learn the entire task of lane and road following without manual decomposition into road o
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4):541–551, Winter 1989. URL: http://yann.lecun.org/exdb/publis/pdf/lecun-89e.pdf.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks.
L. D. Jackel, D. Sharman, Stenard C. E., Strom B. I., , and D Zuckert. Optical character recognition for self-service banking. AT&T Technical Journal, 74(1):16–24, 1995.
Danwei Wang and Feng Qi. Trajectory planning for a four-wheel-steering vehicle. In Proceedings of the 2001 IEEE International Conference on Robotics & Automation, May 21–26 2001. URL: http://www.ntu.edu.sg/home/edwwang/confpapers/wdwicar01.pdf. rlane marking detection, semantic abstraction, path planning, and control. A small amount of training data from less than a hundred hours of driving was sufficient to train the car to operate in diverse conditions, on highways, local and residential roads in sunny, cloudy, and rainy conditions.
The CNN is able to learn meaningful road features from a very sparse training signal (steering alone).
The system learns for example to detect the outline of a road without the need of explicit labels during training.
More work is needed to improve the robustness of the network, to find methods to verify the robustness, and to improve visualization of the network-internal processing steps.
Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backprop- agation applied to handwritten zip code recognition. Neural Computation, 1(4):541–551, Winter 1989. URL: http://yann.lecun.org/exdb/publis/pdf/lecun-89e.pdf.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 1097–1105. Curran Associates, Inc., 2012. URL: http://papers.nips.cc/paper/ 4824-imagenet-classification-with-deep-convolutional-neural-networks. pdf.
L. D. Jackel, D. Sharman, Stenard C. E., Strom B. I., , and D Zuckert. Optical character recognition for self-service banking. AT&T Technical Journal, 74(1):16–24, 1995.
Large scale visual recognition challenge (ILSVRC). URL: http://www.image-net.org/ challenges/LSVRC/.
Net-Scale Technologies, Inc. Autonomous off-road vehicle control using end-to-end learning, July 2004. Final technical report. URL: http://net-scale.com/doc/net-scale-dave-report.pdf.
Dean A. Pomerleau. ALVINN, an autonomous land vehicle in a neural network. Technical report, Carnegie Mellon University, 1989. URL: http://repository.cmu.edu/cgi/viewcontent. cgi?article=2874&context=compsci.
Danwei Wang and Feng Qi. Trajectory planning for a four-wheel-steering vehicle. In Proceedings of the 2001 IEEE International Conference on Robotics & Automation, May 21–26 2001. URL: http: //www.ntu.edu.sg/home/edwwang/confpapers/wdwicar01.pdf.
Until now quantum computers have had just a fraction of the processing power they are theoretically capable of producing.
But an international team of researchers believe they have finally overcome the main technical problems that have prevented the construction of more powerful machines.
They are currently building a prototype and a full-scale quantum computer – many millions of times faster than the best currently available – could be built in about a decade.
This is a modal window.
Scientists invent invisible underwater robots based on glass eels
Such devices work by utilising the almost magical properties found in the world of the very small, where an atom can apparently exist in two different places at the same time.
Professor Winfried Hensinger, head of the Ion Quantum Technology Group at Sussex University, who has been leading this research, told The Independent: “It is the Holy Grail of science, really, to build a quantum computer.
“And we are now publishing the actual nuts-and-bolts construction plan for a large-scale quantum computer.”
It is thought the astonishing processing power unleashed by quantum mechanics will lead to new, life-saving medicines, help solve the most intractable scientific problems, and probe the mysteries of the universe.
“Life will change completely. We will be able to do certain things we could never even dream of before,” Professor Hensinger said.
“You can imagine that suddenly the sky is the limit.
“This is really, really exciting … it’s probably one of the most exciting times to be in this field.”
He said small quantum computers had been built in the past but to test the theories.
“This is not an academic study any more, it really is all the engineering required to build such a device,” he said.
“Nobody has really gone ahead and drafted a full engineering plan of how you build one.
“Many people questioned, because this is so hard to make this happen, that it can even be built.
“We show that not only can it be built, but we provide a whole detailed plan on how to make it happen.”
The problem is that existing quantum computers require lasers focused precisely on individual atoms. The larger the computer, the more lasers are required and the greater the chance of something going wrong.
But Professor Hensinger and colleagues used a different technique to monitor the atoms involving a microwave field and electricity in an ‘ion-trap’ device.
“What we have is a solution that we can scale to arbitrary [computing] power,” he said.
Fig. 2. Gradient wires placed underneath each gate zone and embedded silicon photodetector. (A) Illustration showing an isometric view of the two main gradient wires placed underneath each gate zone. Short wires are placed locally underneath each gate zone to form coils, which compensate for slowly varying magnetic fields and allow for individual addressing. The wire configuration in each zone can be seen in more detail in the inset. (B) Silicon photodetector (marked green) embedded in the silicon substrate, transparent center segmented electrodes, and the possible detection angle are shown. VIA structures are used to prevent optical cross-talk from neighboring readout zones. Source: Science Journals — AAAS. Blueprint for a microwave trapped ion quantum computer. Lekitsch et al. Sci. Adv. 2017;3: e1601540 1 February 2017
Fig. 4. Scalable module illustration. One module consisting of 36 × 36 junctions placed on the supporting steel frame structure: Nine wafers containing the required DACs and control electronics are placed between the wafer holding 36 × 36 junctions and the microchannel cooler (red layer) providing the cooling. X-Y-Z piezo actuators are placed in the four corners on top of the steel frame, allowing for accurate alignment of the module. Flexible electric wires supply voltages, currents, and control signals to the DACs and control electronics, such as field-programmable gate arrays (FPGAs). Coolant is supplied to the microchannel cooler layer via two flexible steel tubes placed in the center of the modules. Source: Science Journals — AAAS. Blueprint for a microwave trapped ion quantum computer. Lekitsch et al. Sci. Adv. 2017;3: e1601540 1 February 2017
Fig. 5. Illustration of vacuum chambers. Schematic of octagonal UHV chambers connected together; each chamber is 4.5 × 4.5 m2 large and can hold >2.2 million individual X-junctions placed on steel frames. Source: Science Journals — AAAS. Blueprint for a microwave trapped ion quantum computer. Lekitsch et al. Sci. Adv. 2017;3: e1601540 1 February 2017
“We are already building it now. Within two years we think we will have completed a prototype which incorporates all the technology we state in this blueprint.
“At the same time we are now looking for industry partner so we can really build a large-scale device that fills a building basically.
“It’s extraordinarily expensive so we need industry partners … this will be in the 10s of millions, up to £100m.”
Commenting on the research, described in a paper in the journal Science Advances, other academics praised the quality of the work but expressed caution about how quickly it could be developed.
Dr Toby Cubitt, a Royal Society research fellow in quantum information theory at University College London, said: “Many different technologies are competing to build the first large-scale quantum computer. Ion traps were one of the earliest realistic proposals.
“This work is an important step towards scaling up ion-trap quantum computing.
“Though there's still a long way to go before you'll be making spreadsheets on your quantum computer.”
And Professor Alan Woodward, of Surrey University, hailed the “tremendous step in the right direction”.
“It is great work,” he said. “They have made some significant strides forward.”
But he added it was “too soon to say” whether it would lead to the hoped-for technological revolution.
MIT engineers have developed biological computational circuits capable of both remembering and responding to sequential input data.
The group's work, which is described in this week's issue of Science, represents a critical step in the progression of synthetic biology with the integration of DNA-based memory, in particular, pointing the way toward building large computational systems from biological components—computing devices that are living cells—and, ultimately, programming complex biological functions.
More specifically, Nathaniel Roquet and colleagues at MIT's Synthetic Biology Groupwere able to implement within a living cell what's known as a state machine: an abstract mathematical model describing computation as a list of of distinct internal states paired with an associated list of operations (or machine inputs) required to transition from state to state. So: a new state is always the result of an old state taken in combination with new inputs (history matters). State machines happen to describe a very large number of different things, from natural language processing algorithms to neurological systems to something as simple as a vending machine.
In a living cell, DNA is the natural candidate for storing state information. After all, that's what DNA does: store information. What Roquet and co. have created is a framework for chemically manipulating DNA such that states are encoded in DNA sequences. As a storage mechanism, this allows for both conveniently reading out a given state via genetic sequencing and also regulating gene expression via state transitions. In other words, the states can be linked to cellular behavior. The DNA serves as the memory for the state machine. The rest is in how, specifically, the DNA is manipulated and what effect that has on cellular behavior.
This could mean integrating biological state machines into tumor models, where they may be used to genetically surveil the activation of genes that may cause cancer
In their experiments, Roquet and co. programmed E. coli cells to react to several substances commonly used in biological laboratory experiments, including an analogue of the antibiotic tetracycline, a sugar called arabinose, and a chemical called DAPG that helps plants protect their roots from pathogens. The cells could be reprogrammed to other inputs as needed, however.
The actual cell behavior being programmed by the researchers was the expression of genes coding for the production of different fluorescent proteins representing different colors. With three different inputs they were able to produce 16 different combinations of colors.
"Synthetic state machines that record and respond to sequences of signaling and gene regulatory events within a cell could be transformative tools in the study and engineering of complex living systems," Roquet writes. In other words, by implementing a state machine (a computer) in a living cell, it's possible to use that state machine to surveil otherwise impossible-to-observe cellular happenings.
For example, progenitor cells (similar to stem cells) develop into differentiated cell types with specific functions thanks to transcription factors, proteins that help regulate gene expression in cells. Transcription factors have allowed researchers to program both progenitor cells to become certain specific types of functional cells—and also to do the opposite, programming functional cells to behave as undifferentiated cells. However, much about the process remains mysterious. A state machine that could record the DNA transitions resulting from TF activation could go a long way toward not only understanding these processes, but manipulating them as well.
The circuits in the biological state machine are dependent on enzymes called recombinases. These enzymes are activated by various inputs into a cell, such as chemical signals, and act to tweak that cell's DNA. But the tweak that actually occurs depends on the orientation of two DNA sequences known as recognition sites. The important thing is that the effect of changing any two recognition sites (the resulting cellular behavior) depends on how other recognition sites have been altered previously. Hence, memory.
There's really no shortage of potential applications here. The example Roquet gives is in integrating biological state machines into tumor models, where they may be used to genetically surveil the activation of oncogenes (genes that may cause cancer) and deactivation of tumor suppression mechanisms in individual cells.
"This idea that we can record and respond to not just combinations of biological events but also their orders opens up a lot of potential applications," Roquet offers in a statement. "A lot is known about what factors regulate differentiation of specific cell types or lead to the progression of certain diseases, but not much is known about the temporal organization of those factors. That's one of the areas we hope to dive into with our device."
Computers have become "alive," but perhaps not in the way that many of us anticipated. A unicellular organism itself won't ever be packing much computational horsepower, but considered as a building block, the potential is pretty wild.
Smart bricks capable of recycling wastewater and generating electricity from sunlight are being developed by a team of scientists from the University of the West of England (UWE Bristol). The bricks will be able to fit together and create 'bioreactor walls' which could then be incorporated in housing, public building and office spaces
The UWE Bristol team is working on the smart technologies that will be integrated into the bricks in this pan European 'Living Architecture' (LIAR) project led by Newcastle University. The LIAR project brings together living architecture, computing and engineering to find a new way to tackle global sustainability issues.
The smart living bricks will be made from bio-reactors filled with microbial cells and algae. Designed to self-adapt to changing environmental conditions the smart bricks will monitor and modify air in the building and recognise occupants.
Each brick will contain Microbial Fuel Cells (MFCs) containing a variety of micro-organisms specifically chosen to
clean water,
reclaim phosphate,
generate electricity and
facilitate the production of new detergents,
as part of the same process.
The MFCs that will make up the living engine of the wall of smart bricks will be able to sense their surroundings and respond to them through a series of digitally coordinated mechanisms.
Professor Andrew Adamatzky, LIAR Project Director for UWE Bristol, is leading the UWE Bristol team, he said, “The technologies we are developing aim to transform the places where we live and work enabling us co-live with the building.
“A building made from bio-reactors will become a large-scale living organism that addresses all environmental and energy needs of the occupants. Walls in buildings comprised of smart bricks containing bioreactors will integrate massive-parallel computing processors where millions of living creatures sense the occupants in the building and the internal and external environmental conditions.
“Each smart brick is an electrical analogous computer. A building made of such bricks will be a massive-parallel computing processor.”
A photo-bioreactor is a device that can be programmed to utilize a variety of inputs such as
grey water,
microbial consortia (algae and bacteria),
carbon dioxide from the atmosphere, and
different types of nutrient to generate outputs.
These outputs include
'polished' water,
fertiliser,
extractable products (recoverable phosphate),
oxygen,
next generation biodegradable detergents,
electricity,
recoverable biomass,
bio-fluorescence and to a certain extent,
heat.
Professor Ioannis Ieropoulos, Director of the Bristol Bioenergy Centre (BBiC), at the Bristol Robotics Laboratory at UWE Bristol, said, “Microbial Fuel Cells are energy transducers that exploit the metabolic activity of the constituent microbes to break down organic waste and generate electricity. This is a novel application for MFC modules to be made into actuating building blocks as part of wall structures. This will allow us to explore the possibility of treating household waste, generating useful levels of electricity, and have 'active programmable' walls within our living environments.”
Rachel Armstrong, Professor of Experimental Architecture at Newcastle University, UK, who is co-ordinating the project, said, “The LIAR project is incredibly exciting – it is bringing together living architecture, computing and engineering to find a new way to tackle global issues, like sustainability.”
The €3.2m LIAR (Living Architecture) project is co-ordinated by Newcastle University working with experts from the universities of
Technique combines analogue and digital processes in engineered cells.
Stock image of mitotic cells
Living cells are capable of performing complex computations on the environmental signals they encounter.
These computations can be continuous, or analogue, in nature — the way eyes adjust to gradual changes in the light levels. They can also be digital, involving simple on or off processes, such as a cell’s initiation of its own death.
Synthetic biological systems, in contrast, have tended to focus on either analogue or digital processing, limiting the range of applications for which they can be used.
But now a team of researchers at MIT has developed a technique to integrate both analogue and digital computation in living cells, allowing them to form gene circuits capable of carrying out complex processing operations.
The synthetic circuits, presented in a paper published today in the journal Nature Communications, are capable of measuring the level of an analogue input, such as a particular chemical relevant to a disease, and deciding whether the level is in the right range to turn on an output, such as a drug that treats the disease.
In this way they act like electronic devices known as comparators, which take analogue input signals and convert them into a digital output, according to Timothy Lu, an associate professor of electrical engineering and computer science and of biological engineering, and head of the Synthetic Biology Group at MIT’s Research Laboratory of Electronics, who led the research alongside former microbiology PhD student Jacob Rubens.
“Most of the work in synthetic biology has focused on the digital approach, because [digital systems] are much easier to program,” Lu says.
However, since digital systems are based on a simple binary output such as 0 or 1, performing complex computational operations requires the use of a large number of parts, which is difficult to achieve in synthetic biological systems.
“Digital is basically a way of computing in which you get intelligence out of very simple parts, because each part only does a very simple thing, but when you put them all together you get something that is very smart,” Lu says. “But that requires you to be able to put many of these parts together, and the challenge in biology, at least currently, is that you can’t assemble billions of transistors like you can on a piece of silicon,” he says.
The mixed signal device the researchers have developed is based on multiple elements. A threshold module consists of a sensor that detects analogue levels of a particular chemical.
This threshold module controls the expression of the second component, a recombinase gene, which can in turn switch on or off a segment of DNA by inverting it, thereby converting it into a digital output.
If the concentration of the chemical reaches a certain level, the threshold module expresses the recombinase gene, causing it to flip the DNA segment. This DNA segment itself contains a gene or gene-regulatory element that then alters the expression of a desired output.
“So this is how we take an analogue input, such as a concentration of a chemical, and convert it into a 0 or 1 signal,” Lu says. “And once that is done, and you have a piece of DNA that can be flipped upside down, then you can put together any of those pieces of DNA to perform digital computing,” he says.
The team has already built an analogue-to-digital converter circuit that implements ternary logic, a device that will only switch on in response to either a high or low concentration range of an input, and which is capable of producing two different outputs.
In the future, the circuit could be used to detect glucose levels in the blood and respond in one of three ways depending on the concentration, he says.
“If the glucose level was too high you might want your cells to produce insulin, if the glucose was too low you might want them to make glucagon, and if it was in the middle you wouldn’t want them to do anything,” he says.
Similar analogue-to-digital converter circuits could also be used to detect a variety of chemicals, simply by changing the sensor, Lu says.
The researchers are investigating the idea of using analogue-to-digital converters to detect levels of inflammation in the gut caused by inflammatory bowel disease, for example, and releasing different amounts of a drug in response.
Immune cells used in cancer treatment could also be engineered to detect different environmental inputs, such as oxygen or tumor lysis levels, and vary their therapeutic activity in response.
Other research groups are also interested in using the devices for environmental applications, such as engineering cells that detect concentrations of water pollutants, Lu says.
Ahmad Khalil, an assistant professor of biomedical engineering at Boston University, who was not involved in the work, says the researchers have expanded the repertoire of computation in cells.
“Developing these foundational tools and computational primitives is important as researchers try to build additional layers of sophistication for precisely controlling how cells interact with their environment,” Khalil says.
The research team recently created a spinout company, called Synlogic, which is now attempting to use simple versions of the circuits to engineer probiotic bacteria that can treat diseases in the gut.
The company hopes to begin clinical trials of these bacteria-based treatments within the next 12 months.
