End-to-End Deep Learning for Self-Driving Cars

In a new automotive application, we have used convolutional neural networks (CNNs) to map the raw pixels from a front-facing camera to the steering commands for a self-driving car. This powerful end-to-end approach means that with minimum training data from humans, the system learns to steer, with or without lane markings, on both local roads and highways. The system can also operate in areas with unclear visual guidance such as parking lots or unpaved roads.

Figure 1: NVIDIA’s self-driving car in action.

We designed the end-to-end learning system using an NVIDIA DevBox running Torch 7 for training. An NVIDIA DRIVETM PX self-driving car computer, also with Torch 7, was used to determine where to drive—while operating at 30 frames per second (FPS). The system is trained to automatically learn the internal representations of necessary processing steps, such as detecting useful road features, with only the human steering angle as the training signal. We never explicitly trained it to detect, for example, the outline of roads. In contrast to methods using explicit decomposition of the problem, such as lane marking detection, path planning, and control, our end-to-end system optimizes all processing steps simultaneously.

We believe that end-to-end learning leads to better performance and smaller systems. Better performance results because the internal components self-optimize to maximize overall system performance, instead of optimizing human-selected intermediate criteria, e. g., lane detection. Such criteria understandably are selected for ease of human interpretation which doesn’t automatically guarantee maximum system performance. Smaller networks are possible because the system learns to solve the problem with the minimal number of processing steps.

This blog post is based on the NVIDIA paper End to End Learning for Self-Driving Cars. Please see the original paper for full details.

Convolutional Neural Networks to Process Visual Data

CNNs[1] have revolutionized the computational pattern recognition process[2]. Prior to the widespread adoption of CNNs, most pattern recognition tasks were performed using an initial stage of hand-crafted feature extraction followed by a classifier. The important breakthrough of CNNs is that features are now learned automatically from training examples. The CNN approach is especially powerful when applied to image recognition tasks because the convolution operation captures the 2D nature of images. By using the convolution kernels to scan an entire image, relatively few parameters need to be learned compared to the total number of operations.

While CNNs with learned features have been used commercially for over twenty years [3], their adoption has exploded in recent years because of two important developments.

• First, large, labeled data sets such as the ImageNet Large Scale Visual Recognition Challenge (ILSVRC)[4] are now widely available for training and validation. 
• Second, CNN learning algorithms are now implemented on massively parallel graphics processing units (GPUs), tremendously accelerating learning and inference ability.

The CNNs that we describe here go beyond basic pattern recognition. We developed a system that learns the entire processing pipeline needed to steer an automobile. The groundwork for this project was actually done over 10 years ago in a Defense Advanced Research Projects Agency (DARPA) seedling project known as DARPA Autonomous Vehicle (DAVE)[5], in which a sub-scale radio control (RC) car drove through a junk-filled alley way. DAVE was trained on hours of human driving in similar, but not identical, environments. The training data included video from two cameras and the steering commands sent by a human operator.

In many ways, DAVE was inspired by the pioneering work of Pomerleau[6], who in 1989 built the Autonomous Land Vehicle in a Neural Network (ALVINN) system. ALVINN is a precursor to DAVE, and it provided the initial proof of concept that an end-to-end trained neural network might one day be capable of steering a car on public roads. DAVE demonstrated the potential of end-to-end learning, and indeed was used to justify starting the DARPA Learning Applied to Ground Robots (LAGR) program[7], but DAVE’s performance was not sufficiently reliable to provide a full alternative to the more modular approaches to off-road driving. (DAVE’s mean distance between crashes was about 20 meters in complex environments.)

About a year ago we started a new effort to improve on the original DAVE, and create a robust system for driving on public roads. The primary motivation for this work is to avoid the need to recognize specific human-designated features, such as lane markings, guard rails, or other cars, and to avoid having to create a collection of “if, then, else” rules, based on observation of these features. We are excited to share the preliminary results of this new effort, which is aptly named: DAVE–2.

The DAVE-2 System

Figure 2: High-level view of the data collection system.

Figure 2 shows a simplified block diagram of the collection system for training data of DAVE-2. Three cameras are mounted behind the windshield of the data-acquisition car, and timestamped video from the cameras is captured simultaneously with the steering angle applied by the human driver. The steering command is obtained by tapping into the vehicle’s Controller Area Network (CAN) bus. In order to make our system independent of the car geometry, we represent the steering command as 1/r, where r is the turning radius in meters. We use 1/r instead of r to prevent a singularity when driving straight (the turning radius for driving straight is infinity). 1/r smoothly transitions through zero from left turns (negative values) to right turns (positive values).

Training data contains single images sampled from the video, paired with the corresponding steering command (1/r). Training with data from only the human driver is not sufficient; the network must also learn how to recover from any mistakes, or the car will slowly drift off the road. The training data is therefore augmented with additional images that show the car in different shifts from the center of the lane and rotations from the direction of the road.

The images for two specific off-center shifts can be obtained from the left and the right cameras. Additional shifts between the cameras and all rotations are simulated through viewpoint transformation of the image from the nearest camera. Precise viewpoint transformation requires 3D scene knowledge which we don’t have, so we approximate the transformation by assuming all points below the horizon are on flat ground, and all points above the horizon are infinitely far away. This works fine for flat terrain, but for a more complete rendering it introduces distortions for objects that stick above the ground, such as cars, poles, trees, and buildings. Fortunately these distortions don’t pose a significant problem for network training. The steering label for the transformed images is quickly adjusted to one that correctly steers the vehicle back to the desired location and orientation in two seconds.

Figure 3: Training the neural network.

Figure 3 shows a block diagram of our training system. Images are fed into a CNN that then computes a proposed steering command. The proposed command is compared to the desired command for that image, and the weights of the CNN are adjusted to bring the CNN output closer to the desired output. The weight adjustment is accomplished using back propagation as implemented in the Torch 7 machine learning package.

Once trained, the network is able to generate steering commands from the video images of a single center camera. Figure 4 shows this configuration.

Figure 4: The trained network is used to generate steering commands from a single front-facing center camera.

Data Collection

Training data was collected by driving on a wide variety of roads and in a diverse set of lighting and weather conditions. We gathered surface street data in central New Jersey and highway data from Illinois, Michigan, Pennsylvania, and New York. Other road types include two-lane roads (with and without lane markings), residential roads with parked cars, tunnels, and unpaved roads. Data was collected in clear, cloudy, foggy, snowy, and rainy weather, both day and night. In some instances, the sun was low in the sky, resulting in glare reflecting from the road surface and scattering from the windshield.

The data was acquired using either our drive-by-wire test vehicle, which is a 2016 Lincoln MKZ, or using a 2013 Ford Focus with cameras placed in similar positions to those in the Lincoln. Our system has no dependencies on any particular vehicle make or model. Drivers were encouraged to maintain full attentiveness, but otherwise drive as they usually do. As of March 28, 2016, about 72 hours of driving data was collected.

Network Architecture

Figure 5: CNN architecture. The network has about 27 million connections and 250 thousand parameters.

We train the weights of our network to minimize the mean-squared error between the steering command output by the network, and either the command of the human driver or the adjusted steering command for off-center and rotated images (see “Augmentation”, later). Figure 5 shows the network architecture, which consists of 9 layers, including a normalization layer, 5 convolutional layers, and 3 fully connected layers. The input image is split into YUV planes and passed to the network.

The first layer of the network performs image normalization. The normalizer is hard-coded and is not adjusted in the learning process. Performing normalization in the network allows the normalization scheme to be altered with the network architecture, and to be accelerated via GPU processing.

The convolutional layers are designed to perform feature extraction, and are chosen empirically through a series of experiments that vary layer configurations. We then use strided convolutions in the first three convolutional layers with a 2×2 stride and a 5×5 kernel, and a non-strided convolution with a 3×3 kernel size in the final two convolutional layers.

We follow the five convolutional layers with three fully connected layers, leading to a final output control value which is the inverse-turning-radius. The fully connected layers are designed to function as a controller for steering, but we noted that by training the system end-to-end, it is not possible to make a clean break between which parts of the network function primarily as feature extractor, and which serve as controller.

Training Details

DATA SELECTION

The first step to training a neural network is selecting the frames to use. Our collected data is labeled with road type, weather condition, and the driver’s activity (staying in a lane, switching lanes, turning, and so forth). To train a CNN to do lane following, we simply select data where the driver is staying in a lane, and discard the rest. We then sample that video at 10 FPS because a higher sampling rate would include images that are highly similar, and thus not provide much additional useful information. To remove a bias towards driving straight the training data includes a higher proportion of frames that represent road curves.

AUGMENTATION

After selecting the final set of frames, we augment the data by adding artificial shifts and rotations to teach the network how to recover from a poor position or orientation. The magnitude of these perturbations is chosen randomly from a normal distribution. The distribution has zero mean, and the standard deviation is twice the standard deviation that we measured with human drivers. Artificially augmenting the data does add undesirable artifacts as the magnitude increases (as mentioned previously).

Simulation

Before road-testing a trained CNN, we first evaluate the network’s performance in simulation. Figure 6 shows a simplified block diagram of the simulation system, and Figure 7 shows a screenshot of the simulator in interactive mode.

Figure 6: Block-diagram of the drive simulator.

The simulator takes prerecorded videos from a forward-facing on-board camera connected to a human-driven data-collection vehicle, and generates images that approximate what would appear if the CNN were instead steering the vehicle. These test videos are time-synchronized with the recorded steering commands generated by the human driver.

Since human drivers don’t drive in the center of the lane all the time, we must manually calibrate the lane’s center as it is associated with each frame in the video used by the simulator. We call this position the “ground truth”.

The simulator transforms the original images to account for departures from the ground truth. Note that this transformation also includes any discrepancy between the human driven path and the ground truth. The transformation is accomplished by the same methods as described previously.

The simulator accesses the recorded test video along with the synchronized steering commands that occurred when the video was captured. The simulator sends the first frame of the chosen test video, adjusted for any departures from the ground truth, to the input of the trained CNN, which then returns a steering command for that frame. The CNN steering commands as well as the recorded human-driver commands are fed into the dynamic model [7] of the vehicle to update the position and orientation of the simulated vehicle.

Figure 7: Screenshot of the simulator in interactive mode. See text for explanation of the performance metrics. The green area on the left is unknown because of the viewpoint transformation. The highlighted wide rectangle below the horizon is the area which is sent to the CNN.

The simulator then modifies the next frame in the test video so that the image appears as if the vehicle were at the position that resulted by following steering commands from the CNN. This new image is then fed to the CNN and the process repeats.

The simulator records the off-center distance (distance from the car to the lane center), the yaw, and the distance traveled by the virtual car. When the off-center distance exceeds one meter, a virtual human intervention is triggered, and the virtual vehicle position and orientation is reset to match the ground truth of the corresponding frame of the original test video.

Evaluation

We evaluate our networks in two steps: first in simulation, and then in on-road tests.

In simulation we have the networks provide steering commands in our simulator to an ensemble of prerecorded test routes that correspond to about a total of three hours and 100 miles of driving in Monmouth County, NJ. The test data was taken in diverse lighting and weather conditions and includes highways, local roads, and residential streets.

We estimate what percentage of the time the network could drive the car (autonomy) by counting the simulated human interventions that occur when the simulated vehicle departs from the center line by more than one meter. We assume that in real life an actual intervention would require a total of six seconds: this is the time required for a human to retake control of the vehicle, re-center it, and then restart the self-steering mode. We calculate the percentage autonomy by counting the number of interventions, multiplying by 6 seconds, dividing by the elapsed time of the simulated test, and then subtracting the result from 1:

Thus, if we had 10 interventions in 600 seconds, we would have an autonomy value of

ON-ROAD TESTS

After a trained network has demonstrated good performance in the simulator, the network is loaded on the DRIVE PX in our test car and taken out for a road test. For these tests we measure performance as the fraction of time during which the car performs autonomous steering. This time excludes lane changes and turns from one road to another. For a typical drive in Monmouth County NJ from our office in Holmdel to Atlantic Highlands, we are autonomous approximately 98% of the time. We also drove 10 miles on the Garden State Parkway (a multi-lane divided highway with on and off ramps) with zero intercepts.

Here is a video of our test car driving in diverse conditions.

Visualization of Internal CNN State

Figure 8: How the CNN “sees” an unpaved road. Top: subset of the camera image sent to the CNN. Bottom left: Activation of the first layer feature maps. Bottom right: Activation of the second layer feature maps. This demonstrates that the CNN learned to detect useful road features on its own, i. e., with only the human steering angle as training signal. We never explicitly trained it to detect the outlines of roads.

Figures 8 and 9 show the activations of the first two feature map layers for two different example inputs, an unpaved road and a forest. In case of the unpaved road, the feature map activations clearly show the outline of the road while in case of the forest the feature maps contain mostly noise, i. e., the CNN finds no useful information in this image.

This demonstrates that the CNN learned to detect useful road features on its own, i. e., with only the human steering angle as training signal. We never explicitly trained it to detect the outlines of roads, for example.

Figure 9: Example image with no road. The activations of the first two feature maps appear to contain mostly noise, i. e., the CNN doesn’t recognize any useful features in this image.

Conclusions

We have empirically demonstrated that CNNs are able to learn the entire task of lane and road following without manual decomposition into road o 

• Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4):541–551, Winter 1989.
  URL: http://yann.lecun.org/exdb/publis/pdf/lecun-89e.pdf. 
• Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. 
• In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 1097–1105. Curran Associates, Inc., 2012. URL: http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf.
• L. D. Jackel, D. Sharman, Stenard C. E., Strom B. I., , and D Zuckert. Optical character recognition for self-service banking. AT&T Technical Journal, 74(1):16–24, 1995.
• Large scale visual recognition challenge (ILSVRC). URL: http://www.image-net.org/challenges/LSVRC/.
• Net-Scale Technologies, Inc. Autonomous off-road vehicle control using end-to-end learning, July 2004. Final technical report. URL: http://net-scale.com/doc/net-scale-dave-report.pdf.
• Dean A. Pomerleau. ALVINN, an autonomous land vehicle in a neural network. Technical report, Carnegie Mellon University, 1989.
  URL: http://repository.cmu.edu/cgi/viewcontent.cgi?article=2874&context=compsci.
• Danwei Wang and Feng Qi. Trajectory planning for a four-wheel-steering vehicle. In Proceedings of the 2001 IEEE International Conference on Robotics & Automation, May 21–26 2001. URL: http://www.ntu.edu.sg/home/edwwang/confpapers/wdwicar01.pdf.
  rlane marking detection, semantic abstraction, path planning, and control. A small amount of training data from less than a hundred hours of driving was sufficient to train the car to operate in diverse conditions, on highways, local and residential roads in sunny, cloudy, and rainy conditions. 
• The CNN is able to learn meaningful road features from a very sparse training signal (steering alone).
• The system learns for example to detect the outline of a road without the need of explicit labels during training.
• More work is needed to improve the robustness of the network, to find methods to verify the robustness, and to improve visualization of the network-internal processing steps.

For full details please see the paper that this blog post is based on, and please contact us if you would like to learn more about NVIDIA’s autonomous vehicle platform!

REFERENCES

1. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backprop- agation applied to handwritten zip code recognition. Neural Computation, 1(4):541–551, Winter 1989. URL: http://yann.lecun.org/exdb/publis/pdf/lecun-89e.pdf.
2. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 1097–1105. Curran Associates, Inc., 2012. URL: http://papers.nips.cc/paper/ 4824-imagenet-classification-with-deep-convolutional-neural-networks. pdf.
3. L. D. Jackel, D. Sharman, Stenard C. E., Strom B. I., , and D Zuckert. Optical character recognition for self-service banking. AT&T Technical Journal, 74(1):16–24, 1995.
4. Large scale visual recognition challenge (ILSVRC). URL: http://www.image-net.org/ challenges/LSVRC/.
5. Net-Scale Technologies, Inc. Autonomous off-road vehicle control using end-to-end learning, July 2004. Final technical report. URL: http://net-scale.com/doc/net-scale-dave-report.pdf.
6. Dean A. Pomerleau. ALVINN, an autonomous land vehicle in a neural network. Technical report, Carnegie Mellon University, 1989. URL: http://repository.cmu.edu/cgi/viewcontent. cgi?article=2874&context=compsci.
7. Danwei Wang and Feng Qi. Trajectory planning for a four-wheel-steering vehicle. In Proceedings of the 2001 IEEE International Conference on Robotics & Automation, May 21–26 2001. URL: http: //www.ntu.edu.sg/home/edwwang/confpapers/wdwicar01.pdf.

ORIGINAL: NVidia

by Ben Firner, Beat Flepp, Karol Zieba, Larry Jackel, Mariusz Bojarski and Urs Muller

August 17, 2016

First Human Tests of Memory Boosting Brain Implant—a Big Leap Forward

“You have to begin to lose your memory, if only bits and pieces, to realize that memory is what makes our lives. Life without memory is no life at all.” — Luis Buñuel Portolés, Filmmaker

Image Credit: Shutterstock.com

Every year, hundreds of millions of people experience the pain of a failing memory.

The reasons are many:

• traumatic brain injury, which haunts a disturbingly high number of veterans and football players; 
• stroke or Alzheimer’s disease, which often plagues the elderly; or 
• even normal brain aging, which inevitably touches us all.

Memory loss seems to be inescapable. But one maverick neuroscientist is working hard on an electronic cure. Funded by DARPA, Dr. Theodore Berger, a biomedical engineer at the University of Southern California, is testing a memory-boosting implant that mimics the kind of signal processing that occurs when neurons are laying down new long-term memories.

The revolutionary implant, already shown to help memory encoding in rats and monkeys, is now being tested in human patients with epilepsy — an exciting first that may blow the field of memory prosthetics wide open.

To get here, however, the team first had to crack the memory code.

Deciphering Memory

From the very onset, Berger knew he was facing a behemoth of a problem.

We weren’t looking to match everything the brain does when it processes memory, but to at least come up with a decent mimic, said Berger.

“Of course people asked: can you model it and put it into a device? Can you get that device to work in any brain? It’s those things that lead people to think I’m crazy. They think it’s too hard,” he said.

But the team had a solid place to start.

The hippocampus, a region buried deep within the folds and grooves of the brain, is the critical gatekeeper that transforms memories from short-lived to long-term. In dogged pursuit, Berger spent most of the last 35 years trying to understand how neurons in the hippocampus accomplish this complicated feat.

At its heart, a memory is a series of electrical pulses that occur over time that are generated by a given number of neurons, said Berger. This is important — it suggests that we can reduce it to mathematical equations and put it into a computational framework, he said.

Berger hasn’t been alone in his quest.

By listening to the chatter of neurons as an animal learns, teams of neuroscientists have begun to decipher the flow of information within the hippocampus that supports memory encoding. Key to this process is a strong electrical signal that travels from CA3, the “input” part of the hippocampus, to CA1, the “output” node.

This signal is impaired in people with memory disabilities, said Berger, so of course we thought if we could recreate it using silicon, we might be able to restore — or even boost — memory.

Bridging the Gap

Yet this brain’s memory code proved to be extremely tough to crack.

The problem lies in the non-linear nature of neural networks: signals are often noisy and constantly overlap in time, which leads to some inputs being suppressed or accentuated. In a network of hundreds and thousands of neurons, any small change could be greatly amplified and lead to vastly different outputs.

It’s a chaotic black box, laughed Berger.

With the help of modern computing techniques, however, Berger believes he may have a crude solution in hand. His proof?

Use his mathematical theorems to program a chip, and then see if the brain accepts the chip as a replacement — or additional — memory module.

Berger and his team began with a simple task using rats. They trained the animals to push one of two levers to get a tasty treat, and recorded the series of CA3 to CA1 electronic pulses in the hippocampus as the animals learned to pick the correct lever. The team carefully captured the way the signals were transformed as the session was laid down into long-term memory, and used that information — the electrical “essence” of the memory — to program an external memory chip.

They then injected the animals with a drug that temporarily disrupted their ability to form and access long-term memories, causing the animals to forget the reward-associated lever. Next, implanting microelectrodes into the hippocampus, the team pulsed CA1, the output region, with their memory code.

The results were striking — powered by an external memory module, the animals regained their ability to pick the right lever.

Encouraged by the results, Berger next tried his memory implant in monkeys, this time focusing on a brain region called the prefrontal cortex, which receives and modulates memories encoded by the hippocampus.

Placing electrodes into the monkey’s brains, the team showed the animals a series of semi-repeated images, and captured the prefrontal cortex’s activity when the animals recognized an image they had seen earlier. Then with a hefty dose of cocaine, the team inhibited that particular brain region, which disrupted the animal’s recall.

Next, using electrodes programmed with the “memory code,” the researchers guided the brain’s signal processing back on track — and the animal’s performance improved significantly.

A year later, the team further validated their memory implant by showing it could also rescue memory deficits due to hippocampal malfunction in the monkey brain.

A Human Memory Implant

Last year, the team cautiously began testing their memory implant prototype in human volunteers.

Because of the risks associated with brain surgery, the team recruited 12 patients with epilepsy, who already have electrodes implanted into their brain to track down the source of their seizures.

Repeated seizures steadily destroy critical parts of the hippocampus needed for long-term memory formation, explained Berger. So if the implant works, it could benefit these patients as well.

The team asked the volunteers to look through a series of pictures, and then recall which ones they had seen 90 seconds later. As the participants learned, the team recorded the firing patterns in both CA1 and CA3 — that is, the input and output nodes.

Using these data, the team extracted an algorithm — a specific human “memory code” — that could predict the pattern of activity in CA1 cells based on CA3 input. Compared to the brain’s actual firing patterns, the algorithm generated correct predictions roughly 80% of the time.

It’s not perfect, said Berger, but it’s a good start.

Using this algorithm, the researchers have begun to stimulate the output cells with an approximation of the transformed input signal.

We have already used the pattern to zap the brain of one woman with epilepsy, said Dr. Dong Song, an associate professor working with Berger. But he remained coy about the result, only saying that although promising, it’s still too early to tell.

Song’s caution is warranted. Unlike the motor cortex, with its clear structured representation of different body parts, the hippocampus is not organized in any obvious way.

It’s hard to understand why stimulating input locations can lead to predictable results, said Dr. Thoman McHugh, a neuroscientist at the RIKEN Brain Science Institute. It’s also difficult to tell whether such an implant could save the memory of those who suffer from damage to the output node of the hippocampus.

“That said, the data is convincing,” McHugh acknowledged.

Berger, on the other hand, is ecstatic. “I never thought I’d see this go into humans,” he said.

But the work is far from done. Within the next few years, Berger wants to see whether the chip can help build long-term memories in a variety of different situations. After all, the algorithm was based on the team’s recordings of one specific task — what if the so-called memory code is not generalizable, instead varying based on the type of input that it receives?

Berger acknowledges that it’s a possibility, but he remains hopeful.

I do think that we will find a model that’s a pretty good fit for most conditions, he said. After all, the brain is restricted by its own biophysics — there’s only so many ways that electrical signals in the hippocampus can be processed, he said.

“The goal is to improve the quality of life for somebody who has a severe memory deficit,” said Berger. “If I can give them the ability to form new long-term memories for half the conditions that most people live in, I’ll be happy as hell, and so will be most patients.”

ORIGINAL: Singularity Hub

BY SHELLY FAN

ON NOV 15, 2015

Scientists Demonstrate Basics of Nucleic Acid Computing Inside Cells

.

DETAILS: Using strands of nucleic acid, scientists have demonstrated basic computing operations inside a living mammalian cell. Shown examining a cellular “AND” gate are associate professor Philip Santangelo and research scientist Chiara Zurla. (Credit: Rob Felt, Georgia Tech)

Using strands of nucleic acid, scientists have demonstrated basic computing operations inside a living mammalian cell. The research could lead to an artificial sensing system that could control a cell’s behavior in response to such stimuli as the presence of toxins or the development of cancer.

The research uses DNA strand displacement, a technology that has been widely used outside of cells for the design of molecular circuits, motors and sensors. Researchers modified the process to provide both “AND” and “OR” logic gates able to operate inside the living cells and interact with native messenger RNA (mRNA).

The tools they developed could provide a foundation for bio-computers able to sense, analyze and modulate molecular information at the cellular level. Supported by the Defense Advanced Research Projects Agency (DARPA) and the National Science Foundation (NSF), the research was reported December 21 in the journal Nature Nanotechnology.

“The whole idea is to be able to take the logic that is used in computers and port that logic into cells themselves,” said .Philip Santangelo, an associate professor in the .Wallace H. Coulter Department of Biomedical Engineering at Georgia Tech and Emory University. “These devices could sense an aberrant RNA, for instance, and then shut down cellular translation or induce cell death.”

Strand displacement reactions are the biological equivalent of the switches or gates that form the foundation for silicon-based computing. They can be programmed to turn on or off in response to an external stimuli such as a molecule. An “AND” gate, for example, would switch when both conditions were met, while an “OR” gate would switch when either condition was met.

In the switches the researchers used, a fluorophore reporter molecule and its complementary quenching molecule were placed side-by-side to create an “off” mode. Binding of RNA in one of the strands then displaced a portion of nucleic acid, separating the molecules and allowing generation of a signal that created an “on” mode. Two “on” modes on adjacent nucleic acid strands created an “AND” gate.

“Demonstrating individual logic gates is only a first step,” said Georg Seelig, assistant professor of computer science and engineering and electrical engineering at the University of Washington. “In the longer term, we want to expand this technology to create circuits with many inputs, such as those we have constructed in cell-free settings.”

The researchers used ligands designed to bind to specific portions of the nucleic acid strands, which can be created as desired and produced by commercial suppliers.

“We sensed molecules and showed that we could respond to them,” said Santangelo. “We showed that we could utilize native molecules in the cell as part of the circuit, though we haven’t been able to control a cell yet.”

Getting basic computing operations to function inside cells was no easy task, and the research required a number of years to accomplish. Among the challenges were getting the devices into the cells without triggering the switches, providing operation rapid enough to be useful, and not killing the human cell lines that researchers used in the lab.

“We had to chemically change the probes to get them to work inside the cell and to make them stable enough inside the cells,” said Santangelo. “We found that these strand displacement reactions can be slow within the cytosol, so to get them to work faster, we built scaffolding onto the messenger RNA that allowed us to amplify the effects.”

The nucleic acid computers ultimately operated as desired, and the next step is to use their switching to trigger the production of signaling chemicals that would prompt the desired reaction from the cells. Cellular activity is normally controlled by the production of proteins, so the nucleic acid switches will have to be given the ability to produce enough signaling molecules to induce a change.

“We need to generate enough of whatever final signal is needed to get the cell to react,” Santangelo explained. “There are amplification methods used in strand displacement technology, but none of them have been used so far in living cells.”

Even without that final step, the researchers feel they’ve built a foundation that can be used to attain the goal.

“We were able to design some of the basic logical constructs that could be used as building blocks for future work,” Santangelo said. “We know the concentrations of chemicals and the design requirements for individual components, so we can now start putting together a more complicated set of circuits and components.”

Cells, of course, already know how to sense toxic molecules and the development malignant tendencies, and to then take action. But those safeguards can be turned off by viruses or cancer cells that know how to circumvent natural cellular processes.

“Our mechanism would just give cells a hand at doing this,” Santangelo said. “The idea is to add to the existing machinery to give the cells enhanced capabilities.”

Applying an engineering approach to the biological world sets this example apart from other efforts to control cellular machinery.

“What makes DNA strand displacement circuits unique is that all components are fully rationally designed at the level of the DNA sequence,” said Seelig. “This really makes this technology ideal for an engineering approach. In contrast, many other approaches to controlling the cellular machinery rely on components that are borrowed from biology and are not fully understood.”

Beyond those already mentioned, the research team included Benjamin Groves, Yuan-Jyue Chen and Sergii Pochekailov from the University of Washington and Chiara Zurla and Jonathan Kirschman from Georgia Tech and Emory University.

This material is based on work supported by the Defense Advanced Research Projects Agency (DARPA) under contract W911NF-11-2-0068 and by National Science Foundation CAREER award 1253691. The content is solely the responsibility of the authors and does not necessarily represent the official views of DARPA or the NSF.

.

Image shows activation of “AND” gates in cells as observed by fluorescence microscopy.
(Credit: Chiara Zurla, Georgia Tech)

CITATION: Benjamin Groves, et al., “Computing in mammalian cells with nucleic acid strand exchange,” (Nature Nanotechnology, 2015)..http://dx.doi.org/10.1038/nnano.2015.278

Research News

Georgia Institute of Technology

177 North Avenue

Atlanta, Georgia 30332-0181 USA

Media Relations Contact: John Toon (404-894-6986) (.joon@gatech.edu).

Writer: John Toon

ORIGINAL: .Geogia Tech
January 19, 2016

Bridging the Bio-Electronic Divide

New effort aims for fully implantable devices able to connect with up to one million neurons

A new DARPA program aims to develop an implantable neural interface able to provide unprecedented signal resolution and data-transfer bandwidth between the human brain and the digital world. The interface would serve as a translator, converting between the electrochemical language used by neurons in the brain and the ones and zeros that constitute the language of information technology. The goal is to achieve this communications link in a biocompatible device no larger than one cubic centimeter in size, roughly the volume of two nickels stacked back to back.

The program, Neural Engineering System Design (NESD), stands to dramatically enhance research capabilities in neurotechnology and provide a foundation for new therapies.

“Today’s best brain-computer interface systems are like two supercomputers trying to talk to each other using an old 300-baud modem,” said Phillip Alvelda, the NESD program manager. “Imagine what will become possible when we upgrade our tools to really open the channel between the human brain and modern electronics.”

Among the program’s potential applications are devices that could compensate for deficits in sight or hearing by feeding digital auditory or visual information into the brain at a resolution and experiential quality far higher than is possible with current technology.

Neural interfaces currently approved for human use squeeze a tremendous amount of information through just 100 channels, with each channel aggregating signals from tens of thousands of neurons at a time. The result is noisy and imprecise. In contrast, the NESD program aims to develop systems that can communicate clearly and individually with any of up to one million neurons in a given region of the brain.

Achieving the program’s ambitious goals and ensuring that the envisioned devices will have the potential to be practical outside of a research setting will require integrated breakthroughs across numerous disciplines including 

• neuroscience, 
• synthetic biology, 
• low-power electronics, 
• photonics, 
• medical device packaging and manufacturing, systems engineering, and 
• clinical testing. 

In addition to the program’s hardware challenges, NESD researchers will be required to develop advanced mathematical and neuro-computation techniques to first transcode high-definition sensory information between electronic and cortical neuron representations and then compress and represent those data with minimal loss of fidelity and functionality.

To accelerate that integrative process, the NESD program aims to recruit a diverse roster of leading industry stakeholders willing to offer state-of-the-art prototyping and manufacturing services and intellectual property to NESD researchers on a pre-competitive basis. In later phases of the program, these partners could help transition the resulting technologies into research and commercial application spaces.

To familiarize potential participants with the technical objectives of NESD, DARPA will host a Proposers Day meeting that runs Tuesday and Wednesday, February 2-3, 2016, in Arlington, Va. The Special Notice announcing the Proposers Day meeting is available at https://www.fbo.gov/spg/ODA/DARPA/CMO/DARPA-SN-16-16/listing.html. More details about the Industry Group that will support NESD is available at https://www.fbo.gov/spg/ODA/DARPA/CMO/DARPA-SN-16-17/listing.html. A Broad Agency Announcement describing the specific capabilities sought will be forthcoming on www.fbo.gov.

NESD is part of a broader portfolio of programs within DARPA that support President Obama’s brain initiative. For more information about DARPA’s work in that domain, please visit:http://www.darpa.mil/program/our-research/darpa-and-the-brain-initiative.

ORIGINAL: DARPA

OUTREACH@DARPA.MIL
1/19/2016

ORNL cell-free protein synthesis is potential lifesaver

This section of a serpentine channel reactor shows the parallel reactor and feeder channels separated by a nanoporous membrane. At left is a single nanopore viewed from the side; at right is a diagram of metabolite exchange across the membrane.

OAK RIDGE, Tenn., Dec. 29, 2015 – Lives of soldiers and others injured in remote locations could be saved with a cell-free protein synthesis system developed at the Department of Energy’s Oak Ridge National Laboratory.

The device, a creation of a team led by Andrea Timm and Scott Retterer of the lab’s Biosciences Division, uses microfabricated bioreactors to facilitate the on-demand production of therapeutic proteins for medicines and biopharmaceuticals. Making these miniature factories cell-free, which eliminates the maintenance of a living system, simplifies the process and lowers cost.

“With this approach, we can produce more protein faster, making our technology ideal for point-of-care use,” Retterer said. “The fact it’s cell-free reduces the infrastructure needed to produce the protein and opens the possibility of creating proteins when and where you need them, bypassing the challenge of keeping the proteins cold during shipment and storage.”

ORNL’s bioreactor features elegance through a permeable nanoporous membrane and serpentine design fabricated using a combination of electron beam and photolithography and advanced material deposition processes. This design enables prolonged cell-free reactions for efficient production of proteins, making it easily adaptable for use in isolated locations and at disaster sites.

From a functional perspective, the design uses long serpentine channels integrated in a way to allow the exchange of materials between parallel reactor and feeder channels. With this approach, the team can control the exchange of metabolites, energy and species that inhibit production of the desired protein. Through other design features, researchers extend reaction times and improve yields.

“We show that the microscale bioreactor design produces higher protein yields than conventional tube-based batch formats and that product yields can be dramatically improved by facilitating small molecule exchange with the dual-channel bioreactor,” the authors wrote in their paper, published in the journal Small.

The researchers also note that on-demand biologic synthesis would aid the production of drugs that are costly to mass-produce, including orphan drugs and personalized medicines.

Other authors of the paper, titled “Towards Microfluidic Reactors for Cell-Free Protein Synthesis at the Point-of-Care,” are ORNL’s Peter Shankles, Carmen Foster and Mitchel Doktycz.

Funding for this project was provided by the Defense Advanced Research Projects Agency through a collaboration with researchers from Leidos (https://www.leidos.com) and Northwestern University. This accomplishment represents the culmination of research led by Doktycz and Retterer funded by multiple grants from the National Institutes of Health and DOE over the last decade. A portion of the work was performed at the Center for Nanophase Materials Sciences, a DOE Office of Science User Facility.

UT-Battelle manages ORNL for the DOE's Office of Science. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit http://science.energy.gov/.

ORIGINAL: ORNL

Ron Walli, Communications. wallira@ornl.gov, 865.576.0226

December 29, 2015

Forward to the Future: Visions of 2045

DARPA asked the world and our own researchers what technologies they expect to see 30 years from now—and received insightful, sometimes funny predictions

Today—October 21, 2015—is famous in popular culture as the date 30 years in the future when Marty McFly and Doc Brown arrive in their time-traveling DeLorean in the movie “Back to the Future Part II.” The film got some things right about 2015, including in-home videoconferencing and devices that recognize people by their voices and fingerprints. But it also predicted trunk-sized fusion reactors, hoverboards and flying cars—game-changing technologies that, despite the advances we’ve seen in so many fields over the past three decades, still exist only in our imaginations.

A big part of DARPA’s mission is to envision the future and make the impossible possible. So ten days ago, as the “Back to the Future” day approached, we turned to social media and asked the world to predict: What technologies might actually surround us 30 years from now? We pointed people to presentations from DARPA’s Future Technologies Forum, held last month in St. Louis, for inspiration and a reality check before submitting their predictions.

Well, you rose to the challenge and the results are in. So in honor of Marty and Doc (little known fact: he is a DARPA alum) and all of the world’s innovators past and future, we present here some highlights from your responses, in roughly descending order by number of mentions for each class of futuristic capability:

• Space: Interplanetary and interstellar travel, including faster-than-light travel; missions and permanent settlements on the Moon, Mars and the asteroid belt; space elevators
• Transportation & Energy: Self-driving and electric vehicles; improved mass transit systems and intercontinental travel; flying cars and hoverboards; high-efficiency solar and other sustainable energy sources
• Medicine & Health: Neurological devices for memory augmentation, storage and transfer, and perhaps to read people’s thoughts; life extension, including virtual immortality via uploading brains into computers; artificial cells and organs; “Star Trek”-style tricorder for home diagnostics and treatment; wearable technology, such as exoskeletons and augmented-reality glasses and contact lenses
• Materials & Robotics: Ubiquitous nanotechnology, 3-D printing and robotics; invisibility and cloaking devices; energy shields; anti-gravity devices
• Cyber & Big Data: Improved artificial intelligence; optical and quantum computing; faster, more secure Internet; better use of data analytics to improve use of resources

A few predictions inspired us to respond directly:

• “Pizza delivery via teleportation”—DARPA took a close look at this a few years ago and decided there is plenty of incentive for the private sector to handle this challenge.
• “Time travel technology will be close, but will be closely guarded by the military as a matter of national security”—We already did this tomorrow.
• “Systems for controlling the weather”—Meteorologists told us it would be a job killer and we didn’t want to rain on their parade.
• “Space colonies…and unlimited cellular data plans that won't be slowed by your carrier when you go over a limit”—We appreciate the idea that these are equally difficult, but they are not. We think likable cell-phone data plans are beyond even DARPA and a total non-starter.

So seriously, as an adjunct to this crowd-sourced view of the future, we asked three DARPA researchers from various fields to share their visions of 2045, and why getting there will require a group effort with players not only from academia and industry but from forward-looking government laboratories and agencies:

Pam Melroy, an aerospace engineer, former astronaut and current deputy director of DARPA’s Tactical Technologies Office (TTO), foresees technologies that would enable machines to collaborate with humans as partners on tasks far more complex than those we can tackle today:

Justin Sanchez, a neuroscientist and program manager in DARPA’s Biological Technologies Office (BTO), imagines a world where neurotechnologies could enable users to interact with their environment and other people by thought alone:

Stefanie Tompkins, a geologist and director of DARPA’s Defense Sciences Office, envisions building substances from the atomic or molecular level up to create “impossible” materials with previously unattainable capabilities.

Check back with us in 2045—or sooner, if that time machine stuff works out—for an assessment of how things really turned out in 30 years.

# # #

Associated images posted on www.darpa.mil and video posted at www.youtube.com/darpatv may be reused according to the terms of the DARPA User Agreement, available here:http://www.darpa.mil/policy/usage-policy.

Tweet @darpa

ORIGINAL: DARPA

OUTREACH@DARPA.MIL

10/21/2015

BioPartsBuilder: a synthetic biology tool for combinatorial assembly of biological parts

BioPartsBuilder: a synthetic biology tool for combinatorial assembly of biological parts

Kun Yang 1∗ , Giovanni Stracquadanio 1∗ , Jingchuan Luo 2 , Jef D. Boeke 2 and Joel S. Bader 1 †

1Department of Biomedical Engineering, Johns Hopkins University, 3400 N. Charles Street,

Baltimore, MD 21218

2Institute for Systems Genetics and Department of Biochemistry and Molecular Pharmacology,

NYU Langone Medical Center, New York, NY 10016

Abstract

Summary: Combinatorial assembly of DNA elements is an efficient method for building large-scale synthetic pathways from standardized, reusable components. These methods are particularly useful because they enable assembly of multiple DNA fragments in one reaction, at the cost of requiring that each fragment satisfy design constraints. We developed BIOPARTSBUILDER as a biologist-friendly web tool to design biological parts that are compatible with DNA combinatorial assembly methods, such as Golden Gate and related methods. 

It 

• retrieves biological sequences, 
• enforces compliance with assembly design standards, and 
• provides a fabrication plan for each fragment.

Availability: BIOPARTSBUILDER is accessible at http://public.biopartsbuilder.org and an Amazon Web Services image is available from the AWS Market Place (AMI ID: ami-508acf38). Source code is released under the MIT license, and available for download at https://github.com/baderzone/biopartsbuilder.

Full PDF

Contact: joel.bader@jhu.edu

ABSTRACT

Summary: Combinatorial assembly of DNA elements is an efficient method for building large-scale synthetic pathways from standardized, reusable components. These methods are particularly useful because they enable assembly of multiple DNA fragments in one reaction, at the cost of requiring that each fragment satisfy design constraints. We developed BIO PARTS BUILDER as a biologist-friendly web tool to design biological parts that are compatible with DNA combinatorial assembly methods, such as Golden Gate and related methods. It retrieves biological sequences, enforces compliance with assembly design standards, and provides a fabrication plan for each fragment.

Availability: BIO PARTS BUILDER is accessible at http://public.biopartsbuilder.org and an Amazon Web Services image is available from the AWS Market Place (AMI ID: ami-508acf38).

Source code is released under the MIT license, and available for download at https://github.com/baderzone/biopartsbuilder.

Contact: joel.bader@jhu.edu

1 INTRODUCTION

DNA synthesis technologies are improving faster than Moore’s law, allowing the synthesis of genes, pathways (Ro et al. (2.06)), bacterial genomes (Gibson et al. (2.10)), eukaryotic chromosomes (Dymond et al. (2.11); Annaluru et al. (2.14)), and eventually entire eukaryotic genomes. Many projects have individual ‘parts’ as synthetic targets, such as promoters, coding domains, and transcriptional terminators. While individual parts can be characterized, predicting how parts will operate together remains a challenge. Rather than building a single construct, therefore, it can be more efficient to specify multiple alternatives for each part, then use massively parallel synthesis and assembly to generate a combinatorial library that can be screened for the desired function. In particular, Golden Gate assembly is an efficient and effective strategy to assemble combinatorial libraries (Engler et al. (2.09)). However, Golden Gate assembly requires a computationally challenging design step to create ‘standardized’ parts that have compatible overhangs, lack pre-defined restriction sites, and comply with other constraints.

To streamline the process of designing standardized biological parts for Golden Gate assembly, we developed BIO PARTS BUILDER , which retrieves sequence data from different sources and ensures compliance with design standards that are compatible with combinatorial assembly. Though there are tools for automated parts retrieval (Scher et al. (2.14)) and subsequent primer design for DNA assembly (Bode et al. (2.09); Rouillard et al. (2.04)), the choices for Golden Gate assembly are limited. Compared to existing Golden Gate designers (Hillson et al. (2.12.), BIO PARTS BUILDER is distributed open source software and freely modified by both academic and commercial users. BIO PARTS BUILDER also provides a repository system that stores the designed parts and shares data within members associated with the same laboratory. BIO PARTS BUILDER therefore provides useful, new, integrated and extendable functionality for the synthetic biology community.

2.SOFTWARE MODULES

BIO PARTS BUILDER provides an easy interface to retrieve, design and order parts (Fig. 1) that are compatible with Golden Gate (Engler et al. (2.09)), BglBrick (Anderson et al. (2.10)), or user-defined assembly standards.

2.1 Part Retrieval

BIOPARTSBUILDER implements a sophisticated sequence retrieval system to gather data from different sources. Users can submit a list of RefSeq protein/nucleotide accession numbers to retrieve sequences and annotations from NCBI, or for parts without RefSeq accessions or with customized sequences and annotations, users can upload a file in F ASTA or CSV format. As retrieving a large number of arbitrary parts from a genome and upload to the system is tedious, BIO PARTS BUILDER implements an advanced search engine for retrieving parts from annotated genomes, similar to GENOME CARVER software (Scher et al. (2.14)). It parses annotations, generates and stores a search index, and provides access to structured search terms (Table S1-S2. through the Apache SOLR query language. 

2.2.Part Design

Parts imported into BIO PARTS BUILDER can be re-designed according to pre-defined or additional user-defined design standards. Users can customize the design workflow to perform one or more of the following steps. 

• Codon optimization: Users can specify host organism for codon optimization. This option can be left blank for parts that lack protein-coding regions, such as promoters or terminators. This step can be amended also to accomodate vendors’ specific recoding strategies. 
• Restriction enzyme constraints: Combinatorial assembly techniques largely rely on the use of specific restriction enzymes to create a unique assembly. For this reason, BIO PARTS BUILDER provides: 
  1. RESTRICTION ENZYME REMOVER that changes the nucleotide sequence to avoid restriction sites corresponding to a user-specified list of restriction enzymes; and 
  2. RESTRICTION ENZYME LOCATOR that detects the presence of user-specified restriction enzymes without recoding the sequence. BIO PARTS BUILDER organizes data and users by laboratories. People in the same laboratory can share parts and designs that remain private to other laboratories and the public. BIO PARTS BUILDER provides an administration panel specifically for laboratory administrators to manage members and design standards. The initial creator of a new laboratory in BIO PARTS BUILDER automatically becomes the laboratory administrator. 
• Prefix and suffix insertion: Users can specify sequences to be added to the beginning and the end of each part.
• Fabrication: Parts can be larger that the synthesis capability of commercial providers. In this case, BIO PARTS BUILDER splits the sequence in fragments of user-defined length using unique overlaps, which allow unambiguous assembly (see Supplementary Text). 

BIO PARTS BUILDER assigns a unique Job ID to each design task. Users can check the status and error report of design tasks online. When the design task is finished, BIO PARTS BUILDER sends an email to notify the user.

2.3 Order 

Design results are accessible online. And users can also use the ORDER module to collect specific designs and prepare files for ordering parts from companies. The ORDER module creates statistical summaries for user-selected designs and provides tables of parts, constructs and design standards. It generates spreadsheets, sequence files, and summary report files for users to download.

2.4 AutoBuild

To streamline the entire design process, BIO PARTS BUILDER has a fully automated design module, AUTO BUILD , which allows users to retrieve, design and create orders for a batch of parts with one click. This module serves as a convenient ‘wizard’ for users whose needs are met by the most common design standards, which are already defined in the software.

3 DESIGN WORKFLOW EXAMPLE 

Using Autobuild it is possible to quickly design both coding and non-coding parts for Golden Gate using these two workflows. 

• Coding region . In the “Search Genomes” tab, input query “systematic name:YBR019C”; select “Golden Gate - CDS” design standard and assign a name to the Order. Click create parts, then select the part and confirm your design. 
• Non Coding region. In the “Search Genomes” tab, input query “systematic name:YBR019C promoter”; select “Golden Gate - NonCDS” design standard and assign a name to the Order. Click create parts, then select the part and confirm your design. 

Acknowledgement: The authors would like to thank N. Agmon, Z. Xu, D. M. Truong and L. Mitchell for testing the application. 

Funding: This work was supported by Defense Advanced Research Projects Agency [grant number N66001-12.C-402.]

.REFERENCES

• Anderson, J. C. et al. (2.10). Bglbricks: A flexible standard for biological part assembly. Journal of Biological Engineering, 4(1), 1. 
• Annaluru, N. et al. (2.14). Total synthesis of a functional designer eukaryotic chromosome. Science (New York, N.Y.), 344(6179), 55–58. 
• Bode, M. et al. (2.09). Tmprime: fast, flexible oligonucleotide design software for gene synthesis. Nucleic Acids Research, 37(suppl 2., W2.4–W2.1. 
• Dymond, J. S. et al. (2.11). Synthetic chromosome arms function in yeast and generate phenotypic diversity by design. Nature, 477(7365), 471–476. 
• Engler, C. et al. (2.09). Golden gate shuffling: a one-pot dna shuffling method based on type iis restriction enzymes. PLoS One, 4(5), e5553. 
• Gibson, D. G. et al. (2.10). Creation of a bacterial cell controlled by a chemically synthesized genome. science, 32.(5987), 52.56. 
• Hillson, N. J. et al. (2.12.. j5 dna assembly design automation software. ACS Synthetic Biology, 1(1), 14–2.. 
• Ro, D.-K. et al. (2.06). Production of the antimalarial drug precursor artemisinic acid in engineered yeast. Nature, 440(7086), 940–943. 
• Rouillard, J.-M. et al. (2.04). Gene 2.ligo: oligonucleotide design for in vitro gene synthesis. Nucleic Acids Research, 32.Web Server issue), W176–180. 
• Scher, E. et al. (2.14). Genomecarver: harvesting genetic parts from genomes to support biological design automation. In 6th International Workshop on Bio-Design Automation.

ORIGINAL: Oxford Journals

Kun Yang1,*, 

Giovanni Stracquadanio1,*, 

Jingchuan Luo2, 

Jef D. Boeke2 and 

Joel S. Bader1,†

+Author Affiliations

1Department of Biomedical Engineering, Johns Hopkins University, 3400 N. Charles Street, Baltimore, MD 21218

2Institute for Systems Genetics and Department of Biochemistry and Molecular Pharmacology, NYU Langone Medical Center, New York, NY 10016

↵†to whom correspondence should be addressed. Joel S. Bader, E-mail: joel.bader@jhu.edu

Received May 19, 2015.

Revision received October 8, 2015.

Accepted November 9, 2015.