Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Bringing neural networks to cellphones
Method for modeling neural networks’ power consumption could help make the systems portable.   
     In recent years, the best-performing artificial-intelligence systems — in areas such as autonomous driving, speech recognition, computer vision, and automatic translation — have come courtesy of software systems known as neural networks.

But neural networks take up a lot of memory and consume a lot of power, so they usually run on servers in the cloud, which receive data from desktop or mobile devices and then send back their analyses.
Last year, MIT associate professor of electrical engineering and computer science Vivienne Sze and colleagues unveiled a new, energy-efficient computer chip optimized for neural networks, which could enable powerful artificial-intelligence systems to run locally on mobile devices.
Now, Sze and her colleagues have approached the same problem from the opposite direction, with a battery of techniques for designing more energy-efficient neural networks. First, they developed an analytic method that can determine how much power a neural network will consume when run on a particular type of hardware. Then they used the method to evaluate new techniques for paring down neural networks so that they’ll run more efficiently on handheld devices.
The researchers describe the work in a paper they’re presenting next week at the Computer Vision and Pattern Recognition Conference. In the paper, they report that the methods offered as much as a 73 percent reduction in power consumption over the standard implementation of neural networks, and as much as a 43 percent reduction over the best previous method for paring the networks down.
Energy evaluator
Loosely based on the anatomy of the brain, neural networks consist of thousands or even millions of simple but densely interconnected information-processing nodes, usually organized into layers. Different types of networks vary according to their number of layers, the number of connections between the nodes, and the number of nodes in each layer.
The connections between nodes have “weights” associated with them, which determine how much a given node’s output will contribute to the next node’s computation. During training, in which the network is presented with examples of the computation it’s learning to perform, those weights are continually readjusted, until the output of the network’s last layer consistently corresponds with the result of the computation.
“The first thing we did was develop an energy-modeling tool that accounts for data movement, transactions, and data flow,” Sze says. “If you give it a network architecture and the value of its weights, it will tell you how much energy this neural network will take. One of the questions that people had is ‘Is it more energy efficient to have a shallow network and more weights or a deeper network with fewer weights?’ This tool gives us better intuition as to where the energy is going, so that an algorithm designer could have a better understanding and use this as feedback. The second thing we did is that, now that we know where the energy is actually going, we started to use this model to drive our design of energy-efficient neural networks.”
In the past, Sze explains, researchers attempting to reduce neural networks’ power consumption used a technique called “pruning.” Low-weight connections between nodes contribute very little to a neural network’s final output, so many of them can be safely eliminated, or pruned.
Principled pruning
With the aid of their energy model, Sze and her colleagues — first author Tien-Ju Yang and Yu-Hsin Chen, both graduate students in electrical engineering and computer science — varied this approach. Although cutting even a large number of low-weight connections can have little effect on a neural net’s output, cutting all of them probably would, so pruning techniques must have some mechanism for deciding when to stop.
The MIT researchers thus begin pruning those layers of the network that consume the most energy. That way, the cuts translate to the greatest possible energy savings. They call this method “energy-aware pruning.”
Weights in a neural network can be either positive or negative, so the researchers’ method also looks for cases in which connections with weights of opposite sign tend to cancel each other out. The inputs to a given node are the outputs of nodes in the layer below, multiplied by the weights of their connections. So the researchers’ method looks not only at the weights but also at the way the associated nodes handle training data. Only if groups of connections with positive and negative weights consistently offset each other can they be safely cut. This leads to more efficient networks with fewer connections than earlier pruning methods did.
"Recently, much activity in the deep-learning community has been directed toward development of efficient neural-network architectures for computationally constrained platforms,” says Hartwig Adam, the team lead for mobile vision at Google. “However, most of this research is focused on either reducing model size or computation, while for smartphones and many other devices energy consumption is of utmost importance because of battery usage and heat restrictions. It demonstrates large performance gains over computation-focused methods. I hope other researchers in the field will follow suit and adopt this general methodology to neural-network-model architecture design."

  • [Image: MIT-Portable-Neural_0.jpg?itok=G-OEZ4Cw]
Folks promote their head of hair to hairpiece Wigs For Women
suppliers that create hairpieces from them. Before this your hair experiences a number of techniques. Hair is actually cleaned, chemically taken care of Wigs For Black Women
, coloured, then minimize along with designed straight into hairpieces.

It is easier to complement hairpieces through real hair to the hue of the normal head of hair Wigs For Sale
. These could be permed, lower, whack dried, along with formed just like you'd probably do with your own real hair. Hairpieces as a result manufactured possess a more organic appear and are smoother compared to their particular artificial counterparts. Real hair can also be more robust. Thus Wigs
, wigs made out of options furthermore stronger.

They are produced from man-made muscle. In comparison to head of hair hair pieces Wigs For Women
, the particular wigs coming from synthetic hair is not restyled in your own home. Moreover, you can not change their own colour from what needs been recently dyed from the producers.

Artificial wigs are less expensive than those produced from natural splendor, nevertheless to get a better and much more natural seem you may need to acquire a high priced top quality. Artificial hair pieces keep his or her form greater in comparison to wigs produced from Wigs For Cancer Patients
person curly hair.

Men and women sell their head of hair to wig Wigs For Sale
producers which craft wigs at their store. Prior to this your hair undergoes numerous functions. The hair is actually cleaned out, chemical treated Short Wigs
, coloured, and then reduce as well as created into hair pieces.

It can be simpler to go with wigs from natural splendor for the hue of your all-natural curly hair Bobbi Boss Wigs
. These can also be permed, minimize, hit dried, and also formed in the same way you'd probably do with your natural splendor. Hair pieces hence created possess a holistic look and therefore are much softer as compared to their particular artificial brethren. Human hair is also more robust. Thus Wigs For Women
, wigs produced from options in addition are more durable.

These are made from man-made muscles. In comparison to curly hair hairpieces Wigs For Women
, the hairpieces coming from man made curly hair is not restyled in your house. Additionally, you can't change their colour via what has already been coloured from the manufacturers.

Man made hairpieces are less costly than others produced from real human hair, nevertheless to experience a greater plus more organic search you may need to acquire a pricey top quality. Artificial hair pieces maintain their design superior when compared with hairpieces created from Wigs For Women
man curly hair.

People sell their head of hair to wig Wigs For Women
manufacturers who create hairpieces from their store. Prior to this the head of hair undergoes several techniques. Hair is cleansed, chemical treated Wigs For Women
, tinted, then reduce along with crafted straight into wigs.

It can be simpler to complement hairpieces from human hair for the hue of your all-natural hair Wigs For Women
. These may also be permed, minimize, blow dried, and also designed in the same way you would employ your current natural splendor. Hairpieces thus manufactured have a very natural appear and they are softer as compared to their particular manufactured brethren. Real human hair can be more robust. Thus Lace Front Wigs
, hair pieces produced from options furthermore are more durable.

These are created from man-made muscles. In comparison to head of hair hair pieces Hair Extensions
, the particular wigs from manufactured locks can not be restyled in your house. Moreover, you can not alter their own color through what has already been colored by the producers.

Artificial hairpieces are cheaper than these created from human hair, however to experience a better and much more normal seem you may need to acquire a costly high quality. Man made hair pieces preserve their form greater compared to hair pieces made out of Wigs
man head of hair.

Forum Jump:

Users browsing this thread:
1 Guest(s)

Bringing neural networks to cellphones00