Artificial Intelligence: Los Alamos Researchers Find Neural Networks Need “Sleep” Too
Turns out, neural networks benefit from slow-wave signals similar to those in a sleeping human brain.
Researchers from Los Alamos National Laboratory observed that artificial neurons in neural networks became unstable after extended periods spent working on certain kinds of tasks. It appears that these neurons need sleep just like their human counterparts. (NEW ATLAS)
The researchers used a “spiking neural network,” network. It is similar to “how humans and other biological systems learn from their environment during childhood development.”
They were intrigued by the growing instability in the network the longer it ran on a task. The project at hand, in this case, was dictionary training without supervision.
The scientists decided to test the network for sleep deficiency after all other methods to restore stability failed.
Parallels with a sleeping biological brain
The artificial neurons were exposed to slow-wave signals similar to those in a somnolent human brain. The new study found that the artificial neurons perked after this exposure.
After experimenting with different types of white noise signals, the Los Alamos researchers homed in on Gaussian noise. This is a kind of signal comprising a wide range of frequencies and amplitudes. It worked best on the artificial neurons by seemingly having a calming effect and helping them regain stability.
Uncannily, these figures are similar to the kinds of waves found in the human brain during a sleep phase known as slow-wave. This is a phase marked by the deepest sleep. Brain and muscle activity during this period is significantly reduced.
According to Yijing Watkins, lead author of the Los Alamos study, it appeared that the neural networks benefited from the “equivalent of a good night’s rest.”
Los Alamos computer scientist and study co-author Garrett Kenyon said that “the issue of how to keep learning systems from becoming unstable really only arises when attempting to utilize biologically realistic, spiking neuromorphic processors or when trying to understand biology itself.”
“The majority of machine learning, deep learning, and AI researchers never encounter this type of problem since they can perform mathematical operations that regulate the overall dynamical gain of the system.”
Related Story: Daphne Koller Raises $143M For Insitro, Her AI-Bio Startup
Latest Alternative Investment News
Funds and accounts advised by T. Rowe Price Associates, Inc. led an investment round worth $2.5 billion in electric vehicle maker Rivian. Participants in this investment round include Soros Fund…
PE Partnership Terms – Best Practices Callan, an institutional investment consulting firm, released its Private Equity Fees and Terms Study to serve as a reference for institutional investors. This study…
Clear, a blockchain startup founded in 2018, specializes in removing complex frictions in B2B trade. One such pain point is the contractual settlement and reconciliation for roaming agreements between telecom…
Alternative Investment: Vanguard Launches Its First ESG-focused Fixed Income Product For U.S. Investors
Vanguard filed a preliminary prospectus for a corporate bond ETF that will offer U.S. investors investment-grade corporate bonds with additional exposure to environmental, social, and governance (ESG) criteria. Vanguard has…