Tikalon Blog is now in archive mode.
An easily printed and saved version of this article, and a link
to a directory of all articles, can be found below: |
This article |
Directory of all articles |
Odd Neural Networks
July 25, 2022
One of the objects of
physics research is to render the
invisible,
visible Occasionally, this takes some very large and very
expensive equipment, such as the 27
kilometer (17
mile)
circumference Large Hadron Collider and the $1.4 billion
Atacama Large Millimeter Array, the most expensive
ground-based telescope in operation.
Laboratory physicists also need to see the invisible, and
electron microscopes are one obvious equipment example. A lesser known, but more ubiquitous equipment example, is the
oscilloscope, a means to
visualize electrical signals.
The essential
component of an oscilloscope is the
cathode-ray tube (CRT),
invented in 1897 by
German physicist and
Nobel Physics Laureate,
Karl Ferdinand Braun (1850-1918). The CRT graduated from a
laboratory curiosity to a useful
laboratory instrument when
American electronic engineer,
Allen DuMont (1901-1965), began selling a laboratory oscilloscope, called the
DuMont Oscillograph, in 1934.
Allen B. Dumont Laboratories Model 208-B Oscillograph.
I was gifted one of these by my undergraduate alma mater as a bonus for acting as a laboratory teaching assistant after these oscilloscopes had been replaced by newer items.
I used it for more than two decades thereafter in my home workshop, and it was still operational when it was discarded as electronic waste in a cleaning frenzy. It would have been nice to have kept it for historical reasons; but, alas, my home has limited space.
(Exhibit at the Putnam Gallery of Harvard University (Cambridge, Massachusetts), a Wikimedia Commons image by Daderot.)
While I used an oscilloscope to visualize signals in the
design of
audio equipment, such as
electronic music synthesizers, the oscilloscope wasn't the first way that this was done. German physicist and
musician,
Ernst Chladni (1756-1827), found that
sand sprinkled on
vibrating plates will accumulate in certain areas, forming what are now known as
Chladni figures. These acoustic
nodes are regions at which vibrations are minimal. While Chladni used a
violin bow to excite vibrations in these plates, modern
technology allows
excitation at various
frequencies with a
loudspeaker or similar
transducer (see figure)
Left, a drawing showing excitation of vibrations in a Chladni plate using a violin bow. Other images show excitation of plates by an audio transducer at various frequencies. (Left image, a Wikimedia Commons image, figure 12 of page 26 from William Henry Stone, "Elementary Lessons on Sound," Macmillan and Co. (London, 1879). Second, third, and fourth images by Elmar Bergeler, also from Wikimedia Commons. Click for larger image.)
I referenced Chladni's research in my 2012
science fiction novel,
Mother Wode.[1]
After the quantum roadblock, physicists had to devise physical shortcuts to achieve rapid computation of important functions. As usual, they were helped by some very old research. Ernest Chladni, a French physicist at the time of Napoleon, had discovered the strange acoustical modes of vibration in irregular plates. He sprinkled sand on thin metal plates, and when they were excited by a violin bow, the sand would settle into a pattern in places where the amplitude of the vibration was small. After a while, the mathematicians got into the act, and they were able to analytically describe how these vibration modes related to the irregular boundaries of plates. In effect, you could simulate a mathematical function by a properly shaped plate. When nanotechnologists were able to make very small plates that would vibrate at high frequencies, Chladni Computation, the rebirth of analog computers, was born. It was most useful for signal analysis and filtering of data streams.
An
open access paper published early this year by physicists,
computer scientists and
engineers at
Cornell University (Ithaca, New York) and the
NTT Physics and Informatics Laboratories (NTT Research, Inc., Sunnyvale, California) used the acoustic vibration of a Chladni plate in an unusual application - They made an acoustical system that functions like a
deep neural network that is somewhat successful at recognizing the
digits 0-9.[2-5] They also explored other unconventional
neural networks based on
broadband optical pulse propagation in
quadratic nonlinear media (
second-harmonic generating materials), and a
transistor circuit with a
noisy,
nonlinear transient response.[2-4]
Something this
strange is not easily done. The research team needed to develop a way to
train networks whose
physical layers lack any mathematical
isomorphism to conventional artificial neural network layers.[2] In the acoustic plate system, a 3.2
centimeter by 3.2
centimeter titanium plate of one
millimeter thickness was connected at its center by a long
bolt to the
voice coil of a
commercial high-fidelity loudspeaker with its
diaphragm removed.[2] The loudspeaker was driven by an
audio amplifier, and the resultant plate
oscillations were
digitally recorded using a
microphone (see figure).[2]
The vibrating plate digit decoder (left), and a schematic diagram of the system operation (right). As you can see, the loudspeaker is sufficiently sized to adequately vibrate the plate with time-dependent forces that encode the input image data and parameters. (Figs. 4a and 4b of ref. 2, licensed under the Creative Commons Attribution 4.0 International License.[2] Click for larger image.)
The plate system was trained using images from the MNIST
handwritten digit classification task dataset.[6-7] The titanium plate system was selected because it blends the source and selection vibrations in a nonlinear
convoluted way.[4] This odd neural network operates by combining the
pixel data of the input image of a handwritten digit with another representing the
synaptic weights of the digits 0-9.[4]
As study
coauthor and
assistant professor of
Applied and Engineering Physics at Cornell University,
Peter McMahon, is quoted in
Quanta Magazine, "Many physical systems can naturally do some computation way more
efficiently or faster than a
computer can."[5] It seems apparent that deep neural networks act as
approximations of
smooth mathematical functions with a larger number of layers giving better results because their approximation to the function is closer.[4]
Such physical realizations of neural networks have the inherent problem that their training must be done by digital means.[4] The usual
backpropagation method doesn't work, since the physical systems
don't run in reverse. As a consequence, training is done using a
physical model of the system, and not the system itself.[4] Such training allowed the plate system to classify handwritten digits correctly 87% of the time.[4] The researchers conclude that such physical networks would serve as an adjunct to conventional general-purpose hardware and not a replacement.[2]
References:
- Dev Gualtieri, "Mother Wode," Tikalon LLC (May 25, 2012), 290 pp., ISBN:978-0985332501 (via Amazon).
- Logan G. Wright, Tatsuhiro Onodera, Martin M. Stein, Tianyu Wang, Darren T. Schachter, Zoey Hu, and Peter L. McMahon, "Deep physical neural networks trained with backpropagation," Nature, vol. 601 (January 26, 2022), pp. 549-555. https://doi.org/10.1038/s41586-021-04223-6. This is an open access publication with a PDF file here.
- Supplementary Information for ref. 2.
- Logan G. Wright, Tatsuhiro Onodera, Martin M. Stein, Tianyu Wang, Darren T. Schachter, Zoey Hu, and Peter L. McMahon, "Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems," arXiv, April 27, 2021.
- Charlie Wood, "How to Make the Universe Think for Us," Quanta Magazine, May 31, 2022.
- Patrick Grother and Kayee Hanaoka, "Handprinted Forms and Characters, 2nd Edition," National Institute of Standards and Technology Special Database 19, September, 2016.
- Yann LeCun, Corinna Cortes and Christopher J.C. Burges, THE MNIST DATABASE of handwritten digits.
Linked Keywords: Physics; research; invisibility; invisible; naked eye; visible; cost; expensive; scientific instrument; equipment; kilometer; mile; circumference; Large Hadron Collider; Atacama Large Millimeter Array; ground-based telescope; Laboratory; physicist; electron microscope; oscilloscope; data and information visualization; visualize; signal (lectronics); electrical signal; electronic component; cathode-ray tube (CRT); invention; invented; Germany; German; Nobel Prize in Physics; Nobel Physics Laureate; Karl Ferdinand Braun (1850-1918); history of the oscilloscope; laboratory curiosity; laboratory equipment; laboratory instrument; America; American; electronic engineering; electronic engineer; Allen DuMont (1901-1965); DuMont Oscillograph; Allen B. Dumont Laboratories; gift; undergraduate; alma mater; bonus; laboratory; teaching assistant; oscilloscope; decade; home workshop; electronic waste; putzfimmel; cleaning frenzy; history; historical; Harvard University (Cambridge, Massachusetts); Wikimedia Commons; Daderot; design; audio frequency; electronic music; electronic music synthesizer; musician; Ernst Chladni (1756-1827); sand; sprinkle; sprinkled; vibration; vibrating; plane (geometry); plate; Chladni figures; node (physics); violin bow; technology; excitation; frequency; frequencies; loudspeaker; transducer; drawing; Chladni plate; Elmar Bergeler; science fiction; novel; quantum mechanics; roadblock; shortcut; computation; function (mathematics); Napoleon; metal; amplitude; mathematician; mathematical analysis; analytically; simulation; simulate; nanotechnology; nanotechnologist; analog computer; signal processing; signal analysis; electronic filter; filtering; >data stream; open access journal; open access paper; scientific literature; published; computer scientist; computer engineering; computer engineer; Cornell University (Ithaca, New York); NTT Physics and Informatics Laboratories (NTT Research, Inc., Sunnyvale, California); deep neural network; numerical digit; artificial neural network; ultrashort pulse; broadband optical pulse; wave propagation; quadratic function; nonlinear optics; nonlinear media; second-harmonic generation; material; transistor; electronic circuit; noise (electronics); noisy; linearity; nonlinear; transient response; strange; train; physical layer; isomorphism; centimeter; titanium; millimeter; bolt (fastener); voice coil; commerce; commercial; high-fidelity; loudspeaker; diaphragm (acoustics); audio amplifier; oscillation; digital recording; digitally recorded; microphone; vibration; vibrating; plane (geometry); plate; numerical digit; schematic diagram; system; force; image; data; parameter; Creative Commons Attribution 4.0 International License; handwriting; handwritten; convolution; convoluted; pixel; synapse; synaptic; author; coauthor; assistant professor; Applied and Engineering Physics at Cornell University; Peter McMahon; Quanta Magazine; efficiency; efficiently; computer; approximation; smoothness; smooth; backpropagation; irreversible process; doesn't run in reverse; physical model; Dev Gualtieri, "Mother Wode," Tikalon LLC (May 25, 2012), 290 pp., ISBN:978-0985332501.