Cursive Simulation

August 31, 2015

This age of ballpoint and gel pens, and people tapping on keyboards and touchscreens to write, is far removed from the time when writing was done with dip pens. Dip penmanship was mentioned in the enjoyable 1986 animated movie, "Happy New Year, Charlie Brown,"[1] when Linus was scolding Charlie Brown for not having the fortitude to read Tolstoy's War and Peace.

"When Leo Tolstoy was writing War and Peace, his wife, Sonya, copied it for him seven times. And she did it by candlelight, and with a dip pen... And you're telling me you can't even read it once?"[2]

Dip pen writing is enabled by some elementary physical principles, as shown in the figure.

Surface tension pools the ink around the vent hole in a dip pen nib, and capillary action draws it down through the slit to the point. (Modified Wikimedia Commons image.)

I'm admitting my age when I recall that my desk in elementary school had an ink reservoir, and that I practiced penmanship with dip pens. Our penmanship lessons, which all students uniformly detested, taught the Palmer Method. The Wikipedia page on the Palmer Method states that its regimentation was believed to increase "discipline and character, and could even reform delinquents." My wife once saw our penmanship teacher in a bank actually writing using that method.

My handwriting has always been abominable, so I was was an early adopter of computer word processing, using the first version of WordStar on an S-100 computer running CP/M in the very early 1980s. Shortly thereafter, Microsoft Word became ubiquitous in the corporate world, as I migrated over the years from StarOffice, to OpenOffice, to LibreOffice.

Computers are now quite adept at handwriting recognition. My venerable Palm Zire 31 has been doing this for more than a decade using a slightly contrived character set called Graffiti. Computer science has advanced handwriting recognition beyond that stage to making sense of arbitrary script through innovations such as recurrent neural networks, as schematically illustrated in the figure. I wrote about neural networks in the context of solving the knight's tour on a chess board in a previous article (The Knight's Tour, March 7, 2014).

Two-layer artificial neural networks. The input states are at the left, and the connections (arrows) are differently weighted. The left is a standard, feedforward connected network, while the right shows recurrent connections in the first layer. The recurrent data is delayed by a time δt. (Created using Inkscape.)

If you're interested in the inverse problem of producing what looks like handwriting from a text file, there's one easy option; namely, using a font that simulates handwriting. There are a number of these script typefaces, one of the most elegant being Mistral. Alex Graves, a Canadian Institute for Advanced Research Junior Fellow at the Department of Computer Science of the University of Toronto, took a more elegant approach, as described in an arXiv paper.[3-5]

Graves' particular approach uses what's termed a "Long Short-Term Memory" recurrent neural network. As if the name isn't confusing enough, look at the basic network node, as shown in the figure. This is definitely not the simple neural network of a decade ago. The network operates by predicting one data point at a time.[3]

One node of a "Long Short-Term Memory" recurrent neural network, as described in ref. 3. This type of network has been shown to generate an excellent simulation of handwriting. (Fig. 2 of ref. 3, via arXiv.[3])

The neural network required training data obtained "online;" that is, the (x,y) pen positions were obtained as the writing was taking place. The alternative "offline" data would just be images of handwriting.[3] The training data were taken from the IAM online handwriting database (IAM-OnDB), as described in ref. 6.[6] These data were obtained by having people write samples from a standard corpus (the Lancaster-Oslo-Bergen text corpus) on a "smart whiteboard."

Simulations can be as good, or as bad, as desired, depending on the amount of computing power used. Graves' network has a bias factor that allows generation of handwriting in various degrees of perfection, perfect handwriting being the average script of the multitude of writers. At low perfection level, the same letter is written in a slightly different manner at different parts of the script. The same line, rendered under different initial conditions, is shown in the figure.[4]

Handwriting samples generated by a neural network. (From the online generating program in ref. 4.[4])

References:

1. Happy New Year, Charlie Brown, TV Movie (1986, Sam Jaimes, Bill Melendez, Directors) on the Inrenet Movie Database.
2. Quotes from "Happy New Year, Charlie Brown," TV Movie, 1986, on the Internet Movie Database.
3. Alex Graves, "Generating Sequences With Recurrent Neural Networks," arXiv, June 5, 2014.
4. Online Handwriting Application, University of Toronto Computer Science Web Site.
5. Matthew Braga, "This Post Was Handwritten by a Neural Network," motherboard.vice.com, July 23, 2015.
6. M. Liwicki and H. Bunke, "IAM-OnDB - an on-line English sentence database acquired from handwritten text on a whiteboard," Proc. 8th Int. Conf. on Document Analysis and Recognition, vol. 2 (2005), pp. 956-961 (PDF File).