The wand is a handheld that types what you write in the air: hold it like a TV remote, draw a letter, and the letter appears on the screen. A small neural network inside it reads the motion of your hand. This is what that network sees, shown with real recordings from the people who taught it.
The bird app
Let your phone listen to a hedge and Merlin will tell you which bird is singing. Cornell’s team built it by turning each recording into a picture — a spectrogram, time running left to right, pitch bottom to top — and training a convolutional neural network to recognise the songs in the picture, the way a photo app recognises faces. Their own write-up is a good read.
I first ran into that idea in San Francisco in 2011. I was working on audio; Simon Osindero, who lived close by in the Mission, was building neural networks that recognised pictures. Spectrograms are pictures, so in principle his nets could read sound — which is what Merlin does today. Simon has since gone on to Google DeepMind. The wand is the same idea pointed at motion instead of sound.
Three words, then the pictures
Edge. The network runs on the chip inside the wand, 25 to 28 milliseconds per letter; nothing goes to a data centre, and nothing you write leaves your hand unless you send it.
CNN. A convolutional neural network. If you have ever touched an equaliser you have used a filter: a small pattern slid along a signal, reporting how strongly the signal matches it at each moment. A convolutional network is a stack of such filters whose patterns are learned from examples rather than designed. In the textbook picture, the first layer learns small things — a tug downward, a push to the right; the next combines them into a straight line or a curve, the one after that into “a line, then a turn”; a few layers up, that is a letter.
LOSO. Leave one subject out. To find out how well the wand reads a stranger, you train the network on everyone except one person, test it on that person, and repeat for everyone. Every figure for strangers below was measured that way: the network that answered had never seen the writer.
Watch it read
A real recording from our paid study, played at the hand’s real speed — nothing staged. Left: the letter, as the hand draws it. Middle: the actual network that reads it, lighting up stage by stage. Right: its 38 possible answers, each brightening with how sure the network is — watch them flicker while the stroke is unfinished, then commit. It is a bit like watching a brain light up to a stimulus, except every light here is a real number from the real recogniser.
The point isn’t that you can read the letter off the lights — you can’t, and honestly neither can we. The point is that it’s real: this is the network that serves the demo, run on a stranger’s actual air-writing, deciding in front of you. Draw an a and it flirts with circle and o while the hand is still coming round the loop, then lands on a once the stem arrives. Draw b and it commits to the downstroke first. Spell out wand and the answer changes hands, letter by letter. The shapes it confuses are the ones you’d expect — an a with a d, a q, a 9: a loop and a stem, every time.
Who it reads
A writer under 25
23 writers aged 18–24 · 3,312 attempts
A writer over 40
215 writers aged 40 and over
Same eight letters, drawn at each writer’s real size. Both writers sit at their age group’s typical score; the network reading them had never seen either. Held-out, 48 prompted shapes, 302 writers with an age on file.
The network reads the over-40s better than the under-25s — about 70% of characters against 55% — and the gap is there again on a second run with a later model. One in five under-25s wrote micro-letters: under 20 centimetres of hand travel per letter, a letter a few centimetres across, where most people draw one the size of a dinner plate. A network taught on dinner plates reads those about 8% of the time. Twenty-three people is a lead, not a verdict; but the hand that grew up on a glass screen does seem to make small marks, with more stops and starts, and the machine has to learn that too. The full analysis is in the Air-Writing Atlas.
Nine things to take away
Hold it like a TV remote; draw; the letter lands on the screen.
The AI runs on the chip in your hand, in under 30 ms. Nothing leaves the device unless you send it.
% of words exactly right after it has learned your hand — 28 people, re-tested on a later day.
It reads motion. Privacy as physics: it works in the dark and in a pocket, and it has no field of view.
A convolutional network: a stack of learned filters, an audio engineer’s filter bank that taught itself — the same family of network Merlin uses.
paid strangers whose 52,191 air-written letters taught it, with age, sex and handedness attached.
TV, phone, laptop — nothing to install.
Leave one subject out: train on everyone but you, test on you. The stranger’s-eye score.
characters it types: the alphabet, the digits, and a couple of strokes for space and delete.
Fancy a go? Your phone can stand in for the wand: phone-demo.the-wand.ai/write (the browser demo does its reading on our server; the wand does it on the chip).
— John
How this was made
The recordings are from our paid Prolific study (adults who consented to their phone’s motion data being recorded for research; aggregates and anonymous examples only). The lights are the real thing: the actual per-stage activations of the recogniser that serves the demo (a selection of its internal features per stage, each normalised so a typical response reads as bright — a partial view, not the whole network), and the 38 answers on the right are its real output, recomputed on the gesture so far at each moment, which is why they shift as the stroke unfolds. The recogniser reads 38 characters — the alphabet, the digits, and a couple of strokes for space and delete; the study itself prompted 48 shapes. The accuracy figures elsewhere on this page are measured differently — leave-one-subject-out, the network scored on writers it never trained on — over the 2026-06-13 run of 44,709 held-out recordings, on a corpus since grown to the Atlas’s 349 writers and 52,191 letters. The two cohort figures are cold-start character accuracy on the 302 writers with an age on file. The 95% is a personalised figure, kept apart from the cold ones: the pooled word accuracy of 28 writers after a short personalisation pass, re-tested on a later day, on phrases assembled from each writer’s recorded letters rather than typed live (the twelve the network found hard went from 82 to 93.5%; the sixteen it already read well from 95.7 to 96.5%).