I’m going to get pwned one of these days, considering my threshold for giving a random website access to my webcam is apparently mild curiosity and entertainment for 30s.
But more generally, if I’m the sort of person who would allow this I’d probably allow some other stupid permissions if they promised me something novel on the other side.
"An overdriven webcam sensor reconfigures its pixels into an AI-steered optical phased array, discharging the laptop's total electrical power as a lethal, coherent laser pulse the instant your hand moves."
Then it rendered the page load with '<html><h1>"You have been theremin ated".'
This is extremely cool. I activated the camera and it recognized the position of my hands and I was able to play it as shown in the instructions. It's very fun.
Although its still kind of working in roughly the same manner as the original theramin - just using a different frequency of electromagnetic radiation.
Ermm, theremin came first, so actually the air guitar is a theremin-version of a guitar.
So actually this project should be called “Cameramin”, since it has very little do to with air guitar, other than air guitar being derived from the theremin ..
Really cool. I remember one local musician here in Lithuania used a theremin like 20 years ago and i was amazed at the interactivity and the alien sound of it. Now sitting in front of my computer doing the same that many years later brings memories back!
There's a button next to "gyro" labeled "hands." Click on it, and the browser will ask for the camera permission. For me (Chrome, mac), I had to click on the "cal" button next to it the first time to actually switch to the camera.
The camera watches your palms. Move them apart and it gets louder. Bring them together and it goes quiet. Raise them both and the note goes up. There is a sound wave strung between your hands, and it slackens as they close.
Two palms give you four numbers, and one of them is best left alone. Their mean height is the pitch. The distance between them is the volume. The angle of the line joining them sat unused for months and is now the vibrato: see-saw your hands and the note begins to sing. The fourth is where the pair sits horizontally, and it stays free on purpose, so that nobody is pinned to one spot in front of the camera. Their apparent size is a fifth thing and not a position at all; it moves a lowpass and a reverb send, so leaning back makes the room bigger. The angle is independent of the distance, so the vibrato leaves the volume alone, and the string drawn between the hands is already a picture of it.
It began as a phone toy — tilt left and right for volume, forward and back for pitch. The camera came later and turned out to be the better instrument. The phone mode is still there.
No video leaves your machine. The hand tracking is MediaPipe's Hand Landmarker compiled to WASM, running on your own GPU. The tracker and its model are fetched once from a CDN, about 8 MB, and after that you can unplug the network and it still plays.
Most of the work was not the sound. It was the hand that disappears. MediaPipe finds a palm by its plane, so it loses one when you turn your wrist edge-on, or when the room is dark and the webcam falls to fifteen frames a second and a moving hand becomes a smear. At first I held the note for a second after a hand vanished. People said it lagged. I cut it to half a second. People said the sound kept dropping out. Both were right. Now the hold depends on where the hand was last seen: near the edge of the frame a person has genuinely taken their hand away, so 400 ms; deep in frame nobody teleports and the detector merely blinked, so a full second, and no one hears the gap.
The other stubborn thing was the bass. A phone speaker produces nothing below roughly 150 Hz, and raising the gain there does nothing at all — I tried ten decibels and heard silence. So the bottom octave is split off, saturated gently to breed harmonics, and its fundamental is then filtered away. The ear reconstructs a pitch it never heard.
That trick then caused a bug I misread for a week. A tester said the pitch moved in steps in the phone mode, worst in the bass, and guessed it was a frame rate problem. It was not. The phone reports its tilt angle quantised to something close to a whole degree, and I had packed six octaves into eighty degrees of tilt — so one tick of the sensor was most of a semitone. It was audible in the bass specifically because of the exciter: at 70 Hz the ear resolves pitch poorly, but the exciter re-voices the note through harmonics at two and three hundred hertz, which is exactly where the ear resolves cents best. So the fix was not the sound and not the frame rate. It was a narrower range, a wider sweep, and reading the sensor at frame rate instead of once per event.
Nice work. "No video leaves your machine." Either Claude specifically or AI models in general use this phrasing now and No teeth are left un-gnashed as a result. Uggh I wonder why it's so aggravating.
My bad, I used Claude to create the description. The thing is, I'm really bad at presenting a project properly. But thanks for the criticism, I'll take it into account next time.
Best way to get better is practice, and practice is best without the assistant. Think weight lifting, rather than a forklift: the hard part is the point.
Obviously I appreciate this is hard to do when it's so tempting to get the easy convenient and free AI to do it for you, but the more practiced you are, the better you can do when you do need a metaphorical forklift.
I think the main objection is that hacker news supposedly doesn't want AI generated comments, which is going to paradoxically produce the situation where the mods will be overwhelmed with policing AI generated comments and they themselves will be replaced by AI who will then be incentivized to accuse human comments as being AI (because they're not team AI duh) and then there will be nothing but bots here.
But more generally, if I’m the sort of person who would allow this I’d probably allow some other stupid permissions if they promised me something novel on the other side.
Then it rendered the page load with '<html><h1>"You have been theremin ated".'
Perhaps a "high frequency theramin"?
Edit: Cameramin, obviously, d’oh.
https://youtu.be/DnAfXK-hft8
So actually this project should be called “Cameramin”, since it has very little do to with air guitar, other than air guitar being derived from the theremin ..
Seriously though: nicely done, UX here is better than what I came up with.
Call it Cameramin and you’ve got a customer. ;)
Two palms give you four numbers, and one of them is best left alone. Their mean height is the pitch. The distance between them is the volume. The angle of the line joining them sat unused for months and is now the vibrato: see-saw your hands and the note begins to sing. The fourth is where the pair sits horizontally, and it stays free on purpose, so that nobody is pinned to one spot in front of the camera. Their apparent size is a fifth thing and not a position at all; it moves a lowpass and a reverb send, so leaning back makes the room bigger. The angle is independent of the distance, so the vibrato leaves the volume alone, and the string drawn between the hands is already a picture of it.
It began as a phone toy — tilt left and right for volume, forward and back for pitch. The camera came later and turned out to be the better instrument. The phone mode is still there.
No video leaves your machine. The hand tracking is MediaPipe's Hand Landmarker compiled to WASM, running on your own GPU. The tracker and its model are fetched once from a CDN, about 8 MB, and after that you can unplug the network and it still plays.
Most of the work was not the sound. It was the hand that disappears. MediaPipe finds a palm by its plane, so it loses one when you turn your wrist edge-on, or when the room is dark and the webcam falls to fifteen frames a second and a moving hand becomes a smear. At first I held the note for a second after a hand vanished. People said it lagged. I cut it to half a second. People said the sound kept dropping out. Both were right. Now the hold depends on where the hand was last seen: near the edge of the frame a person has genuinely taken their hand away, so 400 ms; deep in frame nobody teleports and the detector merely blinked, so a full second, and no one hears the gap.
The other stubborn thing was the bass. A phone speaker produces nothing below roughly 150 Hz, and raising the gain there does nothing at all — I tried ten decibels and heard silence. So the bottom octave is split off, saturated gently to breed harmonics, and its fundamental is then filtered away. The ear reconstructs a pitch it never heard.
That trick then caused a bug I misread for a week. A tester said the pitch moved in steps in the phone mode, worst in the bass, and guessed it was a frame rate problem. It was not. The phone reports its tilt angle quantised to something close to a whole degree, and I had packed six octaves into eighty degrees of tilt — so one tick of the sensor was most of a semitone. It was audible in the bass specifically because of the exciter: at 70 Hz the ear resolves pitch poorly, but the exciter re-voices the note through harmonics at two and three hundred hertz, which is exactly where the ear resolves cents best. So the fix was not the sound and not the frame rate. It was a narrower range, a wider sweep, and reading the sensor at frame rate instead of once per event.
One HTML file, no build step, MIT: https://github.com/gurov/theremin
Obviously I appreciate this is hard to do when it's so tempting to get the easy convenient and free AI to do it for you, but the more practiced you are, the better you can do when you do need a metaphorical forklift.