THE FOUR QUESTIONS PEOPLE ACTUALLY ASK
Can I record what I play? Not inside the app. There is no record button. Use your operating system screen recorder - the method is below.
Is there a guitar sound? No. There are five: Warm, Bright, Retro Synth, Strings and Horns. Nothing plucked.
How do I play an actual song? Set the key first, then play the scale degrees with your left hand. Full method below.
Why does it sound like noise? You are moving both hands at once. Freeze the right one until the left makes sense.
Start here: stop moving your right hand
Almost everyone opens Gesture Synth, waves both hands at the camera, hears something chaotic and concludes it is a toy that makes random sounds. It is not. It is a two-handed instrument where each hand has a completely separate job, and until you separate them you cannot hear what either is doing.
Your left hand decides which chord. Your right hand decides how that chord sounds. That is the whole design.
So for the first five minutes: hold your right hand still and flat in the middle of the frame, and play only with your left. Once the chord changes start sounding deliberate, bring the right hand in. There is also a Left Hand Only mode in the settings, which is the cleanest way to learn - switch to it, learn the chords, then switch back to Two-Handed.
The left hand: picking chords
Two things happen at once with your left hand. The number of fingers you hold up chooses the scale degree, and the tilt chooses major or minor.
| Left hand | What you get |
|---|
| 1 finger | Degree I - the home chord |
| 2 fingers | Degree II |
| 3 fingers | Degree III |
| 4 fingers | Degree IV |
| 5 fingers | Degree V |
| Index + pinky | Degree VI |
| Index + pinky + thumb | Degree VII |
| Tilt inward | Major |
| Tilt outward | Minor |
Vertical position shifts pitch by a semitone - top half raises, bottom fifth lowers, middle is normal. This does nothing until you enable it. There are separate Raise and Lower toggles in the settings, and if you have not switched one on, moving your hand up and down changes nothing. That catches a lot of people.
The right hand: shaping it
| Right hand | What it changes |
|---|
| Finger count | Chord quality - root position, 1st inversion, major/minor 7th, dominant/diminished 7th |
| Thumb tucked in | Octave up |
| Thumb out | Octave down |
| Tilt inward | More filter - darker, muffled |
| Tilt outward | Less filter - brighter, open |
| Raise the whole hand | Louder |
| Lower the whole hand | Softer |
If you looked in the settings panel and saw Volume and Filter marked Fixed and assumed they were broken - they are not. They say Fixed because your right hand controls them instead of a slider.
The five sounds, and the one that is not there
Warm, Bright, Retro Synth, Strings, Horns. That is the complete list.
People search for a guitar sound on Gesture Synth often enough that it is worth answering directly: there is no guitar, and there is nothing plucked or percussive at all. Every sound here is sustained - it holds as long as your hand holds the shape. That is a design decision, not a missing feature. Hand tracking gives continuous position, not discrete triggers, so the instrument is built around sounds that breathe rather than sounds that are struck.
Which to use:
- Warm - the default and the most forgiving. Start here. Mistakes sound intentional.
- Bright - cuts through, good when you want to hear exactly which chord you landed on while learning.
- Retro Synth - the most characterful and the least forgiving; wrong notes are obvious.
- Strings - slow attack, so it lags slightly behind your hand. Best for slow chord changes.
- Horns - loudest and most present. Use it with the filter tilted inward or it dominates.
Setting the key before you play anything
There is a key selector covering all twelve: A through G#. This matters more than it looks, because the finger-count mapping is relative, not absolute. One finger is always degree I, but which actual chord that is depends entirely on the key you picked.
So the order of operations for playing any piece of music is:
- Find out what key the song is in
- Set that key in Gesture Synth
- Translate the song chords into scale degrees - I, IV, V and so on
- Play those degrees as finger counts
Step 3 is the one people skip. You are not looking for chord names here, you are looking for numbers. If a song in the key of C goes C to G to A minor to F, that is I - V - VI - IV: one finger, five fingers, index-plus-pinky tilted outward, four fingers. Any chord chart that gives Roman numerals, or the Nashville number system, maps straight onto your hand with no translation at all.
A very large amount of popular music sits on I, IV, V and VI. If you can hit those four shapes cleanly you can play along with a great deal of it.
On playing a specific song
One of the more surprising things in our search data is people looking for how to play Duvet on Gesture Synth - the Bôa track that opens Serial Experiments Lain and has had a long second life on social video. We are not going to transcribe it, or any other copyrighted song, here.
The method above is the answer regardless of the song: find a chord chart for it, convert the chords to scale degrees in its key, set that key, play the numbers. That works for anything, and it is a more useful skill than one memorised sequence.
Recording what you play
Gesture Synth has no record or export function. No save, no download, no share link. Whatever you play exists only while you are playing it. For an instrument that went viral on social video, that is the single most common complaint.
So you capture it from outside the app:
- macOS - Shift+Cmd+5, choose a recording area, and in Options make sure an audio input is selected or you will record silence. To capture the browser audio itself rather than your room, you need a loopback audio device.
- Windows - Win+Alt+R opens the Game Bar recorder, which captures the active window with its audio. It will not record the desktop, only an app window, which is fine here.
- iPhone - add Screen Recording to Control Centre. Long-press the button and enable Microphone if you also want to narrate; the app audio is captured either way. Remember Silent Mode kills the app audio entirely.
- Android - the built-in screen recorder in Quick Settings, with Media sounds selected as the audio source.
Two practical notes. Record in landscape if you are on a phone - the hand tracking has more room and the video is more usable. And start recording before you raise your hands, because the first thing the camera sees is the app finding your hand, which takes a second.
Things that will make it sound worse
- Backlighting. A window behind you turns your hand into a silhouette and the tracker loses the finger positions. Face the light.
- Hands too close. It wants the whole hand including the wrist. Roughly an arm length back.
- Changing chord and shaping at the same time. Land the left hand first, then move the right. Simultaneous movement is what produces the noise everyone complains about.
- Strings for fast changes. Its slow attack means fast passages turn to mush. Use Bright or Retro Synth when moving quickly.
- Half-shapes. A finger that is only partly extended reads as ambiguous and the chord flickers. Commit to the shape.
Is this what a gesture synthesizer means generally?
Mostly, people searching for a hand gesture synthesizer or a webcam musical instrument land on this one, because it is by far the best known. The broader category is older than it looks - the theremin has been played by moving hands near an antenna since 1920, and the idea of continuous hand position controlling pitch and volume is directly inherited from it.
What is new is that this runs in a browser tab with no hardware, using a computer vision model to locate your finger joints many times a second. There is no language model and nothing generative involved - the mapping from your fingers to a chord is a fixed rule set written by a person, exactly as laid out in the tables above. The machine learning is in seeing your hand, not in deciding the music. You make all the musical decisions.
FAQ
How do I record on Gesture Synth?
You cannot from inside it - there is no record button, no export and no save. Use your operating system screen recorder instead: Shift+Cmd+5 on macOS, Win+Alt+R on Windows, Screen Recording in Control Centre on iPhone, or the Quick Settings recorder on Android. Check your audio source is set or you will capture a silent video.
Is there a guitar sound on Gesture Synth?
No. The five sounds are Warm, Bright, Retro Synth, Strings and Horns. Nothing plucked or percussive, because every sound sustains while you hold the hand shape rather than triggering once.
What is the Warm synth?
The default sound, and the most forgiving of the five. It has a soft attack and a rounded tone, which means imprecise hand positions still sound musical. Start on Warm and move to Bright once you want to hear your chords clearly.
Can I play a specific song on it?
Yes, with one conversion step. Find the song chords, work out its key, set that key in the app, then translate each chord into a scale degree - I through VII - and play those as finger counts. Chord charts written in Roman numerals or the Nashville number system map onto your hand directly.
Why does it sound random?
You are almost certainly moving both hands at once. The left hand picks the chord and the right shapes it; moving both together changes what and how simultaneously. Hold the right hand still, or switch to Left Hand Only mode, until the chords sound deliberate.
Why does moving my hand up and down do nothing?
Pitch shift is off by default. There are separate Raise and Lower toggles in the settings, and until one is enabled, vertical position has no effect on the left hand.