This is gonna accidentally activate some old forgotten MKULTRA code-key in some poor fella’s amygdala and bang!, they’ve Manchurian Candidated Sonic the Hedgehog

The first time I saw it, I thought about playing a drumkit and whistling a bassline at the same time, with someone else on guitar and vocals, making a full rock sound with two people. But then I was too lazy to try it.

There been some people doing the whole "one man band" thing with a back pack drum, guitar and harmonica. I bet someone crazy enough could do one instrument per limb + harmonica and maybe even a theremin with some remaining appendage too.

I think looking cool matters a lot in live music, maybe even more than the actual music. Blowing into a harmonica while carrying a backpack full of cutlery looks really dorky. But two people with drums+whistling and guitar+vocals can be made to look cool.

Laurie Anderson, with some of her bespoke live MIDI instruments (no actual cutlery AFAIK), got pretty close.

This is why I love playing for contra dancing: no one cares what the musicians look like.

Sounds a bit like imitone https://imitone.com/ (I have no affiliation)

FYI, Imitone has a patent, I don't know what it covers or claims though.

The relevant patent is from 1986, long expired: https://www.jefftk.com/p/ugo-contis-whistle-controlled-synth...

In case you haven't seen it, Imitone's patent is US20170098112A1 (https://patents.google.com/patent/US20170098112A1/en). This one's from 2017 and is still active, and it's for a periodicity analysis system. Hopefully it's not directly relevant to your synth.

That patent is about a particular way of implementing a low-latency pitch tracking algorithm. My tool uses a different pitch tracking algorithm based on zero crossings. It's much simpler and far from novel.

A big difference is that Imitone works well with singing, whereas my algorithm only does a good job with whistling. On the other hand, my algorithm is lower latency when given a whistle as input.

Very cool idea. It would be nice if you could enable your camera and then control mod/pitch with head up/down left/right motions.

Reminds me of this thing where I was trying to make a three-formant speech synthesizer, like a really simplified DECtalk, track my own voice: https://x.com/me_irl/status/2075472573627093145

This is something I think about making every time I sit in front of my PC - thank you and I look forward to trying it!

How feasible would it be to add MIDI out support to this?

I often find my improvised melodies are more interesting when I whistle them, so this would be a handy feature.

I wouldn't expect MIDI to be a good fit, and when I experimented with it a few years ago I wasn't able to get anything I liked. Pitch and volume are continuous, and while this can be encoded in MIDI it's awkward. If you send too many updates you can overwhelm the synth, and if you slide a single note too far it will re-trigger the note. Also when the sound starts you know "there's whistling" before you have a great estimate of the pitch, but MIDI wants to know the initial tone (and many voices have tuned onsets).

But if you want to try it, checking out https://github.com/jeffkaufman/whistle-synth and asking Claude or Codex to add MIDI support would probably work!

there's the artinoise re.corder which is blown and fingered but only generates MIDI (via bluetooth).

Fingering helps a lot: it means you get a very reliable indication of which note is intended, and there are no multi-step slides.

It seems like it will be fun but my headphones just come in under the activation threshold, but I can't see a way to add some input gain

Try setting the gate to a higher number?