Bitwig FFT Resynthesizer and Note Extractor
Tutorial | Oct 06, 2025
In this video, I demonstrate a custom-built FFT resynthesizer and note extractor inside Fitwick Studio, which uses pure sine oscillators to extract and recreate audio frequencies without traditional bandpass filters. The system allows for unique spectral manipulation, including stretching sounds, octave shifting, and removing partials outside a selected musical scale, making it perfect for ambient effects and creative sound design. I also show how this setup can extract MIDI notes from audio and explain the straightforward method behind building it, with presets available for free download.
You can watch the Video on Youtube
- support me on Patreon
- Download FFT - Re-Synth & Note Extractor
Short Overview #
Today, I introduced a new FFT Resynthesizer and Note Extractor that works differently from traditional bandpass filtering or vocoders. This tool uses sine oscillators to analyze and resynthesize audio, allowing for creative effects like spectral stretching, shimmer reverb, and note extraction based on musical scales. The entire process is musically oriented, focusing on notes rather than arbitrary frequency bins, making it especially useful for ambient music and experimental sound design. If you're interested, you can download the presets for free and try them out yourself.
- Introduction to a unique FFT Resynthesizer and Node Extractor built in Fitwick Studio, not based on traditional bandpass filtering
- Uses sine oscillators for each note instead of a fixed frequency grid, making it musically mapped across all notes
- Demonstrates real-time spectral resynthesis, extracting amplitude information at each note and recreating the sound with sine waves
- Features include cutoff for amplitude signal smoothing, octave shifting for creating spectral shimmer effects, and note delay for disperser-like flares
- Option to remove partials not in a chosen musical scale, keeping only harmonics that fit the selected key
- Explains the difference between traditional FFT bin mapping and this method’s musical note-based mapping
- Describes the technical process: multiplying sine and cosine oscillators with the audio, filtering and recombining to extract amplitude and (optionally) phase information
- Showcases a simple note extraction tool that converts audio input to MIDI notes, allowing played notes to trigger synthesizers
- Demonstrates practical applications: ambient sound design, spectral effects, and note extraction
- Patch and presets available for free download, with encouragement to experiment and offer feedback
Introduction #
Today I’m excited to share something truly innovative that I built inside Bitwig Studio. I’ve created my own FFT Resynthesizer and Node Extractor, which stands apart from traditional methods because it does not utilize bandpass filtering. Instead, my approach is more closely related to how a vocoder works, but without the typical bandpass filters. It’s entirely based on sine or partial oscillators. In this summary, I’ll walk you through what this tool does, how it sounds, its features, and the underlying concepts and techniques used.
Demonstration and Sound Capabilities #
Before diving into the technical details, I always like to showcase the results first, knowing that many people prefer to hear what a tool can do right away. For the demo, I used a recognizable audio track, the Rambo theme. My method involves playing a sine partial oscillator at every possible pitch or frequency, then combining it with the incoming audio to extract the amplitude envelope. This amplitude envelope is then used to drive a sine oscillator, creating a resynthesized version of the input audio.
Key Features and Controls #
Cutoff for Amplitude Smoothing #
Unlike traditional filters, the cutoff parameter here applies to the amplitude envelope rather than the audio itself. Lowering the cutoff creates a stretched, ambient sound by smoothing out rapid amplitude changes. Raising the cutoff yields a much more immediate and sometimes harsh or distorted response, acting almost as a spectral distortion device.
Octave Shifting #
Because I'm resynthesizing purely with sine waves driven by the extracted amplitude, I can easily shift the resulting sound by octaves. This is perfect for creative effects like shimmer reverb, allowing you to layer pitched-up versions of the audio with reverb for a lush, ambient texture.
Note Delay #
This feature staggers the onset of each sine partial, with higher partials delayed more than lower ones. It’s inspired by dispersive audio effects and produces characteristic spectral flares, which can be particularly interesting on basslines or other rhythmic material.
Scale Filtering #
Instead of traditional pitch correction, I implemented a "scale off" function. When enabled, only the harmonics that correspond to the selected musical scale are played back. Non-scale frequencies are simply removed, ensuring the resynthesized output fits musically without forcing unrelated frequencies to jump to the nearest scale note.
A Different Approach to FFT #
Traditional FFT techniques split the frequency spectrum into evenly spaced bins, which often do not align musically with standard notes or scales. Some frequencies land between notes, making the results less musically useful. My approach is different: each sine partial is aligned with a musical note, for example from C1 to C2, making for a more musically spread distribution.
Each note in the octave gets its own stack and corresponding sine oscillator, divided into sine and cosine components for further processing.
Technical Explanation and Building Blocks #
How Amplitude Extraction Works #
- Sine Oscillation and Multiplication: For each frequency of interest, I generate a sine oscillator at that frequency.
- Audio Input Multiplication: I multiply the incoming audio signal with the sine oscillator. This yields a new signal, which, after lowpass filtering, contains the amplitude information for the original audio at that exact frequency.
- Filtering and Envelope Following: By applying a steep lowpass filter at this stage, I am able to extract a smooth amplitude envelope for each frequency. The trade-off here is between response speed and clarity, stronger filtering slows things down but reduces noise and extraneous artifacts.
Sine and Cosine for Phase Extraction #
By also using a cosine oscillator (a sine shifted by 180 degrees) and multiplying both the sine and cosine with the input audio, I extract not only amplitude but also phase information for each frequency. Although my current patch does not use this phase data (to keep things simple), the possibility remains for future enhancements.
Amplitude Calculation #
After getting both sine and cosine values, I square them (to remove negative values), sum, and then take the square root. This process gives the amplitude magnitude for each frequency. By tuning the oscillator to a frequency present in the input, this method cleanly extracts its amplitude; if the frequency is absent, the output is near zero.
Scale Filtering Details #
Instead of forcibly correcting pitches, the scale filter simply mutes partials not included in the selected scale, preserving musical integrity without artifacting from pitch jumps.
Note Extraction #
The second main patch I developed is the Note Extractor. It attempts to detect and extract MIDI notes from an incoming audio signal, which you can then use to trigger synthesized sounds or external instruments. While this is a simple and not always perfectly accurate solution, it works well enough for creative use, particularly with less complex source material.
Practical Application #
Users can experiment with these patches for free by downloading them from my Patreon or the video description. The system is especially suited for ambient processing, spectral effects, shimmer reverbs, and experimental sound design. It’s not a commercial-grade, highly precise VST, but it’s an accessible and educational tool for those interested in spectral audio manipulation.
Conclusion #
This project demonstrates a novel approach to audio resynthesis and note extraction, leveraging the power of sine-wave-based spectral analysis rather than traditional FFT binning. It’s relatively simple to implement inside Bitwig's grid environment, and the results are both musically relevant and highly customizable. I encourage others to experiment, download the patches, and explore new sonic directions using these techniques.
Thank you for your interest, and stay tuned for further developments.
Full Video Transcription #
This is what im talking about in this video. The text is transcribed by Whisper, so it might not be perfect. If you find any mistakes, please let me know.
You can also click on the timestamps to jump to the right part of the video, which should be helpful.
Click to expand Transcription
[00:00:00] Hey folks, welcome back.
[00:00:01] Today I want to show you something that's never been done before inside of Fitwick Studio.
[00:00:06] I built my own FFT Resynthesizer and Node Extractor.
[00:00:12] So this is not based on bandpass filtering and it works kind of like on vocoder without
[00:00:18] bandpass filters.
[00:00:19] It's pure sine, partial or sine oscillator based and I'll show you in a minute how it
[00:00:25] works.
[00:00:26] First I want to show you how it sounds, because I know some people are impatient, they want
[00:00:30] to see results first, so let's do this.
[00:00:33] So here I have some audio tracks.
[00:00:55] Okay, so this is the Rambo theme of course and what I do is I play a sine partial for
[00:01:22] every possible pitch, for every possible frequency and then I combine it with the audio signal
[00:01:28] and then I can extract the amplitude and then I use this amplitude to play back a sine oscillator.
[00:01:35] And because it's playing back a sine oscillator we can do some tricks.
[00:01:39] First up we have a cutoff here and the cutoff is not for the audio signal, it's actually
[00:01:43] for the amplitude signal to smooth it out, right?
[00:01:46] So when we pull this down we get a stretched out kind of sound.
[00:01:57] So it's basically perfect for making ambient out of everything, because you stretch the
[00:02:03] sound out.
[00:02:05] If you push this up here you get much much faster response times, because the amplitude
[00:02:10] is changing faster, but it's very unpleasant to hear, but you can do it, maybe it gives
[00:02:22] you some interesting distortion, it's a spectral distortion device at this point probably.
[00:02:30] So yeah you can do this, then because we are playing back a sine partial to play back the
[00:02:34] audio amplitude we can just shift the octave of this, so it plays back one octifier.
[00:02:51] So this is basically perfect if you want to put the reverb on this and make a spectral
[00:02:58] shimmer reverb.
[00:03:22] So you can overlay the original audio with some pitched up version of itself and then
[00:03:28] put the reverb on it and you have a much fatter layer and it sounds kind of cool and can use
[00:03:34] it for ambient or some other interesting ideas or just here like I did as a shimmer, spectral
[00:03:40] shimmer reverb.
[00:03:43] So that's that, then we have here something like note delay, so when you have fast response
[00:03:47] times we can take each sine partial and delay it slightly longer than the partial before
[00:03:57] and it's more like a disperser working on audio, but here it's working on the amplitude
[00:04:03] or delayed amplitude.
[00:04:25] So it gives you this typical spectral flare, maybe you try it out on bass lines, maybe
[00:04:30] it's perfect for color bass, I have no idea, I haven't tried it out yet.
[00:04:34] So this is the note delay, then cut off and this one here is for smoothing, the amplitude
[00:04:41] is pretty self-explanatory and octave shift here, then we have scale off, so when you
[00:04:47] disable this, when it's off or when the scale off is off, then it removes all the partials
[00:04:55] that are not in the scale of what you are selected here with the global scale selector.
[00:05:01] So if you select a D sharp minor for instance, all the partials that are not in the scale
[00:05:08] detected in here are not played back.
[00:05:23] So it's not pitch correction, it's more like let's remove all the harmonics that are not
[00:05:30] in the scale and leave the rest untouched.
[00:05:32] So this is my idea for this here instead of correcting because I had some problems with
[00:05:39] certain partials moving then to the same note and it doesn't sound really really well in
[00:05:47] my opinion, so I leave just everything untouched that's not or that's in the scale and what's
[00:05:54] out of the scale is just removed.
[00:05:56] So leave this on if you want to have all partials, so by default this is also on.
[00:06:10] But sometimes it can help right when you play something on the piano and you want to have
[00:06:14] a sample playing on top, but it's not really in the scale, but it's also not really out
[00:06:19] of the scale.
[00:06:20] So here you can just remove everything that's not in the scale.
[00:06:24] So that's the resynthesizer.
[00:06:26] Also very unusual with this resynthesizer is that every partial is on a note.
[00:06:32] Usually with FFT is you take a frequency range, let's say 20 Hertz to 20 K and then you divide
[00:06:39] this up into equal parts, I don't know 1000 bins or whatever.
[00:06:46] And then sometimes some of these partials are on a note sometimes out of the out of
[00:06:52] the scale sometimes up between the notes and so on.
[00:06:55] So it's not musically spread out.
[00:06:58] That's what I want to say.
[00:06:59] So this one here is very musical spread out.
[00:07:02] I don't know if this is actually a problem.
[00:07:04] Maybe that's because why it sounds this way.
[00:07:07] But I basically put the sine partial on every note.
[00:07:11] So here I have the first octave C1 to C2 and then that's one patch.
[00:07:15] This patch has 12 voice stacks for every note in this octave.
[00:07:20] So C, C1, C#1, D1 and so on.
[00:07:26] And every note gets this patch.
[00:07:28] And this patch uses a sine partial here, which then is the split up into sine and cosine.
[00:07:36] So it's basically sine just phase shifted by 180 degree of phase inverted, phase flipped.
[00:07:45] And then we multiply this with the audio signal and we get the amplitude out of the audio
[00:07:50] signal of exactly this frequency.
[00:07:52] I show you this in a minute how it works.
[00:07:55] But this is basically here the patch.
[00:07:58] So this is the patch here to extract basically the frequency bin and then I use here a sine
[00:08:03] partial to play back what I extracted and add here the amplitude using then this here
[00:08:13] to play back the sine wave.
[00:08:15] And we can here change the octave and so on.
[00:08:17] You can also change here the sine partial for something else if you want to.
[00:08:22] So this is roughly how it works.
[00:08:26] Very simple actually.
[00:08:28] You can also extract the phase of the frequency here of this of this audio signal if you want
[00:08:35] to because we have the sine and the cosine and then you can you get an amplitude for
[00:08:41] the cosine, you get an amplitude for the sine.
[00:08:44] And then if you compare this to the audio signal, you can kind of triangulate where
[00:08:49] the position is of the of the sign in the audio signal.
[00:08:54] And then you get kind of a phase position out of this and then you can add it here also
[00:08:59] to the sine oscillator if you want to.
[00:09:02] But it needed to you know, I needed to add more modules here and I want to kept it simple
[00:09:07] and it also sounded very okay ish to my ears already without cloning the phase, but you
[00:09:14] can do it.
[00:09:15] And maybe I do also a patch here in the future for that.
[00:09:18] Anyway, I just want to show you this here.
[00:09:21] I show you in a minute how it's been built.
[00:09:23] But I want to focus now on another preset here, which is called FFT note extract.
[00:09:31] And it does exactly what what the name says, it's it tries to extract the notes from an
[00:09:38] audio signal.
[00:09:40] So here we have again, all your signals already here tries to find some notes and you can
[00:09:48] just use your synthesizer and play these notes back with the synthesizer.
[00:09:53] But we can also create another instrument track here, arm this and as an input we use
[00:10:00] audio one, which is this audio track here, right?
[00:10:04] So now we receive MIDI notes from this patch here, but we also need to add your of course
[00:10:13] an instrument, for instance, piano take eight.
[00:10:32] Again here, this is the cutoff works exactly like the cutoff in this preset.
[00:10:38] And yeah, if you push this up here, you get faster response times, but also a lot of
[00:10:43] more garbage.
[00:10:49] If you pull this down, you get more of the harmonics or of the longer drawn out notes.
[00:11:11] It works probably better here on this ambient track on this single audio track.
[00:11:36] So let's choose here different preset, then we can use on this one transpose, no transpose,
[00:11:44] push it up one octave, maybe use a quantizer so we can quantize this to the BPM here one
[00:11:50] on 10.
[00:11:51] It's probably not not the correct BPM for this track here, but anyway, maybe make this
[00:11:58] longer, put the reverb on that, just play it on top.
[00:12:24] Yeah, it's fun.
[00:12:40] It's not always detecting the right notes, of course, it's a very simple implementation,
[00:12:45] my amateurish kind of implementation, but it's fun.
[00:12:48] It kind of works, and I learned something, you probably also learned something.
[00:12:53] It's fun to play around with, and it doesn't cost anything can download this for free on
[00:12:59] my Patreon or on my in the description here.
[00:13:04] So why not do it, right?
[00:13:06] Of course, it's not that precise like a VST or any other plugin, but you get you get
[00:13:12] something out of it, and it's it's fun.
[00:13:15] Okay, so now that we have this, I show you how it works.
[00:13:21] And for that, we need an FX grid.
[00:13:25] This one here.
[00:13:27] And all we have to do, it's actually not very hard, it's very simple.
[00:13:32] If you think about it, we take a sine oscillator here, and you put this on a specific frequencies.
[00:13:38] We use a constant here, and pitch, pitch to Hertz, Hertz to pitch.
[00:13:47] We put this on a specific frequency, let's say one kilohertz, 1000 Hertz.
[00:13:54] And then we multiply this with the audio signal.
[00:14:01] We take this in the first input, and then the audio signal, we use the audio signal
[00:14:05] to amplify actually the sine oscillator.
[00:14:09] And then we get something out, let's use an oscilloscope, when we put audio in, of course.
[00:14:15] So use your test tone, test tone, I go to one kilohertz, and I use here bit of amplitude,
[00:14:29] and use your sine, okay.
[00:14:32] So we get something out, and if we change the frequency, you can see it's very wild,
[00:14:40] but when we go back to one kilohertz, actually it looks different, right?
[00:14:46] It looks a bit different.
[00:14:48] So with this knowledge, we can already use a filter, so we try to smooth the signal down.
[00:14:56] So we go into a filter here, maybe use a different color here, to see how long.
[00:15:06] Maybe use a low pass, maybe a very steep one, and we try to use the lowest frequency there
[00:15:11] is.
[00:15:12] At the moment, the lowest one is here, 19.4 Hertz, which is okay, I guess, for this experiment.
[00:15:23] But you can go lower, so you can take a constant here, go back to C3, it's the middle position,
[00:15:30] go in here and say minus one, then push this down.
[00:15:34] So now this is the lowest, I think this goes down to at least zero Hertz, so we can go up
[00:15:42] here to maybe six, or six.
[00:15:46] Let's try this first.
[00:15:47] You can see here, it's actually not that wild anymore as a signal compared to this one,
[00:15:52] maybe use a second oscillator here.
[00:15:55] So it's a different signal just by filtering this.
[00:16:00] So now we change the frequency, and you can see it goes back to zero.
[00:16:06] So we have no amplitude here, except when we go to one kilohertz, then we get something.
[00:16:15] So we can extract or can sense or detect that there is some kind of input on exactly this
[00:16:23] frequency.
[00:16:24] Here it's just one single test tone, of course, but when you have an audio signal, you can
[00:16:30] extract basically single frequencies just with this.
[00:16:34] In my patch, then, I used instead of just using here the sign, I'm also using a second
[00:16:42] sign oscillator here, but this one is offset by 180 degrees, so it's the cosine, basically.
[00:16:51] I can actually take an oscilloscope, it's very hard to see, but it's actually exactly
[00:16:59] the opposite.
[00:17:00] So it fills in the gaps, so when this first oscillator goes down, this one goes up and
[00:17:06] so on.
[00:17:07] So we have the sine and the cosine, but instead of using two oscillators here and one is offset,
[00:17:15] I'm using just one, and I used the dome filter here, which is a new module, which gives us
[00:17:22] here the sine and the cosine as a second output.
[00:17:26] So here I'm doing the same thing with the audio signal, but here with the cosine, and
[00:17:31] go also into a filter with the same position here, same filter position, and here we get
[00:17:42] a different position from this one, and we also use a different color.
[00:17:51] So this is the sine, this is the cosine here, and you can extract actually the phase from
[00:17:56] this.
[00:17:57] So because we have the sine and the cosine, we multiply then this with the audio signal,
[00:18:03] and dependent on the level from the sine and the cosine, we can kind of triangulate where
[00:18:09] the original audio phase position is.
[00:18:15] I don't do this in my patch because it needs more modules, and I don't think it's needed,
[00:18:21] but you can do it.
[00:18:22] So this is how we can extract actually the phase.
[00:18:26] But what we do then is we multiply the signal with itself, so we have only positive value.
[00:18:39] So here, and then we add these two together, just an addition, and then we take the square
[00:18:52] root, what's it called in English, square root from this, and this is our amplitude here.
[00:18:59] So you can see we have the test frequency on one kilohertz, and we have an amplitude,
[00:19:05] and we change the frequency, it goes down.
[00:19:08] We are not on 1k anymore, and we don't get an amplitude or value here until we hit 1k.
[00:19:18] Then we get something.
[00:19:19] And the filtering gives you a response time.
[00:19:23] So when you have this open, you get much, much more wider results here, as you can see.
[00:19:30] It responds much faster, but you get also garbage here for frequencies that are not 1k.
[00:19:36] We try to filter everything out, 1k.
[00:19:43] So that's kind of the thing you try to find, the sweet spot between responsiveness and getting
[00:19:54] actually a real pitch harmonic out of this.
[00:19:57] But it kind of works.
[00:19:58] So this is what I did basically for one frequency, and we can of course, instead of using here
[00:20:04] a test tone, we can use an audio signal here, let this play back, and then see what's going
[00:20:12] on at 500 hertz, there's probably something, and you can see here there's something happening
[00:20:19] at some point, probably more 200 hertz, there's more action.
[00:20:32] And then you can extract basically the amplitude of exactly this frequency out of an audio
[00:20:37] signal.
[00:20:38] So this is what I did.
[00:20:39] This is how it works roughly.
[00:20:42] Like I said, I put you here the presets in the description on my Patreon, you can download
[00:20:49] this, leave a thumbs up, leave a subscription, maybe think about the Patreon subscription
[00:20:55] would be nice, but you can download this for free.
[00:20:58] Okay, you don't need to.
[00:21:00] That's it.
[00:21:01] That's it.
[00:21:02] Thank you so much for watching.
[00:21:03] See you in the next video.
[00:21:04] Bye.
[00:21:04] Thanks for watching.
[00:21:05] See you in the next video.
[00:21:06] Bye.
[00:21:06] Thanks for watching. See you in the next video. Bye.