Tags: posts polarity-music Audio-Effects Audio2Midi Bitwig Plugins Tutorial

Neural Note: A Free VST Plugin for Audio to MIDI Conversion

Tutorial | Jul 20, 2023

The video is about a website called Basic Pitch developed by Spotify, which converts audio to MIDI. However, the drawback is that it runs slowly in the browser. The video then introduces Neural Note, a native VST plugin developed by Damien that can be used in any DAW. It shows how to use Neural Note in Bitwig Studio to convert an audio file of a piano sample into MIDI. The video explains the various settings and features of Neural Note, including note sensibility, split sensibility, minimal note duration, scale quantize, pitch bend mode, and time quantize. However, the video mentions a drawback with the recording feature and suggests improvements for future updates. Overall, the video recommends Neural Note as a useful tool for converting audio to MIDI and provides a link for viewers to download it.

You can watch the Video on Youtube #

Overview

In this video, I showcased Basic Pitch, an open-source audio-to-MIDI conversion tool developed by Spotify. Basic Pitch runs entirely client-side using JavaScript, ensuring no data upload is necessary. To overcome its slow browser-based processing, Damien created Neural Note, a native VST plugin. I demonstrated how to use Neural Note within Bitwig Studio, detailing its features and some minor drawbacks.

Key Points

Questions & Answers

Maybe you dont watch the video, here are some important takeaways:

What is Basic Pitch and how does it work with audio to MIDI conversion?

Basic Pitch is a website developed by Spotify that allows users to convert audio to MIDI. It is an open source tool that works completely client-side with JavaScript, eliminating the need for users to upload anything. The drawback of Basic Pitch is that it can only be used in the browser and can be quite slow due to processing with JavaScript.

What is Neural Note and what are its benefits compared to Basic Pitch?

Neural Note is a native VST plugin developed by Damien that allows users to convert audio to MIDI within their DAW. It is a modified version of the open source Basic Pitch tool developed by Spotify. Neural Note provides a faster and more efficient way to convert audio to MIDI as it is a native plugin rather than a browser-based tool. It is available as a VST3, AU, and standalone plugin, providing flexibility for different DAWs and workflows.

How does Neural Note work within a DAW and what features does it offer?

When using Neural Note within a DAW, users can apply the plugin to an audio track containing the audio they want to convert to MIDI. The plugin then analyzes the audio and recognizes the pitches or notes present. Neural Note offers several features to refine the converted MIDI data, including note sensitivity, split sensitivity, minimal note duration, scale quantize, removal of falsely recognized notes, and pitch bend mode. These features allow users to adjust the plugin's recognition settings to achieve more accurate and desired results.

Are there any limitations or drawbacks to using Neural Note?

One limitation or drawback of using Neural Note is the recording feature. When recording audio with Neural Note, there is a small gap between pressing the record button within the plugin and starting playback on the DAW's transport. This can result in an offset in the recorded MIDI data, requiring users to manually adjust the position of the notes to align them with the beginning of the audio track. It would be beneficial if the developer could implement a feature that synchronizes the recording with the DAW's playback to eliminate this offset.

Transcription

This is what im talking about in this video. The text is transcribed by AI, so it might not be perfect. If you find any mistakes, please let me know.
You can also click on the timestamps to jump to the right part of the video, which should be helpful.

[00:00:00] In some of my recent videos, I showed you this website here called Basic Pitch.
[00:00:05] And it's developed by Spotify. It's an open source tool to convert audio to MIDI.
[00:00:14] It works completely client-side with JavaScript, so you don't have to upload anything.
[00:00:20] And the only drawback was that you had to use the browser and that it was pretty slow, because it was processing with JavaScript.
[00:00:29] So someone took this library, because it's open source of course, and developed some kind of native VST plugin you can use in any DAW.
[00:00:39] It's called Neural Note and the developer is called Damien. Damien runs in.
[00:00:45] I put a link in the description below so you can download this. It's completely free, like I said.
[00:00:50] VST3, AU and standalone also, which is nice.
[00:00:57] I want to show you how this looks like here in Bitwig Studio.
[00:01:03] To test this, we have to use some kind of audio file.
[00:01:07] So maybe I use here some kind of piano sample.
[00:01:18] So we'll start with this here. We already know it's in G minor here and it's 74 BPM, so it's not a mystery what happens in this file.
[00:01:31] So here we have this playing. On this audio track, we put now the neural note plugin.
[00:01:42] And to actually hear something, because when we press play, you can see it's muted.
[00:01:49] We have to enable here the mute out, disable this.
[00:01:55] And then you get this audio file in here. We can either press record, then it play on the transport.
[00:02:08] And then it automatically here recognizes the pitches or the notes in there.
[00:02:17] And we have some notes on the left side here where we can refine the result.
[00:02:22] You can see it's blazing fast. You can see this in real time while you are tweaking here what's happening.
[00:02:29] So we have the note sensibility if you pull this down. Of course, it doesn't recognize any note at all because the sensitivity is zero.
[00:02:36] But you can slowly pull this up and you get more and more notes there.
[00:02:43] And then we have the split sensibility here, which basically means the sensitivity for splitting into individual notes.
[00:02:52] So when you know in here, there are probably just some notes, some single notes held down.
[00:03:00] So right, there's no individual small little notes playing. You can pull this down.
[00:03:06] So we have single longer notes versus you pull this all the way up.
[00:03:12] You have a lot of small little notes, which is not the reality.
[00:03:16] So we pull this all the way down and then we have the minimal note duration.
[00:03:21] So the minimal length of a note, you can see here, 35 milliseconds should be the shortest note.
[00:03:32] That's pretty long. So we can leave it at that.
[00:03:38] Then we have a scale quantize here. It's more or less like the range is more or less like a low cut and a high cut.
[00:03:46] So you can exclude your certain ranges, right? If you know there's a lot of rumble maybe in the or noises inside of the audio sample and you want to cut this away.
[00:03:56] So it's not recognized as notes so you can, you know, shorten the range basically or narrow down the range here to a certain degree.
[00:04:06] Maybe it's something between C3 and C5. I don't know.
[00:04:11] Then we can also narrow down to a scale. If you know exactly this is here, we know it's in G minor.
[00:04:17] So we can use here. This is G and it's G and minor.
[00:04:23] And then you can remove also some falsely recognized notes from the results because we already know it's in G minor.
[00:04:32] So this helps sometimes. If you don't know this, then just leave it at C.
[00:04:37] Dramatic. So we have all the notes in there.
[00:04:41] And then there's also here a pitch bend mode. So it also recognizes bending, which is really nice.
[00:04:50] And it's also recognizing this pretty nicely.
[00:04:54] Then there's a time quantize note to everything line up with the grid, with the beat grid.
[00:04:59] At the moment here it's disabled. I don't know why.
[00:05:03] What the reason for that is maybe it's not implemented yet. I have no idea.
[00:05:08] The big drawback for me is actually the recording feature here.
[00:05:13] As you can see here, we just hit record in here and then it starts recording inside of the plug in.
[00:05:19] And then you have to press play on the transport. And then you have this gap basically.
[00:05:23] That's the gap between me pressing record here and hitting play there.
[00:05:28] And then you get basically here a small little gap. We have to offset then later because now when we drag here the MIDI file onto that,
[00:05:36] it asks if you want to import tempo and time signal. We say no.
[00:05:41] But inside of this note file, you can see here the notes actually starting with an offset.
[00:05:46] So you have to go in there, select everything and just move it to the beginning.
[00:05:52] So this is a small drawback of this plug in. If you use the record feature,
[00:05:58] maybe the developer can implement something to synchronize the record to the playback of the DAW.
[00:06:05] So when we hit record here, it just waits or is armed.
[00:06:09] And then when you hit play on the transport, then it starts recording.
[00:06:13] It would be much better in my opinion.
[00:06:16] So it's recognized here this probably...
[00:06:20] Kind of okay-ish, I would say.
[00:06:29] We probably have to remove some notes here.
[00:06:33] Maybe use here...
[00:06:38] polysynth.
[00:06:40] But you get at least all the information.
[00:06:45] And this is probably a note that goes here.
[00:06:57] This is something we don't need.
[00:07:12] So we can extract the notes pretty easily from this.
[00:07:17] And I want to show you the different drop flow here from...
[00:07:22] Go for this.
[00:07:27] It's in here.
[00:07:30] Neural note.
[00:07:33] And we just want to drag this file into here.
[00:07:36] We can track this directly here.
[00:07:38] No, that's not working for some reason.
[00:07:41] We have to go here to the file section in Bitwig.
[00:07:44] And here all the files are listed from the current project.
[00:07:48] And this is just this file.
[00:07:51] So we can track this in.
[00:07:52] And the benefit of this is basically we start exactly here with the file start.
[00:07:57] We start also with the media file exactly at the beginning, which is really nice.
[00:08:02] Then we track this in here.
[00:08:04] No.
[00:08:05] And...
[00:08:06] So it kind of works nice.
[00:08:19] It's sometimes a bit distracted.
[00:08:21] Here you can see it generates sometimes too much too many notes.
[00:08:26] But I guess that's...
[00:08:28] It's already pretty hard for a computer to actually recognize this.
[00:08:34] But for open source it works pretty nice.
[00:08:37] And if you need sometimes the notes from a sample or from a radio cut or maybe from piano loop like I did here,
[00:08:48] I mean it's for free.
[00:08:49] It's open source and it kind of works nice.
[00:08:52] Why not?
[00:08:53] So I put you the link in the description below.
[00:08:56] Maybe it's helpful to you.
[00:08:58] If you like the video, like the video.
[00:09:01] Subscribe to the channel.
[00:09:03] Thanks for watching and I'll see you in the next video.
[00:09:05] Bye.