Mini Neural Nets for Guitar Effects with Microcontrollers | by Keith Bloemer | Apr, 2023

Less is more

I’ve come to realize that constraint drives creativity and innovation. The more options an artist has, the more difficult it can be to think of a solid idea. Without rules in music, the possibilities are so overwhelming that it would be near impossible to write something listenable. It’s the rules and restrictions that give structure to an idea. Of course knowing when to break those rules is just as important as knowing what they are.

Over the past three years I’ve been fascinated with using deep neural networks for emulating analog guitar amps and effects. I’ve written a handful of audio plugins and tested countless variations of neural networks for emulating guitar gear. Neural networks are CPU intensive, but with PCs you can often throw more computer power at it to achieve a more accurate sound.

For more info on how this works see my articles on neural networks for real-time audio.

But what if you wanted to run a neural net on a cheap microcontroller, with a small (guitar pedal size) device, dedicated to just running an audio effect and nothing else? Small project computers like the Raspberry Pi are powerful enough to run an operating system, supporting applications, and a guitar effect, and have room to spare, which means wasted resources at higher cost. These types of computers use microprocessors, as opposed to microcontrollers. A micro-processor has external memory and I/O, and a micro-controller is a self contained computing system that can run programs on it’s own. Microcontrollers generally don’t need to be as powerful as a PC or smartphone, because it’s fully dedicated to doing one specific thing. Microcontrollers are ideal for a digital guitar pedal, but writing software for them is different from a smartphone or PC, and something I had never done before.

Enter, the Daisy Seed! The Daisy Seed is a $30 embedded platform for music, utilizing the Cortex-M7 ARM microcontroller. It has its own software library (libDaisy) and a DSP library for audio (DaisySP), all open source. For a guitar pedal, you want an input audio buffer for connecting your analog guitar signal to the digital hardware, and an output buffer for matching impedance with your amplifier. This ensures that your tone is preserved going through the effect. PedalPCB sells an interface board called Terrarium ($12) specifically for the Daisy Seed, and it even provides connections for up to 6 knobs, 4 switches, 2 footswitches, input/output mono audio jacks, and 9 volt power (common to guitar effects).

At $42 for the audio hardware, I’m already well below the cost of the Raspberry Pi4 used in my previous NeuralPi guitar pedal. I bought the boards, and thanks to my previous obsession with analog effects, had the remaining hardware on hand to build the Terrarium pedal. Total cost was around $100 (with enclosure, electrical components, knobs/switches), and required basic soldering skills to put together. The existing projects other people had developed for guitar effects worked and sounded great on the completed Terrarium pedal.

BUT, my NeuralPi plugin can’t run on a microcontroller, so I needed to get creative. I decided to use the RTNeural engine to run the neural models and develop minimal c++ code around it using libDaisy and DaisySP. There are several codebases out there for developing on the M-series microcontrollers, but I had no experience with these, so if at all possible I wanted to stick with what I already knew.

Now for those constraints I mentioned: processing power, and memory. The M7 chip operates at a max of 480MHz, and at that speed can only utilize Flash memory, a tiny 128KB! (But to be fair, landing on the moon only took 74KB.) There are other memory areas you can access on the Daisy Seed, but I wanted to maximize my processing speed to run the biggest neural networks possible, to produce the best sound.

When I initially compiled my minimal c++ program with RTNeural, I blew past the 128KB limit. So then I reached out to the creator of RTNeural for help (as I have many times before!). He was able to reduce the compiled footprint by a significant amount, allowing for plenty of space for the required code and model files.

Once my program would load onto the Daisy Seed, I was really curious what size model it could handle. I started with the NeuralPi models (LSTM size 20), but it was too much. It caused the pedal to be unresponsive. Then I figured I’d start low and work my way up to the limit, so I used a LSTM size 4, and success! It would run the effect (a capture of my TS-9 overdrive pedal) but I wasn’t happy with the accuracy. I determined that the most it could handle was an LSTM size 7, which is decent enough for most pedals, and might get some OK sounds on capturing lower gain amps. Keep in mind that the Daisy Seed runs at 48kHz audio, which means 48,000 passes through the neural net each second!

This kind of neural net sampling rate is impressive on any system, but I wanted it to sound even better. Guitarists of recent years are accustomed to digital modelling that is indiscernible from the real thing. Products like NeuralDSP’s Quad Cortex, IK Multimedia’s ToneX, and the more recent Headrush Prime have given guitarists options to do what was previously only possible using a Kemper. Just because my work is tailored to the DIY community doesn’t mean it has to be any less capable! (As the recent open source Neural Amp Modeler has proven.)

There is another form of Recurrent Neural Network (RNN) besides LSTM that I had not tested, the GRU (Gated Recurrent Unit). This network uses approximately 33% less processing power than LSTM for the same internal size. I never did much testing with it because on a PC, the extra processing wasn’t as much of a concern.

The initial test with GRU was promising, I was getting even lower loss values on GRU’s than the same size LSTM. In addition to running faster, the GRU models used up less memory. I determined through trial and error that a GRU of size 10 was about the max I could run through RTNeural on the Daisy seed (vs. 8 with LSTM), and that difference improved loss values by half in some cases. I was getting loss values of less than 0.01 for distortion pedals, and less than 0.02 for low gain amplifiers.

With the current release, NeuralSeed (as I named it) can accurately capture most distortion/overdrive pedals (up to 3 parameterized knobs) and low gain amps. Using no parameterized knobs remains the most accurate, but the flexibility of controls for gain and tone is a cool option in my opinion. Techniques I previously learned have benefitted the overall sound quality. These include improvements in my recording equipment, such as using load boxes to record amps and re-amp boxes for pedals. Starting from a pre-trained model (transfer learning) and using specialized input signals have helped improve accuracy in the model training. By using a technique called “distillation” (training a small model from a larger model), I was able to capture the complex distortion of tube amplifiers at low to medium gain.

Here is the open source code on Github. When loaded on the Terrarium pedal, it has input/output level knobs, wet/dry mixer, 4 EQ boost switches, and up to 3 knobs for controlling a parameterized neural model (gain, bass, and treble, for example). The left footswitch either bypasses or engages the effect. The right footswitch cycles through the available neural amp/pedal models.

Developing for the Daisy Seed has made me appreciate the power of modern day computers. The constraints of the M7 processor (which is a very capable microcontroller!) made me think outside the box to achieve more using less. The NeuralSeed code is open source for anyone to try out on the Daisy Seed / Terrarium pedal, as well as mod and make their own tweaks and improvements. There is also a Colab script for training your own models to run with the NeuralSeed software. Here is a video demo of the finished Neural Seed pedal running several neural network models of actual guitar amps and pedals.

Special thanks to Jatin of ChowDSP for minimizing the RTNeural footprint and helping me solve the GRU implementation! Also thanks to the folks on the Daisy Discord for helping me understand the Daisy Seed and related software, and the Neural Amp Modeler Facebook group for providing training data.