Recent updates

AI Song Checker: How It Works

In case you didn't notice, AI music is one of the hot topics in the industry right now.

A guy named Ben Jordan actually did a video about it not too long ago, and in it he said he'd made his own to tool to detect AI. Super cool! Unfortunately, to my knowledge, he hasn't divulged the way his tool works or let the public try it. The good news is that I'm gonna tell you how mine works 😉

First things first, if you want to give my model a try, you can do that here: https://www.submithub.com/ai-song-checker

I won't pretend it's 100% accurate, but I'd say it's pretty close. I've tested hundreds of songs on my own, and it gets it right around 90% of the time. That’s not perfect, but still pretty solid.

How the Model Works

My AI audio detector uses a Random Forest Classifier, which is really good at yes/no classification problems. It looks at various audio features, then gives you a probability score that suggests whether a track is AI-generated or made by humans.

Most of the core logic behind this approach came from this academic paper.

Training Process

I trained it on two types of audio files:

AI-generated samples
Human-made samples

To keep things balanced, I pulled an equal number of samples from each set (around 2,000 for each). That way, the model doesn’t lean too much one way or the other. Each audio file goes through a bunch of feature extraction steps, and that data trains the Random Forest Classifier.

Feature Analysis

My detector checks 21 features across three main areas:

1. Basic Spectral Features

Spectral Flatness: Shows whether the audio is more noise-like or tone-like
Spectral Rolloff: The frequency under which most of the audio's energy is found
RMS Energy: Overall loudness of the track
Zero Crossing Rate: How often the waveform changes its sign
MFCC Statistics: Captures different timbral qualities

2. Harmonic Analysis

Phase Coherence: How consistent phase relationships are across time
Frequency Band Ratios: Looks at energy spread in different frequency zones
Harmonic Consistency: Checks if harmonic structures stay steady
Harmonic Stability: Gauges how stable harmonic content is over time
Pitch Transition Rate: Measures how quickly the pitch shifts
Harmonic Complexity: Quantifies how complex the harmonies are
Vocals/Music Ratio: Balances the vocal presence vs. instruments

3. Long-Range Pattern Analysis

The model also looks at patterns over three time windows:

3-minute sections (180 seconds)
2-minute sections (120 seconds)
1-minute sections (60 seconds)

It calculates how correlated these sections are and how much they vary, using standard deviation and other measures.

Please don't quiz me on any of these 😆. Ironically, I levered AI (Claude) to help generate this algorithm based on the previously linked Sonics paper and a few other prompts.

Limitations and Considerations

No model is perfect, and here are a few weak spots:

Audio Length: Best results come from tracks 32-180 seconds long
Audio Quality: 196-320kbps gives more reliable data
Genre Bias: Performance can vary across music styles
AI Evolution: As generative tools improve, retraining will be necessary
Mastered AI Tracks: As soon as someone manually masters an AI track (adding a human touch) it struggles to detect
Mixed Elements: If half the track is AI (vocals) and half is human (instrumental) it'll also struggle
No Vocals Present: A number of the measures rely on vocals to be present for accurate assessment, which means instrumental songs (jazz, blues, etc) often fail to give valid results

Next steps

Here’s what I think I'll do next:

History tracking in the UI so you can see songs you've recently tested
Use GPU processing instead of CPU for faster performance
Increase the data set (e.g., from 2,000 samples to 20,000)
Separate the vocals from the instrumentals and analyze each one separately

This was a fun little project, but I should probably focus on other website features for a little while!

Let me know what you think in the comments below.