Recent updates

AI Song Checker V2.0

Things move fast! Especially with such a vocal user base 😆

It's been about 3 weeks since I launched version 1.0, and reactions have been mixed. The good thing about that is that I've been able to take this feedback and refine the model.

Major issues with v1.0

The first version of the detector really struggled in certain situations:

  • Classic songs: all my data was trained on "human" songs from 2022 (submitted via SubmitHub). This meant that classics like Michael Jackson and Louis Armstrong were coming back with the wrong results, as "old" songs weren't part of the data set
  • Processed AI: because the data was built on "pure" AI downloaded directly from Suno and Udio, it was unable to properly detect "modified" AI tracks, such as those that had been mastered or otherwise changed
  • Low quality audio: everything in my original data set was 320kbps, so running analysis on a 128kbps file made for wonky results
  • Similarly different sampling rates (eg, 44khz vs 48khz) caused inconsistencies
  • Short durations: previews from Spotify are typically only 20-30 seconds long, and my algorithm wasn't set up for that
  • MIDI songs: technically human made, right? But they were all coming up as AI

Changes to the training data

  • Added samples from a few more AI platforms: AIVA, Boomy, Donna & Sonauto
  • Included MIDI detection (should be "human")
  • Someone on Reddit kindly pointed out that MIDI files were failing
  • Grabbed a copy of "the 1,000 greatest songs in history" and added that to the human training data (mostly popular songs from the 19th and 20th century)
  • Created a new "processed AI" category for my sample set
  • Previously I was looking at pure AI vs. human, and I was missing the fact that 90% of the AI that gets "released" has been modified/processed in some way (typically through a basic mastering service)
  • Created low quality and high quality versions of each song, allowing me to capture more diversity

Changes to the models

Spectral model:

This model converts audio files into spectrograms (visual representations of sound that show frequency and intensity over time) and then analyzes these "audio pictures" using computer vision techniques to detect whether the audio was made by AI or humans.

When this rolled out it was quite simple, but I've made two major changes:

  • The model now combines two analysis methods - CNNs for spotting small local details and Vision Transformers (ViT) for understanding the big picture, similar to how a human examines something from both up close and far away
  • It now processes both the volume (amplitude) and timing (phase) of sound waves together, while using frequency masking during training to help it recognize AI content even when parts of the audio are modified

Temporal model:

This model examines how audio patterns evolve over time - looking at segments ranging from 15 seconds to 3 minutes to analyze how musical elements flow, change, and relate to each other throughout a piece.

Some of the major changes:

  • The system now examines pitch changes, harmonic complexity, and how different musical elements interact with each other
  • Can identify patterns at multiple timescales - from short phrases to entire sections
  • Breaks down audio into different frequency bands (bass, mid, treble) to spot AI signatures that might be hidden in specific ranges
  • More accurately separates and analyzes different parts of the audio, like distinguishing between melodic instruments and percussion

What's next?

I think there are two things I need to do to make this better:

  1. Add more data to the training set
  2. Explore splitting vocals and instrumentals

Regarding #1, I've added a handy little rating option to each result. I plan to use these responses to help add new data to the training set - particularly in cases where the results were wrong.

Final Thoughts

At the end of the day, this is a fun project for me. I'm have no plans to commercialize it, and I anticipate being able to keep it completely free. I'm not sure it will ever be 100% accurate - things are constantly evolving, and there's a lot of nuance here.

Open dialog is welcome. I'm happy to consider your ideas and suggestions :)