Recent updates

Summarizing Feedback with ChatGPT

By now you've probably played around with ChatGPT. It's really good at doing things like summarizing large bodies of text or helping you improve written content.

That said, I've been a pretty big skeptic about the application of this technology in the music submission space.

For example, there are companies out there that "use AI to analyze your song and tell you what your genres are" - and every time I've tested these they are close, but still pretty far off / unhelpful.

We've also had issues with curators using ChatGPT to improve their feedback, and have cracked down on that pretty hard. We want them to stay authentic. We know ChatGPT is an excellent writer, but it also sounds pretty corny at times.

A few of our competitors have launched "AI features" that help identify the right curators for a song. And to be honest, I'm not sure I fully trust they're actually using AI. I think they've simply written an algorithm and called it AI because a computer executes it. We've got plenty of that kind of human-written code here, and I think we do it better than all of the other platforms. For example, our genre matching and similar artists feature.

All that aside, I think I've finally found a pretty cool use of ChatGPT on SubmitHub: summarizing the feedback you've received so that you can easily extract common themes. It's mostly for fun, but I reckon some will find it helpful!

How it works on the front-end

As a user who has sent a song, you might have noticed that your "Responses" tab now contains a section where you can request a ChatGPT summary.

  • To generate your summary, look for this green box
  • You need a minimum of 5 pieces of feedback
  • This is bad news for those of you with 100% approval rates and no declines 😂
  • Hitting "Click here" kicks off the back-end process (below), and the wait begins!
  • The final result is a nice little summary of your feedback. Two examples below:

How it works on the back-end

The company behind ChatGPT is called OpenAI. They've got an API that allows developers like me to harness the power of ChatGPT using code.

Here's how I set it up:

  • First, I tell ChatGPT what kind of answer I want from them
  • Tweaking these instructions took me a couple hours
  • I've heard some people say a "new skillset" in the market is going to be "expert AI prompter", and I now feel like I have a baseline qualification 😉
  • Got ideas that might improve this prompt? Use the comments below
  • I then send these instructions + the feedback received for that song to OpenAI's servers
  • They take approximately 60 seconds to respond. I wish it was faster, and I imagine eventually it will be
  • The result is pretty cool - and I think it's often worded way better than any of the curators initially intended 😆

Costs involved

OpenAI is pretty cheap with their pricing. Each of these "requests" seems to cost between 500 and 1,000 "tokens", and they charge $0.002 per 1,000. So, for every 500 times that users request a summary of their feedback it's going to cost us about $1.

I actually took a few steps to optimize these costs. The bulk of the "prompt" that gets sent to their API is made up of the feedback received, so I've "shortened" it as follows:

  • Using a package called stopword I can remove filler words like "a," "the," "is," "are," etc.
  • Natural language models like ChatGPT don't need these words to figure out what's going on
  • I've limited the data to the most-recent 25 responses with feedback
  • Some folks out there send to 100+ curators, and if I were to pass all that feedback through the system, ChatGPT would actually come back with a response telling me that I sent too much information and exceeded the "maximum token limit"
  • Just in case there's superfluous feedback (eg, someone wrote 3 paragraphs of fluffy feel-nice text), I slice each bit of feedback to be 200 characters maximum

What comes next?

Eventually I'd like to add this feature to Hot or Not, but I need to keep an eye on these costs first 🤑

Got any ideas for cool ways that ChatGPT can be used on SubmitHub? Use the comments below!