Personalised concert recommendations using Songkick and Apify Actors
I’ve recently directed my automation addiction towards music events.
Loads of artists come through Sydney but Songkick only sends notifications for big-ticket concerts, and quite infrequently at that. What about all of the smaller gigs happening every week? Are there any recommended artists I should check out?
To fill this gap I’ve used the Songkick API to produce an RSS feed that tells me if any artists I track, or artists that are similar/recommended, are coming to town.
The list of “Recommended through” artists shows which of my tracked artists they’re similar to. Sometimes it can be 10 or so artists long, which is usually a good sign I ought to be interested.
Combined with this awesome iOS 12 shortcut for looking up Spotify artists, it’s a pretty sweet workflow.
It’s fully automated in the cloud, for free, using Apify Actors and Crawlers. You can view the code on GitHub and read on below for how it works and how to set up your own.
How it works
For simple automation tasks I often turn to Zapier, but I’ve recently been playing with Apify and enjoying it a lot. I wrote about crawlers in my last post, but Apify also has Actors for complete flexibility.
You point the actor at a GitHub repo with a Dockerfile
and some code and it will run your Docker container automatically, serverless-style. I’ve often wanted a reliable place (read: non-laptop) to automate simple scripts, and this suits my needs well.
Here is how I combined Apify’s Actors, Key-value Store, and a Crawler to produce the final RSS feed.
The following JavaScript routines carry out the necessary API calls and persistence steps:
- One actor pulls all of my tracked artists from the Songkick API, and for each of those fetches 50 recommended artists. The results are persisted to a Key-value Store in Apify. I only run this once a month because it takes a while and my tracked artists don’t change that often.
- Another actor pulls all Songkick events for my “metro areas” and saves any that include the artists I pulled in step 1, again to a Key-value Store. I run this every night.
- A Crawler reads the JSON results from the Key-value Store and produces an RSS item for each event.*
How to set it up for yourself
The README on GitHub has instructions for how to set this up in Apify.
Learnings along the way
The following tips might be useful for your own actors.
Splitting out the routines into different actors
It didn’t make sense to put everything in one actor, so I split them into actors I could schedule separately. Key-value stores were useful for transferring state between different stages.
I also wanted to share some code across different actors, and my initial plan didn’t work out… Apify lets you point each actor at a subdirectory in the git repo to look for a Dockerfile
, so I initially went for something this:
# This doesn't work
src/
common_utils.js
actor_1/
Dockerfile
act.js # Uses ../common_utils.js
actor_2/
Dockerfile
act.js # Uses ../common_utils.js
But, with the way Apify runs your Dockerfile
, you can’t access parent directories to copy them into your container. To solve this, all actors in the repo now share a Dockerfile
which calls a single entry point script. The script checks an environment variable (you can set this per actor in the Apify UI) to work out which actor function it should run each time.
# This does work
Dockerfile # Now all JavaScript is underneath the Dockerfile
src/
entry_point.js # Checks an env variable to run the right act
common_utils.js
actor_1/
act.js
actor_2/
act.js
Installing package.json
’s devDependencies
in the Docker container
My code requires compilation with Babel, which meant I needed to install devDependencies
from package.json
when building the Docker container. I found that a simple npm install
was skipping my devDependencies
because the NODE_ENV
environment variable was being set to production
, I’m guessing by Apify. My Dockerfile
therefore sets NODE_ENV=development
right before using Babel, and then switches it back to production
afterwards.
Future ideas
The set-up works pretty well for me at the moment but I have some additions I’d like to make at some point:
- Add images, or links to YouTube/Spotify. This would be a nice to have—Feedly already adds a cover image automatically, and I can look up the artist in various apps pretty quickly using iOS 12 shortcuts.
- Render the RSS content in HTML (currently it’s plain text), to add more context in a clean way. This isn’t actually supported by Apify’s crawlers (they don’t support
CDATA
elements), but I could achieve it via an actor where you’re allowed to produce XML with a specificcontent-type
.
Wrapping up
You can find the code on GitHub as well as some instructions for how to set it up for yourself. Improvements are welcome!