Despite having forgotten what very little I learned about rugby when forced to play it 25 years ago I worked with Mark Johnstone of Content Hubble (who had to explain what all the statistics actually meant) on this interactive piece which compared the players of the different UK teams ahead of the selection for the British Lions tour.
Remember when that boat got stuck? ‘Satirical’ ‘game’ ahoy!
Do you love the UK as much as whichever right-wing lunatic’s deranged tweets presumably inspired me to make this flag-drawing game? Find out!
I put together the data behind this piece on the most popular songs on Working From Home Spotify playlists to get an idea of what people were listening to during ‘the new normal’ for office workers.
I researched, wrote and then built the interactive element of this piece for Uswitch comparing loading times for websites across different commercial sectors.
I wanted a way to watch a bunch of randomly selected old movie trailers from a certain year, so I built one.
‘Current circumstances’ have led to a lot of previously live events reconfiguring themselves into online versions – comedy nights, pub quizzes, even sick pervert stuff like Dungeons & Dragons and improv.
I help to run Maraoke, a comedy karaoke night where the songs are all rewritten to be about having sex with Pokemon video games. Having had every live gig pulled for the foreseeable future we decided to look at the possibility of an online equivalent.
Online karaoke presents 2 key problems:
The first is about timing. In case for some reason you don’t know what karaoke is: you are provided with the backing track for a song accompanied by a video indicating what the words and when to sing them. You sing them?
Introducing the almost inevitable lag of an online connection ruins this – it would have to vanishingly small (and reliably so) for the singer not to be noticeably behind the music. But even if you could fix this, you’ve still got a logistics problem:
At a live karaoke night, all the participants have to do is give the organisers their name and the song they want to sing. Then sing it.
What’s the online equivalent for this? You could just set up a Zoom call and let people take turns to sing – but how do you get them the backing track and lyrics? If you stream them to the singer, there’s lag. If they’re at the singer’s end, they need to share them back ‘down the line’, which necessitates walking everyone through a fiddly process which will probably vary depending on what version of what software on what operating system on what device they’re using, because computers. (Sorry, you cannot Maraoke because you have installed Skype for Business, please install Skype for Pleasure, etc.)
We experimented with various bits of existing software and there didn’t seem to be a particularly happy solution other than building something ourselves, which was probably not realistic.
Or was it? I did some reading and turns out the modern web browser has quite a lot of interesting stuff built into it: it is relatively easy to get a browser window on my computer to send and receive video and audio to a browser window on your computer, without installing anything special at either end.
WebRTC, as it’s called, just requires some stuff on the server to help the browsers at either end do some initial negotiation to get a more direct connection set up, and we handily already have a lot of what’s needed built into the browser-based software we use to run live nights. (This post has some information about how we use this tech, Rail’s Action Cable, as a sort of ‘remote control’.)
I hastily threw together a prototype for us to try on what would have been our April live date.
The singer saw something like this:
A green-backgrounded video of one of our songs*, the output of their camera in green and the output of the host’s camera in blue.
The host saw something like this:
With the singer’s video and the song video merged into one video at the top, and the output of their own web camera at the bottom. There’s no particular need for the singer to see the host but I was basing this off an example I’d found that was for 2 way communication and wasn’t confident enough to start pulling bits out of it.
The host can then captures the singer video in a piece of kit called OBS which can put the lyrics on top of the video with some Lord of the Rings-style HOLLYWOOD MAGIC, and push the result out over our streaming service of choice, Twitch, where it looks something like this:
Here’s a rough diagram of what the technology is doing to make this whole process happen:
What actually happens at the host end is a lot more complicated than this and on the night involves a hell of a lot of coordination between Ste (who does the on-screen hosting) James (who organises the singers while simultaneously switching between different video and audio feeds as needed), while I can largely just drink beer and loftily claim that I’ll “fix it tomorrow”. This would, ironically, be much easier if we could all be in the same place at once rather than miles apart from each other.
The trial run in April was in some ways a success – we managed to get some singers singing songs and stream that LIVE to the internet, but was very far from perfect.
Problem 1: Sometimes we couldn’t hear people. At all. We were asking the browser to get permission to use their microphone, but not asking WHICH microphone – a computer can have several audio input devices and it turned out the browser was not necessarily using the most obvious one. So I added a ‘soundcheck’ stage so people can sort that out their end and check that the browser can ‘hear’ them before they sign up to do a song.
Problem 2: Getting the overlaid lyrics from a video over the WebRTC connection doesn’t result in very good or consistent presentation – the video is more likely to ‘stutter’ than the audio and on a bad connection the quality will drop, making them look rubbish.
To solve this I added a copy of the video to the host end – we send timing data from the original video across the connection to sync them up, then I do the Lord of the Rings-style HOLLYWOOD MAGIC to put them on top of the singer in the browser, meaning we can have a bigger video window to capture and send to the stream.
I also did a bit of tidying up, removing the video of the host (while allowing an optional audio connection so that Ste can talk to the singer at the start/end of a song if he wants) and adding a bit more feedback about what’s going on and whether things are working.
In terms of what’s going on with the tech this makes the process a little more complicated:
But in practical terms once you’ve got the WebRTC stuff working, changing what you’re sending across it is relatively simple.
Problem 3: Wrangling the singers…
For the first attempt our method of getting the right singer connected at the right moment was to have pairs of singer/host links with a matching (randomly generated) keyword parameter, e.g. ‘snake’. This made it a massive faff to keep track of who was ready to sing then getting them the right link. So for ‘version 2’ I simplified things (by making them much more complicated) – as well as the video window, the host now has a 2nd window to control it – they can see who’s online and ready to sing, then connect to them when ready. The singer just has to complete the ‘soundcheck’ and pick a song.
The first official Maraoke Lockdown took place last Friday (May 1st), and was a massive improvement on our earlier attempt – we lost the recording due to some slightly confusing Twitch settings, but we think something like 27 songs were sung over the course of the evening, not counting some inevitable false starts. Some of them were from other countries! It turns out they have the internet there too!
The main technical issue** was that occasionally the backing track from the singer’s end wasn’t reaching the host end – so a few people were giving unexpectedly a capella performances until we reconnected and started again. Not entirely sure why yet – either we need to mix the two audio tracks into one at the singer end, or something is causing the stream of the video’s audio not be picked up at all, but it feels pretty fixable, especially as the actual connections seem to be pretty reliable now.
The other issues are largely around making it easier for the host to manage the queue of singers, and improving communication with people signing up to sing – the whole process needs to be a bit clearer in terms of what to expect, what they have to do (e.g. WEAR HEADPHONES, TURN YOUR RADIO OFF, ETC) and when, but at least we’re getting clearer about what we need to communicate even if we haven’t managed to do it entirely successfully yet.
And given that 3 weeks ago we had absolutely nothing (well, 500 comedy karaoke songs), I reckon that’s pretty good. We may not have developed the cure for coronavirus, but last night we livestreamed a cat singing a version of 99 Luftballons rewritten, in German, to be about an aeroplane shooting game (I think? I don’t speak German). And isn’t that the real victory?
Is any of the stuff I’ve figured out doing this useful for doing something that isn’t slightly weird video game-based comedy karaoke? Who knows, but happy to talk to anyone who thinks it might (e dot jefferson at gmail dot com/@edjeff). Oh, and you can follow us on Twitch now!
—
* The visual elements of the songs are normally animated on the fly in the browser but there’s no simple way of pushing that animation into a WebRTC connection. There IS a slightly awkward way of capturing tabs as video built into modern browsers and you can save the result off to a file rather than piping it down the internet, so I was able to knock up an automated ‘video recorder’ to save us manually screen recording 500 songs.
** Well, also none of this works in anything other than Chrome yet, but Firefox should support all the tech with a few adjustments and apparently Microsoft Edge is basically Chrome now???? Apologies to IE6 users at this difficult time.
Maraoke is a dumb/excellent live event which is exactly the same as a normal karaoke night, with the same songs, except when you sing the songs it turns out that Coolio’s Gangsta’s Paradise is now about Sonic The Hedgehog for some reason?
This is exactly as stupid as it sounds but people seem to enjoy it – there are now over 500 songs in the roster (many of which are submitted by attendees) and the event has traveled all over the world to baffle/entertain/anger crowds.
I’ve gone from attendee to songwriter to helping run events to accidentally writing a new karaoke software system – most months I can be found behind a computer at the regular event we do at Loading bar in Dalston, with my biggest enemy, this pile of little cards:
I have grown to despise this pile of cards.
The way a Maraoke night works is that people who want to sing write their name and the song on a little card, give it to me and I put it at the back of the pile. Sometimes it will go further forward than the back if it’s a good song and everyone else has been picking rubbish songs.
Then someone will ask if Song X has already been taken and I’ll have to look through the pile one by one because I can’t remember and I’ll say ‘Yes, you can sing Song X!’ only for it to turn out that the reason no-one else has picked it is that we don’t have Song X, or that we do have Song X and someone has picked it but two of the cards have got stuck together. Then the first person will come back and asked to change their song only I can’t find the card because it fell out when I was looking for Song X and now I’ve dropped all the cards and I can’t remember which order anything was in and which card did I just hand to the host to announce next anyway and please stop putting your drinks next to the computer equipment.
If only there was a better way!
The software that powers the night does have a queuing system, but it’s so primitive that it’s rarely used to do anything more than queue up the song that’s immediately next. Plus the whole screen is a big mess of arbitrarily jammed in stuff that seemed like a good idea at the time, and could probably do with an overhaul. Also why is it any of the colour that it is?
First things first, let’s take the sign-up process ‘online’ – people can still pass me a card, but I’ll immediately enter the details on to the system. Probably on my phone so I don’t have to hunch over my laptop like some kind of computer pervert. Since the system is all web browser-based anyway this is just a matter of adding a new page to the Rails app that powers it.
Not only can I queue the song up, but I can automatically check whether it’s already on the queue and get them to pick something else if it has. I’ve also stuck in autocompletion for the song entry using JQueryUI to speed up selecting which song they’re after.
What if I am sick of waiting for all these losers to sing their songs and want to go straight to the top of the queue? A bit of SortableJS adds lets me drag my song up and drop it at the top of the list.
Uh oh, but what happens if I’ve got the queue open on my phone, and also on the laptop that’s actually playing back the songs?
Luckily Ruby on Rails, which makes all of this work, comes with a way for different windows to talk to each other called Action Cable. Literally every tutorial I’ve ever found about this is about making a chatroom app but Maraoke uses it to let the behind the scenes bit tell the ‘laser display board’ (just another browser window that happens to be being displayed by a projector) to start playing the music and displaying the karaoke lyrics, and for the display to feed back information to the control screen.
So to make this new stuff work I just make use of the fact that every window can chat away to every other window, and, e.g. send out a message to all the open windows to refresh the queue when there’s a change.
There’s still one thing missing: with the old card-based system, I’d pass a card up to the host, Ste, to let him know what’s coming next. To replace that I knocked up a page for him to load on his phone that should automatically show what’s at the top of the queue. I even popped in a Veto button should he suddenly remember that the next song contains a joke insulting the mother of Ian Moneybags, who has just turned up and is about to buy Maraoke for A MILLION POUNDS.
And if he doesn’t even want to announce the next singer himself, the system can now do it on the credit screen which displays at the close of each song.
By the time I’d finished hooking all this up it was more or less time to get to a train to Dalston for the February Maraoke, where we did a live ‘alpha test’.
TEST CONCLUSIONS:
It worked! I kept all the cards in a pile as a backup but was pretty quickly confident enough in it that I entered a few requests in directly without making people fill out a card at all.
This was a massive improvement, making it much easier to do stuff like prioritise people who haven’t sung before and deprioritise scoundrels who have snuck two songs into the queue at once. Apart from anything else it reduces the amount of time spent trying to interpret the disgraceful handwriting of today’s youth, bring back National Service and Parker pens I say.
But it was NOT PERFECT:
But basically, a successful test. Maybe at some point we will do away with the cards altogether and SAVE THE PLANET, maybe writing things on little cards is part of the INDEFINABLE MAGIC of Maraoke, who knows?
Maraoke happens every month at Loading Bar in Dalston in London – the next one is on March 13th. Or it happens LITERALLY ANYWHERE IN THE WORLD you would like, if you want to pay us to do that – see the website for details.
I am always looking for more dumb ways to use my haphazardly acquired technical knowledge to make weird creative stuff happen: e dot jefferson at gmail dot com if you’d like to talk about that!
