How We Made Streaming App from YouTube Using Go

A few days ago we have opened sources of one of our in-house projects — yourcast.tv

History behind it

It was a warm and sunny day of October of 2016. I was at my work drinking some coffee and solving some tasks. Everything was fine and there were no signs of trouble. Then Oleg Puzanov came into the building. “Look” — he said, — “I need a script or service or whatever. I need any solution that helps me to create a stream from youtube links. Online stream”. “Why?” — I asked. “Because I want to watch live streams of videos from past conferences” — He answered. “Ok” — I said. — “Give me a few days to think and do some research”.

I took a few days to think and make some tries.

How to Start as a Tech Blogger

The tips to start writing articles as a technical blogger. The best practices that we use at Mad Devs to encourage blog post authoring and public speaking.

First steps

First — we need to know how to get mp4 file from youtube. At this case, youtube-dl project helps us. I didn’t want to do any additional work and tried to make my solution very simple and very stupid.

I took youtube-dl and got a download link of a mp4 video in few minutes.

But there is the main challenge here — HTTP streaming.

I had options:

MP4 HTTP streaming via VLC
Same, but with FFmpeg
Use Nginx RTMP module

OK. I took flash flowplayer and started to dig again.

I achieved working prototype with flash, vlc and HTTP streaming and we started to think how to make service with it.

Making service. Building architecture

The conclusion after we had a working prototype was — OK, let’s make a service from your prototype.

We need an admin interface to manage streams and we need a streamer component to run streams and make them available. We chose Django framework for it and plan was:

Use standard Django admin interface
Make HTTP API for streamer component
Get streams from Django and run it on streamer component

As for Django, it was a very simple realization of it using signals, simple models and serialization.

Go. REST or gRPC

The two approaches and several tools to make HTTP API in Go: REST and gPRC. Speed up the development process and save time on developing web services in Go.

The streamer

As for streamer we chose Go and we need an alternative to youtube-dl for Go programming language. After some research we chose https://github.com/rylio/ytdl. It has clear API and we have no problems to use it.

Streamer component should have an infinite loop and in that loop, we need to get download URL for a youtube video in proper format and then run stream. We decided to move to nginx-rtmp-module for the task because we had some experience in past with it.

The final algorithm was like this:

Get all streams via HTTP API from Django app
Run goroutine per stream with waitgroup for sync
Get download URL from youtube
Spawn FFmpeg process to stream video to nginx-rtmp-module

HTML5

At some first versions playing of live streams was implemented using flash and RTMP. But it doesn’t work for mobile devices and we decided to find a way to move to html5.

After some tries, we found that nginx-rtmp-module with videojs and hls.js and https://github.com/Peer5/videojs-contrib-hls.js works fine for us.

Testing and using

We started to test our implementation with streaming videos from past conferences like DockerCon 2016, GopherCon and others. We used one of our big monitors on the wall to play videos, test it for bugs and educate in a background.