How we made restreaming from YouTube using Go

How We Made Restreaming from YouTube Using Go.

A few days ago we have opened sources of one of our in-house projects — yourcast.tv

History behind it

It was a warm and sunny day of October of 2016. I was at my work drinking some coffee and solving some tasks. Everything was fine and there were no signs of trouble. Then Oleg Puzanov came into the building. “Look” — he said, — “I need a script or service or whatever. I need any solution that helps me to create a stream from youtube links. Online stream”. “Why?” — I asked. “Because I want to watch live streams of videos from past conferences” — He answered. “Ok” — I said. — “Give me a few days to think and do some research”.

I took a few days to think and make some tries.

First steps

First — we need to know how to get mp4 file from youtube. At this case, youtube-dl project helps us. I didn’t want to do any additional work and tried to make my solution very simple and very stupid.

I took youtube-dl and got a download link of a mp4 video in few minutes.

But there is the main challenge here — HTTP streaming.

I had options:

  1. MP4 HTTP streaming via VLC
  2. Same, but with FFmpeg
  3. Use Nginx RTMP module

OK. I took flash flowplayer and started to dig again.

I achieved working prototype with flash, vlc and HTTP streaming and we started to think how to make service with it.

Making service. Building architecture

The conclusion after we had a working prototype was — OK, let’s make a service from your prototype.

We need an admin interface to manage streams and we need a streamer component to run streams and make them available. We chose Django framework for it and plan was:

  1. Use standard Django admin interface
  2. Make HTTP API for streamer component
  3. Get streams from Django and run it on streamer component

As for Django, it was a very simple realization of it using signals, simple models and serialization.

The streamer

As for streamer we chose Go and we need an alternative to youtube-dl for Go programming language. After some research we chose https://github.com/rylio/ytdl. It has clear API and we have no problems to use it.

Streamer component should have an infinite loop and in that loop, we need to get download URL for a youtube video in proper format and then run stream. We decided to move to nginx-rtmp-module for the task because we had some experience in past with it.

The final algorithm was like this:

  1. Get all streams via HTTP API from Django app
  2. Run goroutine per stream with waitgroup for sync
  3. Get download URL from youtube
  4. Spawn FFmpeg process to stream video to nginx-rtmp-module

HTML5

At some first versions playing of live streams was implemented using flash and RTMP. But it doesn’t work for mobile devices and we decided to find a way to move to html5.

After some tries, we found that nginx-rtmp-module with videojs and hls.js and https://github.com/Peer5/videojs-contrib-hls.js works fine for us.

Testing and using

We started to test our implementation with streaming videos from past conferences like DockerCon 2016, GopherCon and others. We used one of our big monitors on the wall to play videos, test it for bugs and educate in a background.

Is It an open source? Where can I find the code?

Yes. We opened sources several days ago and you can find it on our GitHub:

What problems we faced and how we solved them, read here.

Issues, feature requests, contributing and any other requests are welcome.

Try to be a producer of your own TV stream for free, without registration and SMS :)

Custom Software and Mobile App Development.
Major Technologies in Your Tech Stack.

Major technologies you’d want to see in your tech...

Major technologies you’d want to see...

Major technologies you’d want to see in your tech stack: Pros and cons

The world of software development is increasingly dynamic: new technologies, tools, and frameworks emerge on a daily basis. Yes, only a small...

Mar 23, 2021
Application Design Proccess.

Software architecture 101 - introduction to the world...

Software architecture 101 -...

Software architecture 101 - introduction to the world of application design

Hello, everyone!Today, I would like to introduce you to the world of application design.Designing applications can be called a multidisciplinary...

Dec 17, 2020
Running Postgres as Requirement in TeamCity Build.

Running Postgres as a build requirement in TeamCity...

Running Postgres as a build...

Running Postgres as a build requirement in TeamCity build

We use different Continuous Integration tools in our projects. One of them is TeamCity software. A pipeline for TeamCity can be configured easily and...

May 27, 2020