Tuesday, September 18, 2012

Streaming videos over http while encoding and hardsubbing at the same time

So a few days ago I bought the Google Nexus 7. I also have to say that I like to watch Anime. That being said you might already see where I am going, given that you read the title. As you know the Nexus 7 has very limited storage capacity, I even bought the 8 GB version because I didn't see why I should pay 50€ just for 8 GB more storage. Anyways, to understand why I'm encoding the Anime I stream to my tablet, you have to know that recently most Anime Fansub-Groups started releasing their videos in the High 10P H.264 Profile, which the Nexus 7 (and most other devices, including Desktop PC's) can't decode in hardware. Either you try to decode it in software, which is okay on the Desktop, but lags horribly on the Nexus 7, or you go and try to decode it in hardware, which messes up color decoding.
My first approach to tackle this was re-encoding in the High Profile of the H.264 Codec, that, however, would have taken about 3 times the playtime of an episode, while reducing quality.
My second approach, since this long waiting period just isn't worth it, was re-encoding to MPEG4 Part 2, which is a lot easier to encode. The filesize increase was huge though, more than twice the size of the input file, also visual quality is reduced due to the way the colors are encoded (I think?).
These two options given, I first encoded the files in MPEG4 Part 2 and put them on my tablet, with 8 GB of storage, however, it was clear that this was not a solution, so I started playing around with streaming.
The first problem was, that while I was able to encode video- and audio-stream separately if encoding to my harddrive, this was not the case for streaming. For simply encoding and hardsubbing the video-stream I used MPlayer and a fifo pipe which then was encoded by FFmpeg.
The MPlayer line looks like this:

1
mplayer -vo yuv4mpeg:file=/tmp/foo -ao null

and the FFmpeg line like this:

1
ffmpeg -i /tmp/foo -vcodec mpeg4 -sameq /tmp/tmp.mp4

I had to get both streams, audio and video to FFmpeg so it could mux the streams together. At first I tried using only one MPlayer instance, it turned out, that this:


1
mplayer -vo yuv4mpeg:file=/tmp/foo -ao pcm:file=/tmp/bar

in combination with this:

1
ffmpeg -i /tmp/foo -vcodec mpeg4 -sameq -i /tmp/bar -acodec aac -ab 192k /tmp/tmp.mp4

produced a deadlock, MPlayer would not decode the video-stream, because it was waiting for the audio-stream to be read by FFmpeg, which itself was waiting for the video input by MPlayer. So I rearranged the order in the FFmpeg line, but that didn't fix it, MPlayer proceeded to decoding the video-stream, but FFmpeg would still lock up.
The solution was to use two MPlayer instances, one for the audio-stream, one for the video-stream. Now I could write the encoded video to a file, while muxing it with the audio stream. That was not what I was looking for though. What I wanted was streaming the video over the network, so I piped the output of FFmpeg to stdout, which greeted me with the fact that I need to specify a container format to be able to stream to stdout. Therefore I tweaked the FFmpeg line a bit:

1
ffmpeg -i /tmp/foo -vcodec mpeg4 -sameq -i /tmp/bar -acodec aac -ab 192k -f matroska pipe: > /tmp/stream

If I played /tmp/stream now with MPlayer, I could see the encoded video while it was being encoded. This was quite the achievement for me and the only thing left was streaming it over the network.
At first I tried serving the fifo pipe with lighttpd, that obviously failed and lighttpd produced a 404 error. Next thing I tried was using ffserver, but I never really understood how the configuration file is supposed to work. Then I remembered that I once created a fifo pipe on the tablet's internal storage and managed to pipe a data-stream over ssh to that fifo pipe. That however showed some weird behavior, such as the media player not playing the file until the stream was terminated. The solution to my problem was netcat. I just learned about netcat a few days ago and already use it to stream some log-files over telnet, so I figured it should be able to do the job, but how? After quite some research I came across a page that described how to use netcat to imitate a web-server, which is exactly what I needed. So tried using netcat to stream the fifo pipe over some port:

1
nc -l -p 8080 < /tmp/stream

In general that did work when opened the socket, piped the data to stdout and used another pipe to let MPlayer play it like this:

1
wget -O - http://localhost:8080/stream | mplayer -

MPlayer itself would fail to play the stream though, this was because it didn't get a proper HTTP reply, so after a little research I found myself typing this:

1
(echo -e 'HTTP/1.1 200 OK\r\n'; cat /tmp/stream) | nc -l -p 8080

MPlayer accepted that little hack and so did MX Player on my Nexus. So after a few hours of testing and research I was able to hardsub, encode and stream any video file whatsoever. In the end I put everything neatly in a little script so I won't have to remember everything:

TL;DR part:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
#!/bin/bash
if [[ "$2" = "" ]]
then
 skip=0
else
 skip=$2
fi
rm /tmp/bar /tmp/foo /tmp/stream
mkfifo /tmp/foo
mkfifo /tmp/bar
mkfifo /tmp/stream
# dump raw streams to fifo pipes
echo Started audio mplayer
screen -d -m mplayer -ss $skip "$1" -ao pcm:file=/tmp/bar -vo null
echo Started video mplayer
screen -d -m mplayer -ss $skip "$1" -vo yuv4mpeg:file=/tmp/foo -ao null
# encode and mux raw fifo pipe streams
echo Starting ffmpeg pipe
ffmpeg -i /tmp/bar -acodec aac -ab 192k -i /tmp/foo -vcodec mpeg4 -sameq -f matroska pipe: > /tmp/stream 2> /dev/null &
echo Started ffmpeg pipe
# server that bitch some http
echo Starting netcat httpd
(echo -e 'HTTP/1.1 200 OK\r\n'; cat /tmp/stream) | nc -l -p 8080

This script also includes another functionality: you can skip to a given time in the stream, so you don't have to watch the whole stream over and over again.

To open the stream now, direct whatever player you are using at http://<ip of the machine running this>:8080/

No comments:

Post a Comment