In this final post I’ll show how I made the camera video feed available on the Internet.

One thing I wanted from the outset was for Internet clients not to make direct connections to the camera itself. I was a little worried about the ability of the web server and CPU in the camera to cope with multiple clients, and also the security implications of direct access. A second requirement was to have multi platform access – that is, desktop and iOS. This potentially means different streaming video formats.

We have one linux server in the house, which is used for many different things and runs virtual machines. My back-of-an-envelope plan looked something like this:

First step was to create the VM, but remember that the camera feed is in a DMZ using a VLAN, so the VM must live there too. In KVM it’s possible either to send all traffic to a guest system and let it process the VLANs or, you can separate the tagged VLAN traffic in the host system so the guest is dumb and just sees untagged frames. Clearly the latter is preferable so that were the guest to suffer attack from the Internet, it ought not to be able to put traffic onto the house workstation network. The guest is completely within the DMZ.

With that done and a basic Ubuntu system installed, I started work on Apache and VLC (the Swiss Army Chainsaw of video processing). First up, VLC…

Luckily the camera’s video feed comes in MJPEG format with a discoverable URL. The idea is to take this feed, duplicate it, and transcode the respective feeds into something suitable for a desktop browser and for iOS. As a bonus, I’ll timestamp the video to make it easy to tell if the transcoder has crashed (the timestamp would be wrong). After a lot of reading online about how to configure VLC I came up with the following monstrosity:

/usr/bin/cvlc -I dummy http://guest:guest@172.16.30.10:8888/videostream.cgi?rate=0
  --sout='#duplicate{

    dst="transcode{
      width=320,heigh=240,fps=25,vcodec=h264,vb=256,acodec=none,
      venc=x264{profile=baseline,level=30,keyint=30,ref=1},
      sfilter=marq{marquee=\"[%Y-%m-%d %H:%M:%S]\",position=8,size=18}
    }:std{access=livehttp{
        seglen=10,delsegs=true,numsegs=5,
        index=/var/www/streaming/cats.m3u8,
        index-url=/streaming/cats-########.ts},
      mux=ts{use-key-frames},
      dst=/var/www/streaming/cats-########.ts}",

    dst="transcode{
      width=640,heigh=480,fps=25,vcodec=theo,vb=512,acodec=none,
      sfilter=marq{marquee=\"[%Y-%m-%d %H:%M:%S]\",position=8,size=18}
    }:http{mux=ogg,dst=127.0.0.1:8081/catcam.ogg}"

  }'

Of the two transcodes (“dst=”), the second is more straightforward. It creates an Ogg format stream using the Theora video codec, which modern browsers should be able to cope with. This is a video stream being served from VLC’s built-in web server, so I’ll need to proxy it via Apache. The configuration also applies a filter (“sfilter=”) to add a timestamp on the video stream.

The first transcode uses the new HTTP Live Streaming support in VLC. This is a rather elegant specification from Apple (which is why I selected it for the iOS clients) for simple and efficient delivery of streaming video. It creates a set of files and assumes you have a web server to serve them. The files each contain a few seconds of video, and the client retrieves them and plays one after another. The “######” templates an incrementing number within the segment filename. Again, the timestamp is added to the video stream.

CPU load for the above runs at about 60% (in the VM) on the dual core Athlon X2 245e processor. I wrapped the above in an Upstart init file, and just in case VLC gets its knickers in a twist, I added a cron job to periodically stop and start the service.

Now on to Apache. It needs to proxy the Ogg stream and serve the Live Streaming files, and prevent any other access to the web server:

# redirect any non-cat requests to the cat index.html
RewriteEngine on
RewriteCond %{REQUEST_URI} !^/streaming/cats.*
RewriteCond %{REQUEST_URI} !^/stream/catcam.ogg$
RewriteCond %{REQUEST_URI} !^/index.html$
RewriteRule ^(.*) http://%{HTTP_HOST}/index.html [R,L]

ProxyReceiveBufferSize 16384
ProxyRequests On
ProxyVia On
ProxyPreserveHost On

<Proxy *>
    Order deny,allow
    Allow from all
</Proxy>

# VLC server stream
ProxyPass /stream/catcam.ogg http://localhost:8081/catcam.ogg
ProxyPassReverse /stream/catcam.ogg http://localhost:8081/catcam.ogg

Last but not least for this server, we need a web page which offers up the two video streams. This uses an HTML5 video tag:

<!DOCTYPE html>
<html>
    <head>
        <title>Cat Cam</title>
        <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
        <META HTTP-EQUIV="Pragma" CONTENT="no-cache">
    </head>
    <body>
        <h1>Cat Cam</h1>
        <video id="video" autoplay="autoplay">
            <source src="/streaming/cats.m3u8">
            <source src="/stream/catcam.ogg" type="video/ogg; codecs=theora">
            Your browser doesn't appear to support the HTML5 <code>&lt;video&gt;</code> element.
        </video>
    </body>
</html>

All that remains is to enable a NAT rule and firewall pinhole on the home router for the web server (which is, of course, in the DMZ network connected directly to the router).

Let’s see the end result, taken on my iPhone this evening, also demonstrating the automatically activated night vision mode:

It’s nice to be able to check in on the wee beasties when I’m out at work. Other than a lot of reading about VLC, it wasn’t particularly difficult to do, and I think the end result is really quite good.

• The complete series of articles is available at this page

Last time we went through the steps to transcode a recording into the H.264 format, required for playback on iPad (or other Apple devices – iPod, iPhone, Mac OS X, etc).

Today we’ll complete the series by embedding an HTML5 Video player into MythWeb which will stream these files. Of course, this means playback isn’t only possible on iPad, but also any compatible browser platform. The original plan was simply to provide a file download link in MythWeb, but my friend Colin rightly suggested an embedded HTML5 Video player would be much more awesome.

Your MythWeb files are probably installed somewhere like /usr/share/mythtv/mythweb. Open up modules/tv/tmpl/default/detail.php and replace the default embedded player with a new HTML5 Video tag. You should find this around line 797. Replace the two lines comprising the Direct Download hyperlink with the following:

<video controls preload="none" width="360" height="153">
  <source
    src="/h264xcode/<?php echo $program->chanid ?>_<?php echo date( 'YmdHis', $program->recstartts ) ?>.m4v">
  Your browser does not support the video tag.
</video>

The above requires that you make the H.264 files available through your MythWeb web server. In my case, I simply created a symlink from the server’s DocumentRoot to where the H.264 files live (so, change this to reflect your own H.264 file location):

ln -s /mnt/mythtv-video/Videos/iPad /var/www/h264xcode

The result is the following, once the transcoded file appears on disk:

And we’re done! Browsing to the MythWeb site on an iPad shows the embedded player which, when clicked, opens fullscreen for native playback. My deep thanks to all those working on the software used in this series of articles, and I also hope you found it a useful read.

• The complete series of articles is available at this page

Last time we went through the steps to transcode a recording into the H.264 format, required for playback on iPad (or other Apple devices – iPod, iPhone, Mac OS X, etc).

Today we’ll complete the series by integrating this with MythWeb so that a new button appears which sets off the transcode job, and then go on to make the output available for streaming over the web. Remember that there are other ways to watch recordings on the iPad, as listed in the previous article in this series.

First we need to let MythTV know about the new user job command (/usr/local/bin/h264xcode). You can do this either in the mythtv-setup program, or via the MythWeb interface. Personally I use the latter, as stopping the MythTV Backend and running the setup program over an SSH tunnel is fiddly, but I’ll cover both here for completeness.

In mythtv-setup you want to select General and then hit Next until you reach the Job Queue (Job Commands) page:

You can see I’ve filled in the description and command for User Job #1. The command uses parameter variables which are filled in by MythTV (a list of the variables is on the User Jobs MythTV wiki page). Here’s the command in full, as it’s not visible in this screenshot:

/usr/local/bin/h264xcode %CHANID% %STARTTIME%

Alternatively, configure these settings in the MythWeb interface. Click on the big spanner (for Settings) and choose MythTV from the list of buttons on the left. Scroll down and you’ll find a field called UserJob1 and another called UserJobDesc1 just a little below. These take the same values as in the mythtv-setup interface:

Don’t worry about the dummy values in other User Job description fields – unless the Job has a command set, it’s ignored. If you did use MythWeb, you must now restart your MythTV Backend process to get these updated settings.

Hop over to MythWeb, and you should now see a new “H.264 Encode” button for each recording, which fires off a transcode Job that’s also visible in the MythWeb Job Queue:

If you have the disk space, and want this to happen for all recordings automatically, check out the instructions from part 2 of this series. You should tick the Run user job box in MythFrontend settings, and also note that this only takes effect for new Recording Schedules, although I explain a fix for existing schedules at the end of the article. Don’t forget to clean up old transcode files after a while!

The final step on my road to iPad streaming is to embed this video into MythWeb, which I’ll cover in the next article.

• The complete series of articles is available at this page

Last time I showed where you go in MythTV to edit the default job settings, and what happens when you hit the Transcode button in MythWeb or let MythTV transcode automatically after recording.

Today I’m going to continue the series by leaving MythTV to one side, and showing how to create a transcoded recording suitable for playback on an iPad (or other Apple device – iPod, iPhone, Mac OS X, etc). Next time we’ll incorporate this into MythWeb for the final result.

Before that, let’s look at the different ways that playback on Apple platforms might be achieved (more suggestions are welcome, in the comments):

• Install the MythFrontend application natively onto an OS X device.
• Install the VLC Media Player onto an OS X device (as it can handle the ASX Stream link in MythWeb) – in fact this is what I do on my own desktop Mac.
• Install MythPodcaster, a Transcoder and RSS publisher, and subscribe to transcoded recordings in iTunes (leads to syncing them to the iPad).
• Create a MythTV Job to transcode into H.264 format (compatible with all Apple platforms), and add links to the generated files in MythWeb.

It’s the last of these which I’m going to cover in this article. However an honourable mention must be made of MythPodcaster. First, most of the work here is taken from looking at what that excellent tool does. Second, if you don’t want to mess about editing MythWeb, then MythPodcaster has its own web interface for not only managing transcode schedules and profiles, but also browsing and playback. It really is an excellent tool I recommend you check out.

Each MythTV user job is configured as a command which can be executed at the shell command-line. That means the first thing to do is get transcoding working at the command-line for ourselves. The tool for this is ffmpeg, but we must compile a special version which can output H.264. The following instructions are for an Ubuntu system and based on the MythPodcaster InstallationOnMythBuntu page.

I’m going to build a custom version of ffmeg, separate to that installed with my Ubuntu system. The instructions below aren’t intended to be the cleanest solution, but get the job done.

sudo apt-get install build-essential git-core yasm checkinstall zlib1g-dev pkg-config

And we’ll do this work as the mythtv user in a new directory:

sudo mkdir -p /usr/local/lib/ffmpeg-264
sudo chown mythtv:mythtv /usr/local/lib/ffmpeg-264
sudo su - mythtv
cd /usr/local/lib/ffmpeg-264

Build the faac library:

cd /usr/local/lib/ffmpeg-264
wget http://downloads.sourceforge.net/project/faac/faac-src/faac-1.28/faac-1.28.tar.gz
tar -zxvf faac-1.28.tar.gz
cd faac-1.28
./configure --prefix=/usr/local/lib/ffmpeg-264 --enable-static --disable-shared
# some kind of gcc-compatibility bug which needs fixing
sed -i -e "s|^char \*strcasestr.*|//\0|" common/mp4v2/mpeg4ip.h
make && make install

Build the x264 library:

cd /usr/local/lib/ffmpeg-264
git clone git://git.videolan.org/x264.git
cd x264
./configure --prefix=/usr/local/lib/ffmpeg-264 --enable-static --disable-shared
make && make install

Build the libmp3lame library:

cd /usr/local/lib/ffmpeg-264
wget http://downloads.sourceforge.net/project/lame/lame/3.98.4/lame-3.98.4.tar.gz
tar -zxvf lame-3.98.4.tar.gz
cd lame-3.98.4
./configure --prefix=/usr/local/lib/ffmpeg-264 --enable-static --disable-shared
make && make install

Build ffmpeg:

cd /usr/local/lib/ffmpeg-264
wget http://www.ffmpeg.org/releases/ffmpeg-0.8.3.tar.gz
tar -zxvf ffmpeg-0.8.3.tar.gz
cd ffmpeg-0.8.3
./configure --prefix=/usr/local/lib/ffmpeg-264 --extra-version=static --disable-debug --enable-static --extra-cflags=--static --disable-ffplay --disable-ffserver --disable-doc --enable-gpl --enable-pthreads --enable-postproc --enable-gray --enable-runtime-cpudetect --enable-libfaac --enable-libmp3lame --enable-libx264 --enable-nonfree --enable-version3 --disable-devices --extra-ldflags=-L/usr/local/lib/ffmpeg-264/lib --extra-cflags=-I/usr/local/lib/ffmpeg-264/include
make && make install

To make life a little easier, I link to this new binary from a more memorable location:

exit # if you are still the "mythtv" user
sudo ln -s /usr/local/lib/ffmpeg-264/bin/ffmpeg /usr/local/bin/ffmpeg-264

Now we need a magic incantation for ffmpeg-264 which will result in an H.264 file suitable for iPad playback. A good source of possible command line options is the MythPodcaster trancoding profiles XML file. In this file, the various iPad profiles, as their names suggest, will ask for various formats and qualities of output (see each encoderArguments key).

These command line options in turn refer to ffmpeg presets (via -vpre) which should be shipped with ffmpeg in the ffpresets directory. Some of these might work, but I’m pretty happy with two presets from an older version of ffmpeg:

mkdir ~mythtv/.ffmpeg
cd ~mythtv/.ffmpeg
wget https://raw.github.com/FFmpeg/FFmpeg/release/0.6/ffpresets/libx264-main.ffpreset
wget https://raw.github.com/FFmpeg/FFmpeg/release/0.6/ffpresets/libx264-medium.ffpreset

One problem you might have with ffmpeg options is selection of an audio track in recordings with multiple soundtracks (for example normal, and audio-descriptive). There’s no easy way to know which you want (and in my settings I force the choice). Here’s what I settled on, which seems to work OK for SD content, and if you have others please add a comment, below:

-i <infile> -y -map 0:0 -map 0:1 -er 4 -f ipod -acodec libfaac -ac 2 -ab 128k \
    -ar 44100 -deinterlace -vcodec libx264 -vpre medium -vpre main -b 1200k \
    -bt 1200k -maxrate 1200k -bufsize 1200k -level 30 -r 30 -g 90 -async 2 \
    -threads 0 -v 0 <outfile>

These parameters are wrapped up in a script, making things easier for MythTV configuration. This file, called h264xcode is also placed in /usr/local/bin (and obviously, season this to taste, for your system’s recording directories):

#!/bin/bash

CHANID=$1
STARTTIME=$2

INDIR="/mnt/mythtv-video/Videos/MythTV"
OUTDIR="/mnt/mythtv-video/Videos/iPad"

PROG="time nice -n 19 /usr/local/bin/ffmpeg-264"
PARAMS="-y -map 0:0 -map 0:1 -er 4 -f ipod -acodec libfaac -ac 2 -ab 128k -ar 44100 -deinterlace -vcodec libx264 -vpre medium -vpre main -b 1200k -bt 1200k -maxrate 1200k -bufsize 1200k -level 30 -r 30 -g 90 -async 2 -threads 0 -v 0"

LOG="/var/log/mythtv/h264xcode.log"
OUTFILE="${OUTDIR}/${CHANID}_${STARTTIME}"

COMMAND="${PROG} -i ${INDIR}/${CHANID}_${STARTTIME}.mpg ${PARAMS} ${OUTFILE}.m4v.tmp"

echo "h264xcode: ${COMMAND}" >> $LOG
$COMMAND >> $LOG 2>&1
mv -f ${OUTFILE}.m4v.tmp ${OUTFILE}.m4v >> $LOG 2>&1
rm -f ${OUTFILE}.m4v.tmp >> $LOG 2>&1

Let’s work through what’s going on, here. The script takes two arguments – the channel ID and the start time of the recording (we’ll see how MythTV passes these, later). From them it’s possible to locate the original recording file on disk, using the $INDIR variable.

Next a simple log is set up, and the proposed ffmpeg-264 command echoed to that file. Finally the command itself is run, with a low system priority (nice -n 19), and the output directed to the log file. The H.264 file is saved in $OUTDIR.

So far so good. All that remains is to let MythTV know about this new user job command, which we’ll cover in the next article in this series.

Many thanks to Murray Cox, Seb Jacob, and Colin Morey for providing valuable feedback on this post!

• The complete series of articles is available at this page

Last time I enabled transcoding on the back-end server, and showed that hitting the Transcode button in MythWeb fired off a job. Today I’m going to show what that job is doing.

If you have the MythTV Frontend running then you’re all set to begin. Otherwise, log into your MythTV box and start the Frontend application (these are instructions for 0.24):

workstation:~$ ssh -Y mythtv.example.com
mythtv:~$ mythfrontend --windowed --geometry 640x480

Navigate to Utilities/Setup, Setup, TV Settings, General, and hit Next until you end up on the General (Jobs) screen:

First settle your eye on the Default transcoder option in the centre of the screen. There are a number of predefined Transcoders (which we’ll come to shortly), and this is where you choose the one to be run when the Transcode button is pushed in MythWeb.

Below this option are tick boxes for each kind of Job that we saw in my last blog post. Whereas before we simply permitted them to be run manually, or not, these options control whether this is done automatically. However this applies to newly created Recording Schedules only. For example if you already have a Schedule for a TV series, new episodes won’t be auto-transcoded, we need to fix each existing Schedule, and I’ll show how to do that below.

Now we know where to choose the default Transcode profile and enable automatic transcoding for new recordings, let’s look at these mysterious profiles. Navigate to Utilities/Setup, Setup, TV Settings, Recording Profiles, Transcoders to end up on this screen:

You might have spotted that two profiles are called Autodetect from…, and indeed these are used when the Default transcoder we saw above is set to Autodetect. Selecting the Autodetect from MPEG2 profile shows it’s quite simple:

So why have a profile which seems to do nothing? Two reasons:

1. If you have previously run a Commercial-detection Job, the cutpoints for ad breaks will be stored, and the in-place transcode is an opportunity to remove those sections of the video.
2. Sometimes MPEG recordings can contain “glitches” or MythTV has trouble seeking within the video; an in-place transcode is an opportunity to rebuild all the metadata, which helps resolve problems.

It’s for the second reason that I have automatic in-place transcoding enabled even though I don’t use commercial detection. If you want to transcode to save disk space, take a look at the other Transcoders available, and consider either setting one to be the Default transcoder or creating your own.

All that remains is for me to fix the automatic transcode setting for my existing recordings, to be in line with new recordings. The only way I know is by editing the backend database directly, but this is strongly discouraged by the MythTV developers. Anyway, at the mysql prompt (get the password from /etc/mythtv/mysql.txt):

mythtv:~$ mysql -u mythtv -p mythconverg
Enter password:

mysql> update record set autotranscode = 1;
Query OK, 72 rows affected (0.11 sec)
Rows matched: 72  Changed: 72  Warnings: 0

If you want to do the same for Commerical-detection, set the autocommflag field.

Next time I’ll take a break from MythTV and look at how to transcode an MPEG2 file for the iPad using command line tools.

• The complete series of articles is available at this page

What is transcoding? The data saved to disk by MythTV from your TV tuner cards has both a format, and a bit-rate. These are usually particular to how the TV station is broadcast and/or how your tuner hardware works. The idea of transcoding is to change either or both properties, perhaps to enable playback on another device, or to save on disk space used.

MythTV includes a transcoding system and support for some data formats and bit-rates (but not all). In this first session I’ll enable really basic, manual transcoding.

What catches people out is that just hitting the Transcode button in MythWeb will not work unless the backend is allowed to transcode. That’s done in mythtv-setup, the program you probably ran years ago and forgot all about (or never ran).

Log into your MythTV box (these are instructions for 0.24):

workstation:~$ ssh -Y mythtv.example.com
mythtv:~$ mythtv-setup --geometry 640x480

Select General and then hit Next until you reach the Job Queue (backend specific) page:

The notable settings are the six Allow … Jobs tick-boxes. As you can see I have commercial-detection disabled, and the effect is that even if I hit the Flag Commercials button in MythWeb the job will never start (it will just sit in the job queue). There are three types of job:

Edit: updated descriptions due to feedback from a friend – thanks, Colin!

• Commercial-detection: this will attempt to detect and store the commercial break points in TV shows (not a transcode, but may be used by one)

• Transcoding: what you set to happen when you hit Transcode in MythWeb, which (usually) replaces the original recording file
  

• Four User Jobs: scripts you create that transcode (or anything else), which we’ll come to later, and typically don’t replace the original recording file

Hitting Next takes you to the Job Queue (Global) transcode settings page:

An interesting option is Save original files after transcoding (globally). If you transcode and intend to replace the original recording file (as in bullet two, above), this is a way to preserve a backup should things go wrong. If you do, just make sure there’s a cron job or similar to delete these old files as otherwise your disk will fill up.

Finally, hitting Next will show the third transcode-related page where you can configure the User Jobs I mentioned in bullet three, above. I’ll save these for another blog post to come.

So far I’ve allowed transcode jobs but not actually asked MythTV to to run any. Bringing up the MythWeb page for a recorded show (click the thumbnail from Recorded Programs) presents a Transcode button which now works:

For a different show I recently used this button, and on the Backend Status page there’s a corresponding Job Queue section entry showing its progress:

Next time I’ll show what type of transcode is happening, and how to automate this to occur immediately after each recording finishes.