TiVo Community Forum banner
  • TiVoCommunity.com Ambassador Program Now Open! >>> Click Here

pyTivo - Transcoding server

2M views 6K replies 596 participants last post by  mlippert 
#1 ·
pyTivo lets you stream most videos from your PC to your unhacked TiVo. It uses the HMO server protocol, so no hacking of the TiVo is required.

* It will guess if your video is 4:3 or 16:9 and pad your video if it thinks it is needed (looked good on my tests).
* It will not transcode an mpeg that is supported by your TiVo.
* It will stream TiVo recordings (.tivo).

http://pytivo.armooo.net/

* Master changes
* Subfolder changes
 

Attachments

#877 ·
MasterCephus said:
That did it...maybe that should be added to the wiki over at the other site so people will know about that...

I appreciate your help!
No problem, glad to help. I had never tried to run the pytivo service until I read about everyone having trouble. So I gave it try myself and found I had the same problem. I don't remember what I typed on the command line to get the options to show up but saw the debug option and gave it a try and it worked. I like to try and solve problems when if I can and got lucky this time.

Windracer seems to like a puzzle as well. Now that I re-read his post, he even said at the time that guid can be specified in the conf file. I missed that. Thanks bud.

Cheers!
 
#879 ·
PaulS said:
I finally got both pyTivo services up and running.

The only issue I'm still seeing now on the 1280x720 transfers is the audio lagging over the course of playback. At the start, audio and video are synched up. Over the course of playback, the audio will slowly slip. At the end of an hour, the audio's probably lagging the video by about a second or so.
After considering running two servers I eventually just decided to run one HD one, and just let all my SD files get upconverted to 1280x720. Sure it takes a while longer to encode and will take up more space on the S3's drive, but well *shrug*. :)

Regarding audio sync, as an experiment try adding this to your ffmpeg command line: "-async 48000". I'm also experimenting with HD audio sync at the moment, but it is hard because my PC is so old I can't play back the source files properly, and can't tell if any audio sync issues I see on the S3 are due to the transcoding or were in the original source...
 
#880 ·
One other experiment I want to try is removing the part of the ffmpeg command line where it forces all the output to a framerate of 29.97 fps instead of just leaving it the same as the input file. Many of my input files use ~24 fps. Anyone tried removing this option from the command line?
 
#881 ·
I could not find a way to start pyTivoService in below normal priority. But I found a way to launch pyTivo.py that works just the same. This will result in ffmpeg running in below normal priority so that you can actually still use your computer while it is encoding.

You will need to create 4 text files in your pytivo folder. You can put them anywhere, but this example uses the pyTivo application folder on drive F. You will have to adjust the paths to match your configuration.

1. create file PyStart.bat. Edit the file and copy/paste the following:
wscript.exe "f:\pytivo\hidden.vbs" "f:\pytivo\py.bat"

2. create file hidden.vbs. Edit the file and copy/paste the following:
CreateObject("Wscript.Shell").Run """" & WScript.Arguments(0) & """", 0, False

3. create file Py.bat. Edit the file and copy/paste the following:
f:\pytivo\pylow.bat

4. create file Pylow.bat. Edit the file and copy/paste the following:
start /belownormal /b c:\python25\python.exe f:\pytivo\pytivo.py -i input.dat

Or download the files I've attached.

That's it. Double click on pyStart.bat. pyTivo will start in a hidden window at below normal priority. You may want to put a shortcut in your Windows Startup folder that points to pyStart.bat so that it will load every time you start your computer. I have only tested this with Windows XP.

If you are running multiple instances of pyTivo, just add extra lines to PyLow.bat to launch each additional pyTivo server.

To forcefully terminate any open instances of pyTivo/python each time you start pyTivo, add the following line to the top of pylow.bat.
taskkill /IM python.exe /FI "pid gt 0" /F

When you want to run pyTivo at low priority, but not in a hidden window, just double click on pylow.bat.
 

Attachments

#883 ·
mutatedmonkey said:
I have a P4/2.4GHz with 512MB of RAM.

Running pyTivo 2.1 under Ubuntu, I can transcode video to my Tivo faster than real time, and in fact, almost 2x (~35-40fps)

With the same hardware under FreeBSD 6.2, I can only get about 20fps transcoding the same exact video.

What gives?
That's interesting, I was wondering why performance on my FreeBSD box was so low (15 fps or so 2Ghz CPU). One thing I notice is that ffmpeg only gets about 3/5 of the CPU time, the other 2/5 or so goes to pyTivo. Isn't pyTivo just spawning a system process of ffmpeg using subprocess and therefor it should be nearly idle during transcoding? Maybe something is broken/different with regards to the BSD Python port and that accounts for our increased CPU and in turn lower frame rates. Is this behavior seen on flavors of Linux?
 
#887 ·
I use the same box for both FreeBSD and Linux to figure out which one will work best for a file server/bittorrent downloader/pytivo transcoder.

I just use two 40GB hard drives, with either FreeBSD 6.2 on one and Ubuntu Server 7.10 on the other.

The Python pytivo process definitely takes much less CPU when transcoding under Linux.
 
#889 ·
ScottE said:
The ffmpeg line I'm using is:

cmd = [FFMPEG, '-threads', '2', '-i', inFile, '-vcodec', 'mpeg2video', '-r', '29.97'] + select_aspect(inFile, tsn) + ['-comment', 'pyTivo.py', '-acodec', 'ac3', '-ab', AUDIO_BR,'-ar', '48000', '-f', 'vob', '-b_strategy', '1', '-bf', '2', '-coder', 'vlc', '-mbd', 'rd', '-qmin', '1', '-maxrate', '17M', '-bufsize', '1024k', '-b', VIDEO_BR, '-' ]

This works great for me (~45fps) on my E6600 (Dual core 2.4GHz) running NetBSD. pyTivo.conf contains:

audio_br=448k
video_br=8Mi
width=1280
height=720

Depending on your particular system, you might see additional performance increases by adjusting the buffer size used by the pipe that ffmpeg writes to. Just below the "cmd =" line, I changed the ffmpeg call to be:

ffmpeg = subprocess.Popen(cmd, bufsize=512*1024, stdout=subprocess.PIPE)

(added the "bufsize=" argument)

When streaming to Tivo, this reduces the CPU time consumed by the Python interpreter dramatically.

[/QUOTE=ScottE]

Adding the "bufsize=" etc fixed the overly high python usage under FreeBSD, but it screwed up the audio sync.

Adding -async 48000 fixed the sync issues.

Looks like all is well now.
 
#891 ·
mr2828 said:
Regarding audio sync, as an experiment try adding this to your ffmpeg command line: "-async 48000". I'm also experimenting with HD audio sync at the moment,
Where do you input the ffmped command line?
 
#893 ·
Hi all,

I've been trying to get some wmv files to transcode and download to my TiVo. No luck. Here's the output:

Code:
FFmpeg version CVS, Copyright (c) 2000-2004 Fabrice Bellard
Mac OSX universal build for ffmpegX
  configuration:  --enable-mp3lame --enable-gpl --disable-vhook --disable-ffplay --disable-ffserver --enable-a52 --enable-xvid --enable-faac --enable-faad --enable-amr_nb --enable-pthreads --enable-x264 
  libavutil version: 49.0.0
  libavcodec version: 51.9.0
  libavformat version: 50.4.0
  built on Apr 15 2006 08:41:02, gcc: 3.3 20030304 (Apple Computer, Inc. build 1666)

Seems that stream 1 comes from film source: 1000.00 (1000/1) -> 25.00 (25/1)
Input #0, asf, from '/Volumes/Innie 300/russellneufeld/Movies/TV Shows/jkluc1.wmv':
  Duration: 00:03:23.1, start: 5.000000, bitrate: 187 kb/s
  Stream #0.0: Audio: wmav2, 16000 Hz, stereo, 20 kb/s
  Stream #0.1, 25.00 fps(r): Video: WMV3 / 0x33564D57, 384x288
Output #0, vob, to 'pipe:':
  Stream #0.0, 29.97 fps(c): Video: mpeg2video, yuv420p, 544x480, q=2-31, 4096 kb/s
  Stream #0.1: Audio: mp2, 44100 Hz, stereo, 192 kb/s
Stream mapping:
  Stream #0.1 -> #0.0
  Stream #0.0 -> #0.1
[mpeg2video @ 0x45a630]bitrate above max bitrate
Error while opening codec for output stream #0.0 - maybe incorrect parameters such as bit_rate, rate, width or height
Anyone got any ideas? Stupid question - do I not have wmv support in my version of ffmpeg? (I haven't checked if there's a --enable-wmv option.)

Here's the relevant lines from debug.tsxt:

Code:
select_aspect: File
= /Volumes/Innie 300/russellneufeld/Movies/TV Shows/jkluc1.wmv  Type= WMV3 / 0x33564D57  width= 384  height= 288  fps= 25.00  millisecs= 203100  ratio= 133  rheight= 3  rwidth= 4 
select_aspect: File is within 4:3 list.
transcode: ffmpeg command is  /Applications/ffmpegX.app/Contents/Resources/ffmpeg-i/Volumes/Innie 300/russellneufeld/Movies/TV Shows/jkluc1.wmv-vcodecmpeg2video-r29.97-b4096K-maxrate17M-bufsize1024k-aspect4:3-s544x480-commentpyTivo.py-ac2-ab192K-ar44100-fvob-
Thanks in advance,

Russ
 
#894 ·
ScottE said:
The ffmpeg line I'm using is:

cmd = [FFMPEG, '-threads', '2', '-i', inFile, '-vcodec', 'mpeg2video', '-r', '29.97'] + select_aspect(inFile, tsn) + ['-comment', 'pyTivo.py', '-acodec', 'ac3', '-ab', AUDIO_BR,'-ar', '48000', '-f', 'vob', '-b_strategy', '1', '-bf', '2', '-coder', 'vlc', '-mbd', 'rd', '-qmin', '1', '-maxrate', '17M', '-bufsize', '1024k', '-b', VIDEO_BR, '-' ]

This works great for me (~45fps) on my E6600 (Dual core 2.4GHz) running NetBSD. pyTivo.conf contains:

audio_br=448k
video_br=8Mi
width=1280
height=720

Depending on your particular system, you might see additional performance increases by adjusting the buffer size used by the pipe that ffmpeg writes to. Just below the "cmd =" line, I changed the ffmpeg call to be:

ffmpeg = subprocess.Popen(cmd, bufsize=512*1024, stdout=subprocess.PIPE)

(added the "bufsize=" argument)

When streaming to Tivo, this reduces the CPU time consumed by the Python interpreter dramatically.

I've put those two files (my transcode.py and pyTivo.conf) at http://intrepid.warped.com/~scotte/visible/pyTivo/ in case my explanation above doesn't make any sense. :)
By the way, thanks for providing the ffmpeg commands for the ac3 and extra video strategies. It's been very helpful. And it eliminated the audio frame errors reported by VideoReDo when re-writing the Tivo file to an mpg. The errors might be caused by minimum GOP lengths of 1 output by the default pyTivo command while your command results in minimum length 2. At least that's what I noticed in PVAStrumento. EDIT: Doh, should have tested more files. Resync errors are obviously quite common, especially so when altering original framerate.

After testing different bitrates I found that 8Mi or 14Mi both yield the same output file with basically the same bitrates and quality as reported by ffmpeg. My strategy of trying to adjust the bitrate to improve quality was obviously flawed since ffmpeg tries to compress as much as it can regardless of bitrate setting. So I looked for an alternative and found the -qscale setting in the ffmpeg documentation. Which leads me to my question.

Have you tried the qscale setting? I found that by just deleting the bitrate setting ('-b', video_br) from pyTivo's ffmpeg command and replacing it with ('-qscale', '1.4'), it yields an output with equal or better "q" average and a higher average bitrate than using your command at bitrate 8Mi with b_strategy etc... And encoding takes the same amount of time. The difference in file size between the two commands is also negligable. I've only tried it on a few one minute clips but it looks promising. One note of caution though, qscale settings above 1.4 result in the same audio frame errors I described above. You can use settings as low as 0.1, but I doubt there would be enough benefit to justify the extra encoding time.

By only specifying qscale and not bitrate, it appears that ffmpeg uses as high a bitrate as it needs to try to achieve the qscale setting specified, but also adheres to the maxrate 17M setting. Sounds perfect for HD Tivo users who are more concerned about quality than compression. If that is how it works, armooo may want to consider adding a High_Quality option to the conf. If the user says yes, I want highest quality rather than compression, replace the bitrate command with qscale=1.3.

Another benefit to this approach is that it should minimize any additional loss in quality when re-encoding low quality video sources.

Unless you know of any drawbacks to using qscale, I think I'll be replacing my various bitrate servers with a single qscale=1.3 server.
 
#898 ·
is it just me or do the files that you transfer to a S3 using pyTivo make it so the fast forward or rewind then hitting play to resume is on a different scale than shows recorded on the tivo itself. I fast forward 3 times and then hit play and it's way off from the location where I hit play. It's almost as if the offsets are magnified. I end up having to rewind for almost a minute or two worth of video content to get it to the point where it should have resumed. My workaround is to use the slower fast forward and it's a little easier to control, but still acts a little quirky.

Just wondering if I'm the only one with this problem.

David
 
#899 ·
s2kdave said:
is it just me or do the files that you transfer to a S3 using pyTivo make it so the fast forward or rewind then hitting play to resume is on a different scale than shows recorded on the tivo itself. I fast forward 3 times and then hit play and it's way off from the location where I hit play. It's almost as if the offsets are magnified. I end up having to rewind for almost a minute or two worth of video content to get it to the point where it should have resumed. My workaround is to use the slower fast forward and it's a little easier to control, but still acts a little quirky.

Just wondering if I'm the only one with this problem.

David
The files that I've transferred over seem to behave correctly. I'm using ScottE's settings, and all is well.

What resolution, bandwidth and settings are you using ?
 
#900 ·
The FF/RW effect s2kdave described has happened to my transcoded (divx/xvid)+(mp3/ogg) avi's on my S2DT. This is with the '201' version that was the current main build until this month, as well as the subfolder branch.

This is wild speculation, but could it have anything to do with the number of key frames ffmpeg will output to the stream?
 
#901 ·
Yes, what I was describing was not the release that just came out a couple days ago. I'll have to double check the version when I get home. I'll also try the new version and see if I get the same problems. It was a transcoded avi that I had the problems with too. Not sure offhand what the audio and video codec was for the avi though. I believe it was an xvid.
 
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Top