1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

pyTivo - Transcoding server

Discussion in 'TiVo Home Media Features & TiVoToGo' started by armooo, Nov 25, 2006.

  1. epstewart

    epstewart Member

    648
    0
    Mar 1, 2003
    Hi,

    I have a problem with having pyTivoX push HandBrake h.264 output from my Mac to my TiVo Series3. When I do, I get video that is "jumpy" or "jerky" looking. It seems to "jump" or "jerk" slightly at regular intervals of a split second or so. Everything else is normal: audio is fine and in sync with video.

    When I pull the same HB video output file from the TiVo Now Playing list to the TiVo, the video plays perfectly smoothly. I believe that in that scenario pyTivoX is transcoding the video from h.264 to MPEG-2. But in the push scenario (whether initiated from the TiVo or from my computer's web browser; it makes no difference) I believe my h.264 <filename>.mp4 file that was output from HB is left in its original form, and the h.264 decoder on the TiVo is causing playback to be "jumpy/jerky."

    I'm using pyTivoX v. 1.3, incorporating StreamBaby, and HandBrake's GUI version for Mac v. 0.9.4 (not the CLI version). In HB I'm using settings copied from Tivo Native Video Compatibility, specifically under the heading "Handbrake encoding H.264 mpeg4 with AC3 audio for Tivo (from mpeg2 source)."

    Specifically, I'm using

    Code:
    ref=3:mixed-refs=1:bframes=3:b-pyramid=1:weightb=1:analyse=all:8x8dct=1:me=umh:subq=9:psy-rd=1,0.2:direct=auto:keyint=24:min-keyint=2:no-fast-pskip=1:no-dct-decimate=1
    in the HB Advanced Settings pane, Current x264 Advanced Option string. I copied those from the Tivo Native Video Compatibility page.

    In the HB Video pane, I'm using

    Format: MP4 file
    Video Codec: h.264 (x264)
    Framerate (FPS): Same as source
    Quality: Average bitrate (kbps): 5000

    In the HB Picture Settings window, under Size I'm using Anamorphic: Strict, which disables the other settings so they are at their default values. I don't fiddle with the Filters pane of the Picture Settings window.

    In the Audio panel of the HB main window I select as the Audio Codec AC3 Passthru.

    Can someone offer me some tips on how to get HandBrake to produce h.264 output that won't look "jumpy/jerky" when pushed to a TiVo? Many thanks in advance ...
     
  2. wmcbrine

    wmcbrine Ziphead

    10,368
    22
    Aug 2, 2003
    Interesting. No, it turns out it's not the metadata parser that's interpreting it as octal; it's the TiVo.

    The episodeNumber field was just being passed through as a string. A while back, I found that the TiVo would only accept integers, and a bad episodeNumber could mess up other fields. So I attempted to deal with this with this bit of code:

    Code:
    if 'episodeNumber' in data:
        try:
            int(data['episodeNumber'])
        except:
            data['episodeNumber'] = '0'
    but notice that this only resets the episodeNumber if it fails to convert to an integer -- if it passes, the original text is left unaltered. Well, there's only one kind of string that will pass the int() test, but come out oddly on the TiVo, and you found it; congratulations. :) Python says int('0302') = 302, but the TiVo thinks <episodeNumber>0302</episodeNumber> means 194.

    So, I've changed the code to:

    Code:
    if 'episodeNumber' in data:
        try:
            ep = int(data['episodeNumber'])
        except:
            ep = 0
        data['episodeNumber'] = str(ep)
    in my repo now. Thanks.
     
  3. epstewart

    epstewart Member

    648
    0
    Mar 1, 2003
    I'm using pyTivoX v. 1.3, and I have found that sometimes when I push an h.264 file to my TiVo Series3, although the push seems to be completed, the file doesn't actually get sent to my TiVo until much, much later -- not two or three minutes later, but an indeterminate amount of time later.

    Today I was experimenting with different HandBrake options and tried pushing an output file. It didn't seem to arrive any time soon after it had been successfully pushed, so I gave up on it. I went about my business for a few hours and later came back and did some more experimenting. In the course of those later experiments, I noticed that my original pushes from earlier in the day were now in the Now Playing list!

    (They were all of the same file, the only difference being that the file name was in quote marks for some pushes and did not have quote marks for others. It depended on whether the push was initiated from my browser or from my TiVo, I think, but I'm not sure which kind of push added the quotes.)

    Anyway, at other times I've been able to do pushes which happen pretty close to immediately. Can anyone tell me why some pushes get long delayed? Thanks.
     
  4. Rdian06

    Rdian06 New Member

    463
    0
    Apr 12, 2008
    Transfers initiated from the pyTivo web interface are called "Pushes".

    Transfers initiated from the the Tivo's Now Playing list are called "Pulls".

    "Pulls" happen right away because all the communication is local to your home network. That is when you hit the buttons on the remote, the Tivo makes the request for the video right away.

    "Pushes" are an extension of the Tivo online schedule system. That is when you initiate the transfer, pyTivo contacts a Tivo.com server and tells it to schedule the transfer. Your Tivo checks in with the Tivo.com server periodically and picks up new assignments. Once it has the assignment, it then talks to pyTivo to begin the transfer. This is why "Push" requires your Tivo.com username/password and the delay results from waiting for your Tivo to check in with Tivo.com.
     
  5. epstewart

    epstewart Member

    648
    0
    Mar 1, 2003
    Rdian06,

    Thanks for your help. You stated very well what is crucially important for users to know about pyTivoX "pushes." I was already aware of most of it, but what I was fishing for was more along the lines of an explanation for why a push can apparently be delayed not just for a minute or two but for a much longer time.

    I had several pushes queued yesterday while I was experimenting with HandBrake encodes of DVDs. I gave up on them, thinking they had somehow failed despite apparently succeeding. Many minutes had gone by after I initiated the pushes before I gave up. Some hours later I came back and found that all of the pushes had belatedly sent the files to my TiVo. Unfortunately, I don't know exactly how long they were queued, but, again, it was longer than several minutes.

    For lurkers: I believe the Tivo.com server mentioned by Rdian06 is (for most people in the U.S.) mind.tivo.com. When you install pyTivoX on a Mac and run it, you can point your browser to http://localhost:9032/ and see a page appear that has a Web Configuration link (along with links to the shared directories you have put in pyTivoX). Clicking Web Configuration, you see a list of sections at upper left. Click on Global Server Settings, enter mind.tivo.com:8181 in the tivo_mind box and your TiVo Media Access Key number in tivo_mak. Make sure tivo_username contains the username (e-mail address) you use at tivo.com, and tivo_password contains your tivo.com password. Click the Save Changes button. When the editing page reappears, be sure to click the "Soft Reset pyTivo" button so that the changes take effect. Then you will be able to select any of your shared directories, navigate to a video, select a TiVo to push it to, and perform a push. After some indeterminate amount of time, you can go to the TiVo and find the pushed video in your Now Playing list. If the transfer is still incomplete, you should see the blue LED on the front of the TiVo.

    My basic question, then, is why the "some indeterminate amount of time" can occasionally be very long. Is it possibly because the mind.tivo.com server gets overloaded?
     
  6. wmcbrine

    wmcbrine Ziphead

    10,368
    22
    Aug 2, 2003
    There is simply no way for us to know or control what happens on mind.tivo.com. All we can do is send the requests.
     
  7. epstewart

    epstewart Member

    648
    0
    Mar 1, 2003
    Understood ... in case I need to say so, I meant no criticism &#8212; none whatever &#8212; of pyTivo or pyTivoX, which is/are great software.

    The following is possibly due to my not having done enough "homework," or not having enough experience with open source software, but I'll suggest it anyway: It would be nice to see at the pyTivo wiki (without the casual user having to dig for it) a general discussion of how pushing differs from pulling, and how pushing from the computer end, via a browser, requires the user to provide configuration parameters that aren't expressly input when the pushing is initiated from a TiVo. When do video files get transcoded, when played/transferred natively? What is the right tivo_mind parameter, and does it depend on where one lives? Questions like these are essential, and (especially for pyTivoX users) it is not obvious at first that they are important. So I suggest adding to the Automatic install: pyTivoX discussion a link to a page discussing these matters.

    Cheers ...
     
  8. Stormspace

    Stormspace Electrocuted by TiVo

    5,171
    0
    Apr 13, 2004
    Hartsville, SC
    I'm late to this party, but i had a few questions and didn't want to slog through the entire thread.

    • What video fomat(s) does it work with, or is it mpeg2 only same as with Galleon?
    • If it works with other Codec's than mpeg2, does it encode on the fly for Series2 TiVos since they require mpeg2?
    • Can it use the file format galleon uses to set Show/Episode data?
    • Does it run as a service, or do you have to be logged into the machine for it to run?
    • How much system resources does it require? Kmttg kills my media server when it's running.
    • Can it be used with a Roku? (Hopeful/Just askin) :)
     
  9. epstewart

    epstewart Member

    648
    0
    Mar 1, 2003
    In post #2747 quoted above I talked about getting jumpy/jerky video on my TiVo when I used HandBrake to make TiVo-compatible files from a DVD as recommended at Tivo Native Video Compatibility under the heading "Handbrake encoding H.264 mpeg4 with AC3 audio for Tivo (from mpeg2 source)." (I assume a DVD qualifies as an mpeg2 source.)

    I have made some experiments and found the following works OK in HandBrakeCLI — Handbrake with a command-line interface, that is; not necessarily the GUI version — on a Mac:

    Code:
    HandBrakeCLI -i <path to VIDEO_TS folder on DVD> -t 1 -o "<path and filename of output file>.mp4" -f mp4 -O -e x264 -b 5000 -2 -T -a 1 -E ac3 -s 4 -F -x ref=2:bframes=2:me=umh -v --strict-anamorphic
    The "advanced" -x options are much less complex than suggested by "Tivo Native Video Compatibility." I haven't pinned down which option or options in that recommendation caused the jerkiness, but I find the result of the HandBrakeCLI call I show above to be good enough for my purposes. The extra -x options are ones that ostensibly refine the quality of the output video, but I'm not sure they're needed.

    During my experiments I found that a large range of HandBrakeCLI options (the regular options, not the -x advanced ones) do work as expected in making video files destined for a TiVo. You should feel free to experiment; what I suggested above is not holy writ.

    I did find that omitting --strict-anamorphic gave me a video picture that was squeezed horizontally into a box with a 4:3 aspect ratio on the TiVo, though it played on my Mac in MPlayer Extended OSX with the correct dimensions. I was using a DVD that is itself anamorphically encoded at the time. I am not sure --strict-anamorphic is needed, or even works, with DVDs that are not anamorphic.

    I also found to my chagrin that it is possible for gremlins to interfere with HandBrake. I left my DVD in the drive for two nights running and started to get weird video results during my experiments — until I just happened to eject the DVD, use the optical drive for another purpose, and put the DVD back in. It might be a good idea to eject and reinsert any DVD you want HandBrake to rip, for TiVo use or otherwise, if it has been sitting in the drive for any length of time.

    Also, I got burned by failing to pay sufficient heed and originally downloading the 64-bit version of HandBrakeCLI, which gave me incomprehensible errors. My fairly recent-model iMac uses (surprise!) a 32-bit CPU. If yours does too, then make sure you get the 32-bit version of HandBrakeCLI!

    Yet more (sorry): I have a lot of DVD rips from days gone by that do not play on my TiVo. Most or all came from HandBrake and are at least two years old. I don't exactly remember what profile/options I used then. I do know that most were not ripped to play on my iPod, even — now replaced by an iPhone — and I found out at some point the the iPod/iPhone is persnickety about .mp4/.m4v encoding options.

    At some point I did begin doing iPod-compatible rips and/or converting the older rips to iPod compatibility in iTunes. Sadly, AFAIK none of these plays on a TiVo. They'll have to be redone ...

    That said, I'm "dead chuffed" that I can now rip DVDs and play the rips on my TiVo!
     
  10. txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    @epstewart-

    The advanced options for x264 (encoder used by Handbrake) are really meant to improve compression without loss of quality. If you are encoding DVD rips at 5000 kbps, then you have no need for any advanced compression options. When I convert DVD to H.264, I normally target around 2000 kbps. Your DVDs would look excellent at 5000 kbps even just encoding back to MPEG2.

    Since nearly all Region1 DVDs are film content pulled down to NTSC framerates (23.976 fps --> 29.97 fps), the jerkiness that you are talking about is probably the telecined frames. Handbrake will happily IVTC 3:2 material for you (rebuild the original 23.976fps frames). When it does, you will see smooth playback without the jerkiness.

    That being said, I have complained about subtitle shimmering (from hardcoded subs) on 23.976fps H.264 files on my TivoHD. I have found that playback of H.264 files with framerates lower than 25 fps exhibit a subtle up and down movement that makes static images appear to shimmer. This has apparently been fixed on the Tivo Premiere but I have been told that my bug report will very likely never be addressed on Series3/TivoHD. moyekj (author of kmttg) has also done a lot of work with H.264 files on his Series3 units. It sounds like the Series3 may be even more picky about H.264 files than the TivoHD. It is possible that the shimmering issue that I notice at 23.976fps is even worse on the Series3 and that might be the cause of your problems.

    Download MediaInfo and look at a file that displays jerkiness and one that does not. What are the framerates?
     
  11. Gavroche

    Gavroche New Member

    398
    0
    May 26, 2007
    State of...
    Is there a way to use the file name as the episode title in default.txt?
     
  12. wmcbrine

    wmcbrine Ziphead

    10,368
    22
    Aug 2, 2003
    Most people won't need to mess with it, and shouldn't. Only New Zealand users and beta testers should set it, AFAIK.
     
  13. wmcbrine

    wmcbrine Ziphead

    10,368
    22
    Aug 2, 2003
    At least check out the home page, eh?

    It can convert pretty much anything ffmpeg understands to the TiVo's requirements. That's what "transcoding server" means. Yes, on the fly. Yes, it supports Series 2, 3 and 4.

    I don't know what format Galleon uses. Please tell me. I've asked about it before, but got no response. (Please, don't make me reinstall Galleon to find out. The only thing I hate more than Galleon is TiVo Desktop.) I'm guessing that it's a dump of the TiVoVideoDetails XML, with "filename.xml" describing "filename.mpg", sitting next to it in the same directory. I could implement support for that in about five minutes, but I've been holding off until I could get confirmation.

    If the rest is right, but you don't know if the XML files are in TiVoVideoDetails format, just send me some to look at.

    Apart from that, pyTivo has its own metadata format, as well as support for reading metadata from .TiVo files, and from several other kinds of media files.

    It can run as a service if you set it up that way.

    Python tends to be a lot less heavy than Java, IME (even though it's slower -- apart from startup, which is much faster), but YMMV.

    With the Roku as a client, or server? pyTivo will only serve to TiVos (or to anyone else who wants to implement their protocols, but as of now, AFAIK, that's just TiVo). I'm not really familiar with the Roku, but if you can install random software on it, then it should be able to run pyTivo, yes.
     
  14. wmcbrine

    wmcbrine Ziphead

    10,368
    22
    Aug 2, 2003
    Not at present, sorry.
     
  15. Gavroche

    Gavroche New Member

    398
    0
    May 26, 2007
    State of...
    Thats for the quick response. Just thought I'd ask in case I'd missed something.

    Probably what I really need to do is write a converter to scrape the xml data from RSS feeds into filename.txt metadata format.

    Oooooh I have a project!
     
  16. txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    Isn't the filename used if only SeriesId is set in the default.txt? You should be able to create a file named <whatever you want the folder name to be>.containerextension. In your default text file, just use whatever SeriesId you want for the folder. Push that initial file to set the name of the folder. Now, whenever you push anything from that PC folder, it should group in the folder of that first pushed file and be called whatever the filename is. (Remember that if you delete everything from the folder and leave the folder dry on the Tivo for ~2-3 weeks, it will lose the group info and you will need to start over with your initial file to re-create the folder name.)
     
  17. wmcbrine

    wmcbrine Ziphead

    10,368
    22
    Aug 2, 2003
    Anybody?
     
  18. lrhorer

    lrhorer Active Member

    6,924
    0
    Aug 31, 2003
    San...
    AFAIK, everything is bundled in the filename. The format is: SeriesName - EpisodeName (Recorded DOW MTH DD YEAR HR MINAM/PM CHANNEL).TIVO/MPG It looks like it might be possible to embed the episode number, as well, but I don't know how. I also don't know of any way to embed the description.
     
  19. lrhorer

    lrhorer Active Member

    6,924
    0
    Aug 31, 2003
    San...
    The pyTivo forum seems to be down, so I'll post this here. Now, I'm not big into eye candy, and functionally pyTiVo's web server push utility is really great, but I wonder if it wouldn't be possible to spiff it up a bit? Something a little along the lines of TiVoWebPlus' NPL listing would be really nice, although I don't know how difficult the channel logos would be. If nothing else, a little splash of color would be nice, though, and shouldn't be too difficult, at all. At the very least, I find the bright yellow and bright white on black text easier to read than the black on gray of pyTivo's web utility.
     
  20. wmcbrine

    wmcbrine Ziphead

    10,368
    22
    Aug 2, 2003
    Yuck. :thumbsdown: But, you can set your own color preferences in content/main.css. (This is pretty new -- the style elements used to be mixed in with the various templates.)
     

Share This Page