1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

pyTivo - Transcoding server

Discussion in 'TiVo Home Media Features & TiVoToGo' started by armooo, Nov 25, 2006.

  1. May 4, 2007 #421 of 5684
    ocntscha

    ocntscha New Member

    149
    0
    Oct 22, 2003
    I've been continuing to play with pyTivo on Solaris. I was having a performance issue and would like to submit a patch that corrected it. Whenever I would transfer a file it would be at about 1/3 to 1/4 of real time. Odd because issuing the same ffmpeg command in a UNIX shell would convert the files at right around real time. I determined with prstat (more or less the Solaris equivalent to Linux's top) that when a transfer was taking place python would be eating up about 66% CPU and ffmpeg only got 33%. Unlike on my LInux box where its like 98% ffmpeg and 1% python.

    I found where in transcode.py its sending the file over and googled up http://pydoc.org/2.4.1/subprocess.html where I learned about the bufsize option to Popen. Since pyTivo doesn't specify bufsize it defaults to unbuffered, I added the bufsize option and set it to -1 which "means to use the system default". Lo and behold it worked! That one little option gave like a 3 to 4 fold increase in performance, I can transfer right around real time now, ffmpeg eats 98% CPU, python eats 1% just like on my Linux box.

    I don't know if this the best fix, or the correct fix, all I can say is it works. I added the same bufsize=-1 option on my Linux box and it doesn't have any negative effect. I'll leave it to you experts to decide.

    Also included in this patch is just a new, commented section for pyTivo.conf. At the moment I've got 2 pyTivos running on my network, one on Linux one on Solaris so they needed unique guids. One of you programmers already have programmed beacon.py to check for a guid in pytivo.conf and it definately works so I figured it ought to be documented in pyTivo.conf.
    Code:
    diff -L proposed-changes-to-pyTivo191 -r -u proposed-changes-to-pyTivo191 pyTivo/plugins/video/transcode.py
    --- proposed-changes-to-pyTivo191
    +++ pyTivo/plugins/video/transcode.py   2007-05-03 23:31:45.000000000 -0500
    @@ -47,6 +47,7 @@
     def transcode(inFile, outFile, tsn=''):
         cmd = [FFMPEG, '-i', inFile, '-vcodec', 'mpeg2video', '-r', '29.97', '-b', VIDEO_BR] + select_aspect(inFile, tsn) +  ['-comment', 'pyTivo.py', '-ac', '2', '-ab', AUDIO_BR,'-ar', '44100', '-f', 'vob', '-' ]   
         debug_write(['transcode: ffmpeg command is ', ''.join(cmd), '\n'])
    +    ffmpeg = subprocess.Popen(cmd, bufsize=-1, stdout=subprocess.PIPE)
         ffmpeg = subprocess.Popen(cmd, stdout=subprocess.PIPE)
         try:
             shutil.copyfileobj(ffmpeg.stdout, outFile)
    @@ -221,7 +222,7 @@
             return True, True, True, True, True
     
         cmd = [FFMPEG, '-i', inFile ] 
    -    ffmpeg = subprocess.Popen(cmd, stderr=subprocess.PIPE, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
    +    ffmpeg = subprocess.Popen(cmd, bufsize=-1, stderr=subprocess.PIPE, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
     
         # wait 4 sec if ffmpeg is not back give up
         for i in range(80):
    diff -L proposed-changes-to-pyTivo191 -r -u proposed-changes-to-pyTivo191 pyTivo/pyTivo.conf
    --- proposed-changes-to-pyTivo191
    +++ pyTivo/pyTivo.conf  2007-05-03 23:45:54.000000000 -0500
    @@ -22,6 +22,11 @@
     #need to specify the subnet broadcast address(es) of your Tivo boxes
     #beacon=255.255.255.255
     
    +# Global unique identifier of this pyTivo, can be any six digits.  Defaults
    +# to 123456 if not explicitly set here.  If you have 2 or more pyTivos running
    +# on your network, they will each need a unique guid.
    +#guid=123456
    +
     #Output Pixel Width: if you have an HDTV you might want to try 720 or 704
     #Valid: 720, 704, 544, 480, 352
     #width=1440
    
     
  2. May 4, 2007 #422 of 5684
    bschuler2007

    bschuler2007 Frustrated owner

    325
    0
    Feb 25, 2007
    Love pytivo.. but I got a quick question:

    Since spring update 8.3, I had to enlarge my pytivo config file considerably. When I edit the config file manually, I'll have say, Music Videos ABC and then Music Videos DEF right next to each other. When I launch Pytivo configurator.. it suddenly reorganizes my config file and nothing is in order anymore. This matches what I am seeing on tivo. I've tried everything I can think of.. and I can't get it to be sorted.

    SO basically my question is... how does pytivo organize it's folders? Is there anyway to get alphabetically sorted listings of folders? Is it based on folder size content? Time, etc?
     
  3. May 4, 2007 #423 of 5684
    dlfl

    dlfl Cranky old novice

    6,997
    18
    Jul 6, 2006
    Near...
    I just tested this on one of my Windows XP systems (2.8 GHz laptop, wireless .11g to hub, wired hub-to-tivo). I used a 121 sec XviD clip encoded at 1200 kbps, transcoding at 4096kbps. The transfer time was the same with or without the mod, i.e, 67 seconds.

    In other words, it seems to do no harm to Windows systems.
     
  4. May 4, 2007 #424 of 5684
    TreborPugly

    TreborPugly Unreal!

    4,542
    0
    May 2, 2002
    Eastern TN
    Bummer, I got 8.3 last night. I went into a subfolder and out, without anything dying, but I didn't really mess with it. I'll be checking out the updated code though I guess.

    I wasn't paying enough attention to the complaints about 8.3. Does it crash your Tivo or just mess up your Now Playing list?
     
  5. May 4, 2007 #425 of 5684
    dlfl

    dlfl Cranky old novice

    6,997
    18
    Jul 6, 2006
    Near...
    Another user and I have had TiVo crashes when using the video.py file from changeset 192 (Occuring when you select a subfolder file to transcode). However this changeset was never released as a version for downloading.

    On 191 or 187 I haven't seen or heard of crashes, but subfolders are broken for TiVo 8.3. In my case you can sometimes go to a subfolder and successfully transcode a video there, but from then on if you go to any subfolder it is empty.

    Note you can create video shares in pyTivo that are subfolders of other video shares. This at least gives you a solid way to get all your subfolder videos into the NPL without having to move them around on your PC. Unfortunately, neither folder nor file shortcuts seem to work.
     
  6. May 4, 2007 #426 of 5684
    bschuler2007

    bschuler2007 Frustrated owner

    325
    0
    Feb 25, 2007
    Yes.. but then you'll have my issue of trying to organize them in a coherent way. Your shares seem to show up in random order. Not that big of a deal..but would be nice to be able to organize your shares.
     
  7. May 5, 2007 #427 of 5684
    Tobashadow

    Tobashadow Read over there --->

    2,339
    0
    Nov 11, 2006
    Well i got the files to show up again, it seems the conf file was changed somehow when i went back to the 187.

    But when i select a file everything goes as normal but when i hit play i can hear everything just fine but the video looks like im looking thru one of those privacy frosted windows for like a shower. And it's in large blocks of this.

    Oh joy config problems hehe.

    Any pointers or specific files to look in?

    The defualt 182 was perfect before.
     
  8. May 5, 2007 #428 of 5684
    dlfl

    dlfl Cranky old novice

    6,997
    18
    Jul 6, 2006
    Near...
    I would suspect the video bitrate is way to low. This is the "video_br= " line in the config file. If you just delete that line it will default to 4096k which is a good value for most systems. If you need a lower rate (for example if your network connection or computer are slower than average) then try 3072k or 2048k. Quality will be reduced but not to the blockiness you are seeing.

    This parameter (if present) must be in the [Server] section of the file. See "About the file format" in the configuration page of the Wiki for the definition of "being in a section."
     
  9. May 5, 2007 #429 of 5684
    Tobashadow

    Tobashadow Read over there --->

    2,339
    0
    Nov 11, 2006
    Ok im back in business with 187 on both Tivo's now ( i added a second one to my account last night) and everything looks and acts right except for the folder problem on the 8.3 box.

    Turned out the defualt video viewing size setting was messing with it.

    I # it out and everything was fine.
     
  10. May 5, 2007 #430 of 5684
    KRKeegan

    KRKeegan Im lost and confused

    215
    0
    Jul 20, 2004
    Los Angeles, CA
    Ok people as promised I spent about 4 hours yesterday tinking with 8.3 trying to figure out the problem. I have made some good insights, but TiVo really f***ked us on this one.

    So here is what happens.

    It seems that the first time you use the TiVoComeBack feature it works fine, and I think after a period of time a cache in TiVo empties and it will work fine again for another single use.

    But after you use TiVoComeBack once every future request from TiVo becomes messy. TiVo starts requesting the very first folder you used for no reason in every request. And it sticks to this folder, like it is stuck in a cache somewhere.

    I wrote up a summary of where I am at over on the wiki. 8.3Software

    Please anyone read it and tell me what you think.
     
  11. May 5, 2007 #431 of 5684
    TreborPugly

    TreborPugly Unreal!

    4,542
    0
    May 2, 2002
    Eastern TN
    What about an HME interface other than Now Playing? Isn't that what some of the tools for Galleon use?
     
  12. May 5, 2007 #432 of 5684
    morac

    morac Cat God

    8,948
    22
    Mar 14, 2003
    NJ
    Seems about right. Unfortunately, the problem with trying to work around the issue is that it is basically one big hack with unpredictable results.

    The best fix would be to get TiVo to fix the issue themselves. Right now, they're probably not even aware of the problem. Pointing it out to TiVoJerry and asking politely if that was intentional and if not could it be fixed in the next TiVo software release would probably be the best course of action.
     
  13. May 5, 2007 #433 of 5684
    KRKeegan

    KRKeegan Im lost and confused

    215
    0
    Jul 20, 2004
    Los Angeles, CA
    The interface is actually 90% identical to the HMO interface. But no, videos can only appear in the NPL. Galleon uses HME for a lot of features, but even the GoBack option in Galleon appears in the NPL.
     
  14. May 5, 2007 #434 of 5684
    johnkfisher

    johnkfisher New Member

    18
    0
    May 7, 2006
    I'd be cool with 192's limitations in the interim, but unfortunately it causes some tivos, including mine, to reboot. If that can be fixed or if a way to alphabetize the list when we set up multiple shares as a workaround, well, that'd be neat :)
     
  15. May 5, 2007 #435 of 5684
    1up

    1up New Member

    11
    0
    Apr 22, 2007
    Hoping someone can help me out here. I have a Tivo Series2 and started using pyTivo. I can't get any decrypted vob files to show up in the NPL.

    My pyTivo.conf contains:
    Code:
    [Server]
    port=9032
    ffmpeg=c:\Program Files\pyTivo\plugins\video\ffmpeg_mp2.exe
    
    debug=true
    
    video_br=3072K
    
    [My Vids]
    type=video
    path=z:\pemm532
    
    [Torrents]
    type=video
    path=e:\torrents
    
    [DVD]
    type=video
    path=z:\ForTivo
    
    I used the debug function of pyTivo and here's the relevant data:
    Code:
    Seems stream 0 codec frame rate differs from container frame rate: 29.97 (30000/1001) -> 59.94 (60000/1001)
    
    Input #0, mpeg, from 'Z:\ForTivo\VTS_01_0.VOB':
    
      Duration: 00:00:04.9, start: 0.280633, bitrate: 106147 kb/s
    
      Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 720x480, 9800 kb/s, 59.94 fps(r)
    
      Stream #0.1[0x80]: Audio: ac3, 48000 Hz, stereo, 192 kb/s
    
    Must supply at least one output file
    
     
    video_info: Bug in VideoReDo
    video_info: Codec= mpeg2video  width= 720  height= 480  fps= 29.97  millisecs= 4900 
    video_info: ffmpeg output= FFmpeg version SVN-r8047, Copyright (c) 2000-2007 Fabrice Bellard, et al.
    
      configuration: --enable-gpl --enable-pp --enable-swscaler --enable-pthreads --enable-liba52 
    --enable-avisynth --enable-libdts --enable-libfaac --enable-libfaad --enable-libgsm --enable-libmp3lame 
    --enable-libnut --enable-libogg --enable-libtheora --enable-libvorbis --enable-x264 --enable-xvid 
    --enable-amr_nb --enable-amr_wb --cpu=i686 --enable-memalign-hack --extra-ldflags=-static
    
      libavutil version: 49.3.0
    
      libavcodec version: 51.34.0
    
      libavformat version: 51.10.0
    
      built on Feb 21 2007 23:20:24, gcc: 4.3.0 20070126 (experimental)
    
    
    There are multiple errors exactly the same as above for each of the vob files in the directory. I'm not at all knowledgeable about this, but shouldn't the debug data show "--enable-vob" as well? I can't find anywhere in any file where I can change the ffmpeg2 command line options. I was previously using Tivo[.]net and it worked fine with that program. I even tried moving all files into their own directory so that there weren't any subdirectories. Does any have any suggestions?

    Thanks,
    1up
     
  16. May 6, 2007 #436 of 5684
    dlfl

    dlfl Cranky old novice

    6,997
    18
    Jul 6, 2006
    Near...
    .vob files are one kind of mpeg2 file and your debug data indicates it is tivo-compatible and thus should be transfered without even requiring transcoding by ffmpeg. I've never tried to transfer a .vob file but I took one of my TiVo compatible mpeg2 files and changed the extension to .vob and it showed on the NPL and transfered fine (without transcoding).

    What version of pyTivo?
    Per my understanding of the pyTivo code, it should be furnishing your VOB file names to the tivo NPL. As a quick experiment, try renaming the .vob to .mpg, although my experiment indicates it shouldn't matter. Still, it could be a difference between your Series2 and my Series2 DT.

    Heres a link to some good info on ripped vob files. It's a VideoReDo web page but it's helpful even if you don't have the program. In particular note you should ignore the usually small .vob's, i.e. " Do not include VIDEO_TS.VOB or VTS_xx_00.VOB if present. " To get the complete video you need to combine the other .vob's using one of the methods described. The .vob files are split at arbitrary points just determined by a 1 GB max file size, not by logical scene boundaries.
     
  17. May 6, 2007 #437 of 5684
    ocntscha

    ocntscha New Member

    149
    0
    Oct 22, 2003
    I've had great luck with a little freeware program named vob2mpg the few times I've tried it, takes a vob set and makes a normal mpeg out of it, very quickly, few minutes to do the whole movie..

    http://www.svcd2dvd.com/downloads.aspx#VOB2MPG
     
  18. May 6, 2007 #438 of 5684
    Tobashadow

    Tobashadow Read over there --->

    2,339
    0
    Nov 11, 2006

    The bitrate wasnt the problem BUT i did change it to the 3072k spec that you stated above and i cant see a change in quality even on my 62 inch tv (good source count's) , but now on my 802.11b adapter's on both tv im getting a encoding transfer ratio of almost 2:1 now where as i was before having to keep from starting it for 5mins or so after start of transfer to give it enough headstart, now i can instantly hit start playing and it will keep getting further ahead as it goes now.

    BTW i just got done breaking down my folder's to a more single folder type system so that the 8.3 shouldnt bother me in the future and so i can stick with the 187 stable for now.

    Thanks for the help.
     
  19. May 6, 2007 #439 of 5684
    Tobashadow

    Tobashadow Read over there --->

    2,339
    0
    Nov 11, 2006
    When i get a chance i'll upload a .vob to my server machine and try it.

    Ive got the luxury starting as of this weekend of having both my DT in the living room and my older model total black front Single tuner S2 that i just got for the bedroom.

    If there's a difference i'll be able to help.
     
  20. May 6, 2007 #440 of 5684
    Tobashadow

    Tobashadow Read over there --->

    2,339
    0
    Nov 11, 2006
    Both machines threw a error when going into a folder with a unencrypted mpeg2 DVD compatable encoded .vob file in it.

    Both Machines did fine if it was rencoded as a tivo compatable file.

    Seem's Pytivo doesnt like dvd files.
     

Share This Page