1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

pyTivo - Transcoding server

Discussion in 'TiVo Home Media Features & TiVoToGo' started by armooo, Nov 25, 2006.

  1. dlfl

    dlfl Cranky old novice

    7,002
    19
    Jul 6, 2006
    Near...
    Well, now we have a little dilemma: what do we recommend for new users?

    The only significant difference between the "official" (wiki-based) download and wmcbrine's fork is the enhanced plugins (including a photo plugin) in the wmcbrine version.

    My impression is that (1) the wmcbrine plugins are generally superior and (2) they have received fairly adequate testing by now. I've tested the latest snapshot for basic video transcoding functionality also.

    Thus I've put a zip snapshot of the wmcbrine fork on mediafire.com.

    The only advantages of my download over just getting a snapshot from wmcbrine's Git link are:
    1. You get .zip instead of Tar.gz
    2. I'm still including a newer more capable version of ffmpeg (r11051).

    As I've been so thoroughly instructed :eek: there apparently is no harm in just copying one of these zips on top of an existing pyTivo installation. I was concerned about the possibility that the Python compiler might be fooled into not recompiling a new module due to some problem with file time/date stamps (?).
     
  2. KRKeegan

    KRKeegan Im lost and confused

    215
    0
    Jul 20, 2004
    Los Angeles, CA

    Cool, I have a patch that will fix this, but I need to figure out what armooo did with the subfolders svn. I dont see it in git either.

    TreborPugly Can you also tell me if you are getting any other reboots when you do anything else? And if you get any just random reboots that you can't figure out what caused them?

    Kevin
     
  3. KRKeegan

    KRKeegan Im lost and confused

    215
    0
    Jul 20, 2004
    Los Angeles, CA
    Yeah I see this as kind of a problem too. I think we incorporate wmcbrine's ticket fixes into the main source. As for his plugins, I think these can remain as a separate distribution containing only the plugins? The idea of a plugin is that it does not require any changes to the base program. If these plugins are truly autonomous then I think we should make a link on the wiki and reference them as addons to the base program.

    Armooo I need your input on this specifically. And anyone else who has been contributing to the development.

    I also think the 8.3 hack fork should be folded into the main program. Keeping 2 separate forks running seems unnecessary and confusing to users.

    Now I believe your initial reason for splitting the program into 2 forks was a concern over stability. And I agree the 8.3 hack is not perfect and will always probably have some instability. However, if the Hack83 setting is not set to true in the conf file it will not cause pyTivo to be any less stable because the problematic areas will not be executed.

    To further explain I have included the differences of the 3 files which are affected by the Hack83 fork. Config.py, httpserver.py and video.py are the only files altered by the Hack83 fork.

    Config.py - Hack83 adds the following lines 76-84
    Code:
     def getHack83():
         try:
             debug = config.get('Server', 'hack83')
             if debug.lower() == 'true':
                 return True
             else:
                 return False
         except NoOptionError:
             return True
    This is just a function to get the value of the Hack83 setting from the conf file, no big deal. Now I should change the default from true to false so that this function would have to be affirmatively selected.

    httpserver.py - Hack83 imports the config file, no big deal. It also adds a similar debug call to one found in transcode.py, since this same function is included in the standard video.py it isn't a big deal. Then Hack83 replaces
    Code:
             self.send_response(404)
             self.send_header('Content-type', 'text/html')
             self.end_headers()
             t = Template(file=os.path.join(SCRIPTDIR,'templates','unsuported.tmpl'))
             t.query = query
             self.wfile.write(t)
    
    With
    Code:
             if hack83 and 'Command' in query and 'Filter' in query:
                 debug_write(['Unsupported request, checking to see if it is video.', '\n'])
                 command = query['Command'][0]
                 plugin = plugin = GetPlugin('video')
                 if "".join(query['Filter']).find('video') >= 0 and hasattr(plugin,command):
                     debug_write(['Unsupported request, yup it is video send to video plugin for it to sort out.', '\n'])
                     method = getattr(plugin, command)
                     method(self, query)
                 else:        
                     self.send_response(404)
                     self.send_header('Content-type', 'text/html')
                     self.end_headers()
                     t = Template(file=os.path.join(SCRIPTDIR,'templates','unsuported.tmpl'))
                     t.query = query
                     self.wfile.write(t)
             else:
                 self.send_response(404)
                 self.send_header('Content-type', 'text/html')
                 self.end_headers()
                 t = Template(file=os.path.join(SCRIPTDIR,'templates','unsuported.tmpl'))
                 t.query = query
                 self.wfile.write(t)
    This code will not be executed without the Hack83 setting being set to true. Thus I think the changes in this file are safe.

    Video.py - Hack83 imports time, which isn't a big deal. The debug function from transcode.py has also been added, which as I discussed above shouldn't be a problem. Hack83 adds a 150 line function called hack, because this is a function it should only be an issue if it is called, which will only occur if it was selected from the conf file. Finally at line 126 Hack83 adds the following:
    Code:
            if hack83:
                print '========================================================================='
                query, hackPath = self.hack(handler, query, subcname)
                print 'Tivo said: ' + subcname + ' || Hack said: ' + "/".join(hackPath)
                debug_write(['Hack Tivo said: ', subcname, ' || Hack said: ' , "/".join(hackPath), '\n'])
                subcname = "/".join(hackPath)
            
                if not query:
                    debug_write(['Hack sending 302 redirect page', '\n'])
                    handler.send_response(302)
                    handler.send_header('Location ', 'http://' + handler.headers.getheader('host') + '/TiVoConnect?Command=QueryContainer&AnchorItem=Hack8.3&Container=' + "/".join(hackPath))
                    handler.end_headers()
                    return
    This as you can see will only be run if it is selected from the conf file.

    I have enclosed a complete text patch file necessary to update master to hack83 so you can see all the differences.

    Originally I think it made sense to fork this hack hoping that TiVo would fix the issue. But as this has progressed it appears very unlikely that TiVo is going to fix this anytime soon. I think new users and existing users would benefit not having to figure out why there are two different forks of the program.

    Let me know what you think. I am not trying to be difficult I just think it might be easier for users and I have been running the subfolders hack for 9 months with very little issues.

    Kevin
     

    Attached Files:

  4. Chew

    Chew New Member

    284
    0
    Jan 22, 2003
    Thanks!

    Just updated as suggested and it worked perfectly. Thanks again to everybody who helps with and works on this great program!
     
  5. dlfl

    dlfl Cranky old novice

    7,002
    19
    Jul 6, 2006
    Near...
    Strongly agree with folding subfolders into main. Seems simple and low risk.

    Regarding the plugins, it's music and photo.

    The main program currently has no photo plugin so why not just include wmcbrine's ? What are you losing?

    wmcbrine has put a lot of work into the music plugin -- see this post and the thread leading up to it. Unless someone has a complaint about it (or the author himself recommends against it) I favor making it the mainstream code.
     
  6. Jan 1, 2008 #1586 of 5684
    wgw

    wgw New Member

    217
    0
    May 16, 2007
    Jan 12 Update: updated ffmpeg to rev.11143.

    Jan 05 Update: Moved calls for mpg info to cache to speed Tivo display.

    I have created an enhancement for pyTivo if anyone would like to try it out. Be sure to create a new pytivo.conf file using the supplied pytivo.conf.dist.

    http://pytivo.armooo.net/wiki/LatestBeta

    Hopefully this modification will make pyTivo more effortless and easy to use for both new and advanced users.

    Summary of the changes.

    MPEG AUDIO COMPATIBILITY:
    Converts user provided audio bitrates to non-zero multiple
    of 64 for ffmpeg compatiblity.
    Sets max audio bitrate to 384k/448k for S2/S3 tivos respectively.
    Checks audio bitrate of source and transcodes if max audio bitrate exceeded.
    Compares audio_br and max_audio_br and uses the lower of the
    two bitrates to ensure a tivo compatible bitrate is used.
    MAX_AUDIO_BR may be specified in conf to override defaults.
    MAX_AUDIO_BR is both a 'Server' and 'Per Tivo' option.
    You should be able to ignore this setting unless you want to test high audio bitrates. It should not interfere with, or transcode, recordings that you pulled off the Tivo and are sending back. It's mainly to ensure that very high bitrate external sources get downsampled to a bitrate the tivo can actually play.

    MPEG VIDEO COMPATIBILITY:
    Expands use of existing setting MAX_VIDEO_BR=17408K.
    Checks video bitrate of source and transcodes if max video bitrate is exceeded.
    MAX_VIDEO_BR may be specified in conf to override default.
    MAX_VIDEO_BR is a 'Server' option.
    This function, was added so that high bitrate mpegs from sources like HD/Blueray DVD's will be automatically transcoded due to their very high bitrates, rather than be recognized as a tivo compatible mpeg. It should not interfere with, or transcode, HD recordings that you pulled off the Tivo and are sending back. But if you wish to do so, lowering this setting to 13000k should force most HD .mpg files to be transcoded.

    TIVO AUDIO CODEC COMPATIBILITY:
    Checks for HD tivo and uses ac3 audio automatically so that
    5.1 audio is retained, if present in the source.
    Otherwise, mp2 audio is used for S2 compatibility.
    User may still override by specifying their own ffmpeg template.
    This was added in response to complaints about losing 5.1 audio.

    Note: includes wmcbrines photo and music updates.
     
  7. Jan 1, 2008 #1587 of 5684
    wmcbrine

    wmcbrine Ziphead

    10,368
    22
    Aug 2, 2003
    I saw it, but I can't explain it.
     
  8. Jan 1, 2008 #1588 of 5684
    MasterCephus

    MasterCephus New Member

    309
    0
    Jan 2, 2005
    Hueytown,...
    wgw, could you zip that up and upload it to mediafire or PM me and I will post it on mediafire?

    There are a lot of people who dont'/won't use git to get a working copy...while that's good for development, it's not really good for software distribution (IMO of course) for people who are not developers....
     
  9. Jan 1, 2008 #1589 of 5684
    dlfl

    dlfl Cranky old novice

    7,002
    19
    Jul 6, 2006
    Near...
    wgw,
    Does your git fork merge in wmcbrine's plugins?
     
  10. Jan 1, 2008 #1590 of 5684
    wmcbrine

    wmcbrine Ziphead

    10,368
    22
    Aug 2, 2003
    It's there. You can see it from http://repo.or.cz/w/pyTivo.git . Try this:

    git clone git://armooo.net/pyTivo.git
    cd pyTivo
    git checkout origin/subfolders-8.3

    I'm new to git myself, so I'm not sure this is the right way to go about it, but it seems to work.
     
  11. Jan 1, 2008 #1591 of 5684
    KRKeegan

    KRKeegan Im lost and confused

    215
    0
    Jul 20, 2004
    Los Angeles, CA
    ok cool, I will update the subfolders hack then. I am new to this whole git thing too. Hopefully I still have write access to this.

    Kevin
     
  12. Jan 1, 2008 #1592 of 5684
    wgw

    wgw New Member

    217
    0
    May 16, 2007
    git is not required. Click on the "Snapshot" link next to the line that says "SD/HD Tivo Audio/Video/Codec compatibility". This will give you a .tar file which you can extract with WinRar, 7-Zip, or probably just about any file archiver. The .tar file contains the full package. Install it the same as an official release.
     
  13. Jan 1, 2008 #1593 of 5684
    TreborPugly

    TreborPugly Unreal!

    4,542
    0
    May 2, 2002
    Eastern TN
    I can only get obviously pyTivo related reboots if I get into the description of a recording. Just browsing folders, without ever selecting a file doesn't seem to cause it, even with a few sublevels of folders. Once I select a recording, if I back out, the TiVo will crash every time. (Again, this happens either from pulling out after just looking at the file, or after starting to transfer the file and then going back, or after watching the entire file and deleting it or saving it. Anything that tries to take me back to it's subfolder crashes the Tivo.)

    I do sometimes get random reboots, but very infrequently. I also force it to reboot at least once a week, because it gets the "delayed remote response" problem.
     
  14. Jan 1, 2008 #1594 of 5684
    Emrys

    Emrys New Member

    9
    0
    Nov 6, 2002
    I am running the wcmbrine fork from a few posts up, and it fixed my ability to see down 2 folders from the root IE : ROOT -> TV -> SERIES1 -> shows are listed here ( not sure if it's his patches or what that fixed it, but it works ). I am still unable to go a folder deeper, it just clones itself over and over. I can deal with this though. To be able to watch stuff from my fileserver is insane.

    My question is, can we get a solution to not being able to see a second folder structure once we have already been down one previously ?

    I have been trying to read through all this, but this thread is very confusing. In a nutshell, when you go into a folder ( season1 for example ), and then come back up one and try to go into another ( season2 for example ), it shows the contents of the first folder you just backed out of ( season 1 ). I'm not sure if that is a good description. Anyway, the way around it seems to be to back up to root, go into deleted, back up to root, go into another folder, back up to root, then go into the folder you were trying to get into. This works, however it can be a pain. I got this from Ticket # 11 : http://pytivo.armooo.net/ticket/11

    Anybody up to speed on why this happens or how to fix it? Alas, I am not a coder, and wouldn't even know where to begin. Not sure if this is a known limitation or what.

    Series 2 HD ( 9.2a I believe )
    latest and greatest ( wcmbrine fork ) pyTivo code runnin on a gentoo box

    Thanks
     
  15. Jan 1, 2008 #1595 of 5684
    KRKeegan

    KRKeegan Im lost and confused

    215
    0
    Jul 20, 2004
    Los Angeles, CA
    Ok well both of those problems are resolved by a simple one line patch. I am glad to hear you have very few other reboots. I also generally don't have problems using subfolders either.

    Unfortunately I can't get git to output a new change for you. I am assuming Armooo has to approve it? I checked out the subfolders made the change and then committed it. But nothing happened. Anyone know what else I need to do?

    TreborPugly - hopefully Armooo will accept the mods soon and you can download a new update.
     
  16. Jan 1, 2008 #1596 of 5684
    KRKeegan

    KRKeegan Im lost and confused

    215
    0
    Jul 20, 2004
    Los Angeles, CA
    What you are describing is the hiccup added by TiVo in their 8.3 software. I have never actually seen or tried the long solution you have described to fix it(I will have to check this out, i always had to wait for 30+ minutes).

    From the pyTivo end there is only so much we can do to fix this. The two options are:
    Subfolders-Hack - A separate fork of the main pyTivo program, this allows full subfolders use to function. Meaning you can have and browse subfolders infinitely deep. However this is a hack and does have some small possibility of causing your TiVo to reboot while you are browsing files on pyTivo. However for me and for others these reboots are very infrequent.
    Auto_Shares - I have not tried this yet, but from my understanding this will automatically create multiple pyTivo shares in your Now Playing List. This function is present in the wmcbrine version that you have.

    Hope this helps.
     
  17. Jan 1, 2008 #1597 of 5684
    wgw

    wgw New Member

    217
    0
    May 16, 2007
    No, I started with the 12-31-master and added my changes. I had assumed armooo had merged all of wmcbrine's mods and plugins in the latest release, but a file compare indicates those are missing. I'm new to git as well and have not attempted a merge yet. I was going to wait and see if my mod would be accepted as a valid development path for the master and simply be merged into it. I didn't want to stray too far off the beaten path, so to speak.
     
  18. Jan 1, 2008 #1598 of 5684
    Emrys

    Emrys New Member

    9
    0
    Nov 6, 2002
    So from what I am reading you are trying to get this "hack" into the main tree to be enabled with an option in the config file, right ? Should I go back to the 12.03.07 hacked version? How infrequent is infrequent? We watch usually 3 hours of TV a night and frequently move around the guide, but not so much back and forth in the NPL.

    Thanks for the info. I was thinking that this was probably the problem, but thought "hey, I'm way past 8.3, this could be something else".
     
  19. Jan 1, 2008 #1599 of 5684
    MickeS

    MickeS New Member

    25,884
    0
    Dec 26, 2002
    AZ
    This is really confusing with all these forks and different versions...
     
  20. Jan 1, 2008 #1600 of 5684
    KRKeegan

    KRKeegan Im lost and confused

    215
    0
    Jul 20, 2004
    Los Angeles, CA
    Yes that is the one I am talking about. Rebooting would only occur while browsing the pyTivo share if at all. If you are watching regular TV or selecting stuff in the normal NPL you have no worries. Most of the reboots are caused by rapidly moving through the pyTivo share, such as rapifly exiting multiple nested folders back to the pyTivo root. I think my tivo reboots maybe once a month?? It is hard to say exactly, but I doubt you would have a lot of issues. There is currently one outstanding fix that I am hoping armooo adds to the fork and posts a new download link for you. It is listed at Ticket #67 on the wiki.

    I agree, I am not sure I am a fan of the multiple forks using git. I think I would prefer a system where multiple people can contribute changes which are then accepted and put into the main distribution either through a majority vote or direct control of Armooo.
     

Share This Page