1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

pyTivo - Transcoding server

Discussion in 'TiVo Home Media Features & TiVoToGo' started by armooo, Nov 25, 2006.

  1. Dec 9, 2007 #1301 of 5682
    dlfl

    dlfl Cranky old novice

    6,991
    16
    Jul 6, 2006
    Near...
    I inserted this print statement. Following is the command window output for the entire sequence from starting up pyTivo, selecting the USB-DivX share on the NPL and selecting PassageToIndia from that share:


    192.168.0.158 - - [09/Dec/2007 14:40:06] "GET /TiVoConnect?Command=QueryContaine
    r&Container=%2F HTTP/1.0" 200 -
    192.168.0.158 - - [09/Dec/2007 14:40:27] "GET /TiVoConnect?Command=QueryContaine
    r&Container=%2F HTTP/1.0" 200 -
    files:
    ['I:\\Videos\\DivX\\Extras', 'I:\\Videos\\DivX\\PassageToIndia.divx', 'I:\\Video
    s\\DivX\\ThirrdMan-75ms.divx']
    totalFiles = 3
    192.168.0.158 - - [09/Dec/2007 14:40:32] "GET /TiVoConnect?Command=QueryContaine
    r&Container=USB-DivX&SortOrder=!CaptureDate&ItemCount=8&Filter=x-tivo-container%
    2Ftivo-videos,x-tivo-container%2Ffolder,video%2Fx-tivo-mpeg,video%2F* HTTP/1.1"
    200 -
    files:
    ['I:\\Videos\\DivX\\Extras', 'I:\\Videos\\DivX\\PassageToIndia.divx', 'I:\\Video
    s\\DivX\\ThirrdMan-75ms.divx']
    totalFiles = 3
    /TiVoConnect?Command=QueryContainer&Container=USB-DivX\Extras == /USB-DivX/Passa
    geToIndia.divx
    /USB-DivX\PassageToIndia.divx == /USB-DivX/PassageToIndia.divx
    /USB-DivX\ThirrdMan-75ms.divx == /USB-DivX/PassageToIndia.divx


    192.168.0.158 - - [09/Dec/2007 14:40:34] "GET /TiVoConnect?Command=QueryContaine
    r&Container=USB-DivX&SortOrder=!CaptureDate&ItemCount=1&AnchorItem=%2FUSB-DivX%2
    FPassageToIndia.divx&AnchorOffset=-1&Filter=x-tivo-container%2Ftivo-videos,x-tiv
    o-container%2Ffolder,video%2Fx-tivo-mpeg,video%2F* HTTP/1.1" 200 -

    I've highlighted the print statement results in red. Is the '\' in the file_url items going to be OK?

    In the message just before the highlighted output, "Extras" is the name of the subfolder that I have in this share. Does this message make sense?
     
  2. Dec 9, 2007 #1302 of 5682
    dlfl

    dlfl Cranky old novice

    6,991
    16
    Jul 6, 2006
    Near...
    It appears the problem was a '\' that should have been a '/'. The fix (or at least a fix) is to add one line following line 132 in plugin.py:


    Code:
                    if os.path.isdir(os.path.join(file)):
                        file_url = '/TiVoConnect?Command=QueryContainer&Container=' + cname + file_name
                    else:                                
                        file_url = '/' + cname + file_name
    [COLOR="Red"]                file_url = file_url.replace('\\', '/')[/COLOR]
    I highlighted the added line in red. The rest of the (existing) code is just to help you locate the right place.

    This tests OK for me! Including going into a subfolder (manually).

    Thanks to armooo for pointing to the right place in the code! (And for having any code to point to...)
     
  3. Dec 9, 2007 #1303 of 5682
    PaulS

    PaulS Member

    674
    1
    Sep 16, 2002
    Southern NH
    Nice! I'll try this out right now. I'll report back later tonight.
     
  4. Dec 9, 2007 #1304 of 5682
    wmcbrine

    wmcbrine Ziphead

    10,363
    22
    Aug 2, 2003
    Fixes the videos... something further (probably more of the same) is needed for the music.
     
  5. Dec 9, 2007 #1305 of 5682
    PaulS

    PaulS Member

    674
    1
    Sep 16, 2002
    Southern NH
    Same here. Video playback is fine now. Haven't fooled with audio/pictures yet. Nice find, guys!
     
  6. Dec 9, 2007 #1306 of 5682
    dlfl

    dlfl Cranky old novice

    6,991
    16
    Jul 6, 2006
    Near...
    Revision d4219... was just released for download. The only change from 9978.... was the "fix" given in my previous post. Thus, if you edited that fix into 9978, there is no need to download d4219.
     
  7. Dec 9, 2007 #1307 of 5682
    wgw

    wgw New Member

    217
    0
    May 16, 2007
    The fix works if I transfer an mpg, but I'm getting an error when I try to send a file that needs to be transcoded. pyTivo seems to be trying to send the .ts file without transcoding. And Recording History on the Tivo says "This program was not transferred onto this DVR because the program was recorded in an unknown format."

    Code:
    F:\pyTivo\Cheetah\Compiler.py:1508: UserWarning:
    You don't have the C version of NameMapper installed! I'm disabling Cheetah's us
    eStackFrames option as it is painfully slow with the Python version of NameMappe
    r. You should get a copy of Cheetah with the compiled C version of NameMapper.
      "\nYou don't have the C version of NameMapper installed! "
    192.168.0.11 - - [09/Dec/2007 22:16:06] "GET /TiVoConnect?Command=QueryContainer
    &Container=%2F HTTP/1.0" 200 -
    192.168.0.11 - - [09/Dec/2007 22:16:08] "GET /TiVoConnect?Command=QueryContainer
    &Container=MyMovies%2FLost&SortOrder=!CaptureDate&ItemCount=8&Filter=x-tivo-cont
    ainer%2Ftivo-videos,x-tivo-container%2Ffolder,video%2Fx-tivo-mpeg,video%2F* HTTP
    /1.1" 200 -
    192.168.0.11 - - [09/Dec/2007 22:16:10] "GET /TiVoConnect?Command=QueryContainer
    &Container=MyMovies%2FLost&SortOrder=!CaptureDate&ItemCount=1&AnchorItem=%2FMyMo
    vies%2FLost%2Fzzzz.ts&AnchorOffset=-1&Filter=x-tivo-container%2Ftivo-videos,x-ti
    vo-container%2Ffolder,video%2Fx-tivo-mpeg,video%2F* HTTP/1.1" 200 -
    192.168.0.11 - - [09/Dec/2007 22:16:10] "GET /TiVoConnect?Command=TVBusQuery&Con
    tainer=MyMovies%2FLost&File=zzzz.ts HTTP/1.1" 200 -
    192.168.0.11 - - [09/Dec/2007 22:16:16] "GET /MyMovies/Lost/zzzz.ts HTTP/1.1" 20
    0 -
    ----------------------------------------
    Exception happened during processing of request from ('192.168.0.11', 4038)
    Traceback (most recent call last):
      File "c:\python25\lib\SocketServer.py", line 464, in process_request_thread
        self.finish_request(request, client_address)
      File "c:\python25\lib\SocketServer.py", line 254, in finish_request
        self.RequestHandlerClass(request, client_address, self)
      File "c:\python25\lib\SocketServer.py", line 522, in __init__
        self.handle()
      File "c:\python25\lib\BaseHTTPServer.py", line 316, in handle
        self.handle_one_request()
      File "c:\python25\lib\BaseHTTPServer.py", line 310, in handle_one_request
        method()
      File "F:\pyTivo\httpserver.py", line 38, in do_GET
        plugin.send_file(self, container, name)
      File "F:\pyTivo\plugins\video\video.py", line 39, in send_file
        transcode.output_video(container['path'] + path[len(name)+1:], handler.wfile
    , tsn)
      File "F:\pyTivo\plugins\video\transcode.py", line 40, in output_video
        shutil.copyfileobj(f, outFile)
      File "c:\python25\lib\shutil.py", line 24, in copyfileobj
        fdst.write(buf)
      File "c:\python25\lib\socket.py", line 262, in write
        self.flush()
      File "c:\python25\lib\socket.py", line 249, in flush
        self._sock.sendall(buffer)
    error: (10054, 'Connection reset by peer')
    ----------------------------------------
    
     
  8. Dec 9, 2007 #1308 of 5682
    dlfl

    dlfl Cranky old novice

    6,991
    16
    Jul 6, 2006
    Near...
    I suspect the tivo_compatable function in transcode.py is declaring this .ts to be tivo compatible. If ffmpeg says the codec is mpeg2video this can happen. Should be easy to tell if you turn on debug prints.

    Does the same file transcode OK with another version of pyTivo. (With the same version of ffmpeg)?

    It would be easy to add logic to force .ts files to be transcoded, but I wonder if that is always desirable, i.e., are some of them actually tivo compatible?
     
  9. wgw

    wgw New Member

    217
    0
    May 16, 2007
    No, I don't believe .ts files are compatible, only mpg and .tivo are compatible AFAIK.

    I fixed the error by adding the following code before the mpeg2video check.

    Code:
        [COLOR="Red"][B]if (inFile[-3:]).lower() == '.ts':
            debug_write(['tivo_compatible: ', inFile, ' not program stream ends with .ts\n'])
            return False[/B][/COLOR]
    
        if not type == 'mpeg2video':
     
  10. dlfl

    dlfl Cranky old novice

    6,991
    16
    Jul 6, 2006
    Near...
    I think you're right -- your fix should be merged into the next revision. I think this was a chink because TS can be mpeg2 and be declared so by ffmpeg -- but are not tivo compatible.
     
  11. wgw

    wgw New Member

    217
    0
    May 16, 2007
    Yeah, in previous versions, my .ts files were recognized as non-compatible due to high frame rate or high resolution. The recent addition to pass all videos tagged with mpeg2video to HD Tivo's and the mpeg2video tag caused it to be recognized as compatible. Another bug down. Thanks for the help.

    I think I'm just about done writing my mods to prevent incompatible audio from being transferred. But testing will have to wait for another day.
     
  12. ruttmeister

    ruttmeister New Member

    18
    0
    Dec 8, 2007
    West Des...
    I have to say, pyTiVo is great! Thanks to all those who have put in so many hours to make what is getting close to my ultimate server! I'm not a programmer, and don't have any knowledge or experience with any programming language. I've spent the last couple days experimenting with the different versions of pyTiVo, and am now running d421 on an XP system (with the XP firewall on), wired network with two series 2 ST TiVos.

    The subfolder problem is still there and I can't wait for a fix to that, I just wish I could help! I have been most interested in the metadata side of things, and have been quite successful at figuring it out. I've got a few files I would like to post, but I believe I read elsewhere in this thread that I have to have five posts under my belt before I can do that. Oh well. I'll post a few things here that may be of some help to newbies that also don't have any programming experience.

    Currently this is my entire conf file. I found it easiest to just create a new file without all that extra stuff in it. I haven't had any trouble with my TiVos seeing my computer with any of the builds I've used. Everything has worked great so far (except for the subfolders issue :D )! Oh, I suppose I should mention that I'm only working with mpeg2 files, no avi or anything.

    [My Videos]
    type = video
    path = C:\Documents and Settings\Patrick\My Documents\My Videos

    [Test Videos]
    type = video
    path = c:\Video Test Folder

    [Server]
    ffmpeg_prams = -vcodec mpeg2video -r 29.97 -b %(video_br)s -maxrate %(max_video_br)s -bufsize %(buff_size)s %(aspect_ratio)s -comment pyTivo.py -ac 2 -ab %(audio_br)s -ar 44100 -f vob -
    ffmpeg = C:\pyTivo-d421\plugins\video\ffmpeg_mp2.exe
    video_br = 8Mi
    port = 9032
    width = 720
    beacon = 255.255.255.255
    debug = false
    height = 480
    audio_br = 192k

    As far as metadata, I'd like to explain it for those who may still be having problems. First, you must create a txt file with a name like this: <video filename><video extension (.mpg or .tivo)>.txt. So, if you have an mpeg2 video file named Zeitgeist.mpg, you need to have a txt file in the same folder with the name Zeitgeist.mpg.txt.

    Now for the contents. Here is what a basic file looks like, top to bottom, this is everything in that file:

    description : Villians Emory and Oglethorpe kidnap Master Shake and replace him with a "replicant" alien. However, Frylock and Meatwad are not fooled because the "replicant" doesn't really look like Shake.
    isEpisode : true
    seriesTitle : Aqua Teen Hungerforce
    title : Bad Replicant
    originalAirDate : 2004-11-20T06:00:00Z
    episodeTitle : Bad Replicant
    seriesId : SH466293
    time : 2004-11-20T06:00:00Z
    episodeNumber : 17
    displayMajorNumber : 31
    callsign : TOON
    vProgramGenre : Animated
    vProgramGenre : Drama

    OK, there's a lot going on here, and I'm not going to take up a ton of space explaining every detail, I've already used up enough. One important thing, I understand that pyTiVo can only see one command per line, so make sure your description is on the same line. I haven't tested that part yet to be sure, but you sure can! :D But if you make sure that your file looks exactly like this (except with the relevant information instead of this episode of ATHF), it will all show up when you access that file through your TiVo. I have a txt document where I have been compiling a rather detailed explanation of what you can and can't do with all the fields and values of this file. I'm going to try to attach it to this post (it's not completely done, I need to make some changes, but it'll certainly get you on your way), but if that doesn't work, pm me and I'll be happy to send it to you.

    I'm a big fan of MS Excel and I have built a very simple and basic sheet that makes it a good bit easier to create the data above. I would be happy to share that with anyone that is interested.

    Ok, I'll quit now. Thanks again for everything, I'll be keeping up with this post and will continue to work on all this myself! Any comments on my conf file would be appreciated as I haven't really started playing with any of its settings.
     

    Attached Files:

  13. ruttmeister

    ruttmeister New Member

    18
    0
    Dec 8, 2007
    West Des...
    Sorry, the last file wasn't my most up to date. Not many changes, just a bit easier to read.
     

    Attached Files:

  14. PaulS

    PaulS Member

    674
    1
    Sep 16, 2002
    Southern NH
    Thanks to wgw and dlfl, I'm able to use the latest version of pyTivo. Thanks guys!

    A benefit (?) of using this revision is that files are not transcoded as they are sent to my Series3, which leads to a couple of interesting results.

    1. File transfer files have changed dramatically. SD files would always transfer in realtime (or faster), regardless of pyTivo revision. HD transfers were painfully slow, and could take up to 3x as long to transfer as they took to play out. Under the latest pyTivo, these HD files transfer in nearly realtime. A huge improvement. pyTivo's not having ffmpeg do as much work (no more scaling, most likely), so it can chunk through the files much more quickly.

    2. Transferred file video quality isn't so great. This is also likely due to ffmpeg's reduced role in transfers. Videos appear a bit softer, aspect ratios are a bit off, and so on...

    So, it's a trade-off. No free lunches here.

    Currently, this is not a configurable behavior. If you've got a TiVoHD or Series3, pyTivo will automatically instruct ffmpeg to do less work. Although, it would be VERY simple to add another configuration value for pyTivo.conf to revert back to the older, more processor intensive transfers.
     
  15. dlfl

    dlfl Cranky old novice

    6,991
    16
    Jul 6, 2006
    Near...
    Don't forget "Thanks to armooo". He furnished the final hint that led to the "fix". (Not to mention creating pyTivo.)
     
  16. PaulS

    PaulS Member

    674
    1
    Sep 16, 2002
    Southern NH
    Absolutely. Didn't intend to omit armooo. Thanks!
     
  17. ruttmeister

    ruttmeister New Member

    18
    0
    Dec 8, 2007
    West Des...
    I finally picked up a 649 series 2 DT 80hr today and got it up and running. Guess what? subfolders works perfectly. But only on the DT TiVo. It still doesn't on my other two machines. I wonder if the conf file can be set to broadcast two different types of share setups to specific TiVos? Oh, the DT does lose the vProgramGenre entries, but I'll work on that for a workaround or something. Also, with DT, when viewing the program screen of an episode in one of the subfolders on pyTiVo, you can use the channel up/down to jump to the previous/next video in the subfolder. Just so you know...
     
  18. dlfl

    dlfl Cranky old novice

    6,991
    16
    Jul 6, 2006
    Near...
    Using version d4219..., for files that don't have metadata text files paired with them, the name displayed on the NPL is the subfolder containing the file, which is nondescript and confusing. All you have to do to get the actual video file name to display is replace line 105 in video.py with:
    Code:
            metadata['seriesTitle'] = metadata['title']
    
    This will not change the NPL display for files that have metadata text files paired with them.
     
  19. ruttmeister

    ruttmeister New Member

    18
    0
    Dec 8, 2007
    West Des...
    That's a nice fix, thanks.

    I'm having a problem now that I haven't had before. on my 240 series 2 when i try to enter a subfolder, it now says "This Group is empty....." when before it would show the first folder I entered. At least then I could transfer a program from the first folder, now nothing shows up. Any ideas?
     
  20. grahamkent

    grahamkent New Member

    16
    0
    Feb 15, 2003
    Seattle, WA
    Thanks dlfl! Since I've edited a number of my files since downloading the build, can you provide some context to the line that needs to be replaced?

    Thanks!
     

Share This Page