pyTivo - Transcoding server

Discussion in 'TiVo Home Media Features & TiVoToGo' started by armooo, Nov 25, 2006.

  1. txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    Not sure. gonzotek just updated the pytivo install wiki and is pointing to this website for ffmpeg: http://ffmpeg.zeranoe.com/builds/

    Is that where you got yours?
     
  2. wmcbrine

    wmcbrine Well-Known Mumbler

    11,534
    717
    Aug 2, 2003
    I think that output must be from a newer ffmpeg, but with an older version of pyTivo. There's no other way I can account for the "failed at mapVideo/mapAudio" errors.
     
  3. gonzotek

    gonzotek tivo_xml developer

    2,538
    59
    Sep 24, 2004
    Outside...
    Here's the version info of the latest pytivo and ffmpeg builds I've got working. I should explain that the pytivo version is wmcbrine's Jan 3 commit(still latest as of this post). I use git for windows, and apparently it doesn't copy down the timestamps from the repo to my local system, as I had pulled it on the 8th. But I digress :)
    Code:
    INFO:pyTivo:Last modified: Tue Jan 08 19:25:40 2013
    INFO:pyTivo:Python: 2.7.3
    INFO:pyTivo:System: Windows-7-6.1.7601-SP1
    
    Code:
    ffmpeg version 1.1 Copyright (c) 2000-2013 the FFmpeg developers
      built on Jan  8 2013 16:16:22 with gcc 4.7.2 (GCC)
      configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
    isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
    le-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-libg
    sm --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --e
    nable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --e
    nable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-amrwben
    c --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-
    libxvid --enable-zlib
      libavutil      52. 13.100 / 52. 13.100
      libavcodec     54. 86.100 / 54. 86.100
      libavformat    54. 59.106 / 54. 59.106
      libavdevice    54.  3.102 / 54.  3.102
      libavfilter     3. 32.100 /  3. 32.100
      libswscale      2.  1.103 /  2.  1.103
      libswresample   0. 17.102 /  0. 17.102
      libpostproc    52.  2.100 / 52.  2.100
    So I tried two mp4 files I know have compatible codecs. Both transferred fine(specifically: without transcoding h264) as a pull and as a push. Hope this helps.
     
  4. wmcbrine

    wmcbrine Well-Known Mumbler

    11,534
    717
    Aug 2, 2003
    To be clear, I don't think there's any version of FFmpeg that's too new. I think jcthorne had updated his FFmpeg, but not his pyTivo.
     
  5. jcthorne

    jcthorne Well-Known Member

    3,116
    196
    Jan 28, 2002
    Houston
    Thanks guys. Updated ffmpeg again and went and got the latest pytivo and installed it all without any 'extra' family in the house and viola...It works. Was my error somewhere in the previous install.

    Appreciate everyones help and got me going.
     
  6. oregonman

    oregonman Member

    87
    0
    Jul 1, 2002
    Portland OR
    I'm trying to use your updated instructions and I noticed that in step 6, <install folder> is referenced, but I don't think it was explicitly defined. I think that it is the directory created in step 4, but for clarity, it would be good to explicitly define it.
     
  7. gonzotek

    gonzotek tivo_xml developer

    2,538
    59
    Sep 24, 2004
    Outside...
    That's the correct assumption and a good suggestion. I'll see what I can do to make it clearer.
     
  8. lpwcomp

    lpwcomp Well-Known Member

    9,461
    298
    May 6, 2002
    John's...
    A question and an observation.

    The question: Anyone know how "Categories" is set and if there is any way to access them? It is not the same as genre.

    The latest premiere s/w appears to have screwed up the genre entries in the details xml returned by a query. Below is an example. Note that both vSeriesGenre and vProgramGenre have multiple null element entries.

    Code:
    <TvBusMarshalledStruct:TvBusEnvelope xs:schemaLocation="http://tivo.com/developer/xml/idl/TvBusMarshalledStruct TvBusMarshalledStruct.xsd http://tivo.com/developer/xml/idl/TvPgdRecording TvPgdRecording.xsd http://tivo.com/developer/xml/idl/TvBusDuration TvBusDuration.xsd http://tivo.com/developer/xml/idl/TvPgdShowing TvPgdShowing.xsd http://tivo.com/developer/xml/idl/TvDbShowingBit TvDbShowingBit.xsd http://tivo.com/developer/xml/idl/TvBusDateTime TvBusDateTime.xsd http://tivo.com/developer/xml/idl/TvPgdProgram TvPgdProgram.xsd http://tivo.com/developer/xml/idl/TvDbColorCode TvDbColorCode.xsd http://tivo.com/developer/xml/idl/TvPgdSeries TvPgdSeries.xsd http://tivo.com/developer/xml/idl/TvDbShowType TvDbShowType.xsd http://tivo.com/developer/xml/idl/TvDbTvRating TvDbTvRating.xsd http://tivo.com/developer/xml/idl/TvDbInternalRatingList TvDbInternalRatingList.xsd http://tivo.com/developer/xml/idl/TvDbInternalRating TvDbInternalRating.xsd http://tivo.com/developer/xml/idl/TvDbBitstreamFormat TvDbBitstreamFormat.xsd" xs:type="TvPgdRecording:TvPgdRecording"><recordedDuration>PT30M</recordedDuration><vActualShowing><element><showingBits value="991235"/><time>2013-01-14T02:30:00Z</time><duration>PT30M</duration><program><vActor><element>MacFarlane|Seth</element><element>Schaal|Wendy</element><element>MacFarlane|Rachael</element><element>Grimes|Scott</element><element>Baker|Dee Bradley</element></vActor><vAdvisory/><showingBits value="0"/><vChoreographer/><colorCode value="4">COLOR</colorCode><description>Francine regrets deciding to teach Roger the value of hard work; Steve interviews Stan for a school project. Copyright Tribune Media Services, Inc.</description><vDirector/><episodeNumber>711</episodeNumber><episodeTitle>The Adventures of Twill Ongenbone and His Boy Jabari</episodeTitle><vExecProducer/><vProgramGenre><element/><element/></vProgramGenre><vGuestStar><element>Brady|Wayne</element></vGuestStar><vHost/><isEpisode>true</isEpisode><originalAirDate>2013-01-13T00:00:00Z</originalAirDate><vProducer/><series><isEpisodic>true</isEpisodic><vSeriesGenre/><seriesTitle>American Dad</seriesTitle></series><showType value="5">SERIES</showType><title>American Dad</title><vWriter/></program><tvRating value="5">_14</tvRating><RatingList><objectName/><vRatings><element><objectName/><vAdvisory><element>15</element><element>16</element><element>14</element><element>12</element></vAdvisory><InternalRatingSystemId>2</InternalRatingSystemId><InternalRatingValueId>5</InternalRatingValueId></element></vRatings></RatingList></element></vActualShowing><vBookmark/><showing><showingBits value="991235"/><time>2013-01-14T02:30:00Z</time><duration>PT30M</duration><program><vActor><element>MacFarlane|Seth</element><element>Schaal|Wendy</element><element>MacFarlane|Rachael</element><element>Grimes|Scott</element><element>Baker|Dee Bradley</element></vActor><vAdvisory/><showingBits value="0"/><vChoreographer/><colorCode value="4">COLOR</colorCode><description>Francine regrets deciding to teach Roger the value of hard work; Steve interviews Stan for a school project. Copyright Tribune Media Services, Inc.</description><vDirector/><episodeNumber>711</episodeNumber><episodeTitle>The Adventures of Twill Ongenbone and His Boy Jabari</episodeTitle><vExecProducer/><vProgramGenre><element/><element/></vProgramGenre><vGuestStar><element>Brady|Wayne</element></vGuestStar><vHost/><isEpisode>true</isEpisode><originalAirDate>2013-01-13T00:00:00Z</originalAirDate><vProducer/><series><isEpisodic>true</isEpisodic><vSeriesGenre/><seriesTitle>American Dad</seriesTitle></series><showType value="5">SERIES</showType><title>American Dad</title><vWriter/></program><tvRating value="5">_14</tvRating><RatingList><objectName/><vRatings><element><objectName/><vAdvisory><element>15</element><element>16</element><element>14</element><element>12</element></vAdvisory><InternalRatingSystemId>2</InternalRatingSystemId><InternalRatingValueId>5</InternalRatingValueId></element></vRatings></RatingList></showing><startTime>2013-01-14T02:29:58Z</startTime><stopTime>2013-01-14T03:00:00Z</stopTime><bitstreamFormat><vFormat><element><vByte><base64>EjQAAwABAjoBywxXAAAADwAAAAQAAAACAAAAAwAAAA==</base64></vByte></element></vFormat></bitstreamFormat><expirationTime>2038-01-19T03:14:00Z</expirationTime></TvBusMarshalledStruct:TvBusEnvelope>
    Edit: scratch the question. It appears that Categories is all of the vProgramGenre. However, there is an additional bug in the Premiere. On a transfer to the Premiere, whatever the source, if there is only one vProgramGenre entry, Categories gets set to the word "Separator" minus the quotes.
     
  9. Iluvatar

    Iluvatar New Member

    377
    0
    Jul 22, 2006
  10. wuzznuubi

    wuzznuubi Member

    130
    5
    Jan 16, 2013
    Screaming Fast! love the full pipe, almost 100Mb/s transfers, but...

    mp4(H264) pushes work fine, but having problems pulling the same files.

    The pull appears to progress normally until almost the end, but stalls close to the end and doesn't appear on the Premiere's TODO or NPL. After the aborted transfer, history on the Premier says "Not Transferred - This show was not downloaded onto this DVR because it exceeded the expected size."

    Windows7-64bit
    python v2.7.3
    wmcbrine pyTivo version 2013.01.15 from git
    ZERANOE WIN-64 ffmpeg build Jan 13, 2013

    Guessing here... that pyTivo tells my Premiere the size of the mp4 program stream file on my PC, but since pyTivo now remuxes the pull into a ts, it must be making the file too large (compared to what it said it was sending) for the Premiere, which complains by dumping the pulled transfer after it's almost complete.

    Is there any way to pad the filesize pyTivo tells the TiVo in this case? Would that work?
     
  11. wuzznuubi

    wuzznuubi Member

    130
    5
    Jan 16, 2013
    Info for one of the files that pushes, but not pulls...

    ffmpeg output
    Code:
    C:\ffmpeg\ZERANOE\bin\ffmpeg.exe -i "T:\pyTivoVideos\Test Folder\Test File.mp4"
    ffmpeg version N-48810-gaaa7d2f Copyright (c) 2000-2013 the FFmpeg developers
      built on Jan 13 2013 22:06:24 with gcc 4.7.2 (GCC)
      configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-lib
    p3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-amrw
    vorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib --enable-filter=frei0r
      libavutil      52. 14.100 / 52. 14.100
      libavcodec     54. 86.100 / 54. 86.100
      libavformat    54. 59.107 / 54. 59.107
      libavdevice    54.  3.102 / 54.  3.102
      libavfilter     3. 32.100 /  3. 32.100
      libswscale      2.  1.103 /  2.  1.103
      libswresample   0. 17.102 /  0. 17.102
      libpostproc    52.  2.100 / 52.  2.100
    Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'T:\pyTivoVideos\Test Folder\Test File.mp4':
      Metadata:
        major_brand     : isom
        minor_version   : 512
        compatible_brands: isomiso2avc1mp41
        creation_time   : 1970-01-01 00:00:00
        encoder         : Lavf52.32.0
      Duration: 00:49:11.04, start: 0.000000, bitrate: 25189 kb/s
        Stream #0:0(eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 24866 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc
        Metadata:
          creation_time   : 1970-01-01 00:00:00
          handler_name    : VideoHandler
        Stream #0:1(eng): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 320 kb/s
        Metadata:
          creation_time   : 1970-01-01 00:00:00
          handler_name    : SoundHandler
    At least one output file must be specified
    
    MediaInfo 0.7.61 output
    Code:
    General
    Complete name                            : T:\pyTivoVideos\Test Folder\Test File.mp4
    Format                                   : MPEG-4
    Format profile                           : Base Media
    Codec ID                                 : isom
    File size                                : 8.65 GiB
    Duration                                 : 49mn 11s
    Overall bit rate                         : 25.2 Mbps
    Writing application                      : Lavf52.32.0
    
    Video
    ID                                       : 1
    Format                                   : AVC
    Format/Info                              : Advanced Video Codec
    Format profile                           : Baseline@L4.1
    Format settings, CABAC                   : No
    Format settings, ReFrames                : 1 frame
    Format settings, GOP                     : M=1, N=12
    Codec ID                                 : avc1
    Codec ID/Info                            : Advanced Video Coding
    Duration                                 : 49mn 10s
    Bit rate                                 : 24.9 Mbps
    Width                                    : 1 920 pixels
    Height                                   : 1 080 pixels
    Display aspect ratio                     : 16:9
    Frame rate mode                          : Constant
    Frame rate                               : 29.970 fps
    Color space                              : YUV
    Chroma subsampling                       : 4:2:0
    Bit depth                                : 8 bits
    Scan type                                : Progressive
    Bits/(Pixel*Frame)                       : 0.400
    Stream size                              : 8.54 GiB (99%)
    Language                                 : English
    
    Audio
    ID                                       : 2
    Format                                   : AC-3
    Format/Info                              : Audio Coding 3
    Mode extension                           : CM (complete main)
    Format settings, Endianness              : Big
    Codec ID                                 : ac-3
    Duration                                 : 49mn 11s
    Bit rate mode                            : Constant
    Bit rate                                 : 320 Kbps
    Channel(s)                               : 6 channels
    Channel positions                        : Front: L C R, Side: L R, LFE
    Sampling rate                            : 48.0 KHz
    Bit depth                                : 16 bits
    Compression mode                         : Lossy
    Stream size                              : 113 MiB (1%)
    Language                                 : English
    
    VLC media player 2.0.5 Twoflower info
    Code:
    Title: Test File.mp4
    Encoded by: Lavf52.32.0
    Codec:
     Stream 0
      Type: Video
      Codec: H264 - MPEG-4 AVC (part10) (avc1)
      Language: English
      Resolution: 1920x1080
      Frame rate: 29.970029
      Decoded format: Planar 4:2:0 YUV
     Stream 1
      Type: Audio
      Codec: A52 Audio (aka AC3) (a52)
      Language: English
      Channels: 3F2R/LFE
      Sample rate: 48000 Hz
      Bitrate: 320 kb/s
    
    VideoReDo v4.20.7.629 info
    Code:
     File:                                     Name : T:\pyTivoVideos\Test Folder\Test File.mp4
                                               Size : 9.292 GB
                                           Duration : 00:49:10.24
                                           Mux type : MP4
     Video:                                Encoding : H.264
                                      VideoStreamID : x201
                                         Frame rate : 29.97 fps
                                      Encoding size : 1920 x 1080
                                       Aspect ratio : 16:9
                                    Header bit rate : 50.000 Mbps
                                         VBV buffer : 952 KBytes
                                            Profile : Baseline/4.1
                                        Progressive :  Progressive
                                             Chroma : 4:2:0
                                       Entropy mode : CABAC
                                           Bit rate : 23.883 Mbps
     Audio Stream: 1 (Primary)                Codec : AC3
                                             Format : AC3 stream
                                           Channels : 5.1
                                           Language : eng
                                                PID : x202
                                      PES Stream Id : xBD
                                           Bit rate : 320 Kbps
                                      Sampling rate : 48000
                                        Sample size : 16 bits
    
     
  12. wmcbrine

    wmcbrine Well-Known Mumbler

    11,534
    717
    Aug 2, 2003
    Yes -- and pyTivo has done that for a long time. Just not quite enough, in this case, I guess. But it's been years since I ran into this, personally.

    We can of course increase the pad factor, but there's a trade-off: the bigger the estimated size, the more the TiVo wants to clear existing programs to make room. Also, in push, it takes longer for the file to become playable. However, as I look at it, the current padding is only 2%. :) I think we can afford to increase that a bit, and probably should.

    In the meantime -- are you using "ts = on"? If not, try that. If you are, try pre-remuxing the file:

    ffmpeg -i filename.mp4 -vcodec copy -bsf h264_mp4toannexb -acodec copy filename.ts

    filename.ts can then be sent with an exact size.

    Edit: I think I see what's going on here -- pyTivo is using the specified bit rate to estimate, when it should be using the original. This works OK for the videos I've been using (well under 16 Mbps), but not for your file.
     
  13. wuzznuubi

    wuzznuubi Member

    130
    5
    Jan 16, 2013
    Yep
    Code:
    pyTivo.conf file
    [Server]
    beacon = 192.168.255
    ffmpeg = C:\ffmpeg\ZERANOE\bin\ffmpeg.exe
    port = 9032
    tivo_mak = **SECRET**
    tivo_password = **SECRET**
    tivo_username = **SECRET**
    togo_path = pyTivoVideos
    tivodecode = C:\tivodecode\tivodecode.exe
    tdcat = C:\tivodecode\tdcat.exe
    debug = True
    [B]ts = on[/B]
    
    [_tivo_HD]
    
    [_tivo_SD]
    
    [pyTivoVideos]
    force_alpha = True
    type = video
    path = T:\pyTivoVideos
    
    ffmpeg remuxing to ts replied "muxing overhead 7.938006%" and that worked, pull of same file muxed to ts first transferred in about 14 minutes...
    INFO-pyTivo.video.video:[17/Jan/2013 20:37:19] Start sending "T:\pyTivoVideos\Test Folder\Test File2.ts" to PR4
    INFO-pyTivo.video.video:[17/Jan/2013 20:51:34] Done sending "T:\pyTivoVideos\Test Folder\Test File2.ts" to PR4, 10027445112 bytes, 89.42 Mb/s
    Appears in the TiVo NPL (9.38GB) and plays fine.

    P.S. Filesizes;
    File size of original ps (mp4) 9,291,833,710 bytes
    File size after remuxing with ffmpeg to ts 10,027,445,112 bytes
     
  14. TVCricket

    TVCricket HDUI

    141
    1
    Mar 7, 2010
    Was using MetaGenerator before and after the Premiere updates, but no matter what I try, I can't get the episodes to group into Folders. I got one series to transfer over after updating to the latest version, but for whatever reason, Band Of Brothers and Homeland don't group. Can someone please help?
     
  15. lpwcomp

    lpwcomp Well-Known Member

    9,461
    298
    May 6, 2002
    John's...
    Pull or Push?
     
  16. TVCricket

    TVCricket HDUI

    141
    1
    Mar 7, 2010
    Not sure what the difference is. The way I transfer videos is by going to my Premiere and selecting the source folder on my PC and selecting each one individually.
     
  17. lpwcomp

    lpwcomp Well-Known Member

    9,461
    298
    May 6, 2002
    John's...
    If it is from "My Shows", then it is a Pull. It is from "Showcases & Apps" it is a Push.

    Post a couple of your metadata files for a series that should be grouping together. Also, are you running TiVo Desktop? If so, are you certain that you are selecting from a pyTivo share?
     
  18. TVCricket

    TVCricket HDUI

    141
    1
    Mar 7, 2010
    It says My Shows, don't have TiVo Desktop installed.

    This is one of the metadata files of the show that won't Group

    This does group

     
  19. TVCricket

    TVCricket HDUI

    141
    1
    Mar 7, 2010
    I noticed the difference in the *title* so I went in and changed the text file, but it still won't group the videos. What should the MetaGenerator 3 settings be? I tried every possible combination, but can't get it to work like it did before the latest update.
     
  20. moyekj

    moyekj Well-Known Member

    11,918
    664
    Jan 23, 2006
    Mission...
    FYI I took your Band of Brothers metadata file and used it along with a short clip pulled twice to TiVo and it did create a "Band of Brothers" folder with 2 items for me. If you pull the same video (with same metadata file) twice is it grouping?
     

Share This Page