1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New program for 1 step TTG downloads, decryption, encoding - kmttg

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Mar 15, 2008.

  1. Jun 8, 2009 #821 of 10616
    moyekj

    moyekj Well-Known Member

    11,270
    84
    Jan 23, 2006
    Mission...
    I just created a pyTivo_push wiki page summarizing how to use kmttg "custom" job to setup kmttg to push videos to series 3 tivos using pyTivo. This is based on discussions not long ago in this thread and is a good example of how to use kmttg "custom" job feature to do something useful.
    (The wiki page contains a link to the python script used to do the push).
     
  2. Jun 9, 2009 #822 of 10616
    bkdtv

    bkdtv New Member

    7,902
    0
    Jan 9, 2003
    DC Metro Area
    Moyekj,

    Please consider renaming VRD QS Fix to "Stream Fix" or "Fix A/V Errors" and implement a freeware alternative to VideoRedo Quickstream Fix. You can do that with the mpeg2repair utility. The functionality won't be nearly as robust, but it will be sufficient to eliminate a significant percentage of A/V sync errors that would otherwise occur when encoding.

    To do that, I believe you would need to first remux the MPG file into a TS, as that is the container required by Mpeg2Repair. You can remux MPG into TS using command line parameters with TsRemux.

    Code:
    TsRemux v.0.0.21.2
    usage: tsremux input-file output-path [-a] [+b] [+m] [+c]
       -a: do not use async io (default on)
       +b: bypass audio alignment (default off)
       +m: trueHd to ac3 (default off)
       +c: keep console open when done (win2k)
       output extension controls processing:
           ts, m2ts, none for bluray directory
    Mpeg2Repair doesn't have command line options, but there are at least two command line utilities to work around that. Here's one from this thread; it should be placed in the same folder with Mpeg2Repair, and there is only one argument needed, the input file. Another alternative is Mpeg2repairHelper, where you specify both the input and output file as parameters in quotes. These command line tools work by filling in all the appropriate fields in Mpeg2Repair.

    I'm assuming that you would add these programs to the set of helper tools.
     
  3. Jun 9, 2009 #823 of 10616
    moyekj

    moyekj Well-Known Member

    11,270
    84
    Jan 23, 2006
    Mission...
    bkdtv. I'm aware of those tools, but a few issues:
    * Output of mpeg2repair is still in mpeg2 transport stream format which is not very convenient if mpeg2 is your stopping point (but is probably OK if followed by encoding step).
    * One could add yet another free utility to mux from transport stream back to mpeg2 program stream, but seems like chances of introducing A/V sync issues after MPG->TS->mpeg2repair->TS->MPG are pretty high.
    * mpeg2auto still brings up the GUI when it runs (not very nice/feasible if running in service mode)
    * Windows only solution (but then again so is VideoRedo of course)
     
  4. Jun 9, 2009 #824 of 10616
    fyodor

    fyodor Member

    535
    1
    Sep 19, 2006
    Neat! I can't wait to try this out. FYI, I think that there's a typo in the header "to push files to your computer"

    F


     
  5. Jun 9, 2009 #825 of 10616
    bkdtv

    bkdtv New Member

    7,902
    0
    Jan 9, 2003
    DC Metro Area
    moyeki,

    I'm aware of those issues. But what's the alternative? Encode functionality just doesn't work for many people without some sort of stream fix.

    Without VideoRedo installed, you could make that "Fix A/V Errors" option selectable only when "Encode" is selected, and not selectable under autotransfers (service mode?). That makes some sense anyway, because most A/V errors do not noticeably impact playback under Windows; they only come into play when encoding, where the result is out-of-sync audio or choppy video.

    Might there be some way to hide the Mpeg2Repair GUI after the text is entered?
     
  6. Jun 9, 2009 #826 of 10616
    moyekj

    moyekj Well-Known Member

    11,270
    84
    Jan 23, 2006
    Mission...
    Thanks, typo fixed.
     
  7. Jun 9, 2009 #827 of 10616
    moyekj

    moyekj Well-Known Member

    11,270
    84
    Jan 23, 2006
    Mission...
    Seems feasible to tie to encoding step when VRD is not available. You mentioned earlier using some ffmpeg encoding profiles without VRD QS Fix resulted in A/V problems. Can you confirm that running those sources through mpeg2repair and then ffmpeg indeed solves the problems?
    (I've heard mixed reviews about mpeg2repair actually working, and the software is no longer being updated).
     
  8. Jun 9, 2009 #828 of 10616
    bkdtv

    bkdtv New Member

    7,902
    0
    Jan 9, 2003
    DC Metro Area
    Yes, it did.

    I downloaded two 30min SD programs. When played in WMP and VLC, both MPGs were in sync. When the original MPG was re-encoded with ff_ipod_low_res and ff_ipod_high_res by kmttg, both exhibited audio that was 2-3 seconds out of sync.

    I used tsremux to remux the original MPG into TS. Then I ran the TS files through mpeg2repair. After that, I renamed those files with the MPG extension so kmttg could see them. I encoded again with the same ffmpeg profile and the audio was in sync.

    Note mpeg2repair will not fix time gaps, so I don't think it's a solution for the errors that are sometimes introduced by comcut. You would only use it on the original MPG (after TS remuxing).

    Perhaps you should wait on this until I (and others) can do more testing to confirm this is a reliable fix. I pm'd the link to your previous message to a few people who asked me about this problem.

    Edit: I agree that mpeg2repair is a poor substitute for VideoRedo. VideoRedo will fix many problems that mpeg2repair will not, time gaps being one example.
     
  9. Jun 9, 2009 #829 of 10616
    moyekj

    moyekj Well-Known Member

    11,270
    84
    Jan 23, 2006
    Mission...
    OK thanks. So in terms of coding the algorithm would be something like:

    IF windows AND (NOT vrd) AND (encoding enabled) THEN "Fix A/V Errors" enabled

    Then at run time:
    IF ("Fix A/V Errors" enabled) THEN tsremux->mpeg2repair etc. would take the place of the normal VRD QS Fix job.
     
  10. Jun 10, 2009 #830 of 10616
    moyekj

    moyekj Well-Known Member

    11,270
    84
    Jan 23, 2006
    Mission...
    Did some testing tonight with mpeg2repair. I found a three-minute replay episode of Rescue Me that without VRD QS Fix has a lot of timestamp issues. During encoding ffmpeg generates hundreds of these errors:
    [mpeg @ 0x1042960]Invalid timestamps stream=0, pts=26782165, dts=26782166, size=9364

    So I ran it through tsremux and then mpeg2repair and then ffmpeg and still see the errors at exactly same points in ts stream:
    [mpegts @ 0x2db2960]Invalid timestamps stream=0, pts=26782165, dts=26782166, size=160
    Playing back resulting mp4 encoding has gross A/V sync.

    Ran the mpeg through VRD QS Fix instead and then same encoding and you guessed it: no ffmpeg timestamp error messages and perfect A/V sync in the resulting mp4 file.

    So unfortunately mpeg2repair didn't seem to help at all from my brief testing.

    If there is something free out there that works it would indeed be a very nice enhancement to deal with these issues. Wish VideoRedo offered a QS Fix only cheap version of their tool.

    P.S. I ran the same broken timestamp mpeg through handbrake psp profile just to see what happened and the result was even worse than the ffmpeg encoding - broken up audio and jumpy video throughout.

    These experiments to me always lead to the same conclusion: VideoRedo is really a must have tool if you do any kind of video processing... I have VRD QS Fix permanently enabled in my kmttg settings.
     
  11. Jun 10, 2009 #831 of 10616
    zabolots

    zabolots Member

    79
    0
    May 24, 2007
    NW Chicago...
    Which version of VideoReDo do you suggest, Plus or TV Suite?
     
  12. Jun 10, 2009 #832 of 10616
    moyekj

    moyekj Well-Known Member

    11,270
    84
    Jan 23, 2006
    Mission...
    I have no need for DVD authoring, so Plus is what I have. I'm using latest beta version of Plus which basically shares same code set as TV Suite minus the DVD authoring part.
     
  13. Jun 10, 2009 #833 of 10616
    moyekj

    moyekj Well-Known Member

    11,270
    84
    Jan 23, 2006
    Mission...
    ProjectX looks like could be interesting solution for fixing mpeg2 files. It's a Java based solution which means not restricted to Windows only. It allows you to demux files via command line and is supposed to fix problems as it demuxes. It will demux video and audio to separate files, but that's not a problem since for encoding stage you can just use the separate audio and video files as inputs.

    So the flow to fix and then encode would be something like:

    java -jar ProjectX.jar -demux -out /output/dir file.mpg
    ffmpeg -i file.m2v -i file.ac3 ..... file.mp4

    I will try it out tonight on my trouble mpeg2 testcase to see if it works.
     
  14. Jun 10, 2009 #834 of 10616
    bkdtv

    bkdtv New Member

    7,902
    0
    Jan 9, 2003
    DC Metro Area
    After more testing, mpeg2repair is not reliable. Even on the files I did earlier, the A/V still lost sync at a later point (I did only watched the beginning). Hopefully, ProjectX will do better.
     
  15. Jun 10, 2009 #835 of 10616
    bkdtv

    bkdtv New Member

    7,902
    0
    Jan 9, 2003
    DC Metro Area
    Does anyone know of command line utilities to automatically import videos with metadata into (a) Windows Media Player, or (b) iTunes?

    Several other GUI encoding tools, including TiVo Desktop Plus, have that feature.
     
  16. Jun 10, 2009 #836 of 10616
    moyekj

    moyekj Well-Known Member

    11,270
    84
    Jan 23, 2006
    Mission...
    Just logged into home and tried it out on my testcase - result was still pretty gross A/V sync problems even though ffmpeg did not spit out hundreds of warnings during the encode. So no go on this one either...
     
  17. Jun 10, 2009 #837 of 10616
    moyekj

    moyekj Well-Known Member

    11,270
    84
    Jan 23, 2006
    Mission...
    By requests earlier in this thread I added AtomicParsley step that will run automatically following mp4 encoding step in kmttg to add metadata information from pyTivo metadata file to the mp4 file. So if you enabled "metadata" step along with "encode" you will see an "atomic" job will run following the encode. Then if you use "mediainfo" or some other utility you will see the metadata information that was added that presumably iTunes uses. (I don't have/use iTunes so can't verify it).
    (NOTE: This only applies to mp4 or m4v containers)
     
  18. Jun 10, 2009 #838 of 10616
    KateS

    KateS New Member

    7
    0
    Apr 19, 2009
    speaking of metadata¬Ö
    I decrypted and transcoded a .tivo file to .mp4 last week using the existing ff_h264_high_rate profile on kmttg v0p6c. Had to leave town before I got a chance to check out the results. I'm back and ready to work on this over the weekend.

    1. When I tried to open the file using QuickTime Player (Mac), I get the following error: "The movie could not be opened. An invalid sample description was found in the movie." It won't import into iTunes (tried dropping it in and opening from within, neither work.) Haven't found any reference to similar errors. The text file opens ok. Any ideas?

    2. I notice that episode number and description don't seem to be a part of the metadata file created (at least not part of the text file; not sure what's actually embedded in the .mp4 since I can't open it.) Is there any way I can add episode number and description to the metdata file generated, and then (if possible) include it in the .mp4 for importing into iTunes?? I sure miss that about TDM. Perhaps I'm doing something wrong?

    3. Didn't have a chance to try adding -threads to my ffmpeg profile before leaving town. AudioNutz, I have a VH user license as well, and want to use -threads auto (rather than specifying a number.) Is there something I need to do within kmttg so it recognizes my VH license? (You mentioned you thought only the licensed VH works with -threads auto; otherwise it requires a number of threads to be specified¬Ö)

    In any case, perhaps I should get the existing ffmpeg profile working before "enhancing" it with my own added arguments!! hehe

    cheers
     
  19. Jun 10, 2009 #839 of 10616
    moyekj

    moyekj Well-Known Member

    11,270
    84
    Jan 23, 2006
    Mission...
    You can use mediainfo to see what metadata information is stored in the mp4 file.
    As I mentioned above with kmttg configured with working AtomicParsley (included in Mac OS tools package) you need to have "metadata" and "encode" tasks both enabled. Then after encode task completes kmttg uses the information in the pyTivo .txt file to run AtomicParsley to add some of that information to the mp4 file.
    If the pyTivo .txt file doesn't contain a description then that means there was no description available to begin with from the TiVo download or .TiVo file you started from. (In almost all cases I've seen there is some description available for most TiVo show downloads).
     
  20. Jun 11, 2009 #840 of 10616
    moyekj

    moyekj Well-Known Member

    11,270
    84
    Jan 23, 2006
    Mission...
    bkdtv, OK I found something that worked to fix A/V sync issues after encode for my trouble mpeg testcase:

    Run the source trouble mpeg through mencoder as follows:
    mencoder INPUT -oac copy -ovc copy -of mpeg -mpegopts format=dvd:tsaf -vf harddup -o OUTPUT

    Then take the OUTPUT mpeg generated by the above and run it through normal encode process.

    I'd be curious to see if that fixes the issues for your trouble testcases as well.
     

Share This Page