1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New program for 1 step TTG downloads, decryption, encoding - kmttg

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Mar 15, 2008.

  1. Nov 2, 2011 #3461 of 10928
    ThAbtO

    ThAbtO TiVoholic by the bay

    7,283
    60
    Apr 6, 2000
    SF Bay Area
    The video are .mpg with .tivo wrapper/encoded, .avi is also a wrapper.

    If you want to make the files smaller, you need to convert to .mp4.
     
  2. Nov 2, 2011 #3462 of 10928
    ig88

    ig88 New Member

    51
    0
    Feb 1, 2003
    Generically speaking:
    First of all, people all have their personal quirks, so there are going to be different opionions here, based on personalities – there is no right/wrong answer.

    Any time you encode something, you are going to lose some quality

    You already read about the SD vs HD issue. Obviously, HD Is gonna be better in most ways.

    However, the specific problem I was having with SD was playback on the computer, because the computer software wasn't automatically doing some video cleanup that my TV was. But you aren't going to have that problem if your intention is just to offload from TiVo for storage to be later played back on TiVo when you are ready to watch – it is going to look as good as it presently does from your TiVo, so don't sweat the small stuff.


    Regarding stripping commercials, it can be hit or miss. Lots of misses. And it can take a relative amount of time and effort. To me, I would look at it this way:
    Keep it simple.
    Pull it off the TiVo, archive it, put it back on when you are ready to watch.
    Simple, not time consuming.
    Will always work.
    No video loss.
    And you can spend your time doing other things.

    You can just use the traditional TiVo tools to ff/skip commercials.

    My 2¢
     
  3. Nov 2, 2011 #3463 of 10928
    moyekj

    moyekj Well-Known Member

    11,377
    126
    Jan 23, 2006
    Mission...
    I agree with ig88. If the intent is to watch the videos on a TiVo at a later date then just leave them as .TiVo files (no need to even decrypt them to .mpg). No loss of quality and no hassles. Then you can use pyTivo to pull or push them to viewing TiVo as necessary and skip commercials using FF, 30s, etc. With Premieres the transfer speeds are much faster than Series 2 TiVos so even HD .TiVo files transfer quicker than you can watch them.

    Perhaps one thing that will help if you have lots of downloads and want kmttg to auto-categorize them into folders is use folder keyword in file naming configuration option. See Wiki for more details (look at File Naming section):
    http://code.google.com/p/kmttg/wiki/configuring_kmttg
     
  4. Nov 3, 2011 #3464 of 10928
    Hercules67

    Hercules67 TiVo addict

    356
    0
    Dec 8, 2007
    Great answers, especially yours moyeki! Much appreciated.

    Ok, let me get a little bit more specific. You're probably right, and skipping commercials doesn't really bother me with the hacks in place. what I am worried about, is archive size. Over the last 3 years for example I've been saving Christmas shows for the kids, nephews and nieces. They've been set aside and they can view them any time they want, not when the networks show them. I mentioned Burn Notice, but for example with Merlin, I have 3 seasons I am behind. So the size is of the archive... Anyway. Thanks for the help guys. I'll play with Handbrake and the other encodings and see what I get.

    Much appreciated!:up:
     
  5. Nov 3, 2011 #3465 of 10928
    Hercules67

    Hercules67 TiVo addict

    356
    0
    Dec 8, 2007
    Thank you!:up:
     
  6. Nov 3, 2011 #3466 of 10928
    wmcbrine

    wmcbrine Ziphead

    10,450
    60
    Aug 2, 2003
    It does have the advantage of keeping the metadata and video together in one file. On the other hand, I've found that an unencrypted .mpg transfers just slightly faster than the equivalent .TiVo file.
     
  7. Nov 3, 2011 #3467 of 10928
    sanjonny

    sanjonny New Member

    202
    0
    Nov 2, 2008
    Scroll back a bit and check out a few tools I suggested. If you want to edit out commercials and really who likes skipping commercials (#1 reason I still have my replay tv) I would suggest video redo. What I do is use the automated tools in kmttg to fix and adscan. I then manually open the adscan files later in videoredo ( I can do like 20 shows in a few minutes) to check the accuracy of the cuts save the project file and then go back to automated kmttg to cut the commercials using kmttg. I also do the same thing for my replaytv shows and others. So I basically batch download a bunch of files and process them to a point in kmttg, wait till I have a bunch to work on, open them and double check the commercial detection, save them, then batch cut and do whatever other process I need to archive them. For most shows I then run them thru avi.net to convert/shrink to xvid and discard to original tivo mpg to history. My server thanks me as I can take say 40 gb of TiVo mpg and shrink it to 4-8 gb with either low or no perceptible quality loss.

    I agree that feeding back unencrypted mpgs is a bit better on the network side of things. If you are using xvid Avis, you have to use stream baby to view on the TiVo, but not on the computer or most set top boxes.

    If you are already pullingthe files off and plan to keep them awhile, the commercial editing is painless with the manual though quick step. If you want to then save space, I highly recommend the xvid codec and the tools mentioned before, since it plays great on computers and the tivos and other devices with minimal conversion. Unfortunately, if you use h264 to shrink your files or mp4 style, many computers will spend lots of time decoding them resulting in stutters and all that. Xvid doesn't have that problem. I use a set top box that hardware decodes everything, so that isn't an issue, but if I want to play back the file at a later date on my net book, I shrink it using avi.net/xvid. You can save lots of space with minimal quality loss especially if you use the defaults in avi.net, though sometime it calculates based on minutes so a file that is 45 minutes it might tell you to use 700mb when a 44 minute file would be 350, so after a bit of experience you learn to set it at 350. I find for SD content, the quality loss is not even noticeable converting to xvid/avi from SD TiVo but can take a 1.5 gb file down to 300 mb. For HD it is even more efficient. And pretty fast too, if you have a bit of processor power, much faster than the built in encoders in video redo or handbrake.
     
  8. Nov 3, 2011 #3468 of 10928
    txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    Things to consider when moving from S2 to something like a Premiere, if most of the stuff you want to transfer is on cable networks (USA/BBC, etc) then moving away from analog recording (S2) to digital recording (Premiere with cable card) will most likely kill your ability to download those shows to your computer. Many Some cable providers do not allow transfers on anything but network shows.

    As far as commercial removal goes, videoredo is definitely the way to go. There is a tool that comes with VRD that is really irreplaceable for Tivo video editting (Quick Stream Fix...fixes video time stamp errors that cause A/V desync). I end up downloading, decrypting and ad scanning shows using kmttg. I then manually check commercial cuts using VRD and save cut files from there (rather than having kmttg scan and cut without checking). It normally takes me about 2-3 minutes to verify/modify the commercial cuts per 1 hr episode. I then manually transcode from that point. You can easily bring those cut videos back in kmttg and use one of the encoding profiles from there. Or you can open the video in Handbrake...or MeGUI or command line tools. There are a lot of choices.

    Personally, I would transcode to H.264/MP4 with either AAC or AC3 audio. I wouldn't go the MPEG4/AVI route. There is just more support for MP4 files nowadays and the compression is better. It is more demanding on systems for decode, but that isn't much of a problem anymore. For removing inverse telecine or deinterlacing from video while transcoding to H.264/MP4, I think you will find Handbrake easier than most other conversion GUIs.
     
  9. Nov 3, 2011 #3469 of 10928
    lpwcomp

    lpwcomp Well-Known Member

    8,733
    117
    May 6, 2002
    John's...
    Some cable companies are setting the CC1 flag on just about everything. Some aren't. Unless you have actual figures to back it up, the statement that most are is pure speculation.
     
  10. Nov 3, 2011 #3470 of 10928
    txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    Edited. Happy?
     
  11. Nov 3, 2011 #3471 of 10928
    ig88

    ig88 New Member

    51
    0
    Feb 1, 2003
    Couple of other thoughts here:

    If you are REALLY going to watch those shows, its one thing. But if you have so many years of so many shows that you are behind, might need to ask yourself if it's realistic.

    Meanwhile, here is another consideration - maybe look at the problem from a different angle:
    a) Maybe you have some old unused hard drives laying around that can be used to offload the - we only watch this once a year, seasonally content, freeing up that space on your main argive

    b) Purchase something like a four- or eight-bay Drobo. That way you could invest in one hardware storage system, and as you start to outgrow it you can expand by adding additional drives, and/or bigger drives
     
  12. Nov 3, 2011 #3472 of 10928
    lpwcomp

    lpwcomp Well-Known Member

    8,733
    117
    May 6, 2002
    John's...
    Ecstatic.:rolleyes:
     
  13. Nov 3, 2011 #3473 of 10928
    dlfl

    dlfl Cranky old novice

    7,132
    64
    Jul 6, 2006
    Dayton OH
    Just to be a little more specific:
    If your cable co is Time Warner, the probability of copy protection on all but local stations is 100% or darn close to it.

    There is a thread where user reports of copy protection for various cable cos and regions are tracked:
    http://www.tivocommunity.com/tivo-vb/showthread.php?t=451107\
     
  14. Nov 3, 2011 #3474 of 10928
    innocentfreak

    innocentfreak Active Member

    8,986
    6
    Aug 25, 2001
    Florida
    Missing Remote also maintains a list since some people were asking before using Sage TV with CableCARD tuners.
     
  15. Nov 3, 2011 #3475 of 10928
    Hercules67

    Hercules67 TiVo addict

    356
    0
    Dec 8, 2007
    I am ecstatic with the answers I have received. Thanks guys.

    To be honest, I too don't know whether I'll ever catch up with a show I'm 3 years behind on. On the other hand, My wife is facing surgery, and so am I, so we'll be home bound for about a month soon, so that might be what we'll be watching. :)

    Still, I am going to experiment with mp4 and avi as well and see what makes me happier. Obviously, KMTTG doesn't convert to AVI so I need to get some additional tools here.

    Thanks for the insight.

    My PC is dedicated to converting video, but it is a couple of years older....
     
  16. Nov 3, 2011 #3476 of 10928
    ThAbtO

    ThAbtO TiVoholic by the bay

    7,283
    60
    Apr 6, 2000
    SF Bay Area
    You can use VideoReDo TVSuite to take the.tivo and .mpg files and burn to DVD and carry around a DVD Player, if you want portability. It would be lighter than carrying around a laptop.
     
  17. Nov 3, 2011 #3477 of 10928
    moyekj

    moyekj Well-Known Member

    11,377
    126
    Jan 23, 2006
    Mission...
    You can look look at ff_zen profile in kmttg which does create mpeg4 video in avi container using ffmpeg and then perhaps make your own custom encoding profile based on that (copy encode\ff_zen.enc to a different name and edit that file). With kmttg you can basically create any format that ffmpeg, handbrake or any other batch encoder supports since you can create your own encoding profile.
     
  18. Nov 4, 2011 #3478 of 10928
    sanjonny

    sanjonny New Member

    202
    0
    Nov 2, 2008
    I am still playing a bit with profiles, batch files and front ends, but one thing I have found and not quite sure why, is that when I use the handbrake profiles I sometimes suffer out of sync audio or other issues (I qsfix everything) but I never have run into that with either of the two main tools I use to convert to xvid or mp4 or mkv. I know that there is some error checking the guy who wrote the programs has built in to fix that, but I have not gotten a response yet on exactly what it is.

    I will also throw back my two cents about the h264 vs xvid. H264 is more efficient, but I truly cannot play the videos at all on some of my older devices like an older laptop or even my less than a year old net book due to stuttering and such, and encoding/decoding takes double to triple program time on my quad core 3 ghz computer at 100 % utilization. For xvid conversion of hidef it takes about half program time, so most of my tivo downloads are converted to xvid. Mp4 is a container, (getting into a whole world of confusion of container vs encoding) and quite frankly, I find for h264 encodes, it is much better to use the mkv container instead, because it allows multiple audio streams and linking chapters and all that, which means in one file i can have subtitles, director commentary and other audio streams. Mp4 doesn't allow that, only one audio stream and I can't remember about subtitles) The linking thing in mkv/matroska is cool too for example if you have several episodes that all have the same beginning, you can save that part only once (say for BS Galactica) for the whole season and then you have that one file play at the correct point for each episode, but don't use the space up since for 16 episodes, the 2 minute segment is the same so you just call that link and save 32 minutes of disk space.

    Once you have a h264 file in a mp4 or mkv container, you can usually switch containers easily without much processing time, it's getting it into the 264 format that is the problem.

    On size difference, I find that the savings in general is not worth the extra encoding time. For example, a hd 30 minute program that I compress down to say 300 mb on xvid could basically go down to about 200 mb in h264 with about the same quality, (really it is more like 250 for h264 vs 300 for xvid but slight format changes occur) but one encode takes 15 minutes and the other takes about an hour and 8 minutes. That adds up when you start doing a lot of encoding. BUT, I have a new device the hauppauge 1212 that hardware encodes in h264 on the fly. Most of those files I then might repackage in a mkv container but I cannot play them on my net book or such. I specifically got that device because of the cable card nonsense with blocking channels which is variable thru the country. I am about half blocked, my friend has a cable system that blocks everything but the local hd and a friend has Nothing blocked ( boy would that not be nice)

    Since I have spent a lot of time testing out these various things, I have become really familiar with many tools and formats good and bad, I am not really biased on anything other than I have tried them all and ended up where I am at. If I could solve my random sync issues with kmttg encodes, I would probably do all my regular encodes that way (I think I could do xvid thru it with the right profile setup) but part of the reason I think the say avi.net tool works so well is it examines each file (automated thru index.net) and you set the encode by hand for specifics related to that file and then batch encode. My guess is, in that process it corrects the audio sync that handbrake doesn't do in mass encode format. I might be wrong on that, still learning, but all the built in profiles and even some of my own in kmttg encoded do randomly ha e audio sync problems. And again I don't know about xvid encoding, haven't researched that yet, working on other things but I think at worst I can have kmttg call up index.net to run on each file when I get back to that. I am still working on getting kmttg to deal with new formats from the 1212 both h264 .m2ts (bluray) and .ts (h264 transport stream)

    above all, kmttg is the BEST software out there for what it does, and moyekj continually makes it better and fixes bugs, which is even better. I am going above and beyond what it is designed to do, however anyone who archives their video files will start hitting the same walls I am hitting and want to find legal workarounds (the 1212 is one till they close up the analog component outputs) for cable card nonsense.

    And just so you understand, I know exactly where you are coming from. I started out archiving Xmas shows for my wife, and when I got my tivo hd I got better quality shows via digital, which she and my nieces and nephews appreciate every year. Being a long time replaytv devotee, I used wirns which is like kmttg for replaytv though it focuses more on channel guides and recordings than downloads and processing and when I got the TiVo I looked for something similar and that is how I found kmttg, which now is getting into the scheduling and channel guides and such.

    Now I archive lots of shows for the series I like, most of them I watch over and over again.

    It all started with saving some Xmas shows...then Charlie brown, then I thought, hey what about south park for me, I love watching this reruns and years later, look what happened....
     
  19. Nov 4, 2011 #3479 of 10928
    txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    @sanjonny-

    The audio desync issues in Handbrake are most likely due to the way it handles hybrid video (video with varying framerates). Many shows are filmed at 24p but include CGI graphics at a different framerate or include interlaced credits at 29.97fps or if you don't edit out commercials than you can get 29.97fps segments. There are a lot of different scenarios. But the main issue is that Handbrake is perfectly happy to transcode with variable framerates. It is actually pretty difficult to get it to NOT do that (transcode with constant framerate). The sync timing is many times affected by this.

    I run all of my Tivo transcodes through command line batch files. I have found my best results using avisynth and some encoding tool rather than using a GUI. It's more work though.
     
  20. Nov 4, 2011 #3480 of 10928
    sanjonny

    sanjonny New Member

    202
    0
    Nov 2, 2008
    Thanks txporter, so of course my question is, if handbrake can deal with the varying frame rates and such, why can't it also adjust the audio? It seems to be a pretty intelligent tool, but i know after reading many forums that this is an ongoing thing with the sync issues. I guess I don't completely understand qsfix in video redo either because I thought it basically if you selected the right stream option corrected the variances in recording by forcing it all to one resolution, but maybe it doesn't fix framerate? the weird thing is, I believe avi.net is basically a gui front end for handbrake using the xvid converter, but it never has sync issues, so somehow he fixed that. Again I have asked how but have not heard back.
     

Share This Page