1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New program for 1 step TTG downloads, decryption, encoding - kmttg

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Mar 15, 2008.

  1. Oct 1, 2010 #2701 of 10413
    larrs

    larrs Movie Fan-Addict

    1,011
    0
    May 2, 2005
    DFW
    Would it be possible for kmttg to correct for errors in the recording when it downloads to the PC? From time to time (40%) I get partial recordings due to dropouts in the video/audio of the recorded program. Since kmttg reports the size of the file in the NPL, would it be possible to somehow stop this partial recording from happening or even happening in 2 (or more) parts?

    It can really be a pain to find out where the error occurs, go to one Tivo to find that place in the recording, pause right after the error, go to another Tivo, transfer the recording to that Tivo from the paused point, wait for the transfer and then go back to kmttg to download the second section.

    I am sure this is not a simple undertaking and I am not a programmer (and I did not stay in a Holoday Inn Express last night) so I do not know the effort involved. Just an inquiry...
     
  2. Oct 1, 2010 #2702 of 10413
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    I'm using non-cygwin version 7.18.0. I also tried with non-cygwin version 7.20.1 and it worked. However when I switched to cygwin version 7.16.3 it did have the exit code 52 issue as you originally reported, so obviously something wrong with cygwin version. Java mode download worked fine with same file with apostrophe.
    So I guess the lesson here is that cygwin versions MAY give you issues such as this so better off sticking with native windows versions.
     
  3. Oct 1, 2010 #2703 of 10413
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    Something automated I doubt could be implemented. I'm not even sure if downloads with a specified byte offset are possible from a TiVo web server though I haven't looked into that. That may be an interesting option to explore, but even that would have issues since when you download from a TiVo you are not downloading the original file stored on the TiVo - it's converting file on the fly which is the reason why glitches can make that fail in the first place. So because of that I think byte offset is not possible. A time offset would be the ideal solution but I doubt there's a way to specify a time offset for a TiVo download.
     
  4. Oct 1, 2010 #2704 of 10413
    txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    You will also need to open up a command prompt window and run 'HandBrakeCLI --preset-list' from the Handbrake directory to get the list of presets and the flags they set. The list on the website that I linked to you are not up-to-date but the command flags from there are fine.
     
  5. Oct 1, 2010 #2705 of 10413
    wmcbrine

    wmcbrine Ziphead

    10,369
    22
    Aug 2, 2003
    I don't believe they are, via TTG. I tried several ways to do it and got nowhere. They are of course possible (with time offset) via MRV, but I don't know of a non-TiVo implementation of that besides tivoserver, which AFAIK only works with hacked units that have had encryption disabled.

    Interestingly, the TiVo itself will resume failed TTCB transfers using a byte offset, but the method it uses doesn't seem to work in the other direction.
     
  6. Oct 1, 2010 #2706 of 10413
    txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    Ok, I spent a little time today messing around with the profile I sent you. It isn't what you want. The presets set one of the anamorphic modes and it cannot be turned off. Here are a few other versions that I built for you. You can download them and remove the '.txt' and just drop them in your encode folder.

    The first one is what you asked for. The second one includes detelecine which I prefer because it gets rid of the double frames that cause the slight jerks in high motion scenes. The third one includes detelecine and uses quality based encoding rather than average bitrate (your 2500kbps number). You really don't need anywhere near 2500kbps for SD video transcodes. I was working with a 2min sample of Rescue Me (from DVD). Here is an example of the size:
    source - 2min Rescue Me = 106mb
    shelleye profile = 38.3mb
    shelleye2 profile = 31.3mb
    shelleye3 profile = 16.4mb

    Visually, I cannot distinguish between shelleye2 and shelleye3 encodes. I assume that you are using this profile to put onto an Ipod or something? Try an encode or two with the profiles and see what you like.

    NOTE: This profile works well for 720x480 video (what you will get off of NTSC DVDs). Some of the recordings on Tivo are 544x480. Don't know if this profile will work well for that since the width and height are specified and 640>544.
     

    Attached Files:

  7. Oct 2, 2010 #2707 of 10413
    shelleye

    shelleye New Member

    39
    0
    Nov 28, 2002
    Oh my gosh!! Thank you thank you thank you. I only came across this reply by accident, because I was just starting to try and build my own command line, and thought I would re-check the thread to confirm what you posted before. For some reason I wasn't getting thread updates via email. Anyway, I am so grateful, and really looking forward to trying these out. I'm actually using this encode for a Roku box, which I use to stream media and photos from a NAS. The Roku has a few very specific requirements, or else it chokes. My iPod touch, which I also use, is more forgiving, and seems to be able to use the same encodes. I can't wait to see if this works. Did I say thank you? Thank you!!!!
     
  8. Oct 3, 2010 #2708 of 10413
    innocentfreak

    innocentfreak Active Member

    8,950
    3
    Aug 25, 2001
    Florida
    Do most of you use the original air date to keep the shows in order?

    I am trying to find an easy way or tool that will auto convert a show with air date to one with the correct season and episode.

    For example take Supernatural. If I transfer the episode from Friday, it will most likely transfer as Supernatural - 10/1/10 depending on your settings. I am just trying to find a way to convert to Supernatural - Two and a Half Men -S06E02.

    I know there are renaming tools if I have the mpg file with just the show name and season and episode number, but didn't know if there was one for show and airdate.
     
  9. Oct 3, 2010 #2709 of 10413
    ThAbtO

    ThAbtO TiVoholic by the bay

    6,739
    9
    Apr 6, 2000
    SF Bay Area
    I miss having a estimated time remaining for my downloads on kmttg. Its been years since I had forgotten the formula to calculate. I wonder if it can be done for kmttg, even though its only an estimate.
     
  10. Oct 3, 2010 #2710 of 10413
    shelleye

    shelleye New Member

    39
    0
    Nov 28, 2002
    I have tested all three of the profiles, and you are correct, the third one works just as well as the first two, to my eye anyway, and is half the size. I'm frankly baffled: my file sizes are going from 2-4G as .mpg, to 500M as .m4v. Is this truly how much it compresses? I guess it must be, because I'm watching it. But, wow.

    Anyway, I have not run into any problems with syncing or streaming, so it might make sense to standardize the third version as a Handbrake Roku streaming profile. It's working beautifully for me. Did I say thank you? :D
     
  11. Oct 4, 2010 #2711 of 10413
    cweb

    cweb New Member

    106
    0
    May 29, 2004
    kmttg works great with all my shows on all my Tivos, except for this season's Grey Anatomy. I have two episodes the kmttg refuses to download. The shows view as complete. I am able to transfer them to another Tivo with no problem. Kmttg won't even download the copies that successfully transferred to the other Tivo. I have reset and restarted the Tivo's with no success. The message is as follows:

    Download failed to file: G:\TivoTemp\Greys Anatomy\Greys Anatomy 2010-09-30 Shock to the System.TiVo
    Exit code: 52
    % Total % Received % Xferd Average Speed Time Time Time Current
    Dload Upload Total Spent Left Speed

    0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
    0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
    curl: (52) Empty reply from server
    Greys Anatomy 2010-09-30 Shock to the System.TiVo: Too many failed downloads, GIVING UP!!
     
  12. Oct 4, 2010 #2712 of 10413
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    See a few posts up in this thread: The cygwin version of curl has a problem dealing with URLs with special characters such as single quotes (which above show will have). Fix is to either revert to default version of curl installed with kmttg or switch to Java downloads.
    I'm going to hide the cygwin curl version download from kmttg downloads page since it should not be encouraged to use that version.
     
  13. Oct 4, 2010 #2713 of 10413
    txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    Can you send me a link to the Roku stream requirements? As far as I can tell, they should be able to support 720p or 1080i streams (depending on model). I don't see why you need to modify the resolution for SD streams at all.

    Can you try another encode where you modify the 3rd profile and remove the '-w 640 -l 368' flags? This should transcode the video without resolution change. Also, do you have any HD video streams (1080i/720p) that you can re-encode and try? For HD, I would again remove the '-w 640 -l 368' flags and change the '-q 19.0' to '-q 21.0'.

    I think we might simply be able to modify the profile(s) I made earlier for lew/richklein to simply encode with AAC audio rather than passing through AC3. I have attached two new profiles that are simply modifications of the earlier profiles with AAC audio instead. Can you test an encode with those to see if they work for you?
     

    Attached Files:

  14. Oct 4, 2010 #2714 of 10413
    txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    One more update to this profile as well as including both SD/HD version. I am dropping the framerate flag that forces the stream to 23.976fps like the first one had. This should allow FILM, PAL and INTERLACED material to be handled properly.
     

    Attached Files:

  15. Oct 4, 2010 #2715 of 10413
    shelleye

    shelleye New Member

    39
    0
    Nov 28, 2002
    Once again, I almost failed to see your updates. I'm looking forward to trying them out. Here's a link to the roku forum discussion: http://forums.roku.com/viewtopic.php?f=28&t=26058&p=189257&hilit=settings#p189257 (I hope I'm allowed to post that), and here is my cut and paste of the same post: "RockyFord wrote: I spent about 12 hours this past weekend trying to find the magic sauce for encoding videos in Handbrake. Here are my results: Premise: I have .mpg (full mpeg2 dvd rips) and .avi (mp4) with 5.1ac3 audio in most cases. The settings that work best for me and seem to reproduce the proper frame without wierd letterboxing or cropping:

    Container:MP4 File, Large File Size and Web Optimized checked.

    Picture:Anamorphic is set to None. Width is set to 640 (half of 1280), Height auto sets for aspect ratio (Using half width really seems to allow the codec to pack in the detail. The TV upscales the picture beautifully and my file sizes stay manageable. Cropping: Custom and I zero all values.

    Video Filters: All off except for Decomb Default

    Video: Video Codec: H.264(x264) Framerate:29.97 Avg. Bitrae:2500 or 3000 (Constant Quality doesn't seem to work well with roku because the bitrate may climb too high in spots)

    Audio: Use only one track of audio, I found that when I had two tracks of audio that were different the results became unpredictable. I use the AC3 passthru (DTS) for the files that have it already inside and the AAC works well for the files that don't have 5.1."

    To which somebody replies "You can actually drop the average bitrate to 1500 - 1600 without a loss of quality, this will make smaller files."

    I follow all settings, except that I don't do custom cropping, since automatic cropping seems to do the same thing....?

    EDIT: one note about frame rates: Roku seems to choke on anything but a framerate of 29.97, which I have confirmed by trying to use some old encodes. So that value probably can't be messed with. There seems to be some debate about AC3 vs. AAC audio (most say AC3, or two audio streams will confuse Roku, some think you can do AC3 passthrough), but since I don't have 5.1 surround sound I go with the sure choice.

    I will wait to hear from you before I mess with the profiles you sent me, in case you want me to try something different based on these more experienced opinions. My plan, if this works, is to change my workflow completely, and use KTTMG to batch process all of my DVD rips. I will also post my rave review over on the Roku forums. Tivo and Roku are, in my opinion, a match made in heaven. :)
     
  16. Oct 4, 2010 #2716 of 10413
    txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    Did you say that you used my 2nd and 3rd profiles already to encode some videos? Those are set to run de-telecine which will result in 23.976fps video (usually). If you got good results on the Roku with those, then it sounds like the framerate point may be incorrect. Please confirm. You can use MediaInfo or GSpot to confirm the framerate (among other things).

    Also, it looks like the Roku will also accept AC3 audio, which means that you can probably use the tivo_sd/hd_crf profiles without any need for modification (this assumes that de-telecine and frame size are not a problem).

    Finally, you can drop the -l 368 portion of the flags, but I think the aspect ratio will be off. I was getting a framesize of 640x432 without forcing the height to 368, which is why I did force it.
     
  17. Oct 4, 2010 #2717 of 10413
    shelleye

    shelleye New Member

    39
    0
    Nov 28, 2002
    I just went back and checked the encodes I did with your profile, and saw, to my shock, that half of them are showing up as 23.976 frame rates. So far I haven't had any difficulty playing them. But I know I (and others) had to re-encode multiple .m4v files that had a 23.976 frame rate. Most (but not all) of those files would freeze some time into streaming, and never recover. Once they were re-encoded at 29.97, they played fine.

    In your opinion, as a person with no Roku, but obvious knowledge of Handbrake, what would be the best profile/profile settings for me to start with? The only thing I care about is that the stream not choke (which seems to mean, most of the time, 29.97), and that I preserve the original aspect ratio of the film.

    (Can I ask you a question? Why would the Roku forum guy have set the width to 640 in the first place? He seemed to feel that it gave the best picture quality, but I see your point, that anything that starts out at 720 would do best if you just leave it alone. 640 seemed to be the magic number for him because it's half of 1280.)

    I apologize for dragging you into what turned out to be a much longer conversation than I meant for it to be. Hopefully I will be on my way soon.
     
  18. Oct 4, 2010 #2718 of 10413
    txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    The 640 number looks to have come from an effort to minimize file size. He said he was using half the width of 1280 (720p video). Based on this page, it looks like all of the new Rokus will play 720p video and 2 of 3 will play 1080p. Based on that, I think there is zero reason to worry about framesize adjustments. I would just encode without either -w or -l flags meaning input resolution = output resolution. Most of the time, 1080p is shorthand for 1080p24 which is 23.976 fps. I think Roku intends to support 23.976fps framerates, so you should be ok. BUT that isn't to say that you won't see problems at 23.976fps. TivoHD has some playback issues with 23.976fps material that are not present at 25+ fps, so it can happen.

    Personally, I would suggest using the hb_tivo_sd(hd)_crf profiles. That allows you to keep the surround sound AC3 audio if you can use it, and removes the telecine frames if they are present. The deeper I get into video editting/transcoding, the easier it is for me to see the subtle problems with video. I really hate the telecine judder. You will see this if you force FILM video to 29.97fps. Most everything that you watch nowadays in the US is FILM rate (TV shows, movies).

    What I would do is create a short clip (maybe 2-5 mins) and encode it various ways and test them out on your Roku. Ideally, I would like to find a profile that works on both Tivo (probably Premiere since it has best h.264 support) and Roku and use that.

    To start with, I would take a short clip and use the shelleye3 profile I created:
    1. shelleye3
    2. remove '-w 640 -l 368' = verify that standard SD resolution plays without a problem
    3. same as 2 + replace '--aencoder faac --ab 160 --mixdown dpl12 --arate 48 --drc 0.0' with '-E ac3' = verify that AC3 audio works
    4. same as 3 + remove '--detelecine' = compare 23.976fps video to 29.97fps video

    If you don't see a problem with test #3, then you will be fine using the hb_tivo_sd(hd)_crf profiles I posted above.
     
  19. Oct 4, 2010 #2719 of 10413
    shelleye

    shelleye New Member

    39
    0
    Nov 28, 2002
    A succinct and clear troubleshooting path. I will do as you say and post my results, here and in the Roku forum, in hopes of helping others who are moving into this brave new world. (I spent some time searching that forum and the internet, thinking somebody MUST have come up with a best practices encode, but nobody has, yet....)
     
  20. Oct 4, 2010 #2720 of 10413
    txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    I probably should have added a #5 and that is to try an encode using the hb_tivo_sd(hd)_crf profile. I forgot that there are actually some x264 setting differences between the Normal Profile in Handbrake and the High Profile which I use in hb_tivo_sd_crf profile. The ones that I created for you were based off of the Normal Profile.
     

Share This Page