1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New program for 1 step TTG downloads, decryption, encoding - kmttg

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Mar 15, 2008.

  1. May 13, 2008 #181 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    Version 0.4d just released has an option for that.
    In GUI mode you don't even have to go to job manager to kill them all - simply closing the application in GUI mode will terminate all active jobs since it calls an exit procedure to do just that. In Windows however if you kill the job from Task Manager there is no detectable signal sent to Perl that I can trap to call the exit procedure. In UNIX there is a way to trap kill signals, but not in windows from Task Manager from what I've found, so I don't see a way of doing this. The only way I can trap a signal in Windows it is if you run script from a command window and use Ctrl-C to stop execution.

    In version 0.4c I already added file cleanup options to the config so you can enable those now for automatic file cleanup.

    I'll look into it - probably the logic will be if the "auto.history" file already exists then add entries to it as jobs are processed from the GUI. I can add title names to the auto.history file, though I was trying to keep entries as short as possible since over time that file can grow to be pretty large.
     
  2. May 13, 2008 #182 of 10411
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    Excellent, thanks that is a huge help to me, already got my system cranking away on a bunch of mpgs!

    Awesome! Sorry, I didn't realize that if i closed it normally that it does close all of the threads, that's perfect! I should be able to hit the close dialog button and will just have to wait for it to actually do the close. Although that shouldn't be a problem with the new version.

    I was thinking it might make sense to add an option under the "auto transfers" menu item that "marks selected as processed" or something.. then you could just add the files selected in the gui to the history file.. And maybe make it optional to add the description to the history file.. good point about it getting pretty big..

    Thanks,
    Ron
     
  3. May 13, 2008 #183 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    I will release an update tonight addressing these - there is a new "auto transfers" menu entry for adding selections to the history file and the history file entries now include show title in addition to Program ID. Also, if running in GUI mode and the history file exists then processed entries are also now added to the history file so that files you process in GUI mode aren't processed again in batch/service mode.
     
  4. May 13, 2008 #184 of 10411
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    Ok, you are taking just way too long to implement this stuff.. :) just kidding, you rock!

    Thanks,
    Ron
     
  5. May 14, 2008 #185 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    New version released:
    ------------------
    v0.4e (05/14/2008)
    ------------------
    ENHANCEMENTS:
    * History file entries now include show title in addition to Program ID.
    * There is a new "Auto transfers->Add selected to history file" menu entry for
    adding selections from the GUI shows list to the history file.
    * If history file is created then successfully downloaded shows in GUI mode are now
    also added to the history file so they aren't processed again in batch/service mode.
     
  6. May 14, 2008 #186 of 10411
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    Ok, now i'm on to the next part.. figuring out the commercial cutting.. yikes! Comskip seems to do a pretty decent job of finding the commercials, but the comcut part seems to really mess up the video stream. I get some pretty bad effects on the cut. then when it encodes it really throws off the audio sync at those breaks. Am I doing something wrong or is that pretty typical with comskip and mencoder?

    Thanks,
    Ron
     
  7. May 14, 2008 #187 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    Yes, for a few programs I tried comcut=mencoder really messes up the sync, for others it's fine. You are generally better off using an mpeg editor such as VideoRedo or Womble to do the cuts. You can have comskip generate cut files for those programs and then import the cut files into them and let them handle the cutting which they do much more effectively. If I want precision commercial removal then I bypass comskip completely and just do it manually in Womble since I can do it very efficiently (takes me about 5 mins to cut out commercials in a 1 hour show).
     
  8. May 14, 2008 #188 of 10411
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    I wonder why memcoder would do worse than something like womble? I mean, setting aside if the places that the cut is made are really the start or end of a commercial or not, I would think the actual process of cutting out the parts marked as commercials and re-combining the mpg file should be pretty much the same with the two, right? Does womble use a different way to do the join? if so are there other utilities that would do it the same as womble but in a batch mode?

    There are some shows that I might want to go through the process manually, but for the most part i would accept just a rough cut of the commercials, but the audio and video really need to be in sync for it to be usable at all.

    Thanks!
    Ron
     
  9. May 14, 2008 #189 of 10411
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    Going to try to explain again what i was thinking with batch mode running from the gui. What I thought would be cool is if we could basically have the batch functionality from within the gui. basically you'd press a batch button or something and the kmttg would go out and figure out what to download, add it to the process list and you could see the process in the gui. Could even have it loop until you press another button (stop batch) or something. So instead of doing all the batch stuff in the background as a service or from a scrip it could happen in the gui.

    make any sense? Hard to do?

    thanks,
    Ron
     
  10. May 14, 2008 #190 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    Understood. On the surface it sounds like it would not be too hard to implement, but I would have to take a closer look to see if there are any complications.
     
  11. May 15, 2008 #191 of 10411
    leiff

    leiff Member

    513
    1
    Aug 24, 2005
    Great program. I'm still running 3.e version since I'm not interested in multiple requests or auto transfers. Tivo can only transfer 1 show at a time right? So newer versions won't transfer shows any faster right? Just more easy to automatically cue more shows?
     
  12. May 15, 2008 #192 of 10411
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    Actually the advantage of the multiple requests is that after the first show is downloaded and the encoding starts on that show the next show can start downloading while the first is encoded. In this way the second show will probably finish downloading before the first one finishes encoding, so the third show can be downloading while the second one encodes, etc...

    The way it is in 3.e the download has to finish, then it is encoded, once encoded the next one starts to transfer. so it will take a lot longer to download and encode more than one show.

    That said, I think the queue processing still needs some work. I don't think transfers always happen as soon as they could. I still have to spend some time to figure out exactly what happens, but sometimes just an encoding job is running even though there are other transfers in the queue.

    Ron
     
  13. May 16, 2008 #193 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    ------------------
    v0.4f (05/15/2008)
    ------------------
    ENHANCEMENTS:
    * A new menu entry is available: Auto Transfers->Run in GUI
    This is like running the Auto Transfers batch mode but inside the GUI instead of
    a background process. This will automatically match up your titles and keywords
    setup in Auto Transfers configuration and queue up all appropriate tasks to be
    run in the GUI such that you can monitor jobs just like normal in GUI mode.

    BUG FIX:
    * Previously copy protected and still recording shows were not being skipped by kmttg
    in Auto Transfers mode leading to failures when attempting to download those shows.
    This has now been fixed.
     
  14. May 16, 2008 #194 of 10411
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    once again, you rock!

    Ron
     
  15. May 16, 2008 #195 of 10411
    Prince Nez

    Prince Nez New Member

    1
    0
    Dec 7, 2007
    moyekj,

    First off, this is a great little program. I just found this the other day and my dream Tivo setup is now almost complete.

    Here's what I'm trying to accomplish and it would involve a few minor tweaks of your program.

    I have auto transfers setup in kmttg for my most watched shows. It grabs them cuts out the commercials and encodes them. Then I'll have Tivo Desktop auto-transfer them back to the Tivo sans commercials.

    Things that would help with this...

    1. Can there be an option for a separate conversion directory from the output directory? This way the completed files could be moved at the end of the process to the directory that Tivo Desktop is setup to auto-transfer from. That way it won't try to transfer any of the files that are created during the conversion process.

    2. It would need to have an option to put the completed files into separate directories. Preferably the name of the show. This way Tivo Desktop will put the episode back into the same Tivo folder that it grabbed the original from.

    3. The metadata file will need to be renamed when the conversion is finished.

    4. mencoder is horrible. :-( Comskip does a decent job though. You mentioned a couple of commercial programs other then mencoder. Do any of these integrate seamlessly with your program?

    Hopefully you understand where I'm going with this.

    Thanks for any help.
     
  16. May 17, 2008 #196 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    There are many different things people may want to do as a post-processing step so I don't know that it's a good idea to try and specifically code in every one of them. I have been suggesting that people use a custom batch script (e.g a .bat file) in place of the normal "encoder" executable to do what they want. That combined with receiving the file name as an argument makes it flexible to do whatever you want with the file, including the encoding step embedded withing your batch file if you are using encoding.

    I need to change kmttg to be smarter about naming the metadata file based on what actions were selected such that re-naming the metadata file shouldn't be necessary. That will be in next update.

    Yes as it is right now with mencoder cuts making audio/video sync problems for many shows it's pretty useless, so I should explore another automated program for performing the cuts. Currently I do the video editing phase manually using Womble sometimes using comskip generated cut file as a starting point. It only takes me about 5 mins for a 1 hour show. It would still be nice to have a fully automated method for cases when I don't care so much about precision commercial removal. Supposedly VideoRedo has a batch mode where one can supply a cut file and have it do the cuts, but I don't own VideoRedo (and don't really need it since I have Womble instead). There are probably other batch mode mpeg2 editors out there that may fit the bill, probably some Googling and experimentation are in order. If you happen to run into one that looks promising let me know...

    One thing I'm curious about out in your methodology is why do you re-encode the files to a non-mpeg2 file and then send them back to your Tivos? Note that Tivo Desktop then has to convert them back to mpeg2 so you can play them back on your Tivos. It would be better quality and faster to just feed back to the Tivos as mpeg2 files.
     
  17. May 17, 2008 #197 of 10411
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    This is a pretty powerful option. Currently I'm doing this so that I can run a two pass encoding on the input file. I also make a copy of the metadata file with the .avi.txt extension and can delete any intermediate files I want from here.

    It would be great to have access to more variables though, for example the "encoded ext" from the config file could be used when creating the metadata file copy to automatically use whatever extension the file is encoded to.

    Also, it would be awesome if a destination directory could be user defined for each auto transfer entry and then be either passed to the encoder batch file or have kmttg automatically place the file along with it's metadata text file in that destination as Prince Nez requested. This way you could put several types of files in a directory or have nested directories for your destination files without having to move them manually.

    Yes, this would be excellent, although it would be nice to have the option to create a metadata file for any or all of the files output. *mpg.txt, avi.txt and maybe even the cut file.. Right now I'm keeping both the encoded avi and the mpg files since I haven't had much luck with the comskip part yet, I want to have the original mpg to process when I finally get that working. So I am putting the AVIs and MPGs in different places and keep the corresponding metadata file along with them.

    Yes Yes Yes, something here is a must for me. I'm going to try to spend some time this weekend and early next week looking for a solution as well. There has to be a utility out there that can do the cutting and recombining well. I'll let you know if I find anything too.

    I'm also doing this (reencoding and then sending back to the tivo) except that i'm not using desktop at all, i'm using pyTivo. The main reason I want to reencode is to save space. It's amazing, as I'm sure you know, how fast even a 750 gb or larger drive can fill up with 1 gb+ mpg files from the tivo. So I reencode to xvid and can get 4 or 5 or more times as much stuff on the drive without really loosing too much in quality.

    Actually, once I get everything right I think I'll end up with better quality than I had before. This is because before I set the quality of the recording pretty low so that I wouldn't fill up the tivo hard drive too quickly. Before I would just download things I wanted to keep using desktop and archive them. But it was so unstable that I really didn't use it much to put video back to the tivo. Now with kmttg and pytivo i'm pulling everything off of the tivo and re-encoding and archiving it to a big drive. Since everything gets pulled off I set the recording quality to a much higher value to get a better source for the encoding. At this point the final avi file I end up with is basically the same resolution as what I was recording at before, but the files are about half the size. Once I get the commercial cutting working the files can be even smaller, or I can actually increase the quality a bit and still have about the same size files as I'm shooting for.

    Thanks again for this great tool!!

    Ron
     
  18. May 22, 2008 #198 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    Minor update (been very busy lately)...

    ------------------
    v0.4g (05/22/2008)
    ------------------
    ENHANCEMENT:
    * Smarter naming of metadata file to match final video file name. Previously
    the metadata file was hardcoded to be .mpg.txt.
     
  19. Jun 5, 2008 #199 of 10411
    Prestond

    Prestond New Member

    44
    0
    Feb 4, 2008
    Hi Kevin,

    I want to upgrade to latest version, whats best approach for doing so? I am on v0.3e

    Also wanted to bump this cause its such a awesome program!

    Best Regards,
    Preston
     
  20. Jun 5, 2008 #200 of 10411
    Prestond

    Prestond New Member

    44
    0
    Feb 4, 2008
    Never mind ...I see on your website the instructions:

    "NOTE: If you already an older version installed, you can simply replace your old kmttg.pl file with the new one in this zip file"

    Still an awesome program and deserving of bump...

    :)
     

Share This Page