1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New program for 1 step TTG downloads, decryption, encoding - kmttg

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Mar 15, 2008.

  1. May 8, 2008 #161 of 10405
    moyekj

    moyekj Well-Known Member

    11,128
    27
    Jan 23, 2006
    Mission...
    AFAIK you need a video editor such as VideoRedo or Womble to do that, so no there is no way kmttg can do that kind of thing for you.
     
  2. May 8, 2008 #162 of 10405
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    Awesome! Great job! I'm just starting to use it now, haven't tried out the service part yet, but will asap. I have another question, is there a way to do a two pass xvid encoding using ffmpeg and your script?
     
  3. May 8, 2008 #163 of 10405
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    also, is there a way to initiate the batch transfer while running the script normally?
     
  4. May 8, 2008 #164 of 10405
    moyekj

    moyekj Well-Known Member

    11,128
    27
    Jan 23, 2006
    Mission...
    Not directly using ffmpeg since right now there is a single system call for encoding but multi-pass would require additional ffmpeg calls. However, you could setup a batch script which takes input and output files as arguments then in turn calls ffmpeg (multiple times) and then use that batch script in place of the usual ffmpeg.exe binary as your encoder.
     
  5. May 8, 2008 #165 of 10405
    moyekj

    moyekj Well-Known Member

    11,128
    27
    Jan 23, 2006
    Mission...
    I don't understand the question. Can you elaborate?
    If your question is how to run auto transfers once from command line:
    perl kmttg.pl -b
    That will search for any matching entries, download and process the matches and then exit. If instead you use the -a switch in place of -b it will run in a loop instead of exiting (that is essentially what the kmttg service is).
     
  6. May 8, 2008 #166 of 10405
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    I'm wondering if there is a way to basically do the batch process (mainly starting to process just the files that match that haven't already been downloaded) from within the normal gui. this way i could start the "batch" and have the gui to see the progress.

    I'm trying out the batch mode now and it seems to be hiccuping on transferring from the tivo. I'm not sure yet, but I think it might be trying to initiate more than one transfer from the tivo (maybe one per entry in the maching list?) I'll check into it more as i have time.

    This rocks though! I think allowing more than one process to go at a time is going to help really increase the efficiency!

    Thanks!!!

    Ron
     
  7. May 8, 2008 #167 of 10405
    moyekj

    moyekj Well-Known Member

    11,128
    27
    Jan 23, 2006
    Mission...
    Running in batch mode and GUI mode are in no way connected so you can't start something in batch mode and monitor in GUI mode. For batch mode run you can look at "auto.log" file for a record of what it's doing. In either batch or GUI mode more than 1 download at a time per Tivo is not allowed. However, if you have multiple kmttg programs running at the same time (for example a batch mode run and a GUI mode run) that's one way you can bypass that restriction, which is not recommended.
     
  8. May 9, 2008 #168 of 10405
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    Hmm.. for some reason the batch script does seem to be trying to initiate two (or more) downloads from the tivo at the same time.. I attached my log file so you can see. right after creating all of the metadata files it initiates two separate transfers at the same time. Also, I think it might be good to limit the number of concurrent jobs that try to run at one time.. i tried to initiate a batch transfer of all the files on my tivo and after quite a few curl instances are started error messages pop up and the rest fail to run. If you'd like i can get a log of that too. I also attached my ini file in case you needed something from there..

    Thanks again, this is awesome!

    Ron
     

    Attached Files:

  9. May 9, 2008 #169 of 10405
    moyekj

    moyekj Well-Known Member

    11,128
    27
    Jan 23, 2006
    Mission...
    Thanks for the logs, that was helpful. Yes it looks like when jobs are queued up quickly such as in the batch case more than 1 download per Tivo is getting through and now I can see how by looking at the code. I will have to fix that. Also, I will need to restrict metadata generation to 1 Tivo at a time as well since they are grabbing data from Tivos as well.
     
  10. May 9, 2008 #170 of 10405
    moyekj

    moyekj Well-Known Member

    11,128
    27
    Jan 23, 2006
    Mission...
    v0.4b just released.
    ------------------
    v0.4b (05/09/2008)
    ------------------
    BUG FIX:
    * More than 1 download at a time per Tivo was being allowed through with
    previous version. This has been corrected.
     
  11. May 9, 2008 #171 of 10405
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    So far so good! I'm also trying out a batch file, as you suggested, to do a two pass xvid encode.. working on the first pass now.

    Thanks,
    Ron
     
  12. May 10, 2008 #172 of 10405
    kagi.anirik

    kagi.anirik New Member

    5
    0
    Apr 27, 2008
    Thanks a whole lot for going through all the effort to make this great utility set; it automates a lot of what I wanted to do.

    I gotta beg though... Can we set up a cleanup script that allows me to clean up the files after it has finished processing? And also, is there a way to have it delete the file on the Tivo after downloading?
     
  13. May 10, 2008 #173 of 10405
    moyekj

    moyekj Well-Known Member

    11,128
    27
    Jan 23, 2006
    Mission...
    This has been requested by several people. Option to remove .TiVo file after decrypt is already there. I will add option to clean up files of other steps as well.
    I wish, but there is no way to do that (at least no way anyone has discovered yet).
     
  14. May 10, 2008 #174 of 10405
    moyekj

    moyekj Well-Known Member

    11,128
    27
    Jan 23, 2006
    Mission...
    OK, new version released with more file cleanup options:
    ------------------
    v0.4c (05/10/2008)
    ------------------
    ENHANCEMENTS:
    * Added 2 options for file cleanup in the configuration:
    Remove .edl & .mpg files after comcut
    Remove .mpg file after encode
     
  15. May 10, 2008 #175 of 10405
    wombat94

    wombat94 Member

    62
    0
    Nov 18, 2007
    This tool works great.

    I've installed it on my Windows Home Server, and along with pyTivo, it now completes my home media server toolset.

    The service appears to intall and run fine on WHS (essentially windows server 2003) and all of the 3rd party tools run fine too.

    The file cleanup options are very nice, but one more request would truly make it a "be all and end all" for me.

    Can you add the option to have a user-defined application command line launched at the end of the other jobs for a given download?

    I would like to have a batch file (or powershell script eventually) that I can run after the other jobs for a download so I can move the program to the shares that pyTivo watches, or arbitrarily do other stuff. The extensibility would allow me to do other things eventually as well as I continue to integrate into home server.

    Thanks again for this tool. Tivo Desktop does not play nicely on WHS, and this now provides the critical missing piece - which was the autodownload and decrypt. The COMSKIP/COMCUT are gravy and I need to play with them determine whether they will be useful for me.

    Ted
     
  16. May 10, 2008 #176 of 10405
    moyekj

    moyekj Well-Known Member

    11,128
    27
    Jan 23, 2006
    Mission...
    Note that you could define the "encoder" executable as a script instead of a normal encoder executable to do whatever you wish. For "encoder_args" you can include INPUT keyword so that the .mpg file is passed to your script and then in the script you can do whatever you like. If you are encoding you can embed the encoding within your script and then do the post-processing. The encoder step is the last step kmttg calls which makes it well suited to the task of doing your own custom post-processing.
     
  17. May 11, 2008 #177 of 10405
    wombat94

    wombat94 Member

    62
    0
    Nov 18, 2007
    Aah... couldn't see the forest for the trees on that one.

    That will work perfectly. I don't use the encode job currently since I am just using kmttg to download off of the Tivo so that I free up the internal hard drive space on the Tivo for recording new stuff.

    Great tip and thanks again.

    I'm off to brush up my batch file skills...

    Ted
     
  18. May 11, 2008 #178 of 10405
    moyekj

    moyekj Well-Known Member

    11,128
    27
    Jan 23, 2006
    Mission...
    If you already tried it you probably noticed... You'll get an error by kmttg once your script completes since you are not actually encoding to a file and kmttg will check if file exists. However it shouldn't interfere with anything other than that error message.
     
  19. May 12, 2008 #179 of 10405
    ronaldlw

    ronaldlw New Member

    21
    0
    Apr 29, 2008
    I'm really loving this! Still haven't tried out as a service yet, still tweeking my 2 pass batch file, but i really love it running more than one process in parallel.

    Came across another gotcha though... If you select a bunch of mpg files or I would guess tivo files and initiate encoding it fires up processes for however many you select. This quickly brings my system to a screeching halt.

    I think it would be best to limit the maximum number of processes that can run at once.. either by type of job or just the total. It would be awesome if that limit could be part of the config file so that if someone has 8 cores they can choose to run 7 or 8 or whatever number, and if someone only has one or two they could set the number lower.

    Also, it would be great if when the kmttg script is closed it automatically terminated all of the threads it started up, otherwise i have to go through and close them in task manager. I know you can cancel them through the gui, but in the case when 53 instances of ffmpeg have been started the gui just doesn't work at all. Basically my only choice then is to log-off or reset the system.

    Also, i'd like to second the request for an optional final "clean up" batch file. Just another line in the config area that you can enter the path to a file that is called after the encoder process along with an args line like with the encoder, that way it can be called even if you don't do the encoding step. Also, if there was an additional variable that you could pass to the script that indicated what steps had been preformed on the file (create metadata, decrypt tivo, comskip, ecode) the called program or script could be more intelligent about it's cleanup. In other words we would be able to add say OPTIONS to the args for the cleanup script in the same way we can pass INPUT and OUTPUT.

    Make any sense?

    Oh ya, one other thing that I know would help me, if there was a way to automatically add all of the existing files (or files selected in the gui would be even better) on the tivo to the history file so that if they're already processed they won't have to be processed again in the batch/service mode. And along with that it would be helpful to have the title or file name or something along with the program id in the history file, so that we can look at the file and see what files are marked as processed already.

    Thanks again!
    Ron
     
  20. May 13, 2008 #180 of 10405
    moyekj

    moyekj Well-Known Member

    11,128
    27
    Jan 23, 2006
    Mission...
    New version just released:
    ------------------
    v0.4d (05/12/2008)
    ------------------
    ENHANCEMENTS:
    * Added "active job limit" setting to configuration which limits the number of CPU
    intensive jobs that can run at the same time to this number (defaults to 2).
     

Share This Page