1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New program for 1 step TTG downloads, decryption, encoding - kmttg

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Mar 15, 2008.

  1. Sep 11, 2013 #5721 of 10413
    az1097

    az1097 New Member

    82
    0
    Apr 3, 2013
    On two different occasions, after transferring shows from my PC to a Tivo HD using PyTivo, any show in the Tivo that was not marked "until I delete" was wiped out. Does anyone know of this issue, or have any suggestions? Thanks.
     
  2. Sep 11, 2013 #5722 of 10413
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    pyTivo thread is the place to pose that kind of question. But it could be related to a rogue estimate of the final file size of a transcoding transfer, which makes the TiVo think it doesn't have room to accommodate the transfer and hence deletes anything it can to make room.
     
  3. Sep 11, 2013 #5723 of 10413
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    If you are launching a whole bunch of jobs at once for which you want to manually review commercial cut points you are probably better off launching the jobs only up to the "Ad Detect" phase, so you end up with .Vprj files. Then when ready you can fire up VideoRedo GUI and open the Vprj files, review/modify them and then save them. Once you have 1 or more ready to go, from kmttg FILES tab you can add the respective .mpg files for each one and then choose "Ad Cut" task plus any other subsequent tasks for kmttg to run since all tasks from that point on are batch. (kmttg will know to look for the respective .Vprj files for each .mpg file as a starting point of the "Ad Cut" task).

    In that manner all the batch portions of the kmttg tasks are not interrupted waiting for you to edit Vprj files.

    EDIT: In reviewing kmttg code I see that VRD GUI jobs are not considered "active jobs" so should not count against "active job limit" setting. I will need to update the tooltip to reflect that. So if you are running recent version of kmttg the vrdreview tasks don't count against "active job limit" so should not prevent other active jobs from running.
     
  4. Sep 12, 2013 #5724 of 10413
    Soapm

    Soapm Active Member

    1,564
    0
    May 9, 2007
    So close,...
    That doesn't appear to be what's happening, I currently have 4 encodes going (which are really straining my machine) and all the adcuts, QSFix etc... for the remaining files are sitting queued.

    [​IMG]

    I would like to limit the simultaneous encodes to 2 but let the other parts of the queue continue. I have about 150 files from a program I am trying to store but doing them one at a time like your suggestion would be daunting.

    If I could get kmttg to limit the encodes to 2 but still process the adcuts, qsfix etc... I could edit 10 or so, come back when I have more time and do another 20 or so then later do more and they'll be encoding while I'm going about my normal life.

    Limiting the econdes to 2 would also allow me to use the computer, I'm so bogged down with 4 that I can't really use that computer.
     
  5. Sep 12, 2013 #5725 of 10413
    wmcbrine

    wmcbrine Ziphead

    10,368
    22
    Aug 2, 2003
    The fact that you mention KUID makes me suspect that you have a lot of recordings flagged that way. Yes? Ironically, that's exactly the condition that seems to lead to excessive deletion of the unmarked programs when a new recording is made, or transferred.

    It's true that pyTivo (usually) overestimates the size of what it will send -- intentionally; because the alternative, underestimation, leads to the newly transferred recording being deleted immediately. But it's normally (depending on the source material) not by a large factor. The kind of dramatic result you describe is something I've only heard of in connection with systems that have a large percentage of KUID recordings.

    And yes, this should've been in the pyTivo thread.
     
  6. Sep 12, 2013 #5726 of 10413
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    Soapm, the "qsfix" and "adscan" task are considered active jobs since they are CPU intensive, and they come before the "vrdreview" task, so what you show there is expected behavior - there's 4 active encode jobs running already so a "qsfix" or "adscan" active job won't launch until one of the encode job finishes. The "vrdreview jobs aren't active, but they are waiting for the respective qsfix and adscan jobs to run before they can launch so everything is behaving as would be expected.

    Please read my suggestion again. I didn't say to do 1 at a time. I proposed you run in 2 stages, 1st stage ending at "Ad Detect" - you can queue up as many as you like and fire them all off and create all the .Vprj files you can handle. Then you can start reviewing those with VRD and as they are ready you can kick off kmttg 2nd stage starting with "Ad Cut". If you have 10 .Vprj files ready to go you can kick off all 10 at same time and let kmttg process them in batch mode from there on. While those are running you can use VRD to process more .Vprj files (perhaps limiting active jobs in kmttg to 1 or 2 so system is not overloaded while you are reviewing other .Vprj files).

    NOTE: With this 2 stage approach you would turn off the "Use VideoRedo GUI to review detected commercials" setting.
     
  7. Sep 12, 2013 #5727 of 10413
    Soapm

    Soapm Active Member

    1,564
    0
    May 9, 2007
    So close,...
    I understood about them using the processor which was why I was hoping code could be easily added (or was there) to limit encoding to "X" cores which frees the other cores (or active jobs) to do other processing. Qsfix and adscan would only run when I am actively working those tasks since they'd eventually pause at the VRD stage.

    Allowing all active jobs to encode seems like a bottleneck in the big picture.

    Ok, I will sleep on it so it can digest, I'm probably mentally seeing this as daunting so can only see the room for errors and not the fact that it can easily be done. I'm sure I'm not the first to record every episode of a show and now want to move them off the Tivo. i was just hoping to highlight them all and hit go and not have to keep track of which were done and which are at what stage of processing. Organization isn't my strong suit.
     
  8. Sep 12, 2013 #5728 of 10413
    reneg

    reneg Member

    699
    0
    Jun 19, 2002
    If I understand this right. In the kmttg configuration, under program options, you can configure the "active job limit" and "encoding cpu cores". What this means is that each active job can consume x number of cpu cores for encoding.

    For example, my PC has 12 cpu cores, I have active job limit set to 2, and encoding cpu cores set to 5. This means that when there are 2 encoding jobs running, kmttg is using 2x5 or 10 of my 12 cpu cores. This leaves me with 2 cpu cores to do other things while kmttg is running.
     
  9. Sep 12, 2013 #5729 of 10413
    Soapm

    Soapm Active Member

    1,564
    0
    May 9, 2007
    So close,...
    Does yours work that way? Mines won't process "other things" because the active job limit is maxed and all the allowed cores are dedicated to encoding.

    I've looked at the two step process with a fresh mind and think it's workable. I'll just have to set up folders to keep track of the steps and will probably take the time to do all step 1... Then come back later, reset the settings and do step 2. That's the beauty of a 2TB drive in the Tivo, you don't have to rush to remove the shows...
     
  10. Sep 12, 2013 #5730 of 10413
    ALbino

    ALbino Lurker Series 2

    109
    0
    Jun 24, 2002
    I've only recently discovered using the Remote section of KMTTG, and it's amazing. I can't believe I never played with it before. For example: being able to go through my To Do list and quickly add buffer to the end of recordings is significantly easier than doing it on the TiVo itself. And really, that's an understatement. What takes 3 minutes in KMTTG would take half an hour on the actual box.

    KMTTG Wishlist #1: a way to select multiple recordings and set options to all of them at once (i.e. add 5 minutes to every recording).

    KMTTG Wishlist #2: an "undo" button for when we accidentally right-click and choose Cancel instead of Modify.... whoops :)
     
  11. Sep 12, 2013 #5731 of 10413
    reneg

    reneg Member

    699
    0
    Jun 19, 2002
    Mine processes two jobs at once, so when it is encoding two files, it does not make progress on the other jobs. Encoding is the most time consuming of the steps.

    I use a two stage process for the files I transfer off the Tivo. The first stage I set up as auto transfers which download, metadata, qsfix, and ad detect shows. This leaves me with .mpg, .txt, & .vprj files for each show. When I get around to stage 2 which is usually the next day. I open each .vprj file and review the detected cuts making tweaks as necessary. Then I select the .mpg file in kmttg and start stage 2 process, which does adcut, captions, encode and runs a custom script.
     
  12. Sep 13, 2013 #5732 of 10413
    keenanSR

    keenanSR Member TCF Club

    915
    2
    Oct 12, 2004
    Santa Rosa CA
    I just updated to Java Version 7 Update 40 and the Remote functions no longer work, I get an error that says,

    RemoteInt - (IP=middlemind.tivo.com, port=443): java security.cert.CertificateException: Certificate does not conform to algorithm constraints

    I also tried on previous version kmttg v0p9l and same problem. The rest of the functions seem to work fine.
     
  13. Sep 14, 2013 #5733 of 10413
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    Yep, another user filed a ticket on this problem yesterday. Luckily I managed to find a workaround to the problem via code that is part of next release. In the meantime if you really want to use latest version of Java the workaround is:
    Edit java.security file that is part of the installation, for example on Windows:
    c:\Program Files (x86)\Java\jre7\lib\security\java.security

    Then look for this line:
    jdk.certpath.disabledAlgorithms=MD2, RSA keySize < 1024

    comment it out like this:
    #jdk.certpath.disabledAlgorithms=MD2, RSA keySize < 1024

    Now re-start kmttg and Remote functions should work again.

    (With next kmttg release this workaround won't be necessary as kmttg does the equivalent to the above in code).
     
  14. Sep 14, 2013 #5734 of 10413
    keenanSR

    keenanSR Member TCF Club

    915
    2
    Oct 12, 2004
    Santa Rosa CA
    I'll wait for the next release, I just wanted to be sure it wasn't something wrong at my end, thanks! :)
     
  15. Sep 14, 2013 #5735 of 10413
    R_Lawson

    R_Lawson New Member

    1
    0
    Nov 15, 2010
    comment it out like this:
    #jdk.certpath.disabledAlgorithms=MD2, RSA keySize < 1024


    W7Pro 64bit with Java 7 U40, KMTTG v1p0e and win32 tools v0p9s.
    I didn't have any luck with commenting out the certificate line even with multiple clean installs and reboots. KMTTG appears to work correctly except for the remote functions.
     
  16. Sep 14, 2013 #5736 of 10413
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    Try setting it to empty value instead of commenting out. i.e.:
    jdk.certpath.disabledAlgorithms=
    (Note that you need to re-start kmttg after changing java.security file if you are not doing so).
    Also, you need to edit and save file with Administrator permissions since it's under protected area of Windows.
     
  17. Sep 14, 2013 #5737 of 10413
    keenanSR

    keenanSR Member TCF Club

    915
    2
    Oct 12, 2004
    Santa Rosa CA
    I decided to go ahead and do the edit. Restarted kmttg and it works great, thanks again!

    Reading the file was kind of a pain as the text just wraps around continuously but I did a Find with the whole string and it worked like a charm.
     
  18. Sep 16, 2013 #5738 of 10413
    cherry ghost

    cherry ghost Active Member

    5,281
    6
    Sep 13, 2005
    Chicago
    Is there anyway to automatically change the originalAirDate time in the metadata? Say from T00:00:00Z to T06:00:00Z?
     
  19. Sep 16, 2013 #5739 of 10413
    lpwcomp

    lpwcomp Active Member

    8,081
    2
    May 6, 2002
    John's...
    To what end?
     
  20. Sep 16, 2013 #5740 of 10413
    cherry ghost

    cherry ghost Active Member

    5,281
    6
    Sep 13, 2005
    Chicago
    Showing the correct date in iTunes.
     

Share This Page