1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New program for 1 step TTG downloads, decryption, encoding - kmttg

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Mar 15, 2008.

  1. Apr 28, 2014 #6701 of 10413
    wireman121

    wireman121 Member

    126
    0
    Nov 8, 2008
    OK moyekj,

    I have done a bunch of testing and this is what I've come up with:

    With Auto Transfers enabled, if the check TiVos interval is less than the amount of time it takes to complete a download (and add the title to the auto.history) then it will re-add the metadata task.

    The solution for this should be after Auto Transfer checks to see if there are any titles on the auto transfer list on a NPL, it should check to see if it is already processing a title (which can be done by checking the status of the icon next to the title, it if has a hourglass icon, then it can be considered already processing) and if it is already processing, then it should not re-add the already completed metadata task (or any task related to the currently processing transfer.) If the download fails, then it will fail and clear the rest of the tasks and not add the title to the auto.history file. So then auto transfer will re-add the tasks again at its next interval, which should be expected.

    Somewhere I think this small check was overlooked, but I'm guessing (and I don't claim to know how your code works) it will be a easy if/else added into the auto transfer checking code.

    Thanks!
     
  2. Apr 28, 2014 #6702 of 10413
    rickyTV

    rickyTV New Member

    14
    0
    Mar 16, 2014
    Is it just me or has anyone else seen transfer speed drop by 1/3, down from mid 80 Mbps, down to mid 50's. This is in just the last few days.

    I've got a roamio 4 channel and it's running 20.4.1. I rebooted everything and no change. Even rebooted my cisco switch the two are on.
     
  3. Apr 28, 2014 #6703 of 10413
    moyekj

    moyekj Well-Known Member

    11,148
    33
    Jan 23, 2006
    Mission...
    OK thanks for the concise summary - gives me something to look at. Note that checking for duplicate jobs is an integral part of the code. For example try clicking on a title and press START button more than once and you'll see most tasks will not be duplicate queued. metadata task is an exception because there can be multiple metadata files created from the same source file (depending on your "metadata files" setting in config-Program Options), so that one is a little trickier to deal with.
     
  4. Apr 29, 2014 #6704 of 10413
    moyekj

    moyekj Well-Known Member

    11,148
    33
    Jan 23, 2006
    Mission...
    wireman21, I still can't reproduce the issue. As I posted above, during Auto Transfers, once processing starts for a particular TiVo that TiVo will not be processed again for Auto Transfers until all jobs for that TiVo are complete, so the interval you set is really irrelevant to the process if only talking about that one TiVo. So only way I can imagine there being an issue is with duplicate shows on different TiVos. i.e. I think there's more to it I don't know about yet in your setup.
     
  5. Apr 29, 2014 #6705 of 10413
    wireman121

    wireman121 Member

    126
    0
    Nov 8, 2008
    2 Tivos in my setup. both Premieres.
    Auto Transfers is set to 5 mins.
    For example, tonight 3 shows recorded 9-10 pm (The Tomorrow People on 1 tivo, The Following and Warehouse 13 on another) at 10:05 Auto transfer queued all 3 of them. Once the metadata downloads, the task is deleted from the queue. at 10:10, when auto transfer runs again, it re-adds it to the queue.

    Only reason I noticed this is because for some reason, its messing up the rest of the task list and causing the non-download tasks not to run until all 3 downloads are completed (one at a time)

    As I mentioned, I think auto transfers loop should continue to run even if some shows are being processed, so as additional shows are done recording, (say for example if I have a 10-10:30 recording) they can be added and processed by kmttg without waiting for the first added show to be completely finished before auto transfer loop runs again. I had requested this a while back (maybe over a year) and you had changed it to work this way....so if I am away from my house for days, when I come home all of the shows will be at the VideoReDo Review task, and not just the first show that finished recording..... I feel that it makes the automation process much smoother. Otherwise when I get home, there is only 1 show in VideoReDo review and once I close that, then it would add all the others, which defeats the purpose of having auto transfer in the first place...

    You should be able to just add something into the auto transfer loop to confirm the show its trying to queue up isn't already being processed somehow, to remove the re-adding of the metadata task, which should fix the issue of tasks not being processed correctly...

    God I hope that all makes sense.

    Not sure why you're not seeing it as an issue on your end, What else can I do to help?
     
  6. Apr 29, 2014 #6706 of 10413
    moyekj

    moyekj Well-Known Member

    11,148
    33
    Jan 23, 2006
    Mission...
    Indeed it does. However, there's a problem: If "Use RPC to get NPL when possible" option is enabled, there is no byte offset available in that data, only a time offset. Resume needs a byte offset to work. So it's not simply a question of re-enabling resume for series 4 units - I've got to code in obtaining byte offset if missing as well.
     
  7. Apr 29, 2014 #6707 of 10413
    moyekj

    moyekj Well-Known Member

    11,148
    33
    Jan 23, 2006
    Mission...
    I see there's an option under Auto Transfers setup called "Do not wait for all jobs to finish before processing new ones". That option is off by default. Do you have that turned on? If so then that probably explains it. I vaguely remember adding that option but never use it myself.

    EDIT: Yes I can reproduce your issue now with that turned on. I was never fond of adding that option. Guess I'll have to try to account for it now that I can reproduce the problem.
     
  8. Apr 29, 2014 #6708 of 10413
    wireman121

    wireman121 Member

    126
    0
    Nov 8, 2008
    Yes! that was added per (I think) my request. Now I remember how it was implemented! haha.

    So - Is there a variable in kmttg that the icon (check mark, nothing, hourglass)? I think if you add this in, it will fix everything-

    right before the auto transfer loop adds the tasks to the queue, have an if statement check to see if that show already has an hourglass variable (I know that's not what its called, just trying to get my explanation across) and if it does, then abort adding any tasks related to that show. If there is no hourglass variable, then it is a new show, and process it as normal/add the tasks to the queue.
     
  9. Apr 29, 2014 #6709 of 10413
    lpwcomp

    lpwcomp Active Member

    8,081
    2
    May 6, 2002
    John's...
    Actually, it happens even w/o that option checked. I reproduced the problem with duplicate metadata downloads when the check interval is < than the download time days ago, but I thought he was seeing duplicate downloads, which is why my post indicated that the timing had to be "just right (or wrong".

    Wouldn't having that option on simply prevent it from running immediately? It would still be queued.
     
  10. Apr 29, 2014 #6710 of 10413
    moyekj

    moyekj Well-Known Member

    11,148
    33
    Jan 23, 2006
    Mission...
    When that option is turned off (default setting), kmttg won't process auto transfers again until for that TiVo until all current jobs for that TiVo have ended, so it's never an issue. So with that option off you would have to have duplicate shows on different TiVos for that to happen.

    In any case, I came up with simple fix: If the metadata file already exists then don't queue up a job for it. It doesn't directly relate to auto transfers setup, but makes sense regardless of context and effectively solves this problem.
     
  11. Apr 29, 2014 #6711 of 10413
    moyekj

    moyekj Well-Known Member

    11,148
    33
    Jan 23, 2006
    Mission...
    Strange, resume downloads seemed to be working last night, but when I try again on same show that worked last night it no longer works. So I think the TiVo http server has flaky support for byte offset and it's not reliable enough to enable.
     
  12. Apr 29, 2014 #6712 of 10413
    moyekj

    moyekj Well-Known Member

    11,148
    33
    Jan 23, 2006
    Mission...
    beta kmttg.jar with above fix is available in this zip file if you guys want to test.
     
  13. Apr 29, 2014 #6713 of 10413
    wireman121

    wireman121 Member

    126
    0
    Nov 8, 2008
    Thanks. Will test tonight.
     
  14. Apr 29, 2014 #6714 of 10413
    ThAbtO

    ThAbtO TiVoholic by the bay

    6,736
    8
    Apr 6, 2000
    SF Bay Area
    It still worked for me today, only Java downloads, and no RPC.
     
  15. Apr 29, 2014 #6715 of 10413
    wireman121

    wireman121 Member

    126
    0
    Nov 8, 2008
    Did a quick trial with this and it seems to have fixed both problems I was having. I will try again tonight, there's another 3 shows downloading at the same time...
     
  16. Apr 29, 2014 #6716 of 10413
    moyekj

    moyekj Well-Known Member

    11,148
    33
    Jan 23, 2006
    Mission...
    Tried it again a short while ago. The 1st three times I tried it worked. Each time I cancelled the download as I don't want to wait for it to finish. I was going through all the different download combinations to test if they all work. Starting around the 4th try all of a sudden curl method no longer works. Curl gives:
    "curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume."
    However, the java method does continue to work... Strange.
     
  17. Apr 29, 2014 #6717 of 10413
    ThAbtO

    ThAbtO TiVoholic by the bay

    6,736
    8
    Apr 6, 2000
    SF Bay Area
    Code:
    >> RESUMING DOWNLOAD WITH OFFSET=3832938496 i:\video\KPIX 5 News at Noon -  - (RAW) -  2014.04.29-12.00.TiVo ...
    E:\kmttg\curl\curl.exe --retry 3 --anyauth --globoff --user tivo:MAK --insecure --cookie-jar C:\DOCUME~1\Thabto\LOCALS~1\Temp\cookie784816381881244948.tmp --url http://192.168.1.120:80/download/KPIX%205%20News%20at%20Noon.TiVo?Container=%2FNowPlaying&id=16223 -C 3832938496 --output "i:\video\KPIX 5 News at Noon -  - (RAW) -  2014.04.29-12.00.TiVo" 
    i:\video\KPIX 5 News at Noon -  - (RAW) -  2014.04.29-12.00.TiVo: size=218.57 MB elapsed=0:01:58 (15.54 Mbps)
    ---DONE--- job=download output=i:\video\KPIX 5 News at Noon -  - (RAW) -  2014.04.29-12.00.TiVo
    
    Even curl worked for me just now (v0p9i).
     
  18. Apr 29, 2014 #6718 of 10413
    moyekj

    moyekj Well-Known Member

    11,148
    33
    Jan 23, 2006
    Mission...
    Try it a few times in a row, cancelling the download each time and then re-starting to see if it fails eventually as well.
     
  19. Apr 29, 2014 #6719 of 10413
    bobdawsonvenice

    bobdawsonvenice New Member

    5
    0
    Dec 21, 2009
    This is my 1st time using kmttg and my 1st post - please be gentle.

    I have a new Roamia and it often fails (sometimes restarts) when I try to publish (upload) a .wmv file that I add to an auto-transfer folder on my PC. I use the latest TiVo Desktop Plus (2.8.3).

    I have read that I should use kmttg to first convert the .wmv file to .tivo format. I installed/opened kmttg but find it a little daunting.

    In the FILES section I added a wmv file to be converted. I have opened and added several different encode files that have "tivo" in their filename, but none seem to result in output that has a .tivo format. Can someone tell me which Encoding Profile is for creating a .tivo file from a .wmv input?

    Thanks.
     
  20. Apr 29, 2014 #6720 of 10413
    ThAbtO

    ThAbtO TiVoholic by the bay

    6,736
    8
    Apr 6, 2000
    SF Bay Area
    You would not need to convert to .TiVo file, .wmv is an accepted format along with .mpg, .mp4.
    A .TiVo file is an encrypted file with the Media Access Key.
    You may be having some unforeseen problems in the video file.
     

Share This Page