1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New program for 1 step TTG downloads, decryption, encoding - kmttg

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Mar 15, 2008.

  1. Aug 4, 2010 #2501 of 10413
    moyekj

    moyekj Well-Known Member

    11,151
    34
    Jan 23, 2006
    Mission...
    Make sure service is setup to use same account you run the GUI in. See section on windows service in:
    http://code.google.com/p/kmttg/wiki/auto_transfers
     
  2. Aug 4, 2010 #2502 of 10413
    AudioNutz

    AudioNutz New Member

    318
    0
    Nov 9, 2008
    Is this on Windows or Mac?
     
  3. Aug 4, 2010 #2503 of 10413
    Tiresius

    Tiresius New Member

    37
    0
    Mar 22, 2004
    Mesa, AZ
    Windows XP, Athlon 64 4200+ dual-core. Yeah, I know pretty old, but it's the difference between ffmpeg and HB that's shocking.

    In repeated recodes, it's actually 90 minutes for ffmpeg vs. the hour I said before. But it is 4 1/2 hours for HB.

    FYI, running videoredo stream fix before encoding with ffmpeg did make the sync issue go away. Before running videoredo, I did test -async 100 and 4800 with ffmpeg, no help.
    Odd that the issue didn't occur when using HB, Tivo Desktop +, or Pavtube Converter and did not occur when playing the two original .mpg files that I was testing with.

    For the price of VideoRedo, I could buy Roxio 2010, which would take advantage of the 32 CUDA cores in my Nvidia Graphics card, potentially speeding up my conversion over what my 5 year old CPU can do. I wish they had a trial version that I can test it with.
     
  4. Aug 4, 2010 #2504 of 10413
    AudioNutz

    AudioNutz New Member

    318
    0
    Nov 9, 2008
    Well... If it were a Mac, I would have told you about a bug that I found in handbrake, where speed takes a 4x hit.

    I'm having trouble recalling the exact reason, but it seems that if you give it a set of resolutions, it was doing something funny with the aspect ratio. This happened at 1x the speed of ffmpeg. But... If you coerced the aspect ratio back to be correct, the encoding process needed to scale each frame of the video as it processed, and this took 4x the amount of time.

    Open source developers are typically very difficult to convince that their stuff has a bug... so I gave up, and I'm still using ffmpeg. It's certainly not perfect, but it's what i got...

    Maybe the same 'bug' exists on the Windows side! Try taking out the aspect ratio and resolution settings, and encode a show again. If it speeds back up... Bingo, same bug.

    ....er.... Ummm... "Un-Documented Feature"
     
  5. Aug 4, 2010 #2505 of 10413
    moyekj

    moyekj Well-Known Member

    11,151
    34
    Jan 23, 2006
    Mission...
    Yes, VRD fixes broken timestamps in the original mpeg files. ffmpeg is not good at dealing with broken timestamps. For HB if you double-click on the job in kmttg while it's running you will actually see some information spit out about timestamp correction that HB is doing, and it does so by modifying the audio stream. That is supposed to be what ffmpeg -async does as well but obviously that is broken in ffmpeg...

    In any case, VRD is useful to have to precisely edit out commercials and note that you only really need the $50 VideoRedo Plus for QSFix, you don't need the $100 TVSuite version, so well worth it IMO.

    P.S. At some point I need to get around modifying HB encodes to actually show % progress in kmttg - a relatively simple thing to do I never got around to (mostly because I never use HB).
     
  6. Aug 5, 2010 #2506 of 10413
    ferror

    ferror New Member

    66
    0
    May 3, 2005
    Austin, TX
    That is interesting that it also shows up in XP. I guess it isn't just Windows 7, but I do still believe it may have been a Windows update that started this. I have had no luck solving the issue, but it does appears to be a curl problem. I can also replicate the same behavior (stalled transfers) when running curl with the same parameters (copy/pasted from the kmttg log) directly at the command line outside of kmttg. I noticed there was a newer version of curl so I tried that, but it has no effect on the issue.

    I have found one crude workaround that seems to help a small amount. If I add the parameters: "-y 60 -Y 100" to the curl command line, then curl will at least detect that the transfer has stalled and restart the download. Unfortunately, resuming a download (-C) does not seem to work with the TiVo. I guess the http service on the TiVo doesn't support resume. That said, this does at least seem to be able to kick the transfer and allow it to use the set retry count to get the whole file. It isn't 100%, but helps considerably. It can result in total download times of 3 or 4 times normal, however, because it gets on average 50% into the file and then stalls out. At this point, anything is better than literally having to babysit kmttg 24 hours a day to keep curl going.

    To get kmttg using the above command line parameters, I did have to modify my local copy of kmttg by adding the following lines to src/com/tivo/kmttg/task/download.java beginning at line 99:

    command.add("-y");
    command.add("60");
    command.add("-Y");
    command.add("100");

    Maybe it is time to take this issue over to the curl bug tracker and find out what they think.
     
  7. Aug 5, 2010 #2507 of 10413
    Network_Guy

    Network_Guy New Member

    5
    0
    Apr 25, 2010
    I change the properties so that kmttg would use the Administrator account on WHS and VideoReDo ran perfectly.

    Thanks,
     
  8. Aug 5, 2010 #2508 of 10413
    moyekj

    moyekj Well-Known Member

    11,151
    34
    Jan 23, 2006
    Mission...
    That is odd. curl has been very reliable for me for downloads over the years (with and without using kmttg). It may be interesting experiment for a show that has this problem to try it directly via a web browser to see if there are any issues that way. If that proves more reliable then perhaps a module to bypass using curl for downloads in kmttg could be useful.
     
  9. Aug 5, 2010 #2509 of 10413
    marbordom

    marbordom Member

    38
    0
    Sep 29, 2007
    Bronx, NY
    As I have experienced this I do not think that it is a curl issue per se.
    Only running curl under my Win7 shows the problem. Curl 7.19.7 e.g under Mac OS downloads the entire show w/o stalling. Of course it could be a Windows plus curl issue instead of either or. Just for the fun of it I have installed kmttg v0p7j. See if that makes any difference (according to my own statements it should not) but hell It's Windows - everything is possible !

    I found an 1.5 year old thread on curl which is pretty much what could be happening her:
    http://curl.haxx.se/mail/tracker-2009-02/0022.html
     
  10. Aug 6, 2010 #2510 of 10413
    moyekj

    moyekj Well-Known Member

    11,151
    34
    Jan 23, 2006
    Mission...
    For those with windows having curl hanging issues perhaps try the latest 7.21.0 version which you can download from here to see if behavior is any better:
    http://kmttg.googlecode.com/files/curl-7.21.0_win32.zip
    Rename the current "curl" folder in kmttg installation to something else and then install this one as the new "curl" folder.
     
  11. Aug 6, 2010 #2511 of 10413
    ferror

    ferror New Member

    66
    0
    May 3, 2005
    Austin, TX
    I was imprecise when I said it was a "curl problem," which I realize may have sounded unfair to the curl developers. Considering the fact that curl works as expected on other platforms, and it worked on my own system just a couple weeks ago and nothing else I know of has changed, it is far more likely that Microsoft broke something that affects curl in a recent patch. I have been working on problems like these for enough years to instinctively hop over that step, however, when it is an issue like this one. In a situation where it is a Microsoft mistake that breaks something Microsoft cares little about (curl), the burden to solve the problem will probably fall to the curl developers, unfortunately. Microsoft will not admit it is a bug and will call it a feature if the only impact is on things like curl. So, it is a "curl problem" in that the curl developers are the ones who will most likely end up solving it. :)

    I have been running 7.21.0 on my system for a day now, and it is still exhibiting the same behavior. Maybe I'll look into doing a kmttg modification to have it download the files via a browser rather than curl and see what happens then, as moyekj suggested. That might make for some solid proof in a "bug" report and also serve as a better short term workaround.
     
  12. Aug 6, 2010 #2512 of 10413
    lew

    lew Active Member

    3,965
    3
    Mar 12, 2002
    Not working. Last error Atomic Parsley failed exit code -1073741819 check command. The resulting mp4 files have issues. I can stream with streambaby. Push with streambaby and the tivo doesn't know the file has finished transferring. Eventually it stops and the transferred video vanishes. Use pytivo and it complains about the MOOV atom.

    My intent is to encode mp4 files that will be streamed or pushed back to my tivo. I'm not even sure what Atomic Parsley is suppose to do. I thought it puts the metadata in the mp4 file for ipds. I thought pytivo still needed the .txt for metadata.
     
  13. Aug 6, 2010 #2513 of 10413
    sirfergy

    sirfergy New Member

    288
    0
    May 18, 2002
    Is it possible when using VRD to convert a .tivo to a .wtv to also get all the show properties populated in the .wtv file?
     
  14. Aug 6, 2010 #2514 of 10413
    ThAbtO

    ThAbtO TiVoholic by the bay

    6,739
    9
    Apr 6, 2000
    SF Bay Area
    I reverted back to v0p7j and so far so good.
     
  15. Aug 6, 2010 #2515 of 10413
    wmcbrine

    wmcbrine Ziphead

    10,369
    22
    Aug 2, 2003
    Recentish versions of pyTivo can read metadata directly from some types of video files.
     
  16. Aug 6, 2010 #2516 of 10413
    marbordom

    marbordom Member

    38
    0
    Sep 29, 2007
    Bronx, NY
    This is better than I could have written it. I totally agree. Windows systems are way to alive - they change every day even w/o updates. Things just grow or get corrupted- well. Just to let some users know:
    1) I tried a system restore to a date prior of me seeing the curl problems
    -> I had to fix installing kmttg as a service, re updated java, re-installed kmttg itself but nothing made curl work again
    2) last week I did the latest curl upgrade but used the 64 bit version
    -> no luck with that either
    3) I used another Windows XP Pro desktop, installed kmttg latest version and in less than 24 hours it just downloaded about 30 shows, decoded and cut out commercials => So I'd say it is a Win7 problem even if MS does not agree although builds are depend. on cross/compilers and libs which might differ
    Finally these timeouts issues reported with curl and Windows (not a curl problem itself) still make sense to me. If I'd be sure that the Acer Revo 3100 would just work fine (HDMI audio output etc.) under Linux I would go for that - but then comcut needs to run under some emu like wine ?? - Hmm, lots of work. Maybe I should at first upgrade the HDD im the Tivo ...

    Anyway: If you find a fix I would like to know how I can hop onto that train. I have never compiled the kmttg sources nor modfied them but if that proves to be the "rord of the ling" be it and I will try.

    Great forum by the way !
     
  17. Aug 6, 2010 #2517 of 10413
    marbordom

    marbordom Member

    38
    0
    Sep 29, 2007
    Bronx, NY
    Hi, yes that was one of my first ideas: Upgrade curl !
    I used the 64 bit version first and now the link you posted here.
    Unfort. both did not work on my Win 7. The very first download got stuck after 5 min. I bet a king of TCPIP analysis could find the problem.

    Seems that my MS system is made for sadists .. self torture built in.
     
  18. Aug 7, 2010 #2518 of 10413
    moyekj

    moyekj Well-Known Member

    11,151
    34
    Jan 23, 2006
    Mission...
    I do have a Win 7 machine to test on, though it's not one I normally use. How prevalent is this issue though? Will I have to download 20-30 different shows to get the issue? Haven't seen the issue yet on that machine.

    ThAbtO, the fact that users can reproduce issue outside kmttg using curl seems to take kmttg out of equation, so switching versions is not likely a cure especially since it's same version of curl in both j & l versions anyway...

    EDIT: I've now downloaded over 20 different programs from 2 different S3s using Win7 Toshiba laptop, with downloads concurrent and haven't yet run into this hang issue, so looks like I won't be able to reproduce this myself...
     
  19. Aug 7, 2010 #2519 of 10413
    marbordom

    marbordom Member

    38
    0
    Sep 29, 2007
    Bronx, NY
    I believe I have to figure out a way how to produce a debug output from the curl executable via -v (verbose) and / or a TCP dump or whatever sniffer can be attached under Win 7. kmttg is at no fault here.
     
  20. Aug 7, 2010 #2520 of 10413
    wireman121

    wireman121 Member

    126
    0
    Nov 8, 2008
    Got a TiVo Premiere and set kmttg up to use it instead of my Tivo HD.... Also set it up to download in TS format instead of PS. It downloaded fine, but tivodecode is throwing an error now. see below:

     

Share This Page