1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New program for 1 step TTG downloads, decryption, encoding - kmttg

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Mar 15, 2008.

  1. Aug 7, 2010 #2521 of 10411
    wireman121

    wireman121 Member

    126
    0
    Nov 8, 2008
    Little side note, PS format seems to be downloading at 20Mps while TS format last night maxed out at 11Mps... shouldnt it be the other way around?
     
  2. Aug 7, 2010 #2522 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    TS downloads should be higher than 20Mbps. In any case there are plenty of posts about Premiere TS downloads and/or decryption having issues, so should not be used.
     
  3. Aug 7, 2010 #2523 of 10411
    wireman121

    wireman121 Member

    126
    0
    Nov 8, 2008
    I was afraid to hear that. Is there a discussion somewhere that is working on getting the TS downloads working properly with kmttg, tivodecode, etc? I'd be interested in participating in getting it to be more stable
     
  4. Aug 7, 2010 #2524 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    The TS download problems are purely a Premiere issue, not curl, kmttg or whatever else you use to download in that format. For tivodecode issues with TS streams the latest source code including TS support is available from kmttg downloads section:
    http://kmttg.googlecode.com/files/tivodecode-0.3pre4.tar.gz
    It is based on some anonymous poster who posted patches at the sourceforge site (and mentioned there was still a bug present):
    http://sourceforge.net/tracker/?atid=906362&group_id=183716&func=browse

    i.e. source code is available if someone wants to take a crack at fixing tivodecode. But that still doesn't help if TS downloads from Premiere units are not reliable - only TiVo can fix that and they have much bigger fish to fry when it comes to the Premiere...
     
  5. Aug 7, 2010 #2525 of 10411
    wireman121

    wireman121 Member

    126
    0
    Nov 8, 2008
    Yea, I made sure I have everything updated to the newest version. I'm not sure why its downloading so slow. I restarted a TS download to see if there would be a difference. It started coming at about 50Mbps for maybe 5 seconds, and has since dropped to an average of 10Mbps. I'm not sure what is causing that - I know the network is fine, and if I initiate a PS download it comes a little faster.

    Besides that, I'm going to see if redownloading the TS file works with tivodecode this time, otherwise I'll just be leaving TS off for now until things get fixed...

    I have to agree. I'm not that excited about the Tivo Premiere. I thought, if nothing else, it would be faster to navigate through the menus and whatnot, but it definately is slower than my Tivo HD. And theres no menu option to initiate a restart?

    Its currently at 14.4 version, and I cant figure out how to get it to update to 14.5, not that I actually expect that to fix any of my problems, but tivo.com says 14.5 is newest...
     
  6. Aug 7, 2010 #2526 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    I know it's off topic, but: If I had a Premiere I would just stick to SD menus. From all accounts that is stable and much faster than THD/S3 units. Also PS downloads are much faster than THD/S3 units as well so I would be content with that as well.
     
  7. Aug 8, 2010 #2527 of 10411
    ferror

    ferror New Member

    66
    0
    May 3, 2005
    Austin, TX
    @moyekj: I agree on the Premiere HDUI. I wish they would just offer an option to widen the existing SD menus and use a smaller font. It would still run very fast on the new hardware, but skip all that Flash mess until some other time (or not.) They should even back-port that to the S3/THD as well. Seeing as they call that an HD unit, it seems like it should have menus that optionally support that format optimally.

    @marbordom and others experiencing the stalled downloads in curl on Windows:

    It appears I have found a solution to this issue on my system. I installed the Cygwin version of curl and set kmttg to use that instead of the version that is packaged with it. If you are not familiar, Cygwin is a *NIX-style command shell and tools emulator for Windows. They have ported most of the library code that the *NIX tools require and can compile them to run in Windows, and they actually have done a very good job with most of them. Cygwin is a bit unconventional to most Windows users, so most packages including curl have native Windows distributions that run as standalone apps. I think the advantage in this case is that curl for Cygwin is compiled in a unique way and runs through a different set of APIs compared to the native Windows curl distribution. This looks like it is a fluke that just happens to work in our favor here. Whatever is happening does not seem to effect the Cygwin version. The curl developers will still need to find the root cause of the problem, but this buys us some time by giving us a relatively easy way around the issue for now.

    To do it, just run the Cygwin setup program and then be sure to enable the curl package to be installed. Then change the kmttg curl path to point at c:\Cygwin\bin\curl.exe, or wherever it is installed. The Cygwin curl will run fine even if you are not in the Cygwin shell. Now your log entries should show curl running from that new path and with any luck you will also find that your downloads are working right again. :)
     
  8. Aug 8, 2010 #2528 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    Interesting find ferror. NOTE: You don't need a whole cygwin installation to get it to work. I pulled out curl.exe and all the necessary accompanying cygwin .dll files and made a self-contained package that runs without having cygwin installed. Those having this issue can try out this version of curl:
    http://kmttg.googlecode.com/files/cygwin_curl.zip
    (Rename the kmttg "curl" folder and replace with the one in the above zip file).
     
  9. Aug 8, 2010 #2529 of 10411
    ferror

    ferror New Member

    66
    0
    May 3, 2005
    Austin, TX
    Thanks. Yeah, sorry installing Cygwin is a bit extreme, was just a quick way to get the desired result. I was just thinking about looking at what files would be needed to just run curl and then saw your post. Putting this version of curl into the kmttg dist would actually solve the problem completely, at least for the time being. Then the curl developers fixing it in their windows version would just be an interesting bit of news for building the next bundle.

    EDIT: I have one thing to add regarding the cygwin curl. If you happen to already have cygwin installed on your machine for any other reason, you should use the cygwin setup program to install/update curl and not use a separate version of curl. The reason is that the .dll files will probably get out of sync.
     
  10. Aug 8, 2010 #2530 of 10411
    marbordom

    marbordom Member

    38
    0
    Sep 29, 2007
    Bronx, NY
    That find is awesome. Yesterday I ran kmttg under Ubuntu and even there I found the curl problem - I had to restart the Tivo since resetting the web server did not solve that no connection could be made to the tivo's IP. Now I have to fix my Win 7 before I can trye your solution .. after shrinking the Win7 partition I can not boot into this or the recover partition ...
     
  11. Aug 9, 2010 #2531 of 10411
    marbordom

    marbordom Member

    38
    0
    Sep 29, 2007
    Bronx, NY
    Hi, when I use the unzipped dir under kmttg the curl.exe can not find cygz.dll. Since I do not know which version this it copying one would just be another trial and error. Other than that I do have cygwin installed and the version installed on my Win7 is not running either (gets stalled)
    [xxx@acer]$ /bin/curl --version
    curl 7.20.1 (i686-pc-cygwin) libcurl/7.20.1 OpenSSL/0.9.8o zlib/1.2.5 libidn/1.1
    8 libssh2/1.2.5
    Protocols: dict file ftp ftps http https imap imaps pop3 pop3s rtsp scp sftp smt
    p smtps telnet tftp
    Features: IDN IPv6 Largefile NTLM SSL libz
     
  12. Aug 9, 2010 #2532 of 10411
    ferror

    ferror New Member

    66
    0
    May 3, 2005
    Austin, TX
    Hmm, sounds like maybe the zip needs another file, but if you are running cygwin now anyway, that should do the trick. You are running the exact same version I am running here. Mine actually did stall again once during the afternoon today, but I've seen that happen before for other reasons. Before I switched to the cygwin curl last night, the stalls were happening on almost every file at least once or twice. It was a very acute issue, not just an annoyance. The -y 60 -Y 100 parameters did help to make it eventually get the files downloaded. When I was testing the cygwin curl, I took those parameters out to see how it handles it without them. When it stalled this afternoon, I decided to go ahead and put them back as an additional measure against the issue. Looks like the stalls can still happen, but in my case the cygwin curl reduced those stalls to such a low amount that it took a day of continuous running before I saw it happen again. Sorry you aren't having as much luck with yours. I am not sure what the variables are in this equation, but I think we are close.
     
  13. Aug 9, 2010 #2533 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    Yeah I guess I missed that dll. I've updated the download to include it and verified that it runs on my Win 7 machine without cygwin installed:
    http://kmttg.googlecode.com/files/curl_cygwin.zip

    BTW, I've now downloaded over 40 different shows from my S3s to a Win 7 laptop and have not had a single freeze or incomplete download, so I cannot reproduce this issue at all.
    Are these curl freezes when downloading from Premiere units only?
     
  14. Aug 9, 2010 #2534 of 10411
    ferror

    ferror New Member

    66
    0
    May 3, 2005
    Austin, TX
    For me it happens on either of my TiVo HDs and on my Premiere (which I thought I was getting to replace one the the THDs, but never wound up selling it). It seems to happen more often on the Premiere, than the THDs, but I think this is just because the download speed is faster and so it is that much more likely to fail. Due to the faster transfer speed, it may also increase the likelihood of getting a stall. I have not tried it on anything other than win7 but marbodom said he saw it on Ubuntu as well, which was a big surprise to me. Maybe the issue goes a bit deeper than originally thought.
     
  15. Aug 9, 2010 #2535 of 10411
    lew

    lew Active Member

    3,951
    3
    Mar 12, 2002
    Add me to the list of users who solved issues by uninstalling Java (including every update) then installing the current version.
     
  16. Aug 9, 2010 #2536 of 10411
    bwall23

    bwall23 New Member

    73
    0
    Jun 8, 2009
    Is there a reason why seriesId and programId are not appearing in the generated metadata files?

    Running kmttg v0p7l and tiveodecode/tdcat v0.3pre4 on Win7-64.

    Have .tivo files on the PC and want to decode them to mpg, create metadata files, then delete the .tivo files.

    Problem I'm running into is if I tell kmttg to create metadata files, they don't contain seriesId or programId even though that metadata is in the .tivo files.

    If I manually run tivodecode or tdcat on the .tivo files it shows both seriesId and programId in chunk2.
     
  17. Aug 9, 2010 #2537 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    seriesId being left out is intentional, see:
    http://code.google.com/p/kmttg/issues/detail?id=50
    programId is not documented in pyTivo metadata Wiki and thus has never been included.
     
  18. Aug 10, 2010 #2538 of 10411
    bwall23

    bwall23 New Member

    73
    0
    Jun 8, 2009
    Thanks for the quick reply, but whew! Me eyes are too tired too feed all this to my brain right now. And that's a good thing because it couldn't process it all tonight anyway. I'll re-read later when I have time to digest and reply. My first thought was this may have to do with padding recordings (a minute or more early and/or late) and chunk1 vs. chunk2 of metadata where <vActualShowing> vs. <showing> has the data.

    Main concern is getting seriesId in there, programId is secondary, but why leave it out just because it's not documented in pyTivo metadata Wiki, does it make something choke on it?.
     
  19. Aug 10, 2010 #2539 of 10411
    bwall23

    bwall23 New Member

    73
    0
    Jun 8, 2009
    I think I understand now (until a proper xml parser is implemented).

    I can send you samples of metadata from recordings where I;

    A) don't add anything to the scheduled guide recording time.
    B) start recording a minute before the scheduled recording time.
    C) stop recording a minute after the sheduled recording time.
    D) start recording a minute before and stop recording a minute after the scheduled recording time.

    For each instance of A to D above I can include;
    1) recursive query to my TiVo's NPL
    2) query to my TiVo for video details of the recording
    3) output from tivodecode/tcat for that same recording from the .tivo file after transfer to my PC
    4) output from kmttg after trying to create metadata from said .tivo file

    So, if it would help I could try this and send you all 16 files (A1-4, B1-4, C1-4, D1-4).

    i.e.

    Code:
    A) don't add anything to the scheduled guide recording time.
     1) recursive query to my TiVo's NPL
     2) query to my TiVo for video details of the recording
     3) output from tivodecode/tcat for that same recording from the .tivo file after transfer to my PC
     4) output from kmttg after trying to create metadata from said .tivo file
    B) start recording a minute before the scheduled recording time.
     1) recursive query to my TiVo's NPL
     2) query to my TiVo for video details of the recording
     3) output from tivodecode/tcat for that same recording from the .tivo file after transfer to my PC
     4) output from kmttg after trying to create metadata from said .tivo file
    C) stop recording a minute after the sheduled recording time.
     1) recursive query to my TiVo's NPL
     2) query to my TiVo for video details of the recording
     3) output from tivodecode/tcat for that same recording from the .tivo file after transfer to my PC
     4) output from kmttg after trying to create metadata from said .tivo file
    D) start recording a minute before and stop recording a minute after the scheduled recording time.
     1) recursive query to my TiVo's NPL
     2) query to my TiVo for video details of the recording
     3) output from tivodecode/tcat for that same recording from the .tivo file after transfer to my PC
     4) output from kmttg after trying to create metadata from said .tivo file
    Only thing is I can't help you parse XML files and am not interested in getting back into coding (past couple lives of that burnt me out;-)
     
  20. Aug 10, 2010 #2540 of 10411
    moyekj

    moyekj Well-Known Member

    11,134
    27
    Jan 23, 2006
    Mission...
    To be honest my motivation is pretty low to implement proper xml parser just to address this seriesId issue. There are full xml parsers available out there already but that adds a pretty big bloat of a library to include in kmttg.jar so don't really want to go there. For video transfers to TiVos they end up without programId anyway so not sure what the point is to include that field in metadata file? For seriesId I can understand wanting it in there since it does affect grouping when transferring back to TiVos. I'll look into that more when time permits - right now my real job is taking up most of my time.
     

Share This Page