New program for 1 step TTG downloads, decryption, encoding - kmttg

[Twoflower]

Wow. I'm so used to random technology problems nobody can explain that I'm surprised to see so many people with the same problem I have! That boosts my spirits that I'm not alone in it.

I normally just point my web browser at my Tivo Premiere on my LAN and download files that way. I can't get the official Tivo Desktop software to work for some reason, but straight downloads by HTTP worked fine... until last night.

Is there a way to fix this other than setting the clock back? I'm not sure what you mean by "use java for downloads". (Kind of a neophyte here.)

Edit: Mighta helped if I'd read page one of the thread. I got a copy of KMTTG and installed it, found the setting in question. I'm in the middle of "disable transfers, wait, reenable them" as a solution which I'd googled up for this problem before, but once that's done, I'll try using this. VERY impressive program so far! Even connects to my videoredo!

EditEdit: WOOHOO! IT WORKS! KMTTG, you are my hero. I'm able to download my shows again! Thanks to everyone for the suggestions.

[magilladke]

Originally Posted by notting

Dumping the HTTP headers on mine showed it giving a session id cookie that was already expired (Sat Feb 16 00:00 GMT), so the client doesn't send it back to the Tivo. Hence, no session id.

Setting the clock back on the computer 'fixes' it.

Just wanted to say "thanks" for everyone posting the fixes. I am on traveling and needed to get this working. I did the date rollback first, then saw the java option.

I'm a programmer and have seen this type of issue before with javascript (and certificates). Glad someone dumped the headers to see what was really going on.

I think kmttg is fantastic and hope the underlying issue gets fixed soon. Thanks again!

[wmcbrine]

Originally Posted by True Colors

kmttg comes to the rescue, once again

Technically, the fact that it works in kmttg's Java mode is a bug in kmttg. It's a case of two wrongs making a right.

[moyekj]

Originally Posted by wmcbrine

Technically, the fact that it works in kmttg's Java mode is a bug in kmttg. It's a case of two wrongs making a right.

It's actually not a bug since it is coded intentionally to accept any certificate, even expired ones as in this case.

[unitron]

The bug, apparently, is in the TiVos themselves, since rolling back the PC clock lets it work again.

[magilladke]

Originally Posted by wmcbrine

Technically, the fact that it works in kmttg's Java mode is a bug in kmttg. It's a case of two wrongs making a right.

People were seeing the issue with Tivo Desktop as well. This is why I figured it was more of a Tivo issue. Either way I'm just happy people were able to post a work around. Thanks everyone!

[wmcbrine]

Originally Posted by moyekj

It's actually not a bug since it is coded intentionally to accept any certificate, even expired ones as in this case.

It's a cookie, not a certificate.

[morac]

Found another work around for people using Firefox. Install the Cookie Manager+ add-on and edit the "sid" cookie to expire at a later date. In Firefox this appears to "stick" so going back to the TiVo web page doesn't cause the cookie to expire again. I don't use Chrome, but it looks like this add-on does the same thing.

Note curl has an option to read cookies from a file (--cookie). TiVo Desktop actually does that when it tries to download a program. It looks for a .cookie file in save to folder on the PC. The name of the file is the same name as the .TiVo file with a .cookie appended to it. So for example, if the TiVo file is "Curiosity - ''Did God Create the Universe'' (Recorded Aug 7, 2011, DSCHD).TiVo" the the cookie file would be "Curiosity - ''Did God Create the Universe'' (Recorded Aug 7, 2011, DSCHD).TiVo.cookie". Theoretically if you create a cookie file for curl to read and put an unexpired sid cookie in there, it should work. See this page.

I'm not sure how TiVo's going to go about fixing this for old boxes as I've fairly certain the cookie is coded into the box's software. When I looked at my browser cookies I found that all TiVo box cookies expired Feb 15, 2013. It was probably hard coded into the code to do that. Premieres can be updated, but with S2 and S3 not getting updates, I don't see how they can be fixed.

[morac]

Just to expand on what I wrote above. Type the following curl command to download the box's cookies:

curl -c cookies.txt -d "tivo:MAK" http://tivoip:80

The current expiration date listed is the number "1360972800", which is the number of seconds since midnight of January 1, 1970. So simply change that number to when you want the cookie to expire. For example if you want it to expire next year, change it to 1392508800.

You can then use "curl -b cookies.txt" to read in the cookie file. As I mentioned above TiVo Desktop tries to read in a .cookie file with the same name as the .TiVo file it is downloading. It seems like it would be possible to trick it into using the cookie file.

Unfortunately I've found that the TiVo returns a different SID value each time curl makes a request, but apparently the TiVo doesn't care what SID is sent as long as it's valid. I "created" a new cookie for my S3 and gave it the same SID cookie value that my Premiere gave me and could download from my S3 using Firefox.

[Fofer]

So we change the date back on our computer to fix this? Can we then change it back? What a mess.

[morac]

Originally Posted by Fofer

So we change the date back on our computer to fix this? Can we then change it back? What a mess.

That or build your own cookie as I described above. I've found that the TiVo box (at least the S3 OLED) doesn't care what the value of the SID is, as long as it is in a valid format and is sent.

[BigJon]

Many thanks to notting, morac, unitron and the others who figured this out....and have pointed to this thread from the at least one other that is out there ranting on this subject. Just to be clear, is there an assumption here in the community that this is a planned obsolescence strategy on the part of TiVo to force folks out of perfectly good TiVo S2's and S3's? (I don't normally buy into conspiracy theories, but....). They just had a promotion to pay 99 bucks for a lifetime service plan on your S3/HD if you bought a Premier 4 through them. So, once again, are we not likely to see a software revision to the S3/HD's (or the S2's for that matter) that will fix this?

And for those of us who are not yet Jedi Masters with pytivo, and some of the other tricks out there, what is the easiest most foolproof way to update the cookie file on a TiVo HD TCD652160 upgraded to a 2tb drive, using TiVo Desktop as the PC/transfer interface (I can't just roll back my clock and leave it there for other reasons as the PC does other stuff beyond TiVo server duty). I am not entirely clear on what medium to use to type the command lines to set the next cookie expiration date out 25 years if that's what I want to do (and, if I want a 25 year expiration, why not?).

Thanks again for the help some of the members have put into this.

BJ

[moyekj]

BigJon, if you use kmttg for downloads an easy workaround was already discovered and posted in this thread.
File-Configure-Program Options-Use java for downloads instead of curl

Can't help you with TiVo Desktop as I don't like to use it and don't have a full installation of it. I doubt TiVo will let this problem sit for too long without offering a workaround of their own.

[jbrown13]

Set my clock back to 2/13/2013 and Desktop started transferring again. Another bug from the TiVo software group that can't seem to release new versions without creating issues.

[opieant]

moyekj: For what it's worth, I modified kmttg to do the following as a workaround since my use of kmttg requires curl support:

- Add "--head" to the usual curl command line, and run curl so it only caches the cookie without trying to download
- Parse the value for the "sid" cookie from the cookie file
- Add the "sid" cookie to the usual curl command line with --cookie (or -b, if you prefer), and run curl to download the file

My implementation of this makes use of some of my previous modifications to kmttg, so unfortunately my code can't just be added directly into the current source. I just wanted to suggest this approach since it works nicely, but you may have a better idea.

[pL86]

Thanks to everyone who figured this out - too bad I didn't find this answer before I reinstalled Windows. What a hassle...

One question - does the date need to be set before February 13 every time before transferring a recording to the PC? Or is once enough and going forward, this step doesn't need to be done?

[Fofer]

Originally Posted by jbrown13

Set my clock back to 2/13/2013 and Desktop started transferring again. Another bug from the TiVo software group that can't seem to release new versions without creating issues.

To be fair, this bug was probably there since the first version, it's just that the date of Feb. 16th 2013 triggered it.

[Fofer]

Originally Posted by morac

That or build your own cookie as I described above. I've found that the TiVo box (at least the S3 OLED) doesn't care what the value of the SID is, as long as it is in a valid format and is sent.

Now I am curious, how does building a browser cookie on my computer take care of this? Do iTiVo and kmttg use and share these same cookies?

[moyekj]

Originally Posted by opieant

moyekj: For what it's worth, I modified kmttg to do the following as a workaround since my use of kmttg requires curl support:

- Add "--head" to the usual curl command line, and run curl so it only caches the cookie without trying to download
- Parse the value for the "sid" cookie from the cookie file
- Add the "sid" cookie to the usual curl command line with --cookie (or -b, if you prefer), and run curl to download the file

My implementation of this makes use of some of my previous modifications to kmttg, so unfortunately my code can't just be added directly into the current source. I just wanted to suggest this approach since it works nicely, but you may have a better idea.

OK thanks. I tried this out and got it to work. Run 1st with --head and without an output file, parse stdout to grab the sid and then run a 2nd time normal way with --cookie sid=xxxx. This also means the cookie file (--cookie-jar) is no longer needed when doing it that way. I'll keep the code on hand in case it becomes necessary for next release.

[morac]

Originally Posted by Fofer

Now I am curious, how does building a browser cookie on my computer take care of this? Do iTiVo and kmttg use and share these same cookies?

The TiVo http server is simply looking for a SID cookie, it doesn't seem to care if it gets one that doesn't match the one it set. The reason it's currently failing is because no cookie is sent since it expired. Building your own violates RFC standards, but it works so who cares.

[opieant]

Originally Posted by morac

That or build your own cookie as I described above. I've found that the TiVo box (at least the S3 OLED) doesn't care what the value of the SID is, as long as it is in a valid format and is sent.

I just checked this out since I had to see it to believe it. Sure enough, it works, and formatting doesn't seem to matter either. The TiVo seems to accept any string of at least one character as a valid session ID. Considering that, it's amazing they bothered to write the code to check if the "sid" cookie is sent at all. The programmers just needed to slack off a tiny bit more and this transfer problem never would have cropped up.

[tluxon]

Originally Posted by morac

Just to expand on what I wrote above. Type the following curl command to download the box's cookies:

curl -c cookies.txt -d "tivo:MAK" http://tivoip:80

The current expiration date listed is the number "1360972800", which is the number of seconds since midnight of January 1, 1970. So simply change that number to when you want the cookie to expire. For example if you want it to expire next year, change it to 1392508800.

You can then use "curl -b cookies.txt" to read in the cookie file. As I mentioned above TiVo Desktop tries to read in a .cookie file with the same name as the .TiVo file it is downloading. It seems like it would be possible to trick it into using the cookie file.

Unfortunately I've found that the TiVo returns a different SID value each time curl makes a request, but apparently the TiVo doesn't care what SID is sent as long as it's valid. I "created" a new cookie for my S3 and gave it the same SID cookie value that my Premiere gave me and could download from my S3 using Firefox.

I was able to see that the cookies.txt files created from each of my TiVoHDs contained the "1360972800" value, but I'm not clear on whether you were describing making it a global (all the cookies in the TiVo box) change or if it needs to be sent as an argument for each download request. When I commanded

curl -b cookies.txt -d "tivo:MAK" http://tivoip:80

to upload the edited cookies.txt file to the TiVo, it doesn't seem to have done anything because a subsequent

curl -c cookies.txt -d "tivo:MAK" http://tivoip:80

still created a cookies.txt file containing the "1360972800" value.

I guess I'm not quite understanding this...

[moyekj]

Originally Posted by tluxon

I guess I'm not quite understanding this...

All you need to do for curl to work is use:
--cookie sid=abc

As discussed above the TiVo http server doesn't seem to care what the actual sid value is as long as there is one.

[tluxon]

Originally Posted by moyekj

All you need to do for curl to work is use:
--cookie sid=abc

As discussed above the TiVo http server doesn't seem to care what the actual sid value is as long as there is one.

So the idea isn't to change the expiry time value of the cookie residing on the TiVo - but to just tweak the download request by substituting a dummy "cookie" string for the standard cookie string? Does that require downloads to be done on a manual basis or can this tweak be done on a more global basis?

[moyekj]

v0p9l version released with above discussed fix/workaround for curl downloads. Also contains new Season Passes Export button to export to csv file among some other things. See release_notes for all the details.

[morac]

Originally Posted by tluxon

So the idea isn't to change the expiry time value of the cookie residing on the TiVo - but to just tweak the download request by substituting a dummy "cookie" string for the standard cookie string? Does that require downloads to be done on a manual basis or can this tweak be done on a more global basis?

It depends on what program you are using to do the download. You can add a cookie that doesn't expire to most web browsers. TiVo Desktop uses curl, but it specifies it's own parameters so it's not really something you can automate. Third party programs that use curl can be changed as needed. So except for TiVo Desktop it can be done "globally".

I'm wondering if it would be possibly to replace the curl.exe in the TiVo Desktop folder with a batch file that simple calls curl.exe with the required "--cookie" argument. That would work depending on whether or not TiVo Desktop tried to run "curl" or "curl.exe". If TiVo Desktop calls curl.exe specifically there are still ways of doing this, but it becomes more difficult. At that point it would be easier to modify the source code of curl to hard code the cookie so a command line argument isn't needed.

[tluxon]

Originally Posted by moyekj

v0p9l version released with above discussed fix/workaround for curl downloads. Also contains new Season Passes Export button to export to csv file among some other things. See release_notes for all the details.

That was quick. Thanks!

[howards]

Originally Posted by moyekj

v0p9l version released with above discussed fix/workaround for curl downloads. Also contains new Season Passes Export button to export to csv file among some other things. See release_notes for all the details.

Thanks so much for the quick release fix for kmttg. Works fine now using curl. If only Tivo Desktop engineers were as responsive as you!

[howards]

Originally Posted by morac

I'm wondering if it would be possibly to replace the curl.exe in the TiVo Desktop folder with a batch file that simple calls curl.exe with the required "--cookie" argument. That would work depending on whether or not TiVo Desktop tried to run "curl" or "curl.exe". If TiVo Desktop calls curl.exe specifically there are still ways of doing this, but it becomes more difficult. At that point it would be easier to modify the source code of curl to hard code the cookie so a command line argument isn't needed.

I gave this a try. Didn't work - failed with "unknown error." I think this means Tivo Deskop calls curl.exe (which I renamed to something else).

[caddyroger]

Ok I just tried the new kmttg_v0p9l. When I start a program it will start for a couple of seconds then repeats. After about 5 tries it will go to this or similar.

<h2>Bad Request</h2>
Download failed to file: <parsed filename>.TiVo
Exit code: 0
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 31 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 31 0 31 0 0 86 0 --:--:-- --:--:-- --:--:-- 31000
0 39 0 39 0 0 55 0 --:--:-- --:--:-- --:--:-- 55

It does this about 4 time and the it will end. I reset the the clock back to 02-13-2013, and still did not help.
I reinstalled kmttg_v0p9k and set the clock back to 02-13-2013