TiVo Community Forum banner
  • TiVoCommunity.com Ambassador Program Now Open! >>> Click Here

New program for 1 step TTG downloads, decryption, encoding - kmttg

2M views 13K replies 921 participants last post by  mattack 
#1 ·
EDIT: This program has evolved a lot since this 1st post and now is written in Java and much easier to install than before (very easy on Windows and Mac OSX)... See http://sourceforge.net/projects/kmttg/ for details or visit the end of this thread for more up to date information.

kmttg is a Perl/Tk program I wrote to facilitate TivoToGo (TTG) transfers that can download, create pyTivo metadata, decrypt, run comskip & comcut (commercial detection and removal) and re-encode multiple shows you select from your Tivos all in 1 step.
You can select one or more shows at a time and then with one click of a button the program will download all the selected items, with the options of also automatically creating a metadata file for pyTivo, decrypting .TiVo files to .mpg, running comskip (commercial detection and removal program), and automatically re-encoding to a more portable format using mencoder, ffmpeg or any other command line encoder of your choosing. The program queues up multiple jobs and displays time, size and speed statistics for ongoing jobs.

For more information, screenshots and download visit:
http://sourceforge.net/projects/kmttg

Web page contains windows executables for all 3rd party tools used. The only other requirement of course is you must have Perl installed. Runs under Windows or Linux - tested with WinXP SP2 & Linux Red Hat Enterprise 4.

If you do try this out would appreciate some comments/feedback. For any programmers out there feel free to tinker and make improvements yourself.
 
See less See more
#4,370 ·
I noticed after getting the fall update, that any shows pulled to my TiVos show the transfer date in kmttg instead of the OAD.

The OAD appears correctly on the TiVo.

Has anyone else noticed this?
same here
AFAIK, kmttg has always shown the record date/time, not the OAD. TiVo has done something odd here as the record date/time still displays properly on the TiVo.
 
#4,371 ·
AFAIK, kmttg has always shown the record date/time, not the OAD. TiVo has done something odd here as the record date/time still displays properly on the TiVo.
That's right, kmttg has always used <CaptureDate> from the NPL XML as the date, so I guess for pulls <CaptureDate> is being set to the time the show is pulled? For those seeing this are you pulling with pyTivo and associated pyTivo metadata file?
 
#4,372 ·
I noticed after getting the fall update, that any shows pulled to my TiVos show the transfer date in kmttg instead of the OAD.

The OAD appears correctly on the TiVo.

Has anyone else noticed this?
I'm not sure I follow exactly what you are saying. Where are you looking in kmttg to find this date, and how can the date be both correct and incorect on the Tivo?
 
#4,373 ·
That's right, kmttg has always used <CaptureDate> from the NPL XML as the date, so I guess for pulls <CaptureDate> is being set to the time the show is pulled? For those seeing this are you pulling with pyTivo and associated pyTivo metadata file?
I'm pulling with pyTivo and an associated pytivo metadata file. I'm also using the default.txt pyTivo option to force the OAD to be used.

I'm not sure I follow exactly what you are saying. Where are you looking in kmttg to find this date, and how can the date be both correct and incorect on the Tivo?
Shows pulled before the update show the OAD in kmttg. Shows pulled after the update show the pull date in kmttg. In both cases, the OAD is correctly displayed in the NPL.
 
#4,375 ·
I'm pulling with pyTivo and an associated pytivo metadata file. I'm also using the default.txt pyTivo option to force the OAD to be used.
And you're saying that was working fine before 20.2.2 software right? Which implies something changed in metadata handling for pulls (and apparently pushes as well since seriesId handling for grouping purposes seems to have changed). I think this topic is more relevant in pyTivo thread by the sound of it.
 
#4,376 ·
And you're saying that was working fine before 20.2.2 software right? Which implies something changed in metadata handling for pulls (and apparently pushes as well since seriesId handling for grouping purposes seems to have changed). I think this topic is more relevant in pyTivo thread by the sound of it.
Except that he is seemingly saying everything is fine on the TiVo for content transferred both pre- and post-upgrade, but that something in kmttg is now not showing the OAD. If kmttg ever shows the OAD, I would like to know about it.
 
#4,377 ·
Except that he is seemingly saying everything is fine on the TiVo for content transferred both pre- and post-upgrade, but that something in kmttg is now not showing the OAD. If kmttg ever shows the OAD, I would like to know about it.
No I think he's saying he setup pyTivo pulls such that the <CaptureDate> would get set to OAD and that used to work fine, but now CaptureDate is being set to the time of transfer instead of OAD in the metadata file with the new TiVo software.
 
#4,378 ·
No I think he's saying he setup pyTivo pulls such that the <CaptureDate> would get set to OAD and that used to work fine, but now CaptureDate is being set to the time of transfer instead of OAD in the metadata file with the new TiVo software.
moyekj is correct.

Both before and after the TiVo update OAD shows correctly in the NPL and sorts by OAD as desired. The dates on the TiVo are displayed correctly.

The kmttg date that is displaying differently is on the kmttg list of shows. This date displays the OAD for shows pulled before the update, if I used the default.txt when I pulled the shows, and the transfer date for shows pulled after the update.
 
#4,380 ·
Once again, kmttg does not display the OAD in the list, it displays the capture date/time, which used to be the same as the record date/time. Apparently, with the latest update, the Premieres are differentiating between the two. The TiVo displays the OAD ( as first aired) and the record date/time. It does not display the capture date.

Having time : OAD in the metadata simply tells pyTivo to use the OAD as the record date/time that it puts in the XML for the transfer.
 
#4,381 ·
moyekj is correct.
Fine, but what moyekj said and what you continue to say are completely different.

Both before and after the TiVo update OAD shows correctly in the NPL and sorts by OAD as desired. The dates on the TiVo are displayed correctly.
Yeah, I got that.

The kmttg date that is displaying differently is on the kmttg list of shows. This date displays the OAD for shows pulled before the update
That's my point. Kmttg has never shown the OAD in the list of shows from the TiVos, to my knowledge. I wish it did, and indeed I requested that very feature 3 months ago in this post. See the subsequent conversations.

if I used the default.txt when I pulled the shows, and the transfer date for shows pulled after the update.
I must be missing something, here. First of all, when you say "pull" I presume you mean pull from the PC to the TiVo (GoBack) not from the TiVo to the PC (TTG, or in context kmttg). To my knowledge no contents of any default.txt file has anything to do with the kmttg list of shows. The date shown by kmttg, both in the TiVo NPL window and the log file window is and always has been the date the show was recorded, according to the TiVo's response to the XML request from kmttg.

Hmm. Maybe I am getting a glimmer of what you are saying. Are you saying the date reported by the TiVo only for shows transferred back to the Tivo from the PC are now being reported differently? If so, then moyekj is most certainly correct that this is a pyTivo / TiVo issue, since kmttg only reports what the TiVo tells it. OTOH, it also doesn't help me, at all, since I don't care about the OAD of transferred material, but do often care about the OAD of material not yet transferred from the TiVo to the PC.
 
#4,382 ·
My apologies for not understanding the code (I&#8217;m not a programmer) or using the correct terminology and therefore lengthening what I thought was a shorter conversation.

Perhaps a screen capture from kmttg will help (attached).

The top show was pulled from my PC to the TiVo before the update. It correctly displays the OAD from the metadata file. The bottom show has exactly the same metadata, but was transferred after the update. The kmttg date for the bottom show reflects the time I started transferring the show.

The OAD displays correctly on my TiVo for both shows

I've also attached the default.txt and metadata files in case that clarifies anything.

My original point was that the TiVo reports the dates correctly, but kmttg is reporting the dates differently after the update.

Is this something that can be fixed in kmttg, is it a pyTivo issue, or is it a bug introduced on the TiVo side?

Ed
 

Attachments

#4,383 ·
I duplicated what you guys are seeing with the new TiVo software by experimenting with some pyTivo pulls.

From what I could see from NPL XML (behavior with new Premiere software):
There is a new entry called <ShowingStartTime> in the XML in addition to <CaptureDate>. The ShowingStartTime is influenced by metadata (time first else originalAirDate) while now the CaptureDate is purely the time at which you made the transfer. If there is no time information in metadata then ShowingStartTime is close to but not equal to CaptureDate.

Older software XML (such as my S3 OLED unit):
There is no <ShowingStartTime> entry, only <CaptureDate>.

So it looks like the proper course of action now would be to use <ShowingStartTime> if available, else <CaptureDate>. I'll have to implement this for next release.
 
#4,384 ·
My apologies for not understanding the code (I'm not a programmer) or using the correct terminology and therefore lengthening what I thought was a shorter conversation.

Perhaps a screen capture from kmttg will help (attached).

The top show was pulled from my PC to the TiVo before the update. It correctly displays the OAD from the metadata file. The bottom show has exactly the same metadata, but was transferred after the update. The kmttg date for the bottom show reflects the time I started transferring the show.

The OAD displays correctly on my TiVo for both shows

I've also attached the default.txt and metadata files in case that clarifies anything.

My original point was that the TiVo reports the dates correctly, but kmttg is reporting the dates differently after the update.

Is this something that can be fixed in kmttg, is it a pyTivo issue, or is it a bug introduced on the TiVo side?

Ed
It has nothing to do with code or terminology. The problem is that you continue to misunderstand what you are seeing. Before the update there were two date fields, the OAD and the recording date. What kmttg was displaying was the recording date, which it sees as "Capture Date". Now there are three fields, the OAD, the date of the original recording, and the date it was transferred which is what kmttg now sees as the "Capture Date". Only the first two are displayed by the TiVo.

As I said before, the "time : OAD" entry in your default.txt file simply instructs pyTivo to set the record date/time to the same value specified for the originalAirDate.

It is not a bug in pyTivo, the TiVo s/w or kmttg and in fact, IMHO, makes "capture date" more accurate as it now reflects the actual date/time the recording was made on that TiVo, rather than the date/time of the source recording.

Having said that, I am in favor of the proposed change to kmttg.

One more thing. If you ever end up with episodes of a series whose OAD is prior to @1970, having time : OAD is problematic as it might cause the TiVo to crash.
 
#4,386 ·
Quick encoding question ...

I have been having problems with encoding recordings from CBS (1080i) downloaded via kmttg. The download and decrypt works fine, but ffmpeg seems to hang, leaving just a 42mb file. I am using this custom profile I created for use with AirVideo:

Code:
# Description (single line, keep short)
<description>
airvideo h264 + 2 channel aac encoding (1080i source)

# Encode command or script (single line)
# Known keywords: FFMPEG, HANDBRAKE, MENCODER, PERL, INPUT, OUTPUT, PWD, CPU_CORES, SRTFILE
<command>
/etc/tivo/airvideo/ffmpeg-for-airvideo -y -i INPUT -threads CPU_CORES -flags +loop -g 30 -keyint_min 1 -bf 0 -b_strategy 0 -cmp +chroma -refs 1 -coder 0 -me_range 16 -subq 5 -partitions +parti4x4+parti8x8+partp8x8 -trellis 0 -sc_threshold 40 -i_qfactor 0.71 -qcomp 0.6 -ss 0.0 -vcodec libx264 -vf crop=1920:1080:0:0,scale=800:448,pad=800:448 -aspect 800:448 -async 1 -f mp4 -crf 24 -qmin 24 -r 29.97 -ar 48000 -ac 2 OUTPUT

# Encoded output file extension
<extension>
m4v
I use the same profile for encoding shows off of other 1080i channels (like NBC) just fine, it only seems to fail on CBS shows, and even then not ALL the time, but enough to be annoying. I've seen this on BBT, HIMYM, and CSI.

If I run the kmttg-generated encode command-line manually, I'll get this:

Code:
Output #0, mp4, to '/etc/tivo/togo/The Big Bang Theory - The Extract Obliteration (2012-11-01).m4v':
  Metadata:
    encoder         : Lavf53.0.3
    Stream #0.0: Video: libx264, yuv420p, 800x448 [PAR 1:1 DAR 25:14], q=24-31, 200 kb/s, 2997 tbn, 29.97 tbc
    Stream #0.1: Audio: aac, 48000 Hz, 2 channels, s16, 64 kb/s
Stream mapping:
  Stream #0.0 -> #0.0
  Stream #0.1 -> #0.1
Press [q] to stop encoding
Input stream #0.1 frame changed from rate:48000 fmt:s16 ch:6 to rate:48000 fmt:s16 ch:2

< snip out encoding frames ... and then >

frame=  524 fps= 11 q=30.0 size=    2156kB time=16.32 bitrate=1082.5kbits/s dup=
[mpeg @ 0x206cf60] Invalid timestamps stream=0, pts=1620028, dts=1620029, size=29737
So it seems to be hanging on that "invalid timestamps" message. Is this an example of where something like QSF would help? I'm running kmttg on Linux so I don't have access to VideoRedo but I'm wondering if that's even the cause. Any ideas? Thanks!
 
#4,387 ·
Quick encoding question ...
So it seems to be hanging on that "invalid timestamps" message. Is this an example of where something like QSF would help? I'm running kmttg on Linux so I don't have access to VideoRedo but I'm wondering if that's even the cause. Any ideas? Thanks!
Yes, QSF most likely would fix the problem. Note that on Linux (or any platform) you can use ProjectX for QSF step, so no need for VRD as long as you don't mind losing captions.
 
#4,388 ·
Yes, QSF most likely would fix the problem. Note that on Linux (or any platform) you can use ProjectX for QSF step, so no need for VRD as long as you don't mind losing captions.
Cool, I'll give that a shot, thanks!

edit: that worked! I've added the QSF step to all my auto-transfers from CBS which should take care of the problem. Thanks again!
 
#4,389 ·
Anybody else with 20.2.2 software seeing some crazy Episode Numbers in kmttg NPL?

For example, "Two and a Half Men" recorded Thu 11/1 is giving me episode # 25849648.
I thought perhaps it was some weird bug in kmttg but checking the NPL XML from the TiVo that's exactly how it shows up there.
Then I thought perhaps shows with more than 9 seasons perhaps were screwed up with 20.2.2 but I see for example "Dancing with the Stars" which is season 15 with correct episode #s so that's not it. Perhaps it was just a glitch in guide listings for that particular "Two and a Half Men" episode...
 
#4,390 ·
Anybody else with 20.2.2 software seeing some crazy Episode Numbers in kmttg NPL?

For example, "Two and a Half Men" recorded Thu 11/1 is giving me episode # 25849648.
I thought perhaps it was some weird bug in kmttg but checking the NPL XML from the TiVo that's exactly how it shows up there.
Then I thought perhaps shows with more than 9 seasons perhaps were screwed up with 20.2.2 but I see for example "Dancing with the Stars" which is season 15 with correct episode #s so that's not it. Perhaps it was just a glitch in guide listings for that particular "Two and a Half Men" episode...
Well both my Premieres have 20.2.2 and I am seeing weird episode #s for "Two and a Half Men", but the two facts are not related. The most recent episodes show ep #s of 25849648, 25819482, and 25819479 but they were recorded on and remain on my THD.
 
#4,391 ·
Well both my Premieres have 20.2.2 and I am seeing weird episode #s for "Two and a Half Men", but the two facts are not related. The most recent episodes show ep #s of 25849648, 25819482, and 25819479 but they were recorded on and remain on my THD.
OK thanks, so I think that proves it's a guide issue of some sort, not related to 20.2.2.
 
#4,392 ·
v0p9a version just released.

This contains new RPC "Deleted" tab which allows you to see and recover shows from Recently Deleted folder.

Also adds a "Include History" boolean to the "Won't Record" tab that when enabled means past history will be included in the table so that one can explore why certain shows of interest did not record in the past.

Finally includes update discussed above to properly list record dates for shows transferred to TiVos running the new 20.2.2 software.

See release_notes for all the details.
 
#4,393 ·
So after receiving the latest tivo update, I'm getting the following error when trying to transfer within kmttg:

---DONE--- job='REMOTE NP List' TiVo=LivingRoom
>> DOWNLOADING D:\from_tivo\Shark Tank (11_02_2012).TiVo ...
C:\kmttg\curl\curl.exe --retry 3 --anyauth --globoff --user tivo:MAK --insecure --cookie-jar C:\Users\Fdisk\AppData\Local\Temp\cookie3583094416204174099.tmp --url http://192.168.1.109:80/download/Shark Tank.TiVo?Container=/NowPlaying&id=227259 --output "D:\from_tivo\Shark Tank (11_02_2012).TiVo"
D:\from_tivo\Shark Tank (11_02_2012).TiVo: size=0.00 MB elapsed=0:00:07 (0.00 Mbps)

Server Busy

Download failed to file: D:\from_tivo\Shark Tank (11_02_2012).TiVo
Exit code: 0
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 31 0 31 0 0 283 0 --:--:-- --:--:-- --:--:-- 283
0 31 0 31 0 0 283 0 --:--:-- --:--:-- --:--:-- 0
0 51 0 51 0 0 217 0 --:--:-- --:--:-- --:--:-- 217
0 51 0 51 0 0 217 0 --:--:-- --:--:-- --:--:-- 0
Warning: Transient problem: HTTP error Will retry in 1 seconds. 3 retries
Warning: left.
Throwing away 51 bytes

0 51 0 51 0 0 653 0 --:--:-- --:--:-- --:--:-- 653
0 51 0 51 0 0 653 0 --:--:-- --:--:-- --:--:-- 0
Warning: Transient problem: HTTP error Will retry in 2 seconds. 2 retries
Warning: left.
Throwing away 51 bytes

0 51 0 51 0 0 544 0 --:--:-- --:--:-- --:--:-- 544
0 51 0 51 0 0 544 0 --:--:-- --:--:-- --:--:-- 0
Warning: Transient problem: HTTP error Will retry in 4 seconds. 1 retries
Warning: left.
Throwing away 51 bytes

0 51 0 51 0 0 544 0 --:--:-- --:--:-- --:--:-- 544
0 51 0 51 0 0 544 0 --:--:-- --:--:-- --:--:-- 0
Shark Tank (11_02_2012).TiVo: Download attempt # 2 scheduled in 10 seconds.
No rows selected
No rows selected

Any ideas?
 
Top