TiVo Community Forum banner
  • TiVoCommunity.com Ambassador Program Now Open! >>> Click Here

New program for 1 step TTG downloads, decryption, encoding - kmttg

2M views 13K replies 921 participants last post by  mattack 
#1 ·
EDIT: This program has evolved a lot since this 1st post and now is written in Java and much easier to install than before (very easy on Windows and Mac OSX)... See http://sourceforge.net/projects/kmttg/ for details or visit the end of this thread for more up to date information.

kmttg is a Perl/Tk program I wrote to facilitate TivoToGo (TTG) transfers that can download, create pyTivo metadata, decrypt, run comskip & comcut (commercial detection and removal) and re-encode multiple shows you select from your Tivos all in 1 step.
You can select one or more shows at a time and then with one click of a button the program will download all the selected items, with the options of also automatically creating a metadata file for pyTivo, decrypting .TiVo files to .mpg, running comskip (commercial detection and removal program), and automatically re-encoding to a more portable format using mencoder, ffmpeg or any other command line encoder of your choosing. The program queues up multiple jobs and displays time, size and speed statistics for ongoing jobs.

For more information, screenshots and download visit:
http://sourceforge.net/projects/kmttg

Web page contains windows executables for all 3rd party tools used. The only other requirement of course is you must have Perl installed. Runs under Windows or Linux - tested with WinXP SP2 & Linux Red Hat Enterprise 4.

If you do try this out would appreciate some comments/feedback. For any programmers out there feel free to tinker and make improvements yourself.
 
See less See more
#3,220 ·
I'm trying a bunch of things to find a workable, automated, linux-only solution. I'm willing to give up on comcut, but I don't want VideoReDo (Windows).

I downloaded 4 episodes of a show. The mpeg 2 files are all about 3.5G and play fine in VLC. I tried 4 encodings and checked them in VLC.

  • 'hb_tivo_hd', 1.27G, In sync in VLC
  • 'ff_h264_high_rate', 1.02G, Won't play in VLC
  • 'ff_h264_med_rate', 540M, Out of sync in VLC
  • 'ff_tivo_hd', 1.89G, Out of sync in VLC

Ok, I thought. The HandBrake encoding is in sync!

Then I PUSHED it to the TiVo with pyTivo, where it was badly out of sync.
Then I PULLED it to the TiVo with pyTivo (ffmpeg transcode to mpeg2), where it was in sync. Also, it transcoded/transferred slower than anything I've ever seen. (And I use pyTivo a lot.)

Anyone have a linux solution?
Anyone have the technical details of "the timestamp problem"?

Thanks,
tlc
 
#3,221 ·
Hello I have been lurking for a while now, and just want to say thanks for all the help. And that I am loving kmttg.

Second, is there a way to get around the ten post limit, for viewing links and sending PM's? I find most of my answers through searches and don't like to post on any board unless its necessary, so the ten post requirement seems weirdly counterproductive and high.

Thanks

Bill
 
#3,223 ·
Greetings!

I'm having trouble auto-transferring news programs. First off, I'm using XBMC as my primary video device having relegated the TiVo with the lifetime subscription to do all of my cable recording, then using kmttg to move the contents over to XBMC where I have much more control over the experience.

I have three auto-transfer lines, one that gets everything, .*, and one each for the two news shows I want recorded and do have "Treat each recording as unique" selected for those two. The title keyword match hits, but then nothing happens. It's been set up this way for a couple of weeks. Everything else is coming through fine.





Any ideas?

2011_07_08_12:26:44 Title keyword match: '.*' found in 'fox 40 news at 6'
2011_07_08_12:26:44 (ProgramId=SH011785410000 already processed => will not download)
2011_07_08_12:26:44 Title keyword match: 'fox 40 news at 6' found in 'fox 40 news at 6'
2011_07_08_12:26:44 Title keyword match: '.*' found in 'indoors out'
 
#3,224 ·
Not enough details given on the matching portion of the log, but from the one line shown the .* match didn't download because ProgramId is already in auto.history file. News shows typically don't have unique ProgramId's, so you have to use the "Treat each recording as unique" setting to get them.
 
#3,225 ·
The first two lines are from the first Title keyword match. I understand it skips because it has the ProgramId in the auto.history file. The third line hits the on the Title keyword match which DOES have "Treat each recording as unique" selected. It doesn't follow with an "already processed" but it doesn't download, either. It just moves on to the next match.

2011_07_08_12:26:44 Title keyword match: '.*' found in 'fox 40 news at 6'
2011_07_08_12:26:44 (ProgramId=SH011785410000 already processed => will not download)
2011_07_08_12:26:44 Title keyword match: 'fox 40 news at 6' found in 'fox 40 news at 6'
2011_07_08_12:26:44 Title keyword match: '.*' found in 'indoors out'

Is there some other portion of the log file that would be more helpful?
 
#3,226 ·
Hey everyone, I recently decided to drop DirecTV and bought a Tivo Premiere to record OTA HD. I am using kmttg to grab and decrypt recordings from the TiVo and would now like to convert them for use on the Apple TV. My usual handbrake profile will convert the mpg files to m4vs, and they look really nice, but a 1 hour 720p show is taking ~5 hours to complete. This is the same profile I have been using to convert DVD rips for use on my AppleTV.

I do not care about compressing the file size at all 4.57GB can stay 4.57 GB, I'm not going to keep these for archival purposes, I just want to be able to watch something I record upstairs on the TiVo downstairs on my AppleTV2 and have the quality nearly the same, or acceptable when viewed on a projector. The main thing I am looking for here is speed and quality vs size. Can someone recommend what settings I would use to accomplish this, or if handbrake is not the tool to accomplish this is there a better way to perform a fast, nearly lossless, MPG2 to M4V conversion?

Thanks,
 
#3,227 ·
The first two lines are from the first Title keyword match. I understand it skips because it has the ProgramId in the auto.history file. The third line hits the on the Title keyword match which DOES have "Treat each recording as unique" selected. It doesn't follow with an "already processed" but it doesn't download, either. It just moves on to the next match.

2011_07_08_12:26:44 Title keyword match: '.*' found in 'fox 40 news at 6'
2011_07_08_12:26:44 (ProgramId=SH011785410000 already processed => will not download)
2011_07_08_12:26:44 Title keyword match: 'fox 40 news at 6' found in 'fox 40 news at 6'
2011_07_08_12:26:44 Title keyword match: '.*' found in 'indoors out'

Is there some other portion of the log file that would be more helpful?
I would guess that what is happening is since the ".*" matches the program and is rejected, then that show is put in reject bucket (not to process) for any subsequent matches. To test that you can temporarily disable the ".*" auto transfer and then fire up auto transfers in GUI mode to see what happens.
 
#3,228 ·
Can someone recommend what settings I would use to accomplish this, or if handbrake is not the tool to accomplish this is there a better way to perform a fast, nearly lossless, MPG2 to M4V conversion?

Thanks,
I use kmttg to convert the .tivo file to .mpg file that is saved onto a NAS.

My computer uses Intel Q6600 cpu and Nvidia GT240 gpu.

Source file spec - 1920x1080 resolution, 59.94 frame rate

I use the Apple TV profile because my cellphone has 960x540 native resolution.

Conversion to Apple TV spec via Handbrake runs ~22fps, 100% cpu. Handbrake *estimates* this 1 hour HD show will take 80 min to finish. I didn't actually do the transcode because I wanted to continue using the computer.

Conversion to Apple TV spec via Badaboom2 ran 33fps, 35% cpu. It took 46 min to finish.

Transcoded file looks acceptable on cellphone, netbook, and 65 inch HDTV.
 
#3,229 ·
I'm seeing 0p8c hang when I try to cancel a download. I ran into it twice today. Since it's java, simply sampling it won't be useful, right? Is there a way to get useful java-y info about where it's hung?

This is 0p8c as I said, on a Mac, on 10.5.8. (yeah yeah, upgrade)

whatever version I was using before, I never had a problem of cancelling a download hanging.
 
#3,230 ·
I'm seeing 0p8c hang when I try to cancel a download. I ran into it twice today. Since it's java, simply sampling it won't be useful, right? Is there a way to get useful java-y info about where it's hung?

This is 0p8c as I said, on a Mac, on 10.5.8. (yeah yeah, upgrade)

whatever version I was using before, I never had a problem of cancelling a download hanging.
Nothing has changed with code to kill jobs in a long time so I doubt it's anything different in 0p8c causing issue. What download method are you using? curl or java? Are you combining download & decrypt as one? If using curl then you can use try using command line "kill" to terminate the curl job. (Not familiar with Mac but since it's unix like you can use "ps" command line to find the curl pid and then "kill pid" or "kill -9 pid" to kill it). You can also use "top" to find running curl command and kill it. Since it's Mac there's likely some GUI to kill jobs available as well I would guess...
 
#3,231 ·
I'm seeing 0p8c hang when I try to cancel a download. I ran into it twice today. Since it's java, simply sampling it won't be useful, right? Is there a way to get useful java-y info about where it's hung?

This is 0p8c as I said, on a Mac, on 10.5.8. (yeah yeah, upgrade)

whatever version I was using before, I never had a problem of cancelling a download hanging.
I don't have any problems canceling jobs on 0p8c on Win XP. Is your Java up to date?
 
#3,232 ·
v0p8d version just released. Highlight of the release is a new Season Premieres tab in the iPad remote control GUI. This is a utility to search for new upcoming season & series premieres. There is also option to schedule season passes for resulting entries displayed in the table.

Other enhancement of note is addition of Record button in the Will Not Record tab so that you can schedule to record entries on selected TiVos. Useful if you have more than 1 Premiere - you can now get list of entries that will not record on 1 Premiere and optionally schedule to record selected entries on another Premiere.

Consult release_notes Wiki page for all the details of changes in this new release.

Consult remote_control Wiki page for details on the new Season Premieres tab of the remote.
 
#3,233 ·
I am having some issues lately with some of my actual TiVo recordings and I am trying to narrow down if it is signal related or if the drive might be going.

I recorded Franklin & Bash and when my friend went to watch it, it was all pixelated and blocky. I figured I would see if KMTTG could transfer it and also to see if it played fine on my PC though I don't know why.

It transferred 30 minutes of the hour show and decrypted what transferred. Here is the odd thing. The TiVo recording is really broken up and unviewable. The stripped MPG plays fine without issues. Any ideas?

I figured if the .TiVo file had issues the .MPG would also, but was surprised when it didn't.
 
#3,234 ·
I am having some issues lately with some of my actual TiVo recordings and I am trying to narrow down if it is signal related or if the drive might be going.

I recorded Franklin & Bash and when my friend went to watch it, it was all pixelated and blocky. I figured I would see if KMTTG could transfer it and also to see if it played fine on my PC though I don't know why.

It transferred 30 minutes of the hour show and decrypted what transferred. Here is the odd thing. The TiVo recording is really broken up and unviewable. The stripped MPG plays fine without issues. Any ideas?

I figured if the .TiVo file had issues the .MPG would also, but was surprised when it didn't.
Very interesting. Is the TiVo running hot? Memory problems would probably cause more catastrophic strangeness. It could be a failure of the decoder hardware in the TiVo. Or maybe the TiVo is unable to stream the data from the drive to the decoder fast enough. Possible reasons could be drive problems (inadequate throughput? re-seeks?), excessive file fragmentation (should be unlikely with DVR file systems), insufficient hardware buffering (assumes fragmentation; could be tested by cloning to a drive with more cache), and ... what else?
 
#3,235 ·
v0p8e version just released. Highlights include addition of full virtual remote with buttons and keyboard support in addition to the advanced navigation buttons and some "combo" buttons:


Another highlight is ability to re-prioritize Season Passes.
Also new configuration option under *Tivos* tab to not initiate download of NPL lists automatically for each TiVo on kmttg startup.

See release_notes for details and consult remote_control Wiki for updated documentation on the iPad remote control.

NOTE: If upgrading from a previous version of kmttg note that this release contains a new "rc_images" folder as part of zip file which must be in same folder as kmttg.jar. The icons in that folder are used by the updated Remote Control.

Special thanks to "innocentfreak" for help testing this version.
 
#3,237 ·
This may not belong but It the closes I believe. I am using kmttg latest build. I have it set to automatically download programs using videoredo quickstream fix. It downloads the tivo programs ok but when the QS is done the program size is 0. what could be causing this?
What version of VRD are you using? Does it happen for every show or just particular ones? If you have the "Enable VideoRedo QS Fix video dimension filter" option enabled what is kmttg reporting as the video dimension it is giving VRD to use? If you bring up VRD GUI manually and run QS Fix with video filter enabled is the issue still there?
 
#3,238 ·
I just started using this application yesterday and I am very pleased with what it allows me to do. It saves me many hours trying to manage the programs I have amassed. The GUI is very good and I am deep in decoding and cutting commercials out of my movie library.

I've only found had one problem and it is likely I missed something in the setup. I am running Windows 7 and I downloaded kmttg_v0p8e.zip 10 hours after you posted it. The problem is in setting up the auto transfer. It works fine when I do a dry run in GUI but when I try to install the service, I get a path not specified. Which is correct as I do not have a service\win32 folder in the Temp1_kmttg_v0p8e.zip directory (where Windows installed the application).

Thinking it might be an oversite on the new release (I apologize in advance for thinking this was possible), I downloaded version d and found the same problem. I even did a search on *.bat for all my drives and none of the service.bat files you list are on my machine.

I found no reference to this on the last 5 forum pages, so this must be something I did wrong. What am I missing?
 
#3,239 ·
I just started using this application yesterday and I am very pleased with what it allows me to do. It saves me many hours trying to manage the programs I have amassed. The GUI is very good and I am deep in decoding and cutting commercials out of my movie library.

I've only found had one problem and it is likely I missed something in the setup. I am running Windows 7 and I downloaded kmttg_v0p8e.zip 10 hours after you posted it. The problem is in setting up the auto transfer. It works fine when I do a dry run in GUI but when I try to install the service, I get a path not specified. Which is correct as I do not have a service\win32 folder in the Temp1_kmttg_v0p8e.zip directory (where Windows installed the application).

Thinking it might be an oversite on the new release (I apologize in advance for thinking this was possible), I downloaded version d and found the same problem. I even did a search on *.bat for all my drives and none of the service.bat files you list are on my machine.

I found no reference to this on the last 5 forum pages, so this must be something I did wrong. What am I missing?
I just downloaded kmttg_v0p8e.zip and unzipped it to a fresh folder and I verified that "service" folder at same level as kmttg.jar with "conf" and "win32" sub-folders are extracted out of it. There are also other required folders at same level as kmttg.jar namely "VRDscripts", "rc_images" and "encode". If that's not what you are seeing perhaps use a different unzip program.
NOTE: It almost sounds like perhaps you are running kmttg.jar directly from within the zip file or something? If that is so don't do that. You should unzip everything to a new folder and the double-click on kmttg.jar to run it.
 
#3,240 ·
What version of VRD are you using? Does it happen for every show or just particular ones? If you have the "Enable VideoRedo QS Fix video dimension filter" option enabled what is kmttg reporting as the video dimension it is giving VRD to use? If you bring up VRD GUI manually and run QS Fix with video filter enabled is the issue still there?
The day before 1 out of 5 did not QF. Last night 3 out 3 did not QF. I am using vrdtvsh264-4-20-6-691a. Do not know what you mean by what is kmttg reporting as the video dimension it is giving VRD to use. how do i check that?
did not try the manual gui will try now.
 
#3,241 ·
Do not know what you mean by what is kmttg reporting as the video dimension it is giving VRD to use. how do i check that?
did not try the manual gui will try now.
When kmttg starts the qsfix job and you have that option enabled kmttg will report what video dimensions were detected and passed on to VRD with a message such as:
VideoRedo video dimensions filter set to: x=1920, y=1080
If you don't have that option enabled in kmttg then you won't see such a message.
 
#3,242 ·
I initially downloaded the zip file to a local drive (P:) and used Windows Explorer to view the contents of the zip file. I double clicked on the kmttg.jar file in this view and windows set up a directory on the C: drive under my user name. It was from this directory that I have been opening the application. It does not have the Service directory within it.

Just now, I copied the zip file to another directory on the P: drive and used Windows Explorer to extract the contents. W/E extracted the files in the same directory on the P: drive as contained the zip file. This directory does contain the service directory.

It was necessary to go to the win32 subdirectory and install the service as administrator and it is still necessary to start and stop the service in the same manner, but at least now auto transfer is running.

I was my error in allowing W/E to set up the application without performing the extract first. Thank you for your assistance.

Just to verify the process, I set the interval for 3 mins. The 1st recorder has only 1 - 30 minute program and the 2nd had several 90 min programs. The autolog shows both recorders were queried initially and transfers were stated on both. The 1st recorder finished quickly.

Subsequent queries have been for the 1st recorder only. Is this because when the 2nd recorder is actively transferring programs it is not queried?
 
Top