1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New program for 1 step TTG downloads, decryption, encoding - kmttg

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Mar 15, 2008.

  1. Apr 2, 2009 #541 of 10412
    AudioNutz

    AudioNutz New Member

    318
    0
    Nov 9, 2008
    I've been using the KMTTG on my Mac for several months, and I've recently decided to start using the "Auto Transfer" feature.

    When using this feature, I've noticed that it leaves the mp4 extension off the final encoded files. It leaves the period, but truncates the mp4. "Lie to Me." This ONLY happens with Auto Transfers, and not with shows that I manually download.

    Has anyone seen this? Are there any hints on solving this?
     
  2. Apr 2, 2009 #542 of 10412
    AudioNutz

    AudioNutz New Member

    318
    0
    Nov 9, 2008
    It seems that someone else is seeing this error... Hmmm... I'm running in GUI mode though.

    EDIT: I'm also getting missing information in the file name of the metadata. Instead of "Lie to Me.mp4.txt" it's "Lie to Me..txt".
     
  3. Apr 2, 2009 #543 of 10412
    moyekj

    moyekj Well-Known Member

    11,141
    31
    Jan 23, 2006
    Mission...
    AudioNutz, which version are you using? If you switched to 0.5k and are using a built in encoding profile did you update the auto configuration to fix the encode names? (names changed in 0.5k)
     
  4. Apr 2, 2009 #544 of 10412
    AudioNutz

    AudioNutz New Member

    318
    0
    Nov 9, 2008
    First, I was using 0.5h when I first noticed this issue. I thought I would be clever, and I upgraded to 0.5k, but I observed the same problem. I too assumed that an update might/could help. However, since it did not, I reverted back to 0.5h.

    Second, since I'm using a different encode profile than any of your provided profiles, I didn't need to update the auto configuration, but it certainly was one of the things that I checked, since it made the most sense.

    THEN... I started poking around, and remembered that kmttg was VERY particular about the names of the encode profiles. (No spaces) I also noticed that all of your provided profiles are all lower case. Upper vs. Lower case has not been a problem in the past. I have always used a profile that had mixed upper/lower case in it, but... I thought I would switch to all lower-case in the encode profile name, and give it a shot.

    This has helped on one computer, but not the other.

    When I thought I had it solved on both, so I switched to the "Background" option instead of the GUI. (The "Run in GUI" doesn't seem to look for new shows at the "Check TiVos at Interval" unless you actually force it to by clicking "Run in GUI". This defeats the purpose of automation)

    So... I've got one working, and one with a background process that's jammed hard. Since I have no idea what the (UNIX) "Process Name" is to kill it, I'm waiting for an opportunity for a Re-Start this particular machine. Since this machine is the file server for my entire house I'll need to wait until the 1st machine is done with its processing.

    Questions:
    1. You wouldn't happen to know the name of the process that these background auto transfers use, would you?
    2. You wouldn't happen to know how to make "Run in GUI" actually check at the 10min interval, or whatever you set it to?
     
  5. Apr 2, 2009 #545 of 10412
    AudioNutz

    AudioNutz New Member

    318
    0
    Nov 9, 2008
    qtip96: Does the name of your encoding profile have capital letters in it?
     
  6. Apr 2, 2009 #546 of 10412
    moyekj

    moyekj Well-Known Member

    11,141
    31
    Jan 23, 2006
    Mission...
    Spaces in profile names would definitely be a problem. Upper or lower or mixed case should not matter at all, but if you do that you have to be careful that the exact case is consistent wherever you use it. (i.e. case sensitivity does matter in perl). Ultimately the file that matters for auto transfer configuration is the "auto.ini" file. So you should double check the names saved in that file.
    To your other questions:
    1. Background auto transfers run as "perl kmttg.pl -a". You shoud be able to kill it by starting kmttg in GUI mode on the appropriate machine and selecting Auto Transfers->Background Job->Disable
    2. Auto Transfers -> Run in GUI was a request by a user a while back to just have an easy way to test auto transfer setup while in GUI (which is easier than debugging problems in background mode). There really is not an easy way to set it to loop at user specified interval in GUI mode and that could be confusing/annoying if you are trying to do something else with the GUI in the interim. So while conceivably it would be possible I'm not inclined to do it.

    If there is an issue with background auto transfers then that really needs to be debugged and fixed. However since you mention it seems to be working on one machine and not another I would lean towards a config issue of some sort on the one that's not working.
     
  7. Apr 2, 2009 #547 of 10412
    AudioNutz

    AudioNutz New Member

    318
    0
    Nov 9, 2008
    This makes total sense now. If you look at the auto.ini file that is generated by the GUI, I notice that it always generates everything in lower case, EVEN IF you type it in upper/lower, and EVEN IF you have your profile named with upper/lower. I'll just switch to all lower case, and it solves this issue. Otherwise, I would have had to edit the auto.ini file manually every time I edited it, to put the upper case characters back in... Not worth it...



    Yeah, I tried that first. No luck. I can "Enable" the background processes to begin with the GUI menu option, but I can't stop them that way. One other thing: If I have the background processes going, and I select the menu option for "Enable" once again, it tried to begin the automatic transfers again, gives me errors about too many downloads already happening etc... And then it completely HOSES everything. I think this is why I needed the re-start, however knowing the name of the process that I need to kill might have helped me out of this without the re-start. As you know, this MAY be due to the fact that I'm running this in the Mac environment. Maybe we've found the ONE thing that doesn't work on a Mac. :)


    Gotcha! I agree that it does help with the de-bugging. I'm switching to the full background mode as soon as I can get the little bugs worked out.


    After my re-start, I noticed that I had one machine running 0.5h and the other on 0.5k. (The 0.5h was the server, which was the one that was not working) I switched the server to 0.5k, and then did a test run in GUI mode after the restart. This seemed to work fine, so I killed all of these, and switched to background mode. It seems to be working correctly for now, but I'm going to watch it closely for a few days.

    I'm betting that either 1) I hosed myself by "Enabling" the background process twice. or 2) There's something in 0.5k that is helping this process work better than 0.5h.

    Hmmm... I see "perl" as a running process on my Macs, but not "perl kmttg.pl -a". The next time I get hosed up I'll try killing this process. I did try this when I was hosed up earlier, but there were three of them running. (I'm guessing because of my multiple "Enable") Killing all three of these completely locked up the machine.

    Or... Maybe I'll try a "perl kmttg.pl -x" argument in an X11 or Terminal window...
     
  8. Apr 2, 2009 #548 of 10412
    moyekj

    moyekj Well-Known Member

    11,141
    31
    Jan 23, 2006
    Mission...
    Don't know how you are checking for running processes but depending how you do it you will only see the main executable. So "perl" is the main executable and "kmttg.pl -a" are the arguments to it.
    On unix systems "ps -ef" or for other flavors "ps -auxww" show you processes plus command line arguments, don't know about Macs specifically.
    (NOTE: Probably a typo but -x argument doesn't mean anything to kmttg. I think you meant -a right?)
     
  9. Apr 2, 2009 #549 of 10412
    AudioNutz

    AudioNutz New Member

    318
    0
    Nov 9, 2008
    I usually use the "Activity Monitor". It looks similar to the "Task Manager" on a PeeeCeee if you look at the "Processes" tab and the "Performance" tab at the same time.

    [​IMG]



    I'll try that. I'll admit that like most Mac users I'm like the guy on the right in the Apple commercials, and I just sorta expect stuff to work, without getting into the command-line interface too much.


    Actually, no, I meant that. I thought that -a was the command line to start the process, and I looked in your perl application, and I noticed where a -x argument was meaning "Exit". I put one and one together and arrived at that assumption. (I got this working on my Macs by doing a bunch of assumptions.)

    Sooo... I'll ask...
    Is the -a argument a switch to turn the automatic processing on and off?

    By the way.. THANK YOU for building this wonderful tool. I don't know how I lived without it. (I was using VisualHub before)
     
  10. Apr 2, 2009 #550 of 10412
    MikeAndrews

    MikeAndrews Registered abuser

    14,222
    1
    Jan 17, 2002
    Northern...
    I use top, but them I spoke Unix shell long before I bought a Mac - because it had a Unix shell I coudl put to good use if I hated everything else. I didn't hate everything else - well I hate some stuff like some aspects of networking setup ....

    I never got GUI-impaired. :D:D
     
  11. Apr 2, 2009 #551 of 10412
    MikeAndrews

    MikeAndrews Registered abuser

    14,222
    1
    Jan 17, 2002
    Northern...
    I'm putting shows in thier own directory which works great, execept it screws up when I have shows on auto-transfer. It will auto transfer existing shows if they're not duplicated in the default directory.

    On windows, I tried putting shortcuts but that didn't work. On a Mac, Unix, or Linux it could be solved with a hard or soft link in the base directory.

    I could put in an empty file named the same, but those would make the a player go boom if I ever tried to play the wrong version.

    How about support for a unique destination folder on each auto-transfer?
     
  12. Apr 2, 2009 #552 of 10412
    moyekj

    moyekj Well-Known Member

    11,141
    31
    Jan 23, 2006
    Mission...
    AudioNutz, the only command line arguments supported are -h, -b, -a, -d
    -h => Print help and exit
    -b => Batch mode (no GUI) and perform a single auto download loop and then exit
    -a => Batch mode (no GUI) and perform repeated auto downloads in a loop (using interval specified by check_tivos_interval in auto.ini) without exiting
    -d => Spit out some verbose debugging information
     
  13. Apr 2, 2009 #553 of 10412
    moyekj

    moyekj Well-Known Member

    11,141
    31
    Jan 23, 2006
    Mission...
    Are you running auto transfer from GUI or in batch mode? In batch mode each time a successful auto download happens kmttg is supposed to add an entry to auto.history file with programId of the program to prevent it from auto processing the same show over and over. If running from auto transfer from GUI, then the auto.history file is only updated if it exists in the first place. So if you are always running from GUI then just create an empty auto.history file and then that file will get updated as you auto download.
     
  14. Apr 2, 2009 #554 of 10412
    AudioNutz

    AudioNutz New Member

    318
    0
    Nov 9, 2008
    Got it. That's why I can't kill the process unless I do it in the Activity Monitor. Now that I've got things sorted, it does allow me to quit these processes without re-starting the machine.
     
  15. Apr 3, 2009 #555 of 10412
    AudioNutz

    AudioNutz New Member

    318
    0
    Nov 9, 2008
    After running in background mode for a couple of days now, I now see why I wanted to run it in the GUI. It sure would be nice to have the ability to actually see what's being processed. What if the GUI had the ability to monitor what was happening in the background auto processing? ;)


    It might not help me in my particular case, since I'm running KMTTG on two different machines, so I'm thinking of writing a little Xcode application to monitor the log files on the two machines, (One for each TiVo) and give me a status report in a little application window on my laptop (or SmartPhone) or any other machine in the house.

    Then I would never need to look at the screens of these two machines via ARD... ;)
     
  16. Apr 6, 2009 #556 of 10412
    fyodor

    fyodor Member

    523
    0
    Sep 19, 2006
    Hi

    I was wondering what type of encode times people were seeing with handbrake. I've found that it takes at least 5 times as long as FFMPEG for comparable bitrates. I've been trying handbrake encodes using the parameters listed on the streambaby compatibility page. Converting a three minute movie clip takes at least forty minutes on a 2.5 ghz pentium dual core processor. Is this typical?
     
  17. Apr 6, 2009 #557 of 10412
    moyekj

    moyekj Well-Known Member

    11,141
    31
    Jan 23, 2006
    Mission...
    fyodor that hanbrakeCLI recipe that you see in that wiki page I posted there yesterday after some feedback in streambaby thread. Two things with that recipe that make it slower to begin with:
    * It's setup as 2 pass encoding (-2 argument)
    * It will re-encode audio (though that should not be that much overhead)
    Note that immediately above the handbrake recipe given in the wiki page is a roughly equivalent (for video) ffmpeg recipe that is 1 pass and simply copies audio, so that should run much quicker and give comparable results. I think in general ffmpeg is a faster encoder than handbrakeCLI so I would tend to stick with it over handbrake if you have a choice.
     
  18. Apr 6, 2009 #558 of 10412
    Yoav

    Yoav New Member

    1,048
    0
    Jan 12, 2007
    I just want to correct this meme that is being circulated and repeated.

    ffmpeg is using the EXACT same code for encoding h.264 (which is what we're talking about). It's libx264 from the videolan project (the guys who make vlc). Therefore, unless it's compiled with better optimization options, it will not run any faster or slower.

    IF you're seeing differences in speed between mencoder, handbrake and ffmpeg, it's because you're using different options to the encoding (like 2-pass, larger blocks, more reference frames, etc -- all those have a HUGE effect on the time). Or it is because 'reading, rescaling, anti-aliasing, changing frame-rate, etc' is done. Each of these programs use a different method to achieve those (although none of those account for major performance differences).

    In the old days, ffmpeg had a faster mpeg-2 and mpeg-4 encoder, until projects like mplayer switched to using ffmpeg's encoder (Handbrake doesn't support those encoding outputs). That's when the general opinion that 'ffmpeg is faster at everything' started.

    ffmpeg can also be faster because people are accidentally using the 'mpeg-4' encoder instead of h.264 encoder, and thinking that it's the same thing (it's a lower-quality encoder that also spits out an .mp4 file, so people get confused).

    Ok rant over :) sorry...
     
  19. Apr 6, 2009 #559 of 10412
    fyodor

    fyodor Member

    523
    0
    Sep 19, 2006
    I guess that I've generally not been thrilled with the video quality I've gotten with FFMPEG. The handbrake encodes (and the Mencoder encodes that I did back in my SageTV days) seemed to deliver much better quality for the same bitrate. That being said, it's been a long time since I've use FFMPEG, so I'll give the command lines you suggested a shot. Thanks for the advice.

     
  20. Apr 6, 2009 #560 of 10412
    moyekj

    moyekj Well-Known Member

    11,141
    31
    Jan 23, 2006
    Mission...
    As Yoav mentioned ultimately ffmpeg, mencoder & handbrakeCLI all use libx264 for encoding, so given the same encoding setup you should get comparable results for all of them. The difficulty of course is since each uses different command line options it's sometimes difficult to setup identical encodings so that you can compare encoding time, quality, etc. From the wiki post the ffmpeg command you would setup in kmttg .enc file is:
    Code:
    FFMPEG -y -i INPUT -vcodec libx264 -b 2000k -refs 3 -flags2 +mixed_refs+wpred+bpyramid+dct8x8-fastpskip -bf 3 -me_method umh -subq 9 -g 24 -keyint_min 2 -acodec copy -f mp4 OUTPUT
    The roughly equivalent one using handbrake is (except this re-encodes audio as well where ffmpeg is doing an audio copy):
    Code:
    HANDBRAKE -i INPUT -f mp4 -O -p -e x264 -b 2000 -a 1 -E ac3 -B 160 -R 0 -6 dpl2 -D 1 -s 1 -x ref=3:mixed-refs=1:bframes=3:weightb=1:direct=auto:b-pyramid=1:me=umh:subq=9:analyse=all:8x8dct=1:no-fast-pskip=1:psy-rd=1,0.2:keyint=24:min-keyint=2:no-dct-decimate=1 -v -o OUTPUT
    (And of course in both cases you can adjust -b argument to control desired video bit rate)
     

Share This Page