1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New program for 1 step TTG downloads, decryption, encoding - kmttg

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Mar 15, 2008.

  1. Jan 9, 2010 #1741 of 10616
    moyekj

    moyekj Well-Known Member

    11,267
    81
    Jan 23, 2006
    Mission...
    In SVN I changed timeout from 10 to 60 seconds so next release will get that change (of course you can build your own kmttg.jar from SVN as well if you want change right away).
     
  2. Jan 12, 2010 #1742 of 10616
    moyekj

    moyekj Well-Known Member

    11,267
    81
    Jan 23, 2006
    Mission...
    In next version I'm planning on upgrading to newer version of ffmpeg. Unfortunately that means x264 encoding profiles & those with aac audio all have to be updated. I've got everything working on windows but since I don't have Mac to test with I need help from any Mac users to ensure things are still working. So if you have a Mac and would like to help out please visit this issue and give the beta version a shot:
    http://code.google.com/p/kmttg/issues/detail?id=79
    (NOTE: Please install in a fresh/new directory and let kmttg download the helper tools). I'm not sure if the ffmpeg in there is universal or PPC which is one of the things I need feedback on.
     
  3. Jan 13, 2010 #1743 of 10616
    luvmytivo

    luvmytivo New Member

    15
    0
    Nov 13, 2000
    Burbank,...
    I'm running 7g on a mac, and for some reason the Atomic Parsley step is not running on my auto jobs. The .m4v.txt file is sitting there right next to the .m4v but according to my logs it's just download, decrypt, encode, and move on - no AP calls at all.

    I have double checked the AP location in the configuration, and it points to a working copy of AP (the one that came with the kmttg download), so any other troubleshooting steps I can try out? I've got 80 files downloaded that are in the process of decrypting and encoding in an auto job, and I'm gonna have to stop it and start over...
     
  4. Jan 13, 2010 #1744 of 10616
    moyekj

    moyekj Well-Known Member

    11,267
    81
    Jan 23, 2006
    Mission...
    Are the encoding jobs finishing normally? i.e. Do you see the following message in the log files:
    Code:
    encoding job completed: ...
    ---DONE--- job=encode output=file.m4v
    
    Other than that as long as AP is properly configured and both file.m4v & file.m4v.txt exist it should be scheduling the job. Here's the relevant code to determine if to schedule AP or not:
    Code:
                
    
                // Schedule an AtomicParsley job if relevant
                if (file.isFile(config.AtomicParsley)) {
                   job.metaFile = job.encodeFile + ".txt";
                   if ( ! file.isFile(job.metaFile) ) {
                      job.metaFile = job.mpegFile_cut + ".txt";
                   }
                   if ( ! file.isFile(job.metaFile) ) {
                      job.metaFile = job.mpegFile + ".txt";
                   }
                   if ( file.isFile(job.metaFile) &&
                        (job.encodeFile.toLowerCase().endsWith(".mp4") ||
                         job.encodeFile.toLowerCase().endsWith(".m4v")) ) {
                      // Schedule AP job
    ...
    
     
  5. Jan 13, 2010 #1745 of 10616
    luvmytivo

    luvmytivo New Member

    15
    0
    Nov 13, 2000
    Burbank,...
     
  6. Jan 13, 2010 #1746 of 10616
    luvmytivo

    luvmytivo New Member

    15
    0
    Nov 13, 2000
    Burbank,...
     
  7. Jan 13, 2010 #1747 of 10616
    moyekj

    moyekj Well-Known Member

    11,267
    81
    Jan 23, 2006
    Mission...
    That's very odd because it's the exact same code being called whether you do it in GUI or auto mode...
     
  8. Jan 14, 2010 #1748 of 10616
    dwalzer

    dwalzer New Member

    6
    0
    Jan 14, 2010
    Hi, I am running kmttg v0p7g on an old laptop (Win XP SP3, Pentium 1.6GHz, 512MB RAM). It's slow but I don't care about the speed. It connects to a Tivo HD over my home network. The issue is that it does not complete downloading the entire .tivo files from the Tivo, even though I don't get any error messages. For example, this is from the kmttg.log file:

    Code:
    How I Met Your Mother - The Pineapple Incident
    Recorded Mon 12/21/2009 01:30 AM on 140=LIFE, Duration = 20 mins
    Ted relies on his friends to fill in the gaps left in his memory by a night of partying. 
    C:\Documents and Settings\dwalzer\Desktop\kmttg\tivo_output\How I Met Your Mother - The Pineapple Incident (12_21_2009).TiVo: size=727.54 MB elapsed=00:17:30 (5.81 Mbps)
    ---DONE--- job=download output=C:\Documents and Settings\dwalzer\Desktop\kmttg\tivo_output\How I Met Your Mother - The Pineapple Incident (12_21_2009).TiVo
    The file in question is a 20-minute clip, but the file size on the Tivo is about 1GB (according to kmttg). However, it only downloads a partial file -- in this case, 727MB. I've done repeated tests; the percentage downloaded varies and seems random. Following this, it seems to decrypt and re-encode the file normally, but obviously only on the partial file it downloaded. Any ideas why it is doing this? Thanks so much for any help!
     
  9. Jan 14, 2010 #1749 of 10616
    moyekj

    moyekj Well-Known Member

    11,267
    81
    Jan 23, 2006
    Mission...
    This seems a little odd:
    Recorded Mon 12/21/2009 01:30 AM on 140=LIFE, Duration = 20 mins
    i.e. On the TiVo itself it appears to be a 20 minute recording. If there are commercials it should be 30 minutes. Did you check the recording on the TiVo itself to see if it is complete there? Could be that it is intentionally a partial 20 minute recording in which case see below.

    What does Mediainfo report as the total time for the kmttg downloaded TiVo file?

    A couple of notes on file sizes reported by TiVo:
    1. The sizes are those as stored on TiVo. When it transfers to a computer it remuxes and re-encrypts (CPU intensive and hence why transfers are slow) and file sizes always decrease. Since it's not predictable how much the file sizes decrease during transfers kmttg will use the TiVo reported file size and as a result the transfer will always complete before 100% progress indication.
    2. File sizes for partial recordings reported by TiVo can be way off.
     
  10. Jan 14, 2010 #1750 of 10616
    dwalzer

    dwalzer New Member

    6
    0
    Jan 14, 2010
    Yes, on the Tivo it's just a partial recording of the program, but that's OK. I've been using it as a test file simply because its 1GB file size makes it easy to see total compression in the final encoded output. All that aside, my point is that the file does not complete downloading -- I might end up with only 15 minutes of the original 20, or maybe only 2 minutes. As I said, it seems random how much I get.
     
  11. Jan 14, 2010 #1751 of 10616
    moyekj

    moyekj Well-Known Member

    11,267
    81
    Jan 23, 2006
    Mission...
    That is very odd. I've heard of cases of partial transfers due to some problem with TiVo doing the TTG conversion, but usually those always fail at same spot, not random.
    kmttg uses "curl" to do the transfer and reports any problems there may be. Next time you initiate a transfer double click on the download job in job monitor and you can see the curl output statistics and messages as it downloads. Perhaps look for anything unusual there. Also make sure you have enough room available to where you are downloading files to.
    Other than that if you try and download via a web browser do you get random results as well?
     
  12. Jan 14, 2010 #1752 of 10616
    dwalzer

    dwalzer New Member

    6
    0
    Jan 14, 2010
    The problem just occurred again. I watched the download quit prematurely, and got the following error message from curl:

    curl: (56) Received problem 3 in the chunky parser
     
  13. Jan 14, 2010 #1753 of 10616
    moyekj

    moyekj Well-Known Member

    11,267
    81
    Jan 23, 2006
    Mission...
    Sounds like erratic responses from your TiVo web server. Have you tried rebooting your TiVo? What's your network setup? Do you have wireless in the picture?
     
  14. Jan 14, 2010 #1754 of 10616
    txporter

    txporter One sec, almost done

    666
    0
    Sep 17, 2006
    Austin, TX
    Agreed. Sounds like wireless network drop-out.
     
  15. Jan 14, 2010 #1755 of 10616
    dwalzer

    dwalzer New Member

    6
    0
    Jan 14, 2010
    I have not tried rebooting the Tivo yet. Network setup is solid -- both Tivo and PC are hardwired directly to the Fios router; no wireless. I'm trying to isolate if the problem is worse when it's working on a queue of tasks as opposed to just a single file. Maybe I'm just overloading the PC. It is an old machine after all (even though I've left Active Job Limit and Encoding CPU Cores at 1).
     
  16. Jan 14, 2010 #1756 of 10616
    moyekj

    moyekj Well-Known Member

    11,267
    81
    Jan 23, 2006
    Mission...
    I don't think the PC is the problem. A download doesn't take much in terms of resources. The download speed is throttled by TiVo slow processor which means disk I/O is pretty low and there should be not much CPU overhead involved. Guess you can bring up Task Manager to check but I think you will find that for the download task itself there shouldn't be much overhead.
     
  17. Jan 14, 2010 #1757 of 10616
    dwalzer

    dwalzer New Member

    6
    0
    Jan 14, 2010
    OK, thanks a million for all your help! I will go noodle around with it some more, reboot the Tivo, and report back when I think I've solved it.
     
  18. Jan 14, 2010 #1758 of 10616
    VideoGrabber

    VideoGrabber New Member

    114
    0
    Sep 11, 2003
    > I've heard of cases of partial transfers due to some problem with TiVo doing the TTG conversion, but usually those always fail at same spot, not random. <

    I ran into that a lot in the early days of TTG, but an update a year ago took care of most of it. Though it's happening again now, consistently on one channel that the cableco started inserting SD commercials into HD broadcasts. So my grief has returned. (And it's something other than just the Dimension Changes, since TTG handles those fine, and I clean them up in VideoReDo.)

    When TTG fails in this way, it's always at the same byte, which is a discontinuity in the stream. It also doesn't matter externally how the transfer is being handled (web browser, TiVoDesk, DesktopLite, kmttg/curl, etc.) Even though the TiVo can "play thru" whatever the problem is in the stream, the internal TTG software will throw up it's hands and give up, even though it knows there's GB's more data in the stream. Pretty sad, actually. (I.e., anything the TiVo video engine can play, TTG should be able to transfer, and simply skip over the packets it has a problem with.)

    > Next time you initiate a transfer double click on the download job in job monitor and you can see the curl output statistics and messages as it downloads. <

    I'm not seeing that. Where is it located/accessed?

    - Tim
     
  19. Jan 14, 2010 #1759 of 10616
    moyekj

    moyekj Well-Known Member

    11,267
    81
    Jan 23, 2006
    Mission...
    Yes, that's what I was referring to. In this case since the behavior is erratic/random so I believe it's a different problem to what you describe above, but maybe not.

    The list below the CANCEL JOBS button has a list of currently active and queued jobs. If you double-click on an active "download" task it pops up a "Task stdout/stderr" window showing you the output of curl in stderr pane.
    This applies to any kind of job that is currently running and where you may be interested in seeing program stdout/stderr.
    (The above feature is documented and there is also a tooltip that tells you as much if you hover over a job.)
     
  20. Jan 14, 2010 #1760 of 10616
    VideoGrabber

    VideoGrabber New Member

    114
    0
    Sep 11, 2003
    > The list below the CANCEL JOBS button has a list of currently active and queued jobs. If you double-click on an active "download" task it pops up a "Task stdout/stderr" window showing you the output of curl in stderr pane. <

    Cool. Thanks! (though a monospaced font would make the columns line up better.)

    - Tim
     

Share This Page