1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New program for 1 step TTG downloads, decryption, encoding - kmttg

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Mar 15, 2008.

  1. Jul 11, 2017 #11101 of 11243
    worachj

    worachj Well-Known Member

    1,276
    41
    Oct 13, 2006
    Eagan, MN
    Trying to duplicate the problem. Not having great success determining what toggles the problem on and off.

    Multiple shows are queued in kmttg for download and commercial ship. Some shows have autoskip points, some do not. While watching the windows resource manager I notice the download speed drop 40% when kmttg went comskip => ffcut/ffmpeg. The show had autoskip points so comskip wasn't done. All following downloads had the slower download speed.

    Something about FFFix=>Comskip=>FFcut/ffmpeg while another download is occurring triggers the problem. Not sure what clears the problem back for normal download speeds.
     
  2. Jul 12, 2017 #11102 of 11243
    worachj

    worachj Well-Known Member

    1,276
    41
    Oct 13, 2006
    Eagan, MN
    I'm able to duplicate the problem following the above process. Something about FFFix=>Comskip=>FFcut/ffmpeg while another download is occurring triggers the problem.

    Java/library not releasing resources problem? That's probably why I didn't notice the problem until recently. I upgraded to a newer 64 bit Java release when I notice that somehow I was switched to the 32 bit version.
     
  3. Jul 12, 2017 #11103 of 11243
    SugarBowl

    SugarBowl Member

    801
    2
    Jan 4, 2007
    Cary, NC
    Does the sling box capture feature work with a slingbox M1 ?
     
  4. Jul 12, 2017 #11104 of 11243
    moyekj

    moyekj Well-Known Member

    11,473
    181
    Jan 23, 2006
    Mission...
    Don't know, I've only used/tested it with Slingbox 350.
     
  5. Jul 12, 2017 #11105 of 11243
    SugarBowl

    SugarBowl Member

    801
    2
    Jan 4, 2007
    Cary, NC
    Does this tell you anything ?

    >> CAPTURING SLINGBOX TO C:\Apps\slingbox_2017_07_12_14_37_23.ts ...

    "C:\Perl64\bin\perl.exe" "C:\Apps\kmttg_v2.1k\slingbox\rec350.pl" -stdout -ip 192.168.1.44 -port 5301 -pass ***** -vbw 4000 -vs 16 | "C:\Apps\kmttg_v2.1k\ffmpeg\ffmpeg.exe" -fflags +genpts -i - -vcodec copy -acodec ac3 -ab 224k -r 29.97 -y -f mpegts "C:\Apps\slingbox_2017_07_12_14_37_23.ts"

    Capture to file failed: C:\Apps\slingbox_2017_07_12_14_37_23.ts

    Exit code: 1

    ffmpeg version N-79000-g66edd86 Copyright (c) 2000-2016 the FFmpeg developers

    built with gcc 5.3.0 (GCC)

    configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmfx --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib

    libavutil 55. 19.100 / 55. 19.100

    libavcodec 57. 28.100 / 57. 28.100

    libavformat 57. 28.100 / 57. 28.100

    libavdevice 57. 0.101 / 57. 0.101

    libavfilter 6. 39.102 / 6. 39.102

    libswscale 4. 0.100 / 4. 0.100

    libswresample 2. 0.101 / 2. 0.101

    libpostproc 54. 0.100 / 54. 0.100

    cmd: 103 err: 2 8 at C:\Apps\kmttg_v2.1k\slingbox\rec350.pl line 140.

    pipe:: Invalid data found when processing input
     
  6. Jul 12, 2017 #11106 of 11243
    moyekj

    moyekj Well-Known Member

    11,473
    181
    Jan 23, 2006
    Mission...
    This means something:
    Code:
    cmd: 103 err: 2 8 at C:\Apps\kmttg_v2.1k\slingbox\rec350.pl line 140.
    
    Line 140 in rec350.pl is:
    Code:
    die "cmd: $opcode err: $stat $dlen" if $stat && $stat != 0x0d && $stat != 0x13; # error code from SB
    
    i.e. The Slingbox is returning error message, so either the login/password setup is not right or the Perl script doesn't work with your model Slingbox.
     
  7. Jul 12, 2017 #11107 of 11243
    SugarBowl

    SugarBowl Member

    801
    2
    Jan 4, 2007
    Cary, NC
    The logins are correct.. Looks like the M1 (and probably the M2) doesn't work.

    Thanks for confirming.
     
  8. Jul 13, 2017 #11108 of 11243
    ClearToLand

    ClearToLand Old !*#$% Tinkerer!

    685
    60
    Jul 9, 2001
    Central Jersey
    Yesterday I unpacked and set up my third Roamio (a new OTA 1TB), adding to my existing 'network' of a refurb Roamio Basic 500GB and a refurb Roamio OTA 500GB. While doing so, I decided to change my 'Naming Convention' and thus re-named the two existing Roamios. After all three had the names I wanted, I put each into Standby and then Power-Cycled them one-at-a-time. Both TiVo.Com and each individual TiVo now have the correct, new names.

    kmttg, on the other hand, now has MULTIPLE names for the same IP (OLD name, NEW name, NEW name w/ typo, NEW name generated by TiVo.Com, etc...). So, naturally, I restarted kmttg - NG. I SEARCHed the kmttg Wiki (found nothing about renaming a TiVo; doesn't look like you can do it; must DELETE and re-ADD, either automatically or manually, AFAICT) and TCF (TiVo name is not correct on KMTTG). Since the TCF thread is from 2011, I'm assuming 'something' in kmttg has changed:
    • What are the consequences of re-naming (DELETE / re-ADD) a TiVo in kmttg?
      .
    • AutoSkip.ini (a CRLF / Windows-style file) has multiple references to the OLD TiVo names.
      - Is it safe to edit these names (Find & Replace w/ Notepad) to the new names? (while kmttg is not running of course)
      .
    • config.ini (a LF / *NIX-style file) has both the OLD and NEW TiVo names associated with the same IP(s).
      - Should I just DELETE the unwanted names in File -> Configuration -> TiVo ?
      - Or should I edit the file with NotePad++? (while kmttg is not running of course)
    Thanks! :)
     
  9. Jul 13, 2017 #11109 of 11243
    ThAbtO

    ThAbtO TiVoholic by the bay

    7,525
    85
    Apr 6, 2000
    SF Bay Area
    Just delete the Tivo with the old names from KMTTG.
     
  10. Jul 13, 2017 #11110 of 11243
    ClearToLand

    ClearToLand Old !*#$% Tinkerer!

    685
    60
    Jul 9, 2001
    Central Jersey
    Then you lose the relationship between the TiVo name and the entry in AutoSkip.ini.

    I have almost 3TB of shows backed up to an External HDD with .tivo, .srt and .edl files. I had high hopes that when pyTivo or pyTiVo Desktop brought them back, AutoSkip would still be available. But, in my very limited 'experimentation', it doesn't seem to be. :( [My first 'guess' is that 'something' in the .tivo file header is changed when the show is re-loaded onto a TiVo unit and the 'MotherShip' no longer sees it as a 'Skip Data Available' show. :oops: I've been meaning to ask @Dan203 about this but recent health problems got me sidetracked.]

    I'll wait to see what Kevin has to say before I do anything that 'might' cause damage.

    Thanks.
     
  11. Jul 13, 2017 #11111 of 11243
    moyekj

    moyekj Well-Known Member

    11,473
    181
    Jan 23, 2006
    Mission...
    Yes, in AutoSkip.ini you can replace old names with new ones and delete the old names from config.ini.
    Of course can't hurt to keep backups of original files as they were before you make any edits.

    NOTE: For AutoSkip.ini the originating TiVo name shouldn't matter as far as getting AutoSkip functionality, it's the contentId that really matters. That's why for example if you have the same show with same contentId recorded on a different TiVo in your house, AutoSkip will still work, as will AutoSkip on a Mini.
    I *think* only side effect of having wrong TiVo names in the AutoSkip.ini file is that they won't be indicated with "S" in kmttg NPL table column 1.
     
    ClearToLand likes this.
  12. Jul 13, 2017 #11112 of 11243
    ClearToLand

    ClearToLand Old !*#$% Tinkerer!

    685
    60
    Jul 9, 2001
    Central Jersey
    When I bring back a show from my External HDD via either pyTiVo or pyTiVo Desktop, I no longer see the 'Skip' icon on either the TiVo or in kmttg (as of ~ a month ago was my last 'experiment'):
    • Am I doing something wrong?
      .
    • Are others reading this GETTING the 'Skip' icon?
      .
    • Did I previously ask this and forgot? :oops:
    The *ONLY* purpose for the .edl file, that I'm currently aware of, is input for ComSkip (not AutoSkip). Is this correct?
     
  13. Jul 13, 2017 #11113 of 11243
    moyekj

    moyekj Well-Known Member

    11,473
    181
    Jan 23, 2006
    Mission...
    Perhaps with TTCB (pyTiVo) "contentId" is being lost which is the basis for how TiVo finds SkipMode data. Pretty easy to test if that's the issue. Look at an entry in kmttg AutoSkip.ini and get it's contentId and then compare to contentId you see for the show transferred back to a TiVo (bindkey j in kmttg will give you the info). If they don't match then that's the problem. Also if you are just transferring .TiVo files back to your TiVo then for sure a lot of metadata will be lost. Be sure to transfer back decrypted file with pyTivo metadata file that has proper programId entry.

    EDIT: Note as far as TiVo SkipMode goes I don't think SKIP will ever show on TiVo for any file you transfer back even with correct programId entry, but AutoSkip I think should work in that situation (assuming contentId matches what kmttg AutoSkip.ini has).
     
    Last edited: Jul 13, 2017
  14. Jul 13, 2017 #11114 of 11243
    ClearToLand

    ClearToLand Old !*#$% Tinkerer!

    685
    60
    Jul 9, 2001
    Central Jersey
    Yes, all I've been saving is the PS / 'Slow' Format transfer .tivo file along with the CC in the .srt and the 'cuts' in the .edl. If I understand you correctly, the .tivo I'm putting back is now missing 'tons' of metadata and in order to preserve this metadata I need to decrypt to .mpg and save the metadata to .txt and forget completely about restoring / transferring back the .tivo.

    So, am I also correct that the .tivo file stored on my External HDD *STILL* has the full metadata *AND* if I decrypt it to .mpg (and save the metadata in .txt if I don't already have it on the External HDD) and let pyTiVo transfer THAT back to a TiVo, the 'Skip' icon should re-appear (and all of the old AutoSkip data in AutoSkip.ini will 'magically' work again)? :cool:

    Hmmm... All this time I was 'assuming' that the .tivo IN (TiVo-to-PC) was EXACTLY equal to the .tivo OUT (PC-to-TiVo) - NG huh... o_O I'm surprised that I haven't read much about this here on TCF; the penalties of arriving late to the game...

    What is TTCB (before I go off GOOGLE'ng)? Is it possible for either version of pyTiVo to NOT lose "contendId" with a code change? Or are they locked into TTCB along with its' deficiencies?

    [HR]=====
    OK. I see the "contentId" string in the "programId" field of the .txt file after the hyphen. It appears that I started also saving .txt files back around May, so before then I'll see if I can still extract the information from the saved .tivo file on the External HDD. PROGRESS! :D

    BTW, last month I set up a Python Development Environment on my old Vista PC and 'hacked' into the pyTiVo Desktop (v1.62 IIRC) source code (I knew nothing about how to code in Python before this; just used the online 'Guide' to what the commands did) to add the displays I wanted to see. *NOW* I understand how random and unstable TS / 'Fast' Format transfers can be (i.e. the TS Sync Error keeps moving! :eek: ). For me, a picture (or as I call it some DEBUG display code) WAS worth a thousand words. :thumbsup:

    Although I only have problems with (few and far apart; two shows since I came back to TiVo in 2015) H.264 files (forcing me to use TS / 'Fast' Format transfers), I'll be buying my own copy of VideoReDo TVSuite the next time it goes on sale just to avoid the task of temporarily having to use pyTiVo Desktop with a high number of retries in order to archive a show.
     
    mlippert likes this.
  15. Jul 13, 2017 #11115 of 11243
    moyekj

    moyekj Well-Known Member

    11,473
    181
    Jan 23, 2006
    Mission...
    You will NOT get SkipMode (SKIP icon on the TiVo) for any program you transfer back regardless of what metadata you use. You may retain kmttg AutoSkip functionality however, and transferring back files to TiVo with proper programId is the most complete way to preserve metadata on the TiVo itself when using the HDUI.

    TTCB = TiVo To Come Back = transfer back to TiVo
     
    ClearToLand and mlippert like this.
  16. Jul 14, 2017 #11116 of 11243
    greenawayj

    greenawayj New Member

    10
    1
    Nov 16, 2006
    Hi all. I'm a long time kmttg and VRD user (just upgraded from v4 to v5 this week) but generally not on this board as much. I mostly watch on my Tivo or MRV streaming to minis but I do archive a small subset of recordings once a month or so for safe keeping and/or taking shows offline (with commercials removed) to watch through Plex when traveling. I find Plex sync to be way better / faster than Tivo downloads. That said, these tools together are amazing pieces of work and I've seen the Kevin's past rejections of donation offers, but if you ever change your mind, I'm in for a decent donation since this app has been a great value to me over the years.

    Anyway, I had been way behind on this thread and the general new capabilities in kmttg for a long time. I just spent parts of the past 5 days catching up on the last 15 months or so of posts, mostly focusing on messages about auto-skip from skip mode and generating vprj cut files from that data. I think I found something not working as I expected - and for the life of me, I don't think anyone else had mentioned this previously (but I could certainly have missed it).

    Here's the issue: When running in TS container mode (but mp2 recording) AND after running a QS fix (and backing up the original ts file for comparison), I can see in vlc player that the original ts file (renamed back to .ts again from .bak) is slightly longer than the resulting .ts file after qsfix. (usually in the 0-40 seconds range, but I had at least one with more than a minute cut by qsfix). I never paid attention in the past if qsfix had cut any run time. As a result of this cut, the vprj file generated from skip mode data gets slightly out of sync later in the show (presumably after wherever qs fix had to cut (a lot of) frames to fix things - I guess that is what is happening with the qs fix 'cuts'). This ends up leaving parts of commercials in the cut show and cutting parts of the program. This is obviously because the skip mode cut points were based on the full run time show and not the shorter length post-qsfix. Obviously, when using vrd ad detect, this is never a problem since the ad detect scan happens after the qs fix on the post-qsfix file.

    My current solution to this issue is to just exclude the qs fix step in the workflow - I haven't actually watched these shows completely to see if there are major a/v sync issues, but in spot checking a few in vlc, I don't see noticeable sync issues without qs fix.

    My relevant settings (when seeing this error) were:
    • programs have SKIP and "S" (autoskip from skipmode runs nightly through windows scheduler)
    • main: checked: decrypt, qs fix, ad cut, encode - encoding uses a vrd profile. not checked: ad detect - auto generated vprj from SKIP MODE, metadata, captions, custom
    • file settings: backup orginal .mpg
    • program options: tivolibre
    I don't totally understand what qs fix does or how (generally, my understanding it it helps prevent audio/video sync issues which I don't really notice that much when I don't run qs fix, BUT there's no harm in qs fix, so why not). In this case, does it possibly make sense to run qs fix on the mpg or ts file AFTER the SKIP MODE generated vprj cuts instead of after decode and before cuts? (Admittedly, given my lack of complete understanding here, that could be a totally idiotic question? maybe it can't be run at that point or would not work on a file already chopped up)

    On a related note (peripherally): The particular shows I was seeing this happen to were Blackish S3 E1-E7. These are all mp2 recording, but I had been trying to just use TS for all show downloads since I had several PS failures on mp4 content before I started catching up on this tread and realized the issue. (I don't care about captions so TS limitations as I understand them were fine for me). However, some (not all) of these 7 consecutive episodes ended up with only audio and no video (just green screen frames happening at some point in the process - I cant remember right now). I have since switched back to PS for mp2 recordings and just left out the qs fix step. I'd be happy to reproduce this and put relevant files on a dropbox if necessary. I was just surprised to see some of these work just fine in TS but others fail repeatedly until I did them as PS.

    Sorry for the long note. Didn't think it would take this much to explain my observations (and be a little cordial and appreciative out of the gate!)
     
    leswar likes this.
  17. Jul 14, 2017 #11117 of 11243
    moyekj

    moyekj Well-Known Member

    11,473
    181
    Jan 23, 2006
    Mission...
    greenawayj, yes QS Fix especially on TS transfers which tend to have issues during TTG transfers (from TiVo side) can cut out several seconds of video as you are seeing. So ultimately you still need to review the cut points before proceeding with the Ad Cut step. So I would recommend you enable the "Use VideoRedo GUI to review detected commercials" option in Options--VideoRedo as part of your flow if you are having this problem persistently.

    Note that I mostly use AutoSkip for hands free watching of programs still on the TiVo, not for generating cut files, so I don't know how prevalent this problem is, maybe others that use AutoSkip for generating cut files can comment further on their flow.
     
  18. Jul 14, 2017 #11118 of 11243
    ggieseke

    ggieseke Well-Known Member

    4,265
    80
    May 30, 2008
    My usual workflow is to QS Fix manually from .TiVo files to .ts files with VRD, then apply the downloaded skip points to the .ts file. The cuts tend to get slightly behind the further I go into a file, but they're still a godsend and I use kmttg for so much more than that. I think that's due to the QS Fix process, not kmttg.

    I would like to echo greenawayj's post. Even if you don't solicit donations like I do for DvrBARS & MFSR, give us a way to contribute something. I have a Benjamin with your name on it.
     
  19. Jul 15, 2017 #11119 of 11243
    worachj

    worachj Well-Known Member

    1,276
    41
    Oct 13, 2006
    Eagan, MN
    I believe my problem with slower download speeds are caused by the "superfetch" window service. Something with the superfetch service and Java not playing nicely with each other.

    I haven't noticed the problem since disabling the "superfetch" windows service. Though not sure how disabling it will effect the performance of other window programs/stuff.
     
    mlippert likes this.
  20. Jul 17, 2017 #11120 of 11243
    worachj

    worachj Well-Known Member

    1,276
    41
    Oct 13, 2006
    Eagan, MN
    I found out something new this morning, at least for me. I didn't know that autoskip worked on a Premiere if you're streaming the recording from a Bolt or Roamio. WOW, neat! I no longer need to strip commercials for the Premiere I use to watch shows while ridding the exercise bike.
     
    moyekj likes this.

Share This Page