1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

pyTivo - Transcoding server

Discussion in 'TiVo Home Media Features & TiVoToGo' started by armooo, Nov 25, 2006.

  1. PaulS

    PaulS Member

    684
    2
    Sep 16, 2002
    Southern NH
    Was that for HD or SD sources ? I'm assuming HD...
     
  2. PaulS

    PaulS Member

    684
    2
    Sep 16, 2002
    Southern NH
    wgw has made a change to his pyTivo branch HERE to accomodate for fixes made in 9.3a :

    Note that this change is to the MAX VIDEO BITRATE, which is responsible for handling peaks of activity during encodes.
     
  3. lrhorer

    lrhorer Active Member

    6,924
    1
    Aug 31, 2003
    San...
    I read it and I understand it. Would it help if I pointed out I used to be a Video Engineer, and I am now a Network Engineer?

    Of course there is, but the effects I have seen have nothing to do with the transmission bit rate. The transmission bit rate is determined by a number of network and workstation factors. Transmission rates in excess of 116 Mbps have been reported without issues using the backport drivers. I am using the stock drivers, and so rarely enjoy much greater than 17 Mbps from the PC to the TiVo using pyTiVo or Galleon. From the TiVo to the PC I usually get about 30 Mbps, sometimes up to 40 Mbps using TyTool on the 100M Ethernet port.

    I submit the use of the phrase "most conclusively" is ill advised. Certainly the results of my own testing do not bear out the assertions you are making. As always, one person's mileage may vary compared with another's.

    'Not any longer. Release 9.3 has fixed the issues. Galleon, TiVo Desktop, and pyTiVo with no configuration options now can all deliver picture perfect HD content without transcoding. As an example, my copy of Judgment at Nuremberg is 2:58:29 in length and consists of 23,214 MB of steam data, which is an average of 17.34 Mbps. Power DVD reports peak bit rates as high as 18.82 Mbps during the first 3 minutes of the film. It now transfers perfectly without transcoding at over 16 Mbps. Previously it had problems in the video even if the network transfer were as slow as 5 Mbps, and indeed I noted qualitatively the same level of artifacts no matter what the network transfer speed. Conversely, if the video were transcoded to limit the video bit rate, few or no video problems were observed even at network transfer speeds in excess of 16 Mbps.
    Since every single one of the HD videos on my server came from the TiVos themselves, and since a number of them frequently exceed 17 Mbps, this statement was demonstrated to be total nonsense even before the release of 9.3. Since 9.3 has completely eliminated all such artifacts including in streams whose peaks exceed 20 Mbps, it's doubly nonsense. With the exception of a number of SD videos (mostly Bogart films and Star Trek episodes), very few of the 300+ videos on the server average less than 12Mbps, and most average above 15Mbps. Several of the Blue Planet episodes peak in excess of 20.0 Mbps. All were originally recorded on the TiVos - many prior to 9.3, and since 9.3 they all play perfectly when transferred back to one of the TiVos from the server.

    Setting video_br to 12 results in HORRIBLE looking video if the original is 1080i in excess of 15Mbps.

    I am well aware of what is involved in networking and the various parameters affecting network performance, thanks.

    I have seen that report. It is not backed up by my own testing.

    My video server has a 3.0GHz dual core Athlon 64 running Debian "Etch" Linux. I never saw it hit 100% utilization on both cores when transcoding, but then I really wasn't paying all that much attention to the CPU utilization. It certainly was able to transcode and transfer at near-real time, which in the case of most of the testing I did was around 12 Mbps, sometimes exceeding 16Mbps.

    Since the TTCB transfers are TCP, your guess lacks credibility. TCP itself would have slowed the transfers if any of the packets were corrupt after assembly at the Rx port. Not only that, but since 9.3 works just fine with the same kernel as 9.2, displaying no issues with said kernel, your guess holds water about as well as a sieve. It was almost surely an issue with the TiVo's encryption routine, although I suppose it's possible the MFS code could have been responsible. The fact the artifacts would show up in exactly the same spot for a given video no matter how fast or slow the transfer or what other network processes were in play on the TiVo as well as the fact MRV transfers were just fine at faster than real-time lend credence to the faulty encryption hypothesis, and make the MFS hypothesis less likely. None of these facts support the notion there was a problem with the network drivers, and the fact the TiVo was more than capable of encrypting not one but TWO streams well in excess of 16 Mbps pretty much blows the notion a 12 Mbps network stream was too fast for the TiVo in any sense right out of the water.

    There were no assumptions of any sort in the statements you quoted.
     
  4. mbklein

    mbklein Foreign Contaminant

    9,979
    0
    Oct 24, 2000
    Evanston, IL
    I realize this is an old post, but I've been digging around and discovered that the listings at zap2it.com contain consistent series/episode IDs to my guide data. For example, the zap2it URLs for today's episode and the episode guide for Live at Lincoln Center are:
    And the same episode from the XML version of my TiVo's Now Playing list:
    ProgramId = the "pgmID" parameter from the episode detail URL (with a couple leading zeros truncated), and SeriesID = a slightly altered version of the sId parameter from the Episode Guide (likewise). That should make it pretty easy to figure out the proper IDs without having to muck around in the guide data (or wait for an episode to record so you can hijack its ID).
     
  5. Rdian06

    Rdian06 New Member

    463
    0
    Apr 12, 2008
  6. mbklein

    mbklein Foreign Contaminant

    9,979
    0
    Oct 24, 2000
    Evanston, IL
    Ah, OK. Sorry about that. :) Even with search, it's kinda hard to keep up with what's already been figured out.
     
  7. Oct 2, 2008 #1967 of 5755
    Ely105

    Ely105 New Member

    25
    0
    Sep 7, 2008
    Not sure if there is a thread already out there on this. I have read through many posts and tried several things. I have a TivoHD with 9.4 and 2TB of storage. I am trying to find out if there is a tweaking guide for ffmpeg to get out better performance on the transcoding of videos. I'm a bit of an HD snob so i prefer to have videos that are HD, or at least semi HD. I am using a reasonably newer dual-core PC.

    For several of the videos I am trying to watch, i get between 19-25 fps. Most of these are at 24 fps, so i'd really like to get a little more throughput so that I can watch immediately and it could keep up.

    The files that I am using are MKV and are 1280 x 544 or similar Res. they are h.264 with AC3 streams.

    Things I have tried:
    I have tried to force the resolution settings, but i find that it doesn't change much, and ffmpeg is doing fine at detecting the audio/video.

    I have reduced the BW from the default of 8, down to 4k. This seemed to help, but of course it comes at a quality penalty. So i'm trying to see if I can keep the bitrate up or at least in the 8k range.

    I've added "-threads 2" to the ffmpeg command, but i am only seeing about 50-60% CPU (core) utilization. I don't know if the x264 and mpeg2 codecs really thread much to take advantage of multiple cores? Are there any other ways to help improve core utilization?

    I was wondering if the ffmpeg (from windows installer) is optimized for the latest processors with all the mmx, sse and other extensions? I have read some people using Mencoder as an alternative to ffmpeg, but it seems there are problems there as well.

    I suppose one option would be for me to re-encode (or remux) the .mkv files into .mp4 files and use tivostream on them. But I like the ability to throw just about any video at pytivo and having it transcode. I know there is a little quality loss in the process, but from what i've seen so far, it's pretty damn good.

    -Mark
     
  8. Oct 2, 2008 #1968 of 5755
    Kershek

    Kershek Member

    125
    0
    Oct 10, 2002
    Mesa, AZ
  9. Oct 2, 2008 #1969 of 5755
    Ely105

    Ely105 New Member

    25
    0
    Sep 7, 2008
    Thanks for the suggestions.

    Yes, I did try to raise the process priority, but no real diff.

    i think it's a 2 Ghz based system, but it's not overclockable.

    I need to compare it to another system I have and see what the difference in perf is.

    It's pulling the file off my NAS, but I get between 20-30MB/s transfer speeds so i don't think that's an issue.

    I will look for any fragmentation issues.

    I looked at memory utilization and it seems like there is no issues there.

    It's running on a fairly lean copy of XP with not a lot of other stuff running.
     
  10. Oct 8, 2008 #1970 of 5755
    Jkjowers

    Jkjowers New Member

    29
    0
    Jan 25, 2004
    I have also been trying to figure out how to group things on my Tivo. I suppose I will have to sound like an idiot... please define "one that is currently in your TiVo's guide data".

    Thanks :eek:
     
  11. Oct 8, 2008 #1971 of 5755
    ObiWanJenkins

    ObiWanJenkins New Member

    151
    0
    Dec 31, 2004
    How do I get the Windows installer to see that I have Python installed? I've read numerous posts that mention it has to find it, but nothing to instruct how to direct it.
     
  12. Oct 8, 2008 #1972 of 5755
    Rdian06

    Rdian06 New Member

    463
    0
    Apr 12, 2008
    "one that is currently in your TiVo's guide data" = A show that is being broadcast on one of the feeds into your Tivo. So if you just have over the air signals going into your Tivo, your Tivo guide data sent from Tivo corporate will only include your locals. No HBO or Showtime seriesIDs. Likewise if a show isn't being actively broadcasted, it won't show up in the Tivo guide data.
     
  13. Oct 8, 2008 #1973 of 5755
    Rdian06

    Rdian06 New Member

    463
    0
    Apr 12, 2008
    How did you install Python? Assuming you used the Windows installers from python.org, the Windows installer will just find it.

    If you installed Python 2.6 that JUST came out a few days ago, then the Windows installer won't find it. The person who maintains the Windows installer hasn't been active lately and the last update was back in early May. No idea when he'll have time to work on it again.
     
  14. Oct 8, 2008 #1974 of 5755
    mbklein

    mbklein Foreign Contaminant

    9,979
    0
    Oct 24, 2000
    Evanston, IL
    Interesting tidbit: I used the seriesID of a show I knew I'd never record as a way to group a bunch of related recordings that I couldn't get a valid seriesID for. The seriesTitle and episodeTitle behaved as expected, but when I sort folders by name, the folder (which starts with 'M') shows up sorted with the T's because the seriesID I hijacked is for a show that starts with T. So TiVo displays what I want, but sorts it according to its internal seriesID index.

    So if you hijack a seriesID for your own use, make sure the first few letters of its seriesTitle match your show's seriesTitle.
     
  15. Oct 8, 2008 #1975 of 5755
    Rdian06

    Rdian06 New Member

    463
    0
    Apr 12, 2008
    Strange, I thought on my S3 when I hijacked a seriesID, the folder name (after transferring two episodes) became that of the hijacked series title as taken from the guide data and not the seriesTitle I specified. That is if you just transfer one record, then it uses your seriesTitle. But once the second recording with the hijacked seriesID is transferred, the grouping kicks in and the title is taken from the guide.

    I'll have to retest later. mbklein, what Tivo do you have?
     
  16. Oct 8, 2008 #1976 of 5755
    ObiWanJenkins

    ObiWanJenkins New Member

    151
    0
    Dec 31, 2004
    I installed python 2.5.2 x64 from python.org through the normal Windows installer. But pyTivo doesn't detect it.
     
  17. Oct 9, 2008 #1977 of 5755
    Rdian06

    Rdian06 New Member

    463
    0
    Apr 12, 2008
    Yeah, the installer is definitely at fault. The x64 part is probably throwing off the version check it's trying to do. Unfortunately krkeegan never checked in the source for the installer so I can't fix it easily.

    You are better off installing pyTivo manually using wgw or wmcbrine's fork version. Essentially you download a tar.gz file, extract it to a folder, configure your pyTivo.conf, and then run "python pyTivo.py" from a command prompt after cd'ing to the pyTivo folder. pyTivo will run from console that way and print it's output to the command prompt. The only real magic that the Windows installer does is setup running pyTivo as a service.
     
  18. gonzotek

    gonzotek tivo_xml developer

    2,408
    4
    Sep 24, 2004
    Outside...
    I think it's here:
    http://repo.or.cz/w/pyTivo/krkeegan...c;hb=be94f986de5e0d119fb181e71e19c80257c0acd9

    I've done several NSIS installers for other projects and was on the beta-test team for it for awhile, but it's been a few years since my last major interaction with NSIS and I am far to busy with a full-time job and night classes to tackle it right now, but if you do and run into any trouble, post about and I'll try to help.

    We could really use an updated installer 'for the masses'. :)
     
  19. Rdian06

    Rdian06 New Member

    463
    0
    Apr 12, 2008
    Cool. I swore I looked for those and couldn't find them. I see where the problem is with the Python version check. Seems like an easy fix, but I've never built an NSIS installer before and I'm really low on time this month.

    Try to look at it after I get done with my "honey dos" for the weekend.
     
  20. ObiWanJenkins

    ObiWanJenkins New Member

    151
    0
    Dec 31, 2004
    I will give that a try. I'm beginning to think Vista is horrible. :D
     

Share This Page