Is something seriously wrong with TiVo’s guide data?

Discussion in 'TiVo Coffee House - TiVo Discussion' started by Puppy76, Jan 16, 2020.

  1. Jan 17, 2020 #21 of 345
    mbernste

    mbernste Active Member

    766
    84
    Apr 6, 2003
    Piscataway, NJ

    Advertisements

    The Richard Pryor show aired last night (Thursday). This makes sense since I don't think this type of show would typically preempt 20/20.
     
  2. Jan 17, 2020 #22 of 345
    tenthplanet

    tenthplanet Well-Known Member

    1,607
    478
    Mar 5, 2004
    I know Tivo takes flack for Rovi guide data errors but I'm finding posts on the internet that Gracenote is having bad data also. There is no perfect guide data. :eek:o_O
     
  3. Jan 18, 2020 #23 of 345
    Puppy76

    Puppy76 Well-Known Member

    1,438
    45
    Oct 6, 2004
    As of yesterday the switchover seems to have happened okay...everything is broadcasting on their new channels.

    I can’t believe how little warning we got, and wish Tivo had thrown up a warning themselves rather than just vanishing half my stations.

    and I’m still pissed about us just giving away tons of our broadcast space AGAIN

    but at least it seems to be working.
    Found out during this the Hydra channel power scan thingee feature is completly broken though.
     
  4. Jan 18, 2020 #24 of 345
    Luke M

    Luke M Member

    502
    22
    Nov 5, 2002
    The guide data has been unreliable since the switch from Gracenote to Rovi. Some programs are screwed up so often that I resort to manual recordings.
     
  5. Jan 18, 2020 #25 of 345
    Bigg

    Bigg Cord Cutter

    7,393
    804
    Oct 30, 2003
    Hartford-...

    Advertisements

    Yes. It's been an issue since they switched to Rovi. Mine is a mess even a few days out, and then it gets cleaned up, and there's never a full two weeks of data, back in the Gracenote days it was rock solid 2 weeks out.
     
  6. Jan 18, 2020 #26 of 345
    Bigg

    Bigg Cord Cutter

    7,393
    804
    Oct 30, 2003
    Hartford-...
    This looks like the repack issue. They totally botched that too. They could have pulled telemetry data from the boxes to see when the stations moved over in each market, but no, they made people manually file requests to have channels moved over one at a time, and stuff didn't record in the meantime.
     
  7. Jan 19, 2020 #27 of 345
    JoeKustra

    JoeKustra in the other Alabama TCF Club

    20,122
    3,710
    Dec 7, 2012
    Ashland, PA...
  8. Jan 20, 2020 #28 of 345
    Puppy76

    Puppy76 Well-Known Member

    1,438
    45
    Oct 6, 2004
    I’d just noticed that SD versus HD issue myself the past week or two. Stumptown has been getting labeled correctly (says and is HD) but a number of other shows are getting labeled SD which clearly aren’t.

    And man, my ABC station is only like 2.2GB/hour. I wish all my major stations got at least 4.x...although at least my storage space goes further. And there’s a huuuuuuuuge difference between 1.3-1.5/hour and 2GB/hour on MPEG-2. A bigger difference IMO between that then between 2GB and 7GB.
     
  9. Jan 20, 2020 #29 of 345
    JoeKustra

    JoeKustra in the other Alabama TCF Club

    20,122
    3,710
    Dec 7, 2012
    Ashland, PA...
    When Letterman left The Late Show there was a small pause and then the new set with Colbert was a real blowout. At that time my CBS feed had no sub-channels. I could understand why OTA was always said to look better. But then over the course of a year the station added 3 (and now 4) sub-channels. The quality is why I stay with 1080p. The channel went from 16+Mbps down to now 10Mbps. My basic cable channels, like SyFy and TNT are better. I'm still MPEG-2, and see no change in the future. Last I checked USA was 12.57Mbps and ABC was 6.71Mbps. I don't watch a lot of ABC or Fox (except sports).

    I do wonder what quality streaming will bring. So far every time I check, they are still PCM but not yet MP3. Soon.

    Oh yeah, the guide is a joke. Not funny either. Go to Friday and there's a chance for reasonable accuracy. Pass that it's a waste. I remember TVGOS from Rovi. It was a 7-day guide. Some things don't change.
     
  10. Jan 21, 2020 #30 of 345
    tapokata

    tapokata Active Member

    491
    181
    Apr 26, 2017
    Sacramento, CA
    A variable in streaming video quality is the efficiency of the codec. H.264 results in a steam that is about 50% smaller than MPEG-2, so that full bandwidth MpEg channel that was humping at 19 Mbps provides equivalent data at ~9 Mbps. H.265 drops that rate even further, so the encoded file now produces reasonable results at ~5 Mbps.

    While a 5 Mbps MPEG-2 format file can look like the dogs breakfast, a 5 Mbps H.265 can have the picture quality of full bandwidth HD Mpeg-2 stream. ATSC 3.0 can deliver up to 57 mbps in the current 6 Mhz channel space.
     
    Bigg likes this.
  11. Jan 21, 2020 #31 of 345
    Puppy76

    Puppy76 Well-Known Member

    1,438
    45
    Oct 6, 2004
    I disagree about that bitrate looking bad with MPEG-2. Imo lower still looks great. 1.5GB/hour is where it starts looking bad-ish with 720p/60fps, although even that is watchable. IMO it’s a fast diminishing returns thing past 2GB/hour, not that it doesn’t look better, and not that H.265 wouldn’t be an awesome upgrade to use thr same bandwidth better.

    Between using H.265, HDR (if anything will support it...) and better reception/error correction, I’m potentially excited by ATSC 3.0......IF the transition actually works, and we actually have easy/cheap hardware to receive it. Including from Tivo...
     
  12. Jan 21, 2020 #32 of 345
    Bigg

    Bigg Cord Cutter

    7,393
    804
    Oct 30, 2003
    Hartford-...
    Those numbers have to be off if you're using OTA. If you're on Comcast, they make a LOT more sense, as they could be re-compressing your locals to MPEG-4 and then those numbers would add up.

    OTA used to be 19.3mbps, which is about 9GB/hour. Today, OTA typically ranges from 8mbps to 12mbps, which would be 4GB/hour to 6GB/hour, with a few few O&O stations that aren't channels sharing as high as 17mbps, or about 8GB/hour. There may be a few as low as 6-7mbps if they are in a really tight stat mux, which would be 3-4GB/hour. There is no way anyone can do MPEG-2 HD in 2.2GB/hour.

    Are you on Comcast, or looking at file sizes for half hour shows?

    Also, the encoders have gotten WAY more efficient. Today, a good MPEG-2 encoder can deliver the same quality that used to require 19.3mbps in about 12mbps. The problem is, stations always want more, so instead of delivering great HD quality, they put two HD channels on one or a ton of subchannels, and now we have 8mbps channels that look like what 12mbps channels looked like 5 years ago.

    Comcast started tri-muxing MPEG-2 HD around 2012, and it looked pretty awful, at about 12.5mbps average for each of the 3 HD channels. By the time they were done with MPEG-2 and moving to MPEG-4, they had gotten the same awful quality down to 9mbps CBR, where they would "slot" channels without having to do local/regional stat muxes. Imagine if they had put their encoding power towards quality and left them tri-muxed! They would have looked good. Not being happy with that, they then moved those advancements in encoding to MPEG-4, but instead of putting 7 HDs per QAM in a stat mux, which is about equivalent to what DirecTV does with generally good results, they put 9-10 HDs per QAM CBR so that they could be "slotted", and it looks like total garbage. The crazy part is that their total garbage quality is actually an amazing example of efficient encoding, it's just bit starved to such an extreme that it looks like garbage. At the rate they are going, we can expect 2mbps HEVC for unicast IPTV, which would probably look just about as bad, but with all the advancements in encoding rolled into smaller and smaller streams.
     
  13. Jan 22, 2020 #33 of 345
    Puppy76

    Puppy76 Well-Known Member

    1,438
    45
    Oct 6, 2004
    They can and they do. Lower than that. Like I said, I had a station that used to be 1.3-1.5GB/hour for 720p. That was very watchable, but had obvious macro blocking with much action. 2GB/hour with MPEG2 looks good, and would look amazing with H.265. There’s a much bigger difference going from 1.5 to 2GB/hour than from 2 to 7.

    no, and no.

    also, I don’t know what the max possible is per station, but the highest I’ve ever seen was ~7GB/hour for a station carrying only a single channel, so I’ve assumed that must be roughly the limit.

    Although thinking about it, I’ve got a station doing ~2GB right now, + ~2.2, + ~2.8ish, which would add up to 7GB, but there’s some left over for what I’m guessing is an SD channel, Ion I think, which presumably is 1GB or less.


    I’ve wondered what difference there is in encoders, noticed a few oddities and differences between different stations here. My ABC station occasionally had weird...sort of shifting of colors to pink briefly here and there, though I haven’t seen that recently.

    That would be about 5.5GB/hour, which is really high, and ought to look fantastic with MPEG2.

    Ugh, last time I dealt with Comcast, they were randomly chopping off the sides of video on some channels.
     
  14. Jan 22, 2020 #34 of 345
    Chuck_IV

    Chuck_IV Active Member

    774
    75
    Jan 1, 2002
    Connecticut
    I have various problems ALL the time since we were forced to start using Rovi’s putrid guide data.

    From once in a while completely missing guide data to wrong guide data to the VERY frequent “generic” episode data where it describes the show in general but not the specific episode. While Tribune’s data wasn’t perfect, is was EXPONENTIALLY better than the cr@p we have now.

    I am convinced Rovi is just LAZY and do the absolute MINIMUM to get by with their guide data and that is why they probably charge a cheaper price than others, which is why companies use them.

    And yes, my frustration with their guide data has built to this over time.


    Sent from my iPhone using Tapatalk
     
    Last edited: Jan 22, 2020
  15. Jan 22, 2020 #35 of 345
    Bigg

    Bigg Cord Cutter

    7,393
    804
    Oct 30, 2003
    Hartford-...
    How many channels are on that station? I've heard of some 3 or 4 HD broadcasters, which is beyond extreme levels of compression, and even 4 720p channels would still be about 2GB/hour, not 1.3-1.5GB/hour, and those would look like complete trash. Unless it's a weather loop subchannel or something else that compresses really, really well....

    What market are you in and what channel are you claiming uses that extreme of a bitrate? I want to cross reference with rabbitears.tv to see what their data shows.

    19.3mbps. That's the spec. Calculated out, that's 8.69GB before metadata/DRM. I don't think any US channels are still broadcasting at the full 19.3mbps OTA. A few small cable providers still run a handful of sports channels at about 19mbps, half of a QAM. Most markets today don't have anything above 13mbps. Back in the early 2000's, many channels were running a single HD channel at 19.3mbps. There's really no reason to broadcast at that bitrate now, they can do the same quality at 12-15mbps and still have a subchannel or two. 19.3mbps would also blow up people's DVRs even worse.

    It's 8.69GB. With a stat mux, you'd have to record all channels at the same time, otherwise the numbers won't add up. And there is no guarantee that they're using all the bandwidth either. They often arbitrarily limit bandwidth or have a subchannel that's offline at that point in time placeholding for some future channel. I emailed our local PBS about that, as their VQ sucked and they had about 5mbps missing from their broadcast, after a year or more they finally fixed their encoder to use the full bandwidth and get the main channel up to 12mbps where it looks pretty decent. I can't say I caused them to fix it, but more emails can't hurt to keep things moving along.

    There are many different encoders out there, the newer they are the more efficient they are. The practical limit for MPEG-2 HD is basically 2HD/4SD, and even that has somewhat of an impact on VQ, where the image is "flatter" and doesn't have the "pop" and "wow" that a 12mbps or higher encode would have.

    It would look good today, but back then, that was horrendous. Today, they could stat mux 5 HDs onto a QAM and it would be tolerable quality wise. It's only the last few years that broadcasters have been able to get more than one HD channel in an OTA broadcast without everything going to hell.

    Comcast is extremely over-aggressive in terms of video compression.

    For reference:

    (typical bitrate range used for OTA MPEG-2)
    19.3mbps - 8.69GB/hr
    12mbps - 5.4GB/hr
    8mbps - 3.6GB/hr

    (used for Comcast MPEG-4)
    4.1mbps - 1.8GB/hour
    3.8mbps - 1.7GB/hour
    3.5mbps - 1.6GB/hour

    6mhz OTA broadcast: 19.3mbps
    6mhz cable QAM: 38.8mbps
     
  16. Jan 22, 2020 #36 of 345
    Charles R

    Charles R Well-Known Member

    628
    284
    Nov 9, 2000
    USA
    Outside of our local PBS (which used to look the worse) all of the OTA channels are barely on the side of watchable. I have already stopped recording a few shows over the issue however if they get any worse I'm going to simply write them off (totally). They simply don't care... at all.

    Our local ABC even has a repeatable (every few minutes) image breakup/flash at the bottom of the image. It's virtually identical every time and I have seen it on three different tuners and the signal strength is fine. It's a sad state when virtually all streaming looks far superior to OTA - every time I switch and see the "mess" I get closer to saying bye... to their SD image at best.
     
    Bigg likes this.
  17. Jan 22, 2020 #37 of 345
    spiderpumpkin

    spiderpumpkin Say no to Hydra!

    296
    174
    Dec 1, 2017
    In my area the local OTA ABC, CBS, NBC, CW and PBS I record are far superior to those broadcast by Comcast.
     
  18. Jan 22, 2020 #38 of 345
    Puppy76

    Puppy76 Well-Known Member

    1,438
    45
    Oct 6, 2004
    I wouldn’t call it extreme, much less beyond extreme, but I’ve got multiple stations with 4 channels and multiple with 3.

    1.3-1.5 doesn’t look like “complete trash” at all, just noticeably compressed.

    2GB/hour looks good.

    And you’re assuming they’d divide the bandwidth up equally, which they’ve never done. Now they’ve got 2x720 and 1x 1080 station, plus an SD Ion station, doing roughly 2/2.2/2.8ish GB now (plus something for the SD station, I’m not sure what). Used to be 1.5/2.2/4ish, and again there’s a MUCH bigger difference between 1.5 and 2GB than between 2 and 7. That extra 500MB/hour makes a gigantic difference.

    huh, so that’s about 1.5GB more per hour than I’ve ever seen for an individual station. Don’t know how much overhead there is. Currently that station is doing 7GB/hour between the three main networks, and something for the SD one. I’m curious how much, but not curious enough to bother recording something on it lol. Normally in my experience stations here tend to use about 1GB/hour for SD, but who knows.

    Maybe I should try to find out what that fourth channel is using!

    For that matter, my PBS station might be missing something, as it’s around 4.8GB, then 1GB each for World and PBS Kids.

    This station has been doing 2 HD channels that look good since 2016, and before that a different one did 2 good looking HD stations. I think it was since maybe April of last year all 3 HD networks on it have looked good though. It’s been a while now. Even before that though the third one was entirely watchable, just not ideal with fast action.

    I think my FOX station is doing like 6.4/GB/hour, which is now the highest of anything here. I use it probably the least though ever since the Disney sale.

    I ought to save that max bitrate for broadcast for future reference.

    Sounds like ATSC 3.0 will be lower, but with better reception, and it’s a moot point since H.265 is so much more efficient. Hulu is normally only 3Mb/s H.264 for very good looking 1080p, as I recall! That’s significantly lower than my worst HD station, which is handling HD just fine with MPEG2 at ~4.5Mb/s.

    Hence why I’m potentially excited by better reception and H.265! (And possibly 4K and possibly HDR! Maybe H.265 encoding first broadcast will get better too.)
     
    Last edited: Jan 22, 2020
  19. Jan 22, 2020 #39 of 345
    Puppy76

    Puppy76 Well-Known Member

    1,438
    45
    Oct 6, 2004
    I guess I’m lucky that all of mine work, although I’m also lucky that most of mine are locally owned, not Sinclair nonsense, so maybe that’s why.

    my abc station did used to sort of subtly skew the colors pink now and then, more of a curiosity than a real issue, but they seem to have that under control.

    and my nbc station has 3 tiny dots on screen all the time, which I’ve assumed are some watermarking, and again, more a curiosity than a real annoyance, thankfully.
     
  20. Jan 22, 2020 #40 of 345
    Bigg

    Bigg Cord Cutter

    7,393
    804
    Oct 30, 2003
    Hartford-...
    You must have MPEG-4 locals then?

    You're claiming bitrates of roughly 4mbps for MPEG-2, which is beyond extreme compression for MPEG-2 HD.

    What market are you in? I want to see the rabbitears.tv data.

    Then it's not 3mbps or it's not MPEG-2. The borderline for complete trash on OTA HD is somewhere in the 6-8mbps range. I want to see rabbitears.tv data. Are they using MPEG-4? There are a few OTA stations using MPEG-4, most TVs, but not all can view it. Any TiVo other than the TCD648 (OLED Series 3) should be fine with MPEG-4 OTA.


    It would be in a stat mux. You can bias a stat mux, but it still changes the bandwidth allocation multiple times a second. You also keep using GB/hr which is the wrong unit of measure. Broadcast is measured in mbps. You can roughly convert in your head, 10mbps is roughly 5GB/hour, but not quite because you have 3600 seconds in an hour but 8000MB in a GB (technically 8096 if you want to be really precise).

    Again, I am not aware of a single station in the US broadcasting 19.3mbps on a single video channel, but that's what a physical channel broadcasts. That's the spec. The highest currently in the US is 16-17mbps, and most are 6-12mbps. At 6mbps, they can barely squeeze 2HD/4SD, which is currently the maximum any encoder manufacturer or broadcast engineer is going to claim they can do, although several channels don't care at all about quality and now have 3HD channels or more on a single broadcast, which pretty much guarantees awful VQ. You might be able to get away with 3x720p with no SD subchannels, as 720p compresses easier than 1080i.

    If you can ID all channels that are broadcasting on that physical channel and record them all at the exact same time, you can figure out if they have "dead" or "wasted" bandwidth, but I doubt it with that channel load. PBS is the most likely culprit for "dead" bandwidth due to having a limited budget for dialing in their encoders properly. WEDH-DT had "dead" bandwidth for quite some time, and they are in a relatively major market (30ish DMA) that's adjacent to WGBH, WGBY, and WNET.

    I think we need to check your definition of "good".

    That's 14.2mbps which is quite respectable. NBC has gotten the most aggressive with channel sharing in a lot of markets (thanks, Comcast). You can get stunning HD these days out of 12mbps or higher, acceptable out of 8mbps or higher. Bitrate alone doesn't mean good VQ though, back in the MPEG-2 cable days I had a Comcast channel at 17mbps that looked mediocre at best.

    Total apples and oranges. Supposedly HEVC is twice as efficient as H.264, which is twice as efficient as MPEG-2, so the 8mbps bar we hit today should eventually get down to 2mbps HEVC. However, even that's apples and oranges, as MPEG-2 encoding is very mature, HEVC is not. Further, the larger a stat mux group, the better results you can get, so using a mix of HD and SD channels on ATSC 3.0 will result in very efficient stat muxing.

    Lastly, comparing Hulu (unless it's the live TV) to broadcast is again apples and oranges, since that stuff is offline encoded, which is always more efficient than online encoding. Even for streaming TV, they can use a VBR encode to a certain extent in order to save bandwidth while still offering better quality when it needs more bandwidth, and it's not up against one or two or three other channels in a stat mux, it's just some sort of constrained VBR. Netflix is notorious for really incredible encoding at remarkably low bitrates, because they throw both a massive amount of horsepower at their encoding, and a lot of intelligence. They're entirely offline, so they can have several racks of machines working on different resolutions and codecs of the same file at the same time, and they can spend minutes encoding every second of video. They're using some sort of narrow AI to detect the type of video or content in each frame of sequence of video, and then adjust the encoding down to match the content so that they use just enough bitrate at any given time to create stunning results. Broadcast doesn't have any of that relative luxury.
     

Share This Page

spam firewall

Advertisements