TiVo Community
TiVo Community
TiVo Community
Go Back   TiVo Community > Main TiVo Forums > TiVo Home Media Features & TiVoToGo
TiVo Community
Reply
Forum Jump
 
Thread Tools
Old 01-10-2014, 10:39 AM   #31
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,879
MPEG compression is a "lossy" data compression method. Algorithms such as Lempel-Ziv-Welch closely examine a data stream looking for ways to perfectly represent the stream using different symbols from the original in order to reduce the volume of data required to reproduce the information. Depending greatly on the actual symbols used to represent the original stream, LZW-based and similar compression utilities can generally be expected to reduce the size of a file by about a factor of two, and the original data stream can be recovered from the new, smaller file perfectly without the loss of a single bit. That is a major reason why zip, arj, gzip, etc. are ubiquitously used to transfer programs and data around the internet. These algorithms could be applied as well to photos, music, and video, but doing so would only reduce a 3 Gbps HD video stream to about 1.5 Gbps. That is not anywhere nearly good enough to make storing and transferring digital music, photos, and video on the internet or over the air, for that matter, practical.

Fortunately, if we analyze things like photos and video in a different way, there tend to be large areas that are approximately repetitive. In addition, significant amounts of both visual and audible data can be discarded without the human brain being able to tell the difference, or at least not very much or very often. Because of this, we can employ "lossy" compression algorithms to translate the music, photos, or video into much lower bandwidth bitstreams or much smaller files. The recovered output is not at all identical to the original data, but it is close enough for our purposes.

The question is, "How much data do we throw away?" The answer to that depends on several variables. The first is the nature of the original data. Video (sans the audio) tends to lend itself to greater compression than any other data of which I am aware. Large areas of the screen may be approximately duplicated from one page of video to the next. One relies on the fact the video only changes a little bit from one frame to the next, so the data stream reproduces the picture in full, but then subsequent frames are only represented in the data stream by the changes from the previous page, not the entire page itself. By choosing prudently what we consider enough of a difference how much of a change needs to be represented in the picture at all, we can reduce the data needed to reproduce the next page. Where the line between "prudent" and "imprudent" lies depends upon how much degradation of picture quality we are willing to accept. If one has lower standards, the compression can be greater. As Soapm mentioned, this may depend on the quantity of alcohol one has consumed.

Another variable is the amount of processing power and time available. Analyzing the data more extensively can produce a much tighter bitstream for qualitatively virtually the same results. If one has a monster CPU array with unlimited time, comparatively rather small bitstreams can be created that produce very pleasing results. If one must compress the data in real time with a reasonably economical processor, the stream is going to require rather more bandwidth.

Of course, the actual content itself has much to do with it. Some video lends itself to greater compression and other video to less without unpleasant artifacts in the output. Other than changing the resolution, the individual doing the authoring doesn't really have much control over the content, but the other factors are much more within his control. That said, most people are gong to want (or absolutely need) to "set it and forget it" when it comes to the compression parameters, and may have rather limited patience in terms of the amount of time required to get things done. (Some compression utilities also have very limited amounts of control offered to the user.) That being the case, rather than recode every movie or at least part of it, observe the PQ, and adjust the compression parameters, one usually just quickly finds a set of compression parameters that almost always produced good results and then sticks with those parameters. That is fine for most of us, but if space is a major concern, one may wish to fiddle with the compression on an individual basis to produce the tightest practical bitstream.

So yes, 2 hours or even significantly more can fit on a DVD, but one must make compromises in one or more areas to achieve it.

Last edited by lrhorer : 01-10-2014 at 10:53 AM.
lrhorer is offline   Reply With Quote
Old 01-10-2014, 08:33 PM   #32
Dan203
Super Moderator
 
Dan203's Avatar
 
Join Date: Apr 2000
Location: Nevada
Posts: 25,581
Quote:
Originally Posted by lrhorer View Post
doing so would only reduce a 3 Gbps HD video stream to about 1.5 Gbps.
Your calculation is a little off. 1920*1088*24*29.97 = 1.5Gbps and that's before chorma sub-sampling* is taken into account. In reality video is transmitted at 4:2:0, which means it's really 1920*1088*12*29.97 = 751Mbps. Throw in your 2:1 compression from LZW and you're down to 375Mbps. Still not practical to transmit, but significantly smaller then 1.5Gbps.

* Instead of RGB, where each pixel has 8 bits of Red/Green/Blue video uses YUV where the image is separated into one brightness value (luma) and two color values (chroma). In standard broadcast the luma is full resolution, but both of the chroma signals are only 1/4 resolution. So basically each block of 4 pixels is the same color but each individual pixel varies in brightness based on the luma signal.
__________________
Dan Haddix
Super Moderator
Developer for VideoReDo
Dan203 is offline   Reply With Quote
Old 01-11-2014, 03:41 PM   #33
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,879
Quote:
Originally Posted by Dan203 View Post
Your calculation is a little off. 1920*1088*24*29.97 = 1.5Gbps and that's before chorma sub-sampling* is taken into account. In reality video is transmitted at 4:2:0, which means it's really 1920*1088*12*29.97 = 751Mbps. Throw in your 2:1 compression from LZW and you're down to 375Mbps. Still not practical to transmit, but significantly smaller then 1.5Gbps.
That assumes 1920*1088*24*30. Transmitting 1920*1088*24*60 = 3 Gbps, although of course you are correct about employing YUV coding rather than RGB - but that is just another means of compression. Unless I am much mistaken, 3 Gbps is the maximum bit rate specification for HDMI 1.0. HDMI 2.0 I vaguely seem to recall may even be more. In any case, however, even 100 Mbps would be rather impractical to store and transmit, and it would most definitely have been so when the specs for Blu-Ray and HD QAM were created. In another 5 years, 100Mbps video bitstreams might be very practical, although I suspect unnecessary. 1080p x 60 video is getting to be close to the limit where any visual artifacts (other than compression artifacts, of course) are no longer discernable. Quadrupling the resolution to around 2160p should take the bandwidth of a "perfect" compressed video to about 60 - 80 Mbps, and at that point I don't think further improvement will be of significant value, regardless of picture size. Not, mind you, that I even believe 1080p video will be abandoned for 2160p in the next 5 or even 10 years.
lrhorer is offline   Reply With Quote
Old 01-12-2014, 03:54 PM   #34
Dan203
Super Moderator
 
Dan203's Avatar
 
Join Date: Apr 2000
Location: Nevada
Posts: 25,581
No one transmits 1080p@60fps. 1080i@29.97fps is the max for broadcast. Most broadcasters store the "masters" as I frame only MPEG-2 at ~50Mbps with 4:2:2 YUV. I,m sure the production companies save their shows as film, tapes, or lossless digital, but the broadcasters and sindicators have too much stuff and just can't store files that big. One of our biggest business clients is a sindicator and they still get a lot of stuff on tape and have to digitize it themselves. The broadcast sector tends to be pretty far behind technologically compared to even what we can do with our desktop PCs. You'd also be amazed how many times a show is recoded before it hits your TV just to accommodate old equipment. I always get a little chuckle when someone says they don't want to recode to H.264 because they want to preserve the "pristine original". That video has already been recoded like a half dozen, or more, times before it even reaches their DVR.
__________________
Dan Haddix
Super Moderator
Developer for VideoReDo
Dan203 is offline   Reply With Quote
Old 01-12-2014, 05:24 PM   #35
gonzotek
tivo_xml developer
 
gonzotek's Avatar
 
Join Date: Sep 2004
Location: Outside Phildadelphia
Posts: 2,233
Quote:
Originally Posted by Dan203 View Post
I always get a little chuckle when someone says they don't want to recode to H.264 because they want to preserve the "pristine original". That video has already been recoded like a half dozen, or more, times before it even reaches their DVR.
Yeah, but why add one more, if you have the space available to store the mpeg2? Recoding to H.264 is still computationally expensive. I do it - for device compatibility reasons, but not on videos I expect to only send back to the tivo. And I treat the resultant output as disposable, keeping the 'original' copy, since that's the best I have access to and it's just easier not to bother. At the rate I personally transfer and store content, disk space/cost simply isn't an issue.
__________________
Follow @pytivo on Twitter for project updates and more!
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
A Web app for Roku Remote Control
gonzotek is offline   Reply With Quote
Old 01-13-2014, 09:28 PM   #36
Dan203
Super Moderator
 
Dan203's Avatar
 
Join Date: Apr 2000
Location: Nevada
Posts: 25,581
Yeah but we get people who complain about the one GOP we have to recode around an edit point. They'll actually do I frame edits just to avoid a dozen frames from being recoded.
__________________
Dan Haddix
Super Moderator
Developer for VideoReDo
Dan203 is offline   Reply With Quote
Old 01-13-2014, 11:09 PM   #37
moyekj
Registered User
 
Join Date: Jan 2006
Location: Mission Viejo, CA
Posts: 9,253
Quote:
Originally Posted by Dan203 View Post
Yeah but we get people who complain about the one GOP we have to recode around an edit point. They'll actually do I frame edits just to avoid a dozen frames from being recoded.
Guilty as charged! I love how easy it is to set cut points on I-Frames in VRD.
__________________
Roamio Pro, Elite, Premiere
Cox - Motorola CableCards & TAs
Slingbox 350 via TiVo Mini & TiVo Stream for remote viewing

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
moyekj is offline   Reply With Quote
Reply
Forum Jump




Thread Tools


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Advertisements

TiVo Community
Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
vBulletin Skins by: Relivo Media

(C) 2013 Magenium Solutions - All Rights Reserved. No information may be posted elsewhere without written permission.
TiVoŽ is a registered trademark of TiVo Inc. This site is not owned or operated by TiVo Inc.
All times are GMT -5. The time now is 09:21 PM.
OUR NETWORK: MyOpenRouter | TechLore | SansaCommunity | RoboCommunity | MediaSmart Home | Explore3DTV | Dijit Community | DVR Playground |