TiVo Community
TiVo Community
TiVo Community
Go Back   TiVo Community > Main TiVo Forums > TiVo Premiere DVRs
TiVo Community
Reply
Forum Jump
 
Thread Tools
Old 06-01-2012, 04:05 PM   #1
brentil
Registered User
 
brentil's Avatar
 
Join Date: Sep 2011
Posts: 309
TiVo Premiere Benchmarking

This thread is a breakout from my Performance thread that was more of a discussion thread than a details thread which is what this one will focus on. This thread will cover a variety of different benchmarks run on my TiVo Premiere to try to glean performance enhancements or regressions.

My testing process includes running each test a minimum of 3 times excluding outliers that are obvious flukes. The times are averaged together then compared against other runs.

Completed Benchmarks
Upcoming Benchmarks
  • TiVo 20.2.1.1 DNS benchmarks

__________________

  • To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

  • To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

Last edited by brentil : 06-01-2012 at 09:27 PM.
brentil is offline   Reply With Quote
Old 06-01-2012, 04:06 PM   #2
brentil
Registered User
 
brentil's Avatar
 
Join Date: Sep 2011
Posts: 309
TiVo 14.8c vs 14.9.2.2 benchmarks

All tests were performed a minimum of 3 times, most were done 5. All runs are averaged while the outliers are ignored as is initial loads unless noted. Timing was done using a digital stopwatch and values were typically rounded to whole seconds unless I felt the 0.5s value was needed to indicate a feeling of being that bit faster.

This image contains the data values in seconds comparing the previous 14.8c versus the new 14.9.2.2 along with the % difference.



This image is a graph of the % increase showing the overall trend in increases.



Aggregating this data ignoring the 0% values shows an average gain of 46% or nearly a 50% performance gain in the TiVo Premiere non-Elite with the new software that contains the new HDUI/Flash & 2nd core enabled. This is definitely a solid performance gain from primarily the 2nd core without a complete HDUI rewrite with a new multi-threaded Flash runtime.

To me what this shows is that the main system was CPU limited for a majority of background tasks the UI relies on as the UI itself probably was as well. Providing true hardware threading the systems behind the UI (IO, database, network, etc) can now run more efficiently and finally concurrently.

Oddly the moving of a Season Pass form bottom to top decreased in speed, where I had expected to see a solid improvement here. This task is possibly one of the most DB intensive in my opinion as it requires a significant amount of reorganization and validation. My guess would be the process needs to be optimized for more efficient memory/cpu use due to heavy thrashing.

In terms of software improvements this is super solid gains. I can't wait to see what the v16 software and v2.0 UI bring to us! (Note: at the tiem we thought it might be called v16 but in reality it is what we now know as v20.)
__________________

  • To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

  • To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

Last edited by brentil : 06-01-2012 at 04:12 PM.
brentil is offline   Reply With Quote
Old 06-01-2012, 09:18 PM   #3
mattack
Registered User
 
Join Date: Apr 2001
Location: sunnyvale
Posts: 16,924
I don't need/want all of the tests redone, but do you have any idea about the Elite?

and exactly what is "boot time" until? Until you can get to Now Playing? Until it starts a recording in progress (i.e. gets all of the cablecard data)?

I know, we shouldn't care about boot time, but ESPECIALLY when "playing" with Tivos (e.g. installing bigger hard drives), or power outages (get a UPS, blah blah), it's an excruciatingly long wait for ~10 minutes for my Tivo HD to get back to recording again.. (I do have a 'bad' drive software wise that was rebooting for a long time.. Thought it was bad hardware and/or hard drive itself, but now I'm mostly convinced that the drive is somewhat scrambled since it gets "confused" and loses the Now Playing list.. Putting the ORIG drive back in works fine.. I should have done it last weekend, but I mean to copy the 'bad' drive to the new 2 TB drive I got and let ANOTHER TivoHD do the fsck on it.. I know you can't play recordings from one tivo on another, but I'm hoping that at that level, it doesn't know, and I don't have to go without my Tivo for a day..)
mattack is offline   Reply With Quote
Old 06-01-2012, 09:26 PM   #4
brentil
Registered User
 
brentil's Avatar
 
Join Date: Sep 2011
Posts: 309
TiVo 14.9.2.2 vs 20.2.1.1 benchmarks

I want to point out before you actually look at the values below that most of these items should be classified as MOE (Margin Of Error) and should therefore be considered 1:1 realistically. Getting down to the difference between 1s & 2s could be easily explained as my finger hitting the stopwatch early on my last set of benchmarks and hitting it late on these when in reality they're the same set of values. There are 3 things I think you should take away from these values.
  1. That 20.2.1.1 is equivilant to 14.9 in terms of speed for the most part in relation to 14.8 as their base comparison. We will probably continue to see more new features over performance gains.
  2. The new YouTube & Neflix apps are slow to start. This is not unexpected for the first iteration of these radically new designs. The TiVo versions are now much more on par with other system's versions while still using a much older/slower hardware subsystem comparatively to a Roku/Phone/Xbox.
  3. There seems to be a good bit of caching going on now. Looking at the benchmarks you will notice I put 0s in some fields and I really mean 0s. There were items that once loaded were instantly there when you went back to them.



__________________

  • To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

  • To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
brentil is offline   Reply With Quote
Old 06-01-2012, 09:34 PM   #5
brentil
Registered User
 
brentil's Avatar
 
Join Date: Sep 2011
Posts: 309
Quote:
Originally Posted by mattack View Post
I don't need/want all of the tests redone, but do you have any idea about the Elite?

and exactly what is "boot time" until? Until you can get to Now Playing? Until it starts a recording in progress (i.e. gets all of the cablecard data)?

I know, we shouldn't care about boot time, but ESPECIALLY when "playing" with Tivos (e.g. installing bigger hard drives), or power outages (get a UPS, blah blah), it's an excruciatingly long wait for ~10 minutes for my Tivo HD to get back to recording again..

...
The Elite/XL4 is identical enough in terms of CPU/RAM that you should not see a difference.

The boot time is the measure of time from a cold boot to the TiVo video showing (it begins to show). Measuring the time it takes after that is not a reliably measurable or comparable value since it can be impacted by too many things like CCs, over the air, SDV, # of tuners, etc.

The reason boot time is there and is the first in the list is because it was the very first thing we saw running faster when the Elite came out because its OS enabled the 2nd CPU which we had never had before.
__________________

  • To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

  • To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
brentil is offline   Reply With Quote
Old 06-01-2012, 09:37 PM   #6
moyekj
Registered User
 
Join Date: Jan 2006
Location: Mission Viejo, CA
Posts: 9,063
Having 2 additional tuners & 4 programs buffering to disk all the time I think does slow down the Elite somewhat compared to 2 tuner Premier units.
__________________
Roamio Pro, Elite, Premiere
Cox - Motorola CableCards & TAs
Slingbox 350 via TiVo Mini & TiVo Stream for remote viewing

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
moyekj is offline   Reply With Quote
Old 06-01-2012, 11:51 PM   #7
ig88
Registered User
 
ig88's Avatar
 
Join Date: Feb 2003
Posts: 51
Wouldn't it be nice if we never had to worry about benchmarks?
__________________
Series I
Series II
Series III
Series IV XL
OS X, iPad, iPad2
ig88 is offline   Reply With Quote
Old 06-02-2012, 01:58 AM   #8
BigJimOutlaw
Registered User
 
Join Date: Mar 2004
Posts: 1,526
I thought my Premiere was still a bit sluggish until I bought an Elite. I WISH there was no difference, but it's fairly obvious. The hardware is largely identical but the Elite just doesn't keep up with my remote presses as well as the Premiere does.

In theory the hard drive is supposed to be able to keep up with the extra tuners, but I wonder how likely it is that the extra overhead is slogging it down. Or if not, what else it could possibly be.
BigJimOutlaw is online now   Reply With Quote
Old 06-02-2012, 09:33 AM   #9
brentil
Registered User
 
brentil's Avatar
 
Join Date: Sep 2011
Posts: 309
Well instead of complaining start benchmarking! Since all I have is a basic Premiere that is all I can do tests on, so it would take one of you doing benchmarks to get an idea if there are any glaring differences or not.


Quote:
Originally Posted by ig88 View Post
Wouldn't it be nice if we never had to worry about benchmarks?
Benchmarks are like Jello pudding pops, there's always room for more. Because without benchmarks you never know the truth of a change.
__________________

  • To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

  • To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
brentil is offline   Reply With Quote
Old 06-04-2012, 11:07 AM   #10
ghuido
Registered User
 
Join Date: May 2007
Posts: 311
Woah ... surprised on the season pass 20.2 time decreases ....
ghuido is offline   Reply With Quote
Old 06-04-2012, 12:02 PM   #11
brentil
Registered User
 
brentil's Avatar
 
Join Date: Sep 2011
Posts: 309
Quote:
Originally Posted by ghuido View Post
Woah ... surprised on the season pass 20.2 time decreases ....
The problem with that test is it can be very subjective. As moving the show then also re-evaluates all of your upcoming recordings for the next 2 weeks dealing with conflicts and recording order. So the show I picked to move could in itself cause the longer lag time because of how many upcoming shows it has, the number of conflicts it creates, the time of year so the number of showings vs reruns, and the other season passes I've added/removed since the last time I did it.

That's why a lot of these need to be taken with a grain of salt because there are a lot of error sources that can cause them to change between testing times. Realistically I needed to have done the tests right before an upgrade and then right after to minimize the number of changes. However... I complete forgot to do the 20.2 benchmarks when it came out and so we're skipping right to 20.2.1.1 months later.
__________________

  • To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

  • To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
brentil is offline   Reply With Quote
Reply
Forum Jump




Thread Tools


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Advertisements

TiVo Community
Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
vBulletin Skins by: Relivo Media

(C) 2013 Magenium Solutions - All Rights Reserved. No information may be posted elsewhere without written permission.
TiVoŽ is a registered trademark of TiVo Inc. This site is not owned or operated by TiVo Inc.
All times are GMT -5. The time now is 03:01 PM.
OUR NETWORK: MyOpenRouter | TechLore | SansaCommunity | RoboCommunity | MediaSmart Home | Explore3DTV | Dijit Community | DVR Playground |