TiVo Community
TiVo Community
TiVo Community
Go Back   TiVo Community > Underground Playground > TiVo Upgrade Center
TiVo Community
Reply
Forum Jump
 
Thread Tools
Old 08-25-2014, 08:00 PM   #61
jmbach
Registered User
 
jmbach's Avatar
 
Join Date: Jan 2009
Posts: 836
To many steps before testing the results. I have been guilty of that before.
If it is the same as your previous unit, use that original disk for the image and use the copy and rearrange scripts to place the image on the units drive. Then do a C&DE to marry the drive to the unit. If there is room left, then expand.
__________________
"Delay is preferable to error" - Thomas Jefferson
"If I have seen further it is by standing on the shoulders of Giants" - Sir Isaac Newton
jmbach is offline   Reply With Quote
Old 08-31-2014, 01:16 PM   #62
HerronScott
Registered User
 
Join Date: Jan 2002
Location: Staunton, VA
Posts: 990
Jim,

So I'm looking to go back and align my son's HD which I had upgraded from the original 160GB drive to a 2TB drive using comer's JMFS tool. I noticed that its partition order appears to be the default with just an added large 14th MFS media region 3 partition.

HD original 1,13,2,3,4,5,6,7,8,9,10,12,11
HD upgraded with JMFS 1,13,2,3,4,5,6,7,8,9,10,12,11,14

S3 OLED original 1,13,2,3,4,5,6,7,8,9,10,12,11
S3 OLED upgraded with WinMFS 1,13,11,2,3,4,5,6,7,8,9,10,12,14,15

I'm curious about 2 things when compared to the partitions on my 2 S3 OLED's and wasn't sure if you knew the answers.

First would be it looks like the original S3 OLED partition order was the same as the HD so I'm assuming that when I used WinMFS to upgrade those to 1TB drives originally that it moved partition 11. I'm curious as to why it made the partition order change.

Second, I'm curious about WinMFS adding another MFS application region to go along with the added MFS media region when it expands a drive (although a small one) where JMFS did not. I'm curious about the reasons and wonder if there are any reasons with the expanded HD not having it.

Scott
__________________
  1. S3 HD - 2TB - Lifetime (son's)
  2. S3 OLED - 2TB - Lifetime
  3. S3 OLED - 2TB - Lifetime
  4. S1 Phillips - 120GB - Lifetime transferred to S3
  5. S1 Sony - 30GB - Lifetime transferred to S3
HerronScott is offline   Reply With Quote
Old 08-31-2014, 05:24 PM   #63
jmbach
Registered User
 
jmbach's Avatar
 
Join Date: Jan 2009
Posts: 836
Scott,

Good questions. As far as WinMFS is concerned. It has an option when you copy the partitions to the new drive to optimize the copy. It has been a while since I played with WinMFS, but it moves the last partition to the front to try to keep the OS and MFS app partitions closer to the center of the drive for an optimized layout. At least that is the thought.

JMFS actually creates two partitions, an app and a media partition and then coalesces it into one partition. This is because the Premiere units have a partition 14 already and since the MFS header only allows up to sda15 and any more are ignored, so JMFS has to coalesce the added partition pair so the unit will recognize and use the added space.

Jim
__________________
"Delay is preferable to error" - Thomas Jefferson
"If I have seen further it is by standing on the shoulders of Giants" - Sir Isaac Newton

Last edited by jmbach : 08-31-2014 at 05:36 PM.
jmbach is offline   Reply With Quote
Old 08-31-2014, 11:06 PM   #64
nooneuknow
TiVo User Since 2007
 
Join Date: Feb 2011
Location: Cox Cable Market, NV
Posts: 3,099
WinMFS, if used to image from a file, will mess with the partition layout, regardless of the "optimize" option being checked. If actually uses a stock S2 layout if not optimized, and will make a S2 "optimized" layout if you check that option. This may not be 100% correct. It's what I was told when I inquired about why WinMFS wouldn't just restore the backup file to the drive as-is (as-was) for my TiVo HDs.

This is why I started only using WinMFS for copying from source to target drives, which IIRC doesn't mess with the drive layout. JMFS or any variant of dd or dd rescue will also leave the partitions in the layout they started out as. I would then use WinMFS only for operations that didn't have the the optimize function as an option (like to "supersize").

If all the images you have are molested by WinMFS, you either should get an unmolested image, or use instant cake to build a 1TB (max) drive, as a foundation to start with.

jmbach gave a good description of the specifics of what WinMFS tries to do, and why, as well as how JMFS operates.
__________________
Cisco tuning adapters should never be used inline (using the TA coax OUT port) to connect a TiVo, if MoCA is in use. Use a splitter w/PoE filter on leg to TA, use other leg for the TiVo. Enjoy!

Last edited by nooneuknow : 09-01-2014 at 01:16 AM. Reason: specifics
nooneuknow is offline   Reply With Quote
Old 09-02-2014, 12:15 AM   #65
HerronScott
Registered User
 
Join Date: Jan 2002
Location: Staunton, VA
Posts: 990
Quote:
Originally Posted by nooneuknow View Post
WinMFS, if used to image from a file, will mess with the partition layout, regardless of the "optimize" option being checked. If actually uses a stock S2 layout if not optimized, and will make a S2 "optimized" layout if you check that option. This may not be 100% correct. It's what I was told when I inquired about why WinMFS wouldn't just restore the backup file to the drive as-is (as-was) for my TiVo HDs.

This is why I started only using WinMFS for copying from source to target drives, which IIRC doesn't mess with the drive layout.
The 1TB drives were drive to drive copies from the original 250GB drives using WinMFS back in 2008 and I would not have selected optimized format unless it's the default.

Quote:
Originally Posted by nooneuknow View Post
If all the images you have are molested by WinMFS, you either should get an unmolested image, or use instant cake to build a 1TB (max) drive, as a foundation to start with.
Well given that the current layout on both S3 OLED's has functioned well since 2008 including a number of version upgrades from the 9.4 that was on them at the time of the upgrade to the current 11.0, I'm probably not going to worry about starting from scratch at this point.

Scott
__________________
  1. S3 HD - 2TB - Lifetime (son's)
  2. S3 OLED - 2TB - Lifetime
  3. S3 OLED - 2TB - Lifetime
  4. S1 Phillips - 120GB - Lifetime transferred to S3
  5. S1 Sony - 30GB - Lifetime transferred to S3
HerronScott is offline   Reply With Quote
Old 09-02-2014, 12:26 AM   #66
HerronScott
Registered User
 
Join Date: Jan 2002
Location: Staunton, VA
Posts: 990
I completed the alignment on my son's HD today keeping the partition layout the same and using the script to first copy the drive to a spare 2TB drive which aligned and resized the partitions and then using the script to copy them back to the original drive (after verifying the temp drive booted and worked successfully).

It certainly appeared that the HD is noticeably more responsive to both my son and I after the alignment in displaying large lists of shows and in scrolling through upcoming shows using the channel up/down buttons while viewing the show information. I had really noticed this hesitation/slowness when I had originally upgraded it to the 2TB drive but wasn't sure if it was related to it being an HD (versus S3 OLED) or the larger drive or the advanced format.

Scott
__________________
  1. S3 HD - 2TB - Lifetime (son's)
  2. S3 OLED - 2TB - Lifetime
  3. S3 OLED - 2TB - Lifetime
  4. S1 Phillips - 120GB - Lifetime transferred to S3
  5. S1 Sony - 30GB - Lifetime transferred to S3
HerronScott is offline   Reply With Quote
Old 09-02-2014, 02:57 AM   #67
nooneuknow
TiVo User Since 2007
 
Join Date: Feb 2011
Location: Cox Cable Market, NV
Posts: 3,099
Quote:
Originally Posted by HerronScott View Post
I would not have selected optimized format unless it's the default.
I think it is the default setting (checked).

But you seem to be done, and happy with your results. So, I'll leave the subject alone, until the next time it comes up.
__________________
Cisco tuning adapters should never be used inline (using the TA coax OUT port) to connect a TiVo, if MoCA is in use. Use a splitter w/PoE filter on leg to TA, use other leg for the TiVo. Enjoy!
nooneuknow is offline   Reply With Quote
Old 09-04-2014, 01:38 AM   #68
telemark
Registered User
 
Join Date: Nov 2013
Posts: 812
Part 2

Blue pill folk might want to skip over this post, because there may be no way to fix.

Aligning the start of the partitions via the partition map is half the equation.
Better analogy, it's like the base of a pyramid.
[every layer needs to use 4k blocks, but also depends on the layer below to be correctly aligned first]

For illustration, If you're building a drive image from scratch, you'd make sure every partition started on a even boundry. When you format the filesystems next, you would specify to the formatter the minimum block size should be 4k as well. Then for every application that managed it's own data structures on that filesystem, you'd have to leave hints to it, hey, this drive doesn't like <4k so keep your back and forth chatter to units of 4k.

Since we already have an image made for us from Tivo, we're inheriting and then propagating wherever that data is already within a partition since we're doing perfect bit/byte level copies.

This would probably be set "wrong" if the image was built before 4k drives existed.

Checking an S4 320GB image for example I get:

Code:
# dumpe2fs /dev/loop4  | grep "Block size"
Block size:               1024 (root)

# dumpe2fs /dev/loop7  | grep "Block size" 
Block size:               1024 (root)

# dumpe2fs /dev/loop9  | grep "Block size" 
Block size:               4096 (var)

# dumpe2fs /dev/loop14  | grep "Block size" 
Block size:               4096 (sql db)
That's the filesystem, then the database would need to be checked as well. Might be in here somewhere:
https://bugzilla.mozilla.org/show_bug.cgi?id=416330

Does any of this matter? Maybe not, I depends how often data is read / written off-alignment vs on-alignment, and disk I/O is dominated by MFS video anyway so that performance would eclipse Linux operations. The Linux bits were easier to check though and every little bit helps in the end.

Last edited by telemark : 09-04-2014 at 02:52 AM.
telemark is offline   Reply With Quote
Old 09-04-2014, 04:33 PM   #69
nooneuknow
TiVo User Since 2007
 
Join Date: Feb 2011
Location: Cox Cable Market, NV
Posts: 3,099
Quote:
Originally Posted by telemark View Post
Does any of this matter? Maybe not, I depends how often data is read / written off-alignment vs on-alignment, and disk I/O is dominated by MFS video anyway so that performance would eclipse Linux operations. The Linux bits were easier to check though and every little bit helps in the end.
Even if it fails to produce a performance difference in use in a TiVo, being unaligned increases RMW (Read,Modify,Write) write operations. If everything about a drive, what is on it, and how the host device interacts with it made every read/write aligned, there would be no RMW write operations. Every unaligned write, resulting in it being a RMW, overwrites two whole adjacent underlying 4k physical sectors, if the write straddles two physical sectors, rather than one. It also increases the number of seek operations to complete the same operations.

If you are saying what I believe you are, you are correct. Aligning partition start boundaries is only half the equation, leaving the fact that TiVo writes in multiples of 512 byte sectors, rather than multiples of 4k.

The long writes, of long block sizes, of the type when writing the AV streams (which TiVo writes as regular data, rather than using the AV streaming feature), tends to only need the partition alignment, since it's running sequential long writes, that would straddle the physical adjacent consecutive sectors, anyway, as well as writing the whole 4k, anyway.

The partition alignment is even more critical for small writes, as in the things other than AV stream writes. Now factor in not writing in a minimum of 4k blocks, and things get worse.

The biggest inevitability many seem to completely overlook, is data fragmentation, and how it single-handedly creates an explosion of RMW operations, exponentially worsened if the boundary alignment is off.

Why do we care about how many RMW operations happen? It wears out the drive faster, even if your TiVo seems to be running at peak speed. Behind the scenes, inside the drive, the drive cache, with AF 512e (4k emulating 512) drives, is being used to hide the performance penalties. This is why drives with 32MB cache suddenly had 64MB cache. The extra was needed to mask the hobbled internal performance differences, caused by RWM operations. Masking them doesn't stop the drive from making excessive read/write/seek operations to deal with RMW operations.

There seems to be a consensus that modern drives are not holding up as well as many would expect. They should be getting more reliable, not just holding onto reliability, or losing it. RMW operations on platter drives have an effect similar to what happens inside SSDs when you run a disk defragmenter on them (which is a non-no, if you want them to last).

Now, take a standard "green" drive that comes with a built-in internal performance penalty (both in throughput and seeking performance), and use it in a DVR with 4 or 6 tuners. One might think that the aligned partitions in a Roamio will stop RMW operations from happening. This will only be true if/when TiVo stops treating AF 512e drives like native 512 byte sector drives. As soon as data fragmentation sets in, and writes can no longer be truly adjacent and sequential, the drive will start having to seek more, and this is what "wears out" a platter drive, more so than excessive rewrites, which also "wears out" sectors. Both types of "wear" are exacerbated by RMW operations.

I've posted this same info many times in the past, when AF drives hit the market. I was told a great many unkind things, and had to stop posting about it, unless somebody specifically asked about it. Even then, I was accused of over-thinking things, being overly-technical, being OCD, and inciting unfounded worries.

I've been watching for trends in drive image requests, especially those that were requested for use on a replacement drive. It seems like the 4 tuner Premieres have been "wearing out" at a pace which supports my early predictions of decreased drive life longevity.

Then again, when looking for patterns, you tend to find them, even if they aren't really there.

One thing I'd like to understand more about, is how, exactly, Seagate can claim their drives auto-align unaligned writes, eliminating the need to align them at all (partition-wise). I can't seem to find any published data to support this claim.

The other thing I'd like to understand better, is the first logical sector offset on drives, and if this has anything to do with AF, and what it's purpose is. I've googled to extremes on this matter, and can't find anything other than the fact that the offset exists. Does this mean a drive with a first logical sector offset of 16384 starts writing (block zero) at 16384 based on 512 byte logical units? Does it serve a purpose we should know about, and does the offset value mean anything to us when choosing a drive (one offset better than another)?
__________________
Cisco tuning adapters should never be used inline (using the TA coax OUT port) to connect a TiVo, if MoCA is in use. Use a splitter w/PoE filter on leg to TA, use other leg for the TiVo. Enjoy!
nooneuknow is offline   Reply With Quote
Old 09-05-2014, 07:40 AM   #70
ggieseke
Registered User
 
Join Date: May 2008
Posts: 3,046
On Series 4s the root partitions use 1K blocks. Series 3 and earlier models used 1K blocks on the /var partition as well. I don't think there's any practical way around it, but I doubt that it matters very much.

Aligning the partitions to 4K start addresses is probably as close as we're going to get to perfection.
ggieseke is offline   Reply With Quote
Old 09-05-2014, 05:37 PM   #71
nooneuknow
TiVo User Since 2007
 
Join Date: Feb 2011
Location: Cox Cable Market, NV
Posts: 3,099
Quote:
Originally Posted by ggieseke View Post
On Series 4s the root partitions use 1K blocks. Series 3 and earlier models used 1K blocks on the /var partition as well. I don't think there's any practical way around it, but I doubt that it matters very much.

Aligning the partitions to 4K start addresses is probably as close as we're going to get to perfection.
I'm inclined to agree. At minimum, I hope that some who figure aligning partitions is a waste of time (since they can't tell the difference, with the RMW penalties being masked by the drive), might see there's a bigger picture, and at least use aligned start boundaries, if they want to do what they can, to hopefully increase the drive lifespan, relative to what it would be without aligning them.

With the double-whammy of how much harder the drive has to work internally, just due to being an AF 512e drive, then compounded the additional TB/yr workload in, to support more tuners, it would seem entirely logical to say that those looking for a replacement drive, might want to think twice before picking a lowest-price drive, with a lesser warranty, especially when the difference can be as low as $5-$15.

As telemark has suggested (to me), it's probably best to lead with the warranty, then try to briefly explain why that warranty might pay-out, on drives that people often assume will only fail due to defects, or old-age (which is now more relative to other factors), when people ask "which drive?" or "will this drive work?". At least with the Roamio, this is as much as needs to be said, since manually aligning the partitions is not required, and the Idle Mode 3 Timer issue is a non-issue for Roamios.
__________________
Cisco tuning adapters should never be used inline (using the TA coax OUT port) to connect a TiVo, if MoCA is in use. Use a splitter w/PoE filter on leg to TA, use other leg for the TiVo. Enjoy!
nooneuknow is offline   Reply With Quote
Old 09-06-2014, 09:55 AM   #72
HerronScott
Registered User
 
Join Date: Jan 2002
Location: Staunton, VA
Posts: 990
Quote:
Originally Posted by nooneuknow View Post
I'm inclined to agree. At minimum, I hope that some who figure aligning partitions is a waste of time (since they can't tell the difference, with the RMW penalties being masked by the drive), might see there's a bigger picture, and at least use aligned start boundaries, if they want to do what they can, to hopefully increase the drive lifespan, relative to what it would be without aligning them.
As I mentioned above, on the HD that we went back and aligned, it was noticeably faster in certain activities. This is certainly one of the reasons that I wanted to address this before I upgraded my 2 S3 OLEDs. The other reason was possible impacts of misalignment on drive longevity. The 1TB drives that I had initially upgraded to were 6 years old and still working fine (and still passed WD's tests) and I'd really like to see the 2TB drives last just as long. As an aside news that Comcast upgraded Augusta, GA to MPEG4 are disappointing for the life expectancy before we need to upgrade but I can only hope that since we're a small town of 25,000 that it will be a long while before that happens here.

Regarding early drive failures, it does tend to catch your eye when someone reports that their 2 year old 2TB drive has failed already.

Scott
__________________
  1. S3 HD - 2TB - Lifetime (son's)
  2. S3 OLED - 2TB - Lifetime
  3. S3 OLED - 2TB - Lifetime
  4. S1 Phillips - 120GB - Lifetime transferred to S3
  5. S1 Sony - 30GB - Lifetime transferred to S3
HerronScott is offline   Reply With Quote
Old 09-06-2014, 03:47 PM   #73
nooneuknow
TiVo User Since 2007
 
Join Date: Feb 2011
Location: Cox Cable Market, NV
Posts: 3,099
Quote:
Originally Posted by HerronScott View Post
As I mentioned above, on the HD that we went back and aligned, it was noticeably faster in certain activities. This is certainly one of the reasons that I wanted to address this before I upgraded my 2 S3 OLEDs. The other reason was possible impacts of misalignment on drive longevity. The 1TB drives that I had initially upgraded to were 6 years old and still working fine (and still passed WD's tests) and I'd really like to see the 2TB drives last just as long. As an aside news that Comcast upgraded Augusta, GA to MPEG4 are disappointing for the life expectancy before we need to upgrade but I can only hope that since we're a small town of 25,000 that it will be a long while before that happens here.

Regarding early drive failures, it does tend to catch your eye when someone reports that their 2 year old 2TB drive has failed already.

Scott
I agree with you.

I was addressing the large number of pre-existing posts around the forum, where people report that their is no performance difference (even years later), sometimes complete with allegations that those wanting to align TiVo drives, or those considering buying a pre-imaged & aligned drive from WK or DVR_DUDE, are a bunch of OCD worry-warts, creating concern over things that they allege are of no concern, and writing alignment off as having no relevance for a TiVo drive, whatsoever.

Since I was one of the original ones to have concerns about this, I'm the one many of the aforementioned claims of having unfounded concerns are aimed at. So, I can't help but recall getting slammed and snarked at over the subject matter. This is why I've been watching what images are being requested, the specifics of why they need the image, what model TiVo it is for, and how long it took before they needed an image. The trends do seem to finally be revealing themselves. Premieres with 4 tuners, that have been in use for 2 years, seem to be eating drives, almost exactly as I predicted.

I usually prefer that I'm wrong, when I make such predictions, FWIW.
__________________
Cisco tuning adapters should never be used inline (using the TA coax OUT port) to connect a TiVo, if MoCA is in use. Use a splitter w/PoE filter on leg to TA, use other leg for the TiVo. Enjoy!
nooneuknow is offline   Reply With Quote
Reply
Forum Jump




Thread Tools


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Advertisements

TiVo Community
Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
vBulletin Skins by: Relivo Media

(C) 2013 Magenium Solutions - All Rights Reserved. No information may be posted elsewhere without written permission.
TiVoŽ is a registered trademark of TiVo Inc. This site is not owned or operated by TiVo Inc.
All times are GMT -5. The time now is 06:20 PM.
OUR NETWORK: MyOpenRouter | TechLore | SansaCommunity | RoboCommunity | MediaSmart Home | Explore3DTV | Dijit Community | DVR Playground |