TiVo Community
TiVo Community
TiVo Community
Go Back   TiVo Community > Main TiVo Forums > TiVo Series3 HDTV DVRs
TiVo Community
Reply
Forum Jump
 
Thread Tools
Old 08-07-2013, 11:59 AM   #9661
jmbach
Registered User
 
Join Date: Jan 2009
Posts: 566
Did you use WinMFS or MFSLive to do the expansion.
WinMFS has an option to add an external drive that might work better.
__________________
"Delay is preferable to error" - Thomas Jefferson
"If I have seen further it is by standing on the shoulders of Giants" - Sir Isaac Newton
jmbach is online now   Reply With Quote
Old 08-07-2013, 01:21 PM   #9662
Sevenfeet
Gentle Giant
 
Sevenfeet's Avatar
 
Join Date: Jun 2000
Location: Nashville, TN
Posts: 146
I used MFSlive. I couldn't get WinMFS to recognize the drive, even hung off my laptop's SATA port. I'm not sure what I did wrong. MFSlive (and Linux) did see the drive so I proceeded with that.
__________________
Rick
Sevenfeet is offline   Reply With Quote
Old 08-07-2013, 02:56 PM   #9663
jmbach
Registered User
 
Join Date: Jan 2009
Posts: 566
I have seen that happen if you don't run WinMFS with administrator privileges. Might try to divorce the extender drive (usually by unplugging it and booting the tivo) and then restored attach the drive to see if it will initialize and pair the drive up. If not, try WinMFS again with administration privileges to pair the drive. There used to be a KS code you can use to pair the drive. KS 62 was the code. It's still present but was made superfluous because the tivo is supposed to automatically initialize the extender drive once it is plugged in. Could not hurt to try if everything else fails.
__________________
"Delay is preferable to error" - Thomas Jefferson
"If I have seen further it is by standing on the shoulders of Giants" - Sir Isaac Newton
jmbach is online now   Reply With Quote
Old 08-07-2013, 07:47 PM   #9664
ThAbtO
TiVoholic by the bay
 
ThAbtO's Avatar
 
Join Date: Apr 2000
Location: SF Bay Area
Posts: 4,370
Quote:
Originally Posted by Sevenfeet View Post
I used MFSlive. I couldn't get WinMFS to recognize the drive, even hung off my laptop's SATA port. I'm not sure what I did wrong. MFSlive (and Linux) did see the drive so I proceeded with that.
Why would you need to add the expander drive when it will just add on a point to failure. If either drive should fail, your recordings will not be recoverable. You can only add the external with both hooked up to the computer and in WinMFS, with admin rights, I believe its MFSAdd.

With the 1 TB drive, you should have 157 HD Hrs of storage.
__________________
Series 2 (80 hrs,PLS, Retired)
TiVo HD 1TB (157 HD Hrs, PLS, "S3HD")
Roamio, Oh Roamio!

~~~~~

The TiVo Apocalypse begins on Sat 2013 Feb 16 00:00 GMT!
ThAbtO is offline   Reply With Quote
Old 08-07-2013, 08:28 PM   #9665
ThreeSoFar
FourNow...WaitFive
 
ThreeSoFar's Avatar
 
Join Date: May 2002
Location: Baltimore
Posts: 5,291
Quote:
Originally Posted by ThAbtO View Post
Why would you need to add the expander drive when it will just add on a point to failure. If either drive should fail, your recordings will not be recoverable. You can only add the external with both hooked up to the computer and in WinMFS, with admin rights, I believe its MFSAdd.

With the 1 TB drive, you should have 157 HD Hrs of storage.
This. Don't bother.
ThreeSoFar is offline   Reply With Quote
Old 09-06-2013, 11:12 PM   #9666
Keen
Registered User
 
Join Date: Aug 2009
Location: Minneapolis, MN
Posts: 141
Just used jmfs to move my Tivo to a new hard drive. My Tivo's been acting a little flakey for months, and it randomly rebooted and hung last week. So I grabbed a new drive and copied my existing drive over.

At 60MB/sec, the 2TB drive took a good long while to copy, which I left it to do overnight. In the morning, it had completed without seeing any errors on my old drive. Hopefully this fixes the infrequent hangs I've been seeing, and the hang during reboot.
Keen is offline   Reply With Quote
Old 09-08-2013, 04:38 AM   #9667
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
Quote:
Originally Posted by sbguy View Post
A TCD648250B "OLED" with a totally dead 1TB drive.
A new WD20EURS 2TB drive.
A truncated mfsbackup file from 3+ years ago when it had the original 250GB drive and old (8.x?) software.

I restored my old backup to the new 2TB drive, using mfsrestore, without expanding.
I installed this drive in the Tivo, let it run until it got a software update to 11.0k.
I installed it back into my host PC and used WinMFS to expand it using the MFSAdd menu and telling it to ignore the 1TB limit.

That did it.
So I've only read the FAQs on the first page, but was there some recent change in 11.0k...m for OLED S3 that allows for larger the 1TiB MFS partitions. That was the impression I got from the procedure above.
sfhub is offline   Reply With Quote
Old 09-08-2013, 05:14 AM   #9668
unitron
Registered User
 
unitron's Avatar
 
Join Date: Apr 2006
Location: semi-coastal NC
Posts: 12,420
Quote:
Originally Posted by sfhub View Post
So I've only read the FAQs on the first page, but was there some recent change in 11.0k...m for OLED S3 that allows for larger the 1TiB MFS partitions. That was the impression I got from the procedure above.
Yeah, it's been confirmed that you can use a 2TB in any S3, the 648, the 652, the 658, and WinMFS will do the job (but you have to do the expand as a separate step instead of accepting the offer at the end of the copy/restore, for some reason), no need for jmfs, provided you're installing an image that's updated at least to 11.0k (which has been around for a few years now, so it's rare not to have it), or provided you do not expand until letting it phone home and update to k (or maybe straight to m).

jmfs still needed for S4s, though, if you want to copy and expand.
__________________
(thisismysigfile)


"I am altering the deal. Pray I don't alter it any further."

Darth TiVo, 14 February, 2011
unitron is offline   Reply With Quote
Old 09-08-2013, 12:07 PM   #9669
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
Quote:
Originally Posted by unitron View Post
Yeah, it's been confirmed that you can use a 2TB in any S3, the 648, the 652, the 658, and WinMFS will do the job (but you have to do the expand as a separate step instead of accepting the offer at the end of the copy/restore, for some reason), no need for jmfs, provided you're installing an image that's updated at least to 11.0k (which has been around for a few years now, so it's rare not to have it), or provided you do not expand until letting it phone home and update to k (or maybe straight to m).
Awesome, thanks for the info.

I had the failing/bulging caps that I replaced only to find I also had a dying HD. I haven't been around these forums in a while so I had no idea 11.0k added that feature (updated kernel?) To give perspective, my last backup of the OS is 8.1.1.

So is this a reasonable way to upgrade now for 2TB OLED S3 (assuming old image)
  1. Prepare drive
    • DD original image
    • hdparm -M 128
    • wdidle3 (if necessary)
  2. allow OS to upgrade to 11.0k (11.0m in my case)
  3. winmfs mfsadd
  4. winmfs supersize
Is there a better procedure?

From reading past posts, you are pretty much on top of everything. Is there something conclusive about the 4K/512e advanced format on the WD20EURS drive. Is it a situation where there is initial success but issues develop over time, or is everything just theoretical. If there are actual issues, then I'll try and get an older 512b drive, but it will probably take some effort.

As an aside, I have an ESR tester for bad caps and tested them in-circuit (unplugged of course).

Only 2 tested bad, so I replaced those. There were 3 that looked ok, except one had tiny bulge, but could have been my imagination. It tested fine so I didn't replace it. Still had some issues with unstable svideo output, so I went back and looked more closely.

Turns out my late night tired eyes were fooled and I didn't realize those 3 caps are in parallel, so the in-circuit ESR testing was useless. Anyway, replaced all 3 as it turns out after testing out of circuit, 2 of 3 were bad, though one had no signs of bulge and I figured with the high ESR on the failed caps, the remaining cap probably took on more of the burden so its days might be numbered.
sfhub is offline   Reply With Quote
Old 09-08-2013, 03:30 PM   #9670
ThAbtO
TiVoholic by the bay
 
ThAbtO's Avatar
 
Join Date: Apr 2000
Location: SF Bay Area
Posts: 4,370
Quote:
Originally Posted by sfhub View Post
So I've only read the FAQs on the first page, but was there some recent change in 11.0k...m for OLED S3 that allows for larger the 1TiB MFS partitions. That was the impression I got from the procedure above.
The 11.0m (9.3.2c in Series 2) version was just a fix for the cookie bug that stopped downloads.
__________________
Series 2 (80 hrs,PLS, Retired)
TiVo HD 1TB (157 HD Hrs, PLS, "S3HD")
Roamio, Oh Roamio!

~~~~~

The TiVo Apocalypse begins on Sat 2013 Feb 16 00:00 GMT!
ThAbtO is offline   Reply With Quote
Old 09-08-2013, 05:21 PM   #9671
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
Quote:
Originally Posted by ThAbtO View Post
The 11.0m (9.3.2c in Series 2) version was just a fix for the cookie bug that stopped downloads.
Thanks, from the standpoint of upgrading drives, 11.0m should have the same change as in 11.0k to support > 1.1TB partitions, correct?
sfhub is offline   Reply With Quote
Old 09-08-2013, 05:56 PM   #9672
jmbach
Registered User
 
Join Date: Jan 2009
Posts: 566
More than likely since my 2TB OLED S3 updated from 11.0k to 11.0m with no loss of functionality.
__________________
"Delay is preferable to error" - Thomas Jefferson
"If I have seen further it is by standing on the shoulders of Giants" - Sir Isaac Newton
jmbach is online now   Reply With Quote
Old 09-08-2013, 08:57 PM   #9673
unitron
Registered User
 
unitron's Avatar
 
Join Date: Apr 2006
Location: semi-coastal NC
Posts: 12,420
Quote:
Originally Posted by sfhub View Post
Awesome, thanks for the info.

I had the failing/bulging caps that I replaced only to find I also had a dying HD. I haven't been around these forums in a while so I had no idea 11.0k added that feature (updated kernel?) To give perspective, my last backup of the OS is 8.1.1.

So is this a reasonable way to upgrade now for 2TB OLED S3 (assuming old image)
  1. Prepare drive
    • DD original image
    • hdparm -M 128
    • wdidle3 (if necessary)
  2. allow OS to upgrade to 11.0k (11.0m in my case)
  3. winmfs mfsadd
  4. winmfs supersize
Is there a better procedure?

From reading past posts, you are pretty much on top of everything. Is there something conclusive about the 4K/512e advanced format on the WD20EURS drive. Is it a situation where there is initial success but issues develop over time, or is everything just theoretical. If there are actual issues, then I'll try and get an older 512b drive, but it will probably take some effort.

As an aside, I have an ESR tester for bad caps and tested them in-circuit (unplugged of course).

Only 2 tested bad, so I replaced those. There were 3 that looked ok, except one had tiny bulge, but could have been my imagination. It tested fine so I didn't replace it. Still had some issues with unstable svideo output, so I went back and looked more closely.

Turns out my late night tired eyes were fooled and I didn't realize those 3 caps are in parallel, so the in-circuit ESR testing was useless. Anyway, replaced all 3 as it turns out after testing out of circuit, 2 of 3 were bad, though one had no signs of bulge and I figured with the high ESR on the failed caps, the remaining cap probably took on more of the burden so its days might be numbered.
What part of the country are you and your ESR meter in?

If the AF of the WD20EURS was going to cause a problem in S3s, I think enough of us are running them that we'd have run into it by now.

I don't know exactly what's wrong with the drive you have in your 648 currently, or how long it's going to last.

Are there shows on there that you particularly want to try to save?

Do you have a PC running Windows XP or newer and WinMFS installed on it?

Can you open it up and attach two SATA drives directly to SATA ports?

If not, have you got USB-to-SATA adapters?

Do you have a freezer and paper towels?

Have you burned yourself a bootable copy of the MFS Live cd v1.4 and made sure you can boot that PC from it?

Have you made yourself a bootable copy of the current Ultimate Boot CD?
__________________
(thisismysigfile)


"I am altering the deal. Pray I don't alter it any further."

Darth TiVo, 14 February, 2011
unitron is offline   Reply With Quote
Old 09-09-2013, 07:11 AM   #9674
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
Quote:
Originally Posted by unitron View Post
If the AF of the WD20EURS was going to cause a problem in S3s, I think enough of us are running them that we'd have run into it by now.
My original drive actually seemed usable and after replacing the caps, the unit booted again, but I was getting some random reboots and some svideo instability.

In an effort to determine if the drive was the issue I thought I should run a kickstart 54. Kickstart 54 actual tests came back passed, except the SMART query came back with some error #7. Drive could still boot and playback shows, but would randomly reboot half an hour to an hour in, sometimes on playback, sometimes just doing menu operations.

Instead of assuming the drive was going and immediately starting recovery operations (which I now realize I should have done), I thought maybe there were more caps to replace but I wanted to further rule out the drive so I naively thought I should run kickstart 57.

It said it would take 3hrs and was chugging along for about 2hrs, then it rebooted and got stuck in a GSOD loop. I don't know if the drive is actually bad because I timed the GSOD loop and it always rebooted exactly 89 seconds into the GSOD screen, so could be the drive, or something kickstart 57 depends on isn't right. I checked my records and I had added 375mb swap space originally, but it was so long ago, I might have had issues with the bug with swap space signatures > 127mb.

I tried sticking the drive on a winxpsp3 asus box and ran WinMFS but it wouldn't recognize the drive as a TiVo drive, so at this point I figured I could spend another unknown amount of time figuring out if I can recover enough of the drive to salvage shows or I could just image a new drive. I went the latter route because I needed some shows to get recorded.

I've already restored an original image onto a 1TB 512b drive and it is up and running, but I haven't spent any time trying to salvage shows on the old drive yet.

So I came on here seeing if there was more info about the GSOD loop and ended up reading about advances in drive upgrades. I was surprised to find out the 1TB barrier was passed with 11.0k. I started thinking since I have the unit out and disassembled, I might as well put in a 2TB drive. I read about AF and what I thought were theoretical problems, but then I read something about Tluxon and some FF/Rew issues he had over time.

So where I am at now is I have a fully working unit on the 1TB 512b drive.

I am resolved to lose my shows, but will make some attempt at recovering the 750GB Seagate DB35.3 drive.

In parallel I am trying to decide if I should purchase a WD20EURS drive and do the 2TB upgrade. The WD20EURS seems like a great drive for an upgrade except for this AF black cloud that seems to surround it. I can try and get a refurb 512b drive, but I'm really wondering if it is worth the effort, especially if these are just theoretical rather than actual issues.

I went through and looked at the original partitions on the OLED S3 and it looks like they are not 4k aligned at all:
Code:
  Original Partition Map
  Partition map (with 512 byte blocks) on '/dev/hda'
   #:                type name                        length   base      ( size )
   1: Apple_partition_map Apple                           63 @ 1
   2:               Image Bootstrap 1                      1 @ 268618469
   3:               Image Kernel 1                      8192 @ 268618470 (  4.0M)
   4:                Ext2 Root 1                      524288 @ 268626662 (256.0M)
   5:               Image Bootstrap 2                      1 @ 269150950
   6:               Image Kernel 2                      8192 @ 269150951 (  4.0M)
   7:                Ext2 Root 2                      524288 @ 269159143 (256.0M)
   8:                Swap Linux swap                  262144 @ 269683431 (128.0M)
   9:                Ext2 /var                        524288 @ 269945575 (256.0M)
  10:                 MFS MFS application region      589824 @ 270469863 (288.0M)
  11:                 MFS MFS media region         216747657 @ 271649511 (103.4G)
  12:                 MFS MFS application region 2    589824 @ 271059687 (288.0M)
  13:                 MFS MFS media region 2       268618405 @ 64        (128.1G)
I am not sure if it really matters because I don't even know if TiVo internally tries to write out in 4096 blocks or respects 4k blocks once the partitions are aligned. I can imagine that making every write a read-modify-write cycle would be inefficient and possibly result in more wear/tear, but possibly WD handles this in cache before it ever gets to the drive.

So I think I will go with a 2TB upgrade and I'm thinking of getting a WD20EURS drive but haven't purchased it yet. I am willing to try and get a 512b 2TB drive if I can find one from a realiable source and it isn't outrageously expensive.

If I get the WD20EURS I'm trying to figure out if I need to spend some effort getting the partitions 4k-aligned. I was originally planning on a DD then a WinMFS mfsadd but since the original partitions are completely not aligned, that seems to not have any advantage as far as 4k-alignment goes.

As an aside, does mfslive 1.4 mfsadd also work for > 1TB adds?

I'd normally just go and do some experiments, but these 250GB/750GB/1TB/2TB disk operations all take significant time and doing everything "right" the first time around will probably save a lot of frustration.

Sorry if this is TLDR, I'm hopeful someone takes the time to lend their expertise and push me on the right path if I am headed the wrong direction.
sfhub is offline   Reply With Quote
Old 09-09-2013, 09:13 AM   #9675
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
So I had another question about alignment and how partitions map to physical blocks.
Code:
   #:                type name                        length   base      ( size )
 1: Apple_partition_map Apple                           63 @ 1
...
13:                 MFS MFS media region 2       268618405 @ 64        (128.1G)
When the partition map lists base of partition #1 as 1, does the numbering system start at "1" or "0"?

I think it starts at "0" and the first block is the boot record. So in this case
Code:
block
0      MBR
1 -63  Apple_partition_map
64-... MFS medial region 2
If this is correct, I'd want the base of the media partitions to all be evenly divisible by 4.

However if this is incorrect and the 1st block actually starts at "1", then I'd want all the divisions to end in .25

I know from past experience, it is easy to get "off by 1" errors if not being very specific about where things start.

Also since mfscopy (mfslive/restore) is moving partitions around anyway, has someone tried modifying it to 4k align the partitions when it moves them around? It doesn't even need to adjust the partition sizes, just needs to skip enough blocks to align the partitions and adjust the start/end of the zones.
sfhub is offline   Reply With Quote
Old 09-09-2013, 02:45 PM   #9676
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
Quote:
Originally Posted by sfhub View Post
So I had another question about alignment and how partitions map to physical blocks.
Code:
   #:                type name                        length   base      ( size )
 1: Apple_partition_map Apple                           63 @ 1
...
13:                 MFS MFS media region 2       268618405 @ 64        (128.1G)
When the partition map lists base of partition #1 as 1, does the numbering system start at "1" or "0"?

I think it starts at "0" and the first block is the boot record. So in this case
According to this article
http://en.wikipedia.org/wiki/Apple_Partition_Map

For Apple Partition Map systems every block is mapped out, except for Block 0, which is the Driver Descriptor Map.

That means the most obvious and easy way to figure out 4k alignment is to have the base block evenly divisible by 4 (which is what everyone has been doing already).

So the above example looks like this:
Code:
block
0      Driver Descriptor Map
1 -63  Apple_partition_map
64-... MFS medial region 2
So back to the previous question, since it seems mfslive/restore is already moving stuff around has anybody looked into adding the appropriate "spacing" to keep everything 4k-aligned?

Is it possible to just change the start/end of the partitions and zone information so they are 4k aligned, leaving unreferenced "spacers" in between? Or does every block on disk need to be accounted for in an Apple Partition Map, in which case you could create small dummy partitions.
sfhub is offline   Reply With Quote
Old 09-09-2013, 08:32 PM   #9677
unitron
Registered User
 
unitron's Avatar
 
Join Date: Apr 2006
Location: semi-coastal NC
Posts: 12,420
If one is going to run SMART tests on a drive, it's probably better to take it out of the TiVo and use the manufacturer's diagnostic software.

Since WinMFS will let you use a 2TB on S3s now that 11.0k will let the TiVo handle partitions bigger than 1TB (or 1.2 or whatever), I never bothered spending time experimenting with MFS Live to see if it would.

WinMFS isn't exactly brand new, and MFS Live is even older.

I think the WD20EURS is one of those 4k physical/512 virtual drives, meaning it does any necessary bit juggling instead of the TiVo having to.

Like I said...

"If the AF of the WD20EURS was going to cause a problem in S3s, I think enough of us are running them that we'd have run into it by now."
__________________
(thisismysigfile)


"I am altering the deal. Pray I don't alter it any further."

Darth TiVo, 14 February, 2011
unitron is offline   Reply With Quote
Old 09-09-2013, 09:37 PM   #9678
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
Quote:
Originally Posted by unitron View Post
Since WinMFS will let you use a 2TB on S3s now that 11.0k will let the TiVo handle partitions bigger than 1TB (or 1.2 or whatever), I never bothered spending time experimenting with MFS Live to see if it would.
Do you know if WinMFS will add the zones to the existing partition 15 if you add a 2nd time. ie you went from 250GB->1TB->2TB After 250GB->1TB you'd have partition 14/15 (for historical reasons). Now when you try to reupgrade an upgraded drive it could do a single partition 16 (instead of the traditional pair) or it could add the zone to the existing partition or it could just not work.

I'm hoping it justs adds the zone to the existing partition 15.

Quote:
Originally Posted by unitron View Post
I think the WD20EURS is one of those 4k physical/512 virtual drives, meaning it does any necessary bit juggling instead of the TiVo having to.

Like I said...

"If the AF of the WD20EURS was going to cause a problem in S3s, I think enough of us are running them that we'd have run into it by now."
I understand what you are saying, I just like to understand what I'm getting myself into.

All (or almost all) 4k drives are 4k physical/512 virtual, but not all 4k/512 are the same. Western Digital recommends alignment while Seagate does "SmartAlign" which is their predictive cache/lookahead read that is supposed to reduce the occurence of read-modify-write even without alignment.

The issue with all these 4k drives is not whether they will work. Clearly as you say they do. The performance issues, even with benchmarks happens only with random writes. With sequential writes they perform at almost normal speed even if not aligned. The reads are virtually not affected by non-alignment even if random. So you are just left with random writes, where the performance goes down the tubes, probably 3-4x slower.

I'm worried that a newly upgraded TiVo with lots of free space will mostly be doing sequential writes, but over time if you fill up the drive with different size SD/HD shows and remove them, things could get fragmented, shifting you into the random write situation.

As an aside has someone filled up their 2TB drive completely yet? Was the old 1TB partition size error something that would immediately show up at startup or something you'd see later on when the drive filled up?

Alternatively has someone verified the 11.0k kernel actually supports > 1TiB partitions or is it just by experimentation with WinMFS?

I saw something about Jamie having made some signed int->unsigned int change to his custom kernel to support 2TiB partitions, then some waiting game to see if TiVo incorporated the changes, but I never was able to find talk about the kernel changes being included 11.0k other than the experimentations with WinMFS and conclusion that it works.

I am sorry for asking so many questions. While this stuff is probably common knowledge, I'm time-warped from 5yrs ago, so it is somewhat new to me.
sfhub is offline   Reply With Quote
Old 09-10-2013, 01:04 AM   #9679
Soapm
Registered User
 
Soapm's Avatar
 
Join Date: May 2007
Location: So close, yet too far away
Posts: 1,446
Quote:
Originally Posted by sfhub View Post
Do you know if WinMFS will add the zones to the existing partition 15 if you add a 2nd time. ie you went from 250GB->1TB->2TB After 250GB->1TB you'd have partition 14/15 (for historical reasons). Now when you try to reupgrade an upgraded drive it could do a single partition 16 (instead of the traditional pair) or it could add the zone to the existing partition or it could just not work.

I'm hoping it justs adds the zone to the existing partition 15.
WinMFS will always add pairs. Only JMFS can add a single partition. You can also try mfsadd tool from the other forum.


Quote:
Originally Posted by sfhub View Post
The issue with all these 4k drives is not whether they will work. Clearly as you say they do. The performance issues, even with benchmarks happens only with random writes. With sequential writes they perform at almost normal speed even if not aligned. The reads are virtually not affected by non-alignment even if random. So you are just left with random writes, where the performance goes down the tubes, probably 3-4x slower.
I had 2TB drives in two Tivo's for probably two years now so I will assume they're as full as a Tivo will allow them to be and the shows record and play just fine. There might be performance issues but they're not visible using the Tivo.

It's like beefing up your engine to continue to drive 65mph when the car went 65mph with no strain before.

Quote:
Originally Posted by sfhub View Post
I'm worried that a newly upgraded TiVo with lots of free space will mostly be doing sequential writes, but over time if you fill up the drive with different size SD/HD shows and remove them, things could get fragmented, shifting you into the random write situation.
I'm no technical expert but I recall seeing that fragmentation isn't a problem with the MFS design. I think the compensation was in the design.

Quote:
Originally Posted by sfhub View Post
I saw something about Jamie having made some signed int->unsigned int change to his custom kernel to support 2TiB partitions, then some waiting game to see if TiVo incorporated the changes, but I never was able to find talk about the kernel changes being included 11.0k other than the experimentations with WinMFS and conclusion that it works.
I don't think the problem was the kernel. I've used nuetered stock kernels since installing this drive and I have a 1.6 gig partition that's worked since day one. I don't know why TivoHD weren't accepting WinMFS drives before 11.0k but I don't think it was the kernel. However, I used JMFS on my drives so I don't know if that made a difference but I would think 1.6 gig is 1.6 gigs.

You can also see that JMFS added a single partition.

Code:
Partition map (with 512 byte blocks) on '/dev/hda'
 #:                type name                         length   base       ( size )
 1: Apple_partition_map Apple                            63 @ 1          (  31.5K)
 2:               Image Bootstrap 1                       1 @ 309549120
 3:               Image Kernel 1                       8192 @ 309549121  (   4.0M)
 4:                Ext2 Root 1                       524288 @ 309557313  ( 256.0M)
 5:               Image Bootstrap 2                       1 @ 310081601
 6:               Image Kernel 2                       8192 @ 310081602  (   4.0M)
 7:                Ext2 Root 2                       524288 @ 310089794  ( 256.0M)
 8:                Swap Linux swap                   262144 @ 310614082  ( 128.0M)
 9:                Ext2 /var                         524288 @ 310876226  ( 256.0M)
10:                 MFS MFS application region       589824 @ 311400514  ( 288.0M)
11:                 MFS MFS media region          137629696 @ 171919424  (  65.6G)
12:                 MFS MFS application region 2     589824 @ 311990338  ( 288.0M)
13:                 MFS MFS media region 2        171919360 @ 64         (  81.9G)
14:          Apple_Free Extra                          1645 @ 312580162  ( 822.5K)
15:                 MFS MFS media region 3       3594447361 @ 312581807  (   1.6T)


Last edited by Soapm : 09-10-2013 at 01:14 AM.
Soapm is offline   Reply With Quote
Old 09-10-2013, 04:23 AM   #9680
unitron
Registered User
 
unitron's Avatar
 
Join Date: Apr 2006
Location: semi-coastal NC
Posts: 12,420
Quote:
Originally Posted by sfhub View Post
Do you know if WinMFS will add the zones to the existing partition 15 if you add a 2nd time. ie you went from 250GB->1TB->2TB After 250GB->1TB you'd have partition 14/15 (for historical reasons). Now when you try to reupgrade an upgraded drive it could do a single partition 16 (instead of the traditional pair) or it could add the zone to the existing partition or it could just not work.

I'm hoping it justs adds the zone to the existing partition 15.


I understand what you are saying, I just like to understand what I'm getting myself into.

All (or almost all) 4k drives are 4k physical/512 virtual, but not all 4k/512 are the same. Western Digital recommends alignment while Seagate does "SmartAlign" which is their predictive cache/lookahead read that is supposed to reduce the occurence of read-modify-write even without alignment.

The issue with all these 4k drives is not whether they will work. Clearly as you say they do. The performance issues, even with benchmarks happens only with random writes. With sequential writes they perform at almost normal speed even if not aligned. The reads are virtually not affected by non-alignment even if random. So you are just left with random writes, where the performance goes down the tubes, probably 3-4x slower.

I'm worried that a newly upgraded TiVo with lots of free space will mostly be doing sequential writes, but over time if you fill up the drive with different size SD/HD shows and remove them, things could get fragmented, shifting you into the random write situation.

As an aside has someone filled up their 2TB drive completely yet? Was the old 1TB partition size error something that would immediately show up at startup or something you'd see later on when the drive filled up?

Alternatively has someone verified the 11.0k kernel actually supports > 1TiB partitions or is it just by experimentation with WinMFS?

I saw something about Jamie having made some signed int->unsigned int change to his custom kernel to support 2TiB partitions, then some waiting game to see if TiVo incorporated the changes, but I never was able to find talk about the kernel changes being included 11.0k other than the experimentations with WinMFS and conclusion that it works.

I am sorry for asking so many questions. While this stuff is probably common knowledge, I'm time-warped from 5yrs ago, so it is somewhat new to me.
Actually it sounds like you may know some stuff about hard drives that I don't.

Anyway, if you start with a Series 1 that had a small enough hard drive, the image would have one MFS application region partition and one MFS media region partition, the proverbial "MFS pair".

Everything after that came with 2 MFS pairs, for a total of 13 partitions.

If you copied to a larger drive, you could expand (with MFS Tools or its successor MFS Live, or with WinMFS) by adding a third MFS pair, partitions 14 and 15.

If you copied that to yet a larger drive, it turns out that WinMFS can let you expand by making partition 15 larger.

I don't think MFS Live can do that, and won't be able to experiment to find out anytime soon.

Before I discovered (by way of someone else discovering it) that WinMFS would do what I needed, I used it to copy to a 1TB and expand (it won't leave a tiny little "Apple Free" partition the way I could never keep MFS Live from doing), filling up that 1TB drive with 15 partitions.

I then copied that to a 2TB drive using the jmfs cd and let it expand by adding a 16th partition.

(I learned the hard way that jmfs cannot detect and make use of an Apple Free partition the way WinMFS can, so if there is one of any size as the 16th partition on a drive, when you copy that drive to a larger one and expand with jmfs, it creates a 17th partition after that Apple Free one, and on booting the TiVo thinks it has an external drive gone bad hooked up and insists on divorcing it, which means the TiVo deletes that 17th partition and turns your 2TB drive back into one with only 1TB usable)

With all due respect to comer, who gave us jmfs (which is still the only way to expand an S4 drive that I know of), since WinMFS will handle S3 upgrades just fine as long as you're already up to 11.0k or later, I don't see any reason to use anything else, unless you only have a Mac laptop, in which case you're probably stuck with MFS Live if the laptop will boot from it.
__________________
(thisismysigfile)


"I am altering the deal. Pray I don't alter it any further."

Darth TiVo, 14 February, 2011
unitron is offline   Reply With Quote
Old 09-10-2013, 10:10 AM   #9681
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
Thank you so much for the details. Every little bit adds confidence.

Just to be absolutely clear, folks here are successfully running > 1TB partitions on S3 OLEDs using stock 11.0k/m kernels right? I know sometimes when I mod boxes, I can lose track of what I'm running or whether I did something first with a hacked kernel then went back to stock.

Regarding JMFS dealing with the Apple Free partition created at the end of drive by MFS Live, would it have been possible to delete that partition using a partition editor (pdisk?) before running JMFS? I know it's not something you'll go back and try now, but perhaps something to keep in mind if it ever comes up again with someone else's upgrade.

Regarding the 1TiB partition limit in the kernel, the following led me to that idea (I was wrong about jamie's patch being signed to unsigned int, it was 32-bit int to 64-bit int)

From Spike
http://www.mfslive.org/forums/viewto...start=45#p4818
Quote:
Here are the limits that I know of:

You can have only 16 partitons per drive.
128 character device names in the superheader devlist. /dev/hda10 /dev/hda11 .....
1 TiB partition limit due to linux Kernel (use of signed int to check # of sectors )
2 TiB partition limit due to Apple partitition table being used (use of unsigned int to reference # of sectors)
From Spike
http://mfslive.org/forums/viewtopic.php?p=4636#p4636
Quote:
Beta Build 9.3f & 9.3g are released.

- User is prompted "Y" or "N" to limit a partition to 1 TiB when expanding. If you have a drive smaller than 1TiB you won't get that prompt.
If you want to use a 1.5 TB or higher drive, answer "Y" so you can at least use 1.2 TB of it. As of Tivo software 11.0, linux kernel used can not
support any partition higher than 1 TiB so if you answer "N", it won't work properly in your TiVo unless you have the specially hacked linux kernel
installed that will allow it.
From Jamie
Quote:
Here are custom Gen05 and Gen06 kernels built from the 9.4.1 tivo kernel sources. Be sure to read and understand the first post in this thread before attempting to us these custom kernels.
...
An additional patch is include here that changes some 32 bit ints in the tivo ide-disk.c driver to 64 bit ints. This allows the kernel to work with partitions > 1TiB. Without this patch, you'll get "kernel: idedisk_dosectors: Access beyond end of drive" errors with partitions > 1TiB. Note that there is still a 2TiB partition size limit imposed by the ancient apple partition table format the tivo still used.
...
I was also led to the impression that originally the latest versions (at the time) of MFSLive and WinMFS mfsadd would blindly expand to the end of a 2TB drive. Coupled with the 1TiB partition limit in the linux kernel, this made MFSLive incompatible with 2TB drives (unless you started with a 1TB factory image). I think Spike added that WinMFS question about whether to break the 1TB partition limit to stop the blindly expand behavior so people could use 1.5/2TB drives with kernels that didn't support > 1TB.

So I *think* (but could be mistaken) MFSLive mfsadd would do the same expansion to end of drive that WinMFS is doing with 2TB drives, it just wouldn't ask you the question. I can't really verify at the moment since I don't have a 2TB drive (yet).

Thank you for being patient with me, I'm slowly figuring things out and the cobwebs are clearing from long-term neglect.
sfhub is offline   Reply With Quote
Old 09-10-2013, 06:22 PM   #9682
unitron
Registered User
 
unitron's Avatar
 
Join Date: Apr 2006
Location: semi-coastal NC
Posts: 12,420
I tried using

pdisk

to get rid of the Apple Free partition and it doesn't work, in that after you remove it, it puts itself right back. It's just the way the Apple Partition Map works--a non-partioned space gets labeled as an Apple Free partition.

Using WinMFS I put a 648 image (11.0k) on a 2TB and wound up with a 1.6TB partition 15, and it's been working fine for months--updated itself to 11.0m the other day without a hitch and everything.

I don't know if MFS Live will do that or not, I'll have to experiment.
__________________
(thisismysigfile)


"I am altering the deal. Pray I don't alter it any further."

Darth TiVo, 14 February, 2011
unitron is offline   Reply With Quote
Old 09-10-2013, 10:59 PM   #9683
jmbach
Registered User
 
Join Date: Jan 2009
Posts: 566
MFSLive will put a very small Apple_Free partition at the end of the drive after you expand it. Not sure if it does it in all cases, but it did it in all the cases that I have tried. The only way I know how to erase the Apple_Free partition is to do it manually with a hex editor. I have not found a way to accomplish that using pdisk alone.
__________________
"Delay is preferable to error" - Thomas Jefferson
"If I have seen further it is by standing on the shoulders of Giants" - Sir Isaac Newton
jmbach is online now   Reply With Quote
Old 09-10-2013, 11:46 PM   #9684
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
Quote:
Originally Posted by jmbach View Post
The only way I know how to erase the Apple_Free partition is to do it manually with a hex editor. I have not found a way to accomplish that using pdisk alone.
Perhaps you can use
  • dd bs=512 skip=N count=1 if=/dev/zero of=/dev/sda1
Once you determine "N" for partition 16.

Partition 16 should be in the same location for everyone so the command should be universal (and harmless if you don't have a partition 16)

If it needs to be some special format rather than all zeros, then I'm sure you can read from the partition map of a 15 partition drive and save the output to a file to write back onto drives that have partition 16.
sfhub is offline   Reply With Quote
Old 09-11-2013, 12:26 AM   #9685
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
Does anyone know if there are any S3 OLEDs that run MFS64 (ie their "magic" in the Super Header is ebbafeed)

I'm trying to recover a drive stuck in GSOD bootloop and noticed the magic got set to 37353033 (which is apparently why WinMFS wasn't recognizing the drive as a tivo drive anymore) I think the original magic was abbafeed (indicating MFS32), but I am not sure.
sfhub is offline   Reply With Quote
Old 09-11-2013, 12:31 AM   #9686
unitron
Registered User
 
unitron's Avatar
 
Join Date: Apr 2006
Location: semi-coastal NC
Posts: 12,420
Quote:
Originally Posted by jmbach View Post
MFSLive will put a very small Apple_Free partition at the end of the drive after you expand it. Not sure if it does it in all cases, but it did it in all the cases that I have tried. The only way I know how to erase the Apple_Free partition is to do it manually with a hex editor. I have not found a way to accomplish that using pdisk alone.
Apparently one of the "delightful" quirks of the Apple Partition Map is that each partition has info in the header of where all the other partitions are, so you can't just hex erase the last partition and remove it from the Map, you have to hex edit each partition to reflect the removal.
__________________
(thisismysigfile)


"I am altering the deal. Pray I don't alter it any further."

Darth TiVo, 14 February, 2011
unitron is offline   Reply With Quote
Old 09-11-2013, 02:53 AM   #9687
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
Quote:
Originally Posted by sfhub View Post
Does anyone know if there are any S3 OLEDs that run MFS64 (ie their "magic" in the Super Header is ebbafeed)

I'm trying to recover a drive stuck in GSOD bootloop and noticed the magic got set to 37353033 (which is apparently why WinMFS wasn't recognizing the drive as a tivo drive anymore) I think the original magic was abbafeed (indicating MFS32), but I am not sure.
Well I can answer this question now, my S3 OLED is using MFS32 with magic abbafeed.

I was able to recover from my GSOD loop.

I had two ideas on how to get out of it. One way was to try and change the magic from 37353033 back to abbafeed. I *think* that 37353033 magic is what is telling TiVo to do keep going into green screen KS 57 mode so my thinking is if I changed it back to abbafeed, it would just boot normally.

I didn't get around to trying that because my other option ended up working. I just dd_rescued the GSOD loop drive onto a new drive and put the drive back in the TiVo. It was able to complete the KS 57 and I got all my shows back.

Out of 750GB I had 64kb of bad sectors causing my GSOD loop.

Last edited by sfhub : 09-11-2013 at 01:26 PM.
sfhub is offline   Reply With Quote
Old 09-11-2013, 04:02 AM   #9688
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
Quote:
Originally Posted by unitron View Post
Apparently one of the "delightful" quirks of the Apple Partition Map is that each partition has info in the header of where all the other partitions are, so you can't just hex erase the last partition and remove it from the Map, you have to hex edit each partition to reflect the removal.
I found this post about removing apple_free.

http://www.tivocommunity.com/tivo-vb...21#post8758021

According to this post, you zero out block 16 then edit the field in the other partitions that references the total # of partitions to be 15 instead of 16.

So this should be something that can be done relatively easily with dd instead of by hand with hex editor.

Last edited by sfhub : 09-11-2013 at 01:25 PM.
sfhub is offline   Reply With Quote
Old 09-12-2013, 04:26 AM   #9689
unitron
Registered User
 
unitron's Avatar
 
Join Date: Apr 2006
Location: semi-coastal NC
Posts: 12,420
Quote:
Originally Posted by sfhub View Post
Does anyone know if there are any S3 OLEDs that run MFS64 (ie their "magic" in the Super Header is ebbafeed)

I'm trying to recover a drive stuck in GSOD bootloop and noticed the magic got set to 37353033 (which is apparently why WinMFS wasn't recognizing the drive as a tivo drive anymore) I think the original magic was abbafeed (indicating MFS32), but I am not sure.
I wish I knew enough about this stuff to have understood that.
__________________
(thisismysigfile)


"I am altering the deal. Pray I don't alter it any further."

Darth TiVo, 14 February, 2011
unitron is offline   Reply With Quote
Old 09-12-2013, 04:45 AM   #9690
sfhub
Registered User
 
Join Date: Jan 2007
Posts: 1,261
Ok, I believe I have now fully recovered from my user initiated Kickstart 57 GSOD loop.

To recap (no pun intended), S3 OLED had some bad caps, replaced them, unit booted, but had reboot issues, ran kickstart 57, got stuck in GSOD reboot loop.

I tried using WinMFS to copy the drive, but WinMFS wouldn't recognize it. It turns out (I believe) TiVo sets the magic cookie identifier for the MFS Super Header to be 37353033 which then triggers the GSOD mfschk (normally the Super Header magic cookie is abbafeed for mfs32 volumes and ebbafeed for mfs64 volumes). WinMFS will only recognize a drive as a TiVo drive if the Super Header magic cookie is abbafeed or ebbafeed. The reason my drive started the GSOD loop was because it had a very small amount of bad sectors. Kickstart 57 does not like bad sectors at all.

Anyway, I ended up using dd_rescue to copy the bad 750GB drive onto a 1TB drive I had available. As an aside, there were only 64KB bad sectors out of 750GB. I didn't expand but immediately put the 1TB drive back into TiVo to let it finish the Kickstart 57, which it did in about 3 hours.

When it was done, TiVo booted and all my recorded shows were available. I tried connecting to the mothership to refresh guide data which was now 3 days from running out. The connect worked, but the Guide Cache and Indexing were taking forever (over 12 hours and counting).

Until the Guide Cache and Indexing are complete, even though you have guide data in the program guide, none of your season passes will schedule shows to record. Also when you view upcoming episodes, there will be none.

Anyway, I figured the Guide Cache and Indexing was just stuck or hung, possibly due to some issues with the disk.

My solution was to rechoose my video provider:

settings->channels->channel list
press enter to reselect channel provider

What this does is let you choose your zip code and video provider. It will delete your program guide info and redo it from scratch. Your ToDo list is preserved and so is your list of channels you get. This is unlike the Clear Program Information and To Do list option, which will delete your Season Passes.

The new guide download took around 2hrs to download and index and I was back in business. Guide Cache and Indexing were complete.

My ToDo list was still empty, but this was easily solved by going to the ToDo list and reordering 1 season pass entry. After that all the scheduled shows showed up immediately.

Hope this helps someone else who might get stuck in a Kickstart 57 GSOD loop.
sfhub is offline   Reply With Quote
Reply
Forum Jump




Thread Tools


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Advertisements

TiVo Community
Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
vBulletin Skins by: Relivo Media

(C) 2013 Magenium Solutions - All Rights Reserved. No information may be posted elsewhere without written permission.
TiVoŽ is a registered trademark of TiVo Inc. This site is not owned or operated by TiVo Inc.
All times are GMT -5. The time now is 02:22 AM.
OUR NETWORK: MyOpenRouter | TechLore | SansaCommunity | RoboCommunity | MediaSmart Home | Explore3DTV | Dijit Community | DVR Playground |