1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Fixes for MFSTools 2.0 swap problems

Discussion in 'TiVo Upgrade Center' started by Robert S, Jul 31, 2002.

  1. Robert S

    Robert S New Member

    Jul 8, 2002
    The only way to save your recordings and increase swap is to do a pipe transfer (basically MFS Tools 2.0 can only increase swap if it's doing its restore function). It makes sense to do this instead of blessing, even if you're not going over the limit.

    Despite the simplicity of blessing, if you don't increase swap it's very difficult to increase swap if you want to upgrade further and keep your recordings (adding swap would be a very similar process to the 'rescue' hack at the top of this thread, but you'd use pdisk to create a partition to relabel, rather than using the root partition).

    The trouble is, to explain this properly would add at least a page to Hinsdale. Unfortunately, if you don't understand this, you might conclude that a TiVo stuck in a green screen loop is unrecoverable, whereas infact it can be recovered with a clever hack.

    It would be nice to say to everyone who upgrades, 'if you go over 140Gb and you didn't expand your swap, tape a note with the address of the Upgrade Forum inside the lid - you'll know when you need it'.
  2. deek_man

    deek_man New Member

    Jul 26, 2002
    Thanks for the help, Robert. But if one wanted to save their recordings before a green screen freeze AND repair the swap partition for a 200gb system with only 64 mb of swap, could one use:

    mfsbackup -Tao - /dev/hdx | mfsrestore -s 127 -xpi - /dev/hdy /dev/hdz

  3. Robert S

    Robert S New Member

    Jul 8, 2002
    That's what I mean by 'a pipe transfer' - the '|' character is pronounced 'pipe'.

    It's not always possible to do this - if you want to upgrade the A drive of a twin drive machine without losing recordings, for example. You can increase swap between running dd and mfsadd, but the process is just as complex as the 'rescue' procedure.

    I agree that Hinsdale skates over an important issue by not mentioning swap issues. Advising everyone to use -s 127 is a good start, but to cover it properly you'd have to explain:
    • What swap is
    • You need more swap >140Gb
    • Most upgrades can increase swap by using -s 127
    • Most of the rest can increase swap by using a pipe transfer instead of blessing or dd + mfsadd. And that despite the extra pain this causes, it's well worth it.
    • A drive expansions on twin drive machines can not expand swap without losing recordings, but there is a hack you can use if you want.
    • If you go over >140Gb without expanding swap your TiVo will get stuck in a green screen loop if it gets filesystem corruption. If this happens, there's a hack to recover your TiVo without losing your data or recordings.
    As there's already so much in Hinsdale, I can appreciate he's reluctant to add more. New Hinsdale was written from Tiger's documentation, which claims that you don't need more swap if you use MFS Tools 2. This has since been proved false. One could argue that in the light of that discovery, it needs a major overhaul.
  4. mrtickle

    mrtickle New Member

    Aug 26, 2001
    The last bullet point is the single most important omission IMHO. It should be screamed in huge letters at the start of the document!

    I bet there are thousands of people out there with machines >140GB and only the default size swap. Not just MFStools2 upgrades, but older ones too. I only found out by a fluke chance just in time before I did my own upgrade.
  5. BobCamp1

    BobCamp1 Active Member

    May 15, 2002
    Also, upgrade kits should be avoided! Ordering a large pre-blessed drive and installing it will make your Tivo unstable. The full upgrade service has to be used (where you ship the entire Tivo), and you have to be smart enough to insist the service also increase the swap file size using MFSTools 2.0.

    This could put Hindsdale and others who have performed upgrades in trouble, since they have been unintentionally screwing up some (but not all) of the Tivos. And people who were too scared to do the upgrade themselves are surely not going to want to run the emergency fix procedure. Some of these people will demand a free fix or demand to be reimbursed the $100 they had to pay Tivo to have them fix it.

    Without Hindsale's instructions, I would have never attempted the upgrade in the first place. My guess is that around 70% of the people simply want to add a new drive. Those instructions are incomplete. It's not his fault (it's actually Tiger's), but he needs to fix them.
  6. jhburke

    jhburke New Member

    Aug 3, 2002
    I have been following this thread with a lot of interest, and I am impressed with the clever work of the contributors in figuring out the swap problems. However, I'm not so sure that the old "BlessTiVo" method is necessarily bad.

    It seems that large upgrades (ie 30 + 120) have been performed for a while, using the old tools, but no problems reported until MFSTools 2.0 was released. Then, shortly after MFSTools 2.0, there was a torrent of reports about GSOD and failures from inadequate swap. It doesn't seem possible to me that sudden drive failures causing GSOD should suddenly start occuring once 2.0 tools are used - perhaps the problem doesn't occur with mfstools 1.1 or BlessTivo. PTVupgrade stated in an earlier post in this thread that he's never had this type of problem despite never increasing swap. Perhaps mfstools 2.0 increases the swap requirement rather than decreasing it, and perhaps a 30 +120 upgrade with BlessTiVo can recover from GSOD with 64M swap.

    Of course, I don't know the answer, but it might be interesting if someone could test this theory (hint-hint anyone?). Then all the people with older upgrades and default swap may not need to panic yet.

  7. mrtickle

    mrtickle New Member

    Aug 26, 2001
    Systems >140GB have always needed more swap. Always! This is unfortunately not made clear in the guides - that was my point and is my only criticism of the guides. Everyone who has >140GB and only default swap has a TiVo which WILL crash and go into an infinite reboot loop if it ever gets a GSOD! The only get-out is to pull the drives and perform the procedure to give emergency extra swap as detailed earlier in the thread.

    There was a separate problem whereby anyone following the instructions (-s 128) ended up with no swap at all. People who then had GSODs (for whatever reason) then found themselves in trouble. Initially it was thought that less swap would be needed for mfsfix with the bigger blocksize that MFStools uses by default. The complete lack of swap may have induced many of the GSODs. This happened to lots of people at the same time so that's why the issue got a lot of attention.
  8. Robert S

    Robert S New Member

    Jul 8, 2002
    A 150Gb TiVo might just squeak through a GSOD. Merle Corey published his best estimate of the formula needed to estimate the precise amount of swap needed. Above about 155Gb there's no possibility of recovering. The swap requirements for MFS Tools 2.0 are exactly the same as for any other tool, despite the MFST2 docs repeatedly saying it's more efficient.

    I don't think there was a flood of people getting green screens - in fact quite the reverse, there was a nasty gap between embeem pointing out there was a problem and reports coming in from others confirming it.

    I don't think the absence of swap causes any real problems for the media side of TiVo - the guide and indexer get gummed up, but the video side is fine.

    There were a lot of people talking about the risk of a GSOD, but only a few actually experience one.

    This problem has always existed - TiVoMad will increase your swap for exactly the same reason - but larger drives and inaccurate claims of MFS Tools 2's powers make it more visible.
  9. muchmore44

    muchmore44 New Member

    Jul 19, 2002
    I just want to say thanks to Hinsdale and Tiger for giving us TiVo users a way to maximize a terrific technology, as well as to Robert S. and Merle C. for refining the swap expansion/correction process.

    Last night, I upgraded my Hughes DirectTiVo from 35 hours to 106 hours (went from 40 GB to 120 GB) and it couldn't have gone better! It took approximately 1 hour, 30 min, mostly because I was very, very careful - I could have done it faster, but I didn't want to take any chances. MFSTools 2.0 are the best!

    I am the happiest guy in the world right now (at least until the next person upgrades a Tivo!) and wanted to thank you guys for all that you have done.

    You guys are Saints!

  10. BobCamp1

    BobCamp1 Active Member

    May 15, 2002
    Note that your system merely becomes unstable with "only" a 64 MB swap file. "Unstable" in the sense that the Tivo will crash very infrequently, but it will crash so hard as to be unusable. Tivo only runs mfsfix under normal conditions when the file system on the hard drive gets really screwed up.

    This usually happens when the hard drive is failing anyway, so those people simply attributed their misbehaving Tivos for a bad hard drive.

    However, since Tivo doesn't have an "off" switch, and it constantly writes to the hard drive, you're at a low risk every time you unplug the Tivo. And when the power flickers at your house, too.

    Also there is an issue of timing:

    1. 100 and 120GB hard drives just became affordable enough to buy this summer. An 80GB hard drive, even when combined with a Series 2 60 GB Tivo, will not exhibit this problem.

    2. Series 2 units and newer DirecTivos (with their larger factory-shipped capacity) started shipping earlier this year.

    3. MFSTools 2.0 also came out at about the same time. The -s 128 "bug" was causing a different set of problems and was adding to the confusion.

    4. Power to your house (and Tivo) was flickering on and off from thunderstorms this summer. This could trigger mfsfix to run on your Tivo and expose the problem.

    Basically, we were doing bad upgrade procedures the entire time. But it wasn't until this summer that a bunch of things converged to magnify the problem.
  11. hinsdale

    hinsdale New Member

    Apr 30, 2001
    I think this thread might be more useful if someone were to do some more definitive testing to determine the actual upgrade capacity that will fail to recover from GSOD with 64MB Swap (taking into account 16 vs 32MB RAM) - not forumula approximations. I do not have time to do this currently but before unnecessarily concerning people with statements like "your unit will not recover from a GSOD if you are over 140GB", I believe some more testing is in order (results labeled GB or GiB). This issue is not new, and having witnessed posts, interchange, etc of thousands of upgraders adding 120GB drives to their 20, 30, 40, 60GB units without increasing swap and without widespread failure or even any pattern of failure - I am not sure this issue rises to the crisis level portrayed. I would be interested,however, in determining the actual trigger number. If I am not mistaken, my 160GB DirecTiVo with unmodified swap has recovered from GSOD, which makes me dubious of the trigger numbers being fowarded recently.
  12. enigma2175

    enigma2175 Long Member

    Apr 1, 2002
    I may have missed this somewhere in this quite large thread, but couldn't a recovery from an unrecoverable (mfsfix crash because of lack of swap) go like this:

    1. Get diagnostic prompt, append HANDCRAFT=TRUE RUNMYWORLD=FALSE to boot parameters. (assuming a bash prompt modification)

    2. At bash prompt make sure current swap is valid :
    mkswap /dev/hda8
    Check kernel logs to make sure swap is being enabled.

    3. If mfsfix is still crashing (You have way to much space for your own good) create a swapfile on your current /var partition.
    dd if=/dev/zero of=/var/SWAPFILE bs=1024 count=64000
    mkswap /var/SWAPFILE

    4. Add the swap to start at boot. Append
    /var/SWAPFILE swap swap defaults 0 0
    to /etc/fstab

    You now have 128 MB of swap without altering the partition layout. Takes some space in /var, but it should at least get you up and running.

    Has this already been proposed and I just missed the post?
  13. enigma2175

    enigma2175 Long Member

    Apr 1, 2002
    ... or you could put a 32 MB swapfile in / and in /var (I'm not sure if /var is already mounted when mfsfix runs, but it most likely is).
  14. enigma2175

    enigma2175 Long Member

    Apr 1, 2002
    Hmm, I shouldn't post before I think (hint: swap isn't much good on a read-only filesystem :) ) Disregard that last idea... But the swapfile in /var should still work.
  15. Robert S

    Robert S New Member

    Jul 8, 2002
    Glad you're thinking about this issue. A bit of brain-storming never hurts.

    The immediate problem with all your suggestions so far is that DTiVoes and S2's are locked down. The other problem, of course is that this procedure is actually more difficult than the rescue or the manual swap creation (got that written up at last. Thanks to angra).

    If you wanted to use the space on /var for swap, wouldn't it be better to change the partition table to shrink /var and expand partition 8? (You would then run mke2fs on hdx9 and mkswap -v0 on hdx8). This should work on any TiVo.

    My advice is still, expand swap if you can, try not to worry about it if you can't/didn't - just remember the rescue maneuver exists if you need it.
  16. enigma2175

    enigma2175 Long Member

    Apr 1, 2002
    I like to change my partition table as little as possible :D. Assuming this is an upgrade gone wrong, it is very possible there are a number of things in /var that you want to keep. Also, if you have the swap broken up into multiple partitions/files, you can have over 127 MB of swap. I don't know that anybody NEEDS more than 127 MB (with only 16MB of RAM..Yikes!) but it's nice to know that it can be done if needed. Also, if you have (had) a Bash prompt on your TiVo you do not need to take out your drive. The constant swapping of the hard drives back and forth beween the TiVo and the computer is not good for the drive. The interface and jumper pins can get bent or broken, the drive can get dropped, or the controller board can get toasted from an ESD. I prefer to keep my drive in my TiVo whenever possible.
  17. Robert S

    Robert S New Member

    Jul 8, 2002
    Fair enough.

    Merle calculated that 127Mb of swap is enough for the largest currently possible (274Gb) TiVo to recover from a GSOD, but if you're only just over the limit a small swap file on /var might do the trick.
  18. Merle Corey

    Merle Corey New Member

    Aug 25, 2001
    On the one hand, I'm not sure I care. :D My goal when I started testing was to prove one of two things - either that MFST2 was broken and to be avoided at all costs or that it worked (consistently) with the right "adjustments." That the only adjustment needed was a command line flag (and some misconceptions) was pure gravy.

    On the other hand, you raise a valid point - knowing the exact "at risk" threshold could be useful.

    Of course, it's been done. It's in the FAQ, in the segment on Question #7. "Around 150GB" was the result, which dovetails nicely with the predicted 140GiB result.

    It's also been documented by several people that 160GB is too big.

    My replacement TiVo arrives Monday - I can put this to the test then if we think it's really that important. So my questions are:

    * Do we really care where, exactly, it breaks, or is our current information close enough? Is an exact value at all meaningful when it applies only to one specific hardware platform? Is it better to have a more arbitrary but safer threshold value that acts as a general rule of thumb, or to have exact breaking points on all platforms?

    Personally, I think it's close enough. In fact, I'd hedge a little anyway just to play it safe (which is why I still say 140GB most of the time instead of the down to the wire 140GiB/151GB). As Robert has pointed out, that leaves the door open for less painful future upgrades. That said, I'm still willing to look into this.

    Beyond that, nobody is saying there's a crisis - we're just trying to get the word out that people with >140GB configurations may experience greenscreen loops iff they have filesystem errors...and let's face it, (real) filesystem errors aren't that common in non-failing equipment anyway.

    It's a warning. The sky isn't falling, but it may hail later in some areas.

    * If our current information isn't close enough, how close *is* close enough? The exact value? Is confirmation that 160GB is too much but that 140GB is safe good enough?

    Finally, a comment on my formula: It's already proven accurate to within 5MiB on smaller (20 - 120GB) capacities and within 10MiB on larger (240GB). The only reason it's not nailed down tighter than that is that I didn't take the time to determine exact breaking points - the margin of error is the margin from my limited tests, not from the formula. If there's interest, I can also determine the exact margin of error - assuming there's a measurable one - if/when I do any further tests.

    Personally, I don't think there's need for it - we know 64MiB isn't always enough, we know 127MiB is enough for all (128GiB limited) configurations. Does it matter beyond that point?

  19. gigageek

    gigageek New Member

    Aug 23, 2002
    Merle has the right idea, IMO: 127 is an "engineering solution". If it turns out that someone nails down a magic number of 125 (or 118 or ...) for a particular drive capacity, is anyone seriously going to suggest they NOT use 127?

    I also think Hinsdale has a valid point that it is not a mathematical certainty that every >140GB configuration will "not recover" from a GSOD. (The proof, however, is "left to the student". ;) ) While hard drives are fairly reliable, enough of them fail that we probably would have seen more units with the GS/reboot loop if this were guaranteed to be a problem under all circumstances.

    Perhaps Robert's second post could be softened to say:
    This is probably the best advice I've seen on the subject. It tells the reader, "Don't panic, remain calm, your house is probably not on fire. But you might want the phone number of the fire department handy, just in case it catches fire at a later date."

    DCIFRTHS I dumped SDV / cable

    Jan 6, 2000
    I have been following this thread since it's inception, and there has been a lot of information. I would like to ask a very simple question:

    Can I create a TiVo that contains 2 x 120 GB* hard drives using MFS Tools 2.0, and a command line to create a 127 meg swap file, that will successfully complete mfsfix if the TiVo crashes?

    * Please note that when I specify 120GB drives I am going by the size listed on the Maxtor box.

    I'm going to read through the documentation, determine the correct procedure, and then post back here before I upgrade, so that if I make any mistakes, hopefully someone will spot it and point it out to me.

    Thanks all !

Share This Page