1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Fixes for MFSTools 2.0 swap problems

Discussion in 'TiVo Upgrade Center' started by Robert S, Jul 31, 2002.

  1. Sep 6, 2002 #221 of 609
    Robert S

    Robert S New Member

    9,725
    0
    Jul 8, 2002
    Cambridgeshi...
    127Mb swap is a complete fix for all the problems this thread has addressed. It has been tested on a 240Gb TiVo and calculated for a 274Gb TiVo (the largest currently possible).

    If you follow New Hinsdale, MFS Tools 2.0 will expand your swap for you (the -s 127 option).
     
  2. Sep 6, 2002 #222 of 609
    mnc042

    mnc042 MarcBot

    15
    0
    Dec 13, 2000
    Windham, NH USA
    But, reading Hinsdale, if I have a DirecTiVo with a single A drive, and I want to simply add a new, 120GB B drive, isn't there that hole in the instructions that was mentioned back a ways?

    i.e., the instructions basically tell you to backup your A drive, restore that backup on to your new B drive, then expand/mate that new B drive to your original A.

    In this scenario, the A drive's swap is not expanded, and you wind up with 160gb and 64mb swap.

    Or did I miss something in Hinsdale?

    Thanks!
    \marc
     
  3. Sep 6, 2002 #223 of 609
    Robert S

    Robert S New Member

    9,725
    0
    Jul 8, 2002
    Cambridgeshi...
    That's right, Hinsdale's instuctions for that scenario are unhelpful. I don't have any control over what's in Hinsdale - and what he says is right, it's just some of the advice as to when to do what assumes you don't need to expand swap.

    Hinsdale does describe the pipe transfer technique in a different context. Create your A drive with a pipe transfer and MFS Tools 2.0 will copy your recordings and expand your swap (you can even fill the drive and bless in your B drive in one step, if you want).

    To 'fix' New Hinsdale you'd have to add probably a page of discussions of what the swap issues are and how to work around them. For the most part advising people to use -s 127 fixes the problem anyway. The problem seems to be largely theoretical - weaknees say they can't remember ever seeing a machine stuck on a green screen.

    My position - oft repeated - is:

    If you can expand swap when you upgrade, do so (this probably covers 90%+ of upgrades).

    If this is the sort of thing that would worry you, then we have ways of adding swap in every possible upgrade scenario. They are time consuming and/or convoluted.

    If you decide to 'chance it', either because you didn't understand the swap issue when you upgraded, or your upgrade profile doesn't allow you to expand swap easily, you may not hit a green screen in the lifetime of your hard drives. If you do green screen and get stuck, we have a 'rescue' maneuver that will allow your TiVo to recover if running out of swap caused the green screen to stick.
     
  4. Sep 6, 2002 #224 of 609
    mnc042

    mnc042 MarcBot

    15
    0
    Dec 13, 2000
    Windham, NH USA


    Ah okey. So maybe I should pick a different scenario, where I replace my A drive with my new 120GB drive. I notice that in this scenario, since you're doing the restore with the -s 127 option, that should work well. And, I'd do this by:

    In this scenario, I'd either keep my old A drive as a backup on a shelf somewhere, or I could decide to re-bless it as a B drive, as in:

    This sounds like the safest option.

    Agreed, and that is the way I will go.

    Thank you so much for your help!

    \marc
     
  5. Sep 6, 2002 #225 of 609
    DCIFRTHS

    DCIFRTHS I dumped SDV / cable

    2,119
    0
    Jan 6, 2000
    New York
    I will be using two new drives, and I am not going to worry about saving recordings.
     
  6. Sep 7, 2002 #226 of 609
    Merle Corey

    Merle Corey Member

    64
    0
    Aug 25, 2001
    Mount...
    <sigh>

    127MiB is no longer enough for all configurations.

    Jafa really is that cool.

    Theoretically, with quad 200's, the universally safe value is now nuzzling 367MiB of swap. :D

    Literally, that's true.

    Warning: More information about fsfix, MFS, and journaling ahead than anybody really cares about.

    However, for every specific hardware/software combination, there is a point beyond which there is a mathematical certainty that it will not recover with 64MiB of swap. On 16MiB RAM systems, it's around the 151GB/140GiB mark. On 32MiB systems, it *should* be close to the 182GB/170GiB mark.

    fsfix requires a very specific amount of RAM per GiB being repaired; that number isn't going to change, except perhaps in small ways in the form of alterations to MFS between software versions. The bigger change is going to come from either the amount of system RAM (16MiB vs 32MiB) or the memory footprint on that version/platform.

    Assuming that Tiger's -r setting is really changing the default block size (and we have at least some evidence showing that it does), it would appear that fsfix doesn't just operate by looking at the allocation table, it actually requires specific data from each block - quadrupling the block size may mean that it only has to look at one fourth as many blocks, but it also quadruples the amount of data that fsfix needs to snarf from each block. Net gain: 0.

    This makes perfect sense from a journaling standpoint - it would be a bad journal, indeed, that kept the same amount of data for 1MiB and 4MiB block sizes, rendering data recovery *much* more difficult. And yes, I'm implying that the RAM fsfix requires is the amount of RAM needed to load the journal (or some specific subset thereof) into memory. This would be why GSOD looping appears fairly quickly rather than after the amount of time it would take to "process" the maximum supported space of a 64MiB swap. The formula I developed translates into 1 byte of fsfix data per 2048 bytes of filesystem data - a fairly reasonable journal, especially given that 2K of multimedia data isn't going to appear as much more than a stopple... The formula may or may not be dead on accurate, but it is *absolutely* reasonably accurate. (I would suspect that it's actually 1 byte per 2047 bytes of data, but it's going to be damn hard to prove that when dealing with multi-GiB setups and MiB based swaps - I really don't care enough to nail everything down into bytes.)

    At this point it looks like the only way to get fsfix to run in a significantly more efficient manner would be for it to not require that the entirety of MFS space be mapped out at once - in other words, if there were some way to get it to look at one drive - or better yet, one partition - at a time. That, however, is something that I suspect is an MFS "limitation" - if it's all one big virtual space, it probably has to be treated as such for repair purposes.

    (The above is, where applicable, true about fsck as well. It's common to see fsck problems on linux systems with small amounts of RAM and very large partitions when swap is disabled/broken.)

    So let's stop picking nits about swap, and go hound Nick for a release date on the quad drive hack. :D

    MC

    Typos, typos everywhere...
     
  7. Sep 7, 2002 #227 of 609
    hinsdale

    hinsdale New Member

    83
    0
    Apr 30, 2001
    Hinsdale,...
    These numbers sound more reasonable to me... and would mean that you can safely add any drive, 120GB or less (or larger in some circumstances), to any unmodified TiVo using the standard BlessTiVo or mfsadd procedure without increasing swap.

    You can of course easily add any size drive and also increase swap if you dont care about your recordings or dont mind waiting a couple hours for your recordings to copy. Those with 2 drives or previously expanded drives will generally be using the swap expansion methods anyway.

    And as has become evident, swap file increase only becomes required in very rare circumstances to begin with.

    I will be adding some brief swap file requirement notes to the How-To's over the next week (figure out best wording without adding a page) . Just wanted to get some confirmation/concensus of more reasonable numbers (thanks merle) before unneccesarily concerning anyone.
     
  8. Sep 7, 2002 #228 of 609
    hinsdale

    hinsdale New Member

    83
    0
    Apr 30, 2001
    Hinsdale,...
    As I stated before my 160GB DirecTiVo has recovered from GSOD with 64MB swap, and as merles numbers tend to indicate - the threshold number for a DirecTiVo is >180GB so this configuration should not pose any potential problems.
     
  9. Sep 7, 2002 #229 of 609
    mrtickle

    mrtickle Active Member

    2,824
    0
    Aug 26, 2001
    Birmingham, UK
    Not "any" TiVo - all UK TiVos are 40GB to start with so you must NOT add a 120GB drive as a B drive.

    Thanks! If you could make the warning simple and prominent - something like "if you are going over 140GB you must NOT just add a large 2nd drive" then that would be great.
     
  10. Sep 7, 2002 #230 of 609
    hinsdale

    hinsdale New Member

    83
    0
    Apr 30, 2001
    Hinsdale,...
    Sorry, you are correct, the Brits slipped my mind.. "any US TiVo" would have been more accurate.


    I will not be using the 140GB number because it is inaccurate but will be adding a cautionary message basically stating that in the very rare circumstance that an mfsfix utitlity is needed, it will require an increased swap in order to complete if your series 1 standalone is >150GB or if your DirecTiVo or Series 2 unit is >180GB .
     
  11. Sep 7, 2002 #231 of 609
    Merle Corey

    Merle Corey Member

    64
    0
    Aug 25, 2001
    Mount...
    Ok, just to err on the side of caution, I want to make sure this clear: The 32MiB RAM TiVo swap threshold was based entirely on the assumption that it has a similar memory footprint as the 16MiB - in other words, that almost the entire amount of additional RAM (I used 15MiB for my ballpark) can be utilized for fsfix. Since your 160GB DTiVo made it through the GSOD, I'm inclined to think that my assumption was more or less valid (10ish MiB footprint/22ish MiB available for fsfix).

    The entire reason I'd been avoiding making an explicit statement on 32MiB TiVo's is simply because I don't have one to test. It's a best guess on my part, and I'd feel pretty rotten if someone ended up hosed because he saw "180GB is probably safe" and ran with it. I'd be happier if someone with a DTiVo would check this out before it becomes part of the documentation.

    As for phrasing, I agree with mrtickle - keep it really simple and assume that anybody who needs to follow the How-To in a verbatim, step-by-step kind of way (aka the target audience) probably shouldn't be making judgement calls on exactly where the threshold should be. If you feel it's that important to document what we know about swap and thresholds and such, do a separate bit of "advanced" doco specifically on the subject - the most interesting bits are already beyond the scope of the How-To.

    In other words, let "> 140GB" be a general rule for the basic "I just want to add a drive, why are you talking about this linux stuff?" crowd, and keep 150GB & 180GB for the more technically proficient - the people who are most able to make an educated decision, especially keeping in mind future upgrade scenarios. Besides, both of those values are down to the wire, and *neither* has been explicitly tested for exact accuracy. I don't know about you, but I'd feel pretty sheepish if the actual numbers were, say, 149GB and 179GB.

    The bottom line is that we don't need that kind of accuracy - not to prevent leaving it to copy overnight via a pipe transfer, and *certainly* not for saving a measly 63MiB of drive space. If you're really set on listing two thresholds, at least pad 'em a bit - say, 140GB and 160GB. We *know* those are both safe.

    MC
     
  12. Sep 7, 2002 #232 of 609
    mrtickle

    mrtickle Active Member

    2,824
    0
    Aug 26, 2001
    Birmingham, UK
    I still say it needs to be more hard-hitting rather than implying it's rare and doesn't affect many people. It doesn't matter how rare the GSOD is - when it happens it's fatal for people who can't create the "emergency" swap space (and they shouldn't be put in such a situation).

    Far better to say bluntly: "whatever you do, do NOT add a big drive as a B drive if it will take your total above <the number you decide to use>". And then repeat the warning in the section which tells you how to add a B drive.
     
  13. Sep 7, 2002 #233 of 609
    DCIFRTHS

    DCIFRTHS I dumped SDV / cable

    2,119
    0
    Jan 6, 2000
    New York
    I have a question: What you describe does not apply to a person that replaces that A and B drive with a lrger swap, does it?
     
  14. Sep 7, 2002 #234 of 609
    Merle Corey

    Merle Corey Member

    64
    0
    Aug 25, 2001
    Mount...
    No, it doesn't apply when swap has been expanded - we're referring to situations where people keep the factory A drive (or otherwise don't expand swap) and add a large B drive.

    MC
     
  15. R. Kalia

    R. Kalia New Member

    197
    0
    Apr 16, 2002
    After posting (in another context, elsewhere) that I had not seen a GSOD, I immediately got a GSOD.

    The instructions in the 3rd message in this thread got me back up very quickly---many thanks!

    Some idle newbie questions:

    1) why is it that a disk w/o swap can have swap added, but a disk w/64mb swap cannot have swap permanently expanded?

    2) What does the pdisk command "r 5 9" do and why is it needed only if the inactive partition is 4? Looking at the partition list I could see that "r 4 8" switches the functions of partitions 4 and 8, but "r 5 9" seemed to do nothing at all.

    3) Once I put in the temp swap, TiVo healed itself within a few minutes. Does it actually dial in, or does it use files stored somewhere else on the disk?
     
  16. Robert S

    Robert S New Member

    9,725
    0
    Jul 8, 2002
    Cambridgeshi...
    The commands you used are given as the default, the ones with /mad/ are added in parentheses with the words 'if you're using TiVoMad'. Would you like to suggest a clearer form of words for me? The TiVoMad disk doesn't need the magic incantation to boot byteswapping, but the commands are in a different place to MFS Tools.

    The 'disks without swap' have an empty 128Mb partition where MFS Tools failed to create a valid swap partition. The fixes in the top post simply format this partition correctly. If your disk is already full, there's no where to put a bigger swap partition.

    pdisk's r command renumbers partitions. The way it does it is a bit clunky r 7 8 swaps partitions 7 and 8 nicely, but you need two commands to swap 4 and 8 (thanks to angra for pointing that out). Although partition 4 ends up as number 8, the numbers of some of the other partitions are displaced.

    I believe TiVo will dial and collect copies of files (presumably the system animations) it finds corrupt. Presumably your anims were OK and mfsfix just had to fix a small problem. Some people have reported green screens taking hours to recover.

    You did back out to revert to 64Mb of swap, didn't you. (I'm worried I've created a new type of Moron here, although we've obviously not had a documented case yet).
     
  17. R. Kalia

    R. Kalia New Member

    197
    0
    Apr 16, 2002
    I was babbling. I confused Tivomad and MFS Tools.

    In Windows, programs such as PartitionMagic will resize existing partitions and insert new ones and so forth, without losing data. I guess there's nothing like that in Unix; too bad.

    Huh? Bad idea. Who knows, I might get a GSOD again. I'm leaving it the way it is. Partition 4 hasn't been used since 10/2001 so I am sure it isn't needed for anything!


    :)
     
  18. Robert S

    Robert S New Member

    9,725
    0
    Jul 8, 2002
    Cambridgeshi...
    There may be Unix tools that would let you resize Ext2 partitions, but the recordings are stored in a proprietary filing system called MFS. Although MFS Tools can create MFS partitions, it can't resize them.

    I have created an new class of Moron! I wonder if they'll give me credit for it in the FAQ?

    You need the inactive root for the next software update. I don't know precisely what will happed when the next software update comes in, but it'll be really, really nasty - your TiVo will attempt to reformat and install the new software on the partition you're currently using for swap!
     
  19. R. Kalia

    R. Kalia New Member

    197
    0
    Apr 16, 2002
    You mean, the class that is humor-impaired even when smilies are used?
     
  20. Robert S

    Robert S New Member

    9,725
    0
    Jul 8, 2002
    Cambridgeshi...
    Oops!
     

Share This Page