Fixes for MFSTools 2.0 swap problems

[Robert S]

You have probably heard MFS Tools 2.0 has some problems associated with it. We now believe we have solutions for those problems.

The core problem is swap space. To get a safe upgrade with MFS Tools 2.0, make sure you use include the option '-s 127' on your restore - do not go higher than 127 or you'll get no swap at all! Avoid dd + mfsadd profiles if possible (useful for saving recordings), a pipe backup|restore with -s 127 is actually safer.

If you upgraded your TiVo with MFS Tools 2.0, you can use the backdoor codes (http://www.tivocommunity.com/tivo-vb/showthread.php?threadid=26530) to check your kernel log for swap error messages. See below for a detailed explanation of how to do this by Cpen.


The MFS Tools 2.0 documentation claims that 64Mb is enough for any TiVo to recover from a 'green screen' filesystem corruption error. This is not true. If you go over about 140Gb total storage space, TiVo will need more than 64Mb of swap to recover from a green screen, 127Mb appears to be enough for a 274Gb (largest possible capacity) TiVo to recover.

If your TiVo gets stuck in green screen - reboot loop, you may be able to save it by temporarily using your inactive root partition as emergency swap.

See the next post from me below for detailed instructions on how to do this.

If you are upgrading the A drive on a twin drive TiVo and keeping recordings, you may not be able to use MFS Tools 2.0 to increase swap. See the next post from me for details of how to manually add a swap partition.


If you used MFS Tools with -s 128, and therefore got no swap at all, below are some methods that you can use to activate your swap without reinstalling. Choose the one that suits your circumstances best:

The Linux kernel on the TiVo can only use the first 127Mb of a swap partition, even if you've used -s 512 to try and get 512Mb swap.

Ways to fix the problem

There are several ways to fix this problem, pick the one that works best for you.

1. Use shell access to TiVo to run mkswap directly.
If you can get a shell on your TiVo, this is the easiest general solution.

2. Use dd to duplicate existing swap partition
Simple method, but doesn't increase size of swap available, so doesn't fix green screen issue. Will get you working fast.

3. Use copy of mkswap on tools CD
Must have bootable CD.

4. Modify boot scripts to make TiVo run mkswap itself
Quite tricky, especially if you don't know Linux, but will work when other methods are unavailable. Won't work on DTiVoes or Series|2.


For the remainder of this guide we shall assume that your new TiVo drive is on Secondary Master and your old TiVo drive (if required) is on Primary Slave.

Some of these methods require access to the TiVo partitions and/or filing systems. You must use a byteswapping boot disk to do this. Byteswapping mode on the MFS Tools 2.0 boot disk
is broken. You can make it work by typing vmlnodma hdb=bswap hdc=bswap hdd=bswap at the boot prompt. You may prefer to use a different TiVo boot floppy or CD instead.

1. Use shell access to TiVo to run mkswap directly.

Use the shell to type

mkswap /dev/hda8
swapon -a

2. Use dd to duplicate existing swap partition

From TiVo boot disk do

dd if=/dev/hdb8 of=/dev/hdc8

This will only take a few seconds.

This will work even if you used -s to get a larger swap partition, but it will only give you 64Mb of swap, which will be enough to get your TiVo working properly.

3. Use copy of mkswap on tools CD

Boot a TiVo boot CD (but not the MFS Tools 2.0 CD) and do

/sbin/mkswap -v0 /dev/hdc8

Thanks to Merle Corey for getting this working. This is now a verified method.

(It has been suggested that under certain circumstances

/sbin/mkswap -v0 /dev/hdc8 130048

may be required to get working swap).

Options 2 and 3 are probably the only options for DTiVos as you can't get a shell or modify boot scripts on those.

4. Modify boot scripts to make TiVo run mkswap itself

[This is a condensed version of the original description (http://tivocommunity.com/tivo-vb/showthread.php?s=&threadid=66256&pagenumber=2).]

You will need both TiVoMad and Dylan's Boot disk.

On boot TiVo (and indeed all Unix systems) runs a script called /etc/rc.d/rc.sysinit (this is broadly equivalent to C:\AUTOEXEC.BAT on a DOS/Windows system). If we add our mkswap command to this file, the swap file will be initialized when the TiVo boots.

The file we want is in the active root partition.

Boot TiVoMad, use Ctrl-C to break out of the upgrade script.

Do

/mad/edit_bootparms hdc -r

It will reply 'root=/dev/hda4' (or hda7)

Boot Dylan's boot disk. Mount active root partition

mount /dev/hdc7 /mnt

or

mount /dev/hdc4 /mnt

edit sysinit

joe /mnt/etc/rc.d/rc.sysinit

add the following line on a line on its own somewhere near the top.

mkswap /dev/hda8

Save (Ctrl-K then X).
Unmount

umount /mnt

Power down, boot TiVo, check logs, return drive to PC, boot TiVoMad's utility disk (Ctrl-C to get command prompt)

Traverse to rc directory

cd /mnt/etc/rc.d

delete modified sysinit

rm rc.sysinit

replace original version from joe backup

mv rc.sysinit~ rc.sysinit

unmount partition

cd /
umount /mnt

Power down and replace drive in TiVo.


Most of the credit for the fixes in this thread goes to the contributors below, especially Merle Corey for his work on characterising the mfsfix/green screen problem and identifying -s 127 as the solution.

[mtg101]

I just need to clarify something for my upcoming upgrade...

I'm planning to take my single 40GB drive system and add a second drive to it (120GB). If I use MFSTools 2.0 and leave the swap space alone, I should be OK - right?

[Robert S]

If you have more than 140Gb of hard drive space in your TiVo, the original 64Mb of swap space does not provide enough memory for the filesystem checker (mfsfix) to repair corruption to the filesystem. If you get corruption, your screen will turn green as mfsfix runs, but rather than fixing the problem, your TiVo will just reboot endlessly. If you get this problem, this post explains how to fix it.

If you are upgrading the A drive on a twin drive TiVo, see the end of this post for important details.

(Note that mfsfix can't fix everything, so you might get stuck like this for other reasons. If you know you increased swap, or you know your TiVo is smaller than 140Gb, these instructions won't help.)

The Plan

TiVo has two partitions for storing the system software, these are referred to as 'root partitions'. Only one is active at a time. The other is used when a software upgrade comes in. The inactive root partition is formatted and the new software load copied on to it. The boot block is then rewritten and the TiVo reboots to the new active partition.

Most of the time therefore, the inactive root partition is unused. We will format it as a swap partition and recofigure the TiVo to use it instead of the normal swap partition. Once TiVo has repaired itself, we will return things to the way they were to leave the inactive root partition available for the next software upgrade.

Do not forget to reverse the partition changes once your TiVo is working again! Terrible things will happen at the next software upgrade if the partitions are not reset!

We will:


Boot so we can read the TiVo partitions
Identify the inactive root partition
Prepare the inactive partition as a swap partiton
Renumber the partitions
Allow the TiVo to repair itself
Restore the original partition numbers


A. Booting so we can read the TiVo disk

Connect your TiVo's A drive as primary master.

TiVo disks can not be read by a PC because the bytes are written in a different order to that expected by the PC's processor. TiVo boot disks are patched to boot in a 'byteswapped' mode that allows the TiVo data to be read.

Most people these days seem to be using the MFS Tools 2.0 CD. MFS Tools 2.0 has internal support for byteswapping so that CD boots in non-byteswapped mode by default. It offers you the option to boot byteswapped, but this doesn't work. To boot the MFS Tools 2.0 CD byteswapped at the 'boot:' prompt you must type

vmlnodma hda=bswap

If you can't get this to work, try Kazymyr's boot CD or TiVoMad's boot CD. If use a different boot disk, hda will be left unswapped. Use one of the other connectors and adjust the commands below accordingly. (We don't currently know how to do this from a floppy, one of the tools we need is only on the CDs).

Series|2 TiVoes are NOT byteswapped. You should be able to boot the MFS Tools 2.0 as normal.

B. Identifying the Inactive Root Partition

The easiest way is to read the boot block.

edit_bootparms hda -r

(If you're using TiVoMad, the command is:
/mad/edit_bootparms hda -r )

It will reply with the name of the active root partition, which will be either /dev/hda4 or /dev/hda7 - the inactive root partition will be the other one.

If edit_bootparms is unavailable, you can try to mount the root partitions:

mount /dev/hda4 /mnt or mount /dev/hda7 /mnt

If you get an error saying 'Must specify filesystem type', you're trying to mount an empty partition, if the prompt comes back with no message, the filesystem has mounted.

You need to confirm that one root partition mounts and the other doesn't. You unmount partitions with:

umount /mnt (note umount, not unmount).

If both partitions mount, look at the dates on the files with:

ls -l /mnt

The active partition will have newer files.

If neither partition mounts, you haven't booted correctly.

C. Prepare the inactive partition as a swap partition.

Do

mkswap –v0 /dev/hda4

or

mkswap –v0 /dev/hda7

D. Renumber the partitions.

pdisk /dev/hda

On TiVoMad, pdisk is /mad/pdiska.

If your inactive root partition is 4, type:

r 8 4
r 5 9

If your inactive root partition is 7, type:

r 8 7

w (write partition table)
y (confirm, returns you to command prompt)

E. Restore drive to TiVo

Let the TiVo run, it may take a long time, and may reboot occasionally, but it should eventually boot up properly. Remember, you must complete this sequence, your TiVo is not safe when it recovers from the green screen.

F. Restore the original partition numbers

Repeat step D above.

You can now safely replace the drive in the TiVo.

Notes

If you want to backup the partition you're about to take over, boot in a byteswapped mode, with a FAT drive attached (the way backups were done before MFS T2), mount the FAT parition as in Old Hinsdale and do:

dd if=/dev/hdX4 of=/mnt/dos/tivopart.bak

Similarly, you can backup the partition table with

dd if=/dev/hdX1 of=/mnt/dos/tivotbl.bak


This has been tested and verified on all types of TiVo.


Thanks to gigageek for working through this properly to document it and proving it works on DTiVoes. See page 10 of this thread for his write-ups, which include so details I've omitted. Thanks also to dougdmy for being the first to try the emergency swap idea and reporting that it works.




Upgrading Twin-drive TiVoes

There is one case that isn't covered by all the previous discussion: What do you do if you've got a TiVo with the original A drive and a large B drive? If you're on a twin with the original drives, you can just pipe both drives onto the new drive, but you're stuck if the old B drive is to be retained.

You can upgrade by using dd to copy the A drive and then using mfsadd to expand it. This gets you a working TiVo, but doesn't expand your swap. You can just wait for it to green screen (may well never happen) and use the rescue procedure above, but if this is the sort of thing that would worry you, you can manually add a swap partition.

The procedure is very similar to the rescue. Start by dd'ing the A drive on to the new drive. Then reboot into a byteswapping mode (you'll probably want to disconnect the old A drive) as above.

Run pdisk and create a new partition:

pdisk /dev/hda

C
12p
128m

You'll also be asked for the partition label and partition type. "Linux Swap" and "Swap", WITH the quotes, are the 'official' answers, but TiVo doesn't check this detail.

Swap the partition labels around

r 12 8
r 9 13

The TiVoes currently on sale have 13 partitions on their A drives. The commands will be different - C 14p 128m <Return> r 14 8 <Ret> r 9 15 <Ret>.

If pdisk complains 'invalid partition' when you try to create the new partition, see post #7 (http://www.tivocommunity.com/tivo-vb/showthread.php?postid=629559#post629559) for a laborious work-around.

Write and exit:

w
y
q

Then prepare your new swap partiton

mkswap -v0 /dev/hda8

and you can now complete the upgrade with mfsadd as normal:

mfsadd -x /dev/hdX /dev/hdY

You should now find your TiVo boots to give you extra space and report a 130 million byte swap partition in the logs.

Thanks to angra for working through this one.

[mtg101]

Originally posted by Robert S
Yes, swap file issues only affect changes to the A drive, but this is not the same as having a perfect upgrade.

You will be vulnerable to mfsfix problems however you do it,


Thanks for your help. However...

You've lost me there I'm afraid - what's mfsfix? I assume it's a daemon that monitors the file system and fixes it?

And MFSTools 2.0 will stop that from running will it? Is there way to avoid MFSTools 2.0 stopping mfsfix, or will I have to use the older tools to avoid this problem?

[Robert S]

If you're wondering what this 'swap' that everyone's going on about in this thread is, here's a quick explanation. This post was originally a reply to mtg101's question, but that answer is out of date now anyway.

Virtual memory is a technique used by most modern operating systems including Windows, Windows NT and all versions of Unix including the Linux variant used by TiVo. Basically it allows you to substitute cheap (but slow) hard drive space for expensive (but fast) DRAM by tracking which parts of DRAM are currently in use and copying the unused bits to the disk, allowing that space to be used for something else. When the bit that was copied out needs to be used again, it's copied back into DRAM, possibly causing something else to be copied out to disk.

The most common VM system is called 'paging' where blocks of memory (typically 4kb each) are considered individually. This requires fairly sophisticated hardware so earlier systems used a 'swapping' model where entire processes would be copied to disk to enable other processes to run. All modern systems page rather than swap, but the term 'swapping' is used where 'paging' would be more correct.

The other salient point is exactly how the stuff being swapped out it handled. Windows, for example uses an ordinary file. Yours is called c:\windows\win386.swp if you're running Windows 9x with the default settings. Unfortunately there's a lot of housekeeping associated with accessing a file in a filing system (updating dates and file sizes etc) that isn't really useful for a swap file. So Linux also supports swap partitions, which give the VM module direct access to a linear chunk of drive space without having to go through a filing system.


Series 1 stand alone TiVoes ship with 16Mb of DRAM and a 64Mb swap partition, all other TiVoes have 32Mb or RAM. The 'green screen of death' is a filesystem repair utility (broadly equivalent to ScanDisk) called mfsfix. mfsfix requires 1/2Mb of memory for every (binary) gigabyte of disk space that it needs to check. Therefore as shipped, Series 1 stand alones have enough swap for mfsfix to complete with 150Gb of disk space, other TiVoes have enough for 180Gb. It takes about 4 minutes for the TiVo to realise that it doesn't have enough swap for mfsfix and at that point the TiVo reboots.

If you run mfsrestore with -s 127 you'll get 127Mb of swap, which is enough for at least 274Gb of disk space, which is the most the original kernel can handle. If you use Todd Miller's (http://tivocommunity.com/tivo-vb/showthread.php?s=&threadid=83342) patched kernel (ftp://ftp.courtesan.com/pub/millert/TiVo/) you can go well beyond that. This will require more swap, which is possible, but not straight-forward

[ADent]

If you use the older tools, when you run tivomad you can tell it to build a bigger swap parition (it asks if the total drive space is going to be over 140GB).

If I am not mistaken the old method (MFSTOOLS 1.x & TiVoMad) has been proven to work up to 256GiB (or at least 240GB).

S2 users of course are limited to just adding a second drive with MFSTools 1.x and you can not reupgrade a unit w/o losing all of the recordings.

There were arguments that a S2 with an added 120GB drive might be OK since they ship with more memory.

[Robert S]

Very rarely when you try to use pdisk to create a swap partition manually or try to use mfsadd to expand your TiVo image, you'll get an error suggesting that the disk is full. The solution to this is to use pdisk to rebuild the partition table. Doing this will make the partition table represent the full size of the new disk and will allow pdisk or mfsadd to work.

Normally both pdisk and mfsadd will recognise that the drive is larger than the partition table indicates and this process will be unnecessary.

What you need to do is print out the drive's partition table and then use pdisk to create a blank partition table and then reenter all the partitions with exactly the same parameters as before. The reinitialised disk will show the full capacity of the drive, so the Apple_Free partition at the end will be much larger than it was before.


For a Series 1 TiVo you will need to boot in byteswapping mode to get into the partition table. For Series 2 TiVoes, the MFS Tools 2.0 CD will boot in the correct mode anyway.

We're using Kazymyr's boot CD with the hard drive on primary slave. If you force byteswapping mode (see the first post of this thread), the MFS Tools 2.0 disk should work just as well.

Hit the enter key when you see Boot: This will boot Kazymyr's CD in default mode.

You will be prompted to login as "Root" Type root and hit enter.

If you want want to make a backup of your partition table so you can recover if things go wrong, proceed as follows:

You'll need a mountable partition to write the backup files to. This is the same process you use to mount a partition for writing an MFS Tools backup during an upgrade.

At the linux shell prompt (/#) type mount /dev/hda1 /mnt

You can make a text file by typing pdisk -l /dev/hdb > pdisk.txt. If you go back to Windows, this file will be C:\pdisk.txt. You can use WordPad to print it out if you wish (Notepad will be confused by the Unix style line terminations).

You can make a backup of the partition table: dd if=/dev/hdb of=/mnt/tivopart.raw count=32. If you dd this file back onto the drive it will restore the partition table to its original condition.

Disconnect from the partition with umount /mnt

This completes the optional backup phase.


At the next linux shell prompt type pdisk /dev/hdb. This gets you into pdisk which is the program used to change your partitions. It will say: Edit /dev/hdb and take you to a Command (? for help):

Type p and hit Enter. This will show you the partition table for your drive. WRITE THIS DOWN EXACTLY AS IT APPEARS (At least make sure you have all the numbers correct). If you're working on an A drive, you may have 13 partitions to deal with. If you're working on a B drive, there may be as few as three.

At the command:, give an i (Enter) command to initialize the partition table. You will have to confirm that you want to do this by typing y (Enter)

Next, give a C (capital C) command which will allow you to enter the information you wrote down previously.

Partition 1 is the partition table. pdisk has already set that up for you, so you start with partition 2.

First block: (This number will be the number under "base" for partition 2)

Length in blocks: (This number will be the number under "length" for partition 2)

Name: For an A drive this will be "Bootstrap 1", for a B drive, this will be "New MFS Application". You need to type the quotes.

Type: "Image" (A drive) or "MFS" (B drive)

Do a p command to check the partition you just added, make sure it matches what you wrote down.

Repeat the C command and enter the information for partiton the remaining partitions. The last partition - Apple_Free/Extra is just a placeholder for the unallocated space on the drive, you don't create it yourself.

Do a p command to check the partitions again. The table should look just like what you wrote down. The size of the Apple_Free partition should be much bigger - the difference between the old and new drives, infact. pdisk lists the partition sizes in 512-byte blocks, so if you divide by 2 you get the size of the partition in kilobytes.

After confirming that your newly created partitions look exactly like what you wrote down, do a w command. This writes your new partitions into the file. You will have to confirm that you want to do this step by typing y and hitting enter.

You will need to quit pdisk and get back to a linux shell prompt by typing q and hitting enter.

At the next linux shell prompt, do a pdisk -l /dev/hdb command. That is a lower case "L", not a one. You should find that the partition table looks as it did, except for the enlarged Extra partition.

At this point the partition table reflects the true size of the disk. You should find that pdisk now allows you to create your new partition or that mfsadd expands the image as expected.


Thanks to mutant for suggesting the rebuild as a work-around for the problem with creating a new swap partition and outlining the procedure (http://www.tivocommunity.com/tivo-vb/showthread.php?postid=1781967#post1781967), to someone whose nick I've forgotten for suggesting mutant's rebuild would work for the mfsadd problem too and to at4tivo for slogging through the rebuild and writing it up in enough detail to be usable by people who've never used pdisk before.

[Cpen]

I upgraded my TiVo over the weekend and just checked the (kernel) log files using backdoors.

It read:

"activating swap partitions"
"adding swap:65532 swap-space (priority-1)"

Wohoo! Looks good!

Those that are having problems have reported the following message:

"activating swap space"
"unable to find swap-space signature"
"swapon: Invalid argument"

I used MFS Tools 2.0 and Hinsdale's "Upgrade Configuration #6" to take my A & B drive from my Phillips 312 and move/expand them (and my stored recordings/content - 3 hours worth recorded on "high") to a brand new (unused) 120 gig hard drive. This debunks an earlier theory by Hi8 stating that "it appears that the problems appear when trying to copy a FULL TiVo".

From what I've seen so far, it appears that anyone that attempted to use the -s 128 to increase their swap size is a good candidate for this swap problem.

I used the command (from Hisdale's upgrade configuration #6): mfsbackup -Tao - /dev/hda /dev/hdb | mfsrestore -xzpi - /dev/hdc

My Hard Drive was a virgin 120MB 5400rpm Maxtor (no prior image on the drive). I followed Hinsdale's instructions, using the MFS Tools 2.0 to boot from a CD that I made with the ISO image (Edit: I was asked by Robert S if I booted into a swapping or non-swapping mode - to the tell the truth, I don't know - whatever Hindale's default instructions said to do, I followed).


Superman (Tiger) where are you!?

P

PS: To check your log file to see if your swap is working:

1. Reboot/Restart your TiVo (It's important to do this, as this is when the swap gets activated. The sucess or failure of the swap activation gets written to a log file that you can review - keep reading)

2. Enable back doors - The Backdoor mode can be entered using the remote. This is done by doing a "Browse By Name" or "Search by Title" or wherever you can get to the Ouija screen... Currently, the only easy way to exit backdoor mode is to reboot the Tivo. After entering this code, you will see "Backdoors Enabled!" appear briefly, and it will return to Tivo Central. You can verify that backdoors are on in the System Information screen.

1.3 systems: Enter "0V1T" and press Thumbs Up.

2.0 systems: Enter "2 0 TCD" and press Thumbs Up.

2.5 systems: Enter "B D 2 5" and press Thumbs Up.

3.0 systems: Enter "3 0 BC" and press Thumbs Up.

2.5.2 systems (DirecTivo Only): Enter "B M U S 1" and press Thumbs Up.

1.5.2 UK systems: Enter "10J0M" and press Thumbs Up.

3. With backdoors enabled press (on your remote) Clear-Enter-Clear-Thumbsup.

4. You're now in the TiVo Log files. Use the right arrow (the right side of the circular pad on your remote) to get to the "var/log/kernel" page

5. Use page up (channel up on your remote) to jump a page at a time and get to the beginning of the log file(you'll have to press channel up many times).

6. Now - Use page down (channel down on your remote) to review the lines of your log file page by page - the "activating swap" line should appear about 3-5 pages down.

7. To get out of the log file view, press the Tivo button or the left arrow on the circle pad.

8. To then disable the backdoors, reboot/restart your Tivo.

What did you find? What type of Tivo did you upgrade? How did you upgrade (configuration upgrade # from Hinsdale's guide)? What command did you use (did you use -s 128?)?

[Bill Reeves]

Originally posted by Cpen
From what I've seen so far, it appears that anyone that attempted to use the -s 128 to increase their swap size is a good candidate for this swap problem.



Another data point to support this theory. I performed my upgrade last week with the following command:

mfsbackup -Tao - /dev/hdc | mfsrestore -s 128 -xzpi - /dev/hda /dev/hdb

(from the Hinsdale How-To guide, step 10, Upgrade Configuration #3, from a Single-Drive TiVo to New A and New B Drives)

and I just checked my kernel log file and sure enough, the "bad" messages were there.

-- Bill

[mtg101]

Originally posted by Robert S
AFAIK mfsfix just fixes file system problems. There has been no word of the scope of this problem or possible fixes, but a few MFS Tools 2.0 have been stuck in a green screen (mfsfix running) -> reboot (mfsfix crashed) loop, suggesting it is real (presumably these cases have faulty backups or faulty drives leading to problems that mfsfix tries to fix).

The only totally safe option at the moment is to use older tools, but there if you go over about 140Gb mfsfix runs out of swap and crashes so you can't safely bless your 120Gb drive and add it to your A drive without increasing swap.

Thanks for the info. As it's my first upgrade I'll stick to the old tools I think. And I'll leave the original drive alone. Just backup and put on new 120GB drive and expand that. Thus I'll always have the original to do upgrades from in the future. That all sounds about as safe as it's getting to me :)

[hinsdale]

Originally posted by Bill Reeves


Another data point to support this theory. I performed my upgrade last week with the following command:

mfsbackup -Tao - /dev/hdc | mfsrestore -s 128 -xzpi - /dev/hda /dev/hdb

(from the Hinsdale How-To guide, step 10, Upgrade Configuration #3, from a Single-Drive TiVo to New A and New B Drives)

and I just checked my kernel log file and sure enough, the "bad" messages were there.

-- Bill

Realizing the problems with the -s swap file creation in Mfs 2.0, I thought I removed all instances of the -s 128 in the 2.0 How-To several weeks ago. Apparently I missed one, and have removed it. The problem with this parameter was that it not only did not increase the swap to 128, it actually resulted in error with no swapfile.

Having witnessed hundreds of upgraders adding large B drives (since the new 120-160 drives were released at end of last year) without increasing swap and no subsequent pattern of reported unrecoverable GSODS, and also having personally experienced several >140GB machines (standalone and directivo) with unmodified swap file recover from GSOD, I would not be overly concerned about swapfile creation larger than 64MB. Although larger swap certainly doesn hurt, I am not convinced that the swap file limitations for mfsfix to function exist any longer and may have been addressed during TiVo software upgrades above 2.0.

[muchmore44]

Hinsdale,

Do you mean that you feel that using Mfs 2.0 and the most recently updated instructions will successfuly allow upgrades WITHOUT any problems related to the swapfile? Sorry if this is a lame question, but befor I complete a TIVO upgrade, I want to be sure.

Thanks,
Muchmore44

Originally posted by hinsdale


Realizing the problems with the -s swap file creation in Mfs 2.0, I thought I removed all instances of the -s 128 in the 2.0 How-To several weeks ago. Apparently I missed one, and have removed it. The problem with this parameter was that it not only did not increase the swap to 128, it actually resulted in error with no swapfile.

Having witnessed hundreds of upgraders adding large B drives (since the new 120-160 drives were released at end of last year) without increasing swap and no subsequent pattern of reported unrecoverable GSODS, and also having personally experienced several >140GB machines (standalone and directivo) with unmodified swap file recover from GSOD, I would not be overly concerned about swapfile creation larger than 64MB. Although larger swap certainly doesn hurt, I am not convinced that the swap file limitations for mfsfix to function exist any longer and may have been addressed during TiVo software upgrades above 2.0.

[Robert S]

Please do the upgrade and let us know what happens. We need people to try this and report so that we have a good basis to answer questions like this from. At the absolute worst you'll end up having to redo the upgrade with the older tools, which doesn't take very long.

My prediction is that if you use MFS Tools 2.0 to restore to a brand new hard drive it will not give you a working swap partition no matter what options you choose.

[DCIFRTHS]

Originally posted by Robert S


My prediction is that if you use MFS Tools 2.0 to restore to a brand new hard drive it will not give you a working swap partition no matter what options you choose.

For example, do you mean two 120 GIG drives?

[Robert S]

Originally posted by DCIFRTHS

For example, do you mean two 120 GIG drives?


I don't think drive size has anything to do with it. Only the A drive has a swap partition, so it doesn't matter if you use one or two drives.

('Options' is a Unix-ism for command line switches (usually preceded by a - ), and I use it in that rather limited sense.)

People have been talking about whether it's the -s option that causes the problem (and earlier speculation on the role of -p and -z).

I have not seen a report where it can be unequivocably stated that MFS Tools 2.0 has correctly initialised a swap partition.

There have been cases where people have upgraded with MFST2 and got working swap, but so far in all those cases the swap was either copied from another TiVo drive or inherited from an earlier TiVo image on the same drive.

Cpen appeared to state that MFST2 had correctly initialised his swap, but he hasn't replied to my request for more details.

[Cpen]

Robert -

Let me know what else you're looking for and I'll post it here - I'm very interested in figuring out what combinations of hardware and upgrade configurations are leading to problems.

Note - I made changes to my earlier post to clarify where you had questions.

P

[Robert S]

I posted the following on the MFS 2.0 Swap Problem?? thread where you first mentioned your upgrade, sorry if you missed it:

Can you tell us more precisely what you did, specifically:

Was the target disk completely blank or did it have an earlier image on it already?

Did you boot into a swapping or non-swapping mode?

Ideally you'd write up your whole upgrade process and they we'd see if it worked for other people too. In the mean time, the two questions address two specific possibilities:
The swap partition may have been inherited from an earlier install (restoring with MFST1.1 and then restoring over that with MFST2 does seem a valid (if slightly boneheaded) way of getting a working swap partition.

The swap bug may be caused by byte-swapping. Most people do their MFST2 upgrades unswapped because that's the default, perhaps the swap initialiser isn't swap aware (and yes, the turn swap does mean several different things in that sentence!)

[Robert S]

Originally posted by Cpen
Note - I made changes to my earlier post to clarify where you had questions.

I see now, too subtle for me, I'm afraid. Sorry about that. Well, I'm completely stumped, I have seen posts from people who appear to have done exactly and got no swap.

[mtg101]

As mentioned above, I'm planning on replacing my drive with a single 120GB A drive. Would it be useful to people if I tried it first with MFSTools 2.0 to see if I get the swap problem?

[lemketron]

Originally posted by Robert S
...I have not seen a report where it can be unequivocably stated that MFS Tools 2.0 has correctly initialised a swap partition.

There have been cases where people have upgraded with MFST2 and got working swap, but so far in all those cases the swap was either copied from another TiVo drive or inherited from an earlier TiVo image on the same drive.

Cpen appeared to state that MFST2 had correctly initialised his swap, but he hasn't replied to my request for more details.

That may well be true. Tonight we re-did the DirecTiVo that I upgrade the other night. The first time we did backup to file, then restore with -s 128 and the swap was bad. Tonight we did another restore without the -s 128. However, since I lack a DSS dish, there was no way to get to the log so to eliminate another lost day of use we also did a "dd" copy of the swap partition from the original 30gig drive.

I heard back later that the machine is working fine now, and has a valid swap.

Sorry this doesn't help figure out whether MFST2 can restore a valid swap or not, but it does establish (for me anway) that -s 128 is bad.

Now if someone can try a backup | restore without the -s 128 to see if MFST2 can make a valid 64MB swap, that would be great.

One other note Robert: We also tried to boot MFST2 with the "swap" option (in order to try the /sbin/mkswap suggestion) but MFST2 wouldn't boot with "swap" on my system. Don't now if this is a known problem or not, but if refused to boot all the way.
--Steve

[Robert S]

Originally posted by mtg101
As mentioned above, I'm planning on replacing my drive with a single 120GB A drive. Would it be useful to people if I tried it first with MFSTools 2.0 to see if I get the swap problem?

Yes please! Start by trying to replicate Cpen's upgrade, if that works reliably I'll have to rewrite the top post in this thread.

To be a worthwhile test you must ensure that hdx8 doesn't contain a valid swap partition. Obviously the first time the disk will be blank, but on subsequent attempts, dd hdx9 onto hdx8 to make sure it's corrupt.

I'm sure you really just want to get your TiVo working, but if you want to try some variation, I would be interested to see if it makes a difference if MFS Tools runs in a byteswapping environment.

[BareMetal]

Now if someone can try a backup | restore without the -s 128 to see if MFST2 can make a valid 64MB swap, that would be great.

I've worked on three series 2 standalone systems. Two of them i used the -s 128 option on the third one I didn't. The first two have no swap, but the third one does.

[sdunne]

I've just moved from a single drive UK model Tivo (Thompson 6020 series) to dual 160Gb Maxtors. I followed the Hinsdale guide, in particular:
Step 7 Option 1) (but backing up to a linux partition) via "mfsbackup -l 32 -6so /mnt/rh73/TivoBackup /dev/hdc"
Step 8 via "mfsrestore -zpi /mnt/rh73/TivoBackup /dev/hdb
Step 10 Upgrade Configuration #3 via "mfsbackup -Tao /dev/hdc | mfsrestore -xzpi - /dev/hda /dev/hdb"

It all seems to have gone ok. System Info is reporting 94 hours 27 mins best quality (330 hours 19 mins in basic).

I checked my kernel logs and it looks like a new 64Mb swap partition was created, one message in there "Adding Swap 65532k swap-space (priority -1)" suggests.

It's since been rebooted without any apparent problems.

Fingers crossed, but leaving out the -s 128 seems to have worked out OK

Stephen

[klincoln]

On Thursday, I successfully upgraded a previously upgraded SA Sony SVR-2000 with MFSTools 2.0. This TiVo began life as a 30hr single drive unit running 1.3 with a locked Quantum 30GB drive. During the "1.3 era", BlessTiVo was used to add a 60GB Maxtor B drive and expand the unit to roughly 100 hours. It ran in this configuration, with OS upgrades along the way to 3.0, until I saw MFSTools 2.0 and 160GB Maxtor drives for $169 at a local computer show.

The upgrade scenario was Hinsdale's UPGRADE CONFIGURATION #5: Any Dual Drive TiVo to New A and New B Drive (replacing both drives), Option #1 MFSTools single step method using four IDE ports and the MFSTools 2.0 boot floppy. The existing drive pair were reasonably full, using 81GB out of an available 91GB. The two 160GB Maxtor upgrade drives were virgin, fresh out of the shrink wrap. No byteswapping was used. The anecdotal evidence so far seems to point towards the -s option being the root of the swap partition problems. Based on this, the -s option was omitted from the MFSTools command line. The following command was used:

mfsbackup -Tao - /dev/hda /dev/hdb | mfsrestore -xzpi - /dev/hdc /dev/hdd

81GB and 21 HOURS LATER (!), after successful completion of the backup/restore, the new 160GB A and B drives were installed in the TiVo. The unit booted and operated properly. Backdoor inspection of the kernel log confirms that a properly initialized 64MB swap partition is activated:

"activating swap partitions"
"adding swap:65532 swap-space (priority-1)"

Since Thursday, I've booted the unit about a half-dozen times, with the swap partition successfully activating each time. All TiVo functions work properly and reasonably promptly. All menus are properly populated and all migrated recordings play correctly. New recordings work fine at all compression levels. Recording capacities are as expected, ranging from 94.5 hours at best quality to 344.5 hours at basic quality. "Save Disk Space" (VBR) is disabled.

My experience would seem to lend support to the growing evidence that MFSTools 2.0 can create a working swap partition provided you don't try and grow the swap partition via use of the -s option (64MB swap partition correctly created on a virgin drive in this case with the -s option omitted). My gut feeling is that this unit would also successfully complete an mfsfix if it suffered file system corruption and a GSOD. I feel that those MFSTools 2.0 upgraded units that are suffering signal 11 exceptions during mfsfix also don't have a properly initialized swap partition because they used the -s option during their upgrades. If someone can supply more info on using mfsassert to trigger mfsfix, I'd be more than happy to test this hypothesis.

Kevin

[DCIFRTHS]

klincoln: Very nice report. It's easy to read with great attention to details. Welcome to the forum.

[jhburke]

I just spent the last few hours trying to fix the swap on my DirecTiVo 2.5.2 which I upgraded about 5 weeks ago using MFStools 2.0 and the 128M swap option (which of course resulted in an unusable swap partition). I have an old 2 drive Philips DTivo and I upgraded to 2 new Maxtor 120GB drives, preserving all the old recordings. I really haven't noticed any problems despite the lack of swap, but I thought I would try to fix it anyway to try to avoid GSOD...

1. I first tried using the MFStools 2.0 boot CD to run mkswap. The byteswapping option on boot is bad; you have to manually type in "vmlnodma hdb=bswap hdc=bswap hdd=bswap" at the boot prompt to use byteswapping. I ran mkswap /dev/hdc8 without errors ("131xxxK bytes"). I ran swapon /dev/hdc8 and checked /var/log/messages ("Adding swap-space 131064K (priority -1)"). However, when I put it back into the DTiVo, the swap was not used with the same errors.

2. Then I tried using the DTiVoMad boot CD (which does byteswapping by default). Same process, same results - swap partition works in the computer, but not back in the DTiVo.

3. Now I tried to copy the swap partition from my original TiVo A drive to the upgrade drive ("dd if=/dev/hdd8 of=/dev/hdc8"). I know this is only a 64M swap file copied to a 128M swap partition, but I figured "why not?!" since I couldn't think of anything else except to reupgrade and wipe out the last 5 weeks of recordings. It showed no errors, and swapon /dev/hdc8 gave a successful message in /var/log/messages (Adding swap-space 65332K (priority -1)). Now, back in the DTiVo and voila!, I now have a functioning 64M swap file, according to both /var/log/kernel and /var/log/messages via backdoors.

Now, what to think?

Will having a 64M swap file in a 128M swap partition cause problems down the road? So far, so good. I can't tell any difference with the speed of menus or Now Playing list, however.

Maybe MFStools 2.0 initializes the swap file correctly (I never tried to mount it from the computer before running mkswap the first time) but DTiVo cannot use a 128M swap file. This doesn't seem entirely true since in an older swap problem thread someone was able to initialize a 128M swap file by running mkswap from the rc.sysinit file in a stand alone TiVo.

Anyway, just posting my results. I'll let everyone know if I find any problems with my current fix.

John

[klincoln]

Thanks DCIFRTHS. I've been a long time lurker who has sponged volumes of valuable information from the generous folks around here each time I've gone thru one of my flurries of TiVo upgrade madness.

To add to yesterday's post, I supposed that an MFSTools 2.0 upgraded TiVo that had a confirmed working swap partition would subsequently complete an mfsfix/GSOD. I was seeking advice on mfsassert to test the theory. Well, I can now report that theory is ONE HUNDRED PERCENT WRONG! After yesterday's post, I put a bash shell up on the serial port via rc.sysinit.author, fired up tivosh, and executed "mfsassert -please" out of /tvbin. The shell console confirmed "MFS filesystem has been marked inconsistent" among the spew from several unhappy threads as the system rebooted...to the desired (?) GSOD. After roughly three or four minutes, it once again rebooted back to the GSOD. While watching this second reboot sequence I was already taking the side off of the PC, and rummaging for the previous A and B drives and my spare IDE cables. I allowed it to continue this reboot/GSOD cycle for roughly an hour before putting it out of its misery. Upon mounting the "dirty" A drive on the PC, the kernel logs confirm repeated signal 11 failures in mfsfix.

So, in my case, even though the swapfile was properly created, the MFS partitions/data were layed down by MFSTools 2.0 in some fashion that indeed gives mfsfix heartburn.

Deciding to copy my original 100 hour configuration back to the new pair of drives again with MFSTools 2.0 and keep recordings, rather than revert to MFSTools 1.1, I looked a little more closely at the MFSTools 2.0 boot floppy environment. My first (21 Hour!) backup/restore was done with byteswapping enabled/DMA disabled (duh)! My comment of yesterday regarding the original backup/restore being done without byteswapping was wrong. My limited Linux skills get lightly sharpened during my TiVo flurries, but then WAY too many sleep periods elapse between uses. After using "hdparm -d1 /dev/hdx" to enable DMA on all four drives, the 81GB backup/restore takes about 90 minutes! This new configuration also worked correctly in all operational respects. And, since this configuration really did result from a non-byteswapped backup/restore environment, I executed the mfsassert command again for the sake of posterity. Been there, done that, same story. The problem with MFSTools 2.0 MFS partitions and mfsfix is not influenced, in my case, by whether the backup/restore environment is byteswapped or not.

Knowing that I can restore it in less than two hours, I'll continue to run this MFSTools 2.0 config with migrated recordings while I wait for a solution to the mfsfix issue. I'll take my chances against a legitimate GSOD until then. Like most of us, I'd be nowhere without these superb tools that are created by people like Tiger. They've MADE my TiVo experience. I'm confident the problem will be corrected.

A TiVo weekend in the A/C from a blistering Washington, DC...

Thanks,
Kevin

[Robert S]

klincoln, if you're in the mood to experiment, would you like to try MFS Tools 2.0 with -r 0 (1 Mb block size)? Some people have expressed the hope that this might create a recoverable TiVo, but no-one's tested it.

[jhburke]

I had an additional thought about the bad swap files:

I never used TiVoMad to upgrade before, but I understand there is an option to expand the swap file when the combined hard drives are greater than 140 GB to help prevent GSOD from mfsfix. I wonder if someone could provide an image of a good 128MB swap file, which I (and others) could dd onto our mfstools 2.0 expanded swap partition to get a good 128MB swap file. This will be even more important if the 1 MB block size mfs partitions (which might not break mfsfix) is the answer to prevent mfsfix errors, because with the small block size and only 64MB swap, mfsfix will run out of memory. My understanding is that it is not possible to get a bash prompt or to edit rc.sysinit on a DTiVo, so this might be the only way to get a good 128 MB swap. Unless, of course, we fall back to mfstools 1.1. Of course, I guess I could get another blank drive, use the old tools, get a good 128MB swap file, and then dd it to my current upgrade. I just might try that next time I have time (maybe next weekend).

John

[Robert S]

That's not a bad idea. If you start with a blank disk, or zero the partition out (use dd to copy /dev/zero over the partition) before running mkswap then a gzipped copy would be tiny. It may well be that any 128Mb swap partition would be a suitable source (subject to byte-swapping considerations).

[klincoln]

klincoln, if you're in the mood to experiment, would you like to try MFS Tools 2.0 with -r 0 (1 Mb block size)? Some people have expressed the hope that this might create a recoverable TiVo, but no-one's tested it.

Robert S, trying the -r 0 crossed my mind as, under the circumstances, MFST2's default larger block size is a strong suspect in my mind for the cause of mfsfix's heartburn. However, given that I want to confine my testing to a 2x137GB target with preserved recordings, I'm hesitant to invest the time to test -r 0 without the known ability to lay down a valid 128MB swap partition to accompany -r 0's smaller block size on this >140GB configuration. Just as jhburke is creatively mulling the same dilemma, I'm in search of a solution. Without a valid 128MB swap partition, I don't hold out much hope for -r 0.

I'll see what comes up between now and next weekend, and maybe give it another whirl then.

Kevin

[Robert S]

Hmm, swap partitions, it really shouldn't be that hard to make them, you know?

I think the swap partition on a TiVo is just the same as any swap partition on any Linux system. If this is true, then the following should work:

Boot any normal Linux install (I used Tom's Root and Boot Disk (http://www.toms.net/rb/) as it's a convenient single-disk Linux.

Create a blank file of the right size. I did

dd if=/dev/zero of=/mnt/swapfile bs=1024k count=128

(obviously you'll need something mounted on /mnt to take the file).

swapfile is 134,217,728 bytes long. Is this the right size for a 128Mb swap partition?

Then

mkswap /mnt/swapfile

I then did

gzip /mnt/swapfile

and it created swapfile.gz at 131kb.

Now, I'm not sure that file is the right size to copy onto your swap partition, so you might want to try the following:

From a swapping environment:

dd if=/dev/hdx8 of=/mnt/swapfile

From an environment where you have a mkswap command available (Tom's would do nicely)

mkswap /mnt/swapfile

and then from a swapping environment

dd if=/mnt/swapfile of=/dev/hdx8

If this works (I have no easy way to test it) you could gzip swapfile and post it for others to use. If swapfile doesn't compress well, once you know the size of the swap partition you could use dd as above to create a file the same size made of zeros (bs=1 count=<filesize>), run mkswap on that and then gzip it.

This is not a risky procedure, as long as the swap file is no larger than the swap partition you won't overwrite anything - the worst than can happen is that the swap won't activate at boot.

[rswift]

I, too, am suffering the invalid swap file after using the -s 128 option in the MFSTools 2.0 boot CD. I have a Sony T60 with a Maxtor 160G A drive + Maxtor 80G B drive. Reported recording time is 204 hours, and has been working fine for a couple of weeks. After checking the threads here, noticed the swapfile issue, enabled back doors, and discovered the problem.
My quandry/question is whether to do anything now, or wait until:
A) My DirecTiVo crashes, then I take the original 40G Quantum and re-upgrade, or
B) Some genius figures out how to create a 128Mb swapfile usable by TiVo with support for fsfix.

Is it worth trying to copy the 64Mb swap file from my original drive onto the new A drive into the 128Mb partition?
I'm not sure I understand the risk of running without a swapfile, vs using a 64Mb swapfile in my 204 hour TiVo. Can 64Mb swap handle a TiVo with less than 140Gb? If so, maybe I should downgrade to just the single 160G Maxtor. I haven't read anything so far that pinpoints the cause of the fsfix, sigma 11 error, but something gave me the impression that may be related to having insufficient swapfile space for large capacity TiVos.

Thanks to all who've made the upgrades possible. The added capacity is well worth any hassles in opening the box and copying the drives.

rswift

[Robert S]

The swapfile issue and green screen recovery aren't linked in your case. It is true that increased swap space is necessary for earlier versions of MFS Tools and earlier versions of TiVo OS, that no longer applies.

There is currently no known way to make a TiVo that can recover from green screen if it's been upgraded with MFS Tools 2.0.

Copying your 64Mb swap partition from your old drive is a good solution for now. (Unless you'd like to try the swap file method I posted just above you, that post was written with some Linux knowledge assumed, I can explain it more carefully if you're interested.)

If you want a 'safe' TiVo (ie, green screen recovers) you will have to repeat the whole upgrade with pre-MFST2 tools - or wait for a fix for MFST2 problems.

[rswift]

Any guess as to the issues with running without ANY swapfile?? Everything seems to be working with no issues right now. Maybe it's because I haven't let the new drives start getting really full yet. My assumption would be that the swapfile might become very important as the lists of Now Playing, Season Passes, etc. start growing very large, and TiVo needs more memory space to play with the lists??
Thanks for the offer. I've not played with Linux at all, and have limited knowledge of UNIX, mostly Solaris. My first inclination is to dd the old swapfile into the new swap partition.
One thought I had is that if the new swap partition created with the -s 128 option is just slightly too small, even by a byte or so, will a properly created 128 Mb swapfile necessarily fit into the space? Not knowing exactly what the -s option really did, it's hard to predict. In my gut, I fully expect to have to re-upgrade using my old 40Gb original TiVo disk, so I've been copying to VCR most of my "must save" shows. YUK!~
Thanks for the help.
~rswift

[Robert S]

It appears that the problem with the missing swap relates to the program guide functions rather than the recording/playback functions (which are mostly handled in hardware). On UK TiVo's the missing swap prevents TiVo from making daily calls, on US machines it just seems to slow everything down (~24hour indexer runs).

On the space issue, my suggestion above was to dd the current swap partition to a file on your hard drive, therefore when you copy it back it will be precisely the right size. It would be nice if you could validate that, but copying your 64Mb swap partition over would make your problems go away very safely.

[rswift]

I may have to wait until next weekend to play with any of these new ideas. I took the day off work today, but can't do that again for a bit. Let's hope a real fix to this, and the GSOD problem are discovered and posted soon. I'm a bit nervous now.
On an unrelated note, I've had smoke coming out of my TiVo on two occasions, yet the thing still lives on! I fried the original TiVo Quantum drive while copying it the very first time. One of the IC's on the IDE circuit board glowed white hot and smoked out. Thought it was toast, but replaced the little circuit board on the back of the drive with one off of a new spare Quantum, and darned if it didn't come back to life. Then, I smoked the motherboard while replacing the internal fan (trying to make TiVo run cooler) and while the fan supply got fried (I now use power from the 4pin hard drive connectors) the TiVo still lives.
Needless to say I open the cover of my T60 with much trepidation...

[BareMetal]

Originally posted by rswift
Any guess as to the issues with running without ANY swapfile?? Everything seems to be working with no issues right now. Maybe it's because I haven't let the new drives start getting really full yet. My assumption would be that the swapfile might become very important as the lists of Now Playing, Season Passes, etc. start growing very large, and TiVo needs more memory space to play with the lists?? =

My experience with no swap, is a series 2 tivo that starts shutting down services and leaves me with a grey screen every other day. I just need to find some time to fix it..

[Ivor]

Robert S said:

"swapfile is 134,217,728 bytes long. Is this the right size for a 128Mb swap partition?"

From http://www.mcc.ac.uk/Documentation/linux-doc/LDP/sag/x1762.html:

"The Linux memory manager limits the size of each swap space to about 127 MB (for various technical reasons, the actual limit is (4096-10) * 8 * 4096 = 133890048$ bytes, or 127.6875 megabytes). You can, however, use up to 8 swap spaces simultaneously, for a total of almost 1 GB.

With newer kernels and versions of the mkswap command the actual limit depends on architecture."


Could this be the cause of the problem, i.e. MFS2 *is* creating a 128Mb swap file/partition, but TiVo is unable to recognize it because of kernel/hardware limitations?

[enigma2175]

From the mkswap man page:

It is roughly 2GiB on i386, PPC, m68k, ARM, 1GiB on sparc, 512MiB on mips, 128GiB on alpha and 3TiB on sparc64.
Note that before 2.1.117 the kernel allocated one byte for each page, while it now allocates two bytes, so that taking a swap area of 2 GiB in use might require 2 MiB of kernel memory.


It sounds like the tivo kernel is new enough to use 2 bytes per page, and that swap partitions can be up to 2 GiB. Maybe I'm wrong, and who knows what TiVo did to the kernel, but I think we're safe in that regard. However, it might be wise to create 2 64MB swap partitions and use them both (or even better, have one on your second disk, better performance.)

[Robert S]

I don't know if this applies to the version of Linux running on TiVo. It certainly might apply to the version on the Tom's Root and Boot Disk I used (quite an old one, something like 1.7 (of TR&B)).

mkswap did say something like 'truncating swap file to ' and gave a number that I think did start with 133. That was what got me paranoid about file lengths, wondering if dd had created a file that was too large for the TiVo's swap partition.

TiVo are unlikely to have fiddled with the VM on their kernel. I would think a swap partition create the way I suggest would work.

It's not easy to create another swap partition on your B drive as that drive will be full of MFS partitions. TiVo doesn't need the speed of multiple swap partition.

I do wonder, though, if you needed emergency swap for a mfsfix whether you could set up a swap file on /var.

However, it does seem that 64Mb of swap is plenty for TiVo running later than TiVo OS 2.0 and it's only when you get no swap at all that people seem to have real problems.

[enigma2175]

Yeah, a swap file in /var/ might come in super-handy. However, I generally don't have much room in /var and it is getting smaller all the time. But a simple swap file rather than a partition might make it easier for some people to get their TiVos running again.

[Ivor]

I have a Thompson (UK) TiVo with factory fitted 30Gb and 15Gb drives which I want to upgrade to 2 x 120Gb, probably this weekend. Since I'd prefer to have partitions with 1Mb blocks and an increased swap file (to avoid repercussions with fsfix), I don't mind experimenting a bit. I'll try MFStools with the -r 0 and -s 128 options, followed by Robert's procedure above, and let you know what happens. :-)

[Robert S]

Unfortunately we don't know that -r 0 fixes the GSOD problem and I don't know how to run mfsassert without a shell on the TiVo. Can you add mfsassert -please to the start up script the same way I added mkswap? The system should then reboot and green screen. (Does that sound right klincoln?)

[Ivor]

OK - Once I've confirmed I've got a working 128Mb Swap file, I'll add mfsassert -please to rc.sysinit and see if it recovers.

Question - when fsfix finishes repairing a partition successfully, does it usually reboot the box or just continue with startup? If it reboots, I'd need to interrupt the process at some point to prevent getting into a perpetual loop <break - fix - reboot - break - fix - reboot ...>.

[Brad Bishop]

I went back last night and checked my DirecTiVo which I had upgraded with a 80GB drive. It looked fine as far as the swap file goes. I did notice, though, that it was at 64MB instead of what I had been reading here (128MB). I understand what swap files do but I am left wondering if there's some great increase in performance or need for a 128MB swap file.

Also, I'm thinking about taking out the 80GB drive and giving it to my daughter (she's building a PC) and putting a 120GB drive inside it. I remember reading a while ago that TiVo would slow down noticably with drives over 100GB. Is this still an issue or is it related to the swap file size or it being corrupted?

Thanks,

Brad

[Robert S]

Ivor, you'd probably need to catch it on that first reboot and reset the sysinit before letting it green screen.

Brad, I don't think swap size affects the user speed I upgraded with TiVoMad and went to 128Mb and that was noticably slower than the original 40Gb config. That drive then died and MFS Tools 2.0 came out while the drive was being replaced. With MFS Tools 2.0 the TiVo feels very similar to how it did with the 40GB despite now being a 120 and having 11 pages in NP, even with no swap at all! The problem with no swap was that the indexer got stuck, not that the TiVo was unresponsive.

I think you should be find with 64Mb of swap, but there are several methods in this thread for fixing any swap problems you may have.

[Cpen]

I upgraded to a single 120GB Hard drive. I've noticed no change in speed - more importantly, neither has my wife (beleive me, when she knows I've been monkeying with something, she's quick to let me know if I broke it or made it worse).

In the absence of the swap file problem (I don't have the swap problem - did not use -s 128), I think the only real slowdown you will see is in some "live" calculations made by TiVo - For example, the Season Pass manager - it may take much longer to calculate changes/additions as it has many more scenarios and data to sort through to calculate available space, potential conflicts, priorities, etc. It's this kind of live calculation that will get slower with more available/stored information to take into account! (I don't have enought season passes, wish lists, etc to have this get noticably slower, but with the new hard drive, I'm well on my way).

One live process that probably wont be effected is searching for programs to record. You're still working with the same amount of guide data (2 weeks).

I'm guessing that the swap file problem caused MAJOR slow downs because TiVo's indexing engine used the swap file to speed up indexing. Without the ability to index, a bunch of stuff probably get hosed - Program Guide, searching for programs to record, etc.

[deek_man]

I upgraded my Phillips HD112 about three weeks ago and just checked the (kernel) log files using backdoors.

It read:

"activating swap partitions"
"adding swap:65532 swap-space (priority-1)"

I guess that means everything is ok.

I used MFS Tools 2.0 and Hinsdale's Upgrade Configuration #2 (dated July 8, 2002) to restore my original 14 hour drive backup on to a virgin 100 gb Maxtor and I also added a second virgin 100 gb Maxtor.

The restore command I used is as follows:

mfsrestore -zpi /mnt/dos/tivo.bak /dev/hdb

My original backup was made using:

mfsbackup -6so /mnt/dos/tivo.bak /dev/hdc

I used the following command to expand the two drives:

mfsadd -x /dev/hdc /dev/hdb (note: no -s command)

I now have a 250 hour machine and have had no problems that I'm aware of over the past several weeks of use. Hope this helps.;)

[Robert S]

It's actually the mfsrestore command that affects swap, by the time you get to the mfsadd stage, your swap is already set up.

Anyway, thanks for the input.

[deek_man]

Sorry about that. I know where to plug in the cables but my understanding is a little superficial The restore command I used is as follows:

mfsrestore -zpi /mnt/dos/tivo.bak /dev/hdb

My original backup was made using:

mfsbackup -6so /mnt/dos/tivo.bak /dev/hdc

Swap is ok and box works great (see previous post). Hope that's a little more helpful.

[Ivor]

Update:

I started my upgrade yesterday, having first made and tested a divorced backup of my original drives.

mfsbackup -Tao - /dev/hda /dev/hdb | mfsrestore -r 0 -s 128 -xzpi - /dev/hdc /dev/hdd was used to copy my original a and b drives to the new ones using standard sized blocks and an expanded 128Mb Swap partition.

As expected the swap partition signature was not recognized by TiVo, and when the new drives were placed in the system this resulted in a perpetual reboot/GSOD/pause/reboot loop.

The disks were returned the disks to my PC to try and fix the swap partition which was expected to be found on /dev/hdc8.

TiVoMad disagreed, however, giving the following:
Partition check:
hda: hda1 {my Windows hdd}

hdc:Signature 9214, be16 Signature 1492
Blocks in Map - 10
mac st=1 sz=3f name='Apple' t='Apple_partition_map' bim=10
hdc1 mac st 51ff040 sz=1000 name='Bootstrap 1' t='Image' bim=10
hdc2 mac st=5200040 sz=1000 name='Kernel 1' t='Image' bim=10
hdc3 mac st=5201040 sz=40000 name='Root 1' t='Ext2' bim=10
hdc4 mac st=5241040 sz=1000 name='Bootstrap 2' t='Image' bim=10
hdc5 mac st=5242040 sz=1000 name='Kernel 2' t='Image' bim=10
hdc6 mac st=5243040 sz=40000 name='Root 2' t='Ext2' bim=10
hdc7 mac st=5283040 sz=40000 name='Linux swap' t=Swap' bim=10
hdc8 mac st=52c3040 sz=40000 name='/var' t='Ext2' bim=10

hdc9 } various MFS regions
hdc10 }
hdc11 }
hdc12 }
hdc13 }
hdc14 }

hdc15 ... name='Extra' t='Apple_Free' bim=10
hdc16 {blank}

I haven't had a chance to look at my original drives again, but it would seem that the partition labels do not match their contents. The log files I expected to have found in '/var' were all on hdc9; hdc8 was unmountable and hdc7 appeared to contain a root directory rather than a swap file. This is about as far as I got before I had to stop.

Any ideas why the partition names appear to have changed?

Ivor.

[Robert S]

I think this is just the rather confusing layout of the boot-up partition table list. If you use pdisk you'll see the labels are correct, and your experiments with mount also verified it. (7 is root, 8 is unmountable because it's swap and 9 is var.)

If you look closely you'll see that the first partition (Apple_partition_map) doesn't have a number and hdc16 doesn't point at anything. Perhaps on this list the partition names are at the end of the lines and wrap around to the beginning of the next line on an 80 col screen?

[Ivor]

That would make most sense - in fact it seems quite obvious now that you've pointed it out, I'm surprised I didn't think of it! :-)

In any case, I'm about to revisit fixing the swap file shortly. Wish me luck! :-)

[Ivor]

Robert was right - pdisk displays the partitions correctly, the version shown at boot up was what confused me! Anyways...

The new A disk was removed from my Tivo and rc.sysinit was modified to add mkswap /dev/hda8 at the start of the file. The disk was then returned to the TiVo and the box rebooted.

Mkswap ran and the swap file was created successfully (from the kernel log at next reboot):

mkswap: warning: truncating swap area to 130752kB
Setting up swapspace, size = 133885952 bytes
...
Activating swap partitions
Adding Swap: 130684k swap-space (priority -1)


So far so good, but then after a while it continued with:

Time set to: Thu Aug 8 22:58:36 2002
Have a nice day.
Checking for additional disk...
hdb: Generic ATA management
Starting EventSwitcher...
Filesystem is inconsistent - cannot mount!
Writing 207040 bytes to OSD at address 0
Filesystem assert: cbMap >= sBucket at fszone.C line 170 in FsZone::FsZone(bool, enum FsZoneOwner, long unsigned int, long unsigned int, long unsigned int, long unsigned int, long unsigned int, long unsigned int, bool = false)
Filesystem flagged as inconsistent!
Tmk Assertion Failure: cbMap >= sBucket
FsZone::FsZone(bool, enum FsZoneOwner, long unsigned int, long unsigned int, long unsigned int, long unsigned int, long unsigned int, long unsigned int, bool = false), line 170 (fszone.C)
Tmk Fatal Error: Thread main <78> died due to signal -2


at which point, following a few seconds of GSOD, the box reboots...

Back to the drawing board. :)

[kingmiwok]

I upgraded a series 2 AT&Tivo 40 G to a 120/120 with MFS2.0 about a 5 days ago. Unfortunately I used the dreaded -s 128 option and did not discover the swap problems discussed here untill afterward. Dang.

Symptoms: It boots OK and will run for about a day, then I get a UI lockup. It keeps recording though, I just can't get menu control. After reboot it runs fine for another 24hr or so.

I have every reason to believe I have the swapfile problem, but when I access the logs with the backdoor, I do not actually find the....

"activating swap space"
"unable to find swap-space signature"
"swapon: Invalid argument"

....entry. Actually, I can't find anything listed on the swap file activation? Am I looking in the wrong place? var/log/kernel, right?

Is it significant that I can not find this in the log file? I should find it within the time-area for the most recent boot, correct? What other activity in in the log in the same general area?

Any input would be apreciated.

Note: I did find, in another log file, a line that stated (something like) "swap file size 0000000". All the other files/partitions listed had significant file sizes.

Thanks.

[Robert S]

You'll see it fairly early on in the boot sequence, the date will be in January because the clock has not been set at that point. There will be three lines of error messages if it fails, but just one logging success.

[Ivor]

kingmiwok - if you are going to take the drives out anyway to fix the swap file, you might find it quickest to use the TivoMad or Dylan's Boot disk to mount the /var partition (number 9 on the Tivo A drive), load the kernel log into an editor and then search for 'swap'. As Robert says, it should be a few lines before the box corrects the date and time.

The swap file is easy enough to fix by putting mkswap /dev/hda8 at the start of the /dev/hdaX/etc/rc.d/rc.sysinit file (where X is either 4 or 7 depending on your config). See Robert's post at the start of this thread for full details.

--oo0oo--

I completed my second rebuild earlier this morning after running
mfsbackup -Tao - /dev/hda /dev/hdb | mfsrestore -s 256 -xzpi - /dev/hdc /dev/hdd
to create a drive with a 256Mb Swap partition but leaving out the -r 0 parameter [standard block sizes]. I then got TiVo to recreate it's own swap partiton by adding mkswap /dev/hda8 to rc.sysinit as above and tested it by running a full program call. This seems to be working fine [no GSOD] I'm not going to tempt fate by running mfsassert -please on this system despite the larger swap file, since I'm fairly certain this would fail.

Interestingly mkswap did use the full 256Mb allocated to it, although the partition is definitely 256Mb and the MFST2 readme suggests it could use up to 511Mb. Regardless, it is still using the same 133885952 bytes as before, just short of the original 133890048 bytes limitation for early versions of linux...

Thanks to everyone involved for all their help! :)

Ivor.

[kingmiwok]

Thanks for the replies/advise. Much appreciated. I'll tackle it this weekend and report anything out of the ordinary.

Slightly OT... I noticed you guys are from the UK. Many of the posts here are, which leads me to believe (incorrectly?) that Tivo may be more of a phenomenon in the UK than in the US.

In the US there is somewhat of a name recognition for Tivo, but it is hardly a household term. Among my closer friends (mostly computer geek types here in Silicon Valley), NONE have a Tivo. And even though you can't get me to shut up about the virtues of Tivo if asked, I can't get them to take the plunge and get one. It's almost as though they don't get why it's better than their VCR, or don't care. They seem to prefer to watch DVDs they have bought/rented.

I think maybe they see it as just another unwanted distraction competing for their time. We are all in our 30s+. Maybe it's more popular with a younger crowd, but I don't believe so.

Is Tivo huge in the UK? Lots of name recognition with the general population? (I have a fascination with why some products take off and others flop, or never get broad acceptance. Yet will be very popular elsewhere.)

Again, thanks for the tech help.

[Ivor]

Slightly OT... I noticed you guys are from the UK. Many of the posts here are, which leads me to believe (incorrectly?) that Tivo may be more of a phenomenon in the UK than in the US.

This thread http://www.tivocommunity.com/tivo-vb/showthread.php?s=&threadid=70772 suggests there are about 35000 TiVo users in the UK and about 422000 worldwide. That's maybe just over 1 box per 800 households here which would explain why I don't know anyone else who has one... The UK board can probably account for this better than I can. :)

[kingmiwok]

Hmmm, something is not working.

I tried fixes 2 and 4 with (I think) no luck. Tivo is up and running again, but I think my swap file is in the same condition as beforehand and I'll have a lockup in a day or so.

Note: I am not Linux savvy at all, but I do understand what the commands are supposed to do. I'm vcomfortable with DOS so working from a prompt does not bother me.

In a nutshell,
I boot on MadTivo disk and try to run edit_bootparms /dev/hdc -r
I get "file not found" (or whatever it says exactly)

If I try to mount a drive from a Dylan boot, I get a variety of responses, none completely positive. It often wants to know the file system type.

I tried to dd over the 64M swap from the orig Tivo drive and the reply I got back was
0+0
0+0
That seemed wrong.

I notice during boot that it does not read the partitions and can not identify the file system type. It finds the drives OK though.

Question: At the beginning of this thread, for repair method 2, is the dd command correct? States going from hdb8 to hdc9. Should be hdb8 to hdc8, yes?

So if it's possible to guide me a bit... what am I doing wrong? Thanks.

[Robert S]

Yes, well spotted, the swap partition is number 8, of course. You can stare at these things for ages and not see them, you know?

Assuming your working drive is on hdb and your new drive is on hdc your dd command would be

dd if=/dev/hdb8 of=/dev/hdc8

It should say something like 128 records transferred.

edit_bootparms is in the mad directory on TiVoMad, so you need to do

/mad/edit_bootparms ...

(I've fixed this in the top post too).

mount should give no response if it completes. If it asks for a filesystem type you can take it you're either mounting the wrong partition or a byte-swapped one, Linux can auto-detect FAT, Ext2 and NTFS partitions.

Thankyou for your comments, hope things start working for you soon.

[rswift]

Originally posted by kingmiwok

I tried to dd over the 64M swap from the orig Tivo drive and the reply I got back was
0+0
0+0
That seemed wrong.


Kingmiwok, that is exactly what happened to me yesterday. I dug out my original TiVo A drive and slammed in my 2 new TiVo drives, expecting to do a 5 minute dump of the old swap onto the new partition, but got the 0+0 in and 0+0 out message just like you did.
I had all the drives on a second controller card , and used
dd if=/dev/hde8 of=/dev/hdg8, where I connected the original TiVo disk to the primary master on the second controller, and the new TiVo drive (with the bad 128M swap partition) as the secondary master (and the new TiVo B drive on the secondary slave ).
I used the MFSTools 2 boot CD, and all drives were recognized on bootup, so I have no idea what the problem was.
I'm not familiar with Linux, either, so rather than risk doing something bad to my original disk (I have no backup image), I just bailed out, hoping for some help or suggestions online.
My TiVo started acting wierd on Saturday am, rebooting 6 times in fairly rapid succession. It would work for about 5-20 minutes between reboots, then go again. It seems to have fixed itself, or at least settled down for a bit. Its been up for about 36 hours without any issues.

Any new info or suggestions appreciated.
rswift

[kingmiwok]

Robert S:
Thanks for the reply. Based on your comments and what has happened, I have the general feeling that my drives are not being detected correctly when I boot Linux. I'm thinking its not running the commands, mount or dd, etc because it is not seeing the drives in their complete form to begin with.

For example, cd'ed around a bit and spotted the edit_bootparms in the mad dir. It still blew me off when I tried to run it!

I'm not sure why, but maybe my motherboard settings are confusing things.
I think I'll retry with everything dummied down, or do it on another computer where I'm not afraid of messing up the BIOS settings. (I'm always anxious about changing settings on a computer I need for other "real" work.)

rswift:
I feel your pain. I'm doing more rapid reboots now after trying to fix it. Nice. I'm backed though, so I feel a fairly safe in trying things without paying too bad of a price.

My only recommendation it to do what I mentioned above. Maybe try it again with a very simple computer setup. No add-on second controller card, no cards in the slots, the absolute minimum. I think you want HD detection turned off in the BIOS as well. Look at your boot sequence very closely too. Even though my boot was detecting the drives as a physical device, it had lots of comments that indicated that it was not reading the FATS and partitions completely. See if you're getting those too.

I'll post up any success/failure. I'll try to get to it within the next couple days

If it does not work for me, I'm going give up the programs I have recorded since the upgrade and redo the whole process. I was interested in learning a little bit more about the inner workings and fixing the swap directly, but it's becoming a drag. My limited linux skill are really beginning to show.

[rswift]

thanks, kingmiwok. I probably won't crack the cover again until the weekend rolls around, or at least Thursday. I looked at the boot sequence very carefully and didn't see anything that I thought was troublesome. I used the exact same configuration when I copied and expanded my original TiVo drive that I'm trying to use now to repair swap, so I'd think that I would have run into problems the first time around.
I'm also curious as to when it's necessary to boot into byteswapped mode. I tried that with the MFSTools CD and got an error and freeze. Had to reset the PC.
Thanks again for the response.

rswift

[jhburke]

Rswift:

I had success copying the swap file by booting into byteswapping mode. The easiest way to do this is to boot from the older MFSTools disk (i.e. Kazymir's Boot Disk). Byteswapping mode is broken on the new MFSTools 2.0 disk (you can read the thread at the top of this forum) but it can be done if you type "vmlnodma hdb=bswap hdc=bswap hdd=bswap" at the "Boot:" prompt. The error in the boot disk is that the wrong kernel is invoked if you type "boot: swap" . It loads the dma kernel which doesn't work for byteswapping, which causes the lockup. Read the isolinux.cfg file on the cd if you want to see. I think it will work for you in byteswapping mode.

John

[Merle Corey]

So, being the curious sort, and having an extra 40gb drive, I just finished running a proof of concept.

Summary: mfsrestore -r 0 works.

Anyway, here's what I did:

My TiVo: Philips HDR-212, with TurboNet and 100GB HD (TiVoMad style expansion performed Aug '01).

Booted with the MFST2 CD (default/dma enabled mode). Backed up my current TiVo drive with:

mfstools backup -6s -o /mnt/bak/mybackup /dev/hdb

Shutdown, disconnected good drive, installed test drive.

Restored with:

mfstools restore -x -p -s 128 -r 0 /dev/hdb -i /mnt/bak/mybackup

After completion, I rebooted with the TurboNet 3.0 install CD and installed (for tnlited, primarily).

Slapped the drive in the TiVo. Booted. Telneted in, ran:

mkswap /dev/hda8
swapon -a

Rebooted. Swap was accepted.

Ran:

mfsassert -please

Waited nervously through the green screen. It didn't green screen loop - once through the green screen, then back to the happy TiVo intro vid and life as usual.

So MFSTools 2.0 can (apparently, in my case) be made to work completely with a little finagling and not using the defaults.

Anybody else want to make a test run and either confirm or refute?

MC

[Ivor]

That's restoring using a minimal backup, right? i.e. no programs saved?

My attempt at using -r 0 (whilst expanding drives and saving programs) broke even before I had a chance to run mfsassert -please, and didn't recover (see earlier in thread). If I can get hold of a suitable disk I'll try restoring my emergency backup with mfstools restore -x -p -s 128 -r 0 etc. and let you know what happens. It would also be interesting to see how this works on a >140Gb total setup (which requires the larger swap file).

[Merle Corey]

Originally posted by Ivor
That's restoring using a minimal backup, right? i.e. no programs saved?

Correct. Time (not being patient enough to leave my TiVo down that long for a proof of concept) and space (going from 100gb to 40gb, only having a 2.5gb partition for holding any backup images) were my primary reasons for not attempting the complete backup/restore.

Besides, that's the best kind of backup to test -r 0 from, the way I figure - it guarantees all MFS partitions are MFSTools 2.0 created.


It would also be interesting to see how this works on a >140Gb total setup (which requires the larger swap file).

Theoretically, it should make no difference - having repaired the swap and gotten it recognized at 128mb, it shouldn't matter what size the overall setup is... But you're absolutely right, it'll be interesting to see if it works regardless.

MC

[rswift]

Any thoughts on a repair solution for a DTiVo? Doing the mkswap after the drive is back in TiVo isn't really a viable option. I've read elsewhere in this post of doing a mkswap while the drive is out of TiVo, then after the drive is returned to TiVo, the swap isn't readable.

I'm still curious about which copy, backup, restore type commands need to be done in byteswap mode vs. non-byteswap.

Thanks.
rs

[sbourgeo]

So what's the consensus on new expansions?

I have an HDR-312 where I had previously replaced the 30G drive with an 80G drive with TiVoMad/mfstools 1.1.

Based on the Hinsdale directions, I can just put the 80G and my new 160G drive in my PC, boot with the mfstools 2.0 cdrom and run "mfsadd -x" to recognize the new drive.

According to a post from hinsdale in this thread and in tiger's original mfstools 2.0 announcement, the 64M of swap that I currently have should be sufficient. Is that a fair assumption?

I just want to make sure I have enough swap to recover from GSOD, if necessary.



Steve

[Robert S]

The reason Tiger thinks MFS Tools 2 doesn't need enlarged swap is that MFS Tools 2 uses larger blocks sizes, so mfsfix needs less space to run. However, MFS Tools 2.0 breaks mfsfix anyway, so this is hard to verify.

You should be OK if you use mfsadd with -r 0, but this hasn't really been verified. This doesn't use the larger block size, so mfsfix should complete, but you'll have to trust that both Hinsdale's and Merle Corey's upgrades are representative of your situation.

[dobbie1]

I have been following the discussion with much interest, hoping to confirm that the lockups I am receiving on a daily basis now are the result of not having created a swap space when I upgraded my Series2 unit. I don't know if MFS Tools 2 has been updated but I used the version first released. I checked my log files and it does not appear that I have any swap space on the unit. I plan on trying to use the option 2 to fix but since I am not familiar with Linux, but do follow directions very well, are the commands listed for the option 2 fix all I will need execute in order to create a swap? Any comments, suggestions will be appreciated.

Regards,
dobbie1

[Robert S]

First make sure you really do have a swap problem. You should see three lines giving a clear error message about problems activating swap.

Method 2 should work for you, although I don't think anyone's actually tried it on a Series 2. S2's don't need byteswapping, so you should just boot the MFST2 CD as normal.

[Merle Corey]

Originally posted by Robert S
You should be OK if you use mfsadd with -r 0, but this hasn't really been verified.

Not thoroughly, anyway. More datapoints are definitely better.

Of course, Steve's also looking at a >140GB capacity, so he has a choice between playing guinea pig or playing it safe.

Specifically, while -r 0 works (at least some of the time), you (Steve) will theoretically need the expanded swap space as well, and will need to do the song and dance to expand it (Robert nailed a good summary of the possible methods in the first message in this thread).

The guinea pig aspect comes in with not expanding the swap - according to one of Hinsdale's comments somewhere in all of this, the 64MB swap "limitation" may have been resolved in later versions of the TiVo software. If you're the gambling type, you can try not expanding and see what happens.

Of course, failure would essentially result in loss of whatever you've got on the drive, requiring you to reimage... :D

Me, I think I'm going to get myself a new drive and play a little more - I'll run another test scenario and see exactly what happens with >140GB capacity, 64MB swap, and that ever so enjoyable mfsassert -please.

I'll post my results when I've got them, but nobody hold off anything on my account - I haven't even ordered the new drive yet. I'm at least a week or two away from being able to sit down and play again.

And on a not-unrelated note, does anybody have any idea what's become of Tiger in the last few months?

MC

[Robert S]

Steve's in a rather sticky situation already. He can't just expand swap because his A drive is already full. He just wants to add a B drive. This should work, there's even a chance it might survive a green screen, but there's no easy way to guarantee it.

It might be safer to do a pipe transfer saving all recordings (see MFS Tools README, RESTORE section) and using the -xp, -r 0 and -s 128 options to move everything onto the new drive, expanding swap (will need to use one of the repair methods at the top) and then, once the new drive is verified, mfsadd the old drive as the B drive.

By the way, Merle, what size was your backup image? Did MFS Tools create a new pair of MFS partitions in addition to the original TiVo-created pair?

[sbourgeo]

Originally posted by Robert S
Steve's in a rather sticky situation already. He can't just expand swap because his A drive is already full. He just wants to add a B drive. This should work, there's even a chance it might survive a green screen, but there's no easy way to guarantee it.

It might be safer to do a pipe transfer saving all recordings (see MFS Tools README, RESTORE section) and using the -xp, -r 0 and -s 128 options to move everything onto the new drive, expanding swap (will need to use one of the repair methods at the top) and then, once the new drive is verified, mfsadd the old drive as the B drive.


Actually, I thought about doing that...

If I just run mfsadd to partition the 160G drive, then I really don't have the ability to create another swap partition before (since mfsadd would wipe it out) or after (no space left on new drive) I run it. I wish I had selected 128M swap size when I upgraded the first time, but didn't see the need to at that time... :(

I may just hold off on adding the 160G for a little while. I have a bunch of recordings that I'd prefer to keep, so that is my top priority at the moment.

Maybe by then Tiger will get back from vacation or from doing REAL work and can look into this stuff. :)



Steve

[Merle Corey]

The backup image is a stripped down, formerly upgraded, 20 hour image. (The original upgrade was to 100gb - my understanding is that the -s was required in order to destroy the expanded partitions and let everything be done over). When I restored via -x, it created a new pair of MFS partitions to fit the 40gb drive I was dropping back on to.

If you like, I can re-run the process tonight without -x in the restore and confirm that only a base 20 hour image is restored, and rerun afterwards with mfsadd -x -r 0 /dev/hdb. (Running mfsinfo in between to get the particulars.)

You're exactly right regarding adding the B drive, and making the new drive into the A being the safest route; I knew I was missing an obvious problem there.

MC

[Merle Corey]

Originally posted by sbourgeo
If I just run mfsadd to partition the 160G drive, then I really don't have the ability to create another swap partition before (since mfsadd would wipe it out) or after (no space left on new drive) I run it.

You can do it during - if you do the piped transfer Robert refers to, you'll preserve all your shows, plus create a new swap on the fly, in the process of migrating everything from the 80GB to the 160GB. You can do the expansion at the same time, and even use -p to optimize the layout.

After that's done (and tested, and the swap fixed), you would run mfsadd to make your 80GB the B drive to your new 160GB A drive.

It's a slow process, mind you, but it's definitely the safest route, and accomplishes everything you need to do.

MC

[Robert S]

That's fine, Merle. I just wanted to be sure you hadn't put a 40Gb image on a 40Gb drive, which would've meant no MFS Tools 2.0-created partitions and rendered your success with mfsfix meaningless.

The two new partitions mean this was a valid test.

I suppose showing the same upgrade fails with -r 2 would mean something, but we're pretty sure that's the case. A test on a big drive is the really interesting case, particularly if -r 0 doesn't slow things down too much (keep the -p partition layout changes).

(Just paranoid, really.)

[Merle Corey]

Originally posted by Robert S
I suppose showing the same upgrade fails with -r 2 would mean something, but we're pretty sure that's the case. A test on a big drive is the really interesting case, particularly if -r 0 doesn't slow things down too much (keep the -p partition layout changes).

Hey, I can aim for failure, too. I'll run with -r 2 tonight just to prove the point (probably).

How big is big? I was planning on just going to 120GB for my new purchase, and merging it with the 40GB for large testing. That's more than a single 160GB, but it's way smaller than any dual-large config.

(I'll freely admit that my curiosity and testing aren't entirely selfless - my 100GB drive has started making the kachunk-kachunk-kachunk of doom, and I'd rather get it out of my TiVo before it's dead. Gotta make sure I've got a clean upgrade path. :D)

MC

[Robert S]

If you can get a 120 + 40 to recover from a green screen, I'd be convinced (already close to convinced). You'd then have a two-drive configuration, much larger than any standard TiVo, which would cover most of the potential problems your current experiments haven't.

[Merle Corey]

Ok, so my current laundry list is:

40 w/ -r 2: Proof of failure

120+40 w/128MB swap & -r 0: Proof of success in large setups.

120+40 w/64MB swap & -r 0: Proof of possible mfsfix swap requirement resolution (in 3.0 at least).

Am I missing anything? Any requests? I'm limited by hardware to SA series 1 only, so I can't do any DTiVo experimentation.

MC

[dobbie1]

[QUOTE]Originally posted by Robert S
[B]First make sure you really do have a swap problem. You should see three lines giving a clear error message about problems activating swap.

If the swap is okay, would I not expect to see some message indicating the swap was activated?

Regards,
dobbie1

[sbourgeo]

Originally posted by Merle Corey


You can do it during - if you do the piped transfer Robert refers to, you'll preserve all your shows, plus create a new swap on the fly, in the process of migrating everything from the 80GB to the 160GB. You can do the expansion at the same time, and even use -p to optimize the layout.

After that's done (and tested, and the swap fixed), you would run mfsadd to make your 80GB the B drive to your new 160GB A drive.

It's a slow process, mind you, but it's definitely the safest route, and accomplishes everything you need to do.

MC

I'd rather hold off on the klugey stuff (like the swap workarounds) until the process is ironed out a little.

The TiVo I want to add it to is currently our "production" model and I want to minimize the risk as much as possible.


Steve

[Robert S]

If your swap is OK you'll see one line (very easy to miss) saying something like 'Activating swap <size>, priority -1'.

[dobbie1]

I rebooted, then activated backdoors, went to /var/log/kernel and did not see any entry as described above. The only entry close was "Initialize with 1 live caches."

Thanks for your responses.

[Merle Corey]

Originally posted by dobbie1
I rebooted, then activated backdoors, went to /var/log/kernel and did not see any entry as described above. The only entry close was "Initialize with 1 live caches."

Straight from my kernel log:

Jan 1 00:00:15 (none) kernel: Activating swap partitions
Jan 1 00:00:15 (none) kernel: Adding Swap: 130684k swap-space (priority -1)

It's very early in the boot process (you can tell because the date/time isn't set yet), but it's pretty easy to miss. The line you're looking at is around (ballpark number, YMMV) 100 lines too far down. (Or, an unknown number too soon, depending on where your logfile starts.)

Hope that helps!

MC

[horwitz]

So what is the latest "best idea" for upgrading? I am going from a factory HDR312 (13.6+13.6) to 120. MFSTools 2.0 a la Hinsdale's page (what about all these flags people are talking about? -xp, -r 0, -s 128, etc.)? Will the old 1.1 solution on Hinsdale's page work; what disadvantages are there to the old procedure (aside from it taking longer)? BTW, I hope to upgrade to 120+120 sometime in the not-too-distant future.

[Robert S]

As you might guess, this is still a matter for debate, however, I think we can say the following:

If you want to be absolutely safe, stay away from MFS Tools 2.0! Use the older, proven tools in the original Hinsdale.

If you have to use MFS Tools 2.0 (there are some upgrades that can not be done with the old tools), avoid the -s option (increase swap partition file) and use -r 0 to force 1Mb block size.

The -x and -p options (you've combined them as -xp) cause MFS Tools to fill the hard drive with Tivo partitions and use an alternate partition layout that's believed to be faster.

If you want the fastest possible TiVo and are prepared to accept the risk that your TiVo might lose all its recordings without warning (if you get filesystem corruption), use MFS Tools 2.0 as described in new Hinsdale, but check your swap as described in the first post in this thread.

[SAFW]

So does anyone know of a method to get MFStools 1.0 working with devices recognized as hdf, hdg, hdh, and hdi (the Promise RAID controller takes up hda, hdb, hdc, and hde)? How about TiVoMad? Is there an integrated restore/expand option like -zxpi in MFStools 2.0?

MFStools 2.0 worked like a charm for me with -zxpi, but now I'm concerned that I'll get the GSOD somewhere down the line.

SAFW

[Merle Corey]

I ran several tests tonight, and here are the results:

First up, swap space tests. (All tests are to the same 40GB drive.) First with:

mfsrestore -p -x -r 2 -i /mnt/bak/mybackup /dev/hdb

I can confirm that the "default" swap works fine out of the gate. As expected, a 64mb swap partition was found on first boot; it's not a restored swap either, as the swap on my old drive was 128MB. So we have a little more confirmation on that front.

A little weirder: I ran the experiment again, this time with the 128MB swap (-s 128), and rebuilt the swap (mkswap /dev/hda8 ; swapon -a) via telnet. The swap definitely mounted on the command, and remounted after rebooting, but no further log messages were generated by it. So everyone who couldn't find any message at all? You may not be crazy, and your swap may be fine. Don't ask me, I can't explain this one.

A third time through, and I tried another swap prep mechanism: Building the swapfile with the mkswap on the bootdisk. I used: mkswap -v0 /dev/hdb8

This worked. Flawlessly. Tricks involved were having byteswapping enabled and forcing the old style (-v0) swapfile. The default was to a new style swap - I knew something was up as soon as that happened because the messages were completely different. Forcing -v0 generated the same messages as doing mkswap via telnet to TiVo, and worked immediately on insertion.

I'd love to hear from someone with a DTiVo who's willing to try this. rswift? I believe you were looking for a mechanism to fix your swap?

Now for the really weird part: It successfully ran through mfsassert -please with -r 2. This surprised me so much that I ran it again after zeroing out the drive - and it worked again. So just for giggles, I ran it through a third time, this time without specifying any -r value (theoretically defaulting to -r 2, but I wasn't taking it for granted). It worked yet again.

The only conclusions I can draw are either:

1) -r works for all values and drive sizes, and any mfsassert/fsfix issues are unrelated

2) -r works for all values of drives under a certain size, but fails for drives/combos over a certain size

3) -r works with a stripped out backup (-s) but not for modifying "existing" data - preservation of programs, adding a B drive to an already expanded A drive, etc

I'd love to prove #1 is true, but I suspect the reality is #2 or #3. It may be that only "native" MFST2 mods work, and that piggybacking with other expansion mods doesn't. It may be yet another space issue. I can't work any further on most of these until my new drive arrives (some time early next week, best guess).

If anybody has anything further they'd like me to try, let me know. I plan to try again, this time to do a combo of:

mfsrestore -p -r 0 -i /mnt/bak/mybackup /dev/hdb

and

mfsadd -r 2 -X /dev/hdb

to simulate item #3 (as if the data had been dd'd over) and see if that makes anything break, but I'm not going to have time to do that until some time this weekend.

MC

[stormsweeper]

I think I was one of the first to report this on my SVR-2000. One part of the problem is that the older swap version is limited to 127MB. So you should get a message that the space was truncated to 133890048 or so when you run mkswap from the tivo.

Since then it's been happy.

[dougdmy]

I posted this in another thread, but thought that it was more appropriate to this thread.
__________

I used mfstools-2.0 and ran well for a couple of months. Then a few days ago, I had the GSOD/reboot loop problem. I had so many recordings on there, too. But I was very concerned about my Season Passes and Wishlists. Here's what I did:

I took out the two 120GB drives from my TiVo and used mfstools-1.1 to create a backup image. I got a message stating something about an inode problem and that it was trying to read the backup table (or something to that effect). The backup successfully completed. Then I restored the image using mfstools-1.1 to a new 80GB drive. I put the 80GB drive in my TiVo. On power-up, I got the GSOD and much to my delight, it did not reboot. It spent about 20 minutes fixing itself and then it rebooted and all seemed well. I deleted what was listed in my Now Showing list, powered down, and TiVoMadded the 80GB drive to utilize the whole thing.

So far (one day later) all seems well.

This got me thinking. Can I use mfstools-1.1 to create a backup image of my original 120GB+120GB configuration (streams and all) and then use mfstools-1.1 to restore it? Hopefully during the restore process mfstools-1.1 will create the mfs partitions in a way that will allow mfsfix to do its job and I'll have minimal programming loss.

Has anyone else tried this? It's an expensive thing for me to try because I would have to go out and buy enough disk space to store the huge backup file.

I appreciate any and all suggestions or ideas.

Doug (who is very sad because he lost the entire season of The Shield that he hasn't watched yet)

[Robert S]

I think this has been tried, without success. One obvious question would be whether MFST1 can read the larger blocks created by MFST2 (your minimal backup only captures the original TiVo partitions (1Mb blocks, even on an MFST2 restore) and doesn't touch the new partitions).

[Merle Corey]

For what it's worth, I'm starting to suspect that the block size problem isn't.

This morning's test run involved making a fresh backup of my original 100GB (20 hour image, TiVoMad expanded) drive with MFST1:

mfstools backup -6so /mnt/bak/mybackup /dev/hdb

I restored with MFST1 to a 40GB drive:

mfstools restore -zi /mnt/bak/mybackup /dev/hdb

This gave me a 20 hour, unexpanded image on my 40GB drive. Booted, confirmed functionality, and even ran mfsassert -please for kicks (fsfix was successful). I then went back and expanded with MFST2:

mfsadd -r 2 -x /dev/hdb

Booted again, confirmed everything was golden, mfsassert -please, and yet another successful run of fsfix.

I'm starting to nurse a pet theory, and that theory is that the decreased swap needed by the larger block size may not be directly proportional to the increase in MFS block size.

To put it differently: Kevin (klincoln) is the only other person to test MFST2 via mfsassert with a known good swap. He tested with dual 160's and a 64MB swap - if I'm right, he may need 128MB swap to make it work, even with the default (-r 2) MFST2 block size. Kevin? You still watching this? Wanna take a crack at it?

Meanwhile, I can't find a way to break things with a 40GB drive, so there's nothing more I can do for now. My new drive is arriving Wednesday, and I should be done with burning it in by Friday. As an added bonus, I may be able to lay my hands (temporarily) on another 120GB drive, allowing me to test an extremely large (though 34GB shy of maxed out) config.

MC

[Robert S]

How about reducing your swap? You could try swapoff to turn off all swap. You could also try manufacturing a smaller swap file and using that. If you could find a threshold below which mfsfix wouldn't complete, that would be an interesting find.

However, if you look at embeem's post on Tiger's MFS Tools 2 thread, it looks like he understands the swap issue.

[Merle Corey]

Originally posted by Robert S
How about reducing your swap?
I'm a little confused now. We already know that swap plays a critical role in successfully running fsfix. Or are you suggesting the establishment of multiple data points, whereby we can determine the minimum amount of swap required by, say, 40GB at -r 0 vs -r 2, and by extension determine whether the the swap size is directly proportional to the block size?

Heck, I think I may go do that anyway now, just for kicks. :D The real testing may once again have to wait for me to get the new drive, though, as I'm going to be playing with really small partitions to scale it down for 40GB (ballpark "minimum" at -r 0 should be around 18MB, meaning a theoretical 9MB for -r 2).
However, if you look at embeem's post on Tiger's MFS Tools 2 thread, it looks like he understands the swap issue.
I didn't get that out of it. He just asserted that there was a problem that prevented fsfix from running. He wasn't even certain that it was a MFST2 problem, and said he didn't have the resources to check.

It may be that we're flogging a dead horse at this stage, but I'm not going to be satisfied without definitive success either.

MC

[Robert S]

Yes, exactly. If reducing the swap breaks mfsfix on a 40Gb (and it may well be it's too small, but you don't have a bigger drive ATM), you should be able to find a threshold that separates sufficient from insufficient swap. It might then be possible to estimate how much swap a given configuration requires, and to test the theory that higher -r numbers require less swap for mfsfix.

To some extent you can't add much more to our knowledge because we need more data from other TiVoes (there are enough people reporting GSOD loops to suggest it's a real problem). On the other hand, if you're enjoying the investigation I'm happy to suggest new experiments. You've certainly shown that saying 'MFST2 breaks mfsfix' is an over-simplification.

I've PM'd embeem to ask what his config was - there's no earthly point in arguing what someone else meant :)

[jhburke]

I just wanted to post my confirmation of Merle Corey's findings about running mkswap from the boot disk. I had previously tried to run mkswap from the MFSTools 2.0 cd boot disk, and ended up with a swap file which I could mount from the boot disk, but would not mount from the TiVo. Now, thanks to Merle, I ran "mkswap -v0 /dev/hdc8" from a byteswapped boot cd, and my kernel logs now show "Adding Swap: 130744k swap-space (priority -1)". I also received the error message from the boot disk about truncating the swap file from 133xxxK to 130xxxK as Merle also noted. I don't notice anything running any faster on the TiVo, however.

I'd be willing to try mfsassert -please, but since I have a DTiVo, I can't easily get a prompt. I guess I'll just have to wait for the green screen!

BTW, I've had 2 TiVos now for greater than 2 and 1 years, and have never seen the green screen, either before or after upgrading. Are they common or am I just having good luck?

John

[Robert S]

I ran "mkswap -v0 /dev/hdc8" from a byteswapped boot CD

What does -v0 do? (Sorry, too lazy to look at mkswap manpage :) )

As for mfsassert, unless mfsfix completes it will blow away your current TiVo install, forcing you to restore from backup - so not recommended unless you've got a spare disk like Merle.

[horwitz]

I did an upgrade last night (thanks to Tiger and Hinsdale, of course and to all the folks who answered recent questions (e.g., Robert S)) of an HDR312 from 13.6+13.6 to a single 120. Everything seems to working fine. I tried to look through the log file for "activating swap" (just after a reboot), but never found that text. Will it always be there? (I never even found "swap".) TIA

[Robert S]

It should be there. The error messages from a failed swap activation are much easier to see. If you're not having problems with indexing, I think it's safe to assume you're OK.

[Merle Corey]

Ok, so I flubbed my guesstimate before. The magic number for getting fsfix to fail under -r 0 on a 40GB drive is 12MB of swap or less. 13MB works.

Useful trivia: If you have a TiVo that's greenscreen looping due to bad swapfile, but is otherwise of sufficient size, you can run mkswap -v0 /dev/hdX8 from the (byteswapped) boot disk and the TiVo will recognize the corrected swap, allowing fsfix to complete.

In other words, anyone who is up the creek because of having used -s 128...? It's fixable, without reimaging, even if the greenscreen loop has already begun.

Not-as-useful trivia: mkswap accepts a size argument, as long as the size isn't larger than the target partition. Very useful for, say, playing with swap sizes. :D

Useful datapoint: fsfix fails with signal 11 when there's not enough swap space. This is the same signal Kevin got with his dual 160's and a 64MB swap.

Robert, the short answer your question: -v0 sets an old style swap, the same variety TiVo uses. The mkswap included in the bootdisks defaults to the new style, which is incompatible. (Of course, the newer version is also more efficient, on those systems that recognize it...)

John, glad to hear it worked for you. I can't really answer your question about greenscreens - I've had my TiVo's (2) for a year, and never had one that I hadn't caused. Others aren't as lucky. The swap shouldn't make any performance difference - having run top on my TiVo a few times, I can confirm that the swap doesn't get touched in normal use, and even most abnormal use. I'm not an authority on the subject, but it may only really get utilized when fsfix is running.

I'm off to generate a magic number for -r 2. I'm not certain that I'm going to be able to determine a scale based on these results alone, so I may redo this process when the new drive arrives.

MC

(Edit: Typo fix)

[Robert S]

So, your theory at the moment is that TiVoes caught in a green screen loop are just running out of swap afterall?

Two suggestions for making more swap for emergencies:

Create a swap file on /var (hda9 is 64Mb, of which most is probably rubbish).

Create a swap partition on your inactive root partition. hda4/7 is 128Mb, you can have multiple swap partitions/file active even if no single swap entity can be more than 128Mb.

Both of those are fairly heavy duty Unix hacks (and wouldn't work on DTiVoes), but they might provide an emergency rescue option.

I was beginning to wonder if you were a special case, but being able to break/fix mfsfix by tweaking the swap size suggests these results will apply to all TiVoes. You may very well have found a fix for the second MFS Tools 2.0 problem.

[Merle Corey]

Yep, that's my working theory.

Oh, and a note on the emergency swap suggestions: If you need to implement one of these, you'll also need to update /etc/fstab in the active root partition to include the added swap space(s). Theoretically speaking, if you have a single drive config, you could also put a second drive in place with a swap partition or two on it. Again, caveats regarding fstab and hackable TiVo's apply.

I finished the second phase of testing, and got some interesting results. The magic swap number for -r 2? 13MiB.

Either the improvement is subtle and beyond my measurement (I moved in increments of 1MiB, so theoretically, there could be up to 1023KiB difference) or the improvement isn't significant on a 40GB drive (a possibility, since only about 18GB is providing "new" space). Either way, it looks like the swap requirements for MFST2 may be almost exactly the same as they were for all previous methods.

It wouldn't be a bug in MFST2 as much as a serious misconception as to how fsfix works. On top of that, it seems to indicate that 64MiB of swap probably isn't enough for the larger drive configs.

For what it's worth, I hacked out some estimates of swap requirements and system RAM utilization. Specifically, about 6MiB of system RAM is used in the fsfix process, and the total RAM (system & swap) needed for a capacity is about 512KiB per GiB of total drive capacity.

It can be approximated as:

(SwapMiB + 6MiB) = (DriveCapacityGiB*(512KiB/GiB))/(1024KiB/MiB)

Or more simply:

.5*DriveCapacityGiB = TotalSwapMiB

If I'm right, this means that people with dual 160's should have enough swap at 128MiB, regardless of the method used. Of course, this is on a SA w/16MiB RAM - people with 32MiB are probably in slightly better shape.

(For the pedantically inclined, I specified the difference between binary GiB/MiB/KiB and decimal GB/MB/KB. This isn't something I usually pay much attention to, but it plays a significant role here - DriveCapacity in GiB vs GB is the difference between "just enough" and "not enough" swap at the 256GiB/274GB mark.)

I'll run these tests again when I've got a 120GB drive in place - it should at least tell me how significant the difference is between -r 0 and -r 2, but I'm not expecting big changes in the results.

Meanwhile, I think we've definitely got some food for thought. :D

MC

(Edited to clean up some phrasing)

[dougdmy]

Originally posted by Merle Corey

Useful trivia: If you have a TiVo that's greenscreen looping due to bad swapfile, but is otherwise of sufficient size, you can run mkswap -v0 /dev/hdX8 from the (byteswapped) boot disk and the TiVo will recognize the corrected swap, allowing fsfix to complete.

In other words, anyone who is up the creek because of having used -s 128...? It's fixable, without reimaging, even if the greenscreen loop has already begun.


I'm still getting my feet wet with this and I don't want to do anything irreversible. So I'd just like to clarify this before I do it.

First my history---

I used mfstools-2.0 to expand a 80+80 DTiVo to a 120+120 DTiVo by using this command:

mfsbackup -Tao - /dev/hda /dev/hdb | mfsrestore -xzpi - /dev/hdc /dev/hdd

According to the README with mfstools-2.0, -s 64 is the default for the swap file size. I verified this by looking at the partition table, which has the following entry for /dev/hdc8:

8: Swap Linux swap 131072 @ 23894080 ( 64.0M)

So now on to my question: If I run

mkswap -v0 /dev/hdc8

and put the drives back in my DTiVo, there's a good chance that the reboot loop will be history and that fsfix will be able to complete. Am I correct?

Thanks in advance for your comments. I'm probably being a bit overly cautious, but I just want to be sure.

Doug

[mrtickle]

Originally posted by Merle Corey

John, glad to hear it worked for you. I can't really answer your question about greenscreens - I've had my TiVo's (2) for a year, and never had one that I hadn't caused. Others aren't as lucky. The swap shouldn't make any performance difference - having run top on my TiVo a few times, I can confirm that the swap doesn't get touched in normal use, and even most abnormal use. I'm not an authority on the subject, but it may only really get utilized when fsfix is running.


Here in the UK there is so much guide data that the swap is needed for indexing after the daily call. You may have seen some reports of people's tivos crashing after two days...

[Merle Corey]

Originally posted by dougdmy
So now on to my question: If I run

mkswap -v0 /dev/hdc8

and put the drives back in my DTiVo, there's a good chance that the reboot loop will be history and that fsfix will be able to complete. Am I correct?

Unfortunately, if I'm right, no.

You have a 64MB swap, which may not be enough for your dual 120's - if my findings are correct. More to the point, a 64MB swap (probably) won't be helped by rerunning mkswap, simply because MFST2 seems to know how to make a 64MB swap without breaking.

You would fall into the category of needing emergency swap space, but that would only be possible if you've hacked your DTiVo's prom to allow modifications. How to do that is beyond the scope of this particular thread - try a search on "DTiVo" and "bash" either here or on google, but it's definitely not for the faint of heart, and goes way past just adding drive space. Even if you do go that route, you'll still need to eventually redo your TiVo drives to permanently expand the swap to 128MB.

Ultimately, you may be better off restoring from backup and writing off what you've got recorded. :(

MC

(Edited for clarification.)

[Merle Corey]

I never said it explicitly last night, so I'll say it now for the sake of clarity: If I'm right, the ">140GB needs 128MB of swap" rule of thumb is still true under MFST2.

That would mean - in theory - that anyone with 64MB of swap and >140GB could be subject to greenscreen looping, and may want to redo their TiVo expansion to use 128MB of swap.

Keep in mind, though, that right now this is a theory. Just because it was true once doesn't mean it'll be true always, but it's certainly disturbing news. I hope to add further confirmation to this once my new drive arrives, but it ultimately needs to be tested by more people.

Oh, and thanks to mrtickle for the swap info - now we know two things it's used for. :D

MC

[dougdmy]

Originally posted by Merle Corey


You would fall into the category of needing emergency swap space.....



Thanks for your helpful information. Currently hda4 is my active partition and hda7 is unused. I did mkswap on hda7 and edited /etc/fstab. I put the drives back into the TiVo and I got the green screen and no reboot! It seems to be working.

Of course I realize that this is a temporary fix and if fsfix successfully completes, I will have to rebuild (soon) from my backup. But at least I can watch some of my more important shows. I feel like I'm sitting on a TiVo TiMe BoMb!

Doug

[dobbie1]

[QUOTE]Originally posted by Merle Corey
[B]

Straight from my kernel log:

Jan 1 00:00:15 (none) kernel: Activating swap partitions
Jan 1 00:00:15 (none) kernel: Adding Swap: 130684k swap-space (priority -1)

It's very early in the boot process (you can tell because the date/time isn't set yet), but it's pretty easy to miss.

I know I probably keep missing the entry, but for the life of me I cannot find an activating swap partitions in my Series 2 standalone. The only indication I can find of swap is in \var\log\messages.

However, yesterday I took the upgraded drive A out and tried to run the option 3, mkswap. It did not work. I rebooted, activated backdoors and in the log \var\log\messages, Swap Total = 0, and Swap Free = 0.

Today I took both of the upgraded drives out and replaced with the original 60G TiVo drive. When I rebooted and activated backdoors then checked \var\log\messages, Swap Total = 65532, Swap Free = 65520. I still could not find anywhere in any of the logs where the swap partition was activated.

For the last month or so, the TiVo had been experiencing random lockups, normally after the daily call when the data load had finished processing. I intend to run the original drive for a week to see if I begin experiencing lockups, if not then I will put a second drive in and bless with MFSTools 1.1.

Hopefully, this info will help.

Regards,
dobbie1

[Robert S]

Why don't you just dd the original swap partition over? (Method 2 on the top post.) Remember Series 2's are not byteswapped, so use the MFS Tools 2.0 CD and you should be able to see the partition table. I don't think this method has been verified on an S2, so let us know how you get on.

[kingmiwok]

doobie1:
Just for your sanity... I have a series 2 AT&Tivo and can find no mention of activating the swaps either. (This is after upgrading to a 120/120 config using the -s 128 option and tools v.2.0.) My swap reads 0 and 0 in the \var\log\messages as well. I'm thinking now that series 1 and 2 behave differently in this way

I'm locking up every 24 hrs or so and I'm patiently waiting for a sure fire fix. I have not had success dd-ing the old swap over, or running mkswap, but I may have had unrelated linux problems while trying to do the fix, so YMMV.

I hope to get a chance to try again in the next week, so I'll post up results.

[dobbie1]

Originally posted by kingmiwok
doobie1:
Just for your sanity... I have a series 2 AT&Tivo and can find no mention of activating the swaps either.

Thanks! At least I'm not in the boat alone.

It maybe the weekend before I have the opportunity to try and "dd" the backup and see if that works. Will post the results.

Regards,
dobbie1

[Agent86]

I'm confused. Someone throw me a lifejacket :)

I have an AT&T Series 2. It came with a 40GB drive. When I got it, I used MFSTools 1.1 to add a 120GB drive to it.

When MFSTools 2.0 arrived, I wanted to expand my A drive, but not lose recordings. I DDed the 40GB drive to a 120GB drive, and then expanded the DDed image on the 120GB drive.

I assume, based on what I have read, that this means my swap partition is indeed intact. So my question is pretty simple:

Do I have to do anything (i.e. bump swap to 128MB), or am I all set as is?

If I need to do anything, what do I need to do (comands to run, etc).

Someone save me! :)

- Agent 86

[Robert S]

You're OK for now. You carried your original swap over with dd, so that's fine (you will experience serious problems within hours if you have no swap at all). However, if you get a file system corruption problem - a green screen - you'll lose everything and have to restore from backup. The swap extending hacks we've been proposing require some fairly heavy Unix hacking and Series 2's are locked against that sort of thing.

If you do have to restore that TiVo's hard drives for whatever reason, it's probably best to use -s 128 to expand your swap. We don't seem to have figured out a reliable way to fix the swap problem that causes on S2's yet, so don't do that just yet.

[Merle Corey]

dobbie1, kingmiwok:

Just another confirmed sanity check - series 1 also displays the swap info you found in /var/log/messages - it's as valid a confirmation as the one in /var/log/kernel.

What steps, exactly, did you follow when using mkswap? What boot disk, byteswapping or no, and did you use any command line parameters?

Theoretically speaking (I haven't been able to talk a friend into letting me dissect his S2 for some reason, so I can't say for sure...), you should use a non-byteswapping boot mode, but the command you use should still be:

mkswap -v0 /dev/hdX8

Where X is the drive letter assigned by linux.

For the record, has anyone with a series 2 been able to successfully build a 128MB swap?

I've been hooked up with an image now; I'll post if I learn anything new from it.

MC

(edited to remove image request)

[Agent86]

Robert:

Thanks for the lifejacket :).

I'm pretty familiar with Linux, so command lines here and there don't scare me too much. I'm no low-level programmer or anything, but I can handle dd, cfdisk, and all their friends :).

Hopefully we'll find a solution soon.

- Agent 86

[dobbie1]

Originally posted by Merle Corey
dobbie1, kingmiwok:

What steps, exactly, did you follow when using mkswap? What boot disk, byteswapping or no, and did you use any command line parameters?
MC

I used the mkswap parameters listed under option 3, but I belive they were different that what is shown now. I used noswap, but after looking at my boot cd's today, I may have inadvertenly used the MFSTools2 disk. However, I did find time today to try and "dd" the swap file with the commands listed under option2, only to find that I had no space left on the upgraded drive and when I tried to reboot with that drive to delete some programs, I ended up with a very "green" screen. I assumed my file structure was now corrupt and went back to the original drive with a blessed drive b.

I will be happy to try and ftp you a copy of my original Series2 image, with the 30 software. I am not sure why there isn't one on the ftp site. I ftp'd Stan Simmons (?) the image over a month ago.

Regards,
dobbie1

[Robert S]

Sorry dobbie, the original mkswap parameters were wrong, the current ones have been tested on a Series 1.

Is there something odd about the S2 partition structure?

[Merle Corey]

Originally posted by Robert S
Is there something odd about the S2 partition structure?

I was wondering that myself; that's why I wanted the image - so I could see if there's anything unusual about it.

Right now the only noteworthy thing is that it's a 13 partition structure, similar to the S1 Sony... But I'm pretty sure we knew that already.

Other than that, the layout appears to be exactly the same - /var on 9, roots on 4 and 7, swap on 8. I'm going to take a closer look at the swap and confirm that it's still the old version, but that's also fairly likely to be the case.

MC

[Merle Corey]

And it's confirmed, series 2 still utilizes the old style swap - in other words, -v0 is required or mkswap will create something that the TiVo can't read.

(PM me if you really wanna know how to find out - it's easy - everything you need to know is in the man page for mkswap. :D)

I don't think there's much more to fiddle with at this point - what we need now is someone with a series 2 to try it out with the -v0 parameter.

MC

[Agent86]

So what has to be done? You know, the usual newbie stuff - commands to run, switch options, and option values (like swap size).

If it helps, I'm running dual 120gb drives in this unit.

Is it possible to do this without losing my programs, or do I have to start from scratch?

I have a feeling I might not like the answer to this one, based on prior posts in the thread :(.

- Agent 86

[Robert S]

Not at all, we just want you to do method #3 from the top post with the new options. It shouldn't affect the rest of your setup.

Seeing as you already have working swap, perhaps Merle can suggest a sequence of commands to reduce your swap slightly and then increase it back to 64Mb so we can be sure it's working.

[Agent86]

I think I can handle that :).

But don't I need to get MORE then 64 to make my TiVo happy when things go sour?

So ideally we'd drop it to some lower number, and then boost it to some higher number (i.e. 128, or whatever is needed for dual 120s).

Just feed me the proper command lines, and I'll make sure my fingers get them right :).

Also, does Kazymyr's boot cd (v2.6i) have this tool on it? Do I boot in byteswapped or non-byteswapped mode?

- Agent 86

[Robert S]

Well, you only have room for 64Mb of swap, which is not enough for your system to recover from a GSOD. You should have done a pipe transfer, which would have saved your recordings and left you with a 128Mb hole where your swap partition should be.

What we're trying to figure out is that last step - repairing swap partition on an S2. I don't think there's any reason to think it's impossible, but people have been having problems.

DTiVoes and S2's are locked down, so you can't do the hacking required to add additional swap for emergencies. I think DTiVoes can be unlocked by hacking the boot PROM, but S2's are currently unhackable, so the only way to make your TiVo safe is to repeat the upgrade and this time expand swap to 128Mb (there's no point in going beyond 128Mb, even though MFS Tools 2.0 allows it, as the Linux kernel TiVo uses can only use 128MB per partition or file).

128Mb should be enough for any TiVo to recover from a green screen, subject to confirmation when Merle's stocked up on big drives :)

[dobbie1]

I will give it a try with the 80G drive that had been expanded. Listed below are the steps, I think I should take.

1. Low level format the drive to get back to a clean drive.
2. Restore the backup with MFSTools2 using the default parameters as I used originally.
3. Expand drive with MFSTools2.
4. Return to TiVo and check swap file, there should not be any.
5. Remove drive from TiVo and with MFSTools 1 run mkswap with -v0 parameters.

Will this verify what is required. My system is a Series2 Standalone.

Regards,
dobbie1

[Agent86]

Well balls! :)

So, since its the off season its probably the best time to get this fix out of the way. It stinks that I'll lose all my season's passes, thumb ratings, and recordings, but if its the only option its the only option. I suppose that's why its called "hacking".

I'll just watch the few "new" shows I have (like Monk) over the next couple of days and then prep for surgery.

My backup image is a virgin image made with MFSTools 1.1. So what do I have to do to restore it to the 120GB drive, expand it, and make 128MB of swap properly?

My parents also have a Series 2 with the stock 40GB and a 120GB B drive - which means they fall victim to the same plague. Where is this piped procedure I can follow so that I can upgrade their A drive and extend the swap as well?

I really don't want to have to do all this ever again :).

- Agent 86

[stormsweeper]

You could just make a new backup image and keep all your settings, but not recordings.

[Merle Corey]

Agent86:

You don't have to lose anything (other than recordings). Make a new backup of your TiVo in its current state with:

mfsbackup -6so /mnt/bak/tivoagain.bak /dev/hdX

You'll preserve the season passes, etc, but flush the recordings.

When you go to do the 'mkswap', you'll need to do it in non-byteswapped mode, as that's how S2's work.

And thank you again for setting me up with the S2 image - it was a big help.

---

dobbie1:

Depending on what the "default parameters (you) used originally" are, yes, that's the process in a nutshell. If you used -s 128, then yes, you should have no swap. If you didn't specify, or used -s 64, you should have a working 64MB swap.

Technically, you don't need to low-level format the drive either, if you're looking to save a little time.

MC

[Agent86]

What command do I use to properly restore the backup?

Do I still use the -s 128, and then use mkswap -v0 to initialize it properly?

- Agent 86

[Merle Corey]

Today's quote of the day is: "A foolish consistency is the hobgoblin of small minds." --Ralph Waldo Emerson

So I got curious as to why, exactly, -s 128 failed. It didn't make much sense - I did a little digging, and it turns out that -s 128 fails to produce working swap because it initializes it improperly. It tries to do the right swap type (version 0), but munges the format.

This, however, triggered a suspicion and I just proved it true: We're (mostly) familiar with the "truncating to 130048kB" message that mkswap produces when making a version 0 swap of 128MB. The reason for that message is simple - that's the largest swap size supported by version 0 in a single partition/file.

130048kB/1024 = 127MB.

Oops.

The bug in MFST2 is that it fails to adjust for improper swap sizes fed to it on the command line. This is compounded by the README claiming that all swap sizes up to 512MB are valid.

So, I ran another test, this time using -s 127. It worked just fine.

In other words, the following produces a working restore with a maxed out swap partition:

mfsrestore -zxp -s 127 -i backupfile /dev/hdX

Well, Robert, we were looking for a simpler way to explain how to get working swap... I think I found it. :D

MC

(edit for typos)

[Robert S]

A86, you might want to wait until we have a verified method for doing swap on an S2. If you want to keep your SP's etc a new backup will preserve those (won't even have to make a guide call:)).

Restore with

mfstool restore -s 128 -xpi /mnt/dos/tivo.bak /dev/hdc

Which will restore the backup, increase swap size (broken), expand your partitions to fill the drive and optimise your partition layout for extra speed.

Whatever scheme we settle on for fixing swap, you should be able to do it from the MFS Tools 2.0 CD without rebooting because Series 2's aren't byteswapped.

dobbie, you don't particularly need to clear the disk first, I was slightly concerned earlier that you might inherit working swap from your previous install, but I don't think that applies here. Rather than a format, which is very slow, you could dd from /dev/zero over your drive. Set bsize and count to write 1Gb of data and you'll trash all the relevant stuff.

As I said, S2's aren't byteswapped, so MFS Tools 2.0 CD is the one to use, S1's must not use MFST2 CD because they are byteswapped. I would be quite happy for you to do mkswap before you test the restore, as long as you can verify that it worked.

[Agent86]

Sweet!

So all I have to do is boot up with MFSTools 2.0, and run

mfsbackup -6so /mnt/bak/tivoagain.bak /dev/hdX

and then follow that up with

restore -s 127 -zxpi /mnt/bak/tivoagain.bak /dev/hdX

and I'll be golden?

(The above is a combo of your restore and Merle's fix)

- Agent 86

[Merle Corey]

Originally posted by Robert S

Whatever scheme we settle on for fixing swap, you should be able to do it from the MFS Tools 2.0 CD without rebooting because Series 2's aren't byteswapped.


Actually, a reboot is required between the restore and running mkswap. I don't know why, but mkswap won't recognize the new partition until a reboot has taken place.

Of course, we may not need mkswap any more... :D

MC

[Merle Corey]

Originally posted by Agent86

and I'll be golden?


Yeah, that's the way it looks. I just re-confirmed again.

On the one hand, it's nice to finally have the kinks in MFST2 resolved. On the other, I feel pretty silly about that 127MB vs 128MB thing.

Ah well. Live and learn.

MC

[Robert S]

Merle, 90% of discoveries are things that people, having heard them, say 'I could've thought of that'. Trust me, the 127Mb thing is not something you should feel ashamed of, quite the reverse, be proud. In fact I feel proud just for giving you a bit of prodding to help you on your way.

So, to finish up, we need to get this into the right places, obviously Hinsdale would be a good place to start. We'll have to wait for Tiger to resurface before we can get his MFS Tools docs changed, but we should probably try and get a mod to change his top post on the MFS Tools 2.0 announce thread.

Also, we need to get the emergency swap strategy worked out so people who come here with 64Mb swap and a green screen loop can learn how to save their TiVoes. It would obviously be good if someone who's actually done this could write it up.

Anyway, I'll get on rewriting the top post on this thread.

[Merle Corey]

Oh, I'm not ashamed of it at all - I'm pretty puffed, truth be told. I just had a very large "D'oh!" moment when I realized what was going on, especially in light of that "128MB is such a nice, round number" post that I made elsewhere yesterday. :D

(And in all fairness, the limit is actually 133890048 bytes - about .3MB less than 128MB. 127MB just has the distinction of being the largest whole number within the limit. Theoretically speaking, Tiger should probably just have -s just be a toggle to activate the larger swap rather than a configurable value.)

I'll go ahead and confirm the swap requirements for very large configs (and I've made arrangements to get that second 120GB), but I really don't think there are any big surprises left for us to find.

I pm'd Hinsdale yesterday with regard to still needing *ahem* 128MB swap for larger configs. I haven't heard anything back yet, nor has there been an update to the How-To, but I assume he's just busy.

Beyond that, it looks like we're finally putting a wrap on this thing.

(Can I get a "woohoo!"?)

MC

[stormsweeper]

Originally posted by Merle Corey
The bug in MFST2 is that it fails to adjust for improper swap sizes fed to it on the command line. This is compounded by the README claiming that all swap sizes up to 512MB are valid.

So, I ran another test, this time using -s 127. It worked just fine.


Yeah, I had that suspicion, as well. I even asked if anyone had tried specifying a swap more than 64MB and less than 128 but didn't get a response then. ( http://www.tivocommunity.com/tivo-vb/showthread.php?s=&threadid=65945&perpage=20&highlight=127&pagenumber=2#post595529 ) At the time I had fixed my problem and didn't have a suitable spare drive for testing anymore.

As to why you're having to reboot between restoring and running mkswap - the drive's partition map needs to be reread. I'm not enough of a linux guru to know how to do that from the command line, unless forcing a reread in pdisk will do it.

[hinsdale]

Merle and Robert, I received your emails and thank you for the updates and good work on narrowing down the cause. I had mentioned to several people that emailed me over the last few weeks to try -s 127 and report their results as a possible solution to the swap file failure (apparently no one did this or reported their results). As stormsweeper mentioned this was one of the first theories floated as to the failure of the -s 128 creation. I hadnt had time to test this myself and wasnt following the thread closely enough to realize that this had not yet been tested.

I will be updating the How-To to incorporate the larger swap partition creation.

[Robert S]

Alternative suggestion for emergency swap for green screen recovery:

Use pdisk to transpose the labels on hdx8 and hdx4/7, run mkswap on hdx8.

Avoids problem with hacking fstab or rc.sysinit to activate swap - so may even work on DTiVoes and Series|2's. Would have to execute it just perfectly and back out correctly too.

[DCIFRTHS]

(Can I get a "woohoo!"?)

MC


woohoo :D

I haven't contributed anything technical to this discussion, but I have been following it VERY closely. This is really good news. You guys invested a lot, and I want to say - Thanks.

I really hope that everything is okay with Tiger. Hopefully he just has a new job, and hasn't had time to check in here....

[stormsweeper]

Originally posted by Robert S
Alternative suggestion for emergency swap for green screen recovery:

Use pdisk to transpose the labels on hdx8 and hdx4/7, run mkswap on hdx8.

Avoids problem with hacking fstab or rc.sysinit to activate swap - so may even work on DTiVoes and Series|2's. Would have to execute it just perfectly and back out correctly too.

Eek, and this would probably break a future upgrade too. I wouldn't recommend keeping this config for any longer than absolutely necessary. Be sure to change the partition map before you make a new backup.

[BobCamp1]

I have been watching this thread with interest, as I want to upgrade my Tivo but I'm waiting for the problems with MFSTools to get resolved. (Thanks to everyone for all that work, I wish I had that much spare time!)

Hinsdale has already updated the instructions (man that was fast!) and has put in the -s 127 parameter.

However, isn't there a problem with swap file size when adding a very large B drive? I thought Merle's research (and other people's information) indicated that if the combined capacity was greater than 140GB, then you needed a 127 MB swap file. However, Hinsdale's instructions for adding a larger B drive don't mention this.

So, if I have a single 30GB A drive and add a 120GB B drive, that equals 150 GB. But according to Hinsdale's instructions (step 10, config #1), I'm just typing in:

mfsadd -x /dev/hdc /dev/hdb (Boot CD and Floppy users command)

Doesn't this just use my original 64MB swap file? Isn't that too small to avoid the GSOD/reboot problem with my new 150 GB capacity?

I'm thinking of copying my old A drive to the new 120 GB drive (using the -s 127 parameter to create a larger swap file), then using my old A drive as the new B drive. Unless there is something in MFSTools that lets me expand my swap file on the old A drive while preserving recordings.

Besides, when you have a small amount of RAM you generally need a bigger swap file. All series 1 units with upgraded hard drives should probably use a 127 MB swap file since they only have 16MB of RAM. What's an extra 63 MB for a swap file when you're adding a 120 GB hard drive?

Am I just being paranoid, or should I use the 127 MB swap file for my upgrade?

[Robert S]

I would definitely recommend upgrading swap if at all possible. If you do a pipe transfer (will take several hours) to the new drive, you can save all your recordings, expand swap and stay in a single-drive configuration for now and add a large B drive later with mfsadd.

Alternatively if you just use mfsadd to add a B drive as long as your machine is a Series 1 standalone, you can add extra swap if your machine gets caught in a green screen loop. At 150Gb you're right on the limit for mfsfix, the true limit is believe to be between 150 and 155Gb, so you may or may not have problems.

[mrtickle]

Originally posted by hinsdale
Merle and Robert, I received your emails and thank you for the updates and good work on narrowing down the cause. I had mentioned to several people that emailed me over the last few weeks to try -s 127 and report their results as a possible solution to the swap file failure (apparently no one did this or reported their results). As stormsweeper mentioned this was one of the first theories floated as to the failure of the -s 128 creation. I hadnt had time to test this myself and wasnt following the thread closely enough to realize that this had not yet been tested.

I will be updating the How-To to incorporate the larger swap partition creation.

I see it's now changed to 127 in the guide, thanks. I can't find any mention though of the fact that you have to make the big drive the A drive, because otherwise the bigger swap won't fit. Or does MFStools2 create the swap on the B drive?

[Robert S]

No, MFS Tools can't put a swap partition on the B drive (although this is possible in principle, it's probably not desirable and you would have to modify TiVo system files after a software update).

It's quite tricky to explain, but it's important to explain to be people that they need to choose upgrade profiles that allow MFS Tools 2 to increase swap for them. So things like dd + mfsadd and using mfsadd to add a B drive to an unmodified A drive should be deprecated in favour of pipe transfers, which have the same effect in almost all cases with only a little more pain.

[Cpen]

Robert:

So - When I upgraded my Philips 312 (2-drive) TiVo to 120 MB, I did not use the -s command (at all). As a result, my upgrade used the default 64MB swap file (confirmed by looking at var logs). Should I not be concerned because my total hard drive size is under 140MB?

or

Do I need to re-run my upgrade and use -s 127

I think I'm good to go, as I did not use -s128 (I have a working 64MB swap) and my hard drive space is under 140 MB (which I beleive means a 64MB swap will work just fine). If this is true, then I was just lucky - go figure.

P

[Robert S]

That's our current belief. Merle is planning to do some more experiments on big drives, so we may get really solid answers to these questions.

If we can get my partition relabeling idea proved out (and I think we're still waiting on an S2 owner successfully using mkswap), then our advice to people with working TiVoes over the limit would be to wait for the green screen to happen and then use emergency swap, rather than trash a perfectly good system.

Assuming no-one pops up in the next few days pointing out a major problem we've missed, I think we can say that MFS Tools 2.0 is a safe and effective upgrade tools when used within our guidelines.

[Cpen]

Robert:

Thanks. I was hoping that this thread would drive resolution/closure to the outstanding issues w/regard to use of MFSTools 2.0.

Hopefully this will close out the chapter of uncertainty for MFSTools 2.0 and allow people to upgrade w/greater confidence!

Thank you for (doggedly) pursuing the solution(s). It's very much appreciated.

P

[Merle Corey]

A little more swap confirmation:

Got both my new drive and my borrowed drive today - both 120GB.

Random details:

I ran short on time, so didn't fine tune the results beyond general range.

A 120GB requires somewhere between 45MB and 50MB of swap to successfully run fsfix.

A 240GB config requires between 100MB and 110MB to successfully run fsfix.

Expected results:

.5 * CapacityGiB = 6MB RAM + N swap

120GB capacity --> 50MB swap
240GB capacity --> 105MB swap

This makes the formula good enough for government work. :D More to the point, it stands up for at least moderate accuracy.

Most importantly, of course, is the fact that we have (more) proof that very large configs require more swap than smaller configs.

Cpen - this also means that you should be fine with your config.

MC

[SAFW]

By the proposed formula a dual 137 GB system will require ~121 MB swap, or 6 MB shy of the current swap limitation. If that 6 MB of hard RAM weren't there, it would put the swap size at 127 MB. Are the hard drive limitation issue and the swap limitation issue connected in some way?

[Robert S]

I suppose the connection is powers of 2 - both the 128GiB disk limit and the 128MiB swap limit arise from the number of bits allocated to an address lookup. The 1/2 MiB per 1 GiB is balanced by the 2 drives.

So the fact it lines up so precisely isn't coincidental, but one more or fewer bit on either one would mess it up or make it well clear.

[Merle Corey]

I'll tackle this with more depth.

The swap limitation is because of the age of the TiVo kernel - I'm not sure when that limitation was broken, but it's been obsolete for at least four years or so. It probably also happens to have been current when the TiVo was undergoing initial development.

The drive limitation (128GiB) conforms to the ATA spec. Up until late last year, 128GiB was the defined maximum addressable space supported by an EIDE drive, using 28 bits of address space. The new ATA/ATAPI-6 spec provides for 48 bits of address space - a nominal maximum of 144 petabytes, or around 150,000 hours of basic recording time. That's quite a ways off, though - right now we know these as the 160GB, 180GB, and 200GB drives, of which TiVo can currently only utilize 137GB/128GiB.

But back to your original question: No, they're not really connected in any meaningful way - they're two independent standards, the only common factor being fsfix.

It may be that TiVo saw this coming in one form or another, and optimized fsfix so that it would run within the limitations of the platform, even though a "stock" TiVo doesn't even come close to pushing those limitations. Honestly though, I tend to suspect coincidence on this one, a serendipitous arrangement of two different base 2 numbers and the memory requirements of a filesystem repair tool.

You know, what Robert said. :D

The other thing worth noting is that's the formula for a 16MB SA TiVo - the 32MB TiVo's are going to be in better shape.

If somebody with either upgraded RAM in an S1 SA or a hacked PROM in their DTiVo would like to determine the usage in a 32MB TiVo, it's pretty easy. Find the breaking point - the minimum amount of swap needed for fsfix to run given a specific capacity in GiB. Using the formula:

R + S = m*C

Where R is the system RAM utilized in MiB, S is the minimum working swap space in MiB, m is the multiplier in (MiB per GiB), and C is the capacity in GiB. You'll know values for m (.5), C, and S, so solve for R. Basic algebra.

MC

(Correcting typos and reminding myself that getting my hands on an S2 won't help generate the 32MB equation.)

[Robert S]

Recovering a TiVo that's GSOD'd and sticks

Originally posted by brentf in another thread
How can I determine which is the inactive partition? Also does pdisk need any parameters or be executed from a particular location? Once I doe this should I expect a green screen and wait for mfsfix to run or should it boot normally?

See the top post in this thread for tips on how to detect the active partition. If you haven't had a software update since the upgrade, one of them will mount and the other won't.

You need to tell pdisk the device node for the drive you want to work on.

Once you're done, yes, restart the TiVo and let the green screen complete. The TiVo should reboot and come up as normal.

Then go back and reverse the renaming of the partitions, if you forget this you'll be in real trouble at the next software update!

pdisk's internal help is probably good enough for you to figure out what to do. If you could take notes of what you do and post them it would really help get this down.

You'll need a byteswapped boot disk to see the partitions on a Series 1 TiVo. Therefore you can't use the MFS Tools 2.0 bootdisk. If you use a TiVoMad boot disk, note that pdisk is actually /mad/pdiska on his disks.

[brentf]

Thanks Robert. I'll read this thread to get up to speed and report the results. To orient this thread I'm getting GSOD reboots after adding a 120GB Maxtor to an already expanded 80GB "A" drive. So the plan is as follows:

mkswap -v0 (on inactive root partition)
use pdisk to swap partition ID's on the inactive root and swap paritions
Boot Tivo and allow to repair
pdisk the partitions back again to ready for next software update.

Let me know if I've missed something.

[BobCamp1]

Using Merle's data points of (120GB, 50MB) and (240GB,110 MB), and assuming a linear relationship, I get the equations:

swap needed in series 1 Tivo (in MB) = (0.5*total capacity (in GB)) - 10

and

total capacity (GB) = (S1 swap size (MB) + 10) * 2

So a 64k swap file allows you to have up to 148 GB in capacity.

A 127 MB swap file would allow a capacity of up to 274 GB, or two 137 GB drives.

These numbers are a little conservative, which means it should be safe to put two maximum capacity hard drives in your Tivo as long as you use a 127 MB swap file. It also means the critical "capacity" is probably around 150 GB. (My planned upgrade puts me right at 150 GB, so I'll increase my swap file size).

Note that a series 1 Tivo seems to be using around 10 MB of RAM to run mfsfix, instead of 6 MB as was posted eariler.

Series 2 is currently a guess. Worst case says no additional RAM is used, so the series 1 equation holds (max. capacity of 150GB w/64MB swap). Best case is all additional 16MB of RAM are used for mfsfix, which makes the max. capacity (64+10+16) *2 = 180 GB w/64 MB swap file. That assumes that almost everything else for series 2 Tivo's is the same as in series 1, which is a pretty weak assumption. But if you want to play it safe, I'd treat the series 2 just like a series 1 and increase the swap file for capacities over 150 GB.

Hopefully, the equations can be tweaked as more data comes in.

[Robert S]

Actually given the 'common code base' of 3.0, I think it's entirely probably that S2's give an extra 16Mb of RAM to mfsfix and even if extra network stuff is added later, the chances are that most of that won't load until after mfsfix is called.

I would recommend increasing swap whatever else you're doing - it only costs you 63Mb of disk space. Even if you're under the limit, hard drives will continue to fall in price and if you want to go up to a bigger drive having extra swap makes it safer.

What we really need, of course is confirmation that the partition swapping technique works on DTiVo and S2.

[Merle Corey]

Originally posted by BobCamp1
Using Merle's data points of (120GB, 50MB) and (240GB,110 MB), and assuming a linear relationship, I get the equations:

Two important factors here:

1) The 120GB/50MiB and 240GB/110MiB were NOT minimum known good values. Rather, they were semi-arbitrary values chosen because they fell on the "good" side of the equation. I didn't have enough time to generate specific minimum good swap values through guess-and-check, and I'm not going to for a while - my TiVo died a fiery death yesterday thanks to the electrical storms in the area.

2) You're flip-flopping between GiB and GB - the former is base-2 and is how the computer reports drive space, while the latter is base-10 and is how hard drive companies refer to disk space.

50MiB + 6MiB = 56MiB.
56MiB * 2 GiB/MiB = 112GiB
112GiB is roughly equal to 120GB

110MiB + 6MiB = 116MiB
116MiB * 2 GiB/MiB = 232MiB
232GiB is roughly equal to 249GB.

So the 50MiB of swap may be the exactly-right number for a 120GB drive, but 110MiB is more than is strictly needed for 240GB.

(For reference, the known "exactly right" values we've currently got are 13MiB for 40GB and 64MiB for 140GiB. Note the different GB and GiB suffixes for those.)

Finally, the reason we cared about this at all was to establish two things - that, contrary to the MFST2 docs, 64MiB swap wasn't sufficient for all capacities, and that 127MiB of swap was sufficient for all currently possible capacities.

MC

[gigageek]

Originally posted by Robert S
What we really need, of course is confirmation that the partition swapping technique works on DTiVo and S2.

Ask and ye shall receive. I just purchased 3 Seagate ATA IV 80GB drives to upgrade my Sony SVR-2000 (SA) and my Sony SAT-T60 (DTiVo).

The SVR-2000 currently has a Maxtor 40GB A-drive and Maxtor 80GB B-drive. Two of the Seagate drives will ultimately end up here. The T60 is currently unmodified, and the remaining Seagate drive will replace the factory A-drive (which will be placed in safe storage - you can never have too many backups).

Before I get to the permanent configurations, however, I will upgrade the DTiVo with a pair of 80GB drives using -s 127. I plan to start later this evening, and post my results when I'm done. Hopefully, this will provide some small measure of payback to the forum for the outstanding support I've received here over the years.

So someone please refresh my memory about what I'm looking for after the upgrade?

Greg

P.S. This is a little bit off-topic, but in case you were wondering why I'm replacing two perfectly functional Maxtor drives in my SVR-2000 with only 2x80GB disks (as opposed to 2x120GB, which would obviously be for the increased capacity)... Mostly, it's because the Maxtors are too noisy for my taste. It's not head-seeking noise, it's the whine of the drive motors, and it's always been just a little bit annoying. It's always been the same volume (for well over a year), so I don't think either drive is failing, but both of my TiVos are in the great room and I'd like them to be quiter. (Silent would be my first choice, but I'll settle for "quiet enough to not be heard in the breakfast nook 30 feet away".)

[Robert S]

That's not what we want to test. We are absolutely confident that you will get a safe upgrade with working swap.

What we want to test is putting a swap partition on your inactive root partition and transpose the partition labels for the main swap partition and the inactive root parition. This breaks lots of important things on the TiVo, but gives it 128Mb of swap, which would allow a TiVo that was stuck on a green screen for lack of swap to recover.

We are pretty sure this will work on stand alone series 1's, but DTiVoes and Series|2 are locked down to prevent modifications to the root filing system. We therefore don't know if the partition table is also checked.

If you would like to test this on the SVR, that would be very helpful. All we really need to know is that the DTiVo boots and recognises the swap partition in this configuration, we're pretty sure that the green screen will complete if that happens.

This is a fairly heavy Unix hack, which we haven't got written down yet, but there are a couple of outlines further up this thread.

[gigageek]

Originally posted by Robert S
This is a fairly heavy Unix hack, which we haven't got written down yet, but there are a couple of outlines further up this thread.

I may be your guy, but I might need a little bit of help. I've got 5 years experience as a sysadmin on SGI boxes, so I'm pretty familiar with Unix (the SGI flavor, anyway). I've also got a functioning 2-drive SVR-2000 (hacked with Dylan's Boot Disk & BlessTiVo), an unmodified T-60, three 80GB disk drives, and a willingness to hack. Heaven forbid, I've even got the time to do it this weekend. The best part is that, even if the swap hack fails on my new configuration, I've still got the fully-functional drives to start over with. :D

If you give me instructions, I'll be happy to test-drive your theories. Just keep in mind my inexperience with the Linux flavor of Unix when giving me the instructions. And do I need to preserve the recordings, or can I do the speedy upgrade?

In the meantime, I'm re-reading the earlier posts in this thread that mention the heavy-duty swap hack.

[Robert S]

It shouldn't be destructive, but you might prefer to play it safe.

We don't care about the recordings on the drive, or the total drive size, we already have that sorted. You might want to do a quick restore to one of your as yet unused drives to sacrifice for this test.

You should find Linux fairly comfortable if you know Irix. You'll probably need a TiVoMad or Kazymyr boot CD.

What you need to do is

Identify your inactive root partition (see the top post on this thread for detailed methods, but one will mount and the other won't).

Turn your inactive root partition into a swap partition by using mkswap. You'll need to use the -v0 option to create an 'old style' swap partition. (This is documented quite well in the top post, although the target will be hda4 or 7 instead of 8)

It might be a good idea to hose the original swap partition with 'dd -if=/dev/zero of=/dev/hdx8' to be sure the substitution is real. (If you restore with -s 128 you can be sure you've got no 'real' swap without doing this step.)

Use pdisk (TiVoMad calls it pdiska and puts it in /mad) to transpose the partition numbers on the normal swap partitions and the inactive root partition. pdisk's help should be enough to figure this out.

Boot the TiVo and report what happens, in particular, check the kernel logs and verify the swap activation.

Use pdisk to reverse the transposition and release the inactive root partition for its intended purpose. If you intend to use this disk you'll need to recreate the hosed swap partition, otherwise forget about it and do whatever the drive was intended for.

Obviously the more detail you can report on what you did and what happened, the better.

[brentf]

Robert, I'm about to give this a shot, but based on what I can gather, this is just a "get it back up" procedure. If this is true, is there any approach to enlarge the permanent swap file? Just a hairbrain, but what would happen if I restored my 80 Gig "A" drive to my 120 Gig "B", then did an expansion building the 128Meg swap space? It would obviously truncate any data that was on the old "B" but would it recover?

[Robert S]

It depends on whether you want to keep your recordings. The -s option only works on restore rather than mfsadd, so if you do a standard backup/restore to your current drive set with -s 127, you'll increase the swap partition size, but lose all your recordings.

The only way to do this and keep your recordings would be to do a pipe transfer to two new 120Gb drives (would take several days!).

Your TiVo should be fine if it recovers from the green screen. If it green screens again it probably means one of your hard drives is going, so you'll want to make arrangements to move to new drives anyway.

[kyoto]

Hi!

I just completed the upgrade using MFS Tools 2.0. It took a couple of hours as I chose to copy over my recordings. The only oddity I noticed was that the TiVo Suggestions were cleared. I intiated a daily call and some appeared.

I really appreciate the work you did to figure out the swap issue. I had read the article on upgrading the TiVo in the Sept 2002 Home Theater magazine and ordered 2 120 gb drives and a drive bracket from 9th Tee. As they arrived, I saw the post on the swap problems and started to sweat. I was very relieved and thankful when I saw you work out the resolution to this problem.

Now the arguments with my wife over her year old recordings can end :D !

Thanks!!!

kyoto
ready for the new TV and college football seasons

[gigageek]

Originally posted by Robert S
Use pdisk (TiVoMad calls it pdiska and puts it in /mad) to transpose the partition numbers on the normal swap partitions and the inactive root partition.

>>> pdisk's help should be enough to figure this out. <<<



I'm trying to get all my ducks in a row before I get started, but I'm stumbling over the pdisk step. The closest relative to pdisk tha I'm ware of in IRIX is "fx", but it operates very differently from pdisk. I'm still guessing that a mis-step here will render parts (or all) of the drive (and my efforts to help) unusable.

I ran pdisk -i and issued "?" for help, but this "help" was pretty thin, even by Unix standards.:rolleyes: I don't have a "real" version of Linux on which to read the pdisk man page, so I'd be grateful for some extra help. Can you post (or PM) either the whole pdisk man page, or at least the relevant sections of it? MTIA.

[Robert S]

Don't read the man page, do pdisk /dev/hdc (or whatever) and look at the help available there. I seem to recall it was plenty. pdisk won't trash your disk unless you save the changes - you can mess about as much as you like.

[gigageek]

Originally posted by Robert S
Don't read the man page, do pdisk /dev/hdc (or whatever) and look at the help available there. I seem to recall it was plenty. pdisk won't trash your disk unless you save the changes - you can mess about as much as you like.

I started with the DirecTiVo (previously unmodified single-disk SAT-T60). I physically removed my XP system drive from the system (all those years of paranoid sysadmin training tought me that you can't destroy what isn't installed) and connected my TiVo A-drive to primary master, and a spare 850MB FAT32 drive to primary slave. I mounted the FAT32 drive as /mnt/dos, made my backup (mfsbackup -6so /mnt/dos/tivo.bak /dev/hda), rebooted to XP, burned the backup to CD, rebooted to linux (mfstools2 boot floppy), mounted the CD as /mnt/dos, and restored the image to an 80GB Seagate drive (mfsrestore -s 128 -zpi /mnt/dos/tivo.bak /dev/hda).

I ran pdisk /dev/hda and was told:
pdisk: No valid block 1 on '/dev/hda'

Before I started, I had tried to format the Seagate drive as FAT32 under XP - until I relized that XP only allows you to format NTFS. (I was going to dump the backup to the Seagate drive instead of cannibalizing the other FAT32 drive from another system.) Since the Seagate drive was connected to the system while XP was running, even though that was before I made/restored the TiVo images, does this mean that I've got the dreaded XP contamination mentioned at the end of Hinsdale's how-to? If so, how can I get rid of it?

[Robert S]

As you have a Series 1 machine, you will need to boot in a byteswapping mode. Byteswapping is broken on the MFS Tools 2.0 CD (someone want to post the boot parameters to fix it?), so you'll have to use a TiVoMad or Kazymyr CD.

Primary Master is left unswapped so you can access a FAT drive for backup. (This is mostly irrelevant as MFS Tools 2.0 has internal support for byteswapping, but you need to understand it you want to access the partitions yourself.)

XP probably has rendered that Seagate unusable, you'll probably have to restore the backup to that drive again. It is possible to reverse this change, but there's nothing special on that disk yet anyway.

You can format FAT partitions from Tom's Root and Boot Disk (http://www.toms.net/rb/) with fdisk (you want a type 0c partition) and mkdosfs - it's more flexible and faster than using the equivalent DOS tools.

[Gomer Pyle]

Robert S.:

I feel like an idiot, but the fact is, I am Linux illiterate, and need some hand holding...

I upgraded from a 40GB+80GB (created with Dylan's boot disk + BlessTiVo) to a 120GB+80GB (using the dd command, per Hinsdales guide, preserving recordings, on a Series 1 Sony, with the boot floppy off of MFS2 CD [which will NOT boot on my machine...]). I knew there was a swap file issue circling around, but went ahead and did the upgrade anyway (sounds like not such a good idea, now).

From my understanding of reading this thread the past few hours, my current set-up will run fine until something goes wrong that would require the TiVo to try to repair itself. At that point, it will be dead, since it has the original 64 MB swap file, which is not large enough to support a rebuild of a 200 GB system. I could in theory, at that point, make an emergency swap file that could get me out of the immediate jam, but not solve the problem permanently - is that close to a correct description?

Your solution in the first post, which sounds like the solution for a Series 1 SA unit, is to:

1. Use shell access to TiVo to run mkswap directly.

Use the shell to type

mkswap /dev/hda8
swapon -a

Is this understanding correct? Where does this command sequence specify swap file size? Is a -127 modifier not required for this command? Is this a permanent fix that will survive future upgrades (as currently understood), or is this an emergency fix?

Thanks for any reply.

[gigageek]

Originally posted by Robert S
As you have a Series 1 machine, you will need to boot in a byteswapping mode. Byteswapping is broken on the MFS Tools 2.0 CD (someone want to post the boot parameters to fix it?), so you'll have to use a TiVoMad or Kazymyr CD.


I get more confused the farther I get into this... That Seagate drive that pdisk didn't like after the upgrade booted just fine in my DTiVo. All my "stuff" was still there (except the actual recordings, obviously).

I re-installed my original DTiVo drive in my PC (primary slave), with the FAT32 drive as primary master & CD-ROM as secondary master. I booted the TiVomad floppy, interrupted the upgrade script, and executed:
#/mad/edit_bootparms /dev/hdb -r

All I got was the usage screen telling me what the parameters are (but no useful information).

I booted the MFSTools2.0 CD and (later) Kazymyr's Boot CD. With both boot CDs, I (alternately) specified the following boot parameters:
vmlinuz hdb=bswap
vmnodma hbdswap
<accept defaults>

(I picked up the kernel boot parameters from jhburke on the 2nd and 4th pages of this thread. Is there a functional difference between vmlinuz and vmnodma, other than display colors?)

I could get pdisk to give me a partition map, but only when accepting the default kernel boot parameters. If I specified hdb=bswap during boot, pdisk protested about "No valid block 1 on '/dev/hdb/" on my original (unhacked) DTiVo A-drive.

Under no circumstances could I get edit_bootparms to tell me anything other than the usage information (in either mad31 or mad32 on tivomad disk).

[Robert S]

Gomer, those instructions are for creating a normal swap partition on a machine which MFS Tools 2.0 left with no swap at all (we now know how to avoid this, but until a revised version comes out it remains a trap for the unwary and I want people to know they can fix it without running MFS Tools again).

The 'emergency swap' routine is still being worked out, but it will be /much/ more involved.

You were in the 'nightmare scenario' anyway - there's no way you could have increased your swap without losing your recordings or buying another big drive.

[Robert S]

GG, you're just running into byteswapping problems.

The Seagate drive is byteswapped, the way TiVo likes it, so your PC won't be able to see it unless you boot into a byteswapped environment. That means all the tools - edit_bootparm, pdisk, mount - require you to get byteswapping right. If you get it right, Linux will print the TiVo partition table as it boots. (MFS Tools 2.0 has internal support for byteswapping, so you don't normally need to worry).

Byteswapping on MFS Tools 2.0 is broken unless you use that special boot line. The reason is that it uses the vmlinuz kernel instead of vmnodma when it boots in byteswapping mode. The difference is that vmnodma has no DMA. DMA is incompatible with the byteswapping patch - if you activate them both you get DMA, but not byteswapping.

Kazymyr by default boots with swapping on hdb,c and d.

If you can't get edit_bootparms to work, try mounting the two root partitions. The one that fails to mount is your inactive root (given your current issues, make sure that the other one does mount).

[BRENT]

What is the difference between the swapon entries? Do they each provide the same result?

mkswap /dev/hda8
swapon /dev/hda8

mkswap /dev/hda8
swapon -a /dev/hda8

mkswap /dev/hda8
swapon -a

[Robert S]

I think swapon -a activates all the swap partitions listed in /etc/fstab whereas swapon <filename> activates the swap partition or swap file pointed to by filename. swapon -a <filename> probably produces an error.

[BRENT]

I used the first entry to correct my swap. The first and second entry were posted as the fix in another thread. The last one listed is in this thread. Just to clarify, which is the correct or best entry?

[Robert S]

The first and third are essentially equivalent as /dev/hda8 is in fstab on a TiVo, the middle one is probably a mistake.

It doesn't really matter as long as mkswap succeeds - you'll get your swap on the next reboot anyway.

[gigageek]

Robert S,

I appreciate all your help, but it's still a little bit unclear to me exactly which tools I'm using/testing, and in which order. I would be really grateful for some EXPLICIT instructions (at least as far as the general steps, if not the actual syntax). What I think you want me to do is:

1. Boot MFSTools 2.0 with 'vmlnodma hdb=bswap' (where hdb is my DTiVo A-drive).
2. Use MFStools2 to backup to a new drive:
#mfsbackup -6so /mnt/dos/tivo.bak /dev/hdb
3. Stop here to burn a CD-R of the backup. I know this step isn't really necessary, but I'm pretty anal about having multiple backups (you can never have too many).
4. Boot MFSTools 2.0 with 'vmlnodma hdb=bswap' (where hdb is now the new Seagate 80GB drive).
5. Restore from the backup with -s128:
#mfsrestore -s 128 -zpi /mnt/dos/tivo.bak /dev/hdb
6. Boot the restored Seagate drive in my DTiVo, verify that I get the swap errors in /var/log/kernel:
kernel: Activating swap partitions
kernel: Unable to find swap-space signature
kernet: swapon: /dev/hda8: Invalid argument

All of the above steps will get me the "swap-hosed" upgrade that everyone who used mfsrestore -s 128 has, correct?

From this point, I'm not terribly clear what EXACTLY you want me to test. I've burned up most of the weekend groping around blindly, and the wife is getting irritated about not having TiVo while I'm working this out. Pretty soon I'm going to have to abandon ship & give her back a functional TiVO.

What I think you're asking me to do is:

7. Put the upgraded Seagate 80GB drive back in the PC (hdb), boot Kazymyr's Boot CD (with defaul boot parameters).
8. Determine the active root partitiion:
#/mad32/edit_bootparms hdb -r
If this doesn't work (which it hasn't for me thus far), try mounting /dev/hdb4; if hdb4 mounts, this is my INACTIVE root partition (but still verify /hdb7 DOES mount).
(Assume for the rest of this discussion that 'root=/dev/hda4', so the INACTIVE root is /dev/hdb7.)
9. Create a swap file on INACTIVE partition from previous step:
#mkswap -v0 /dev/hdb7
10. Swap the partition label of the original TiVo swap partition (/dev/hdb8) and the inactive root partition (hdb7) using pdisk.
11. Put the swapped-partition drive back in the TiVo and see if it boots. If it does, check /var/logs/kernel for relevant messages.

Is this was you've got in mind? If not, please give EXPLICIT instructions.

[Gomer Pyle]

Robert S.

Thanks for the reply. It certainly was not clear to me that the initial post in this thread regarded units only with no swap at all. Sounds like I will be much better off in the long run if I just scrap my recordings and restore with a defined 127 MB swap. Guess I'll do that now. There really isn't much on there other than suggestions right now, so now is a good time to kill it.

You assistance is greatly appreciated.

[Robert S]

Step 1 is unnecessary, MFS Tools 2.0 has internal support for byte-swapping, so you can just boot normally if all you want to do is a backup/restore/expand.

I'm not quite sure where you are at the moment, if you already have a TiVo backup, restore that to the new drive. If you don't have a suitable backup, then making one is a good idea.

'-s 128' will /definitely/ get you no swap, only check it if your really paranoid. You can go straight on to the rescue procedure.

Now is the time to boot MFS Tools 2.0 with the magic incantation. If you have a Kazymyr or TiVoMad CD, that will work just as well (they're all basically variations on Kazymyr's original). You probably will have to reboot, even if you booted in a byte-swapped mode for the restore.

Step 8, if a partition mounts it's probably the active partition. I'll assume this is a typo. Make sure you get one mounting and one not mounting. If neither mount your boot environment is wrong, if both mount you'll have to look at the file dates with 'ls -l', the newer files will be on the active partition.

Other than those incidental point, that's exactly what we want. If you can write up a similar list from what you actually do, I can write that up into a How-To. I have a list of how to do it right - the beta testers' job is to work out what might go wrong.

[gigageek]

Robert S, thanks for the guidance. I think we have a solution here, although I'm not sure what else it might break. I have performed the following steps on my previously unhacked Sony SAT-T60 DTiVo:

1. Install FAT32 drive on hda, DTiVo A-drive on hdb. Verify the BIOS recognizes both drives before booting MFSTools 2.0 CD with default boot parameters. (Note: I think this is where most of my early problems were. Well, this and CDD: Clue Deficit Disorder.)

2. Create backup of the DTiVo A-drive:
# mkdir /mnt/dos
# mount /dev/hda1 /mnt/dos
# mfsbackup -6so /mnt/dos/dtivo.bak /dev/hdb

3. Stop here to burn a CD-R of the backup. I know this step isn't really necessary, but I'm pretty anal-retentive about having multiple backups (you can never have too many). Plus, by restoring from the CD-R backup, this verifies that the CD copy of that backup will indeed create a usable drive.

4. Remove XP drive from PC. Install FAT32 drive on hda, new Seagate 80GB drive on hdb, CD-ROM on hdc. Verify the BIOS recognizes all drives before booting MFSTools 2.0 CD with default boot parameters.

5. Restore from the CD-R backup:
# mkdir /mnt/dos
# mount /dev/hdc /mnt/dos
# mfsrestore –s 128 –zpi /mnt/dos/dtivot60.bak /dev/hdb

6. Boot the restored Seagate drive in my DTiVo, verify that I get the swap errors in /var/log/kernel:
kernel: Activating swap partitions
kernel: Unable to find swap-space signature
kernel: swapon: /dev/hda8: Invalid argument

7. Reinstall new Seagate 80GB drive on hdb (FAT32 drive on hda, CD-ROM on hdc). Verify the BIOS recognizes all drives before booting MFSTools 2.0 CD with "vmlnodma hdb=bswap".

8. Determine active root partition:
# cd
# mkdir /mnt/tivo/4
# mount /dev/hdb4 /mnt/tivo/4
# ls –l /mnt/tivo/4
<most-recent date was June 14, slightly more than 2 months ago>

Per Robert S, this suggests that hdb4 is the active partition. To verify, I did:
# umount /dev/hdb4
# mkdir /mnt/tivo/7
# mount /dev/hdb7 /mnt/tivo/7
mount protested with "mount: you must specify the filesystem type."
From this, I concluded that root=/dev/hdb4, and hdb7 is the INACTIVE partition.

I suppose that if I were slightly more clever (or more familiar with TiVo disk partitions), I could have drawn the same conclusions from the partition map:

Partition map (with 512 byte blocks) on '/dev/hdb'
#: type name length base ( size )
4: Ext2 Root 1 262144 @ 75153472 (128.0M)
7: Ext2 Root 2 262144 @ 75423808 (128.0M)
8: Swap Linux swap 262144 @ 75685952 (128.0M)

9. Create a swap file on the INACTIVE root partition (hdb7):
# mkswap –v0 /dev/hdb7
output from the above command was:
mkswap: warning: truncating swap area to 130752kB
Setting up swapspace version 0, size = 133885952 bytes

10. Reorder the original TiVo swap partition (hdb8) and the inactive root partition with newly-created swap (hdb7):
# pdisk /dev/hdb
Command (? For help): r <for “reorder partition entry in map”>
Partition number: 8
New number: 7
Command (? For help): w <for “write the partition table”>
Writing the map destroys what was there before. Is that okay? [n/y]: y

This reordered the partition table as shown:

Partition map (with 512 byte blocks) on '/dev/hdb'
#: type name length base ( size )
7: Swap Linux swap 262144 @ 75685952 (128.0M)
8: Ext2 Root 2 262144 @ 75423808 (128.0M)

11. Install pdisk-modified A-drive back into DTiVo, power up, enable back doors, search /var/log/kernel:
kernel: Activating swap partitions
kernel: Adding Swap: 130744k swap-space (priority -1)

12. Does this warrant a WOOHOO!?

[gigageek]

Since I don't know whether TiVo cares about the partition names and types (it seems unlikely to me, but what do I know?), I used pdisk to delete & recreate partition names to match what they looked like before I reordered them in step 10 (previous post). Continuing where I left off:

13. Use pdisk to delete/recreate partitions 7 and 8.
# pdisk /dev/hdb
Command (? For help): d (for "delete partition")
Partition number: 7
Command (? For help): C (for "create with type specified")
Partition number: 7
First block: 75685952 Use the number shown for partition 8 for YOUR DRIVE in Step 8 above.
Length in blocks: 262144
Name of partition: "Root 2" (quotes are required)
Type of partition: Ext2
Command (? For help): d (for "delete partition")
Partition number: 8
Command (? For help): C (for "create with type specified")
Partition number: 8
First block: 75423808 Use the number shown for partition 7 for YOUR DRIVE in Step 8 above.
Length in blocks: 262144
Name of partition: "Linux swap" (quotes are required)
Type of partition: Swap
Command (? For help): w (for "write the partition table")
Writing the map destroys what was there before. Is that okay? [n/y]: y

This reordered the partition table as shown:

Partition map (with 512 byte blocks) on '/dev/hdb'
#: type name length base ( size )
7: Ext2 Root 2 262144 @ 75685952 (128.0M)
8: Swap Linux swap 262144 @ 75423808 (128.0M)

I didn't even have to recreate the swap space on partition 8. (Actually, I tried, but mkswap correctly pointed out that there was already swap space on that partition: I didn't really modify the contents of the various partitions, I just moved entries around in the partition table.)

14. Install pdisk-modified A-drive back into DTiVo, power up, enable back doors, search /var/log/kernel:
kernel: Activating swap partitions
kernel: Adding Swap: 130744k swap-space (priority -1)

Robert S: Will this fix the GSOD/reboot loop that some people have?

[gigageek]

After working out the steps that (I hope) will repair the swap partition and end the GSOD/reboot cycle on boxes that previously upgraded with -s 128, I upgraded my Sony SAT-T60 DTiVo using -s 127. A quick check of /var/log/kernel revealed:

kernel: Activating swap partitions
kernel: Adding Swap: 130044k swap-space (priority -1)

So, just as Robert S stated (ever so clearly) a few pages back, -s 127 definitely will give you a good backup with active swap . :D :D :D

All my recordings are present & play (resuming from exactly where I left off prior to upgrading). System Information says my recording capacity is now "Variable, up to 67 hours" (single 80GB drive). So, according to SI, I doubled the size of the drive and got LESS than double the recording capacity (despite the fact that there's no OS on the extra 40GB of space I added). Does this seem right to people? :confused:

Prior to upgrade, SI said "Variable, up to 35 hours" but stuff started getting deleted when the drive got between 25 & 30 hours of stuff recorded. Was the 35 hours hard-coded on the unhacked boxes because that's what they were marketed as? In any case, an extra 30 hours will carry me long enough for my local computer store to get another Seagate 80GB drive in stock ;) (I bought the last 3 they had on Friday, and 2 of them are slated for my SA).

Next up: Back up my Sony SVR-2000 with -s 128 and verify the fix I posted earlier this morning will repair the swap partition. What's the little emoticon for "gesture of fingers crossed for luck"?

[Robert S]

Outstanding work giga!

The only bit I didn't quite follow was the back-out. It didn't apply to you because you already had a partition large enough to be the inactive root, but everyone else will want to revert to exactly the partition table they had before they changed it so their TiVoes will take the next software upgrade.

If you repeat the procedure, could you jot down the precise pdisk sequence needed to get back to where you were. I think it's obvious, but I'd like to write 'this has been tested'.

Also, how could you tell from the partition table which is the active root partition? I don't think a software upgrade changes the partition names, so either Root 1 or Root 2 could be active depending on how many updates you've had.

[gigageek]

Originally posted by Robert S
1. If you repeat the procedure, could you jot down the precise pdisk sequence needed to get back to where you were.

2, Also, how could you tell from the partition table which is the active root partition? I don't think a software upgrade changes the partition names, so either Root 1 or Root 2 could be active depending on how many updates you've had.

Thanks, Robert. It's nice to give back to the TiVo community that has helped me before. People should probably take that approach IRL more often...

1. The precise pdisk sequence needed to get back to where I was WHEN? I think I understand that you're saying it was fortuitous (for me) that I used -s 128 because that meant the inactive root & swap partitions were the same size & I could just swap them. I'm pretty sure this won't work if someone did -s 512 (or any number greater than 128). Do you now want me to delete & recreate partitions 7 and 8 to look like they did in Step 8 (before I swapped them)?

2. DOH!!! I guess I'm not so clever as I thought I was when I posted that. I thought "Root 1" meant active, "Root 2" meant inactive. (This is why I always reserve the right to be wrong.)

[Robert S]

If you have a 128Mb hole, like you did, you can just make a swap partition on that. The people who will need this hack have a full disk with a 64Mb swap partition, but they can take over their inactive root partition to be a larger swap partition until their TiVo fixes itself, but they'll need their original, 128Mb root partition for the next upgrade.

So we need the pdisk sequence to get back to how the partitions were before you changed them with pdisk.

[gigageek]

I restored the A-drive to its original partitions, although I stumbled at one step (I forgot to do mkswap after switching partitions back).

15. Reinstall new Seagate 80GB drive on hdb (FAT32 drive on hda, CD-ROM on hdc). Verify the BIOS recognizes all drives before booting MFSTools 2.0 CD with "vmlnomda hdb=bswap".

16. Use pdisk to delete/recreate partitions 7 and 8.
# pdisk /dev/hdb
Command (? For help): d (for "delete partition")
Partition number: 7
Command (? For help): C (for "create with type specified")
Partition number: 7
First block: 75685952 Use the number shown for partition 8 for YOUR DRIVE in Step 8 above.
Length in blocks: 262144
Name of partition: "Linux swap" (quotes are required)
Type of partition: Swap
Command (? For help): d (for "delete partition")
Partition number: 8
Command (? For help): C (for "create with type specified")
Partition number: 8
First block: 75423808 Use the number shown for partition 7 for YOUR DRIVE in Step 8 above.
Length in blocks: 262144
Name of partition: "Root 2" (quotes are required)
Type of partition: Ext2
Command (? For help): w (for "write the partition table")
Writing the map destroys what was there before. Is that okay? [n/y]: y

This reordered the partition table back to the way it was in Step 8 (before I swapped partitions) as shown:

Partition map (with 512 byte blocks) on '/dev/hdb'
#: type name length base ( size )
7: Ext2 Root 2 262144 @ 75423808 (128.0M)
8: Swap Linux swap 262144 @ 75685952 (128.0M)


At this point, I re-installed the drive back in the DTiVo, but it couldn't mount the swap because I had forgotten to create it. (DOH!!) Back to the PC, again boot MFSTools 2.0 CD with "vmlnomda hdb=bswap", then:
# mkswap -v0 /dev/deb8

17. Install pdisk-modified A-drive back into DTiVo, power up, enable back doors, search /var/log/kernel:
kernel: Activating swap partitions
kernel: Adding Swap: 130744k swap-space (priority -1)


Once I figured out what is was that I was supposed to be doing, this whole process wasn't that painful (except for my occasional ineptness, e.g., forgetting to run mkswap at end of Step 16). I'm pretty certain that the difficulties I had getting started were related to one of two things:

A. Forgetting to make certain the BIOS recognized all devices connected to the system. This step doesn't exist on the SGI boxes I'm most familiar with, and I don't upgrade PCs often enough for this to become an ingrained habit. I'm pretty certain I've developed this habit now, although I'm equally certain it will wear off after about 5 or 6 sleep cycles.

B. Getting byte-swapping hosed up on the different boot floppies/CDs that I used. As with all things Unix, the learning curve was pretty steep here. To be clear, any how-to for fixing the swap issue should include the following statement (or similar): Accept the default boot parameters when booting the MFSTools 2.0 CD for running mfsbackup/mfsrestore/mfsexpand, but be advised that pdisk will not run properly; when booting this CD to run pdisk, use "vmlnomda hdX=bswap" boot parameters (where X is the TiVo drive ID), but be advised that mfsbackup/mfsrestore/mfsexpand will not work properly.

NOTE: I never did figure out how to make pdisk work with TiVomad floppy or Kazymyr's Boot CD, mostly because I stopped trying once I got pdisk to work with MFSTools 2.0 CD. I'm certain it was just operator malfunction.

Robert: I assume the process for repairing a 64MB swap partition that locks on GSOD/reboot loop would be exactly the same, but the block numbers would be different. I'd rather not spend the time verifying this if you concur, and I'm going to need that drive soon. (I'm currently working the same hack on my SVR-2000.) Are we done with the DTiVo swap-fix hack?

[Robert S]

Brilliant!

You're quite right, once you've done it, everything that seemed so difficult seems trivial - I'm sure if you boot Kazymyr now, everything will work perfectly. That's mainly why I wanted someone else to do it - I knew you'd fall into any hole I left I my explanations, so I can now be sure everything is covered.

Can you not use r to renumber the partitions back the way they were? Doesn't r 8 7 do the trick? Or do the labels get mucked up?

[gigageek]

Originally posted by Robert S
Can you not use r to renumber the partitions back the way they were? Doesn't r 8 7 do the trick? Or do the labels get mucked up?

I'm pretty sure you could do that, but I had hosed up the labels in Step 13 (operator malfunction).

Rather than re-label the partitions when I do my SVR-2000, I'll just swap them, do mkswap, verify swap activation in tivo, and then swap them back. I'll post how it works out.

[Robert S]

OK, I've written your work up and posted it as the second post on this thread (fortunately I have several posts on the first page, and that post contained outdated info).

If you'd like to read it through and point out mistakes/improvements...

[gigageek]

This hack was performed on a Sony SVR-2000 that was hacked in March or April 2001 with Dylan’s Boot Disk. This SA TiVo had a 30GB (Maxtor 33073H3) A-drive and an 80GB (Maxtor 98196H8) B-drive.

1. Install the drives in the following order:
hda: 30GB TiVo A-drive (Maxtor 33073H3)
hdb: 80GB TiVo B-drive (Maxtor 98196H8)
hdc: CD-ROM
hdd: FAT32 drive
Verify the BIOS recognizes all drives before booting MFSTools 2.0 CD with default boot parameters.

2. Create backup of the TiVo A-drive:
# mkdir /mnt/dos
# mount /dev/hdd1 /mnt/dos
# mfsbackup -6so /mnt/dos/svr2000.bak /dev/hda /dev/hdb

3. Stop here to burn a CD-R of the backup. I know this step isn't really necessary, but I'm pretty anal-retentive about having multiple backups (you can never have too many). Plus, by restoring from the CD-R backup, this verifies that the CD copy of that backup will indeed create a usable drive.

4. Install the drives in the following order:
hda: FAT32 drive
hdb: new 80GB (Seagate ST380021A) drive
hdc: CD-ROM
Verify the BIOS recognizes all drives before booting MFSTools 2.0 CD with default boot parameters.

5. Restore from the CD-R backup:
# mkdir /mnt/dos
# mount /dev/hdc /mnt/dos
# mfsrestore –s 128 –zpi /mnt/dos/svr2000.bak /dev/hdb
The -s 128 hoses up the swap partition (intentionally), as documented earlier in this post.

6. Boot the restored Seagate drive in my TiVo, verify that I get the swap errors in /var/log/kernel:
kernel: Activating swap partitions
kernel: Unable to find swap-space signature
kernel: swapon: /dev/hda8: Invalid argument

7. Reinstall new Seagate 80GB drive on hdb (FAT32 drive on hda, CD-ROM on hdc). Verify the BIOS recognizes all drives before booting MFSTools 2.0 CD with “vmlnodma hdb=bswap”.

8. Determine active root partition:
# cd
# mkdir /mnt/tivo/4
# mount /dev/hdb4 /mnt/tivo/4
# ls –l /mnt/tivo/4
<most-recent date was June 11, slightly more than 2 months ago>

Per Robert S, this suggests that hdb4 is the active partition. To verify, I did:
# umount /dev/hdb4
# mkdir /mnt/tivo/7
# mount /dev/hdb7 /mnt/tivo/7
mount protested with “mount: you must specify the filesystem type.”
From this, I concluded that ACTIVE root=/dev/hdb4, and hdb7 is the INACTIVE partition.

9. Create a swap file on the INACTIVE root partition (hdb7):
# mkswap –v0 /dev/hdb7
output from the above command was:
mkswap: warning: truncating swap area to 130752kB
Setting up swapspace version 0, size = 133885952 bytes

10. Reorder the original TiVo swap partition (hdb8) and the inactive root partition with newly-created swap (hdb7). The original partition table was:
Partition map (with 512 byte blocks) on '/dev/hdb'
#: type name length base ( size )
7: Ext2 Root 2 262144 @ 56929344 (128.0M)
8: Swap Linux swap 262144 @ 57191488 (128.0M)
# pdisk /dev/hdb
Command (? For help): r <for “reorder partition entry in map”>
Partition number: 8
New number: 7
Command (? For help): w <for “write the partition table”>
Writing the map destroys what was there before. Is that okay? [n/y]: y

This reordered the partition table as shown:

Partition map (with 512 byte blocks) on '/dev/hdb'
#: type name length base ( size )
7: Swap Linux swap 262144 @ 57191488 (128.0M)
8: Ext2 Root 2 262144 @ 56929344 (128.0M)

11. Install pdisk-modified A-drive back into TiVo, power up, enable back doors, search /var/log/kernel:
kernel: Activating swap partitions
kernel: Adding Swap: 130744k swap-space (priority -1)

12. Reinstall new Seagate 80GB drive on hdb (FAT32 drive on hda, CD-ROM on hdc). Verify the BIOS recognizes all drives before booting MFSTools 2.0 CD with “vmlnodma hdb=bswap”.

13. Reorder swap and inactive root partitions back to their original conditions:
# pdisk /dev/hdb
Command (? For help): r <for “reorder partition entry in map”>
Partition number: 8
New number: 7
Command (? For help): w <for “write the partition table”>
Writing the map destroys what was there before. Is that okay? [n/y]: y

This reordered the partition table back to the way we started, as shown:

Partition map (with 512 byte blocks) on '/dev/hdb'
#: type name length base ( size )
7: Ext2 Root 2 262144 @ 56929344 (128.0M)
8: Swap Linux swap 262144 @ 57191488 (128.0M)

14. Create swap space on TiVo swap partition (hdb8). This was never created if upgrade was performed with –s 128.
# mkswap –v0 /dev/hdb8
output from the above command was:
mkswap: warning: truncating swap area to 130752kB
Setting up swapspace version 0, size = 133885952 bytes

15. Install pdisk-modified A-drive back into TiVo, power up, enable back doors, search /var/log/kernel:
kernel: Activating swap partitions
kernel: Adding Swap: 130744k swap-space (priority -1)

Another satisfied TiVo box. :cool:

[gigageek]

Boy, I wish I had your instructions when I started on this. ;)

My suggestions (and they are for MINOR changes):
1. Rename section C to "Prepare the inactive partition as a swap partition." (Change in 2 places.) I think this makes it just a little bit clearer for someone unfamiliar with TiVo disk partitions.
2. Include in section B the alternate method of attempting to mount partitions 4 and 7: the active partition mounts, the inactive partition will not. This allows the whole process to be run from MFSTools 2.0 CD. People with low-bandwidth connections will appreciate not having to download another large boot image if all they have is the MFSTools 2.0 CD.
3. The title for section D didn't get underlined properly due to a vB Code typo.

One other typo that's in your first post: The synatx for edit_bootparms is
# /mad/edit_bootparms hdb -r
(not /dev/hdb).

And the reason why I could never get edit_bootparms to work was because I was using MFSTools or (plain) TiVomad, not DTiVomad.

"It's all become clear to me now."
--Dave Bowman

[Robert S]

Thanks for that. If anyone else spots something that's wrong, unclear or out-of-date with those top posts, do say. The intention is to summarise the current info that's in this thread without people having to read down to here to get it.

[deek_man]

First, thanks to all the folks who have worked so hard to solve the swap problem. I have a couple of quick questions about fixing this problem when/if it occurs and about the instructions for a fix in Robert S's second post on the first page of this thread.

A couple of months back, I upgraded a Philips series 1 standalone using two 100gb Maxtor drives for a total of 200 gigs. I used MFSTools 2.0 but did not specify a swap partition. I checked the log files and I have 64 mb of swap. I understand this could be problematic if mfsfix runs to repair corrupted files. This has not happened yet and the box appears to be operating normally. Here's the questions:

1) Do you still believe we should not fix the swap if the box is operating normally and only initiate a fix if the green screen freeze occurs?

2) Regarding the fix directions: Do you type "vmlnodma hda=bswap" after booting normally with the MFSTools 2.0 CD? (I think that's what step A. is saying.)

3) In step D. of the directions, you renumber the partitions and then in step F. you restore the original partition numbers. Does this mean that the inactive root partition (4 or 7) is the opposite partition for Step F compared to what it was in Step D?

Again, thanks for all the help.

[Robert S]

64Mb of swap will only cause a problem if you green screen. There's no way to increase swap without losing your recordings, so if you want to keep them you don't have much choice. The emergency fix should work well enough for you to feel safe with 64Mb of swap.

You type the magic incantation at the boot: prompt - it changes the way it boots. Would you like to try it and suggest a clearer form of words?

You want to get the partition table back the way it was before you started. Thus the partition numbers are the same as in step D.

Thankyou for you comments.

[drewba]

I'm thinking that there is still a hole in the Hinsdale-How-To. In step 8 it directs us to mfsrestore the image to the new larger upgrade drive using an -s 127 switch to test the backup. However, assuming the backup works, those adding a B drive use their original A drive and run an mfsadd to add the new B drive. Thus the swap has never been actually been increased. Am I looking at that correctly?

[Robert S]

That's not how I read it, he's telling you to restore to the new drive, expanding swap. If you want to use your old A drive too you should add it as a B drive.

If you want to keep your recordings, it's better to do a pipe transfer rather than just blessing a B drive.

[drewba]

Originally posted by Robert S
That's not how I read it, he's telling you to restore to the new drive, expanding swap. If you want to use your old A drive too you should add it as a B drive.

If you want to keep your recordings, it's better to do a pipe transfer rather than just blessing a B drive.

But Upgrade Configuration #1 in step 10 is adding a new B drive to any Single Drive TiVo and keeps your recordings. I would hazard a guess that this is the most common upgrade, but it seems to me that doing this upgrade following the directions will result in a swap file that isn't increased to 127MB.

[Robert S]

The only way to save your recordings and increase swap is to do a pipe transfer (basically MFS Tools 2.0 can only increase swap if it's doing its restore function). It makes sense to do this instead of blessing, even if you're not going over the limit.

Despite the simplicity of blessing, if you don't increase swap it's very difficult to increase swap if you want to upgrade further and keep your recordings (adding swap would be a very similar process to the 'rescue' hack at the top of this thread, but you'd use pdisk to create a partition to relabel, rather than using the root partition).

The trouble is, to explain this properly would add at least a page to Hinsdale. Unfortunately, if you don't understand this, you might conclude that a TiVo stuck in a green screen loop is unrecoverable, whereas infact it can be recovered with a clever hack.

It would be nice to say to everyone who upgrades, 'if you go over 140Gb and you didn't expand your swap, tape a note with the address of the Upgrade Forum inside the lid - you'll know when you need it'.

[deek_man]

Thanks for the help, Robert. But if one wanted to save their recordings before a green screen freeze AND repair the swap partition for a 200gb system with only 64 mb of swap, could one use:

mfsbackup -Tao - /dev/hdx | mfsrestore -s 127 -xpi - /dev/hdy /dev/hdz

:confused:

[Robert S]

That's what I mean by 'a pipe transfer' - the '|' character is pronounced 'pipe'.

It's not always possible to do this - if you want to upgrade the A drive of a twin drive machine without losing recordings, for example. You can increase swap between running dd and mfsadd, but the process is just as complex as the 'rescue' procedure.

I agree that Hinsdale skates over an important issue by not mentioning swap issues. Advising everyone to use -s 127 is a good start, but to cover it properly you'd have to explain:
What swap is
You need more swap >140Gb
Most upgrades can increase swap by using -s 127
Most of the rest can increase swap by using a pipe transfer instead of blessing or dd + mfsadd. And that despite the extra pain this causes, it's well worth it.
A drive expansions on twin drive machines can not expand swap without losing recordings, but there is a hack you can use if you want.
If you go over >140Gb without expanding swap your TiVo will get stuck in a green screen loop if it gets filesystem corruption. If this happens, there's a hack to recover your TiVo without losing your data or recordings.As there's already so much in Hinsdale, I can appreciate he's reluctant to add more. New Hinsdale was written from Tiger's documentation, which claims that you don't need more swap if you use MFS Tools 2. This has since been proved false. One could argue that in the light of that discovery, it needs a major overhaul.

[mrtickle]

The last bullet point is the single most important omission IMHO. It should be screamed in huge letters at the start of the document!

I bet there are thousands of people out there with machines >140GB and only the default size swap. Not just MFStools2 upgrades, but older ones too. I only found out by a fluke chance just in time before I did my own upgrade.

[BobCamp1]

Also, upgrade kits should be avoided! Ordering a large pre-blessed drive and installing it will make your Tivo unstable. The full upgrade service has to be used (where you ship the entire Tivo), and you have to be smart enough to insist the service also increase the swap file size using MFSTools 2.0.

This could put Hindsdale and others who have performed upgrades in trouble, since they have been unintentionally screwing up some (but not all) of the Tivos. And people who were too scared to do the upgrade themselves are surely not going to want to run the emergency fix procedure. Some of these people will demand a free fix or demand to be reimbursed the $100 they had to pay Tivo to have them fix it.

Without Hindsale's instructions, I would have never attempted the upgrade in the first place. My guess is that around 70% of the people simply want to add a new drive. Those instructions are incomplete. It's not his fault (it's actually Tiger's), but he needs to fix them.

[jhburke]

I have been following this thread with a lot of interest, and I am impressed with the clever work of the contributors in figuring out the swap problems. However, I'm not so sure that the old "BlessTiVo" method is necessarily bad.

It seems that large upgrades (ie 30 + 120) have been performed for a while, using the old tools, but no problems reported until MFSTools 2.0 was released. Then, shortly after MFSTools 2.0, there was a torrent of reports about GSOD and failures from inadequate swap. It doesn't seem possible to me that sudden drive failures causing GSOD should suddenly start occuring once 2.0 tools are used - perhaps the problem doesn't occur with mfstools 1.1 or BlessTivo. PTVupgrade stated in an earlier post in this thread that he's never had this type of problem despite never increasing swap. Perhaps mfstools 2.0 increases the swap requirement rather than decreasing it, and perhaps a 30 +120 upgrade with BlessTiVo can recover from GSOD with 64M swap.

Of course, I don't know the answer, but it might be interesting if someone could test this theory (hint-hint anyone?). Then all the people with older upgrades and default swap may not need to panic yet.

John

[mrtickle]

Systems >140GB have always needed more swap. Always! This is unfortunately not made clear in the guides - that was my point and is my only criticism of the guides. Everyone who has >140GB and only default swap has a TiVo which WILL crash and go into an infinite reboot loop if it ever gets a GSOD! The only get-out is to pull the drives and perform the procedure to give emergency extra swap as detailed earlier in the thread.

There was a separate problem whereby anyone following the instructions (-s 128) ended up with no swap at all. People who then had GSODs (for whatever reason) then found themselves in trouble. Initially it was thought that less swap would be needed for mfsfix with the bigger blocksize that MFStools uses by default. The complete lack of swap may have induced many of the GSODs. This happened to lots of people at the same time so that's why the issue got a lot of attention.

[Robert S]

A 150Gb TiVo might just squeak through a GSOD. Merle Corey published his best estimate of the formula needed to estimate the precise amount of swap needed. Above about 155Gb there's no possibility of recovering. The swap requirements for MFS Tools 2.0 are exactly the same as for any other tool, despite the MFST2 docs repeatedly saying it's more efficient.

I don't think there was a flood of people getting green screens - in fact quite the reverse, there was a nasty gap between embeem pointing out there was a problem and reports coming in from others confirming it.

I don't think the absence of swap causes any real problems for the media side of TiVo - the guide and indexer get gummed up, but the video side is fine.

There were a lot of people talking about the risk of a GSOD, but only a few actually experience one.

This problem has always existed - TiVoMad will increase your swap for exactly the same reason - but larger drives and inaccurate claims of MFS Tools 2's powers make it more visible.

[muchmore44]

I just want to say thanks to Hinsdale and Tiger for giving us TiVo users a way to maximize a terrific technology, as well as to Robert S. and Merle C. for refining the swap expansion/correction process.

Last night, I upgraded my Hughes DirectTiVo from 35 hours to 106 hours (went from 40 GB to 120 GB) and it couldn't have gone better! It took approximately 1 hour, 30 min, mostly because I was very, very careful - I could have done it faster, but I didn't want to take any chances. MFSTools 2.0 are the best!

I am the happiest guy in the world right now (at least until the next person upgrades a Tivo!) and wanted to thank you guys for all that you have done.

You guys are Saints!

Thanks,
Muchmore44

[BobCamp1]

Note that your system merely becomes unstable with "only" a 64 MB swap file. "Unstable" in the sense that the Tivo will crash very infrequently, but it will crash so hard as to be unusable. Tivo only runs mfsfix under normal conditions when the file system on the hard drive gets really screwed up.

This usually happens when the hard drive is failing anyway, so those people simply attributed their misbehaving Tivos for a bad hard drive.

However, since Tivo doesn't have an "off" switch, and it constantly writes to the hard drive, you're at a low risk every time you unplug the Tivo. And when the power flickers at your house, too.

Also there is an issue of timing:

1. 100 and 120GB hard drives just became affordable enough to buy this summer. An 80GB hard drive, even when combined with a Series 2 60 GB Tivo, will not exhibit this problem.

2. Series 2 units and newer DirecTivos (with their larger factory-shipped capacity) started shipping earlier this year.

3. MFSTools 2.0 also came out at about the same time. The -s 128 "bug" was causing a different set of problems and was adding to the confusion.

4. Power to your house (and Tivo) was flickering on and off from thunderstorms this summer. This could trigger mfsfix to run on your Tivo and expose the problem.

Basically, we were doing bad upgrade procedures the entire time. But it wasn't until this summer that a bunch of things converged to magnify the problem.

[hinsdale]

I think this thread might be more useful if someone were to do some more definitive testing to determine the actual upgrade capacity that will fail to recover from GSOD with 64MB Swap (taking into account 16 vs 32MB RAM) - not forumula approximations. I do not have time to do this currently but before unnecessarily concerning people with statements like "your unit will not recover from a GSOD if you are over 140GB", I believe some more testing is in order (results labeled GB or GiB). This issue is not new, and having witnessed posts, interchange, etc of thousands of upgraders adding 120GB drives to their 20, 30, 40, 60GB units without increasing swap and without widespread failure or even any pattern of failure - I am not sure this issue rises to the crisis level portrayed. I would be interested,however, in determining the actual trigger number. If I am not mistaken, my 160GB DirecTiVo with unmodified swap has recovered from GSOD, which makes me dubious of the trigger numbers being fowarded recently.

[enigma2175]

I may have missed this somewhere in this quite large thread, but couldn't a recovery from an unrecoverable (mfsfix crash because of lack of swap) go like this:

1. Get diagnostic prompt, append HANDCRAFT=TRUE RUNMYWORLD=FALSE to boot parameters. (assuming a bash prompt modification)

2. At bash prompt make sure current swap is valid :
mkswap /dev/hda8
Check kernel logs to make sure swap is being enabled.

3. If mfsfix is still crashing (You have way to much space for your own good) create a swapfile on your current /var partition.
dd if=/dev/zero of=/var/SWAPFILE bs=1024 count=64000
mkswap /var/SWAPFILE

4. Add the swap to start at boot. Append
/var/SWAPFILE swap swap defaults 0 0
to /etc/fstab

You now have 128 MB of swap without altering the partition layout. Takes some space in /var, but it should at least get you up and running.

Has this already been proposed and I just missed the post?

[enigma2175]

... or you could put a 32 MB swapfile in / and in /var (I'm not sure if /var is already mounted when mfsfix runs, but it most likely is).

[enigma2175]

Hmm, I shouldn't post before I think (hint: swap isn't much good on a read-only filesystem :) ) Disregard that last idea... But the swapfile in /var should still work.

[Robert S]

Glad you're thinking about this issue. A bit of brain-storming never hurts.

The immediate problem with all your suggestions so far is that DTiVoes and S2's are locked down. The other problem, of course is that this procedure is actually more difficult than the rescue or the manual swap creation (got that written up at last. Thanks to angra).

If you wanted to use the space on /var for swap, wouldn't it be better to change the partition table to shrink /var and expand partition 8? (You would then run mke2fs on hdx9 and mkswap -v0 on hdx8). This should work on any TiVo.

My advice is still, expand swap if you can, try not to worry about it if you can't/didn't - just remember the rescue maneuver exists if you need it.

[enigma2175]

If you wanted to use the space on /var for swap, wouldn't it be better to change the partition table to shrink /var and expand partition 8? (You would then run mke2fs on hdx9 and mkswap -v0 on hdx8). This should work on any TiVo.


I like to change my partition table as little as possible :D. Assuming this is an upgrade gone wrong, it is very possible there are a number of things in /var that you want to keep. Also, if you have the swap broken up into multiple partitions/files, you can have over 127 MB of swap. I don't know that anybody NEEDS more than 127 MB (with only 16MB of RAM..Yikes!) but it's nice to know that it can be done if needed. Also, if you have (had) a Bash prompt on your TiVo you do not need to take out your drive. The constant swapping of the hard drives back and forth beween the TiVo and the computer is not good for the drive. The interface and jumper pins can get bent or broken, the drive can get dropped, or the controller board can get toasted from an ESD. I prefer to keep my drive in my TiVo whenever possible.

[Robert S]

Fair enough.

Merle calculated that 127Mb of swap is enough for the largest currently possible (274Gb) TiVo to recover from a GSOD, but if you're only just over the limit a small swap file on /var might do the trick.

[Merle Corey]

Originally posted by hinsdale
I think this thread might be more useful if someone were to do some more definitive testing to determine the actual upgrade capacity that will fail to recover from GSOD with 64MB Swap

On the one hand, I'm not sure I care. :D My goal when I started testing was to prove one of two things - either that MFST2 was broken and to be avoided at all costs or that it worked (consistently) with the right "adjustments." That the only adjustment needed was a command line flag (and some misconceptions) was pure gravy.

On the other hand, you raise a valid point - knowing the exact "at risk" threshold could be useful.

Of course, it's been done (http://www.tivocommunity.com/tivo-vb/showthread.php?s=&threadid=23389). It's in the FAQ, in the segment on Question #7. "Around 150GB" was the result, which dovetails nicely with the predicted 140GiB result.

It's also been documented (http://www.tivocommunity.com/tivo-vb/showthread.php?s=&threadid=33594) by several people that 160GB is too big (http://www.tivocommunity.com/tivo-vb/showthread.php?s=&threadid=28211).

My replacement TiVo arrives Monday - I can put this to the test then if we think it's really that important. So my questions are:

* Do we really care where, exactly, it breaks, or is our current information close enough? Is an exact value at all meaningful when it applies only to one specific hardware platform? Is it better to have a more arbitrary but safer threshold value that acts as a general rule of thumb, or to have exact breaking points on all platforms?

Personally, I think it's close enough. In fact, I'd hedge a little anyway just to play it safe (which is why I still say 140GB most of the time instead of the down to the wire 140GiB/151GB). As Robert has pointed out, that leaves the door open for less painful future upgrades. That said, I'm still willing to look into this.

Beyond that, nobody is saying there's a crisis - we're just trying to get the word out that people with >140GB configurations may experience greenscreen loops iff they have filesystem errors...and let's face it, (real) filesystem errors aren't that common in non-failing equipment anyway.

It's a warning. The sky isn't falling, but it may hail later in some areas.

* If our current information isn't close enough, how close *is* close enough? The exact value? Is confirmation that 160GB is too much but that 140GB is safe good enough?

Finally, a comment on my formula: It's already proven accurate to within 5MiB on smaller (20 - 120GB) capacities and within 10MiB on larger (240GB). The only reason it's not nailed down tighter than that is that I didn't take the time to determine exact breaking points - the margin of error is the margin from my limited tests, not from the formula. If there's interest, I can also determine the exact margin of error - assuming there's a measurable one - if/when I do any further tests.

Personally, I don't think there's need for it - we know 64MiB isn't always enough, we know 127MiB is enough for all (128GiB limited) configurations. Does it matter beyond that point?

MC

[gigageek]

Originally posted by Merle Corey
Personally, I don't think there's need for it - we know 64MiB isn't always enough, we know 127MiB is enough for all (128GiB limited) configurations. Does it matter beyond that point?

Merle has the right idea, IMO: 127 is an "engineering solution". If it turns out that someone nails down a magic number of 125 (or 118 or ...) for a particular drive capacity, is anyone seriously going to suggest they NOT use 127?

I also think Hinsdale has a valid point that it is not a mathematical certainty that every >140GB configuration will "not recover" from a GSOD. (The proof, however, is "left to the student". ;) ) While hard drives are fairly reliable, enough of them fail that we probably would have seen more units with the GS/reboot loop if this were guaranteed to be a problem under all circumstances.

Perhaps Robert's second post could be softened to say:
If you have more than 140Gb of hard drive space in your TiVo, the original 64Mb of swap space might not provide enough memory for the filesystem checker (mfsfix) to repair corruption to the filesystem.


Originally posted by Robert S
If you go over 140Gb and you didn't expand your swap, tape a note with the address of the Upgrade Forum inside the lid - you'll know when you need it'.

This is probably the best advice I've seen on the subject. It tells the reader, "Don't panic, remain calm, your house is probably not on fire. But you might want the phone number of the fire department handy, just in case it catches fire at a later date."

[DCIFRTHS]

I have been following this thread since it's inception, and there has been a lot of information. I would like to ask a very simple question:

Can I create a TiVo that contains 2 x 120 GB* hard drives using MFS Tools 2.0, and a command line to create a 127 meg swap file, that will successfully complete mfsfix if the TiVo crashes?

* Please note that when I specify 120GB drives I am going by the size listed on the Maxtor box.

I'm going to read through the documentation, determine the correct procedure, and then post back here before I upgrade, so that if I make any mistakes, hopefully someone will spot it and point it out to me.


Thanks all !

[Robert S]

127Mb swap is a complete fix for all the problems this thread has addressed. It has been tested on a 240Gb TiVo and calculated for a 274Gb TiVo (the largest currently possible).

If you follow New Hinsdale, MFS Tools 2.0 will expand your swap for you (the -s 127 option).

[mnc042]

If you follow New Hinsdale, MFS Tools 2.0 will expand your swap for you (the -s 127 option).

But, reading Hinsdale, if I have a DirecTiVo with a single A drive, and I want to simply add a new, 120GB B drive, isn't there that hole in the instructions that was mentioned back a ways?

i.e., the instructions basically tell you to backup your A drive, restore that backup on to your new B drive, then expand/mate that new B drive to your original A.

In this scenario, the A drive's swap is not expanded, and you wind up with 160gb and 64mb swap.

Or did I miss something in Hinsdale?

Thanks!
\marc

[Robert S]

That's right, Hinsdale's instuctions for that scenario are unhelpful. I don't have any control over what's in Hinsdale - and what he says is right, it's just some of the advice as to when to do what assumes you don't need to expand swap.

Hinsdale does describe the pipe transfer technique in a different context. Create your A drive with a pipe transfer and MFS Tools 2.0 will copy your recordings and expand your swap (you can even fill the drive and bless in your B drive in one step, if you want).

To 'fix' New Hinsdale you'd have to add probably a page of discussions of what the swap issues are and how to work around them. For the most part advising people to use -s 127 fixes the problem anyway. The problem seems to be largely theoretical - weaknees say they can't remember ever seeing a machine stuck on a green screen.

My position - oft repeated - is:

If you can expand swap when you upgrade, do so (this probably covers 90%+ of upgrades).

If this is the sort of thing that would worry you, then we have ways of adding swap in every possible upgrade scenario. They are time consuming and/or convoluted.

If you decide to 'chance it', either because you didn't understand the swap issue when you upgraded, or your upgrade profile doesn't allow you to expand swap easily, you may not hit a green screen in the lifetime of your hard drives. If you do green screen and get stuck, we have a 'rescue' maneuver that will allow your TiVo to recover if running out of swap caused the green screen to stick.

[mnc042]

Originally posted by Robert S
[B]That's right, Hinsdale's instuctions for that scenario are unhelpful.

Ah okey. So maybe I should pick a different scenario, where I replace my A drive with my new 120GB drive. I notice that in this scenario, since you're doing the restore with the -s 127 option, that should work well. And, I'd do this by:

Hinsdale does describe the pipe transfer technique in a different context. Create your A drive with a pipe transfer and MFS Tools 2.0 will copy your recordings and expand your swap

In this scenario, I'd either keep my old A drive as a backup on a shelf somewhere, or I could decide to re-bless it as a B drive, as in:

(you can even fill the drive and bless in your B drive in one step, if you want).

This sounds like the safest option.

If you can expand swap when you upgrade, do so (this probably covers 90%+ of upgrades).

Agreed, and that is the way I will go.

Thank you so much for your help!

\marc

[DCIFRTHS]

Originally posted by Robert S
127Mb swap is a complete fix for all the problems this thread has addressed. It has been tested on a 240Gb TiVo and calculated for a 274Gb TiVo (the largest currently possible).

If you follow New Hinsdale, MFS Tools 2.0 will expand your swap for you (the -s 127 option).

I will be using two new drives, and I am not going to worry about saving recordings.

[Merle Corey]

<sigh>

127MiB is no longer enough for all configurations.

Jafa really is that cool. (http://www.tivocommunity.com/tivo-vb/showthread.php?s=&threadid=74723)

Theoretically, with quad 200's, the universally safe value is now nuzzling 367MiB of swap. :D

Originally posted by gigageek
I also think Hinsdale has a valid point that it is not a mathematical certainty that every >140GB configuration will "not recover" from a GSOD. (The proof, however, is "left to the student". ;) )

Literally, that's true.

Warning: More information about fsfix, MFS, and journaling ahead than anybody really cares about.

However, for every specific hardware/software combination, there is a point beyond which there is a mathematical certainty that it will not recover with 64MiB of swap. On 16MiB RAM systems, it's around the 151GB/140GiB mark. On 32MiB systems, it *should* be close to the 182GB/170GiB mark.

fsfix requires a very specific amount of RAM per GiB being repaired; that number isn't going to change, except perhaps in small ways in the form of alterations to MFS between software versions. The bigger change is going to come from either the amount of system RAM (16MiB vs 32MiB) or the memory footprint on that version/platform.

Assuming that Tiger's -r setting is really changing the default block size (and we have at least some evidence showing that it does), it would appear that fsfix doesn't just operate by looking at the allocation table, it actually requires specific data from each block - quadrupling the block size may mean that it only has to look at one fourth as many blocks, but it also quadruples the amount of data that fsfix needs to snarf from each block. Net gain: 0.

This makes perfect sense from a journaling standpoint - it would be a bad journal, indeed, that kept the same amount of data for 1MiB and 4MiB block sizes, rendering data recovery *much* more difficult. And yes, I'm implying that the RAM fsfix requires is the amount of RAM needed to load the journal (or some specific subset thereof) into memory. This would be why GSOD looping appears fairly quickly rather than after the amount of time it would take to "process" the maximum supported space of a 64MiB swap. The formula I developed translates into 1 byte of fsfix data per 2048 bytes of filesystem data - a fairly reasonable journal, especially given that 2K of multimedia data isn't going to appear as much more than a stopple... The formula may or may not be dead on accurate, but it is *absolutely* reasonably accurate. (I would suspect that it's actually 1 byte per 2047 bytes of data, but it's going to be damn hard to prove that when dealing with multi-GiB setups and MiB based swaps - I really don't care enough to nail everything down into bytes.)

At this point it looks like the only way to get fsfix to run in a significantly more efficient manner would be for it to not require that the entirety of MFS space be mapped out at once - in other words, if there were some way to get it to look at one drive - or better yet, one partition - at a time. That, however, is something that I suspect is an MFS "limitation" - if it's all one big virtual space, it probably has to be treated as such for repair purposes.

(The above is, where applicable, true about fsck as well. It's common to see fsck problems on linux systems with small amounts of RAM and very large partitions when swap is disabled/broken.)

So let's stop picking nits about swap, and go hound Nick for a release date on the quad drive hack. :D

MC

Typos, typos everywhere...

[hinsdale]

Originally posted by Merle Corey
However, for every specific hardware/software combination, there is a point beyond which there is a mathematical certainty that it will not recover with 64MiB of swap. On 16MiB RAM systems, it's around the 151GB/140GiB mark. On 32MiB systems, it *should* be close to the 182GB/170GiB mark.


These numbers sound more reasonable to me... and would mean that you can safely add any drive, 120GB or less (or larger in some circumstances), to any unmodified TiVo using the standard BlessTiVo or mfsadd procedure without increasing swap.

You can of course easily add any size drive and also increase swap if you dont care about your recordings or dont mind waiting a couple hours for your recordings to copy. Those with 2 drives or previously expanded drives will generally be using the swap expansion methods anyway.

And as has become evident, swap file increase only becomes required in very rare circumstances to begin with.

I will be adding some brief swap file requirement notes to the How-To's over the next week (figure out best wording without adding a page) . Just wanted to get some confirmation/concensus of more reasonable numbers (thanks merle) before unneccesarily concerning anyone.

[hinsdale]

Originally posted by mnc042



In this scenario, the A drive's swap is not expanded, and you wind up with 160gb and 64mb swap.



As I stated before my 160GB DirecTiVo has recovered from GSOD with 64MB swap, and as merles numbers tend to indicate - the threshold number for a DirecTiVo is >180GB so this configuration should not pose any potential problems.

[mrtickle]

Originally posted by hinsdale


These numbers sound more reasonable to me... and would mean that you can safely add any drive, 120GB or less (or larger in some circumstances), to any unmodified TiVo using the standard BlessTiVo or mfsadd procedure without increasing swap.



Not "any" TiVo - all UK TiVos are 40GB to start with so you must NOT add a 120GB drive as a B drive.


I will be adding some brief swap file requirement notes to the How-To's over the next week (figure out best wording without adding a page) . Just wanted to get some confirmation/concensus of more reasonable numbers (thanks merle) before unneccesarily concerning anyone.

Thanks! If you could make the warning simple and prominent - something like "if you are going over 140GB you must NOT just add a large 2nd drive" then that would be great.

[hinsdale]

Originally posted by mrtickle


Not "any" TiVo - all UK TiVos are 40GB to start with so you must NOT add a 120GB drive as a B drive.

Sorry, you are correct, the Brits slipped my mind.. "any US TiVo" would have been more accurate.


Originally posted by mrtickle

Thanks! If you could make the warning simple and prominent - something like "if you are going over 140GB you must NOT just add a large 2nd drive" then that would be great.

I will not be using the 140GB number because it is inaccurate but will be adding a cautionary message basically stating that in the very rare circumstance that an mfsfix utitlity is needed, it will require an increased swap in order to complete if your series 1 standalone is >150GB or if your DirecTiVo or Series 2 unit is >180GB .

[Merle Corey]

Originally posted by hinsdale
(figure out best wording without adding a page) .Just wanted to get some confirmation/concensus of more reasonable numbers (thanks merle) before unneccesarily concerning anyone.

Ok, just to err on the side of caution, I want to make sure this clear: The 32MiB RAM TiVo swap threshold was based entirely on the assumption that it has a similar memory footprint as the 16MiB - in other words, that almost the entire amount of additional RAM (I used 15MiB for my ballpark) can be utilized for fsfix. Since your 160GB DTiVo made it through the GSOD, I'm inclined to think that my assumption was more or less valid (10ish MiB footprint/22ish MiB available for fsfix).

The entire reason I'd been avoiding making an explicit statement on 32MiB TiVo's is simply because I don't have one to test. It's a best guess on my part, and I'd feel pretty rotten if someone ended up hosed because he saw "180GB is probably safe" and ran with it. I'd be happier if someone with a DTiVo would check this out before it becomes part of the documentation.

As for phrasing, I agree with mrtickle - keep it really simple and assume that anybody who needs to follow the How-To in a verbatim, step-by-step kind of way (aka the target audience) probably shouldn't be making judgement calls on exactly where the threshold should be. If you feel it's that important to document what we know about swap and thresholds and such, do a separate bit of "advanced" doco specifically on the subject - the most interesting bits are already beyond the scope of the How-To.

In other words, let "> 140GB" be a general rule for the basic "I just want to add a drive, why are you talking about this linux stuff?" crowd, and keep 150GB & 180GB for the more technically proficient - the people who are most able to make an educated decision, especially keeping in mind future upgrade scenarios. Besides, both of those values are down to the wire, and *neither* has been explicitly tested for exact accuracy. I don't know about you, but I'd feel pretty sheepish if the actual numbers were, say, 149GB and 179GB.

The bottom line is that we don't need that kind of accuracy - not to prevent leaving it to copy overnight via a pipe transfer, and *certainly* not for saving a measly 63MiB of drive space. If you're really set on listing two thresholds, at least pad 'em a bit - say, 140GB and 160GB. We *know* those are both safe.

MC

[mrtickle]

I still say it needs to be more hard-hitting rather than implying it's rare and doesn't affect many people. It doesn't matter how rare the GSOD is - when it happens it's fatal for people who can't create the "emergency" swap space (and they shouldn't be put in such a situation).

Far better to say bluntly: "whatever you do, do NOT add a big drive as a B drive if it will take your total above <the number you decide to use>". And then repeat the warning in the section which tells you how to add a B drive.

[DCIFRTHS]

Originally posted by mrtickle
I still say it needs to be more hard-hitting rather than implying it's rare and doesn't affect many people. It doesn't matter how rare the GSOD is - when it happens it's fatal for people who can't create the "emergency" swap space (and they shouldn't be put in such a situation).

Far better to say bluntly: "whatever you do, do NOT add a big drive as a B drive if it will take your total above <the number you decide to use>". And then repeat the warning in the section which tells you how to add a B drive.

I have a question: What you describe does not apply to a person that replaces that A and B drive with a lrger swap, does it?

[Merle Corey]

Originally posted by DCIFRTHS
What you describe does not apply to a person that replaces that A and B drive with a lrger swap, does it?
No, it doesn't apply when swap has been expanded - we're referring to situations where people keep the factory A drive (or otherwise don't expand swap) and add a large B drive.

MC

[R. Kalia]

After posting (in another context, elsewhere) that I had not seen a GSOD, I immediately got a GSOD.

The instructions in the 3rd message in this thread got me back up very quickly---many thanks!

Some idle newbie questions:

1) why is it that a disk w/o swap can have swap added, but a disk w/64mb swap cannot have swap permanently expanded?

2) What does the pdisk command "r 5 9" do and why is it needed only if the inactive partition is 4? Looking at the partition list I could see that "r 4 8" switches the functions of partitions 4 and 8, but "r 5 9" seemed to do nothing at all.

3) Once I put in the temp swap, TiVo healed itself within a few minutes. Does it actually dial in, or does it use files stored somewhere else on the disk?

[Robert S]

The commands you used are given as the default, the ones with /mad/ are added in parentheses with the words 'if you're using TiVoMad'. Would you like to suggest a clearer form of words for me? The TiVoMad disk doesn't need the magic incantation to boot byteswapping, but the commands are in a different place to MFS Tools.

The 'disks without swap' have an empty 128Mb partition where MFS Tools failed to create a valid swap partition. The fixes in the top post simply format this partition correctly. If your disk is already full, there's no where to put a bigger swap partition.

pdisk's r command renumbers partitions. The way it does it is a bit clunky r 7 8 swaps partitions 7 and 8 nicely, but you need two commands to swap 4 and 8 (thanks to angra for pointing that out). Although partition 4 ends up as number 8, the numbers of some of the other partitions are displaced.

I believe TiVo will dial and collect copies of files (presumably the system animations) it finds corrupt. Presumably your anims were OK and mfsfix just had to fix a small problem. Some people have reported green screens taking hours to recover.

You did back out to revert to 64Mb of swap, didn't you. (I'm worried I've created a new type of Moron here, although we've obviously not had a documented case yet).

[R. Kalia]

Originally posted by Robert S
The commands you used are given as the default, the ones with /mad/ are added in parentheses with the words 'if you're using TiVoMad'. Would you like to suggest a clearer form of words for me?
I was babbling. I confused Tivomad and MFS Tools.


The 'disks without swap' have an empty 128Mb partition where MFS Tools failed to create a valid swap partition. The fixes in the top post simply format this partition correctly. If your disk is already full, there's no where to put a bigger swap partition.
In Windows, programs such as PartitionMagic will resize existing partitions and insert new ones and so forth, without losing data. I guess there's nothing like that in Unix; too bad.


You did back out to revert to 64Mb of swap, didn't you. (I'm worried I've created a new type of Moron here, although we've obviously not had a documented case yet).
Huh? Bad idea. Who knows, I might get a GSOD again. I'm leaving it the way it is. Partition 4 hasn't been used since 10/2001 so I am sure it isn't needed for anything!


:)

[Robert S]

There may be Unix tools that would let you resize Ext2 partitions, but the recordings are stored in a proprietary filing system called MFS. Although MFS Tools can create MFS partitions, it can't resize them.

I have created an new class of Moron! I wonder if they'll give me credit for it in the FAQ?

You need the inactive root for the next software update. I don't know precisely what will happed when the next software update comes in, but it'll be really, really nasty - your TiVo will attempt to reformat and install the new software on the partition you're currently using for swap!

[R. Kalia]

Originally posted by Robert S
I have created an new class of Moron!
You mean, the class that is humor-impaired even when smilies are used?

[Robert S]

Oops!

[R. Kalia]

My apologies if this has been answered before in this very long thread or elsewhere...

To fix my swap problem, can I "upgrade" my 120G A drive to another 120G drive (this one properly prepared), while keeping the recordings? Or will it not work because the new drive has less free space because of the bigger swap space? (BY current drive is NOT anywhere near full with recordings.)

Obviously, I have no idea how the relevant steps actually work.

If my proposed method works I can then re-use the old A drive to replace my current smaller B drive, so buying a second 120G drive is worth it.

[Robert S]

No, there's no way to resize the MFS partitions, even if you're copying them to new media. You would be able to do this with a 160Gb drive, but they're much more expensive.

Try not to worry about it - green screens are very rare and the 'rescue' maneuver at the top of this thread will recover you if you do ever see one.

[Blademan007]

I've read through this thread, and I understand that I'll going to need to to worry about swap. I just upgraded a DTivo with a 160GB Maxtor (137GB recognized), which then promptly started to fail. Addicted to Tivo in 1 months as I am :cool: , I promptly picked up a 120GB WD 7200rpm drive (May as well max the ******* out since I love it :D ), while waiting for the replacement Maxtor. The DTivo has 32MB and I'm entering dangerous territory with 2 drives.

So if I'm going from old 160GB Maxtor (which sporadically works for shorts periods after refrigeration) to new 120GB A + 160GB B drives in a DirecTivo, which is the ideal mfsrestore command I should use to ensure adequate swap and minimize chances of GSOD?

(assuming: mfsbackup -aqo - /dev/hdc | mfsrestore -xpi - /dev/hda /dev/hdb )

Thanks for the bandwidth, and great stuff you guys have going on here!!!

[Robert S]

Are you trying to take your recordings with you? I'd wouldn't have thought you'd have anything worth taking in a month. It'll take many hours (perhaps more than a day!) to copy your recordings.

To take your recordings do

mfsbackup -Tao - /dev/hdc | mfsrestore -s 127 -xpi - /dev/hda /dev/hdb

(you would want the 160 to be the A drive to make room for the extra swap)

To just take your guide data, but not recordings (will take maybe 15 minutes)

mfsbackup -so - /dev/hdc | mfsrestore -s 127 -xpi - /dev/hda /dev/hdb

(Makes no difference which new drive is the A drive).

[Blademan007]

Robert,

Thanks for the response, and yes I do have 113hrs (per mfsbackup of recordings) that I would like to take to the new drive. I need to save to vcr...

I would prefer the 120GB 7200rpm drive as my A drive, since I notice the guide and other operations are much faster, and the 160GB as the B drive. Will the same command work for that configuration and still allow for enough swap?

Old 160GB --> New 120GB 7200rpm A + 160GB B.

[Robert S]

Possibly - MFS Tools does have some flexibility when it comes to laying out the MFS partitions, although I don't know if it can put A drive partitions on a B drive if it can't fit them on the A drive. You'll have to experiment a bit (you'll immediately get an error message if there's not enough room).

You don't need to be too worried about swap. Although I advise everyone to increase it if they can, green screens are very rare and the 'rescue' maneuver from the top of this thread will recover you from a green screen if you do get one.

Perhaps just print out the relevent post and tape it inside the TiVo incase you ever need it?

[BobCamp1]

Just wanted to let everyone know that the latest Hindsdale upgrade guide (dated Sept. 29) for MFSTools 2.0 addresses all the swap file issues covered in this thread. It now has notes warning people about the potential problem, and advises them on the correct upgrade procedure should they wish to avoid the problem.

Many thanks to Hindsdale, Robert S., Merle Corey, and everyone else for investigating and resolving the issues.

Now stop bugging these people so we can end this thread. :D

[jimjamsgolf]

Thanks everyone.

With hindsight I have left out some information but I have success.

I am not sure I had bad blocks on the disk as such as I think the drive could read then given enough attempts.

My hda drive was not an original but an upgraded one with 127/128Mb of swap, I'm too paranoid to try to reuse the original.

What I did was;

used the dd method to copy the data on hard drive suspected as being faulty to another drive (while I went to bed). Next I backuped and then restored all of the data onto the hda drive (which had/has 127 or 128Mb swap) and finally added my larger drive as hdb.

It was suggested that I could use fdisk/pdisk to play with the partitions on the hdb drive, does anyone know if fdisk/pdisk actually works on any boot cd? I think I used the turbonet boot/nic_install CD but I think I thought both pdisk and fdisk failed as the could not read the partiton table. When the kernel booted all the harddrive partitons were detailed correctly.

As an aside has anyone got a recent kernel to read the Tivo/Mac partiton table after applying the partition patch? I know it does not apply cleanly, I get the correct magic nuber but then no partition information.


Thanks

[comfysofa]

Right - i understood none of that lot - but i think my requirements are simple.......ive already done the large a drive upgrade (160gig) and want to put in a 120 gig as a B drive - what commands to i put in ----( i already know the one to stretch the drive over to the new b drive) but confused to the swap file bit???? + i ve already done the network bit if thats a hinderance or help???

Anyone got a walkthrough.....

regards

Andy

PS looking forward to a reply cos the 120 gig disc got delivered today and im itiching to get it installed.....

[Robert S]

Well, if your A drive is already a 160 and you don't want to lose your recordings, there's nothing you can do about swap.
If you increased your swap with -s 127 when you upgraded, then you've already got enough swap for a 257Gb TiVo to recover from a green screen (it doesn't make any difference for any other function). If you didn't, there's nothing you can do about it now - just use the 'rescue' maneuver if you do ever green screen.

The network stuff is irrelevent.

Put your A and new B drive on the primary bus, boot MFS Tools 2.0 and do

mfsadd -x -r 4 /dev/hda /dev/hdb

and you'll have a 257Gb TiVo.