Fixes for MFSTools 2.0 swap problems

[Robert S]

You have probably heard MFS Tools 2.0 has some problems associated with it. We now believe we have solutions for those problems.

The core problem is swap space. To get a safe upgrade with MFS Tools 2.0, make sure you use include the option '-s 127' on your restore - do not go higher than 127 or you'll get no swap at all! Avoid dd + mfsadd profiles if possible (useful for saving recordings), a pipe backup|restore with -s 127 is actually safer.

If you upgraded your TiVo with MFS Tools 2.0, you can use the backdoor codes (http://www.tivocommunity.com/tivo-vb/showthread.php?threadid=26530) to check your kernel log for swap error messages. See below for a detailed explanation of how to do this by Cpen.


The MFS Tools 2.0 documentation claims that 64Mb is enough for any TiVo to recover from a 'green screen' filesystem corruption error. This is not true. If you go over about 140Gb total storage space, TiVo will need more than 64Mb of swap to recover from a green screen, 127Mb appears to be enough for a 274Gb (largest possible capacity) TiVo to recover.

If your TiVo gets stuck in green screen - reboot loop, you may be able to save it by temporarily using your inactive root partition as emergency swap.

See the next post from me below for detailed instructions on how to do this.

If you are upgrading the A drive on a twin drive TiVo and keeping recordings, you may not be able to use MFS Tools 2.0 to increase swap. See the next post from me for details of how to manually add a swap partition.


If you used MFS Tools with -s 128, and therefore got no swap at all, below are some methods that you can use to activate your swap without reinstalling. Choose the one that suits your circumstances best:

The Linux kernel on the TiVo can only use the first 127Mb of a swap partition, even if you've used -s 512 to try and get 512Mb swap.

Ways to fix the problem

There are several ways to fix this problem, pick the one that works best for you.

1. Use shell access to TiVo to run mkswap directly.
If you can get a shell on your TiVo, this is the easiest general solution.

2. Use dd to duplicate existing swap partition
Simple method, but doesn't increase size of swap available, so doesn't fix green screen issue. Will get you working fast.

3. Use copy of mkswap on tools CD
Must have bootable CD.

4. Modify boot scripts to make TiVo run mkswap itself
Quite tricky, especially if you don't know Linux, but will work when other methods are unavailable. Won't work on DTiVoes or Series|2.


For the remainder of this guide we shall assume that your new TiVo drive is on Secondary Master and your old TiVo drive (if required) is on Primary Slave.

Some of these methods require access to the TiVo partitions and/or filing systems. You must use a byteswapping boot disk to do this. Byteswapping mode on the MFS Tools 2.0 boot disk
is broken. You can make it work by typing vmlnodma hdb=bswap hdc=bswap hdd=bswap at the boot prompt. You may prefer to use a different TiVo boot floppy or CD instead.

1. Use shell access to TiVo to run mkswap directly.

Use the shell to type

mkswap /dev/hda8
swapon -a

2. Use dd to duplicate existing swap partition

From TiVo boot disk do

dd if=/dev/hdb8 of=/dev/hdc8

This will only take a few seconds.

This will work even if you used -s to get a larger swap partition, but it will only give you 64Mb of swap, which will be enough to get your TiVo working properly.

3. Use copy of mkswap on tools CD

Boot a TiVo boot CD (but not the MFS Tools 2.0 CD) and do

/sbin/mkswap -v0 /dev/hdc8

Thanks to Merle Corey for getting this working. This is now a verified method.

(It has been suggested that under certain circumstances

/sbin/mkswap -v0 /dev/hdc8 130048

may be required to get working swap).

Options 2 and 3 are probably the only options for DTiVos as you can't get a shell or modify boot scripts on those.

4. Modify boot scripts to make TiVo run mkswap itself

[This is a condensed version of the original description (http://tivocommunity.com/tivo-vb/showthread.php?s=&threadid=66256&pagenumber=2).]

You will need both TiVoMad and Dylan's Boot disk.

On boot TiVo (and indeed all Unix systems) runs a script called /etc/rc.d/rc.sysinit (this is broadly equivalent to C:\AUTOEXEC.BAT on a DOS/Windows system). If we add our mkswap command to this file, the swap file will be initialized when the TiVo boots.

The file we want is in the active root partition.

Boot TiVoMad, use Ctrl-C to break out of the upgrade script.

Do

/mad/edit_bootparms hdc -r

It will reply 'root=/dev/hda4' (or hda7)

Boot Dylan's boot disk. Mount active root partition

mount /dev/hdc7 /mnt

or

mount /dev/hdc4 /mnt

edit sysinit

joe /mnt/etc/rc.d/rc.sysinit

add the following line on a line on its own somewhere near the top.

mkswap /dev/hda8

Save (Ctrl-K then X).
Unmount

umount /mnt

Power down, boot TiVo, check logs, return drive to PC, boot TiVoMad's utility disk (Ctrl-C to get command prompt)

Traverse to rc directory

cd /mnt/etc/rc.d

delete modified sysinit

rm rc.sysinit

replace original version from joe backup

mv rc.sysinit~ rc.sysinit

unmount partition

cd /
umount /mnt

Power down and replace drive in TiVo.


Most of the credit for the fixes in this thread goes to the contributors below, especially Merle Corey for his work on characterising the mfsfix/green screen problem and identifying -s 127 as the solution.

[mtg101]

I just need to clarify something for my upcoming upgrade...

I'm planning to take my single 40GB drive system and add a second drive to it (120GB). If I use MFSTools 2.0 and leave the swap space alone, I should be OK - right?

[Robert S]

If you have more than 140Gb of hard drive space in your TiVo, the original 64Mb of swap space does not provide enough memory for the filesystem checker (mfsfix) to repair corruption to the filesystem. If you get corruption, your screen will turn green as mfsfix runs, but rather than fixing the problem, your TiVo will just reboot endlessly. If you get this problem, this post explains how to fix it.

If you are upgrading the A drive on a twin drive TiVo, see the end of this post for important details.

(Note that mfsfix can't fix everything, so you might get stuck like this for other reasons. If you know you increased swap, or you know your TiVo is smaller than 140Gb, these instructions won't help.)

The Plan

TiVo has two partitions for storing the system software, these are referred to as 'root partitions'. Only one is active at a time. The other is used when a software upgrade comes in. The inactive root partition is formatted and the new software load copied on to it. The boot block is then rewritten and the TiVo reboots to the new active partition.

Most of the time therefore, the inactive root partition is unused. We will format it as a swap partition and recofigure the TiVo to use it instead of the normal swap partition. Once TiVo has repaired itself, we will return things to the way they were to leave the inactive root partition available for the next software upgrade.

Do not forget to reverse the partition changes once your TiVo is working again! Terrible things will happen at the next software upgrade if the partitions are not reset!

We will:


Boot so we can read the TiVo partitions
Identify the inactive root partition
Prepare the inactive partition as a swap partiton
Renumber the partitions
Allow the TiVo to repair itself
Restore the original partition numbers


A. Booting so we can read the TiVo disk

Connect your TiVo's A drive as primary master.

TiVo disks can not be read by a PC because the bytes are written in a different order to that expected by the PC's processor. TiVo boot disks are patched to boot in a 'byteswapped' mode that allows the TiVo data to be read.

Most people these days seem to be using the MFS Tools 2.0 CD. MFS Tools 2.0 has internal support for byteswapping so that CD boots in non-byteswapped mode by default. It offers you the option to boot byteswapped, but this doesn't work. To boot the MFS Tools 2.0 CD byteswapped at the 'boot:' prompt you must type

vmlnodma hda=bswap

If you can't get this to work, try Kazymyr's boot CD or TiVoMad's boot CD. If use a different boot disk, hda will be left unswapped. Use one of the other connectors and adjust the commands below accordingly. (We don't currently know how to do this from a floppy, one of the tools we need is only on the CDs).

Series|2 TiVoes are NOT byteswapped. You should be able to boot the MFS Tools 2.0 as normal.

B. Identifying the Inactive Root Partition

The easiest way is to read the boot block.

edit_bootparms hda -r

(If you're using TiVoMad, the command is:
/mad/edit_bootparms hda -r )

It will reply with the name of the active root partition, which will be either /dev/hda4 or /dev/hda7 - the inactive root partition will be the other one.

If edit_bootparms is unavailable, you can try to mount the root partitions:

mount /dev/hda4 /mnt or mount /dev/hda7 /mnt

If you get an error saying 'Must specify filesystem type', you're trying to mount an empty partition, if the prompt comes back with no message, the filesystem has mounted.

You need to confirm that one root partition mounts and the other doesn't. You unmount partitions with:

umount /mnt (note umount, not unmount).

If both partitions mount, look at the dates on the files with:

ls -l /mnt

The active partition will have newer files.

If neither partition mounts, you haven't booted correctly.

C. Prepare the inactive partition as a swap partition.

Do

mkswap –v0 /dev/hda4

or

mkswap –v0 /dev/hda7

D. Renumber the partitions.

pdisk /dev/hda

On TiVoMad, pdisk is /mad/pdiska.

If your inactive root partition is 4, type:

r 8 4
r 5 9

If your inactive root partition is 7, type:

r 8 7

w (write partition table)
y (confirm, returns you to command prompt)

E. Restore drive to TiVo

Let the TiVo run, it may take a long time, and may reboot occasionally, but it should eventually boot up properly. Remember, you must complete this sequence, your TiVo is not safe when it recovers from the green screen.

F. Restore the original partition numbers

Repeat step D above.

You can now safely replace the drive in the TiVo.

Notes

If you want to backup the partition you're about to take over, boot in a byteswapped mode, with a FAT drive attached (the way backups were done before MFS T2), mount the FAT parition as in Old Hinsdale and do:

dd if=/dev/hdX4 of=/mnt/dos/tivopart.bak

Similarly, you can backup the partition table with

dd if=/dev/hdX1 of=/mnt/dos/tivotbl.bak


This has been tested and verified on all types of TiVo.


Thanks to gigageek for working through this properly to document it and proving it works on DTiVoes. See page 10 of this thread for his write-ups, which include so details I've omitted. Thanks also to dougdmy for being the first to try the emergency swap idea and reporting that it works.




Upgrading Twin-drive TiVoes

There is one case that isn't covered by all the previous discussion: What do you do if you've got a TiVo with the original A drive and a large B drive? If you're on a twin with the original drives, you can just pipe both drives onto the new drive, but you're stuck if the old B drive is to be retained.

You can upgrade by using dd to copy the A drive and then using mfsadd to expand it. This gets you a working TiVo, but doesn't expand your swap. You can just wait for it to green screen (may well never happen) and use the rescue procedure above, but if this is the sort of thing that would worry you, you can manually add a swap partition.

The procedure is very similar to the rescue. Start by dd'ing the A drive on to the new drive. Then reboot into a byteswapping mode (you'll probably want to disconnect the old A drive) as above.

Run pdisk and create a new partition:

pdisk /dev/hda

C
12p
128m

You'll also be asked for the partition label and partition type. "Linux Swap" and "Swap", WITH the quotes, are the 'official' answers, but TiVo doesn't check this detail.

Swap the partition labels around

r 12 8
r 9 13

The TiVoes currently on sale have 13 partitions on their A drives. The commands will be different - C 14p 128m <Return> r 14 8 <Ret> r 9 15 <Ret>.

If pdisk complains 'invalid partition' when you try to create the new partition, see post #7 (http://www.tivocommunity.com/tivo-vb/showthread.php?postid=629559#post629559) for a laborious work-around.

Write and exit:

w
y
q

Then prepare your new swap partiton

mkswap -v0 /dev/hda8

and you can now complete the upgrade with mfsadd as normal:

mfsadd -x /dev/hdX /dev/hdY

You should now find your TiVo boots to give you extra space and report a 130 million byte swap partition in the logs.

Thanks to angra for working through this one.

[mtg101]

Originally posted by Robert S
Yes, swap file issues only affect changes to the A drive, but this is not the same as having a perfect upgrade.

You will be vulnerable to mfsfix problems however you do it,


Thanks for your help. However...

You've lost me there I'm afraid - what's mfsfix? I assume it's a daemon that monitors the file system and fixes it?

And MFSTools 2.0 will stop that from running will it? Is there way to avoid MFSTools 2.0 stopping mfsfix, or will I have to use the older tools to avoid this problem?

[Robert S]

If you're wondering what this 'swap' that everyone's going on about in this thread is, here's a quick explanation. This post was originally a reply to mtg101's question, but that answer is out of date now anyway.

Virtual memory is a technique used by most modern operating systems including Windows, Windows NT and all versions of Unix including the Linux variant used by TiVo. Basically it allows you to substitute cheap (but slow) hard drive space for expensive (but fast) DRAM by tracking which parts of DRAM are currently in use and copying the unused bits to the disk, allowing that space to be used for something else. When the bit that was copied out needs to be used again, it's copied back into DRAM, possibly causing something else to be copied out to disk.

The most common VM system is called 'paging' where blocks of memory (typically 4kb each) are considered individually. This requires fairly sophisticated hardware so earlier systems used a 'swapping' model where entire processes would be copied to disk to enable other processes to run. All modern systems page rather than swap, but the term 'swapping' is used where 'paging' would be more correct.

The other salient point is exactly how the stuff being swapped out it handled. Windows, for example uses an ordinary file. Yours is called c:\windows\win386.swp if you're running Windows 9x with the default settings. Unfortunately there's a lot of housekeeping associated with accessing a file in a filing system (updating dates and file sizes etc) that isn't really useful for a swap file. So Linux also supports swap partitions, which give the VM module direct access to a linear chunk of drive space without having to go through a filing system.


Series 1 stand alone TiVoes ship with 16Mb of DRAM and a 64Mb swap partition, all other TiVoes have 32Mb or RAM. The 'green screen of death' is a filesystem repair utility (broadly equivalent to ScanDisk) called mfsfix. mfsfix requires 1/2Mb of memory for every (binary) gigabyte of disk space that it needs to check. Therefore as shipped, Series 1 stand alones have enough swap for mfsfix to complete with 150Gb of disk space, other TiVoes have enough for 180Gb. It takes about 4 minutes for the TiVo to realise that it doesn't have enough swap for mfsfix and at that point the TiVo reboots.

If you run mfsrestore with -s 127 you'll get 127Mb of swap, which is enough for at least 274Gb of disk space, which is the most the original kernel can handle. If you use Todd Miller's (http://tivocommunity.com/tivo-vb/showthread.php?s=&threadid=83342) patched kernel (ftp://ftp.courtesan.com/pub/millert/TiVo/) you can go well beyond that. This will require more swap, which is possible, but not straight-forward

[ADent]

If you use the older tools, when you run tivomad you can tell it to build a bigger swap parition (it asks if the total drive space is going to be over 140GB).

If I am not mistaken the old method (MFSTOOLS 1.x & TiVoMad) has been proven to work up to 256GiB (or at least 240GB).

S2 users of course are limited to just adding a second drive with MFSTools 1.x and you can not reupgrade a unit w/o losing all of the recordings.

There were arguments that a S2 with an added 120GB drive might be OK since they ship with more memory.

[Robert S]

Very rarely when you try to use pdisk to create a swap partition manually or try to use mfsadd to expand your TiVo image, you'll get an error suggesting that the disk is full. The solution to this is to use pdisk to rebuild the partition table. Doing this will make the partition table represent the full size of the new disk and will allow pdisk or mfsadd to work.

Normally both pdisk and mfsadd will recognise that the drive is larger than the partition table indicates and this process will be unnecessary.

What you need to do is print out the drive's partition table and then use pdisk to create a blank partition table and then reenter all the partitions with exactly the same parameters as before. The reinitialised disk will show the full capacity of the drive, so the Apple_Free partition at the end will be much larger than it was before.


For a Series 1 TiVo you will need to boot in byteswapping mode to get into the partition table. For Series 2 TiVoes, the MFS Tools 2.0 CD will boot in the correct mode anyway.

We're using Kazymyr's boot CD with the hard drive on primary slave. If you force byteswapping mode (see the first post of this thread), the MFS Tools 2.0 disk should work just as well.

Hit the enter key when you see Boot: This will boot Kazymyr's CD in default mode.

You will be prompted to login as "Root" Type root and hit enter.

If you want want to make a backup of your partition table so you can recover if things go wrong, proceed as follows:

You'll need a mountable partition to write the backup files to. This is the same process you use to mount a partition for writing an MFS Tools backup during an upgrade.

At the linux shell prompt (/#) type mount /dev/hda1 /mnt

You can make a text file by typing pdisk -l /dev/hdb > pdisk.txt. If you go back to Windows, this file will be C:\pdisk.txt. You can use WordPad to print it out if you wish (Notepad will be confused by the Unix style line terminations).

You can make a backup of the partition table: dd if=/dev/hdb of=/mnt/tivopart.raw count=32. If you dd this file back onto the drive it will restore the partition table to its original condition.

Disconnect from the partition with umount /mnt

This completes the optional backup phase.


At the next linux shell prompt type pdisk /dev/hdb. This gets you into pdisk which is the program used to change your partitions. It will say: Edit /dev/hdb and take you to a Command (? for help):

Type p and hit Enter. This will show you the partition table for your drive. WRITE THIS DOWN EXACTLY AS IT APPEARS (At least make sure you have all the numbers correct). If you're working on an A drive, you may have 13 partitions to deal with. If you're working on a B drive, there may be as few as three.

At the command:, give an i (Enter) command to initialize the partition table. You will have to confirm that you want to do this by typing y (Enter)

Next, give a C (capital C) command which will allow you to enter the information you wrote down previously.

Partition 1 is the partition table. pdisk has already set that up for you, so you start with partition 2.

First block: (This number will be the number under "base" for partition 2)

Length in blocks: (This number will be the number under "length" for partition 2)

Name: For an A drive this will be "Bootstrap 1", for a B drive, this will be "New MFS Application". You need to type the quotes.

Type: "Image" (A drive) or "MFS" (B drive)

Do a p command to check the partition you just added, make sure it matches what you wrote down.

Repeat the C command and enter the information for partiton the remaining partitions. The last partition - Apple_Free/Extra is just a placeholder for the unallocated space on the drive, you don't create it yourself.

Do a p command to check the partitions again. The table should look just like what you wrote down. The size of the Apple_Free partition should be much bigger - the difference between the old and new drives, infact. pdisk lists the partition sizes in 512-byte blocks, so if you divide by 2 you get the size of the partition in kilobytes.

After confirming that your newly created partitions look exactly like what you wrote down, do a w command. This writes your new partitions into the file. You will have to confirm that you want to do this step by typing y and hitting enter.

You will need to quit pdisk and get back to a linux shell prompt by typing q and hitting enter.

At the next linux shell prompt, do a pdisk -l /dev/hdb command. That is a lower case "L", not a one. You should find that the partition table looks as it did, except for the enlarged Extra partition.

At this point the partition table reflects the true size of the disk. You should find that pdisk now allows you to create your new partition or that mfsadd expands the image as expected.


Thanks to mutant for suggesting the rebuild as a work-around for the problem with creating a new swap partition and outlining the procedure (http://www.tivocommunity.com/tivo-vb/showthread.php?postid=1781967#post1781967), to someone whose nick I've forgotten for suggesting mutant's rebuild would work for the mfsadd problem too and to at4tivo for slogging through the rebuild and writing it up in enough detail to be usable by people who've never used pdisk before.

[Cpen]

I upgraded my TiVo over the weekend and just checked the (kernel) log files using backdoors.

It read:

"activating swap partitions"
"adding swap:65532 swap-space (priority-1)"

Wohoo! Looks good!

Those that are having problems have reported the following message:

"activating swap space"
"unable to find swap-space signature"
"swapon: Invalid argument"

I used MFS Tools 2.0 and Hinsdale's "Upgrade Configuration #6" to take my A & B drive from my Phillips 312 and move/expand them (and my stored recordings/content - 3 hours worth recorded on "high") to a brand new (unused) 120 gig hard drive. This debunks an earlier theory by Hi8 stating that "it appears that the problems appear when trying to copy a FULL TiVo".

From what I've seen so far, it appears that anyone that attempted to use the -s 128 to increase their swap size is a good candidate for this swap problem.

I used the command (from Hisdale's upgrade configuration #6): mfsbackup -Tao - /dev/hda /dev/hdb | mfsrestore -xzpi - /dev/hdc

My Hard Drive was a virgin 120MB 5400rpm Maxtor (no prior image on the drive). I followed Hinsdale's instructions, using the MFS Tools 2.0 to boot from a CD that I made with the ISO image (Edit: I was asked by Robert S if I booted into a swapping or non-swapping mode - to the tell the truth, I don't know - whatever Hindale's default instructions said to do, I followed).


Superman (Tiger) where are you!?

P

PS: To check your log file to see if your swap is working:

1. Reboot/Restart your TiVo (It's important to do this, as this is when the swap gets activated. The sucess or failure of the swap activation gets written to a log file that you can review - keep reading)

2. Enable back doors - The Backdoor mode can be entered using the remote. This is done by doing a "Browse By Name" or "Search by Title" or wherever you can get to the Ouija screen... Currently, the only easy way to exit backdoor mode is to reboot the Tivo. After entering this code, you will see "Backdoors Enabled!" appear briefly, and it will return to Tivo Central. You can verify that backdoors are on in the System Information screen.

1.3 systems: Enter "0V1T" and press Thumbs Up.

2.0 systems: Enter "2 0 TCD" and press Thumbs Up.

2.5 systems: Enter "B D 2 5" and press Thumbs Up.

3.0 systems: Enter "3 0 BC" and press Thumbs Up.

2.5.2 systems (DirecTivo Only): Enter "B M U S 1" and press Thumbs Up.

1.5.2 UK systems: Enter "10J0M" and press Thumbs Up.

3. With backdoors enabled press (on your remote) Clear-Enter-Clear-Thumbsup.

4. You're now in the TiVo Log files. Use the right arrow (the right side of the circular pad on your remote) to get to the "var/log/kernel" page

5. Use page up (channel up on your remote) to jump a page at a time and get to the beginning of the log file(you'll have to press channel up many times).

6. Now - Use page down (channel down on your remote) to review the lines of your log file page by page - the "activating swap" line should appear about 3-5 pages down.

7. To get out of the log file view, press the Tivo button or the left arrow on the circle pad.

8. To then disable the backdoors, reboot/restart your Tivo.

What did you find? What type of Tivo did you upgrade? How did you upgrade (configuration upgrade # from Hinsdale's guide)? What command did you use (did you use -s 128?)?

[Bill Reeves]

Originally posted by Cpen
From what I've seen so far, it appears that anyone that attempted to use the -s 128 to increase their swap size is a good candidate for this swap problem.



Another data point to support this theory. I performed my upgrade last week with the following command:

mfsbackup -Tao - /dev/hdc | mfsrestore -s 128 -xzpi - /dev/hda /dev/hdb

(from the Hinsdale How-To guide, step 10, Upgrade Configuration #3, from a Single-Drive TiVo to New A and New B Drives)

and I just checked my kernel log file and sure enough, the "bad" messages were there.

-- Bill

[mtg101]

Originally posted by Robert S
AFAIK mfsfix just fixes file system problems. There has been no word of the scope of this problem or possible fixes, but a few MFS Tools 2.0 have been stuck in a green screen (mfsfix running) -> reboot (mfsfix crashed) loop, suggesting it is real (presumably these cases have faulty backups or faulty drives leading to problems that mfsfix tries to fix).

The only totally safe option at the moment is to use older tools, but there if you go over about 140Gb mfsfix runs out of swap and crashes so you can't safely bless your 120Gb drive and add it to your A drive without increasing swap.

Thanks for the info. As it's my first upgrade I'll stick to the old tools I think. And I'll leave the original drive alone. Just backup and put on new 120GB drive and expand that. Thus I'll always have the original to do upgrades from in the future. That all sounds about as safe as it's getting to me :)

[hinsdale]

Originally posted by Bill Reeves


Another data point to support this theory. I performed my upgrade last week with the following command:

mfsbackup -Tao - /dev/hdc | mfsrestore -s 128 -xzpi - /dev/hda /dev/hdb

(from the Hinsdale How-To guide, step 10, Upgrade Configuration #3, from a Single-Drive TiVo to New A and New B Drives)

and I just checked my kernel log file and sure enough, the "bad" messages were there.

-- Bill

Realizing the problems with the -s swap file creation in Mfs 2.0, I thought I removed all instances of the -s 128 in the 2.0 How-To several weeks ago. Apparently I missed one, and have removed it. The problem with this parameter was that it not only did not increase the swap to 128, it actually resulted in error with no swapfile.

Having witnessed hundreds of upgraders adding large B drives (since the new 120-160 drives were released at end of last year) without increasing swap and no subsequent pattern of reported unrecoverable GSODS, and also having personally experienced several >140GB machines (standalone and directivo) with unmodified swap file recover from GSOD, I would not be overly concerned about swapfile creation larger than 64MB. Although larger swap certainly doesn hurt, I am not convinced that the swap file limitations for mfsfix to function exist any longer and may have been addressed during TiVo software upgrades above 2.0.

[muchmore44]

Hinsdale,

Do you mean that you feel that using Mfs 2.0 and the most recently updated instructions will successfuly allow upgrades WITHOUT any problems related to the swapfile? Sorry if this is a lame question, but befor I complete a TIVO upgrade, I want to be sure.

Thanks,
Muchmore44

Originally posted by hinsdale


Realizing the problems with the -s swap file creation in Mfs 2.0, I thought I removed all instances of the -s 128 in the 2.0 How-To several weeks ago. Apparently I missed one, and have removed it. The problem with this parameter was that it not only did not increase the swap to 128, it actually resulted in error with no swapfile.

Having witnessed hundreds of upgraders adding large B drives (since the new 120-160 drives were released at end of last year) without increasing swap and no subsequent pattern of reported unrecoverable GSODS, and also having personally experienced several >140GB machines (standalone and directivo) with unmodified swap file recover from GSOD, I would not be overly concerned about swapfile creation larger than 64MB. Although larger swap certainly doesn hurt, I am not convinced that the swap file limitations for mfsfix to function exist any longer and may have been addressed during TiVo software upgrades above 2.0.

[Robert S]

Please do the upgrade and let us know what happens. We need people to try this and report so that we have a good basis to answer questions like this from. At the absolute worst you'll end up having to redo the upgrade with the older tools, which doesn't take very long.

My prediction is that if you use MFS Tools 2.0 to restore to a brand new hard drive it will not give you a working swap partition no matter what options you choose.

[DCIFRTHS]

Originally posted by Robert S


My prediction is that if you use MFS Tools 2.0 to restore to a brand new hard drive it will not give you a working swap partition no matter what options you choose.

For example, do you mean two 120 GIG drives?

[Robert S]

Originally posted by DCIFRTHS

For example, do you mean two 120 GIG drives?


I don't think drive size has anything to do with it. Only the A drive has a swap partition, so it doesn't matter if you use one or two drives.

('Options' is a Unix-ism for command line switches (usually preceded by a - ), and I use it in that rather limited sense.)

People have been talking about whether it's the -s option that causes the problem (and earlier speculation on the role of -p and -z).

I have not seen a report where it can be unequivocably stated that MFS Tools 2.0 has correctly initialised a swap partition.

There have been cases where people have upgraded with MFST2 and got working swap, but so far in all those cases the swap was either copied from another TiVo drive or inherited from an earlier TiVo image on the same drive.

Cpen appeared to state that MFST2 had correctly initialised his swap, but he hasn't replied to my request for more details.

[Cpen]

Robert -

Let me know what else you're looking for and I'll post it here - I'm very interested in figuring out what combinations of hardware and upgrade configurations are leading to problems.

Note - I made changes to my earlier post to clarify where you had questions.

P

[Robert S]

I posted the following on the MFS 2.0 Swap Problem?? thread where you first mentioned your upgrade, sorry if you missed it:

Can you tell us more precisely what you did, specifically:

Was the target disk completely blank or did it have an earlier image on it already?

Did you boot into a swapping or non-swapping mode?

Ideally you'd write up your whole upgrade process and they we'd see if it worked for other people too. In the mean time, the two questions address two specific possibilities:
The swap partition may have been inherited from an earlier install (restoring with MFST1.1 and then restoring over that with MFST2 does seem a valid (if slightly boneheaded) way of getting a working swap partition.

The swap bug may be caused by byte-swapping. Most people do their MFST2 upgrades unswapped because that's the default, perhaps the swap initialiser isn't swap aware (and yes, the turn swap does mean several different things in that sentence!)

[Robert S]

Originally posted by Cpen
Note - I made changes to my earlier post to clarify where you had questions.

I see now, too subtle for me, I'm afraid. Sorry about that. Well, I'm completely stumped, I have seen posts from people who appear to have done exactly and got no swap.

[mtg101]

As mentioned above, I'm planning on replacing my drive with a single 120GB A drive. Would it be useful to people if I tried it first with MFSTools 2.0 to see if I get the swap problem?

[lemketron]

Originally posted by Robert S
...I have not seen a report where it can be unequivocably stated that MFS Tools 2.0 has correctly initialised a swap partition.

There have been cases where people have upgraded with MFST2 and got working swap, but so far in all those cases the swap was either copied from another TiVo drive or inherited from an earlier TiVo image on the same drive.

Cpen appeared to state that MFST2 had correctly initialised his swap, but he hasn't replied to my request for more details.

That may well be true. Tonight we re-did the DirecTiVo that I upgrade the other night. The first time we did backup to file, then restore with -s 128 and the swap was bad. Tonight we did another restore without the -s 128. However, since I lack a DSS dish, there was no way to get to the log so to eliminate another lost day of use we also did a "dd" copy of the swap partition from the original 30gig drive.

I heard back later that the machine is working fine now, and has a valid swap.

Sorry this doesn't help figure out whether MFST2 can restore a valid swap or not, but it does establish (for me anway) that -s 128 is bad.

Now if someone can try a backup | restore without the -s 128 to see if MFST2 can make a valid 64MB swap, that would be great.

One other note Robert: We also tried to boot MFST2 with the "swap" option (in order to try the /sbin/mkswap suggestion) but MFST2 wouldn't boot with "swap" on my system. Don't now if this is a known problem or not, but if refused to boot all the way.
--Steve

[Robert S]

Originally posted by mtg101
As mentioned above, I'm planning on replacing my drive with a single 120GB A drive. Would it be useful to people if I tried it first with MFSTools 2.0 to see if I get the swap problem?

Yes please! Start by trying to replicate Cpen's upgrade, if that works reliably I'll have to rewrite the top post in this thread.

To be a worthwhile test you must ensure that hdx8 doesn't contain a valid swap partition. Obviously the first time the disk will be blank, but on subsequent attempts, dd hdx9 onto hdx8 to make sure it's corrupt.

I'm sure you really just want to get your TiVo working, but if you want to try some variation, I would be interested to see if it makes a difference if MFS Tools runs in a byteswapping environment.

[BareMetal]

Now if someone can try a backup | restore without the -s 128 to see if MFST2 can make a valid 64MB swap, that would be great.

I've worked on three series 2 standalone systems. Two of them i used the -s 128 option on the third one I didn't. The first two have no swap, but the third one does.

[sdunne]

I've just moved from a single drive UK model Tivo (Thompson 6020 series) to dual 160Gb Maxtors. I followed the Hinsdale guide, in particular:
Step 7 Option 1) (but backing up to a linux partition) via "mfsbackup -l 32 -6so /mnt/rh73/TivoBackup /dev/hdc"
Step 8 via "mfsrestore -zpi /mnt/rh73/TivoBackup /dev/hdb
Step 10 Upgrade Configuration #3 via "mfsbackup -Tao /dev/hdc | mfsrestore -xzpi - /dev/hda /dev/hdb"

It all seems to have gone ok. System Info is reporting 94 hours 27 mins best quality (330 hours 19 mins in basic).

I checked my kernel logs and it looks like a new 64Mb swap partition was created, one message in there "Adding Swap 65532k swap-space (priority -1)" suggests.

It's since been rebooted without any apparent problems.

Fingers crossed, but leaving out the -s 128 seems to have worked out OK

Stephen

[klincoln]

On Thursday, I successfully upgraded a previously upgraded SA Sony SVR-2000 with MFSTools 2.0. This TiVo began life as a 30hr single drive unit running 1.3 with a locked Quantum 30GB drive. During the "1.3 era", BlessTiVo was used to add a 60GB Maxtor B drive and expand the unit to roughly 100 hours. It ran in this configuration, with OS upgrades along the way to 3.0, until I saw MFSTools 2.0 and 160GB Maxtor drives for $169 at a local computer show.

The upgrade scenario was Hinsdale's UPGRADE CONFIGURATION #5: Any Dual Drive TiVo to New A and New B Drive (replacing both drives), Option #1 MFSTools single step method using four IDE ports and the MFSTools 2.0 boot floppy. The existing drive pair were reasonably full, using 81GB out of an available 91GB. The two 160GB Maxtor upgrade drives were virgin, fresh out of the shrink wrap. No byteswapping was used. The anecdotal evidence so far seems to point towards the -s option being the root of the swap partition problems. Based on this, the -s option was omitted from the MFSTools command line. The following command was used:

mfsbackup -Tao - /dev/hda /dev/hdb | mfsrestore -xzpi - /dev/hdc /dev/hdd

81GB and 21 HOURS LATER (!), after successful completion of the backup/restore, the new 160GB A and B drives were installed in the TiVo. The unit booted and operated properly. Backdoor inspection of the kernel log confirms that a properly initialized 64MB swap partition is activated:

"activating swap partitions"
"adding swap:65532 swap-space (priority-1)"

Since Thursday, I've booted the unit about a half-dozen times, with the swap partition successfully activating each time. All TiVo functions work properly and reasonably promptly. All menus are properly populated and all migrated recordings play correctly. New recordings work fine at all compression levels. Recording capacities are as expected, ranging from 94.5 hours at best quality to 344.5 hours at basic quality. "Save Disk Space" (VBR) is disabled.

My experience would seem to lend support to the growing evidence that MFSTools 2.0 can create a working swap partition provided you don't try and grow the swap partition via use of the -s option (64MB swap partition correctly created on a virgin drive in this case with the -s option omitted). My gut feeling is that this unit would also successfully complete an mfsfix if it suffered file system corruption and a GSOD. I feel that those MFSTools 2.0 upgraded units that are suffering signal 11 exceptions during mfsfix also don't have a properly initialized swap partition because they used the -s option during their upgrades. If someone can supply more info on using mfsassert to trigger mfsfix, I'd be more than happy to test this hypothesis.

Kevin

[DCIFRTHS]

klincoln: Very nice report. It's easy to read with great attention to details. Welcome to the forum.

[jhburke]

I just spent the last few hours trying to fix the swap on my DirecTiVo 2.5.2 which I upgraded about 5 weeks ago using MFStools 2.0 and the 128M swap option (which of course resulted in an unusable swap partition). I have an old 2 drive Philips DTivo and I upgraded to 2 new Maxtor 120GB drives, preserving all the old recordings. I really haven't noticed any problems despite the lack of swap, but I thought I would try to fix it anyway to try to avoid GSOD...

1. I first tried using the MFStools 2.0 boot CD to run mkswap. The byteswapping option on boot is bad; you have to manually type in "vmlnodma hdb=bswap hdc=bswap hdd=bswap" at the boot prompt to use byteswapping. I ran mkswap /dev/hdc8 without errors ("131xxxK bytes"). I ran swapon /dev/hdc8 and checked /var/log/messages ("Adding swap-space 131064K (priority -1)"). However, when I put it back into the DTiVo, the swap was not used with the same errors.

2. Then I tried using the DTiVoMad boot CD (which does byteswapping by default). Same process, same results - swap partition works in the computer, but not back in the DTiVo.

3. Now I tried to copy the swap partition from my original TiVo A drive to the upgrade drive ("dd if=/dev/hdd8 of=/dev/hdc8"). I know this is only a 64M swap file copied to a 128M swap partition, but I figured "why not?!" since I couldn't think of anything else except to reupgrade and wipe out the last 5 weeks of recordings. It showed no errors, and swapon /dev/hdc8 gave a successful message in /var/log/messages (Adding swap-space 65332K (priority -1)). Now, back in the DTiVo and voila!, I now have a functioning 64M swap file, according to both /var/log/kernel and /var/log/messages via backdoors.

Now, what to think?

Will having a 64M swap file in a 128M swap partition cause problems down the road? So far, so good. I can't tell any difference with the speed of menus or Now Playing list, however.

Maybe MFStools 2.0 initializes the swap file correctly (I never tried to mount it from the computer before running mkswap the first time) but DTiVo cannot use a 128M swap file. This doesn't seem entirely true since in an older swap problem thread someone was able to initialize a 128M swap file by running mkswap from the rc.sysinit file in a stand alone TiVo.

Anyway, just posting my results. I'll let everyone know if I find any problems with my current fix.

John

[klincoln]

Thanks DCIFRTHS. I've been a long time lurker who has sponged volumes of valuable information from the generous folks around here each time I've gone thru one of my flurries of TiVo upgrade madness.

To add to yesterday's post, I supposed that an MFSTools 2.0 upgraded TiVo that had a confirmed working swap partition would subsequently complete an mfsfix/GSOD. I was seeking advice on mfsassert to test the theory. Well, I can now report that theory is ONE HUNDRED PERCENT WRONG! After yesterday's post, I put a bash shell up on the serial port via rc.sysinit.author, fired up tivosh, and executed "mfsassert -please" out of /tvbin. The shell console confirmed "MFS filesystem has been marked inconsistent" among the spew from several unhappy threads as the system rebooted...to the desired (?) GSOD. After roughly three or four minutes, it once again rebooted back to the GSOD. While watching this second reboot sequence I was already taking the side off of the PC, and rummaging for the previous A and B drives and my spare IDE cables. I allowed it to continue this reboot/GSOD cycle for roughly an hour before putting it out of its misery. Upon mounting the "dirty" A drive on the PC, the kernel logs confirm repeated signal 11 failures in mfsfix.

So, in my case, even though the swapfile was properly created, the MFS partitions/data were layed down by MFSTools 2.0 in some fashion that indeed gives mfsfix heartburn.

Deciding to copy my original 100 hour configuration back to the new pair of drives again with MFSTools 2.0 and keep recordings, rather than revert to MFSTools 1.1, I looked a little more closely at the MFSTools 2.0 boot floppy environment. My first (21 Hour!) backup/restore was done with byteswapping enabled/DMA disabled (duh)! My comment of yesterday regarding the original backup/restore being done without byteswapping was wrong. My limited Linux skills get lightly sharpened during my TiVo flurries, but then WAY too many sleep periods elapse between uses. After using "hdparm -d1 /dev/hdx" to enable DMA on all four drives, the 81GB backup/restore takes about 90 minutes! This new configuration also worked correctly in all operational respects. And, since this configuration really did result from a non-byteswapped backup/restore environment, I executed the mfsassert command again for the sake of posterity. Been there, done that, same story. The problem with MFSTools 2.0 MFS partitions and mfsfix is not influenced, in my case, by whether the backup/restore environment is byteswapped or not.

Knowing that I can restore it in less than two hours, I'll continue to run this MFSTools 2.0 config with migrated recordings while I wait for a solution to the mfsfix issue. I'll take my chances against a legitimate GSOD until then. Like most of us, I'd be nowhere without these superb tools that are created by people like Tiger. They've MADE my TiVo experience. I'm confident the problem will be corrected.

A TiVo weekend in the A/C from a blistering Washington, DC...

Thanks,
Kevin

[Robert S]

klincoln, if you're in the mood to experiment, would you like to try MFS Tools 2.0 with -r 0 (1 Mb block size)? Some people have expressed the hope that this might create a recoverable TiVo, but no-one's tested it.

[jhburke]

I had an additional thought about the bad swap files:

I never used TiVoMad to upgrade before, but I understand there is an option to expand the swap file when the combined hard drives are greater than 140 GB to help prevent GSOD from mfsfix. I wonder if someone could provide an image of a good 128MB swap file, which I (and others) could dd onto our mfstools 2.0 expanded swap partition to get a good 128MB swap file. This will be even more important if the 1 MB block size mfs partitions (which might not break mfsfix) is the answer to prevent mfsfix errors, because with the small block size and only 64MB swap, mfsfix will run out of memory. My understanding is that it is not possible to get a bash prompt or to edit rc.sysinit on a DTiVo, so this might be the only way to get a good 128 MB swap. Unless, of course, we fall back to mfstools 1.1. Of course, I guess I could get another blank drive, use the old tools, get a good 128MB swap file, and then dd it to my current upgrade. I just might try that next time I have time (maybe next weekend).

John

[Robert S]

That's not a bad idea. If you start with a blank disk, or zero the partition out (use dd to copy /dev/zero over the partition) before running mkswap then a gzipped copy would be tiny. It may well be that any 128Mb swap partition would be a suitable source (subject to byte-swapping considerations).

[klincoln]

klincoln, if you're in the mood to experiment, would you like to try MFS Tools 2.0 with -r 0 (1 Mb block size)? Some people have expressed the hope that this might create a recoverable TiVo, but no-one's tested it.

Robert S, trying the -r 0 crossed my mind as, under the circumstances, MFST2's default larger block size is a strong suspect in my mind for the cause of mfsfix's heartburn. However, given that I want to confine my testing to a 2x137GB target with preserved recordings, I'm hesitant to invest the time to test -r 0 without the known ability to lay down a valid 128MB swap partition to accompany -r 0's smaller block size on this >140GB configuration. Just as jhburke is creatively mulling the same dilemma, I'm in search of a solution. Without a valid 128MB swap partition, I don't hold out much hope for -r 0.

I'll see what comes up between now and next weekend, and maybe give it another whirl then.

Kevin

[Robert S]

Hmm, swap partitions, it really shouldn't be that hard to make them, you know?

I think the swap partition on a TiVo is just the same as any swap partition on any Linux system. If this is true, then the following should work:

Boot any normal Linux install (I used Tom's Root and Boot Disk (http://www.toms.net/rb/) as it's a convenient single-disk Linux.

Create a blank file of the right size. I did

dd if=/dev/zero of=/mnt/swapfile bs=1024k count=128

(obviously you'll need something mounted on /mnt to take the file).

swapfile is 134,217,728 bytes long. Is this the right size for a 128Mb swap partition?

Then

mkswap /mnt/swapfile

I then did

gzip /mnt/swapfile

and it created swapfile.gz at 131kb.

Now, I'm not sure that file is the right size to copy onto your swap partition, so you might want to try the following:

From a swapping environment:

dd if=/dev/hdx8 of=/mnt/swapfile

From an environment where you have a mkswap command available (Tom's would do nicely)

mkswap /mnt/swapfile

and then from a swapping environment

dd if=/mnt/swapfile of=/dev/hdx8

If this works (I have no easy way to test it) you could gzip swapfile and post it for others to use. If swapfile doesn't compress well, once you know the size of the swap partition you could use dd as above to create a file the same size made of zeros (bs=1 count=<filesize>), run mkswap on that and then gzip it.

This is not a risky procedure, as long as the swap file is no larger than the swap partition you won't overwrite anything - the worst than can happen is that the swap won't activate at boot.

[rswift]

I, too, am suffering the invalid swap file after using the -s 128 option in the MFSTools 2.0 boot CD. I have a Sony T60 with a Maxtor 160G A drive + Maxtor 80G B drive. Reported recording time is 204 hours, and has been working fine for a couple of weeks. After checking the threads here, noticed the swapfile issue, enabled back doors, and discovered the problem.
My quandry/question is whether to do anything now, or wait until:
A) My DirecTiVo crashes, then I take the original 40G Quantum and re-upgrade, or
B) Some genius figures out how to create a 128Mb swapfile usable by TiVo with support for fsfix.

Is it worth trying to copy the 64Mb swap file from my original drive onto the new A drive into the 128Mb partition?
I'm not sure I understand the risk of running without a swapfile, vs using a 64Mb swapfile in my 204 hour TiVo. Can 64Mb swap handle a TiVo with less than 140Gb? If so, maybe I should downgrade to just the single 160G Maxtor. I haven't read anything so far that pinpoints the cause of the fsfix, sigma 11 error, but something gave me the impression that may be related to having insufficient swapfile space for large capacity TiVos.

Thanks to all who've made the upgrades possible. The added capacity is well worth any hassles in opening the box and copying the drives.

rswift

[Robert S]

The swapfile issue and green screen recovery aren't linked in your case. It is true that increased swap space is necessary for earlier versions of MFS Tools and earlier versions of TiVo OS, that no longer applies.

There is currently no known way to make a TiVo that can recover from green screen if it's been upgraded with MFS Tools 2.0.

Copying your 64Mb swap partition from your old drive is a good solution for now. (Unless you'd like to try the swap file method I posted just above you, that post was written with some Linux knowledge assumed, I can explain it more carefully if you're interested.)

If you want a 'safe' TiVo (ie, green screen recovers) you will have to repeat the whole upgrade with pre-MFST2 tools - or wait for a fix for MFST2 problems.

[rswift]

Any guess as to the issues with running without ANY swapfile?? Everything seems to be working with no issues right now. Maybe it's because I haven't let the new drives start getting really full yet. My assumption would be that the swapfile might become very important as the lists of Now Playing, Season Passes, etc. start growing very large, and TiVo needs more memory space to play with the lists??
Thanks for the offer. I've not played with Linux at all, and have limited knowledge of UNIX, mostly Solaris. My first inclination is to dd the old swapfile into the new swap partition.
One thought I had is that if the new swap partition created with the -s 128 option is just slightly too small, even by a byte or so, will a properly created 128 Mb swapfile necessarily fit into the space? Not knowing exactly what the -s option really did, it's hard to predict. In my gut, I fully expect to have to re-upgrade using my old 40Gb original TiVo disk, so I've been copying to VCR most of my "must save" shows. YUK!~
Thanks for the help.
~rswift

[Robert S]

It appears that the problem with the missing swap relates to the program guide functions rather than the recording/playback functions (which are mostly handled in hardware). On UK TiVo's the missing swap prevents TiVo from making daily calls, on US machines it just seems to slow everything down (~24hour indexer runs).

On the space issue, my suggestion above was to dd the current swap partition to a file on your hard drive, therefore when you copy it back it will be precisely the right size. It would be nice if you could validate that, but copying your 64Mb swap partition over would make your problems go away very safely.

[rswift]

I may have to wait until next weekend to play with any of these new ideas. I took the day off work today, but can't do that again for a bit. Let's hope a real fix to this, and the GSOD problem are discovered and posted soon. I'm a bit nervous now.
On an unrelated note, I've had smoke coming out of my TiVo on two occasions, yet the thing still lives on! I fried the original TiVo Quantum drive while copying it the very first time. One of the IC's on the IDE circuit board glowed white hot and smoked out. Thought it was toast, but replaced the little circuit board on the back of the drive with one off of a new spare Quantum, and darned if it didn't come back to life. Then, I smoked the motherboard while replacing the internal fan (trying to make TiVo run cooler) and while the fan supply got fried (I now use power from the 4pin hard drive connectors) the TiVo still lives.
Needless to say I open the cover of my T60 with much trepidation...

[BareMetal]

Originally posted by rswift
Any guess as to the issues with running without ANY swapfile?? Everything seems to be working with no issues right now. Maybe it's because I haven't let the new drives start getting really full yet. My assumption would be that the swapfile might become very important as the lists of Now Playing, Season Passes, etc. start growing very large, and TiVo needs more memory space to play with the lists?? =

My experience with no swap, is a series 2 tivo that starts shutting down services and leaves me with a grey screen every other day. I just need to find some time to fix it..

[Ivor]

Robert S said:

"swapfile is 134,217,728 bytes long. Is this the right size for a 128Mb swap partition?"

From http://www.mcc.ac.uk/Documentation/linux-doc/LDP/sag/x1762.html:

"The Linux memory manager limits the size of each swap space to about 127 MB (for various technical reasons, the actual limit is (4096-10) * 8 * 4096 = 133890048$ bytes, or 127.6875 megabytes). You can, however, use up to 8 swap spaces simultaneously, for a total of almost 1 GB.

With newer kernels and versions of the mkswap command the actual limit depends on architecture."


Could this be the cause of the problem, i.e. MFS2 *is* creating a 128Mb swap file/partition, but TiVo is unable to recognize it because of kernel/hardware limitations?

[enigma2175]

From the mkswap man page:

It is roughly 2GiB on i386, PPC, m68k, ARM, 1GiB on sparc, 512MiB on mips, 128GiB on alpha and 3TiB on sparc64.
Note that before 2.1.117 the kernel allocated one byte for each page, while it now allocates two bytes, so that taking a swap area of 2 GiB in use might require 2 MiB of kernel memory.


It sounds like the tivo kernel is new enough to use 2 bytes per page, and that swap partitions can be up to 2 GiB. Maybe I'm wrong, and who knows what TiVo did to the kernel, but I think we're safe in that regard. However, it might be wise to create 2 64MB swap partitions and use them both (or even better, have one on your second disk, better performance.)

[Robert S]

I don't know if this applies to the version of Linux running on TiVo. It certainly might apply to the version on the Tom's Root and Boot Disk I used (quite an old one, something like 1.7 (of TR&B)).

mkswap did say something like 'truncating swap file to ' and gave a number that I think did start with 133. That was what got me paranoid about file lengths, wondering if dd had created a file that was too large for the TiVo's swap partition.

TiVo are unlikely to have fiddled with the VM on their kernel. I would think a swap partition create the way I suggest would work.

It's not easy to create another swap partition on your B drive as that drive will be full of MFS partitions. TiVo doesn't need the speed of multiple swap partition.

I do wonder, though, if you needed emergency swap for a mfsfix whether you could set up a swap file on /var.

However, it does seem that 64Mb of swap is plenty for TiVo running later than TiVo OS 2.0 and it's only when you get no swap at all that people seem to have real problems.

[enigma2175]

Yeah, a swap file in /var/ might come in super-handy. However, I generally don't have much room in /var and it is getting smaller all the time. But a simple swap file rather than a partition might make it easier for some people to get their TiVos running again.

[Ivor]

I have a Thompson (UK) TiVo with factory fitted 30Gb and 15Gb drives which I want to upgrade to 2 x 120Gb, probably this weekend. Since I'd prefer to have partitions with 1Mb blocks and an increased swap file (to avoid repercussions with fsfix), I don't mind experimenting a bit. I'll try MFStools with the -r 0 and -s 128 options, followed by Robert's procedure above, and let you know what happens. :-)

[Robert S]

Unfortunately we don't know that -r 0 fixes the GSOD problem and I don't know how to run mfsassert without a shell on the TiVo. Can you add mfsassert -please to the start up script the same way I added mkswap? The system should then reboot and green screen. (Does that sound right klincoln?)

[Ivor]

OK - Once I've confirmed I've got a working 128Mb Swap file, I'll add mfsassert -please to rc.sysinit and see if it recovers.

Question - when fsfix finishes repairing a partition successfully, does it usually reboot the box or just continue with startup? If it reboots, I'd need to interrupt the process at some point to prevent getting into a perpetual loop <break - fix - reboot - break - fix - reboot ...>.

[Brad Bishop]

I went back last night and checked my DirecTiVo which I had upgraded with a 80GB drive. It looked fine as far as the swap file goes. I did notice, though, that it was at 64MB instead of what I had been reading here (128MB). I understand what swap files do but I am left wondering if there's some great increase in performance or need for a 128MB swap file.

Also, I'm thinking about taking out the 80GB drive and giving it to my daughter (she's building a PC) and putting a 120GB drive inside it. I remember reading a while ago that TiVo would slow down noticably with drives over 100GB. Is this still an issue or is it related to the swap file size or it being corrupted?

Thanks,

Brad

[Robert S]

Ivor, you'd probably need to catch it on that first reboot and reset the sysinit before letting it green screen.

Brad, I don't think swap size affects the user speed I upgraded with TiVoMad and went to 128Mb and that was noticably slower than the original 40Gb config. That drive then died and MFS Tools 2.0 came out while the drive was being replaced. With MFS Tools 2.0 the TiVo feels very similar to how it did with the 40GB despite now being a 120 and having 11 pages in NP, even with no swap at all! The problem with no swap was that the indexer got stuck, not that the TiVo was unresponsive.

I think you should be find with 64Mb of swap, but there are several methods in this thread for fixing any swap problems you may have.

[Cpen]

I upgraded to a single 120GB Hard drive. I've noticed no change in speed - more importantly, neither has my wife (beleive me, when she knows I've been monkeying with something, she's quick to let me know if I broke it or made it worse).

In the absence of the swap file problem (I don't have the swap problem - did not use -s 128), I think the only real slowdown you will see is in some "live" calculations made by TiVo - For example, the Season Pass manager - it may take much longer to calculate changes/additions as it has many more scenarios and data to sort through to calculate available space, potential conflicts, priorities, etc. It's this kind of live calculation that will get slower with more available/stored information to take into account! (I don't have enought season passes, wish lists, etc to have this get noticably slower, but with the new hard drive, I'm well on my way).

One live process that probably wont be effected is searching for programs to record. You're still working with the same amount of guide data (2 weeks).

I'm guessing that the swap file problem caused MAJOR slow downs because TiVo's indexing engine used the swap file to speed up indexing. Without the ability to index, a bunch of stuff probably get hosed - Program Guide, searching for programs to record, etc.

[deek_man]

I upgraded my Phillips HD112 about three weeks ago and just checked the (kernel) log files using backdoors.

It read:

"activating swap partitions"
"adding swap:65532 swap-space (priority-1)"

I guess that means everything is ok.

I used MFS Tools 2.0 and Hinsdale's Upgrade Configuration #2 (dated July 8, 2002) to restore my original 14 hour drive backup on to a virgin 100 gb Maxtor and I also added a second virgin 100 gb Maxtor.

The restore command I used is as follows:

mfsrestore -zpi /mnt/dos/tivo.bak /dev/hdb

My original backup was made using:

mfsbackup -6so /mnt/dos/tivo.bak /dev/hdc

I used the following command to expand the two drives:

mfsadd -x /dev/hdc /dev/hdb (note: no -s command)

I now have a 250 hour machine and have had no problems that I'm aware of over the past several weeks of use. Hope this helps.;)

[Robert S]

It's actually the mfsrestore command that affects swap, by the time you get to the mfsadd stage, your swap is already set up.

Anyway, thanks for the input.

[deek_man]

Sorry about that. I know where to plug in the cables but my understanding is a little superficial The restore command I used is as follows:

mfsrestore -zpi /mnt/dos/tivo.bak /dev/hdb

My original backup was made using:

mfsbackup -6so /mnt/dos/tivo.bak /dev/hdc

Swap is ok and box works great (see previous post). Hope that's a little more helpful.

[Ivor]

Update:

I started my upgrade yesterday, having first made and tested a divorced backup of my original drives.

mfsbackup -Tao - /dev/hda /dev/hdb | mfsrestore -r 0 -s 128 -xzpi - /dev/hdc /dev/hdd was used to copy my original a and b drives to the new ones using standard sized blocks and an expanded 128Mb Swap partition.

As expected the swap partition signature was not recognized by TiVo, and when the new drives were placed in the system this resulted in a perpetual reboot/GSOD/pause/reboot loop.

The disks were returned the disks to my PC to try and fix the swap partition which was expected to be found on /dev/hdc8.

TiVoMad disagreed, however, giving the following:
Partition check:
hda: hda1 {my Windows hdd}

hdc:Signature 9214, be16 Signature 1492
Blocks in Map - 10
mac st=1 sz=3f name='Apple' t='Apple_partition_map' bim=10
hdc1 mac st 51ff040 sz=1000 name='Bootstrap 1' t='Image' bim=10
hdc2 mac st=5200040 sz=1000 name='Kernel 1' t='Image' bim=10
hdc3 mac st=5201040 sz=40000 name='Root 1' t='Ext2' bim=10
hdc4 mac st=5241040 sz=1000 name='Bootstrap 2' t='Image' bim=10
hdc5 mac st=5242040 sz=1000 name='Kernel 2' t='Image' bim=10
hdc6 mac st=5243040 sz=40000 name='Root 2' t='Ext2' bim=10
hdc7 mac st=5283040 sz=40000 name='Linux swap' t=Swap' bim=10
hdc8 mac st=52c3040 sz=40000 name='/var' t='Ext2' bim=10

hdc9 } various MFS regions
hdc10 }
hdc11 }
hdc12 }
hdc13 }
hdc14 }

hdc15 ... name='Extra' t='Apple_Free' bim=10
hdc16 {blank}

I haven't had a chance to look at my original drives again, but it would seem that the partition labels do not match their contents. The log files I expected to have found in '/var' were all on hdc9; hdc8 was unmountable and hdc7 appeared to contain a root directory rather than a swap file. This is about as far as I got before I had to stop.

Any ideas why the partition names appear to have changed?

Ivor.

[Robert S]

I think this is just the rather confusing layout of the boot-up partition table list. If you use pdisk you'll see the labels are correct, and your experiments with mount also verified it. (7 is root, 8 is unmountable because it's swap and 9 is var.)

If you look closely you'll see that the first partition (Apple_partition_map) doesn't have a number and hdc16 doesn't point at anything. Perhaps on this list the partition names are at the end of the lines and wrap around to the beginning of the next line on an 80 col screen?

[Ivor]

That would make most sense - in fact it seems quite obvious now that you've pointed it out, I'm surprised I didn't think of it! :-)

In any case, I'm about to revisit fixing the swap file shortly. Wish me luck! :-)

[Ivor]

Robert was right - pdisk displays the partitions correctly, the version shown at boot up was what confused me! Anyways...

The new A disk was removed from my Tivo and rc.sysinit was modified to add mkswap /dev/hda8 at the start of the file. The disk was then returned to the TiVo and the box rebooted.

Mkswap ran and the swap file was created successfully (from the kernel log at next reboot):

mkswap: warning: truncating swap area to 130752kB
Setting up swapspace, size = 133885952 bytes
...
Activating swap partitions
Adding Swap: 130684k swap-space (priority -1)


So far so good, but then after a while it continued with:

Time set to: Thu Aug 8 22:58:36 2002
Have a nice day.
Checking for additional disk...
hdb: Generic ATA management
Starting EventSwitcher...
Filesystem is inconsistent - cannot mount!
Writing 207040 bytes to OSD at address 0
Filesystem assert: cbMap >= sBucket at fszone.C line 170 in FsZone::FsZone(bool, enum FsZoneOwner, long unsigned int, long unsigned int, long unsigned int, long unsigned int, long unsigned int, long unsigned int, bool = false)
Filesystem flagged as inconsistent!
Tmk Assertion Failure: cbMap >= sBucket
FsZone::FsZone(bool, enum FsZoneOwner, long unsigned int, long unsigned int, long unsigned int, long unsigned int, long unsigned int, long unsigned int, bool = false), line 170 (fszone.C)
Tmk Fatal Error: Thread main <78> died due to signal -2


at which point, following a few seconds of GSOD, the box reboots...

Back to the drawing board. :)

[kingmiwok]

I upgraded a series 2 AT&Tivo 40 G to a 120/120 with MFS2.0 about a 5 days ago. Unfortunately I used the dreaded -s 128 option and did not discover the swap problems discussed here untill afterward. Dang.

Symptoms: It boots OK and will run for about a day, then I get a UI lockup. It keeps recording though, I just can't get menu control. After reboot it runs fine for another 24hr or so.

I have every reason to believe I have the swapfile problem, but when I access the logs with the backdoor, I do not actually find the....

"activating swap space"
"unable to find swap-space signature"
"swapon: Invalid argument"

....entry. Actually, I can't find anything listed on the swap file activation? Am I looking in the wrong place? var/log/kernel, right?

Is it significant that I can not find this in the log file? I should find it within the time-area for the most recent boot, correct? What other activity in in the log in the same general area?

Any input would be apreciated.

Note: I did find, in another log file, a line that stated (something like) "swap file size 0000000". All the other files/partitions listed had significant file sizes.

Thanks.

[Robert S]

You'll see it fairly early on in the boot sequence, the date will be in January because the clock has not been set at that point. There will be three lines of error messages if it fails, but just one logging success.

[Ivor]

kingmiwok - if you are going to take the drives out anyway to fix the swap file, you might find it quickest to use the TivoMad or Dylan's Boot disk to mount the /var partition (number 9 on the Tivo A drive), load the kernel log into an editor and then search for 'swap'. As Robert says, it should be a few lines before the box corrects the date and time.

The swap file is easy enough to fix by putting mkswap /dev/hda8 at the start of the /dev/hdaX/etc/rc.d/rc.sysinit file (where X is either 4 or 7 depending on your config). See Robert's post at the start of this thread for full details.

--oo0oo--

I completed my second rebuild earlier this morning after running
mfsbackup -Tao - /dev/hda /dev/hdb | mfsrestore -s 256 -xzpi - /dev/hdc /dev/hdd
to create a drive with a 256Mb Swap partition but leaving out the -r 0 parameter [standard block sizes]. I then got TiVo to recreate it's own swap partiton by adding mkswap /dev/hda8 to rc.sysinit as above and tested it by running a full program call. This seems to be working fine [no GSOD] I'm not going to tempt fate by running mfsassert -please on this system despite the larger swap file, since I'm fairly certain this would fail.

Interestingly mkswap did use the full 256Mb allocated to it, although the partition is definitely 256Mb and the MFST2 readme suggests it could use up to 511Mb. Regardless, it is still using the same 133885952 bytes as before, just short of the original 133890048 bytes limitation for early versions of linux...

Thanks to everyone involved for all their help! :)

Ivor.

[kingmiwok]

Thanks for the replies/advise. Much appreciated. I'll tackle it this weekend and report anything out of the ordinary.

Slightly OT... I noticed you guys are from the UK. Many of the posts here are, which leads me to believe (incorrectly?) that Tivo may be more of a phenomenon in the UK than in the US.

In the US there is somewhat of a name recognition for Tivo, but it is hardly a household term. Among my closer friends (mostly computer geek types here in Silicon Valley), NONE have a Tivo. And even though you can't get me to shut up about the virtues of Tivo if asked, I can't get them to take the plunge and get one. It's almost as though they don't get why it's better than their VCR, or don't care. They seem to prefer to watch DVDs they have bought/rented.

I think maybe they see it as just another unwanted distraction competing for their time. We are all in our 30s+. Maybe it's more popular with a younger crowd, but I don't believe so.

Is Tivo huge in the UK? Lots of name recognition with the general population? (I have a fascination with why some products take off and others flop, or never get broad acceptance. Yet will be very popular elsewhere.)

Again, thanks for the tech help.

[Ivor]

Slightly OT... I noticed you guys are from the UK. Many of the posts here are, which leads me to believe (incorrectly?) that Tivo may be more of a phenomenon in the UK than in the US.

This thread http://www.tivocommunity.com/tivo-vb/showthread.php?s=&threadid=70772 suggests there are about 35000 TiVo users in the UK and about 422000 worldwide. That's maybe just over 1 box per 800 households here which would explain why I don't know anyone else who has one... The UK board can probably account for this better than I can. :)

[kingmiwok]

Hmmm, something is not working.

I tried fixes 2 and 4 with (I think) no luck. Tivo is up and running again, but I think my swap file is in the same condition as beforehand and I'll have a lockup in a day or so.

Note: I am not Linux savvy at all, but I do understand what the commands are supposed to do. I'm vcomfortable with DOS so working from a prompt does not bother me.

In a nutshell,
I boot on MadTivo disk and try to run edit_bootparms /dev/hdc -r
I get "file not found" (or whatever it says exactly)

If I try to mount a drive from a Dylan boot, I get a variety of responses, none completely positive. It often wants to know the file system type.

I tried to dd over the 64M swap from the orig Tivo drive and the reply I got back was
0+0
0+0
That seemed wrong.

I notice during boot that it does not read the partitions and can not identify the file system type. It finds the drives OK though.

Question: At the beginning of this thread, for repair method 2, is the dd command correct? States going from hdb8 to hdc9. Should be hdb8 to hdc8, yes?

So if it's possible to guide me a bit... what am I doing wrong? Thanks.

[Robert S]

Yes, well spotted, the swap partition is number 8, of course. You can stare at these things for ages and not see them, you know?

Assuming your working drive is on hdb and your new drive is on hdc your dd command would be

dd if=/dev/hdb8 of=/dev/hdc8

It should say something like 128 records transferred.

edit_bootparms is in the mad directory on TiVoMad, so you need to do

/mad/edit_bootparms ...

(I've fixed this in the top post too).

mount should give no response if it completes. If it asks for a filesystem type you can take it you're either mounting the wrong partition or a byte-swapped one, Linux can auto-detect FAT, Ext2 and NTFS partitions.

Thankyou for your comments, hope things start working for you soon.

[rswift]

Originally posted by kingmiwok

I tried to dd over the 64M swap from the orig Tivo drive and the reply I got back was
0+0
0+0
That seemed wrong.


Kingmiwok, that is exactly what happened to me yesterday. I dug out my original TiVo A drive and slammed in my 2 new TiVo drives, expecting to do a 5 minute dump of the old swap onto the new partition, but got the 0+0 in and 0+0 out message just like you did.
I had all the drives on a second controller card , and used
dd if=/dev/hde8 of=/dev/hdg8, where I connected the original TiVo disk to the primary master on the second controller, and the new TiVo drive (with the bad 128M swap partition) as the secondary master (and the new TiVo B drive on the secondary slave ).
I used the MFSTools 2 boot CD, and all drives were recognized on bootup, so I have no idea what the problem was.
I'm not familiar with Linux, either, so rather than risk doing something bad to my original disk (I have no backup image), I just bailed out, hoping for some help or suggestions online.
My TiVo started acting wierd on Saturday am, rebooting 6 times in fairly rapid succession. It would work for about 5-20 minutes between reboots, then go again. It seems to have fixed itself, or at least settled down for a bit. Its been up for about 36 hours without any issues.

Any new info or suggestions appreciated.
rswift

[kingmiwok]

Robert S:
Thanks for the reply. Based on your comments and what has happened, I have the general feeling that my drives are not being detected correctly when I boot Linux. I'm thinking its not running the commands, mount or dd, etc because it is not seeing the drives in their complete form to begin with.

For example, cd'ed around a bit and spotted the edit_bootparms in the mad dir. It still blew me off when I tried to run it!

I'm not sure why, but maybe my motherboard settings are confusing things.
I think I'll retry with everything dummied down, or do it on another computer where I'm not afraid of messing up the BIOS settings. (I'm always anxious about changing settings on a computer I need for other "real" work.)

rswift:
I feel your pain. I'm doing more rapid reboots now after trying to fix it. Nice. I'm backed though, so I feel a fairly safe in trying things without paying too bad of a price.

My only recommendation it to do what I mentioned above. Maybe try it again with a very simple computer setup. No add-on second controller card, no cards in the slots, the absolute minimum. I think you want HD detection turned off in the BIOS as well. Look at your boot sequence very closely too. Even though my boot was detecting the drives as a physical device, it had lots of comments that indicated that it was not reading the FATS and partitions completely. See if you're getting those too.

I'll post up any success/failure. I'll try to get to it within the next couple days

If it does not work for me, I'm going give up the programs I have recorded since the upgrade and redo the whole process. I was interested in learning a little bit more about the inner workings and fixing the swap directly, but it's becoming a drag. My limited linux skill are really beginning to show.

[rswift]

thanks, kingmiwok. I probably won't crack the cover again until the weekend rolls around, or at least Thursday. I looked at the boot sequence very carefully and didn't see anything that I thought was troublesome. I used the exact same configuration when I copied and expanded my original TiVo drive that I'm trying to use now to repair swap, so I'd think that I would have run into problems the first time around.
I'm also curious as to when it's necessary to boot into byteswapped mode. I tried that with the MFSTools CD and got an error and freeze. Had to reset the PC.
Thanks again for the response.

rswift

[jhburke]

Rswift:

I had success copying the swap file by booting into byteswapping mode. The easiest way to do this is to boot from the older MFSTools disk (i.e. Kazymir's Boot Disk). Byteswapping mode is broken on the new MFSTools 2.0 disk (you can read the thread at the top of this forum) but it can be done if you type "vmlnodma hdb=bswap hdc=bswap hdd=bswap" at the "Boot:" prompt. The error in the boot disk is that the wrong kernel is invoked if you type "boot: swap" . It loads the dma kernel which doesn't work for byteswapping, which causes the lockup. Read the isolinux.cfg file on the cd if you want to see. I think it will work for you in byteswapping mode.

John

[Merle Corey]

So, being the curious sort, and having an extra 40gb drive, I just finished running a proof of concept.

Summary: mfsrestore -r 0 works.

Anyway, here's what I did:

My TiVo: Philips HDR-212, with TurboNet and 100GB HD (TiVoMad style expansion performed Aug '01).

Booted with the MFST2 CD (default/dma enabled mode). Backed up my current TiVo drive with:

mfstools backup -6s -o /mnt/bak/mybackup /dev/hdb

Shutdown, disconnected good drive, installed test drive.

Restored with:

mfstools restore -x -p -s 128 -r 0 /dev/hdb -i /mnt/bak/mybackup

After completion, I rebooted with the TurboNet 3.0 install CD and installed (for tnlited, primarily).

Slapped the drive in the TiVo. Booted. Telneted in, ran:

mkswap /dev/hda8
swapon -a

Rebooted. Swap was accepted.

Ran:

mfsassert -please

Waited nervously through the green screen. It didn't green screen loop - once through the green screen, then back to the happy TiVo intro vid and life as usual.

So MFSTools 2.0 can (apparently, in my case) be made to work completely with a little finagling and not using the defaults.

Anybody else want to make a test run and either confirm or refute?

MC

[Ivor]

That's restoring using a minimal backup, right? i.e. no programs saved?

My attempt at using -r 0 (whilst expanding drives and saving programs) broke even before I had a chance to run mfsassert -please, and didn't recover (see earlier in thread). If I can get hold of a suitable disk I'll try restoring my emergency backup with mfstools restore -x -p -s 128 -r 0 etc. and let you know what happens. It would also be interesting to see how this works on a >140Gb total setup (which requires the larger swap file).

[Merle Corey]

Originally posted by Ivor
That's restoring using a minimal backup, right? i.e. no programs saved?

Correct. Time (not being patient enough to leave my TiVo down that long for a proof of concept) and space (going from 100gb to 40gb, only having a 2.5gb partition for holding any backup images) were my primary reasons for not attempting the complete backup/restore.

Besides, that's the best kind of backup to test -r 0 from, the way I figure - it guarantees all MFS partitions are MFSTools 2.0 created.


It would also be interesting to see how this works on a >140Gb total setup (which requires the larger swap file).

Theoretically, it should make no difference - having repaired the swap and gotten it recognized at 128mb, it shouldn't matter what size the overall setup is... But you're absolutely right, it'll be interesting to see if it works regardless.

MC

[rswift]

Any thoughts on a repair solution for a DTiVo? Doing the mkswap after the drive is back in TiVo isn't really a viable option. I've read elsewhere in this post of doing a mkswap while the drive is out of TiVo, then after the drive is returned to TiVo, the swap isn't readable.

I'm still curious about which copy, backup, restore type commands need to be done in byteswap mode vs. non-byteswap.

Thanks.
rs

[sbourgeo]

So what's the consensus on new expansions?

I have an HDR-312 where I had previously replaced the 30G drive with an 80G drive with TiVoMad/mfstools 1.1.

Based on the Hinsdale directions, I can just put the 80G and my new 160G drive in my PC, boot with the mfstools 2.0 cdrom and run "mfsadd -x" to recognize the new drive.

According to a post from hinsdale in this thread and in tiger's original mfstools 2.0 announcement, the 64M of swap that I currently have should be sufficient. Is that a fair assumption?

I just want to make sure I have enough swap to recover from GSOD, if necessary.



Steve

[Robert S]

The reason Tiger thinks MFS Tools 2 doesn't need enlarged swap is that MFS Tools 2 uses larger blocks sizes, so mfsfix needs less space to run. However, MFS Tools 2.0 breaks mfsfix anyway, so this is hard to verify.

You should be OK if you use mfsadd with -r 0, but this hasn't really been verified. This doesn't use the larger block size, so mfsfix should complete, but you'll have to trust that both Hinsdale's and Merle Corey's upgrades are representative of your situation.

[dobbie1]

I have been following the discussion with much interest, hoping to confirm that the lockups I am receiving on a daily basis now are the result of not having created a swap space when I upgraded my Series2 unit. I don't know if MFS Tools 2 has been updated but I used the version first released. I checked my log files and it does not appear that I have any swap space on the unit. I plan on trying to use the option 2 to fix but since I am not familiar with Linux, but do follow directions very well, are the commands listed for the option 2 fix all I will need execute in order to create a swap? Any comments, suggestions will be appreciated.

Regards,
dobbie1

[Robert S]

First make sure you really do have a swap problem. You should see three lines giving a clear error message about problems activating swap.

Method 2 should work for you, although I don't think anyone's actually tried it on a Series 2. S2's don't need byteswapping, so you should just boot the MFST2 CD as normal.

[Merle Corey]

Originally posted by Robert S
You should be OK if you use mfsadd with -r 0, but this hasn't really been verified.

Not thoroughly, anyway. More datapoints are definitely better.

Of course, Steve's also looking at a >140GB capacity, so he has a choice between playing guinea pig or playing it safe.

Specifically, while -r 0 works (at least some of the time), you (Steve) will theoretically need the expanded swap space as well, and will need to do the song and dance to expand it (Robert nailed a good summary of the possible methods in the first message in this thread).

The guinea pig aspect comes in with not expanding the swap - according to one of Hinsdale's comments somewhere in all of this, the 64MB swap "limitation" may have been resolved in later versions of the TiVo software. If you're the gambling type, you can try not expanding and see what happens.

Of course, failure would essentially result in loss of whatever you've got on the drive, requiring you to reimage... :D

Me, I think I'm going to get myself a new drive and play a little more - I'll run another test scenario and see exactly what happens with >140GB capacity, 64MB swap, and that ever so enjoyable mfsassert -please.

I'll post my results when I've got them, but nobody hold off anything on my account - I haven't even ordered the new drive yet. I'm at least a week or two away from being able to sit down and play again.

And on a not-unrelated note, does anybody have any idea what's become of Tiger in the last few months?

MC

[Robert S]

Steve's in a rather sticky situation already. He can't just expand swap because his A drive is already full. He just wants to add a B drive. This should work, there's even a chance it might survive a green screen, but there's no easy way to guarantee it.

It might be safer to do a pipe transfer saving all recordings (see MFS Tools README, RESTORE section) and using the -xp, -r 0 and -s 128 options to move everything onto the new drive, expanding swap (will need to use one of the repair methods at the top) and then, once the new drive is verified, mfsadd the old drive as the B drive.

By the way, Merle, what size was your backup image? Did MFS Tools create a new pair of MFS partitions in addition to the original TiVo-created pair?

[sbourgeo]

Originally posted by Robert S
Steve's in a rather sticky situation already. He can't just expand swap because his A drive is already full. He just wants to add a B drive. This should work, there's even a chance it might survive a green screen, but there's no easy way to guarantee it.

It might be safer to do a pipe transfer saving all recordings (see MFS Tools README, RESTORE section) and using the -xp, -r 0 and -s 128 options to move everything onto the new drive, expanding swap (will need to use one of the repair methods at the top) and then, once the new drive is verified, mfsadd the old drive as the B drive.


Actually, I thought about doing that...

If I just run mfsadd to partition the 160G drive, then I really don't have the ability to create another swap partition before (since mfsadd would wipe it out) or after (no space left on new drive) I run it. I wish I had selected 128M swap size when I upgraded the first time, but didn't see the need to at that time... :(

I may just hold off on adding the 160G for a little while. I have a bunch of recordings that I'd prefer to keep, so that is my top priority at the moment.

Maybe by then Tiger will get back from vacation or from doing REAL work and can look into this stuff. :)



Steve

[Merle Corey]

The backup image is a stripped down, formerly upgraded, 20 hour image. (The original upgrade was to 100gb - my understanding is that the -s was required in order to destroy the expanded partitions and let everything be done over). When I restored via -x, it created a new pair of MFS partitions to fit the 40gb drive I was dropping back on to.

If you like, I can re-run the process tonight without -x in the restore and confirm that only a base 20 hour image is restored, and rerun afterwards with mfsadd -x -r 0 /dev/hdb. (Running mfsinfo in between to get the particulars.)

You're exactly right regarding adding the B drive, and making the new drive into the A being the safest route; I knew I was missing an obvious problem there.

MC

[Merle Corey]

Originally posted by sbourgeo
If I just run mfsadd to partition the 160G drive, then I really don't have the ability to create another swap partition before (since mfsadd would wipe it out) or after (no space left on new drive) I run it.

You can do it during - if you do the piped transfer Robert refers to, you'll preserve all your shows, plus create a new swap on the fly, in the process of migrating everything from the 80GB to the 160GB. You can do the expansion at the same time, and even use -p to optimize the layout.

After that's done (and tested, and the swap fixed), you would run mfsadd to make your 80GB the B drive to your new 160GB A drive.

It's a slow process, mind you, but it's definitely the safest route, and accomplishes everything you need to do.

MC

[Robert S]

That's fine, Merle. I just wanted to be sure you hadn't put a 40Gb image on a 40Gb drive, which would've meant no MFS Tools 2.0-created partitions and rendered your success with mfsfix meaningless.

The two new partitions mean this was a valid test.

I suppose showing the same upgrade fails with -r 2 would mean something, but we're pretty sure that's the case. A test on a big drive is the really interesting case, particularly if -r 0 doesn't slow things down too much (keep the -p partition layout changes).

(Just paranoid, really.)

[Merle Corey]

Originally posted by Robert S
I suppose showing the same upgrade fails with -r 2 would mean something, but we're pretty sure that's the case. A test on a big drive is the really interesting case, particularly if -r 0 doesn't slow things down too much (keep the -p partition layout changes).

Hey, I can aim for failure, too. I'll run with -r 2 tonight just to prove the point (probably).

How big is big? I was planning on just going to 120GB for my new purchase, and merging it with the 40GB for large testing. That's more than a single 160GB, but it's way smaller than any dual-large config.

(I'll freely admit that my curiosity and testing aren't entirely selfless - my 100GB drive has started making the kachunk-kachunk-kachunk of doom, and I'd rather get it out of my TiVo before it's dead. Gotta make sure I've got a clean upgrade path. :D)

MC

[Robert S]

If you can get a 120 + 40 to recover from a green screen, I'd be convinced (already close to convinced). You'd then have a two-drive configuration, much larger than any standard TiVo, which would cover most of the potential problems your current experiments haven't.

[Merle Corey]

Ok, so my current laundry list is:

40 w/ -r 2: Proof of failure

120+40 w/128MB swap & -r 0: Proof of success in large setups.

120+40 w/64MB swap & -r 0: Proof of possible mfsfix swap requirement resolution (in 3.0 at least).

Am I missing anything? Any requests? I'm limited by hardware to SA series 1 only, so I can't do any DTiVo experimentation.

MC

[dobbie1]

[QUOTE]Originally posted by Robert S
[B]First make sure you really do have a swap problem. You should see three lines giving a clear error message about problems activating swap.

If the swap is okay, would I not expect to see some message indicating the swap was activated?

Regards,
dobbie1

[sbourgeo]

Originally posted by Merle Corey


You can do it during - if you do the piped transfer Robert refers to, you'll preserve all your shows, plus create a new swap on the fly, in the process of migrating everything from the 80GB to the 160GB. You can do the expansion at the same time, and even use -p to optimize the layout.

After that's done (and tested, and the swap fixed), you would run mfsadd to make your 80GB the B drive to your new 160GB A drive.

It's a slow process, mind you, but it's definitely the safest route, and accomplishes everything you need to do.

MC

I'd rather hold off on the klugey stuff (like the swap workarounds) until the process is ironed out a little.

The TiVo I want to add it to is currently our "production" model and I want to minimize the risk as much as possible.


Steve

[Robert S]

If your swap is OK you'll see one line (very easy to miss) saying something like 'Activating swap <size>, priority -1'.

[dobbie1]

I rebooted, then activated backdoors, went to /var/log/kernel and did not see any entry as described above. The only entry close was "Initialize with 1 live caches."

Thanks for your responses.

[Merle Corey]

Originally posted by dobbie1
I rebooted, then activated backdoors, went to /var/log/kernel and did not see any entry as described above. The only entry close was "Initialize with 1 live caches."

Straight from my kernel log:

Jan 1 00:00:15 (none) kernel: Activating swap partitions
Jan 1 00:00:15 (none) kernel: Adding Swap: 130684k swap-space (priority -1)

It's very early in the boot process (you can tell because the date/time isn't set yet), but it's pretty easy to miss. The line you're looking at is around (ballpark number, YMMV) 100 lines too far down. (Or, an unknown number too soon, depending on where your logfile starts.)

Hope that helps!

MC

[horwitz]

So what is the latest "best idea" for upgrading? I am going from a factory HDR312 (13.6+13.6) to 120. MFSTools 2.0 a la Hinsdale's page (what about all these flags people are talking about? -xp, -r 0, -s 128, etc.)? Will the old 1.1 solution on Hinsdale's page work; what disadvantages are there to the old procedure (aside from it taking longer)? BTW, I hope to upgrade to 120+120 sometime in the not-too-distant future.

[Robert S]

As you might guess, this is still a matter for debate, however, I think we can say the following:

If you want to be absolutely safe, stay away from MFS Tools 2.0! Use the older, proven tools in the original Hinsdale.

If you have to use MFS Tools 2.0 (there are some upgrades that can not be done with the old tools), avoid the -s option (increase swap partition file) and use -r 0 to force 1Mb block size.

The -x and -p options (you've combined them as -xp) cause MFS Tools to fill the hard drive with Tivo partitions and use an alternate partition layout that's believed to be faster.

If you want the fastest possible TiVo and are prepared to accept the risk that your TiVo might lose all its recordings without warning (if you get filesystem corruption), use MFS Tools 2.0 as described in new Hinsdale, but check your swap as described in the first post in this thread.

[SAFW]

So does anyone know of a method to get MFStools 1.0 working with devices recognized as hdf, hdg, hdh, and hdi (the Promise RAID controller takes up hda, hdb, hdc, and hde)? How about TiVoMad? Is there an integrated restore/expand option like -zxpi in MFStools 2.0?

MFStools 2.0 worked like a charm for me with -zxpi, but now I'm concerned that I'll get the GSOD somewhere down the line.

SAFW

[Merle Corey]

I ran several tests tonight, and here are the results:

First up, swap space tests. (All tests are to the same 40GB drive.) First with:

mfsrestore -p -x -r 2 -i /mnt/bak/mybackup /dev/hdb

I can confirm that the "default" swap works fine out of the gate. As expected, a 64mb swap partition was found on first boot; it's not a restored swap either, as the swap on my old drive was 128MB. So we have a little more confirmation on that front.

A little weirder: I ran the experiment again, this time with the 128MB swap (-s 128), and rebuilt the swap (mkswap /dev/hda8 ; swapon -a) via telnet. The swap definitely mounted on the command, and remounted after rebooting, but no further log messages were generated by it. So everyone who couldn't find any message at all? You may not be crazy, and your swap may be fine. Don't ask me, I can't explain this one.

A third time through, and I tried another swap prep mechanism: Building the swapfile with the mkswap on the bootdisk. I used: mkswap -v0 /dev/hdb8

This worked. Flawlessly. Tricks involved were having byteswapping enabled and forcing the old style (-v0) swapfile. The default was to a new style swap - I knew something was up as soon as that happened because the messages were completely different. Forcing -v0 generated the same messages as doing mkswap via telnet to TiVo, and worked immediately on insertion.

I'd love to hear from someone with a DTiVo who's willing to try this. rswift? I believe you were looking for a mechanism to fix your swap?

Now for the really weird part: It successfully ran through mfsassert -please with -r 2. This surprised me so much that I ran it again after zeroing out the drive - and it worked again. So just for giggles, I ran it through a third time, this time without specifying any -r value (theoretically defaulting to -r 2, but I wasn't taking it for granted). It worked yet again.

The only conclusions I can draw are either:

1) -r works for all values and drive sizes, and any mfsassert/fsfix issues are unrelated

2) -r works for all values of drives under a certain size, but fails for drives/combos over a certain size

3) -r works with a stripped out backup (-s) but not for modifying "existing" data - preservation of programs, adding a B drive to an already expanded A drive, etc

I'd love to prove #1 is true, but I suspect the reality is #2 or #3. It may be that only "native" MFST2 mods work, and that piggybacking with other expansion mods doesn't. It may be yet another space issue. I can't work any further on most of these until my new drive arrives (some time early next week, best guess).

If anybody has anything further they'd like me to try, let me know. I plan to try again, this time to do a combo of:

mfsrestore -p -r 0 -i /mnt/bak/mybackup /dev/hdb

and

mfsadd -r 2 -X /dev/hdb

to simulate item #3 (as if the data had been dd'd over) and see if that makes anything break, but I'm not going to have time to do that until some time this weekend.

MC

[stormsweeper]

I think I was one of the first to report this on my SVR-2000. One part of the problem is that the older swap version is limited to 127MB. So you should get a message that the space was truncated to 133890048 or so when you run mkswap from the tivo.

Since then it's been happy.

[dougdmy]

I posted this in another thread, but thought that it was more appropriate to this thread.
__________

I used mfstools-2.0 and ran well for a couple of months. Then a few days ago, I had the GSOD/reboot loop problem. I had so many recordings on there, too. But I was very concerned about my Season Passes and Wishlists. Here's what I did:

I took out the two 120GB drives from my TiVo and used mfstools-1.1 to create a backup image. I got a message stating something about an inode problem and that it was trying to read the backup table (or something to that effect). The backup successfully completed. Then I restored the image using mfstools-1.1 to a new 80GB drive. I put the 80GB drive in my TiVo. On power-up, I got the GSOD and much to my delight, it did not reboot. It spent about 20 minutes fixing itself and then it rebooted and all seemed well. I deleted what was listed in my Now Showing list, powered down, and TiVoMadded the 80GB drive to utilize the whole thing.

So far (one day later) all seems well.

This got me thinking. Can I use mfstools-1.1 to create a backup image of my original 120GB+120GB configuration (streams and all) and then use mfstools-1.1 to restore it? Hopefully during the restore process mfstools-1.1 will create the mfs partitions in a way that will allow mfsfix to do its job and I'll have minimal programming loss.

Has anyone else tried this? It's an expensive thing for me to try because I would have to go out and buy enough disk space to store the huge backup file.

I appreciate any and all suggestions or ideas.

Doug (who is very sad because he lost the entire season of The Shield that he hasn't watched yet)

[Robert S]

I think this has been tried, without success. One obvious question would be whether MFST1 can read the larger blocks created by MFST2 (your minimal backup only captures the original TiVo partitions (1Mb blocks, even on an MFST2 restore) and doesn't touch the new partitions).

[Merle Corey]

For what it's worth, I'm starting to suspect that the block size problem isn't.

This morning's test run involved making a fresh backup of my original 100GB (20 hour image, TiVoMad expanded) drive with MFST1:

mfstools backup -6so /mnt/bak/mybackup /dev/hdb

I restored with MFST1 to a 40GB drive:

mfstools restore -zi /mnt/bak/mybackup /dev/hdb

This gave me a 20 hour, unexpanded image on my 40GB drive. Booted, confirmed functionality, and even ran mfsassert -please for kicks (fsfix was successful). I then went back and expanded with MFST2:

mfsadd -r 2 -x /dev/hdb

Booted again, confirmed everything was golden, mfsassert -please, and yet another successful run of fsfix.

I'm starting to nurse a pet theory, and that theory is that the decreased swap needed by the larger block size may not be directly proportional to the increase in MFS block size.

To put it differently: Kevin (klincoln) is the only other person to test MFST2 via mfsassert with a known good swap. He tested with dual 160's and a 64MB swap - if I'm right, he may need 128MB swap to make it work, even with the default (-r 2) MFST2 block size. Kevin? You still watching this? Wanna take a crack at it?

Meanwhile, I can't find a way to break things with a 40GB drive, so there's nothing more I can do for now. My new drive is arriving Wednesday, and I should be done with burning it in by Friday. As an added bonus, I may be able to lay my hands (temporarily) on another 120GB drive, allowing me to test an extremely large (though 34GB shy of maxed out) config.

MC

[Robert S]

How about reducing your swap? You could try swapoff to turn off all swap. You could also try manufacturing a smaller swap file and using that. If you could find a threshold below which mfsfix wouldn't complete, that would be an interesting find.

However, if you look at embeem's post on Tiger's MFS Tools 2 thread, it looks like he understands the swap issue.

[Merle Corey]

Originally posted by Robert S
How about reducing your swap?
I'm a little confused now. We already know that swap plays a critical role in successfully running fsfix. Or are you suggesting the establishment of multiple data points, whereby we can determine the minimum amount of swap required by, say, 40GB at -r 0 vs -r 2, and by extension determine whether the the swap size is directly proportional to the block size?

Heck, I think I may go do that anyway now, just for kicks. :D The real testing may once again have to wait for me to get the new drive, though, as I'm going to be playing with really small partitions to scale it down for 40GB (ballpark "minimum" at -r 0 should be around 18MB, meaning a theoretical 9MB for -r 2).
However, if you look at embeem's post on Tiger's MFS Tools 2 thread, it looks like he understands the swap issue.
I didn't get that out of it. He just asserted that there was a problem that prevented fsfix from running. He wasn't even certain that it was a MFST2 problem, and said he didn't have the resources to check.

It may be that we're flogging a dead horse at this stage, but I'm not going to be satisfied without definitive success either.

MC

[Robert S]

Yes, exactly. If reducing the swap breaks mfsfix on a 40Gb (and it may well be it's too small, but you don't have a bigger drive ATM), you should be able to find a threshold that separates sufficient from insufficient swap. It might then be possible to estimate how much swap a given configuration requires, and to test the theory that higher -r numbers require less swap for mfsfix.

To some extent you can't add much more to our knowledge because we need more data from other TiVoes (there are enough people reporting GSOD loops to suggest it's a real problem). On the other hand, if you're enjoying the investigation I'm happy to suggest new experiments. You've certainly shown that saying 'MFST2 breaks mfsfix' is an over-simplification.

I've PM'd embeem to ask what his config was - there's no earthly point in arguing what someone else meant :)

[jhburke]

I just wanted to post my confirmation of Merle Corey's findings about running mkswap from the boot disk. I had previously tried to run mkswap from the MFSTools 2.0 cd boot disk, and ended up with a swap file which I could mount from the boot disk, but would not mount from the TiVo. Now, thanks to Merle, I ran "mkswap -v0 /dev/hdc8" from a byteswapped boot cd, and my kernel logs now show "Adding Swap: 130744k swap-space (priority -1)". I also received the error message from the boot disk about truncating the swap file from 133xxxK to 130xxxK as Merle also noted. I don't notice anything running any faster on the TiVo, however.

I'd be willing to try mfsassert -please, but since I have