TiVo Community Forum banner
  • TiVoCommunity.com Ambassador Program Now Open! >>> Click Here

MFS Tools 3.2

407K views 2K replies 286 participants last post by  Reryro 
#1 · (Edited)
Announcing the release of MFS Tools 3.2!

"MFS Tools 1.0 was evolutionary in the TiVo upgrade process. MFS Tools 2.0 may prove to be revolutionary." I'll let you can choose your own adjective for MFS Tools 3.2.

Highlights
  • Supports Series 1 through Roamio (and perhaps beyond).
  • Backwards compatible with MFS Tools v1 and v2.
  • Supports 8 TB on Premiere (perhaps more, tested with dual 4 TB drives).
  • Supports 8 TB on Roamio (perhaps more, tested with dual 4 TB drives).
  • Backup and restore (or drive-drive using mfscopy) single or dual drive setups, and preserve recordings if you choose.
  • Supersize the drive, which can be reversed even after GSOD.
  • You can include the alternate boot/kernel/root partitions (useful for customized partition layouts).
  • Automatically includes required loopset files (you no longer need to use the -f flags during backup) when not including recordings (truncated backup)
  • Automatically prunes recordings when a truncated backup is done (sets the MFS flags to cleanup, so the unit will scan objects on startup)
  • Works on drives that have been marked for cleanup (GSOD).
  • Creates a new MFS structure, so drives that have been expanded by adding pairs can be consolidated to just two pairs, just like the original.
  • Restore to drives smaller than the original.

Overview
MFS Tools v3.2 is a set of utilities that allow one to backup/restore/copy/supersize a TiVo drive, without losing recordings, and to expand recording space to take advantage of larger drives. Additionally, there are options that allow more advanced features and finer grain control over the TiVo drive. For example, there are options to revalidate the drive to the Linux kernel, so the partitions can be mounted or copied. The ability to increase/decrease swap, var, SQLite partition sizes on restore. Limit the amount of used space of the drive, limit media partition size, add app/media partition sets, convert between 32-bit and 64-bit MFS layouts, etc, etc, etc. Please take the time to review all of the utilities and their options to fully appreciate what the tools have to offer.

Thanks
A special thanks to Tiger, the original pioneer of MFS. And to all those that contribute here and on other sites that have helped me get the most out of my TiVo units.

History (unverified)
Tiger released the original closed source project in 2001, followed by 2.0 in 2002. In 2004, he released the source code under GPL, and made it available on sourceforge, and announced that he had no plans to continue work on it. He hen returned for a three month period in 2005 and a two month period in 2008 to work on a major update to version 3, with some ambitious new functionality and 64-bit MFS support for the TiVo HD. It has remained in this untested, almost functional, but overlooked state since then.

I received a some anonymous patches that fixed a few bugs, added Premier handling, and got things mostly working one. Since then, I've refined the Premiere support, added Roamio support, added some new utilities, deprecated a few options and added a few more, and cleaned up and debugged the code a bit.

Although there are several new tools, features, options, etc., available in 3.2, there was a fundamental change in the the way the MFS is handled. A simplified explanation is that in v1, the MFS is treated as a whole, and the entire structure is duplicated based on the used blocks. In v3, the inodes and data is copied individually, and during restore the MFS structure is recreated, the inodes and data blocks are allocated, and the inode and data is copied to the new locations.

The downside is that v3 backup/restore is not quite as fast as v1. The upside is that the MFS Media Partitions can now be re-sized, which is a pretty big advantage. Because v1 copied all MFS pairs, the number of times a drive could be backed/restored and then expanded to take advantage of additional disk space was limited. With v3, the restore process creates a drive with only two MFS app/data pairs, just like the original drive, no matter how many pairs the original drive had. And yes, you can now restore to a smaller drive than the original drive (provided the smaller drive has enough space for the recordings or you truncate it).

Notes
Every effort has been made to keep these tools as robust as possible. I have done limited testing (confirmed boot/operation, but not filled drives to capacity) on units from Series 1 through Roamio. The tools can still backup/restore using the original v1 backup format, and should work with existing backups.

I have deprecated several options, and will probably deprecate more to reduce the number of options for novice users. Some of the deprecated options (and additional debug options) will still be available with custom builds.

I have only tested with drives up to 4 TB, but there's been mention of Roamios supporting larger drives with 20.4.6 or later. I was able to setup a Premiere with 8 TB (dual 4 TB drives), but had to limit the media partitions to 2 TB max. This unit passed a forced MFS fsfix scan (GSOD) without issue.

Todo
I still have a few things I would like to add, such as adding SW version, original swap/var/db size to the backup header (extrainfo). I would also like to do some testing with larger drives and additional units.

Known Limitations
Currently, the largest boot drive supported is 4TB (Premiere and newer, 2TB for THD). Backups of drives with the either of the first two app/media partitions is not supported.

Attachment
The iso image has been removed from sourceforge temporarily to address some of the issues reported in this thread regarding large drives.
Please go to sourceforge for the latest release.
 
See less See more
#1,743 ·
So I can confirm that a Bolt on TE4 21.9.7v10 can self format drives as large as 10 TB.
Unfortunately, MFSTools at this time cannot create a native partition layout of that size. It can do it for a drive that is over 4 TB, but I am not sure how large at this time without further testing. Also it will not at this time convert a drive that was expanded using the lowered numbered partitions to a native partition layout.
 
#1,744 ·
So I can confirm that a Bolt on TE4 21.9.7v10 can self format drives as large as 10 TB.
Unfortunately, MFSTools at this time cannot create a native partition layout of that size. It can do it for a drive that is over 4 TB, but I am not sure how large at this time without further testing. Also it will not at this time convert a drive that was expanded using the lowered numbered partitions to a native partition layout.
Can I make my 4TB operate as a 3TB, leaving the extra 1TB out there? I don't care about losing part of the use of it. I mainly want to copy my existing shows over before the drive dies..
And if the 3TB drive is dead dead, sound like I can out the And I am really new to this. I have read and am about to go through pages of instruction looking for the exact instruction to:
1) Backup a 3TB
2) restore a 3TB (maybe restore to a 4TB)
3) or copy from 3TB to 4TB (although I think you said that won't work)
4) alternatively copy from a 3TB to 4TB only partitioning 3TB..
 
#1,745 ·
You can use mfscopy to copy your current drive to your 4TB drive and have the full 4TB for recordings.

Sent from my SM-G988U using Tapatalk
 
#1,748 ·
I have a recently purchased (past year) Bolt Vox with 500GB hard drive. I have purchased a cloning usb drop in ext USB dual drive to get a perfect clone. I now read that the cloning won't work because MFSTools or some other program fixes a sabotage by the Bolt which looks to see if the drive is the same drive that last booted. The message in the thread indicated a perfect result so I jumped on the Inateck Cloning USB drop in and a 2TB SSD which I thought would solve the issue with CMR/SMR. The SSD tried to boot with the HDD led flickering a few times, then it stopped flickering. The TV screen says Starting Up. I has yet to complete the boot process. I read after installing the SSD into the Bolt, that it will trash all my recordings because I inserted the new drive into it. I guess the old/original drive will not work now? I am in the process of buying a 1TB 2.5 WD Red (CMR) HDD so my incompatibilities will go away. The WD site did not show any 2.5 HDDs that were CMR so I bought the 1TB. There is lots of confusion in the newbies myself included, on what works and what doesn't. I bought the 1TB to avoid the CMR and potential to not work due to size above 1TB. Does the Cloning Process work at all for the Bolt? I am thinking that I need MFSTools to do the job. Could someone PM me the program?
 
#1,749 · (Edited)
Where to start.

Since you cloned the original drive to your ssd and it was stuck in the starting screen, put your old drive back in for now. All your old recordings should be just fine. The only time you lose access to your recordings is if the Bolt formats a drive. We are not sure why the Bolt on TE4 stalls on boot with some drives.

You can use mfscopy in MFSTools to copy your original drive up to 4TB without an issue. Although the Bolt on the latest TE4 can use any drive size (tested up to 14TB) MFSTools cannot directly copy to that size as of yet. You can build a drive that big with MFSTools but you cannot simply use the copy command. That is something I hope to fix at some time.

The clone of your original drive will have the exact same recording space of the original drive. You can build it out to use the whole drive for recording space.

Sent from my SM-G988U using Tapatalk
 
  • Like
Reactions: richsadams
#1,751 ·
Where to start.

Since you cloned the original drive to your ssd and it was stuck in the starting screen, put your old drive back in for now. All your old recordings should be just fine. The only time you lose access to your recordings is if the Bolt formats a drive. We are not sure why the Bolt on TE4 stalls on boot with some drives.

You can use mfscopy in MFSTools to copy your original drive up to 4TB without an issue. Although the Bolt on the latest TE4 can use any drive size (tested up to 14TB) MFSTools cannot directly copy to that size as of yet. You can build a drive that big with MFSTools but you can simply use the copy command. That is something I hope to fix at some time.

The clone of your original drive will have the exact same recording space of the original drive. You can build it out to use the whole drive for recording space.

Sent from my SM-G988U using Tapatalk
Can you PM me the MFSTools to use to copy/clone the original drive to a new drive. I bought a WD-Red 1TB because I could not find a WD 2TB CMR drive. Do you know who builds a good 2-3TB CMR drive that can be used in a Bolt?
 
#1,752 ·
Are you wanting to compile the code yourself or are you looking for an ISO.

Sent from my SM-G988U using Tapatalk
 
#1,754 ·
I am trying to copy and expand my 500gb to 1TB WD Red and I find the command interface to be more than a little daunting. I can handle the GUI apps like DV-Bars? and would like to get something that is easier to use if possible. I am getting error about no such file or directory. I am using my NUC with Win10 to load the thumb drive with the 3.32 iso file set up to boot from the TD. I typed the command in exactly as shown above and still get the errors. I don't see how MFSTools could figure out the proper device since that could be different depending on the computer I am using with various HDD configurations. Is JMFS a GUI program?
 
#1,755 ·
You have to identify the drives first using the lsblk command. Then use the mfscopy command to copy one to the other.

Can you post a picture of the error you are getting

JMFS is used on Premiere units only and is more of a hand held process.

Sent from my SM-G988U using Tapatalk
 
#1,759 ·
What errors were you getting with MFSTools.

JMFS 1.04 has a size limitation. So I cannot remember how large of a drive you can expand. If you search well enough, there is a patched version (1.05) that fixes that issue and should work for your 3 TB drive. That is assuming the 2TB drive has not already been expanded. With JMFS, you get only one expansion.

Sent from my SM-G988U using Tapatalk
 
#1,761 ·
The copy process copies and expands at the same time.

You might have gotten the target and source drives reversed. If b you get the error again, post a screenshot so I can see what led up to the error.

Sent from my SM-G988U using Tapatalk
 
#1,762 ·
I guess this is the best thread to put this in as it is an MFSTools 3.32 problem, I think. I apologize if I clogged up the other Bolt Upgrade thread with an MFSTools question.

I am trying to upgrade my Bolt 1TB to 6TB. I downloaded
MFSTools 3.32-devel USB/HD
RAW image file to make bootable USB drives requires direct copy of file contents to USB drive.
md5 hash for compressed raw file: 2a25613140dfa835b5b41a184ba54ac1

I used the HDD Raw Copy Tool in Win10 to copy MFSTools 3.32-devel USB/HD to a 16 GB flash drive.

I disconnected all my HDs from my PC, connected the TiVo Bolt 1 TB drive to the 0 MB SATA connector, and booted off the MFSTools FD. I did not connect the new target drive yet. Baby steps. Before the login I am getting error messages, I login in anyway, which works, and then try the "lsblk -o,name,size,vendor" command to verify where my TiVo original drive is and it does not work. See the screen shot.

Did not go further without verifying the drive location. Can someone let me know what is happening and what to do?
Hoping to make another attempt tomorrow night. I know zip about linux so I would really appreciate some input. Thanks in advance
 
#1,763 ·
Syntaxes looks wrong. Should be:
lsblk -o name,size,vendor

Does not look like what you typed (although it is a little blurry to me)

Linux is very particular about syntax

Sent from my SM-G988U using Tapatalk
 
#1,764 ·
Thanks. Yes syntax. Put a comma after the "-o" should be a space. My bad. Will try again tomorrow night. Really appreciate the quick reply! Sorry about the blurry image. It looked clearer on a bigger screen. What about the error messages above it. The "read/64, error -110" error three lines, and the "device not accepting address 9, error -71" What do these mean? or Does it matter?
 
#1,765 ·
What USB devices are plugged into the computer

Sent from my SM-G988U using Tapatalk
 
#1,767 ·
jmbach, thank you for your prior responses.

I've been trying to copy a 2TB drive to a new 3TB drive. I have not been able to get mfscopy -ai /dev/sdX /dev/sdY to work. I get the volume header error. When I use mfscopy -ai /dev/sda /dev/sdb it started to copy. Does that work the same and if not how do I get the same results as if I used sdX and sdY? Thanks
 
#1,769 ·
Does that work the same and if not how do I get the same results as if I used sdX and sdY?
Using sdX and sdY will not work as these are just examples for the source and target drives. Since you would not have drives specified as sdx or sdy.(Hence the capitals.)
 
#1,770 ·
MFSTOOLS COMMAND QUESTION
I am upgrading my 1TB Bolt to 6TB.
I have successfully copied the 1TB drive to the 6TB drive using the command "mfscopy -a /dev/sda /dev/sdb". The space went from 90% filled to 56%.
I know I have to add 2 TB partitions using mfsadd and mfsaddfix.
Please confirm that I add the partitions one at a time, i.e.
mfsadd
mfsaddfix
mfsadd
mfsaddfix​
I have now again read the other thread "Bolt with 3TB, want to upgrade to 8TB or more" and in that thread the command to use was "mfscopy -am 2040 /dev/sdX /dev/sdY"
What is the difference?
Should I re-copy the 1TB drive to the 6TB drive using "am 2040" version of the command before trying to add 2 TB partitions.
Should I use the "mfsadd -xm 2000 /dev/sdb" command where b is the 6TB target command, or simply "mfsadd /dev/sdb"?
I am in the dark here. Also where can I get a complete set of current commands for the mfstools 3.32 I have downloaded from

MFSTools 3.32-devel USB/HD
RAW image file to make bootable USB drives requires direct copy of file contents to USB drive.
md5 hash for compressed raw file: 2a25613140dfa835b5b41a184ba54ac1

AS always, thanks in advance, really appreciate the assistance.
 
#1,771 ·
Can you post the mfsinfo -d on the drive.

Generally speaking you should use am 2040 when doing the copy process. As mfstools does not copy correctly to anything over 4 TB so we have to limit the initial copy to 4 TB which is what the command does.

Sent from my SM-G988U using Tapatalk
 
#1,774 ·
I am running TE4. I am just going to run the "mfscopy -am 2040 /dev/sda /dev/sdb" and then run
"mfsadd -xm 2040 /dev/sdb"
"mfsaddfix /dev/sdb"

as you have instructed.


If MFSTools is writing an image it should overwrite everything that previously been written. Correct me if I am wrong

What difference does the TE4 vs TE3 version make?
Thanks for all your help.
 
#1,775 ·
It should overwrite the drive. It is always better to erase it first.

Sent from my SM-G988U using Tapatalk
 
#1,777 ·
further to my above post, with the 6TB drive (without runing the "mfsadd...") the Bolt TE4 says capacity is 701 HD, with the original 1TB drive it was 156 HD. 701 HD is more than 4 times as much but less than 6 times as much. I presume there it would not be exact as there is overhead data on the original drive which does not expand. What is the expected capacity with a 6TB drive?
 
#1,779 ·
So when you use m 2040, it will create recording partitions that are larger than what would be created on a normal 4TB drive but still respects the 32 bit limit of the partition size ( which is 4294967295 blocks per partition).
 
#1,781 · (Edited)
Anyone know how get the vhd file to work? I can attach it in the Disc management section but then I see no way to assign a drive letter to it. That is grayed out.

I Might have to try and go old school and try using the bootable flash drive method or worse use the CD ISO. With an old machine I have from 2009.
 
#1,783 ·
The VHD file of MFSTools needs to be attached to a windows virtual machine like Hyper-V and ran. It does not get attached to Windows itself.

Sent from my SM-G988U using Tapatalk
 
  • Like
Reactions: aaronwt
#1,782 ·
You need to use DVRBars to restore this backup file.
 
#1,784 · (Edited)
I'm copying my 1TB Bolt Drive to a 2TB Toshiba drive right now.
I guess it's going to take seven or eight hours?

I ended up going with the bootable USB flash drive method. I realized I only had one desktop machine with Windows Pro on it for Hyper-V. A 2013 HP machine that I only use for my email. And I am not familiar with using VMs.

I did have to use bootsectorfix to get to work with the TiVo drive.
I had to use one of my old unRAID machines from 2011 for the bootable flash drive to work. I could not get it to work with the old 2013 Dell machine I have. And I did not want to use any of my newer, 2018 Dell machines with it.
 
#1,785 · (Edited)
Sweet!!! It automatically expanded the Toshiba MQ03ABB200 drive to use the 2TB. Everything came back up fine with Bolt. All the settings and recordings are still in place. And the cable card is still paired.
Thanks!!
 
#1,787 ·
You don't know until you try. However if you have already had that bolt format a new drive, you have lost access to the recordings on that drive.

What size is the drive in question?

Sent from my SM-G988U using Tapatalk
 
#1,788 ·
The potentially bad drive is an original 3TB drive from a Bolt+ I purchased new in 2017. I wish I had copied it when it was still working.

My goal is to copy it to a new WD Purple 8TB drive.

I need a way to access the drives. Will this work to connect to my laptop? It's a Sabrent USB 3.0 to SATA I/II/III Dual Bay External Hard Drive Docking Station:
https://www.amazon.com/Sabrent-Exte...p/B0759567JT/ref=psdc_160354011_t1_B07WS59SP4
 
#1,789 ·
That should work just fine.

I recommend that you first copy it to a spare drive that is 3TB or more. I would first try to use the cloning feature of the dock. If it clones successfully, then boot that drive in your Bolt. Make sure you can see your shows and it connects to the TiVo servers successfully. If it does, then use that drive to copy to your 8 TB drive.

Sent from my SM-G988U using Tapatalk
 
#1,791 ·
You can. It is just my preference to use temporary drives to try to recover drives with bad spots. If cloning does not work, then you will need to use ddrescue to try to save the information on the drive.

I would run the 8 TB through a diagnostic read-write-read before doing any copying to it.

Sent from my SM-G988U using Tapatalk
 
  • Like
Reactions: gsutkin
#1,798 ·
I am assuming that anyone who clones a TiVo drive is going to want to test it by booting the TiVo with it to see if it is a good usable copy. And as I said at the end of my first post - if the Bolt does not boot because it will not accept an SMR drive, how will he ever know if the clone is actually a good usable copy or not?
 
#1,799 ·
Testing the drive should be fine. SMR drives are problematic in the long term. Short term testing is okay.

Sent from my SM-G988U using Tapatalk
 
  • Like
Reactions: gsutkin
#1,801 ·
Thanks. I’m hoping jmbach is correct and I can use this WD30EFAX as a temporary drive to test in the TiVo and copy to the larger purple drive. If it doesn’t boot in the TiVo, then I will exchange it for a red plus drive and clone again.

Btw, cloning has been going for 18 hours now, and is not complete (the 4th blue LED light has been flashing since this morning). I need to be patient, right?
 
Top