1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

DIY 4TB Premiere

Discussion in 'TiVo Upgrade Center' started by jmbach, Jan 21, 2014.

  1. Jan 21, 2014 #1 of 207
    jmbach

    jmbach der Neuerer

    1,544
    9
    Jan 1, 2009
    Well after a lot of trial and error, I finally managed to come up with a way to expand a premiere drive up to 4TB. This was the result.

    [​IMG]

    I was able to keep all my recorded shows. It took me about 2 days to accomplish this. I will do my best to outline the procedure as soon as I get it written up. It is not for the faint of heart and does require some technical know how. While doing this procedure, I moved the OS to approximately the center of the 4TB drive and 4k aligned the image. It probably was not necessary but I figured if I am maxing out my premiere I want to try to reduce seek times as much as possible. It also requires you to coalesce some partitions and have an extra 2TB drive. Once I feel the write up has the detail needed, I will post it here.
     

    Attached Files:

  2. Jan 21, 2014 #2 of 207
    jmbach

    jmbach der Neuerer

    1,544
    9
    Jan 1, 2009
    [size=+2]Updated procedure 6/9/2015 by jkozee and MFSTools 3.2 which simplifies the process even more. Goto here. If you are unsuccessful with that procedure then try the procedure by cykotix.[/size]

    Depreciated procedure by cykotix which simplified the initial process. Goto here.


    The following information is now obsolete and is currently here for reference.
    Read this post in its entirety first. I would not recommend you to do this if you have any reservations or questions about any step. You attempt this completely at your own risk. There are no warranties or guarantees of success.

    That being said, here is the revised procedure to expand a premiere to 4TB. I am hopeful that comer will come out with a new version of JMFS that addresses some if its current limitations which would greatly simplify this procedure more.

    But first credit and thanks go to ggieseke, comer, cniessen, and JamieP for without their groundwork I may not have been able to accomplish this. Also for ggieseke and cybernut for doing the upgrade which has helped define the steps better. Finally for marwatk who developed Linux scripts to make the copy and rearrange step easier.

    First, in order to expand to 4TB you have to start with a 2TB image. If you have a 2TB XL4 image I suggest you wait until comer has the new version of JMFS. The 2TB XL4 images do not require the rearranging or coalescing of the partitions but it requires you to modify and correct the checksum of the MFS header. If you know how to do this, then you can try it. Instructions are near the end of the post.

    Before proceeding, read cniessen's write up on S3 Consolidation while preserving recordings to better understand MFS structure and limitations.

    The minimum OS to be on to try this procedure is version 20.3.8

    This is for people who have already expanded their current images to 2TB.

    Tools you may need depending if you do this all with Linux or if you want to use some windows tools.
    -Any variant of dd to copy partitions/drives. For windows you can use DvrBARS for copying drives, hex editing program like HxD or disk editing program like iBored for copying partitions/modifying the APM. I like iBored as it has templates that display the APM values.
    - marwatk Linux scripts for the copy and rearrange step of the procedure. Since this was developed, it has greatly simplified that step.
    -pdisk modified for TiVo (found in MFSLive) to edit the partition table. In windows, you can use a hex or disk editing program. However, you will need to know how to manually adjust the APM if you use hex or disk editing programs.
    -hdparm on the JMFS disk to adjust the visible sectors on the temporary expansion drive.
    -JMFS tools

    I will provide details on how I did it but it can be done using any of the tools listed above. When working in windows, make sure all the programs are opened as administrator or ran out of an administrator command window.

    You will first copy the 2TB drive over to the 4TB drive. While doing this you will have to rearrange the partitions a little. Our end goal is to shrink the number of partitions to 14. (An expanded Premiere has 15 partitions) so we can add our own 15th partition. Read cniessen's write up on how to do this. After some discussion with other TCF members, I have come to the conclusion that although my initial method works it is not as eloquent as it should be. So I am revising the method. The final partition layout on the target drive will be 1,11,12,13,15,2,3,4,5,6,7,8,9,14,10. The reason for this is that it will leave partition 10 intact without modifications so that current tools can access the information easier.

    Additional information:
    TCF member marwatk developed a set of Linux scripts that make this step easy. See his post here for how to use the scripts. One side note, when using his scripts, the resultant image is smaller than the original and so the numbers below will be slightly different. If you use the numbers below, you might lose a few minutes of recording time. However, if you use these scripts, I would only use them to copy and rearrange the partitions on the destination drive and not use the coalesce scripts at this time. Once there is a patched version of JMFS publically available which would make using the external 2TB drive unnecessary, then you can use the coalesce script. You would then follow the directions in marwatk post to expand to 4TB and ignore everything else below. (the procedure in this case would be the same to expand to 3TB as to expand to 4TB.)

    Calculate the number of sectors that are free at the end of the current image on the 4TB drive to the physical end of the drive. Take the number of sectors of a 4TB drive (usually 7,814,037,168) minus the end of the 2TB image (usually 3,907,029,168) which gives you a value of 3,907,008,000. We do this so when we run the JMFS command, the zone maps will be updated with the correct number of sectors.

    Hook your 4TB drive and a 2TB drive to a computer and boot JMFS. The 2TB drive needs to be connected either via SATA or eSata for hdparm to work correctly. Once JMFS is booted, drop to the command line. Using hdparm temporarily limit the number of sectors of the 2TB drive to the number you calculated above plus 64. So using the above numbers it would be 3907008064. (Why 64 you asked? Well we have to have space for the partition map on the 2TB drive). If the 2TB drive is /dev/sdc then the command would be “hdparm –N 3907008064 --yes-i-know-what-i-am-doing /dev/sdc”. You can check to make sure it was done correctly running hdparm again with this command “hdparm –N /dev/sdc”. This should show you the number of visible sectors and total actual sectors of the drive.

    Once the 2TB drive has the appropriate number of sectors visible based on the preceding step, expand the 4TB image. If the 4TB drive is /dev/sdb and the 2TB is /dev/sdc, then the command will be "./mfsadd.sh /dev/sdb /dev/sdc". Once it is done you can check the results by typing "./mfslayout.sh /dev/sdb /dev/sdc". Once you verified everything is correct then on to the next step.

    Next we will move the partition information from partition 2 of the 2TB extended drive to partition 15 on the 4TB drive. Then copy the first 2MB of partition 2 on the extended drive to the beginning area of partition 15 on the 4TB drive.


    When that is done, then put the 4TB drive into the TiVo, let it boot and divorce the external drive and you should have a working 4TB drive.
     
    Last edited: Jun 16, 2015
  3. Jan 22, 2014 #3 of 207
    Cybernut

    Cybernut New Member

    42
    0
    Oct 1, 2013
    This is fantastic jmbach!! Kudos!!

    I know you said this is not for faint of heart and will require some technical know-how, which I know I lack. I am hoping though your instruction writeup can guide relative newbies like me to at least a good extent. I know I just installed the 3TB in my Premiere couple weeks ago but would love to try out this more challenging task of getting 4TB in it.
     
  4. Jan 22, 2014 #4 of 207
    Cybernut

    Cybernut New Member

    42
    0
    Oct 1, 2013
    Maybe I'll hold off on upgrading to 4TB from my recently-upgraded 3TB. After reading this post by ggieseke (http://www.tivocommunity.com/tivo-vb/showthread.php?p=9975847#post9975847) I realized that the tools I used to copy over my recordings from my 1TB drive won't work with 2+TB drives. And I've already added quite a number of recordings on my new 3TB drive from shows on channels that I can't dump to a computer using kmttg. So in other words I can move up from my 1TB drive to a 4TB drive (assuming it's easy enough for me to even try once you have the steps written up) but not from 3TB to 4TB drive. So will wait until ggieseke's tools that works with drives larger than 2TB are ready. Unless of course, jmbach, you have solved that problem as well.
     
  5. Jan 22, 2014 #5 of 207
    realfun4u

    realfun4u New Member

    9
    0
    May 11, 2009
    Sanford Florida
    Awesome Post! i want to MAX out my Roamio Pro now lol
     
  6. Jan 22, 2014 #6 of 207
    Cybernut

    Cybernut New Member

    42
    0
    Oct 1, 2013
    This is not for Roamio....this is for Premiere.
     
  7. Jan 22, 2014 #7 of 207
    jmbach

    jmbach der Neuerer

    1,544
    9
    Jan 1, 2009
    Cybernut is correct, this is for premiere only. Ggeiseke is working on something that will work on Roamio (and probably premiere as well).

    Going from a 3TB to 4TB should be possible but will require rearranging the partitions as you copy from the 3TB to 4TB. More to come....

    Sent from my SPH-L710 using Tapatalk
     
  8. Jan 28, 2014 #8 of 207
    Cybernut

    Cybernut New Member

    42
    0
    Oct 1, 2013
    Thanks again to jmbach's immense help, I was able to upgrade my Premiere to 4TB just two weeks after moving to 3TB with his help.

    He has perfected this process. And I agree with his initial post above...this definitely isn't for the faint of heart. It is a pretty complicated process and one must have some technical know how to attempt this...and preferably the right tools like an eSATA dock if trying to move from 3TB to 4TB (USB docks should be ok when moving from the original 1TB drive).
     

    Attached Files:

  9. Jan 29, 2014 #9 of 207
    shamilian

    shamilian Member

    442
    0
    Mar 26, 2003
    NJ
    Has anyone written up the procedure yet ?

    I am looking to go from a previous 2TB P4 to a 4TB P4.. ( not XL model )
     
  10. Feb 3, 2014 #10 of 207
    jmbach

    jmbach der Neuerer

    1,544
    9
    Jan 1, 2009
    Write up in 2nd post of this thread.

    Sent from my SPH-L710 using Tapatalk
     
  11. Feb 14, 2014 #11 of 207
    shamilian

    shamilian Member

    442
    0
    Mar 26, 2003
    NJ
    I used iBored (windows ) and dd (linux) to get down to 14 partitions.
    Instead of using a second disk drive, fixed an issue in JMFS (linux).

    And it works 638 HD Hours.
     

    Attached Files:

  12. Feb 19, 2014 #12 of 207
    wwu123

    wwu123 Member

    39
    0
    May 14, 2008
    I probably don't have the guts to try this anytime soon, but just in case I had a question about Supersizing. I did the 2TB Premiere upgrade in the early days before Supersizing was an available option, so my recording capacity is slightly smaller than it could be.

    If I used that 2TB image to expand to 4TB with this procedure, would I lose even more capacity relative to the 638-640 HD hours people are getting? Would it be advisable to Supersize first?
     
  13. Feb 19, 2014 #13 of 207
    jmbach

    jmbach der Neuerer

    1,544
    9
    Jan 1, 2009
    Since you are using JMFS for part of the procedure, you can supersize while everything is plugged in. I recommend that you supersize right after the image is copied to the 4TB drive and before anything else is done.
     
  14. Feb 21, 2014 #14 of 207
    44ml

    44ml New Member

    6
    0
    Feb 17, 2014
    I have the stock 250GB that I would like to upgrade. Can I move that to a 2TB drive then move the 2TB to a 4TB drive? I have the TCD746320 Premiere.

    Also, is there somewhere I can download a DvrBARS backup of a factory TCD746320?
     
  15. Feb 21, 2014 #15 of 207
    ggieseke

    ggieseke Active Member

    4,018
    11
    May 30, 2008
    PM sent. Jmbach will have to walk you through upgrading it to 2TB, consolidating it, and expanding to 4TB. The only one I've done started with a 2TB XL4 image.
     
  16. Feb 21, 2014 #16 of 207
    44ml

    44ml New Member

    6
    0
    Feb 17, 2014
    PM received. Thank you!

    With DvrBARS, am I right in assuming I can only restore to the same size drive as the backup? I planned on restoring to a 320GB drive, expanding to 2TB with comer's guide (showthread.php?t=455968 [can't post links yet]), then using this guide to expand to 4TB.

    Once I get a 4TB drive working, can I use DvrBARS to make a backup that others can use to simplify the process?

    Most importantly, where's your donate button?
     
  17. Feb 21, 2014 #17 of 207
    jmbach

    jmbach der Neuerer

    1,544
    9
    Jan 1, 2009
    I would restore the image ggeiseke sent you to a 2TB drive using DvrBARS and then let JMFS expand and super-size it. Then when you get a 4TB we can go to the next step. DvrBARS donate button is in the system information menu or you can use the one at the end of the post the guide is in. (which I emailed to ggeiseke but he hasn't used yet in his OP in the DvrBARS thread)

    Sent from my SPH-L710 using Tapatalk
     
  18. Feb 21, 2014 #18 of 207
    ggieseke

    ggieseke Active Member

    4,018
    11
    May 30, 2008
    Once you go past 2TB DvrBARS can't back your drive up anymore. When I wrote it that was the limit on Premieres, so using the Microsoft VHD format was OK (that's the limiting factor).

    You can restore the factory 320GB image to any drive that's at least that big. I would restore it to a 2TB drive and use jmfs to expand it. After that jmbach can walk you through the 4TB expansion if you're handy with hex editors and willing to take the plunge.
     
  19. Feb 21, 2014 #19 of 207
    jmbach

    jmbach der Neuerer

    1,544
    9
    Jan 1, 2009
    That is not a bad idea. Might be able to create a 2TB base image people can use to expand to 4TB. Hmm. Have to see how DvrBARS deals with a highly modified image.
     
  20. Feb 22, 2014 #20 of 207
    wwu123

    wwu123 Member

    39
    0
    May 14, 2008
    To understand the step of copying over the 2TB image to the 4TB disk:
    "I booted windows and used DvrBARS to do a modified backup of the 2TB image and a quick restore to the 4TB drive. You will have to have enough room to store the backup using this method which can be the full 2TB depending on how many recordings you have. (Your other option is using the copy function of JMFS (which uses a variant of dd).)"

    For those of us wanting to do this on a nearly full Tivo, you'll have to have another 2-4 TB pretty empty disk in your comiputer to handle the 2 TB backup image using DvrBARS? But with JMFS you can do a straight copy from HDD to HDD - any downside risks to just doing the copy? I'd hate to have to buy yet another 2+ TB disk as I don't have that much space free on my computer.
     

Share This Page