1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Roamio 4Tb Drive Development Thread

Discussion in 'TiVo Upgrade Center' started by eboydog, Apr 25, 2014.

  1. Jun 4, 2014 #41 of 105
    ggieseke

    ggieseke Active Member

    4,083
    24
    May 30, 2008
    In a perfect world, yes. There's always some waste even on the factory layout. It has always been that way.

    It's slightly less than 16MB in total on your project so I wouldn't sweat it. Coincidently, the layout that a Roamio TRIES to create on a 4TB drive wastes exactly the same amount of space even though it uses a different partition scheme. That's about 9-10 seconds of HD.

    The Weaknees 4TB drive only wastes 5MB, so they could probably squeeze out a least 6 more seconds of recording time. :D
     
  2. Jun 4, 2014 #42 of 105
    ggieseke

    ggieseke Active Member

    4,083
    24
    May 30, 2008
    The zone maps show that the space less than even 10MB blocks at the end of EACH media partition is unused. There's no overlap, which makes sense since they are separated both logically and physically.

    That's what I based my last post on.
     
  3. Jun 4, 2014 #43 of 105
    jmbach

    jmbach der Neuerer

    1,623
    20
    Jan 1, 2009
    Then if one of the media partitions is a multiple of a chunk then more disk would be used and less waste? Even though WK drive has less waste in the partition, does it equal out in the end because they have more free space unallocated in their images?
     
  4. Jun 4, 2014 #44 of 105
    ggieseke

    ggieseke Active Member

    4,083
    24
    May 30, 2008
    The total number of sectors on the drive and the size of the other partitions never works out perfectly with the block size on the media partitions. There will always be unused sectors somewhere.

    The Weaknees layout manages to cut the total wasted space down from 15.78125MB to 5MB, but both media partitions still have some slack. If you make one partition perfect the other one will just have additional unused space.

    A few seconds of recording space either way isn't going to matter to anyone.
     
  5. Jun 4, 2014 #45 of 105
    nooneuknow

    nooneuknow TiVo User Since 2007

    3,554
    0
    Feb 5, 2011
    Cox Cable...
    Just so as not to confuse the rest of the people following this (or trying to), including myself: Are you sure it's physical (4K/4096 byte) sectors, or did you mean the logical emulated ones (512 byte).

    Are "blocks" and "clusters" the same thing, or are they different?
     
  6. Jun 4, 2014 #46 of 105
    jmbach

    jmbach der Neuerer

    1,623
    20
    Jan 1, 2009
    Clusters are groups of blocks.
    I am referring to block size of 512 bytes as this is what all the TiVo calculations are based off of.
     
  7. Jun 4, 2014 #47 of 105
    mrsean

    mrsean Member

    220
    0
    May 15, 2006
    New Jersey
    Anyone notice that Seagate has a new 6GB hard drive out? I wonder if it's possible to get it to work in the Roamio. Imagine 952 hours of HD recording and 6540 SD.:D
     
  8. Jun 4, 2014 #48 of 105
    nooneuknow

    nooneuknow TiVo User Since 2007

    3,554
    0
    Feb 5, 2011
    Cox Cable...
    Thanks.

    So, your post should say "logical" instead of "physical", then, right?

    I know that host devices (like TiVos) often falsely identify emulated sectors as being "physical", but other devices do correctly detect that the physical sector size is 4K/4096 bytes, and the logical (emulated) size is 512 bytes. My laptop's Intel Rapid Storage Technology driver and program does, while Windows 7 seems to see them as 512 byte physical (on the surface, at least).
     
  9. Jun 4, 2014 #49 of 105
    jmbach

    jmbach der Neuerer

    1,623
    20
    Jan 1, 2009
    Probably be easier if I just define the block size I am referring to and not worry about physical / logical.
     
  10. Jun 4, 2014 #50 of 105
    jmbach

    jmbach der Neuerer

    1,623
    20
    Jan 1, 2009
    Hmmm. As you say it is not going to matter overall. I like seeing things used efficiently and it is not since there is some waste involved. Can't have everything.

    If one partition is a multiple of 20480 512 byte blocks then that partition would have zero waste and the other would have about 5MB of waste.
     
  11. Jun 7, 2014 #51 of 105
    telemark

    telemark New Member

    1,544
    1
    Nov 12, 2013
    Thanks for clarifying all that. I found a combination I was happy enough with, taking into account everything.

    I'm finishing off a Linux installer, but I know more people are Windows. Would DvrBARS be able to write a 2.2TB vhd image to a 4TB drive?

    I noticed, but also noticed they're 7200rpm? Maybe the helium type versions have low enough heat and power, I didn't check.

    Should be able to make this work. The other idea that's been floating around is supporting an eSATA RAID enclosure.

    If someone wants to buy or already has this hardware, let us know, cause I think a few people have been thinking about this.
     
  12. Jun 8, 2014 #52 of 105
    ggieseke

    ggieseke Active Member

    4,083
    24
    May 30, 2008
    That should work fine. What I would do is to create a dynamically expanding VHD using Windows Disk Manager that's juuusssssst big enough to hold the last byte that you need to write. 1900MB should be more than enough.

    Create a VMWare or Windows Virtual PC box, attach that VHD, and run your Linux installer. Instant DvrBARS compatible file. I can do all of that in a few minutes if you don't have virtualization software handy.

    P.S. Disk Manager allows you to enter fractions, so you could even make it 1865.91791534423828125MB or whatever value you actually need.
     
  13. Jun 9, 2014 #53 of 105
    cykotix

    cykotix New Member

    18
    0
    May 22, 2014
    I don't have a Roamio so I haven't read up on the differences between it and the Premiere. However, just doing a quick look you guys pointed out that the MFS header is different between a Premiere (0x1492) and Roamio (0x9214). Is that the biggest hurdle with using JMFS? Drive detection? Because if that's the case, we can patch https://github.com/krbaker/jmfs/blob/master/src/tivo/disk/Block0.java and replace:

    Code:
    public static short APPLE_BLOCK0_SIGNATURE = (short)0x1492;
    
    with

    Code:
    public static short APPLE_BLOCK0_SIGNATURE = (short)0x9214;
    
    Or am I oversimplifying?
     
  14. Jun 9, 2014 #54 of 105
    telemark

    telemark New Member

    1,544
    1
    Nov 12, 2013
    That's the idea, but it's every field of bytes longer than 1, that also need to be swapped.

    If someone literally copied the JMFS code to x86 C/C++, it would almost work without modification.

    Java is feature rich enough, that it should be less work to patch it for a reversed byte order.
     
  15. Jun 9, 2014 #55 of 105
    jmbach

    jmbach der Neuerer

    1,623
    20
    Jan 1, 2009
    You could use the signature in block 0 to toggle between byte order.
     
  16. Jun 9, 2014 #56 of 105
    cykotix

    cykotix New Member

    18
    0
    May 22, 2014
    That's what I was thinking, but the bigger issue here is figuring out what parts of the code need to be reversed. If we know exactly what parts are different (which it seems like we do), java appears to have a lovely function, reverseBytes, that can be used to patch those sections based on the block0 signature.

    Also, I'm not a java programmer by any stretch of the imagination. I have no problems reading through code though, patching and subsequently compiling. I don't have a way to test though.
     
  17. Jun 9, 2014 #57 of 105
    telemark

    telemark New Member

    1,544
    1
    Nov 12, 2013
    There are endianness aware object libraries, if used correctly, would only be a couple lines to patch per class that access the disk. This would be the best way to avoid introducing any new bugs.

    ggieske said some fields moved though, which implies to me there's some existing misinterpretations in field length when parsing. It shouldn't be moving, it should be swapping to another position, ideally. So these would have to be fixed at the same time if they exist in JMFS. (another explanation is expanding a field from 32 to 64bits)

    Tivo of course can do whatever they want, but probably took the route of least effort which is to change the minimum.

    I haven't setup a workstation for java yet, and I've been working on other things I thought should come first.

    If you need a test case, parsing a Roamio image (downloaded VHD maybe), would be a decent first milestone. If someone's working on this, let me know and I can make an image better suited to this.

    There are more high-level than low-level programmers these days. The Wikipedia article seems to cover a number of the common examples, so maybe that's a time saving starting point. http://en.wikipedia.org/wiki/Endianness

    Reminder:
    Premiere is MSB / big / Motorola / java order,
    Roamio is LSB / little / Intel / C order,
    both are actually Broadcom (SiByte) MIPS.

    Libraries: http://docs.guava-libraries.googlec...le/common/io/LittleEndianDataInputStream.html
    http://niftilib.sourceforge.net/java_api_html/EndianCorrectInputStream.html
    http://mindprod.com/jgloss/endian.html
    http://my.safaribooksonline.com/book/programming/java/1565924851/data-streams/ch07-22270 / http://jcs.mobile-utopia.com/jcs/10155_LittleEndianInputStream.java

    EDIT: a smaller first milestone would be to, replace instances of DataInputStream with a bi-endian aware subclass. Confirm that it works on the Premiere images still, than flip the endianness bit and track down what breaks.

    Also, Android is bi-endian friendly and uses Java. I personally think it is probably unnecessary, but if stuck, there might be something useful there.
     
  18. Jun 9, 2014 #58 of 105
    telemark

    telemark New Member

    1,544
    1
    Nov 12, 2013
    There's 2-3 theories on the purpose of the guid.

    Which one is right, might change what's the best practices in restoring drive images.

    One is it could be used to generate a Host ID somewhere. Of particular interest is if it's used by the CableCard to recognize whether it's a new host.

    Alternatively, it could be more filesystem related. This is common in desktop OS's. The only example I could come up that would effect life, is if it's used in external drive marriages. There should be some mechanism that prevents secondary expander drives from being incorporated into a different Tivo (an extra-marital affair, so to speak). Allowing such a thing, would cause corruption of the original pairing.

    Or it could be vestigial. I feel like the marriage is just a bit more likely, but would prefer if it fixed the cablecard pairing, because that's the one that could use some improvement, and it would be harder otherwise.
     
  19. Jun 9, 2014 #59 of 105
    jmbach

    jmbach der Neuerer

    1,623
    20
    Jan 1, 2009
    I don't think it is host ID related as my host ID does not change on my cableCARD pairing screen for my Motorola M-Card only the data ID. Not sure about Scientific Atlanta cards.

    The marriage might be. Would need a couple of scenarios to test. One is to take an image and wipe the guid then see if an external drive can be paired. If it can, look at the image and see if a guid was generated. Two, pair an external drive and wipe the guid a difference see if it remains paired and accessible.
    Looks like we need to get ggieseke's knowledge and cykotik and telemark's coding skills (since all ggieseke does with Java is drink it :p ) to get JMFS modified.
     
  20. Jun 10, 2014 #60 of 105
    whereisthelimit

    whereisthelimit New Member

    1
    0
    Jun 9, 2014
    If the data id changes, then to me it seems related to the DRM protection for the stored video, to prevent the data on the disk from being playable when hooked up to another host.

    Sent from my SAMSUNG-SGH-I747 using Tapatalk
     

Share This Page