1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Roamio 4Tb Drive Development Thread

Discussion in 'TiVo Upgrade Center' started by eboydog, Apr 25, 2014.

  1. Jun 10, 2014 #61 of 105
    jmbach

    jmbach der Neuerer

    1,557
    10
    Jan 1, 2009
    The data ID change does not affect the recordings on the drive (premium or not) just my premium channels I receive.
     
  2. Jun 11, 2014 #62 of 105
    nooneuknow

    nooneuknow TiVo User Since 2007

    3,554
    0
    Feb 5, 2011
    Cox Cable...
    It's possibly a YMMV situation, like I described in this post:

    http://www.tivocommunity.com/tivo-vb/showthread.php?p=10137631#post10137631

    While my Cox market makes little use of the CCI-bit, the cablecard pairing authentication is tightly locked-down.

    Cablecard pairing is lost on sector-by-sector drive clones (all the way back to the TiVo HD), even if the drive model and size is identical.

    The CCI-bit protected content is only able to play back if the cablecard that was used to record it is still installed, and authenticated.

    None of this was the case a few cablecard firmware revisions back.

    I'm very likely in a rare market condition here (just like me being in a full 1GHz RF spectrum market, that still carries analog, which uses SDV). If I can do anything to help, being in this market situation, please don't hesitate to ask me (anybody).

    I really want to help. I have an uncle who could probably do what everybody is trying to, and make it work in both Linux and Windows, if he only knew anything, at all, about TiVo. He's willing to tackle any challenge I present to him, provided I can get him to understand the TiVoization factors involved...

    He is an actual programmer, at a very large corporation, who often gets told "That can't be done", only to write the program to do exactly that, and make it work company-wide, and cross-platform.
     
  3. Jun 11, 2014 #63 of 105
    telemark

    telemark New Member

    1,544
    1
    Nov 12, 2013
    Porting JMFS should a straightforward task, which is why I suggested it. I downloaded it just now, and it builds. It should take like a day or weekend to patch it. I honestly never used it before though, so I might have some questions re what it it's intending to do.

    > I really want to help. I have an uncle who could probably do what everybody is trying to, and make it work in both Linux and Windows, if he only knew anything, at all, about TiVo. He's willing to tackle any challenge I present to him, provided I can get him to understand the TiVoization factors involved...

    You make it sound like he doesn't have any Tivos in his house? There are useful tasks for any skill set, you don't have to be programmer.

    > He is an actual programmer, at a very large corporation, who often gets told "That can't be done", only to write the program to do exactly that, and make it work company-wide, and cross-platform.

    Ya, I like those stories. Can you say what city?

    This probably sounds surprising, but I'm not an actual programmer. Because of my skill set, I get tasked with other things, but I do get to technically manage junior developers.

    I have a pretty long list of TODOs, more than I will ever finish, but at the same time I know how to do almost everything I added to the list. Those I don't know how to do, are really hard, unless someone has some specialized skills in some of the tech.

    Maybe we should be having the conversation, what larger projects are worthwhile.
     
  4. Jun 11, 2014 #64 of 105
    lessd

    lessd Active Member

    7,697
    5
    Jan 23, 2005
    CT
    You can play any recorded program on any TiVo that it was recorded on even, if the TiVo now has no service or has no cable card installed. the CCI-Bit has nothing with the play-back, except for some PPV movies that have built in time outs for playback.
     
  5. Jun 11, 2014 #65 of 105
    jmbach

    jmbach der Neuerer

    1,557
    10
    Jan 1, 2009
    This has been my experience as well. As such I believe the recordings are attached to the TiVo serial number rather than the cableCARD.
     
  6. Jun 11, 2014 #66 of 105
    nooneuknow

    nooneuknow TiVo User Since 2007

    3,554
    0
    Feb 5, 2011
    Cox Cable...
    I respect both of you. I'd rather have you loosely "in my corner", than being firmly in the opposite one.

    I've gone out of my way to make sure what I say is as accurate a depiction I can make, using the best possible reasons I can come up with, for what I experience, and can replicate, in my posts.

    Perhaps what is being done in my market is (possibly) illegal, and Cox is only trying to verify it works, they can get away with it, and/or plans to take it wide scale, if no legal hurdles come up. Perhaps it's a TiVo bug, manifesting in my "less than usual" Cox Fibre/Coax 1GHz network w/analog, digital, and SDV, all in use, situation.

    Unless a person is in my market, using the same infrastructure, same cablecards, same firmware, and replicating the ecosystem of the network, as well as on-the-premise equipment, who are they to say it's not happening, or is not possible to happen?

    I've stayed silent on the matter, until telemark came along. I'm looking for fresh eyes, fresh thoughts, and not the Same-Old-Sh*t, I got rained on me the last time I brought this specific matter up, when it first reared its ugly head.

    If I have to take all the data points I can provide to PM, or other private back-channel, I can do that. I'm trying to do "the community thing".

    I can replicate the results without having to make special effort to make them happen. If I still had my older-than-Roamio platforms, I could replicate the results all the way back to the TiVo HD.

    I have never failed to replicate the results I spoke of. I've never had to make everything "exactly just-so", to get/replicate the results.

    Perhaps it's just the choices of descriptions I use, the way I say them, or the best-educated-guess reasons I share, for why I can get what I get, and replicate it.

    The bottom line is that the few CCI-bit protected recordings I end up with, will playback for years without issue. If I remove the cablecard that was authorized when the recordings were made, they won't playback (TiVo error claims no signal was present at the time of the pre-existing recording). Re-inserting and re-pairing the same card, restores the playback (of a recording that has played-back before, without issue). Inserting another card, and pairing it up (or not pairing), results in the aforementioned error, on a recording that had been playing-back for a year, or more.

    I see this as a reason to fully understand the suspected GUID, any linkages to cablecards, any linkages to pairing, and/or any linkages to imaging/cloning.

    I feel being able to get to the bottom of this requires my input (no matter how impossible some claim my experiences are), and/or more participation from others in my market, with the right factors present.

    A perfect example recording is "The Hitchhikers Guide To The Galaxy" (the movie), always recorded from IFCP (considered a "premium", included in a channel "pak"), which does set the cci-bit. I can count on losing access to playing it back, under the circumstances I describe, and can count on restoring playback of it, if I still have the same drive, same cablecard, all in-place (and I haven't since authorized any other card, than the one used during recording).

    So, how I proceed is up to the community (options) :

    1. I retreat again, and work with nobody on this.
    2. I retreat again, and work in private on this.
    3. I continue to work with the community, in the open, and hopefully find some others to join-in, from my market.
     
  7. Jun 11, 2014 #67 of 105
    jmbach

    jmbach der Neuerer

    1,557
    10
    Jan 1, 2009
    noneuknow, I have not said that you were wrong, lying, misleading, etc. I have not accused you of or blamed you for anything. I opened my remark with "my experience" because I know that conditions vary with cable companies, locations, etc. So as not to clutter up this thread I will PM you with a response to your post.

    Let us go back to the subject at hand.

    Perhaps telemark, you can list your TODO list and there might something in there that either I or another TCF member can help with. TODO lists go much quicker with multiple people working on it and perhaps can get it done.
     
  8. Jun 11, 2014 #68 of 105
    nooneuknow

    nooneuknow TiVo User Since 2007

    3,554
    0
    Feb 5, 2011
    Cox Cable...
    Yeah, another nooneuknow "nuke", that I pretty much dropped indiscriminately. Sorry about that...

    I had brought this unusual "lockdown" of content up in the past, when I first observed it, after a cablecard flash update went out.

    It got very ugly, very fast. I was accused of "incorrectly cloning" as I tried to gather data points. I was accused of making things up. It seemed that every comment was pegging me (or my actions) as somehow creating the results, not any part of any system, and so on...

    I had to just back-out, and say to myself "It's not worth it, if this is what I'm going to get as a result".

    The party that really just wouldn't let up on claiming what I observed, and replicated "was impossible", isn't even present (yet).

    I've seen new talent come in, and a new opportunity to possibly get to the bottom of this nagging matter, after having to force myself to not even bring it up all this time.

    Then I go and lash-out, at the first inkling that I'm going to just get a repeat of last time. Realistically, I sabotaged myself.

    I really think this suspected GUID on the drives that has been found, the cablecard pairing data anomaly I have found, and the future of drive changes and upgrades are all part of a design, that I may be able to provide some necessary data on. I just need some help finding what is relevant, and filtering out the rest.

    I'd guess that if the way my market locks-down content wasn't isolated, but was instead the "norm", or became the norm, things would be much different from where I sit, with my POV.

    Let's see if I can fix that post, and get back on track...
     
  9. Jun 11, 2014 #69 of 105
    nooneuknow

    nooneuknow TiVo User Since 2007

    3,554
    0
    Feb 5, 2011
    Cox Cable...
    Yeah, unfortunately, my uncle does not have any TiVos. I'm in the process of parting-out the last unsubbed TiVo HDs I have, which is the last of any "extra" TiVos I have around...

    I'll decline on stating the company he works for, the exact position, etc. It's pretty much nationwide and multinational scale-wise.

    I think of him as ggieseke, except linux is his preferred playground, and Windows is just something he has to make his projects work on. ;)

    At one point, before you came around, I'd asked ggieseke if it would make sense for a two-pronged approach, leaving ggieseke to update DVRBARS, and I'd have my uncle bring JMFS up to date. That conversation kind of died.

    I also wondered how I could expect my uncle to remake JMFS, with just source-code, and no TiVo Roamio (or other model) to work with. I'm not well-off enough to buy him the hardware platforms and drives, and it's not on his to-do list, either...

    I really want to dig into finding any GUIDs that exist on TiVos, and find a way to backup and restore just the data needed to keep the cablecard pairing from being lost, or "zapped". You've already read enough to know the "why" behind this, and my motivations.
     
  10. Jun 11, 2014 #70 of 105
    ggieseke

    ggieseke Active Member

    4,029
    12
    May 30, 2008
    FWIW, I think CableCARD pairing data will always be a black hole that's dependant on your provider. Unless it gets buried somewhere in the tyDb files on the MFS partitions we're screwed there since nobody has come up with a way to read the flash memory on a Roamio yet.
     
  11. Jun 11, 2014 #71 of 105
    lessd

    lessd Active Member

    7,697
    5
    Jan 23, 2005
    CT
    You are the first person on this Forum (that I have read) that has said that a TiVo recorded playback depends on having the cable card in the TiVo, people have moved from one place to another taking their TiVo(s) with them, then putting a new cable card for their new MSO in the TiVo, they have never reported any loss of playback recordings already on the TiVo, I have seen that I can play any recording on a non activated TiVo without its cable card, no problem, but I can't answer for your area or MSO, it just that if this was the way things worked in parts of the country I just assumed (maybe incorrectly) this Forum would have been talking about this playback issue sometime after sept. 2006 when the first cable cards TiVos went on sale, that almost 8 years ago.
     
  12. Jun 11, 2014 #72 of 105
    nooneuknow

    nooneuknow TiVo User Since 2007

    3,554
    0
    Feb 5, 2011
    Cox Cable...
    I'm not sure if I "doubt" it's in the Roamio's flash, or just really "don't want it to be" there... Even if it is, if no GUID change is detected, shouldn't it stay intact? Basically, trick the Roamio into preserving it, thus negating a need to see the flash contents?

    Maybe the guy wanting to know the smallest possible hard drive that a Roamio could prepare (or run, pre-made) was onto something...

    Smaller drives make raw disk editors/hex editors less likely to take years off your life as you scan the data on them, looking for crumbs/clues/changes...

    I swear, if Apple made these things, they'd have been "jailbroken" the day before they went on sale...
     
  13. Jun 11, 2014 #73 of 105
    nooneuknow

    nooneuknow TiVo User Since 2007

    3,554
    0
    Feb 5, 2011
    Cox Cable...
    Nothing about my market reflects the rest of the country, except that we use coax and cablecards (this is the short version of my point).

    I often feel like I'm the only TiVo owner in my market. I get treated like that's the case when I deal with them (Cox).

    Who else, except me, is going to be OCD enough to investigate why "The Hitchhiker's Guide to the Galaxy" (and maybe six other recordings) consistently fail to be watchable every time I change the cablecards?

    Who else, in my market had eight TiVos and eight cablecards at one time, and would move the cablecards around, or replace them every so often?

    Most probably would shrug the loss of a few movies they wanted to hang onto, delete them, and never even know TCF exists, right?

    I'm not trying to be argumentative in this. I see this as a logical answer to the question you pose: "Why only hear this from me?"

    Who messes around with their TiVos, changes drives, whole configurations, and all these other things never meant to be done, just to replicate the results and come here to report them? That seems to be limited to me, when it comes to my market.

    I don't see that as making what I report any less true, or "impossible" and some will flat-out say...

    The content and spirit of this whole thread, and all the DIY threads are not what "most people" do when they buy a DVR...

    While I feel all this has a part in "best practices" for future drive expansions, I also don't want to hijack the thread, over what is just part of a bigger picture...
     
  14. Jun 11, 2014 #74 of 105
    ggieseke

    ggieseke Active Member

    4,029
    12
    May 30, 2008
    YMMV says it all. :D
     
  15. Jun 11, 2014 #75 of 105
    telemark

    telemark New Member

    1,544
    1
    Nov 12, 2013
    So I posted 4 LittleEndian object implementations. #3 had an inconvenient web page. #2 was implemented as a subclass, which is slightly different than the others.

    The Google one, #1, looks complete, but it didn't have an ant build file, and was large. The OReilly one, #4, was simple, but incomplete. I frankensteined them and added 2 missing functions myself.

    I don't know if it's 100% correct but this does build.

    On another attempt I'll try a different combination. The Google library should work out of the box, and the subclass might be fine too.

    Bad news:
    Giving it a 500GB hard drive the ROAMIO freshly produced,
    ./mfslayout.sh /dev/disk1

    java.lang.Exception: No root MFS found
    at tivo.Mfs.addDisks(Mfs.java:311)
    at tivo.Mfs.<init>(Mfs.java:75)
    at tivo.Mfs.<init>(Mfs.java:69)
    at jmfs.MfsLayout.main(MfsLayout.java:42)
    MfsLayout: done

    Good News:
    As baseline, I compiled the krbaker and jmbach github jmfs without modification, and ran them against my PREMIERE backups.

    About half work, but half of them give me:
    ./mfslayout.sh /dev/disk1

    java.lang.Exception: No root MFS found
    at tivo.Mfs.addDisks(Mfs.java:309)
    at tivo.Mfs.<init>(Mfs.java:74)
    at tivo.Mfs.<init>(Mfs.java:68)
    at jmfs.MfsLayout.main(MfsLayout.java:42)
    MfsLayout: done

    I could have corrupt images, or a bad build. Or has this been seen before?
    32bit vs 64bit formats maybe?
     
  16. Jun 12, 2014 #76 of 105
    jmbach

    jmbach der Neuerer

    1,557
    10
    Jan 1, 2009
    I have had that error in the past. If I recall correctly, it is looking for a valid Partition 10 and cannot find it. There is an environment variable you can set to tell it where its at but it never worked for me. Sometimes when this happens, I found that there was an issue reading the APM. Other times there was some corruption in partition 10. The rest I did not have a clue.

    Now JMFS does not recognize a 32bit MFS layout. If you used it on all Premiere images, they are all 64bit MFS, so that should not have been an issue.

    One test would be to see if pdisk from MFSLive CD can read the APM or if dd can recognize the partition by trying to copy it.
     
  17. Jun 12, 2014 #77 of 105
    telemark

    telemark New Member

    1,544
    1
    Nov 12, 2013
    I'm trying the slax live CD now and it is more consistent. Very odd, I can move the same build from a desktop OS to the CD environment and it'll start working.
     
  18. Jun 12, 2014 #78 of 105
    jmbach

    jmbach der Neuerer

    1,557
    10
    Jan 1, 2009
    You have brought back painful memories. I had no clue that dd would understand a TiVo APM for a long time until reading the forums and seeing that members were using it to copy individual partitions. So when I was working on figuring out how to expand a premiere to 4TB, I thought I would just use dd. I had so many issues using dd to copy partitions that I used iBored instead. In retrospect, it might have been related to me using VirtualBox to run the ISOs and it can only access the drive I was working on via USB2 and not eSata. I figured out that was hdparm's problem but never thought it was dd's issue as well.
     
  19. Jun 12, 2014 #79 of 105
    nooneuknow

    nooneuknow TiVo User Since 2007

    3,554
    0
    Feb 5, 2011
    Cox Cable...
    I found out by accident, due to sleep-deprivation (a state that often contributes to some of my meandering/long posts).

    When using GNU dd_rescue intending a full clone, I typed "dev/sda1 dev/sda2" instead of "/dev/sda /dev/sdb". Big "oops!" then "aha!" moment. Luckily, I'd practiced what I preached and was working with a backup copy of sda.

    I don't know the long-term effects of copying the boot sector over the beginning of partition 2 of a Premiere, but it effectively stopped software from installing to that partition via KS52, and didn't kill the unit, even after a KS57&58 failed to fix it (and didn't create a boot-loop). I'd hoped to test if it would "block" a newly downloaded software from installing itself, but never remembered to try it. Another thought was it was a way to take a partition with bad sectors "out of play", letting you limp along on the good alternate.

    Not Roamio-related, AFAIK, but maybe it will give somebody an idea on how to do something they hadn't thought of a way to.
     
  20. Jun 12, 2014 #80 of 105
    nooneuknow

    nooneuknow TiVo User Since 2007

    3,554
    0
    Feb 5, 2011
    Cox Cable...
    I seem to recall slax being used by Comer for at least few things during the making of JMFS, and it sometimes being the choice of others, over the many distros out there.

    I've yet to learn enough about linux to know what distro is best. I'm mostly looking for:

    1. Most drivers already included (especially USB3 onboard and add-on card)
    2. Up to date device drivers (not necessarily for the newest HW)
    3. Best raw copy rate between two USB3 drive docks
    4. I like GNU dd_rescue, and like to run the newest release
    5. I want it to not be shy about using the RAM and resources available, like for caching disk I/O

    I like Live DVDs, or alternately Live USB sticks. Most of my dedicated test systems (guts on a bench) are mid Core2Duo era, with 4-6GB RAM.

    What's suggested for my preferences, and also works well with 3 & 4TB drives, if I'm going to start doing more than talk about wanting to try to help find ways to build large images, and find ways to preserve Roamio content when moving to larger drives?

    I have to start somewhere... Programming/coding is something I know so little about, that I guess all that is left is working with partitions and layouts, and things that can be done with disk/hex editors.
     

Share This Page