1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

PyTivo Video Manager HME App for pyhme

Discussion in 'Developers Corner' started by jbernardis, Apr 20, 2011.

  1. Squirl1899

    Squirl1899 New Member

    15
    0
    Jun 4, 2010
    I have 5 2TB drives attached via various connections (USB 2, USB 3, eSATA, Ethernet) and have different movies on all the different drives. I know that you can set vidmgr to pull by genre, actor, etc but I want to have it use multiple drives for a single share.

    Example:
    Movies
    1) H:\WD 2TB ProBox\Movies
    2) K:\Seagate 2TB ProBox\Movies
    3) J:\WD 2TB USB\Movies

    Animation
    1) F:\Hitachi 2TB ProBox\Animation
    2) K:\Iomega 2TB USB\Animation

    I can't find a way to do it in pyTiVo or vidmgr, any help would be greatly appreciated.

    Code:
    [Server]
    port = 9032
    ffmpeg = C:\pyTiVo\bin\ffmpeg.exe
    tivo_mak = xxxxxxxxxx
    tdcat = C:\pyTiVo\bin\tdcat.exe
    ffmpeg_wait = 10
    tivo_password = xxxxxxxx
    tivo_username = xxxxxxxxxxx
    togo_path = K:\TiVo Recordings
    tivodecode = C:\pyTiVo\bin\tivodecode.exe
    temp_share = C:\pyTiVo\temp
    togo_mpegts = True
    
    [_tivo_SD]
    
    [_tivo_HD]
    bufsize = 4096k
    optres = True
    
    [Movies]
    type = video
    path = G:\Movies Seagate 2TB USB 3
    force_alpha = True
    
    [Animation]
    type = video
    path = F:\Animation
    force_alpha = True
    
    [TV Shows]
    type = video
    path = F:\TV Shows
    force_alpha = True
    
    [Animation - Low Res]
    type = video
    force_alpha = True
    path = J:\Movies That Need Higher Resolution
    
    [Childrens Shows]
    type = video
    path = H:\Childrens
    force_alpha = True
    
    [Movies WD 2TB USB]
    path = J:\Movies WD 2TB USB
    force_alpha = True
    type = video
    
    [Mini Series]
    type = video
    force_alpha = true
    path = F:\Mini Series
    
    [Cartoons]
    path = H:\Cartoons
    force_alpha = true
    type = video
    
    [National Geographic]
    path = H:\National Geographic
    type = video
    force_alpha = True
    
    [Movies NexStar]
    path = L:\Movies Need Cover Art 11-26-11
    force_alpha = True
    type = video
    
    [All Movies]
    force_alpha = True
    path = C:\Users\Michael\AppData\Roaming\Microsoft\Windows\Libraries\All Movies
    type = video
    
    [All Movies Libary-ms]
    force_alpha = True
    path = C:\Users\Michael\AppData\Roaming\Microsoft\Windows\Libraries\All Movies.library-ms
    type = video
    
    
    The last 2 entries were an atempt to add the folders to my windows library and just have pyTivo pull that but it didn't work either. Is this something that can be even be done in vidmgr or pyTiVo?
     
  2. wmcbrine

    wmcbrine Ziphead

    10,364
    22
    Aug 2, 2003
    I have only a dim understanding of Windows Libraries, but I believe they only work through Windows-specific APIs (typical crappy half-assed Windows way of doing things), rather than presenting their contents as part of the filesystem (the Right Way, also known as the Unix Way). pyTivo only uses standard filesystem calls, so it won't see the Windows Libraries. Sorry. Tell Microsoft to fix their ****.

    There is currently no support in pyTivo for combining multiple directories into a single share. But it's a feature others have requested. It might not be out of the question in the future. Meanwhile, I think the best you could do would be to create filesystem-level links (but not "shortcuts" -- they're another proprietary Windows thing) under a common directory.
     
  3. Squirl1899

    Squirl1899 New Member

    15
    0
    Jun 4, 2010
    I agree, pyTiVo can see the share but says there is nothing inside (thanks MS!) I was just trying to find a work around. I also tried symbolic links, hard links, junctions, shortcuts, etc but cannot get pyTiVo to see what is in the actual folders. I basically want this:

    [Movies]
    type = video
    path = G:\Movies Seagate 2TB USB 3, J:\Movies WD 2TB USB, L:\Movies Need Cover Art 11-26-11, J:\Movies Need Cover Art
    force_alpha = True

    This way, all of the different drive contents would show up under a single folder instead of having to remember which movies I have on each drive. I know I can do it alphabetically but that becomes a pain if I run out of room and have to move movies from one drive to another. Not to mention when you start trying to find a movie that may or may not have "The" in the title. "The Polar Express" vs "Polar Express", etc.

    Any suggestions?

    But pyTiVo doesn't recognize (or it is a user error on my part) multiple path's for the same title [Movie].
     
  4. jcthorne

    jcthorne Active Member

    2,721
    3
    Jan 28, 2002
    Houston
    Look into a utility called junction link magic. Then create a single directory called MOVIES and put links under that folder for each of the drives. I did this for a while before moving on to a NAS. Which is really what you need to do. At 5 external drives, you really need to start thinking of an expandable NAS system and move all the drives into it.
     
  5. Squirl1899

    Squirl1899 New Member

    15
    0
    Jun 4, 2010
    TOO FUNNY! I actually just tried Junction Link Magic but it has the same drawback, I can only link a SINGLE folder/directory to another SINGLE folder/directory.

    I had built my own NAS (FreeNAS) but had such a hassle with transfer rates, even though I had a gigabit card in the NAS and a WNDR3700 gigabit router. I also had issues with the SATA controller (tried 3 different types) and spent about 9 months working and tweaking it but it just became too frustrating. When I bought my new laptop with USB 3.0 I figured I would just start using that and bought a 4 bay ProBox USB 3.0 enclosure. I have three 2TB drives in their now but still have the issue of movies on diffferent drives.

    If you have any other suggestions on building a NAS I am wide open.
     
  6. wmcbrine

    wmcbrine Ziphead

    10,364
    22
    Aug 2, 2003
    Let me ask you... if pyTivo were to have a feature that combined multiple directories into one share, would you want it to work like this (as seen from the TiVo):

    Code:
    [share]
     |
     +-[dir1]
     |    |
     | [file in dir1]
     |
     +-[dir2]
          |
       [file in dir2]
    
    or like this:

    Code:
    [share]
     |
     +-[file in dir1]
     |
     +-[file in dir2]
    
    ?
     
  7. Squirl1899

    Squirl1899 New Member

    15
    0
    Jun 4, 2010
    The second one:

    Code:
    [share]
     |
     +-[file in dir1]
     |
     +-[file in dir2]
    
    Basically

    Code:
    [Movies]
     - All movies in folder J:\WD 2TB USB\Movies
     - All movies in folder H:\Iomega 2TB USB\Movies
     - All movies in Folder K:\WD 2TB ProBox NAS\Movies
    
    [Animation]
     - All movies in folder F:\Seagate 2TB ProBox NAS\Animation
     - All movies in folder G:\NexStar 1.5TB\Animation
    Preferably all in alphabetical order even though they are in seperate folders. This way all movies or whatever will be listed in a single share, in alphabetical order. Right now I have about 11 shares that could be cut down to about 5. And I wouldn't have to remember which share had which movies in it. This would be the icing on the cake for pyTiVo, IMHO.

    Hope I explained it well enough, if not, please let me know.
     
  8. wmcbrine

    wmcbrine Ziphead

    10,364
    22
    Aug 2, 2003
    Yeah, I figured you'd say that. It's the more difficult way to implement it. :)
     
  9. jcthorne

    jcthorne Active Member

    2,721
    3
    Jan 28, 2002
    Houston
    First on the junction links. Create a directory called MOVIES. Then create subdirectoies MOVIES\D1
    MOVIES\D2 etc. in each sub, place a link to the root of one of the drives. Then map pytivo video share to MOVIES. Pytivo and vidmgr will see all the files for all subdirectories. Vidmgr can present them as a single list sorted by the way of your choosing.

    On the NAS, I went to an expert and paid a small premium for thier expertise. Bought a Synology 1511 expandable NAS. Has terriffic network speed, a robust hybrid raid that allows mixing of drive sizes and online upgrades, rebuilds and expansions. Cost was not that much more than all the hardware to build one myself of similar caliber and the end result is well supported, works very well and is VERY power efficient as in actual drive power plus 26 watts and the drives spin down when not being used. Well thought out hardware platform, linux based and easily upgraded with user apps.
     
  10. lrhorer

    lrhorer Active Member

    6,924
    0
    Aug 31, 2003
    San...
    He needs a RAID solution.

    Actually, it's more like "fair" network speeds. More than plenty for a handful of videos, though.

    Actually, it's a bit pricey per spindle. The big thing, however, is it only has 5 drive slots and can only expand to 15. Of course, this may very well be enough for Squirl1899, and if so it will cost less than a larger chassis, which (if the Synology will suit him) he does not need. The power consumption is quite good. As mentioned in the other, similar thread, this may be a good fit for him. As with that thread, however, I caution the user that this system may not be sufficient for his ultimate needs. I'm not saying this is the case, but he should consider carefully what his long term needs are likely to be.
     
  11. jcthorne

    jcthorne Active Member

    2,721
    3
    Jan 28, 2002
    Houston
    With its dual Gigabit ports and a proper router, I have no trouble with 4 high capacity connections (2 in going and 2 outgoing) on 4 computers each reaching over 80 MB/sec. Thats about as fast as the technology will allow. And I am not really sure if its the limits of the server or of the computers connected to it as each is no faster when the other is not moving data. IE the limits seem to be on the bandwidth at the users, not the NAS. The 1511with 4 drives or more consistantly tests as one of the fastest ready to run NAS solutions on the market under $10k.



    I looked at the hardware to build a 5 spindle, expandable, hot swappable array myself. It was pertty near $1000 with dual GB networking and high efficiency power supplies, mother board, memory, ultra quiet temp controlled fans etc. Best I can figure, the Synology box ran me about a $200 premium at 10 spindles and $300 when I upgrade to 15.

    There are larger Synology solutions, up to 30 spindles per server even with dual redundant power, but I looked at the likelyhood of my needed more than 40TB before 4TB drives were mainstream. Was not likely in my case. Also its a pretty rare application where more than 40TB in a SINGLE array is good idea anyway. Likely well beyond what a NAS like the 1511 is intended to support. IE large home or small office environment.

    As you suggest, the OP will need to determine his long term needs before investing in ANY solution. Be it ready built or home built. Both are correct answers depending on the need.

    I recognize the fact that I paid a premium for the technology but though I got good value for the money in research, design, support and the Synology Hybrid Raid tech that is not available in DIY solutions. Made sense for ME. Does not make sense for you. No wrong answer, just different. The OP needs to figure out what HE needs.

    OH, and a MERRY CHRISTMAS EVERYONE!
     
  12. lrhorer

    lrhorer Active Member

    6,924
    0
    Aug 31, 2003
    San...
    A lot of the folks over on the mdadm mailing list would call that "extremely poor". Of course, a lot of them are admins of enterprise systems with multiple 10Gbps Ethernet connections to the units. Even my little $150 motherboard and dual core processor can handle 400 MB/sec, though. Of course the speed of a RAID array goes up more or less proportionally to the number of drives.

    No, not at all. That said, the published spec for the Synology is just under 200 MB/sec, which is pretty decent for a pair of 1G connections. To go very much faster, one would need a faster network. One could go with additional Gig-E ports or a 10G Ethernet port, or a technology other than Ethernet, such as Fiberchannel, for example. None of these solutions are available on the Synology, but OTOH, most home users are not going to consider such delivery systems. The Synology's reported performance is decent, and quite more than adequate for deployment in a home environment.

    Yes, with an absolute limit imposed by the network connections. The host capabilities also make a big difference, of course. Using TTG or GoBack, an S3 TiVo is limited to about 2MB/sec or so. While transcoding, most of my workstations running Windows / VideoRedo can't manage more than about 10 MB/sec average. OTOH, my backup server runs flat out at over 110 MB/sec on large files when it does its rsync at 04:00 every morning.

    That may well be true.

    It's certainly possible to spend that and much more. Knock off the premium features, though, and the cost drops. One can go with a single Gig port on the motherboard and forget about the ultra-quiet fans, etc.
    My systems do a fair imitation of a B-17 taking off. The motherboard, CPU, case, and 4 port eSATA controller for my backup system all put together cost less than $375 new. Had I shopped for used, I could have shaved some off that. The case itself holds 8 drives, although only 5 of those could be hot-swappable (accessible from the front). For a time I ran this system without the eSATA controller and had all the drives sitting in the system. As soon as I could afford it, I did buy a 20 drive RAID chassis for $1300, and moved the drives to the external chassis. The chassis (along with a 12 drive chassis purchased earlier) supports both the primary and backup systems.

    You lost me, there. The cost per spindle should go down with increasing numbers of spindles.

    Again, the user needs to make that assessment. The answer determines what will best suit the application.

    Oh, not even. Beyond 6 drives (8 at the most), something more than RAID5 is definitely indicated. RAID6 is very good up to perhaps 16 drives, or 24 with triple parity. At 24 drives or more RAID10 definitely starts looking more appropriate, although it can be appropriate with much smaller drive numbers. RAID60 gets to be a reasonable option, or RAID1 + 5 in those areas, as well. At some point, an iSCSI solution under RAID10 or 2 x RAID1 should be a consideration. For ordinary availability, one parity drive for every 5 - 7 data drives is generally sufficient. For high availability, 1 mirror for each data drive is usually sufficient. For virtual invulnerability, 2 or more geographically diverse mirrors for each data drive are required.

    Exactly. The time required to configure should not be a consideration of any great significance. First of all, it's going to take more or less the same amount of time no matter what the solution, and the configuration time is not relevant over the long term. Maintenance and expandability options are the biggest concern.

    Even that may be an overstatement. Any decent solution makes sense. It's more a question of what is best for one's needs in the long run. No reasonable solution will be a mistake, per se. It's a matter of which one fits his budget yet won't leave him high and dry two years down the line. I started out with a server whose maximum capability was four 160G drives in a RAID0 + 1 configuration. I thought going with the less expensive solution would save me some money. It did not.

    Why does that sound familiar? :)

    Merry Christmas, all.
     
  13. cweb

    cweb New Member

    106
    0
    May 29, 2004
    I have a question about the delete behavior. When you say the cache is saved on exit, is that when I exit vidmgr by doing a ctrl-c from the terminal or when I exit vidmgr from the tivo (e.g. to watch the show that I am now transferring)?

    I am running 2.0i on my mac and am pre building the cache prior to starting hmepye & vidmgr on my computer. At that point when I enter vidmgr I can delete a show as expected. However, if I send a show to my tivo, leave vidmgr on my tivo to watch the show, then re-enter vidmgr to delete the show...the delete no longer works (get the boing) unless I rebuild the cache again (three thumbs down). Since nothing has changed on my computer (I only sent a show) shouldn't the cache still be valid and I should be able to delete the show?

    Also, as previous mentioned, rebuilding the cache via three thumbs down get delete working again. However, rebuilding the cache via BuildCache.py while hem-pye and vidmgr are still running does not re-enable one to delete a file from the tivo. Is this expected behavior?

    Thanks in advance.
     
  14. jbernardis

    jbernardis New Member

    1,072
    0
    Oct 21, 2003
    Princeton NJ
    The cache is only read in when the app first loads - that is when you choose it from the menu with the remote control. So, yes, the scenario you describe in the last paragraph is normal.

    The other situation, though, sounds like it could be a bug, although it hasn't been reported before. If I understand you, you push a video, and then exit the app to watch it without first deleting it. When you go back into the app to delete it, you get the error sound? That's what I read from your description, but I just tried this exact scenario and it worked fine for me. But maybe i'm misunderstanding what you are doing.
     
  15. cweb

    cweb New Member

    106
    0
    May 29, 2004
    I just triple-checked my observations and everything is as I described.

    Vidmgr does NOT appear to be reading the cache when you choose it from the menu via remote control. If I enter the app (via remote) get the "clear boing", exit the app (via tivo remote, still running on computer), then run BuildCache.py (cache should now be up-to-date, right?) and re-enter the app (via tivo remote) I will continue to get the "clear boing". The only way to actually get the delete to work is to a) do a three thumbs down update or b) totally exit Vidmgr and Hme-Pye (via ctrl-c on my computer) run BuildCache.py and the restart Vidmgr and Hme-Pye.


    Your understanding of my ramblings is correct and more concise.

    My guess this has to do with the above situation. Perhaps the cache somehow become invalid (or Vidmgr thinks it is) when one exits the app (via tivo remote).
     
  16. jbernardis

    jbernardis New Member

    1,072
    0
    Oct 21, 2003
    Princeton NJ
    Can you capture the output from pyhme from when you enter vidmgr? It should tell you if it has any difficulty loading the cache. The easiest way might be to run pyhme in the foreground while you are trying this out.

    BTW - loading the cache when the app activates is unconditional. Nothing is retained from one execution to the next. If it didn't load the cache, it would have nothing to present. The other thing is that if it has trouble loading the cache, it rebuilds it which should actually result in correct behavior. So there is something else here that I don't understand yet.

    BTW - according to the logic, the only things that should cause a boing sound when you press clear is if 1) this is a DVD Video, 2) you have multiple references to the same physical file, or 3) you have deleteallowed turned off in your options. If it passed all of these tests, it deletes the file(s) and it doesn't even care if the delete is successful or not.
     
  17. cweb

    cweb New Member

    106
    0
    May 29, 2004
    I have attached a log file. I hope this is the info you were seeking.

    This is what the log should have captured:
    1) Started hme-pye & Vidmgr on computer.
    2) Entered Vidmgr on Tivo (via remote).
    3) Tried to delete show via clear and got boing
    4) Rebuilt cache via three thumbs down.
    5) Tried to delete show, this time got delete confirmation.
    6) Cancelled delete as I wanted to download show.
    7) Sent show via Vidmgr.
    8) Exited Vidmgr app on tivo (via remote) and waited for show download to complete.
    9) Re-entered Vidmgr (via remote)
    10) tried to delete show (via remote), got boing error.
    11) Rebuilt cache via three thumbs down.
    12) Tried to delete show, this time got delete confirmation.
    13) Confirmed with thumbs up, show deleted.
    14) Exited Vidmgr app on tivo (via remote).

    Hope I was concise without to much extra info.
     

    Attached Files:

  18. jbernardis

    jbernardis New Member

    1,072
    0
    Oct 21, 2003
    Princeton NJ
    The log file showed nothing out of the ordinary. The delete failed messages at the end mean nothing - you can ignore them. I have no explanation for the behavior you are seeing. The software works just fine for me, and I have had no similar reports from anyone else.

    That's not to say there isn't a bug - there might still be. We just haven't isolated the exact conditions that bring it to surface.
     
  19. cweb

    cweb New Member

    106
    0
    May 29, 2004
    Anything else I can run that might give you more insight?
     
  20. jbernardis

    jbernardis New Member

    1,072
    0
    Oct 21, 2003
    Princeton NJ
    What is your setting for delete allowed? This is an option in vidmgr.ini but it can also be set locally in each directory.

    The ONLY thing I can think of right now is that this is set to false, but dynamically building the cache (thumbs up three times) has a bug and the flag gets reset to true. I'm not saying this is the case; right now it's just a guess.
     

Share This Page