TiVo Community Forum banner
  • TiVoCommunity.com Ambassador Program Now Open! >>> Click Here

PyTivo Video Manager HME App for pyhme

174K views 734 replies 39 participants last post by  lew 
#1 ·
This app has been discussed in another thread in this forum, but it was suggested that I start my own thread, and so I am.

There is no new version of the code, but I am reposting the link here.

In recent discussion, I learned that the HME does indeed support an HD interface. based on suggestions from wmcbrine, I have figured out how to do this and decided that it is not all that difficult to have a dual-personality app that will work in either SD or HD mode. I am currently working on it - give me a couple of weeks or so.

Obviously there are differences in screen geometry between HD and SD, and there is also additional screen real estate. I could simply increase the size of lists, etc - and that is indeed how I am starting. But in the HD version, I am trying to figure out if I can add additional information as well - such as the recently requested thumbnails.

I have to admit that when I answered the posts earlier this afternoon, I was not into it at all. But since getting home, firing up eclipse, and starting to work on the code, I got right back into the mood.

One question I have for now - does the HME simulator support HD? I can't find out how. I don't have an HD tivo near the computer. Every time I make a change, I have to walk across the house to try it out. It gets tiresome. If the sim supported HD, I could do a lot of debugging there. Also, the sim lets me see the safe areas of the screen (as pointed out by wmcbrine)

Thanks
 
See less See more
#327 ·
Can Galleon have pytivo PUSH an mp4? Has it been updated to HD menus? I admit to never having actually tried Galleon but I like the menus and graphics we have in vidmgr now, just was hoping for a bit more functionality.
 
#328 ·
No, Galleon doesn't interact with pyTivo at all. I was just pointing out another example of an app that can parse metadata (MP3 tags in this case) and present different sorting/filtering options to the user.

Galleon does have a GoBack feature for video files, which I used sparingly back in the old days before pyTivo, streambaby, and vidmgr.
 
#329 ·
I looked at lrhorer's method of using symlinks but its just too much on going work to maintain
I don't do any maintenance, at all. The scripts take care of all the work.

and asking for some sort of automation under windows....well, beyond the scope of most current software. I thought an extension of vidmgr that does this would eliminate the need for most of that and make finding a film much easier.
That's definitely a non-trivial request.

Isn't your Synology NAS Linux based? Why not just modify the scripts I use for use on the Synology system? They will automatically create and delete shares sorted by date, genre, alpha groups, etc.

Heck the db could be built ahead of time by the user and stored as a spreadsheet or other common and easily editable file type.
I do have the scripts create a database, but I've never made any real use of it.
 
#330 ·
That would be beyond my current linux capabilities or desire to learn. I no longer run pytivo or vidmgr on the synology box either. I had those two running fine and then started working toward java and harmonium and really mucked up the works. I failed the primary purpose of the NAS...the RAID. I got all the data back and reflashed the NAS back to an updated stock configuration and went back to running all the interface software under windows since the small interface server that used to function as file server was not going away. It does downloads, pytivo, harmonium and system backups.


Anyway, I am back to running all this under windows. The files are all on the NAS. If you think you can give me the scripts and a bit of direction, I'll give them a go under windows. What language are they in?

I did realize my suggested addition to vidmgr was not trivial. Just thought that managing the vids and presenting them in various sort orders was a logical next step if Jbernardis is interested.
 
#331 ·
I would like to present an idea I have been mulling over for video manager. See what others think, especially Jbernardis since it is HIS program :)
It's a great idea, but unfortunately I don't have the bandwidth to tackle that right now. Maybe for vidmgr 2.0 :)

Of course the downside of a flat file system and relying on metadata is that the PyTivo NPL representation would lose its structure. I'm like you - I have my films all divided into directories by genre, but I also still use the PyTivo pull interface too and I like to see them consistent with one another.
 
#332 ·
Available time I really understand.

I no longer use the pytivo pull interface at all since all my video is stored as mp4 and would have to be transcoded to transfer.

My biggest problem with filing by genre is that so many movies belong in more than one category and when I go looking for one....I always am looking in the wrong one. Perhaps using a browse interface for searching is the wrong approach.

Have you seen any easy way to gather the movie meta data into a single file? Getting all of it to a simple spreadsheet would at least allow me an index to keep handy. Suppose I could relearn visual basic and do it in excel.
 
#333 ·
Anyway, I am back to running all this under windows. The files are all on the NAS. If you think you can give me the scripts and a bit of direction, I'll give them a go under windows. What language are they in?
Bash. I know there are at least a couple of bash interpreters for Windows out there. I have no idea how well they work or whether they support any of the newer bash forms. A better solution might be Take Command.
 
#334 ·
My biggest problem with filing by genre is that so many movies belong in more than one category and when I go looking for one....I always am looking in the wrong one.
You may have lost me, there.

Perhaps using a browse interface for searching is the wrong approach.
If you are going to relegate the function to a PC, rather than the TiVo, then what I would do is write CGI scripts that relegate the functions to a web page. You can use Perl, PHP, or whatever to write the script, and arrange the database any way you like. Indeed, you could create the data base using whatever database manager you like. I'm doing that very thing here.

You will have to either run a web server on one of your machines, or else purchase a web page from one of the myriad web hosting services.

Have you seen any easy way to gather the movie meta data into a single file? Getting all of it to a simple spreadsheet would at least allow me an index to keep handy. Suppose I could relearn visual basic and do it in excel.
Collecting the relevant data into a single file is not difficult. 'Simply parse the metafiles and extract the fields into a database file. A tab-delimited file would work, or perhaps semicolon. Extracting a page for display filtering by genre is also not difficult.
 
#336 ·
Hi

I just added your vidmgr to my pyhme installation...fyi-Pyhme worked fine before this was added.

Iadded to the config, and restarted HME, get the following error:

C:\Program Files\TiVo\hme-vlc kit>start.py
HME Server for Python 0.17
Traceback (most recent call last):
File "C:\Program Files\TiVo\hme-vlc kit\start.py", line 352, in <module>
app = __import__(name)
File "C:\Program Files\TiVo\hme-vlc kit\vidmgr\__init__.py", line 14, in <modu
le>
print time.asctime(), TITLE + " version " + version + " starting"
NameError: name 'time' is not defined

C:\Program Files\TiVo\hme-vlc kit>

It looks like a pretty basic error. Any idea what I did wrong? Looks like it can't even use the time function?
 
#337 ·
Thanks for your usual very detailed response. I often learn quite a bit from them even if they only lead me in directions I had not considered before. That can be a good thing :up:

You may have lost me, there.
By example, From Russia With Love belongs in each of the following Genre

Classics
Action Adventure
Thrillers

In my case its located in the Action & Adventure directory but I would just as likely go looking for it in the Thrillers or Classics sections. Symlinks could help with this but I still am not sure I can make this work effectively under windows. The basic problem being that windows does not keep track of the links. When the base file is moved, renamed or deleted, the symlinks do not get updated and now point to the wrong place.
 
#338 ·
If you are going to relegate the function to a PC, rather than the TiVo, then what I would do is write CGI scripts that relegate the functions to a web page. You can use Perl, PHP, or whatever to write the script, and arrange the database any way you like. Indeed, you could create the data base using whatever database manager you like. I'm doing that very thing here.

You will have to either run a web server on one of your machines, or else purchase a web page from one of the myriad web hosting services.

Collecting the relevant data into a single file is not difficult. 'Simply parse the metafiles and extract the fields into a database file. A tab-delimited file would work, or perhaps semicolon. Extracting a page for display filtering by genre is also not difficult.
I would like the info presented to the user on the tivo, not on a pc. Away from the tivo, a simple spreadsheet database is sufficient. dlfl provided a simple utility to do just that:

metatoexcel

My Synology box has a built in simple web page server so could to that but its not for the pc, I need it on the tivo.

Thanks for the scripts. I am studying them for ideas but at the moment don't see how to use them or similar under windows in an effective manner.
 
#339 ·
Hi

I just added your vidmgr to my pyhme installation...fyi-Pyhme worked fine before this was added.

Iadded to the config, and restarted HME, get the following error:

C:\Program Files\TiVo\hme-vlc kit>start.py
HME Server for Python 0.17
Traceback (most recent call last):
File "C:\Program Files\TiVo\hme-vlc kit\start.py", line 352, in <module>
app = __import__(name)
File "C:\Program Files\TiVo\hme-vlc kit\vidmgr\__init__.py", line 14, in <modu
le>
print time.asctime(), TITLE + " version " + version + " starting"
NameError: name 'time' is not defined

C:\Program Files\TiVo\hme-vlc kit>

It looks like a pretty basic error. Any idea what I did wrong? Looks like it can't even use the time function?
Ping

This seems a fundamental error, but I cannot get around it, please, anybody? :)
 
#340 ·
Ok, got past my previous error, editted out the time part of the line :)

No it seems to work well, except, when I say push file and try to push, I select the tivo to push to, any selection returns a button labelled:

'Exception during push request'

Any ideas why? The console log shows nothing except, 'vidmgr exitting'

I tried to push .avi, .mp4, all caused exceptions as described.


Also, does vidmgr support .mkv files?

I added .mkv to the config and it now sees and describes the episodes, but it gets exception too.

BTW, the hme stuff and pytivo work perfectly......
 
#341 ·
By example, From Russia With Love belongs in each of the following Genre

Classics
Action Adventure
Thrillers

In my case its located in the Action & Adventure directory but I would just as likely go looking for it in the Thrillers or Classics sections.
OK, I see what you are saying. You only have it extant under one category. It was this very dilemma that prompted me to start down this path.

Symlinks could help with this but I still am not sure I can make this work effectively under windows.
It will definitely be more difficult than in Linux. Once again, Take Command may help. Cmd.exe is really awful.

The basic problem being that windows does not keep track of the links. When the base file is moved, renamed or deleted, the symlinks do not get updated and now point to the wrong place.
Linux does not, either. Hypothetically it could be possible for an OS to keep track of symlinks, but it's probably not practical. That means the app has to do it, or else the user. In the files I posted, the `Verify` routine is run manually to add a title to the list of verified movies (which means basically it has been edited to suit my tastes and has been pushed to a TiVo to make sure it transfers without error.) In addition to adding it to the verified list of movies, it also asks the user to classify the video by genre. All of this could fairly easily be automated to pull the info from the metafile, should one choose to do so, and it could be run once an hour, say. The Windows scheduler is quite pathetic, but it is sufficient for this task.

I have set up the `Verify` routine so the user does not have to enter the entire video name. Any part of it is sufficient to bring up a list of all videos whose names contain the string. Thus, for the video "Raiders of the Lost Ark (Recoded Sun Jan 07, 2004, HBOHD)" one need only enter the string "raid" or "ark". Of course, using as common a string as "ark" may bring up more than one video name. On my server, it brings up 14 different names. The user simply enters the number of the video in question from the list. Note coding this was very easy using the Linux command `find`. It will be more difficult under cmd.exe. Take Command may have a suitable function, I don't know.

Once the user has selected the video, the `Verify` routine cleans up all the temporary files that went into creating the video, and then it checks to see if the video is already listed as being verified. If not, it renames the video to eliminate leading articles ( "The" or "A" ) by calling the routine `Rename`.

Once this is all done, the `Verify` script calls the routine `Vlinks`, which creates all the links based upon the users selections. Again, a list of 25 genres is displayed for the user, and the user selects which ones apply. As I mentioned, this could be easily automated to pull the info from the metafile, requiring no user input at all. My biggest issue with this - and the reason I do not do it - is the genre data may be missing or IMO incorrect in the metafile. That, and the genre info in the metafiles encompasses more genre specifications than I care to consider.

At the very end of the `Verify` script, I also run the `GroupAlpha` routine, which creates a symlink to every video from a directory whose name name is equal to the first letter of the name of video (or "0-9" if the first character is a number). This share allows me to find a video by name in the NPL or vidmgr very quickly, despite having literally thousands of videos on the server. I also run the `GroupAlpha` routine under cron twice an hour.

If the user moves the file, the `Verify` routine will need to be run again. (Or if the user is not concerned about the verification aspect, one could simply run `Vlinks` directly on the fully qualified file name.) With little trouble, this could also be easily automated to run every hour or so if the `Vlinks` routine were modified to pull the info from the metadata.

Obviously, none of this takes care of any broken links caused by moving or deleting the file. To handle this, I have the routine `AuditDatabase`, which also runs twice an hour. It first checks for broken links, removing any it finds. Then it updates the database of movies with any links it finds that do not have any corresponding genre entry in the database (this functionality could be removed if the system obtains the genres from the metafiles). Finally, it creates any links that are missing according to the database.

I also manage the video system using the script `VideoScribe`. This typically runs automatically twice an hour, although I frequently run it manually. Among other things, it creates links for all videos that have not yet been verified in a share labeled "Unverified". That way I can easily tell which videos have not been verified, and when I select them to push to one of the TiVos, they are all collected in a single folder named "Unverified". It also calls the script `DateGroup`, which creates links to every video based upon its recording date (not the file create or modify time) in reverse order - newest first. This share allows me to quickly choose from among the videos which have been most recently recorded.
 
#342 ·
I would like the info presented to the user on the tivo, not on a pc. Away from the tivo, a simple spreadsheet database is sufficient. dlfl provided a simple utility to do just that:

metatoexcel
Personally, I would choose a much more portable method than this. It's fairly trivial to implement, even using cmd.exe. 'Just parse the metadata using the cmd.exe "find" command and spit it out to a delimited database file. Excel can read tab or semicolon delimited files.

My Synology box has a built in simple web page server so could to that but its not for the pc, I need it on the tivo.
Short of writing some HME or HMO code, links are the only way I know to handle this. They can be easily implemented directly on the Synology system, even with your TiVo servers running on a Windows PC, or they can be implemented atop SMB on the Windows PC. They don't haveto be symlinks, BTW. If they are on the NAS, then they can be hard links, which might make things easier for you. I chose symlinks because I needed the links to span physical volumes, which hard links cannot do.

Thanks for the scripts. I am studying them for ideas but at the moment don't see how to use them or similar under windows in an effective manner.
The algorythms for the most part will work under any script language, including cmd.exe. The issue is Windows does not provide the vast library of external functions like `find` and `select` that *nix does, nor does cmd.exe provide them internally. It's also vastly less powerful than any of the *nix shells, although it is possibly sufficient to the task at hand, which is not very complex.

It's probably simpler to implement them on the NAS, but certainly it is not impossible in Windows under a Windows based interpreter - even cmd.exe.
 
#343 ·
Ping

This seems a fundamental error, but I cannot get around it, please, anybody? :)
I can't say what's wrong here - this is the third executable line in the file. What version of python are you using? I assume it's not 3.x; vidmgr (and I believe pyhme) requires 2.7 or thereabouts - but hasn't been tested with 3.

I noticed that I didn't have an import statement for time - not sure if it's required - but you could try that - add an "import time" statement above the print statement. I can only assume that if it's necessary, then I was getting the definition of time from the "from hme import *" statement, and you and I are running different versions of pyhme (I'm running 0.18 BTW).

As far as the other issue you are having, vidmgr does not do the actual pushing - it makes an HTTP request to PyTivo to do this. Are you certain that you have your PyTivo information correct in your config file? The message you quoted "Exception ..." occurs when the URL Open request fails. This usually indicates a bad IP address or port for PyTivo, an incorrect TSN or an incorrect share name.
 
#344 ·
Ok. Ill check

I know ip is right as hme runs on pytivo
Box. It reads the pytivo shares as it shows the
Folders

The ports match. I dont need to specify tivo
Ip addrs right? Pytivo finds them, correct?
 
#345 ·
Ok. Ill check

I know ip is right as hme runs on pytivo
Box. It reads the pytivo shares as it shows the
Folders

The ports match. I dont need to specify tivo
Ip addrs right? Pytivo finds them, correct?
That's no guarantee that the IP addresses are correct. Vidmgr does not use pytivo to browse the files - it just goes to the directories themselves. The only time the ip addresses are used is to make the push request.

NOTE: This is the IP address of pytivo that you need to specify - not the IP address of the tivos themselves.
 
#346 ·
I spent some time this weekend getting an education on windows NTFS hardlinks and junction points. Also learning that when they are applied to entries on a NAS server emulating an NTFS file system, things can be a bit different.

I found a useful shell extension for windows file manager that greatly assists with the creation and management of hardlinks. It also shows which files have more than one link by modifying the icon in file manager. Works pretty slick. Moving the links or files does not destroy the links as long as they remain on the one volume.

Using the tools, I was able to move all my movies to an alpha storage system and then create links for a genere storage set.

The end result is that vidmgr now presents the user both alpha and genre menus. Was not as diffecult as I had thought once I determined to learn how they work.

Thanks to lrhorer and orangeboy for the pointers and info. I still do not have it automated but now that I understand the process, that seems almost possible.
 
#349 ·
That's no guarantee that the IP addresses are correct. Vidmgr does not use pytivo to browse the files - it just goes to the directories themselves. The only time the ip addresses are used is to make the push request.

NOTE: This is the IP address of pytivo that you need to specify - not the IP address of the tivos themselves.
Thanks!

I has a typo in my config, 192.168.1.2083 is definitely not valid :)
\
 
#350 ·
Glad you found the problem.

Regarding the time issue, I learned when I got home that I definitely need the import statement. Not sure why it was working for me (and others) but not for you, but the next release of the code will have the import statement. I hope to be releasing it in the next day or so - just doing some testing right now.
 
#351 ·
Hey. I got this working to all 3 tivos

Anyone else see a problem where the wrong
Episode of a show transfers? It pushes,
but not the episode I select

Anyone else ever see that?
 
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Top