TiVo Community Forum banner
761 - 780 of 957 Posts

·
Registered
Joined
·
51 Posts
I was thinking you would have MG3 call the node (javascript not java) server directly as a simple http interface to make mind calls. I could also make a node script that takes parameters like the python script does. I still think the websocket client would be fairly easy to write in c# the harder part is all the parsing you need to do for getting the cookie.

There is at least one request where you need to pass it a real browser user agent or you'll get a error message but a 200 http response.
tivo-scripts/getDomainToken.js at master · lart2150/tivo-scripts
 

·
Cranky old novice
Joined
·
9,508 Posts
Discussion Starter · #762 ·
I did a little googling and now have a picture of how nodejs/javascript works. I’m still hazy about some of your suggestions but I wonder if something like the following would make sense, building on your idea of a node server acting as a simple http interface to make middlemind calls:

Inititally I modify my python script so it communicates with that node server. Since the python RPC requests already use JSON formatted data, I assume this should be easy. I’m assuming the node server would use the required domainToken authorization. And maybe it could use websockets although I’m hazy about what modifications that would imply in the python code, i.e., ranging from almost nothing to not possible at all. My impression is that performance is not a major factor here.

Meanwhile, I will be porting the python code to C# so it is integrated into the MG3 program so the python dependency is eliminated.

I don’t think requiring users to install node.js is a significant issue and I believe the MG3 code can launch the node server at startup.

Now, about the domainToken:
Could the node server implement a call that would get a token? Then if RPC calls fail the C# code could query the node server for a new token and retry. (Or could this fail/get-token/retry be built into the node server?)

BTW is your javascript code to get a domainToken working?

TIA for your help!
 

·
Cranky old novice
Joined
·
9,508 Posts
Discussion Starter · #764 ·
My personal preference would be not to have to install Node.js. Just my personal preference.
Just curious, what is the reason for this? BTW Node.js installation would only be necessary for users who want to do RPC searches.
 

·
Registered
Joined
·
134 Posts
My personal preference would be not to have to install Node.js. Just my personal preference.
Just curious, what is the reason for this? BTW Node.js installation would only be necessary for users who want to do RPC searches.
As an end user I don't have node.js installed for anything else and would prefer not to have yet another run-time engine type thing installed on my PC just for this. At least with the Python engine I have several apps that utilize Python. Just a personal preference thing if it were relatively easy to avoid (and I don't know that it is). That said, I'll install it if it's the only way to fix the RPC searches.

And yes, RPC searches is exactly what I want.
 

·
Old !*#$% Tinkerer!
Joined
·
1,408 Posts
...I don't think requiring users to install node.js is a significant issue and I believe the MG3 code can launch the node server at startup...
My personal preference would be not to have to install Node.js. Just my personal preference.
Just curious, what is the reason for this? BTW Node.js installation would only be necessary for users who want to do RPC searches.
As an end user I don't have node.js installed for anything else and would prefer not to have yet another run-time engine type thing installed on my PC just for this. At least with the Python engine I have several apps that utilize Python. Just a personal preference thing if it were relatively easy to avoid (and I don't know that it is). That said, I'll install it if it's the only way to fix the RPC searches.

And yes, RPC searches is exactly what I want.
As the End User of *FREE* software that I find very useful for my needs, if the Developer, who invests HUNDREDs of *UNPAID* hours creating, maintaining and answering questions about his software as a "Labor of Love", feels that requiring the User to install an additional software package on their PC is NOT a significant issue (and will save him countless hours of development TIME & GRIEF), then I, as a fellow programmer / techie / TiVo User, say " @dlfl do whatever you need" :handok: :cool:.

I, amongst a *HANDFUL* (Sadly, yes - NOT EVERYONE who replies to questions on a regular basis here on TCF presents consistently CORRECT answers; since no one corrects them, I 'ass-u-me' that either they don't realize it *OR* general forum etiquette frowns upon it - i.e. confrontation...) of other TCF "Technically Competent TiVo Users" (a few of whom I have exchanged 'thoughts / ideas / opinions' with via PMs) who choose to freely VOLUNTEER their time to reply to and attempt to help "Non-Technical TiVo Users" really keep wondering "Why do we repeatedly subject ourselves to this unnecessary grief?" (Some have stopped posting; some have left TCF. :( ).

Only a couple of days ago I was CUSSED AT :eek: for asking a TiVo User (who was COMPLAINING about a problem that I was attempting to help him with) several questions for him to answer to aid me in the pursuit of a possible solution. The *ONLY* thing he could keep repeating was "I bought it from Weaknees" :rolleyes:. For some unknown to me reason, *ALL* of his posts suddenly 'disappeared' (I didn't report him?!? Maybe someone else did.). His original posts somewhat remain in my QUOTEs of his posts when replying with my troubleshooting questions: Stuck on Starting Up after Internal HD replacement

Sorry for the off-subject tangent @dlfl but I just wanted to present an opposing opinion to what @Zaphod posted. ;)
 

·
Cranky old novice
Joined
·
9,508 Posts
Discussion Starter · #767 ·
I was thinking you would have MG3 call the node (javascript not java) server directly as a simple http interface to make mind calls. I could also make a node script that takes parameters like the python script does. I still think the websocket client would be fairly easy to write in c# the harder part is all the parsing you need to do for getting the cookie.

There is at least one request where you need to pass it a real browser user agent or you'll get a error message but a 200 http response.
tivo-scripts/getDomainToken.js at master · lart2150/tivo-scripts
By a "real" browser user agent, do you mean for a browser that actually is installed on the computer running the code?

Also, did you notice my post #762 above?

I've been researching .net framework websocket client (for the mg3 C# code) and can't find any counterpart to the domainToken credentials used in your .js code. There is a cookie object that can be set to be sent with ws requests. I wonder if that is it?
 

·
Cranky old novice
Joined
·
9,508 Posts
Discussion Starter · #768 ·
As an end user I don't have node.js installed for anything else and would prefer not to have yet another run-time engine type thing installed on my PC just for this. At least with the Python engine I have several apps that utilize Python. Just a personal preference thing if it were relatively easy to avoid (and I don't know that it is). That said, I'll install it if it's the only way to fix the RPC searches.

And yes, RPC searches is exactly what I want.
I just installed node.js on my Win10 PC and although it went smoothly I have to admit it made me a little uneasy. The installer launches a PowerShell script that goes through a flurry of commands installing a myriad of things apparently needed to support node. There are numerous messages, possibly warnings or errors, that scroll so fast you can't read them. It does create log files that you can go inspect later.

This was several days ago. I've been experimenting, learning JavaScript programming with node.js. So far all seems OK and I don't think the node.js installation ate up a huge amount of disk space, although I don't have a number for that. Considering all the items it installed, the only way I could know that would be to have noted the disk space before installation, which I didn't do. I wonder if the uninstaller removes all those things?
 

·
Registered
Joined
·
51 Posts
For some of the calls to tivoidp.tivo.com you need to send a browser user agent header like chrome or firefox, node and user agents curl don't work. You could look at my code and try and convert it to c# I use some dom parsing and regular expressions. The one wonky thing I do is on line 48 where I limit the number of redirects because the cookie library has a bug.

When you install node there's an option to also install build tools via chocolatey. The build tools are not needed for any of the libraries I use. There is a npm package that can convert a node script into a self contained exe but it's about 25MB.
 

·
Cranky old novice
Joined
·
9,508 Posts
Discussion Starter · #770 ·
I was thinking you would have MG3 call the node (javascript not java) server directly as a simple http interface to make mind calls. ………
I would like to go in this direction. I'm hoping you are offering to create that node server, since I am a long way from knowing how to do it.

I'm curious: is the GetDomainToken function in the Git repo working or still under development?
 

·
Cranky old novice
Joined
·
9,508 Posts
Discussion Starter · #771 · (Edited)
I would like to go in this direction. I'm hoping you are offering to create that node server, since I am a long way from knowing how to do it.

I'm curious: is the GetDomainToken function in the Git repo working or still under development?
Much to my surprise I've managed to create a node server that seems to work for this purpose. I've made the required mods to the rpcSearch python file and I'm testing MG3. Initial results look good.

The certificate that expires is no longer needed, but the "domainToken", which is a currently valid cookie for online.tivo.com, is required. It is easily obtained from the MS edge browser. Assuming testing continues to be successful, some work will be required to create distribution packages and facilitate entering the domainToken cookie. The python code is ported to Python ver. 3.9 so either that version will have to be installed, or a new version of the python executable will be an option. The node server has to be distributed. I will look into the practicality of creating a stand-alone .exe for that too, for those who don't want to install node.js and the required express package. Hopefully, MG3 will be able to launch the server on startup and terminate it upon shutdown.

Hopefully, having the node server automatically obtain/refresh the domainToken cookie can be incorporated, assuming @lart2150 has perfected that code.
 

·
Cranky old novice
Joined
·
9,508 Posts
Discussion Starter · #772 · (Edited)
Ready for testing for the brave! RPC searches are back.
EDIT: Everything originally attached to this post has been updated due to changes and enhancements, and are attached to a later post, so all attachments here have been removed.

Test distro ver. 3.97T1 is contained in the attached zip. EDIT: attachment removed, superseded by MG397T2 attached to later post. But first download the attached NodeServerInstructions.txt file to see what needs to be done to install node.js and other considerations. It includes everything you need to know to get going.

This version requires grabbing a domainToken cookie. This is easy and procedure is in the .txt file.

When you start your first RPC search, the node server will be launched in a small windows command window. You can minimize if if you desire, and later versions will hide it. This will be shut down automatically when you shut down MG3.

Thanks are due to @lart2150 for showing how to do this in node.js!
 

·
Registered
Joined
·
65 Posts
Oh my god, thank you so much for all your hard work!! I have tried all of your instructions, everything looks good. Node.js installation good, typed node main.js saw the server 3000 message ok, saw the names tony, lisa... food in the browser. Retrieved domain token with Edge and put it in the MG3 settings. But when I do an RPC search I get the following error: error getting seriesid:RPC returned nothing. I went through all steps about four times and I have no clue what I might be doing wrong. Any suggestions would be much appreciated. Thanks.
 

·
Cranky old novice
Joined
·
9,508 Posts
Discussion Starter · #774 ·
Oh my god, thank you so much for all your hard work!! I have tried all of your instructions, everything looks good. Node.js installation good, typed node main.js saw the server 3000 message ok, saw the names tony, lisa... food in the browser. Retrieved domain token with Edge and put it in the MG3 settings. But when I do an RPC search I get the following error: error getting seriesid:RPC returned nothing. I went through all steps about four times and I have no clue what I might be doing wrong. Any suggestions would be much appreciated. Thanks.
Please provide some details.
Which windows version.
Did you install python? If so what version? Or are you using rpcSearch129.exe?
Some rpc searches return nothing and that is the message you get. Have you tried several different series?
Did you close the command window after testing the node server?
Do a search for example for seriesId for the series "liimitless". Copy what shows up in the node server command window and paste or attach here.
 

·
Registered
Joined
·
65 Posts
Thanks for getting back to me. I'm using Windows 10, using rpcSearch129.exe. I've tried dozens of series. Closed the command window after test. Below is what comes up in the node server window when I search for limitless.

Server running on port 3000
QUERY = {"domainToken":"wuBVGMDr8ELZ.H8lcrOUGmbWGxn4.1TIftc1XPimadYEnvi4pJS1SXm5ug7fEIi7BItEIogWiueYByGw5xDUgPLC8UcTU.c5ps0QzIBOxo5qi7CzxYZOtAiqt_4aGV9i_8rjpOP7ETtnvOb9zsq_icr4VxCKqjJulvdE9v.tOhmlMU0YmcwYRf4pRP771gOM87Gdxq7SDeYCbgqX_1VEi.NVohfDvnRK","tsn":"tsn:84900019047E035"}
DOMAINTOKEN = wuBVGMDr8ELZ.H8lcrOUGmbWGxn4.1TIftc1XPimadYEnvi4pJS1SXm5ug7fEIi7BItEIogWiueYByGw5xDUgPLC8UcTU.c5ps0QzIBOxo5qi7CzxYZOtAiqt_4aGV9i_8rjpOP7ETtnvOb9zsq_icr4VxCKqjJulvdE9v.tOhmlMU0YmcwYRf4pRP771gOM87Gdxq7SDeYCbgqX_1VEi.NVohfDvnRK
TSN = tsn:84900019047E035
QUERY = {"title":"limitless","orderBy":"strippedTitle","includeBroadcast":"true","includeFree":"true","includePaid":"false","includeVod":"false","levelOfDetail":"medium","count":"50","mergeOverridingCollections":"true","filterUnavailable":"false","collectionType":"series","type":"collectionSearch"}
connected
Request = MRPC/2 253 323
Type: request
RpcId: 1
SchemaVersion: 21
Content-Type: application/json
RequestType: collectionSearch
BodyId: tsn:84900019047E035
ResponseCount: single
X-ApplicationName: Quicksilver
X-ApplicationVersion: 1.2
X-ApplicationSessionId: 2605835

{"title":"limitless","orderBy":"strippedTitle","includeBroadcast":"true","includeFree":"true","includePaid":"false","includeVod":"false","levelOfDetail":"medium","count":"50","mergeOverridingCollections":"true","filterUnavailable":"false","collectionType":"series","type":"collectionSearch","bodyId":"tsn:84900019047E035"}

{
QUERY = {"title":"limitless","namespace":"trioserver","count":"25","bodyId":"tsn:84900019047E035","orderBy":["seasonNumber","episodeNum"],"levelOfDetail":"medium","type":"offerSearch"}
Tivo mind websocket closed 1005
connected
Request = MRPC/2 248 177
Type: request
RpcId: 1
SchemaVersion: 21
Content-Type: application/json
RequestType: offerSearch
BodyId: tsn:84900019047E035
ResponseCount: single
X-ApplicationName: Quicksilver
X-ApplicationVersion: 1.2
X-ApplicationSessionId: 2605835

{"title":"limitless","namespace":"trioserver","count":"25","bodyId":"tsn:84900019047E035","orderBy":["seasonNumber","episodeNum"],"levelOfDetail":"medium","type":"offerSearch"}

{
QUERY = {"levelOfDetail":"medium","count":"5","collectionId":"tivo:cl.185032202","type":"collectionSearch"}
Tivo mind websocket closed 1005
connected
Request = MRPC/2 253 131
Type: request
RpcId: 1
SchemaVersion: 21
Content-Type: application/json
RequestType: collectionSearch
BodyId: tsn:84900019047E035
ResponseCount: single
X-ApplicationName: Quicksilver
X-ApplicationVersion: 1.2
X-ApplicationSessionId: 2605835

{"levelOfDetail":"medium","count":"5","collectionId":"tivo:cl.185032202","type":"collectionSearch","bodyId":"tsn:84900019047E035"}

{
Tivo mind websocket closed 1005
 

·
Cranky old novice
Joined
·
9,508 Posts
Discussion Starter · #776 ·
@ButchKy
I'm perplexed. I have a separate folder where I install MG3 and the node server using the contents of the zips attached to my post, and using rpcSearch129.exe. When I run limitless there my node server window output is identical to yours aside from tsn and domaintoken values, but mine goes on to one additional query. Is it possible you missed selecting the entire output when you copied?

Your node server is connecting to tivo middlemind, which indicates you have a valid domainToken. Recheck your tsn number if you haven't already. Beyond that I don't have any idea what's wrong.
 

·
Cranky old novice
Joined
·
9,508 Posts
Discussion Starter · #777 ·
Everybody: I just discovered that the 3.97T1 zip contains rpcSearch128.py instead of rpcSearch129.py. (sorry, and will provide updated distro ASAP.)

This means those running that distro WITH python 3.x installed will fail at startup with popup message. If however, you have downloaded and installed rpcSearch129.exe then MG3 will run.

Note this has no connection to the problems @ButchKy is having.
 

·
Cranky old novice
Joined
·
9,508 Posts
Discussion Starter · #778 · (Edited)
Test Distro Ver. 3.97T2 in attached zip.

fixed: Correct rpcSearch129.py file included.

Be sure to download the other two attachments to post #772
 

Attachments

·
Cranky old novice
Joined
·
9,508 Posts
Discussion Starter · #780 ·
@ButchKy

For diagnostic purposes download attached main.zip and copy main.js from it, then replace main.js in the nodeServer subfolder with it (AFTER SAVING OFF THE PREVIOUS main.js)

When you do rpc Searches this will cause the node server to display, in its command window, the responses it is sending back to MG3/rpcSearch. You are sending the correct queries and not getting any error returns, apparently just emptiness. This will either verify that or will show responses being sent so we have to figure out why MG3 doesn't receive them.
 

Attachments

761 - 780 of 957 Posts
Top