Which TiVo API for Show List and Forced Playing on TiVo

Discussion in 'Developers Corner' started by Eric2XU, Mar 18, 2016.

  1. Eric2XU

    Eric2XU New Member

    15
    0
    Mar 18, 2016
    Hey all,

    I recently setup my Amazon Echo to Control my Harmony Hub. Got a few sweet NodeJS apps in the middle that glue it all together. Works great for doing what a controller can do.

    However I really want to take things to the next level.

    I want to be able to say "Alex tell TiVo to play the daily show" and in turn my code would go looking for a show called the daily show, figure out which one is latest, then tell the TiVo to put it on.

    I know this is possible as kmttg is able to both get a list of shows as well can remote trigger playing of a show.

    Can anyone help me figure out how to do it? I am assuming its some sort of web call? Given its encrypted I am unable to sniff the traffic on kmttg :(
     
  2. Eric2XU

    Eric2XU New Member

    15
    0
    Mar 18, 2016
    Ok already learned a few things. First kmttg is using MindRPC on port 1413.

    This is some sort of ssl based JSON which requires a client certificate. What I dont understand is how kmttg got that cert and could I use the cert in kmttg?

    Next I have zero idea where to learn what type of communications MindRPC is actually doing let alone do it in NodeJS.

    Anyone have any guidance?
     
  3. gonzotek

    gonzotek tivo_xml developer

    2,538
    59
    Sep 24, 2004
    Outside...
    Hmm interesting project ..it might actually be a lot simpler to leverage the work kmttg already does...it has a web server component that you could probably consume and interact with via node fairly easily. The one thing we'd need that it doesn't do yet (but I'm sure it could be made to) is start playback on the TiVo from the web ui. If you're interested in this route, I'd talk to Kevin in the kmttg thread about adding a 'play on tivo' option to the stream.htm page of kmttg. Beyond this project I can see other times I'd want the same playback option available via the web ui. And I'd be up for testing and maybe even helping with the code for this too!

    There's not really much MindRPC documentation, much of what is known about it is in the kmttg source (and kmttg thread above and this one from the tivo underground forum). The cert is another issue too..

    Btw, I have one but haven't gotten around to Echo programing just yet; but I am using harmonyhubjs-client as the basis for a script syncing my two hubs' activities ;-) One remote for me, one for the wife :)
     
  4. Eric2XU

    Eric2XU New Member

    15
    0
    Mar 18, 2016
    Thanks gonzotek!

    I am going to give MindRPC via Nodes TLS module a try. I was able to get a certificate private key and password (for reasons of not upsetting anyone I am not going to go into details on that part).

    It may be a few days before I have time to give it a go but will see if I can get that to work. The idea of being able to query TiVo directly, get back JSON, and take action is too good to pass up. Given my limited code abilities I may still fail to do so. If I do I would be super happy to help and try to get play put in the web UI. That would likely be enough to use it as a middleware.

    Anyways thanks again for posting back, will let you know how it goes. I will gladly share anything I learn.
     
  5. Eric2XU

    Eric2XU New Member

    15
    0
    Mar 18, 2016
    So I took a little more time on my lunch break today and I am a little stuck.

    If I run curl against the tivo using the certs I pulled from another project (which I am 99% sure are valid) I get "Unknown CA" I get the same message using Node JS TLS connect method with the certs, using Wireshark I can decrypt the exchange using the private key and see the TiVo responding with:

    Transmission Control Protocol, Src Port: 1413 (1413), Dst Port: 53699 (53699), Seq: 4331, Ack: 2015, Len: 7
    Secure Sockets Layer
    TLSv1.2 Record Layer: Alert (Level: Fatal, Description: Unknown CA)
    Content Type: Alert (21)
    Version: TLS 1.2 (0x0303)
    Length: 2
    Alert Message
    Level: Fatal (2)
    Description: Unknown CA (48)

    Here is the CURL output as well.

    slice@pi:/var/tmp$ curl -k --cert tivo.cert \
    > --key tivo.key \
    > -H 'Accept: application/json' \
    > 'https://10.0.0.32:1413/mind/mind11?type=infoGet' \

    curl: (35) error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca

    Anyone have any thoughts? Will pick up again in another day or so, got to get back to work. My only thought is I screwed something up splitting the p12 file into cert and key files although they seem valid enough to be used to decrypt in wireshark. Perhaps I need to supply the full chain in the cert? Although I am not sure how to include more then one public cert in a pem (cert) file.
     
    Last edited: Mar 21, 2016
  6. Eric2XU

    Eric2XU New Member

    15
    0
    Mar 18, 2016
    Ok final update for the day, I think I am past the SSL issues. Adding all public certs in the chain to my .cert file seems to have made the key exchange happy.

    I simply can not wrap my head around MPRC. Its not https but its similar. So instead of using the HTTPS library I am using the TLS for node. I open a connection to port 1413, I past in a header payload from the examples here:

    docs.google.com/document/pub?id=1e4ymm7ROwmW6co2pKENjANGT5xM00VzmzybZ4u8yDE8#h.5befa7kbkyui

    And I get nothing back. Here is my code if anyone is interested. If I can simply figure out how to communicate back and forth with with the TiVo I would be home free:

    Code:
    var tls = require('tls');
    var fs = require('fs');
    process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
    
    var header = (function () {/*
    MRPC/2 225 85
    Type:request
    RpcId:20
    SchemaVersion:7
    Content-Type:application/json
    RequestType:bodyAuthenticate
    ResponseCount:single
    BodyId:
    X-ApplicationName:Quicksilver
    X-ApplicationVersion:1.2
    X-ApplicationSessionId:0x27dc20
    */}).toString().replace('function () {/*', '').replace('*/}', '').trim();
    
    console.log("----------");
    console.log(header);
    console.log("----------");
    
    
    var options = {
    	host : "192.168.1.10"
    	,port : 1413
    	,headers: header
    	,key  : fs.readFileSync('c:\\temp\\tivo.key')
    	,cert : fs.readFileSync('c:\\temp\\tivo.cert')
    };
    
    var client = tls.connect(options, function () {
    	client.write('{"type":"bodyAuthenticate","credential":{"type":"makCredential","key":"1234567890"}}');
    });
    
    client.on('data', function (data) {
    	console.log(data.toString());
    	client.end();
    });
    
     
    Last edited: Mar 21, 2016
  7. Apr 1, 2016 #7 of 27
    Eric2XU

    Eric2XU New Member

    15
    0
    Mar 18, 2016
    Ok final post, got it all working.

    Here's the deal.

    First you need to authenticate to the TiVo. This is pretty simple after you get over the amazing hurdle about the custom protocol. This makes most off the shelf libraries useless. I used Node.js's TLS library which is a SSL wrapper for their direct socket (net) library. Once you have the language sorted, you have create a custom header and body. The first line is special as well, you put MRPC/2 which tells it the version of protocol, next number is the total number or characters in the header including the \r\n (blank line) between the header and body which is part of the header. Final number on the first line is the total characters in the body. Oh and the cert, I dont want to go into details to not piss off TiVo but I do the same thing other projects (sorry for the lack of detail there). Its down right stupid TiVo doesn't have a public API.
    Code:
    MRPC/2 235 85
    
    Type: request
    RpcId: 1
    SchemaVersion: 17
    Content-Type: application/json
    RequestType: bodyAuthenticate
    ResponseCount: single
    BodyId:
    X-ApplicationName: Quicksilver
    X-ApplicationVersion: 1.2
    X-ApplicationSessionId: 0x27b520
    
    {"type":"bodyAuthenticate","credential":{"type":"makCredential","key":"<TiVoMACKey>"}}
    Then the trick is not to close the socket, you have to turn right around on the same connection for the rest of your requests. Each new request you keep the seasonId, change the type to whatever type you are using (type is also in the body) and rev the RpcID by 1.

    Oh the sessionID is a number within a range, forget the range but this is the code I use:
    Code:
    var sessionID = Math.floor(Math.random() * (2612256 - 2539520) + 2539520).toString(16)
    Next do a recordingFolderItemSearch to return all parent folder recordings, I set flatten to false so the return is small and manageable:
    Code:
    "flatten":false,"offset":0,"bodyId":"tsn:848xxxxxxxxxx","type":"recordingFolderItemSearch"}
    You will need the TSN for that, if you dont have it you can run this:
    Code:
    { "type": "bodyConfigSearch", "bodyId": ""}
    Next you loop that looking for a match on results.recordingFolderItem[x].title. Once you find that you are looking for the collectionID and folderItemCount of that record. Go back to the TiVo and run a recordingSearch as many times is equal to folderItemCount. Yup thats right you have to rerun the query as many times as there are recordings (folderItemCount):
    Code:
    '{"bodyId":"tsn:848xxxxxxxxxx","collectionId": "<collectionId>","type":"recordingSearch","offset":<folderItemCount++>}
    You are going to be looking for the results.recording[x].recordingId for the specific show you want to play. Once you have it run uiNavigate on the TiVo:
    Code:
    { "type":"uiNavigate", "uri":"x-tivo:classicui:playback", "parameters": { "fUseTrioId":"true", "fHideBannerOnEnter":"false", "recordingId":"<recordingId>"}}
    That should be it. Hope it helps anyone else that needed to get a jump start on coding for the TiVo.
     
  8. Connor

    Connor Member

    39
    0
    Oct 12, 2002
    Eric2XU,

    Did you get it working completely ? I basic remote stuff working using this..

    https://github.com/natejgreene/alexa_tivo

    But, I want to add some commands like. "Echo, tell Tivo to load netflix" or Echo, tell Tivo to load Amazon". Using the basic remote control stuff won't work.. So, looks like I'm going to have to do some stuff like your doing and add it to my app. Are you willing to share you code on github?

    Thanks, Connor
     
  9. Eric2XU

    Eric2XU New Member

    15
    0
    Mar 18, 2016
    I got it working but ironically I dont use it much because when you start a show thru this method you lose the commercial skip (a non-starter for my wife).

    Anyhow, here is code that will work. You are more then welcome to reuse in your library you are making on Github.

    Code:
    var tls = require('tls');
    var fs = require('fs');
    var async = require('async');
    
    process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
    
    var eol = "\r\n";
    var RpcId = 0
    var options = {
    	host : "192.168.1.11"
    	,rejectUnauthorized: false
    	,port : 1413
    	,pfx : fs.readFileSync('c:\\temp\\xxxxx.p12')
    	,passphrase : "xxxxx"
    	,ca : [fs.readFileSync('c:\\temp\\tivo.ca'), fs.readFileSync('c:\\temp\\tivo.int')]
    	,secureProtocol: "TLSv1_1_method"
    };
    
    function GetSessionID() { 
    	return (Math.floor(Math.random() * (2612256 - 2539520) + 2539520).toString(16))
    }
    
    
    function buildPayLoad(sessionID, type,body) {
    	RpcId++
    	var header = "Type: request" + eol;
    	header = header + "RpcId: " + RpcId + eol;
    	header = header + "SchemaVersion: 17" + eol;
    	header = header + "Content-Type: application/json" + eol;
    	header = header + "RequestType: " + type + eol;
    	header = header + "ResponseCount: single" + eol;
    	header = header + "BodyId: " + eol;
    	header = header + "X-ApplicationName: Quicksilver" + eol;
    	header = header + "X-ApplicationVersion: 1.2" + eol;
    	header = header + "X-ApplicationSessionId: 0x" + sessionID + eol;
    	header = header + eol;
    	
    	var bodyline = body + "\n"
    
    	var firstline = "MRPC/2 " + header.length + " " + bodyline.length + eol
    	return firstline + header + bodyline
    }
    
    function Timer(callback, time) {
    	this.setTimeout(callback, time);
    }
    
    Timer.prototype.setTimeout = function (callback, time) {
    	var self = this;
    	if (this.timer) {
    		clearTimeout(this.timer);
    	}
    	this.finished = false;
    	this.callback = callback;
    	this.time = time;
    	this.timer = setTimeout(function () {
    		self.finished = true;
    		callback();
    	}, time);
    	this.start = Date.now();
    }
    
    Timer.prototype.add = function (time) {
    	if (!this.finished) {
    		time = this.time - (Date.now() - this.start) + time;
    		this.setTimeout(this.callback, time);
    	}
    }
    
    function talkToTivo(type, body, callback, offset) {
    	socket = tls.connect(options, function () {
    		socket.setEncoding('utf8');
    		var sessionID = GetSessionID()
    		
    		var bodyAuth = buildPayLoad(sessionID, 'bodyAuthenticate', '{"type":"bodyAuthenticate","credential":{"type":"makCredential","key":"4685330376"}}');
    		socket.write(bodyAuth);
    		
    		//var tivoReq = buildPayLoad(sessionID, type, body);
    		
    		if (offset != null) {
    			function multiPayload(type, body, offset) {
    				var payload = buildPayLoad(sessionID, type, (body.replace('}', ',"offset":"' + offset + '"}')))
    				socket.write(payload);
    			}
    			
    			for (var i = 0; i < offset; i++) {
    				setTimeout(multiPayload, 100, type, body, i);
    			}
    		} else {
    			var payload = buildPayLoad(sessionID, type, body)
    			setTimeout(function () { socket.write(payload); }, 100);
    		}
    		
    				
    	
    		socket.on('connect', function (blah) {
    			console.log("-------Connected Start----------");
    			
    			console.log("-------Connected end----------");
    		});
    		
    		var str = "";
    		
    		socket.on('data', function (chunk) {
    			str += chunk;
    			timer.add(50);
    		});
    		
    		socket.on('lookup', function (blah) {
    			console.log("-------Lookup Fired START----------");
    			//console.log(blah)
    			console.log("-------Lookup Fired END----------");
    		});
    		
    		socket.on('drain', function (blah) {
    			console.log("-------drain Fired START----------");
    			//console.log(blah)
    			console.log("-------drain Fired END----------");
    		});
    		
    		socket.on('close', function (blah) {
    			console.log("-------close Fired START----------");
    			//console.log(blah)
    			console.log("-------close Fired END----------");
    		});
    		
    		socket.on('timeout', function (blah) {
    			console.log("-------timeout Fired START----------");
    			//console.log(blah)
    			console.log("-------timeout Fired END----------");
    		});
    		
    		socket.on('end', function () {
    			console.log("--------CONNECTION ENDED-----------")
    			//console.log(str)
    			callback(str)
    		});
    		
    		socket.on('error', function (err) {
    			console.log("Error during TLS request");
    			console.log(err);
    			socket.end();
    		});
    		
    		var timer = new Timer(function () { 
    			console.log("[TIMEOUT] OVER!");
    			socket.end();
    		}, 500);
    
    	});
    }
    
    function parseTiVosMess(dataStream) {
    	headerInfo = dataStream.split("\r\n", 1).toString()
    	version = (headerInfo.split(" ", 3))[0]
    	headerSize = parseInt((headerInfo.split(" ", 3))[1]) + headerInfo.length
    	bodySize = (headerInfo.split(" ", 3))[2]
    	body = JSON.parse(dataStream.substring(parseInt(headerSize), dataStream.length).trim());
    	
    	return body
    }
    
    var searchString = 'BonES'
    
    talkToTivo('recordingFolderItemSearch', '{"flatten":false,"offset":0,"bodyId":"tsn:8480001901XXXXX","type":"recordingFolderItemSearch"}', function (tivoOutput1) {
    	tivoSplits1 = tivoOutput1.split("MRPC/2 ")
    	myJSON1 = parseTiVosMess("MRPC/2 " + tivoSplits1[2]);
    	for (var m in myJSON1.recordingFolderItem) {
    		if (myJSON1.recordingFolderItem[m].title.toString().toUpperCase() == searchString.toUpperCase()) {
    			// Found a Match on Name
    			var collectionId = myJSON1.recordingFolderItem[m].collectionId
    			var folderItemCount = myJSON1.recordingFolderItem[m].folderItemCount
    			searchResults = []
    
    			// Asking TiVo for Actual Recordings, have to re-ask for each recording, I use FolderItemCount to loop it in the RPCRequest Functio above
    			talkToTivo('recordingSearch', '{"bodyId":"tsn:8480001901XXXXX","collectionId": "' + myJSON1.recordingFolderItem[m].collectionId + '","type":"recordingSearch"}', function (tivoOutput2) {
    				tivoSplits2 = tivoOutput2.split("MRPC/2 ")
    				console.log(Object.keys(tivoSplits2).length - 2, folderItemCount)
    				for (var i = 2; i < folderItemCount + 2; i++) {
    					myJSON2 = parseTiVosMess("MRPC/2 " + tivoSplits2[i]);
    					if (myJSON2.recording) {
    						//Push any returns to a var called searchResults
    						searchResults.push(myJSON2.recording[0])
    					}
    				}
    				//console.dir(searchResults)
    
    				// Sort my results 
    				searchResultsSorted = searchResults.sort(function (a, b) {
    					return new Date(a["originalAirdate"]).getTime() - new Date(b["originalAirdate"]).getTime()
    				})
    				
    				// Figure out how many shows
    				var totalRecords = Object.keys(searchResultsSorted).length;
    				
    				// Used just for troubleshooting
    				for (var i = 0; i < totalRecords; i++) {
    					console.log(searchResultsSorted[i].originalAirdate);
    				}
    				console.log("Latest: " + searchResultsSorted[totalRecords - 1].originalAirdate + "|" + searchResultsSorted[totalRecords - 1].recordingId);
    				console.log("Oldest: " + searchResultsSorted[0].originalAirdate + "|" + searchResultsSorted[0].recordingId);
    				
    				//////////
    				///
    				/// Finally Launch Show, right now hard coded for first one but using the code above you can craft to do earliest or lastest
    				///
    				/////////
    				//var tivoReq = ['uiNavigate', '{ "type":"uiNavigate", "uri":"x-tivo:classicui:playback", "parameters": { "fUseTrioId":"true", "fHideBannerOnEnter":"false", "recordingId":"' + searchResultsSorted[0].recordingId + '"}}']
    				//RPCRequest(tivoReq, function (daData) {
    				//	console.log(daData)
    				//})
    				talkToTivo('uiNavigate', '{ "type":"uiNavigate", "uri":"x-tivo:classicui:playback", "parameters": { "fUseTrioId":"true", "fHideBannerOnEnter":"false", "recordingId":"' + searchResultsSorted[0].recordingId + '"}}', function (tivoOutput3) {
    					console.log(tivoOutput3)
    				})
    
    			}, folderItemCount)
    		}
    	}
    })
    
     
  10. bradleys

    bradleys It'll be fine....

    4,116
    236
    Oct 31, 2007
    That is an interesting project... Now figure out how to get Echo to change inputs on the TV and that would be cool!
     
  11. Eric2XU

    Eric2XU New Member

    15
    0
    Mar 18, 2016
    I got that working well using Logitech Harmony. I say "turn on <device>" to Echo and my middleware hands off to harmony which does all the heavy lifting.
     
  12. Connor

    Connor Member

    39
    0
    Oct 12, 2002
    Looking to install a Rasberry pi to handle the TiVo. Also want to add a IR blaster to turn on my TV. I just want to say Echo, turn on TV.

    Ideally I want 2 blasters. One for my bedroom and one for living room. Single Ras Pi handling the TiVo Control.

    Thoughts?
     
  13. Connor

    Connor Member

    39
    0
    Oct 12, 2002
    What are you using for the .p1, the .ca and .int files ?
     
  14. windracer

    windracer joined the 10k club

    12,260
    216
    Jan 3, 2003
    St. Pete, FL
    I've actually been working on something like this for the past few days. I saw that project on GitHub but actually tried this one which had a little more work done on it:

    https://github.com/grgisme/alexa_tivo_control

    And my fork is here:

    https://github.com/jradwan/alexa_tivo_control

    I'm still testing, and haven't had anyone try it yet, but you can say things like "Alexa, tell TiVo to launch Plex" or "Alexa, tell TiVo to toggle QuickMode" and it will basically send macros of remote commands over the TCP interface to the TiVo.

    Being able to say "Alexa, tell TiVo to play the Daily Show" would be cool ... I'll have to keep that in mind!
     
  15. Connor

    Connor Member

    39
    0
    Oct 12, 2002
    Hmm.. I've not done much with node.js. I just pulled this down. How is this one invoked? The other one I used would run under node (node app.js) .. I had to modify it ti include https and include my SSL certs..
     
  16. windracer

    windracer joined the 10k club

    12,260
    216
    Jan 3, 2003
    St. Pete, FL
    It still runs under Node, but under the alexa-app-server as I wanted to easily be able to have multiple Alexa skills running and that seemed to be the quickest/easiest way at the time as I was learning.
     
  17. Connor

    Connor Member

    39
    0
    Oct 12, 2002
    Hmm.. is the repo missing something then? Because package.json didn't have anything in it about alexa-app-server, nor does the app.js I say your wiki and it says to run server.js which isn't in the repo either..
     
  18. windracer

    windracer joined the 10k club

    12,260
    216
    Jan 3, 2003
    St. Pete, FL
    I didn't put alexa-app-server in the package.json as a dependency since my skill really runs inside it (a different kind of dependency). It assumes you already have the alexa-app-server installed and running.
     
  19. Connor

    Connor Member

    39
    0
    Oct 12, 2002
    Is this the one your talking about?

    https://www.npmjs.com/package/alexa-app-server

    How do you integrate your app into it? Again, a bit new to node.js
     
  20. windracer

    windracer joined the 10k club

    12,260
    216
    Jan 3, 2003
    St. Pete, FL
    Yes, that's the one. Did you check the Installation Instructions in the wiki? I've got links in the README.md as well.

    I'm new to all of this too (at least, just learned it in the past week or so). Once you have Node installed, you should be able to 'npm install alexa-app-server.' At that point you should be able to run 'node server.js' and see the example apps running. Clone my repo into the example/apps directory of the alexa-app-server, edit the config file, and start the alexa-app-server and you should see my skill get registered as shown in the documentation.
     

Share This Page