TiVo Community
TiVo Community
TiVo Community
Go Back   TiVo Community > Underground Playground > Developers Corner
TiVo Community
Reply
Forum Jump
 
Thread Tools
Old 11-26-2014, 10:33 AM   #1
wr0ngway
Registered User
 
Join Date: Jan 2014
Posts: 10
HMO text encoding for metadata

Does anyone know what charset/encoding the xml response for HMO requests need to be in? I'm running into a encoding display issue with my tivohmo ruby project ( https://github.com/wr0ngway/tivohmo ).

The problem I'm seeing is that metadata (specifically description) that I fetch from plex, has some unicode values (specifically U+2019 - which is a apostrophe), and these show up as empty boxes when browsing the item in the tivo HMO browse UI. However, once I transfer it to the tivo, it does show up correctly as an apostrophe in the details for the show in the "My Shows" UI.

I'm doing everything in utf-8, and both the QueryContainer and TVBusQuery responses are in utf-8. I've tried setting Content-Type response header to 'text/xml', 'application/xml', 'text/xml; charset=utf-8', 'application/xml; charset=utf-8', but makes no difference.

The only place I don't use utf-8 is when transferring the file to the tivo - I generate the TVBusQuery xml as utf-8, but dump the raw bytes into the header since it is a binary stream - which may be why it works in the tivo UI, but not the hmo ui ...? https://github.com/wr0ngway/tivohmo/...server.rb#L163

If I manually create a pyTivo metadata file containing that string, the same things happen.

Any ideas?
wr0ngway is offline   Reply With Quote
Old 11-27-2014, 01:27 AM   #2
davidblackledge
Registered lÜser
 
davidblackledge's Avatar
 
Join Date: Sep 2008
Location: NM
Posts: 258
Well, my first thought is the HMO app is probably implemented with HME, and the fonts available to HME don't handle a lot of characters right (I have a whole character downgrade mapping I do for EWz so it can display pages with fewer boxes), but the HD UI has better fonts.
I'd bet if you had a file name with such a character served by TiVo Desktop, you'd probably have the same behavior.
__________________
EnterWebz.tv
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
+ air qual / lightning maps, minesweeper, 1p/2p card games...

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
David.Blackledge.com Solitaire/FreeCell, Calendar, Map...
hmedev.wikidot.com
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
davidblackledge is offline   Reply With Quote
Old 11-28-2014, 07:59 AM   #3
wr0ngway
Registered User
 
Join Date: Jan 2014
Posts: 10
Ok, thanks, that makes sense. I guess I'll just add transliteration whilst browsing, but leave the full utf8 string in place for the transfer.
wr0ngway is offline   Reply With Quote
Old 11-28-2014, 09:58 AM   #4
davidblackledge
Registered lÜser
 
davidblackledge's Avatar
 
Join Date: Sep 2008
Location: NM
Posts: 258
Quote:
Originally Posted by wr0ngway View Post
Ok, thanks, that makes sense. I guess I'll just add transliteration whilst browsing, but leave the full utf8 string in place for the transfer.
Here's what EWz currently uses:
Code:
			fixTextMap.put((char)0x00a0, " ");
			// en/em space, thinspace
			fixTextMap.put((char)0x2002, " ");
			fixTextMap.put((char)0x2003, "  ");
			fixTextMap.put((char)0x2009, " ");
			
			// commonly used horizontal ellipses character (…)
			fixTextMap.put((char)0x2026, "...");
			// quotes
			fixTextMap.put((char)0x2018, "'"); // ' left
			fixTextMap.put((char)0x2019, "'"); // ' right
			fixTextMap.put((char)0x201C, "\""); // " left
			fixTextMap.put((char)0x201D, "\""); // " right
			// em-dash (and en-dash) appears to be a problem, too
			fixTextMap.put((char)0x2013, "-");
			fixTextMap.put((char)0x2014, "--");
			// how often do the rest of these get used? just slows down the conversion IMO.
//			fixTextMap.put((char)0x201B, "'"); // high 9
//			fixTextMap.put((char)0x201A, ","); // low 9	
//			fixTextMap.put((char)0x201F, "\""); // " high 9
//			fixTextMap.put((char)0x201E, ",,"); // " low 9
			fixTextMap.put((char)0x2022, ""+((char)0x00B7)); // bullet - use mid-dot
			fixTextMap.put((char)0x2122, "tm"); // trademark
//			fixTextMap.put((char)0x301D, "\""); // double prime left
//			fixTextMap.put((char)0x301E, "\""); // double prime right
//			fixTextMap.put((char)0x301F, ",,"); // low double prime

__________________
EnterWebz.tv
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
+ air qual / lightning maps, minesweeper, 1p/2p card games...

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
David.Blackledge.com Solitaire/FreeCell, Calendar, Map...
hmedev.wikidot.com
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
davidblackledge is offline   Reply With Quote
Old 11-28-2014, 11:10 AM   #5
wr0ngway
Registered User
 
Join Date: Jan 2014
Posts: 10
Thanks - I'm actually using iconv for this, but I fed a number of your examples through it to make sure it was doing the right thing. Strangely enough, iconv has been deprecated in ruby for a while, but I still haven't found anything else that transliterates as well as it does, so I'm sticking with it for now

Code:
      def transliterate(s)
        converter = Iconv.new 'ASCII', 'UTF-8'
        converter.transliterate = true
        converter.conv(s) rescue s
      end

wr0ngway is offline   Reply With Quote
Reply
Forum Jump




Thread Tools


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Advertisements

TiVo Community
Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
vBulletin Skins by: Relivo Media

(C) 2013 Magenium Solutions - All Rights Reserved. No information may be posted elsewhere without written permission.
TiVo® is a registered trademark of TiVo Inc. This site is not owned or operated by TiVo Inc.
All times are GMT -5. The time now is 01:13 PM.
OUR NETWORK: MyOpenRouter | TechLore | SansaCommunity | RoboCommunity | MediaSmart Home | Explore3DTV | Dijit Community | DVR Playground |