Comments (5)
Was following your post within the Plex forums and decided to do some digging within the Plex Framework.
The JSON.ObjectFromURL()
default setup is as follows within the Plex Framework:
JSON.ObjectFromURL(url, values=None, headers={}, cacheTime=None, encoding=None, errors=None, timeout=GLOBAL_DEFAULT_TIMEOUT, sleep=0, follow_redirects=True, method=None, max_size=None)
After doing some digging I found that when the response is read it checks if the encoding
value is filled. If so, it then checks the errors
value. Its setup as follows:
def __str__(self):
self.load()
if self._encoding:
if self._errors:
result = str(unicode(self._data, self._encoding, self._errors))
else:
result = str(unicode(self._data, self._encoding))
else:
result = self._data
return result
This shows how the string encoding is done for the JSON response (as-well-as all URL responses within Plex Framework). You might try adding errors="ignore"
to line 124. For more info on valid errors
values, refer here.
Note: I do not use this channel nor have the means to do so.
from tvheadend-ng.bundle.
After removing the raised exception that the author was looking for, this is the real error:
2016-08-06 01:27:02,755 (-b722470) : INFO (__init__:127) - JSON-Request failed: <type 'exceptions.UnicodeDecodeError'>
As far as I can see from the PCAP output, the gzipped response isn't unicode. Hmm.
from tvheadend-ng.bundle.
After doing some digging I found that when the response is read it checks if the encoding value is filled. If so, it then checks the errors value. Its setup as follows:
Thanks for the input. I'm not familiar enough with Python to know what are built-in functions and what belong to imports. Your response has helped me drill down further.
As for your recommendation, if you look in the code at line 127 you'll see that he attempts to make another call with ObjectFromUrl without the encoding parameter.
from tvheadend-ng.bundle.
Couple more notes about the usage with JSON.ObjectFromURL()
, when it returns the json object, it will also try and use the encoding
flag.
class JSON(SubComponent):
def _init(self):
self._lock = self._core.runtime.lock()
def from_string(self, jsonstring, encoding=None):
try:
self._lock.acquire()
return simplejson.loads(jsonstring, encoding)
except simplejson.decoder.JSONDecodeError, e:
self._core.log.warn("Error decoding with simplejson, using demjson instead (this will cause a performance hit) - %s" % str(e.args[0]))
return demjson.decode(jsonstring, encoding)
finally:
self._lock.release()
All of that said, I downloaded your response.gz
and did some test outside of Plex's Framework (and recreated how Plex behaves to track down the error). I found from your data that it gives an error when including the encoding
flag. It gives the following error for me... (note my position will be different from others as I did some formatting to read the file easier, but didn't change the encoding)
'ascii' codec can't encode character u'\xed' in position 1386: ordinal not in range(128)
This happens when I try and recreate the following portion,
str(unicode(self._data, self._encoding, self._errors))
but if I skip the encoding it works fine.
As for your recommendation, if you look in the code at line 127 you'll see that he attempts to make another call with ObjectFromUrl without the encoding parameter.
Correct, but line 31 has the global cache time set to 1 second. I'm assuming when the first JSON.ObjectFromURL()
fails it is still caching the response and reusing the bad cache for the second requests before the 1 second cache time passes.
A workaround, could be to set the JSON.ObjectFromURL()
cache time to zero, i.e.
json_data = dict()
try:
json_data = JSON.ObjectFromURL(encoding='utf-8', url=url, headers=headers, cacheTime=0)
except:
try:
json_data = JSON.ObjectFromURL(url=url, headers=headers, cacheTime=0)
except:
raise Exception("JSON encoding error")
Notes:
- I removed the
values=None
as it does not need to be included since the channel code never calls for values - Please do not copy/paste the above code blindly into the
__init__.py
file, as the spacing will be incorrect and Python will complain and not run. Make sure to use tabs instead of spaces because the author of the__init__.py
file used tabs. - You could also change each
except:
toexcept Exception, e:
and then add aLog.Error(str(e))
after each except to log each exception and track down the errors
from tvheadend-ng.bundle.
'ascii' codec can't encode character u'\xed' in position 1386: ordinal not in range(128)
Ah ha! I searched for that byte and found it. It is part of the description for a Spanish television show.
Correct, but line 31 has the global cache time set to 1 second. I'm assuming when the first JSON.ObjectFromURL() fails it is still caching the response and reusing the bad cache for the second requests before the 1 second cache time passes.
This is true, in fact you can tell this is the case because the log shows the word "fetching" when it uses the cache.
However, while working on this, I switched his code around -- starting without the encoding:
json_data = ""
try:
json_data = JSON.ObjectFromURL(url=url, headers=headers, values=None, encoding=None)
except:
e = sys.exc_info()[0]
if debug == True: Log("JSON-Request without encoding set failed: %s" % e)
try:
json_data = JSON.ObjectFromURL(encoding='utf-8', url=url, headers=headers, values=None, errors=None)
except:
e = sys.exc_info()[0]
raise Exception("JSON-Request with encoding set failed: %s" % e)
except Exception, e:
if debug == True: Log(str(e))
return False
if debug == True: Log("JSON-Request successfull!")
return json_data
Yet, the problem remains.
2016-08-07 13:22:33,320 (-b737470) : INFO (__init__:105) - JSON-Request: getEpgGrid
2016-08-07 13:22:33,352 (-b737470) : DEBUG (networking:166) - Requesting 'http://127.0.0.1:9981/api/epg/events/grid?start=0&limit=2000'
2016-08-07 13:22:33,426 (-b737470) : INFO (__init__:128) - JSON-Request without encoding set failed: <type 'exceptions.UnicodeDecodeError'>
2016-08-07 13:22:46,272 (-b737470) : DEBUG (networking:166) - Requesting 'http://127.0.0.1:9981/api/epg/events/grid?start=0&limit=2000'
2016-08-07 13:22:46,345 (-b737470) : INFO (__init__:136) - JSON-Request with encoding set failed: <type 'exceptions.UnicodeDecodeError'>
2016-08-07 13:22:46,352 (-b737470) : INFO (__init__:149) - Failed to fetch EPG!
I've continued this discussion in the Plex forums, since it looks like the problem isn't with tvheadend-ng:
https://forums.plex.tv/discussion/227780/json-objectfromurl-unicode-issue#latest
from tvheadend-ng.bundle.
Related Issues (20)
- Can't retrieve channel list HOT 20
- howto force transcoding in rasplex HOT 4
- Transcoding on Plex MediaServer HOT 1
- Passing dvb sub-streams to plex. HOT 2
- Buffer problems with Samsung Smart TV Plex App HOT 1
- Issue 27 is still occuring in latest Pull HOT 4
- Issue pulling channel list HOT 1
- Only works for an admin account HOT 2
- Andtorid/IOS HOT 1
- PMS 1.0.0.2261 HOT 23
- Getting connection error
- Not working with plex server 1.2.2.2857-d34b464 HOT 1
- Working over Browser, but not via Android-App HOT 2
- transcoding in plex fails, frame dimension errors. HOT 2
- JSON Encoding Error
- does not work in Browser nor in mobile devices / both android and iOS HOT 2
- TypeError: createTVChannelObject() got an unexpected keyword argument 'includeBandwidths' HOT 1
- Only casts to Chromecast when server is not signed in to Plex account
- Plex version 1.10.0.4523 breaks channel
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tvheadend-ng.bundle.