Git Product home page Git Product logo

Comments (25)

ZeroQI avatar ZeroQI commented on September 26, 2024 1

added option in settings, put preliminary code in the agent but am yet to find how to shortly code this

from hama.bundle.

EndOfLine369 avatar EndOfLine369 commented on September 26, 2024

The simple counter to that is performance. Especially when we consider the volume of some people's libraries and their global relation to those sites. This data is all non-critical information and does not require login to pull so from a security standpoint, it doesn't have much call.

That being said, i'm not saying to not do it but will be ZeroQI's call. 😃

from hama.bundle.

avatar commented on September 26, 2024

@EndOfLine369 :tls performance isn't really an issue anymore, especially if the site uses spdy or http/2. Often the performance over ssl is better.

from hama.bundle.

EndOfLine369 avatar EndOfLine369 commented on September 26, 2024

Its not exactly consistent. But they seem to either average out or go down. Didn't want to try a test against AniDB as it will probably get banned after the second try. This is probably all moot anyway as i'm still not saying no but just wanted to test and share.

http://thetvdb.com/api/A27AD9BE0DA63333/series/81797/all/en.xml

https://thetvdb.com/api/A27AD9BE0DA63333/series/81797/all/en.xml

http://rawgithub.com/ScudLee/anime-lists/master/anime-movieset-list.xml

https://rawgithub.com/ScudLee/anime-lists/master/anime-movieset-list.xml

from hama.bundle.

ZeroQI avatar ZeroQI commented on September 26, 2024

Security wise i don't see the point.
Performance wise buggest gain on github which is once a day at most for ScudLee's xml files

I can maybe add a setting to prefer https but need to know which links can do https and will do a table of variables not to have the double http and https links doubled if identical.

from hama.bundle.

Kezxo avatar Kezxo commented on September 26, 2024

Could also be a privacy thing also, probably not a huge one but everyone's situation is different.
I say it should be added unless it decreases performance or causes other problems.
But it is your call as the first reply said.

from hama.bundle.

avatar commented on September 26, 2024

Does anybody know what happens if hama receives a document with injected html, like in the case of captive portal or injected gunk from an ISP?

Switching to https would make sure that never becomes a problem (assuming verification is turned on)

EDIT: i should mention I use my laptop as a plex server and sometimes travel with it.

from hama.bundle.

Kezxo avatar Kezxo commented on September 26, 2024

Awesome, thanks a lot for working on adding this! 😄 👍
My first thought was just to add an s to the end of the http where it made sense, didn't think of making it optional. I guess it makes sense if https stops working or was performance the issue?

What will the default option be? I noticed that some of the tmdb.org urls are using https already, if you make http the default will you change them to http also?

Wish I had any advice for the code issue.

PS. Did https://api.anidb.net/ always have a trust error? Just noticed this. 😦 Might be good to have an option if it changes often.

from hama.bundle.

ZeroQI avatar ZeroQI commented on September 26, 2024

I am split over something as i have these two choices:
. i mass change the http to https with a list of variables (no failover back to http, short code)
. if the option is set i can try al local level in each call to load https replacing http with https on the fly, then if fails i try to load http, but need to change every bloody call in the source so could create a function to load files, and just implement it there......

Question since i don't use https, is it possible https page load fails but http works ?

from hama.bundle.

EndOfLine369 avatar EndOfLine369 commented on September 26, 2024

hey ZeroQI,
I've got something i'm working up you might like. Will look at finishing it and test to give you later today. Have you found out already if all the URLs we do have can do either http or https? or are there some that only have one option?

from hama.bundle.

ZeroQI avatar ZeroQI commented on September 26, 2024

I did a list of variables in hama source for https conversion. All included worked as https. Didn't find any that didn't to be fair when trying them in a web browser

Thanks for all the submits, code fixes went forward quite a bit thanks to you, was loosing some drive due to lack of time, and you allowed to remove some useless bits in the code and great help... Thanks again

from hama.bundle.

ZeroQI avatar ZeroQI commented on September 26, 2024

EndOfLine369, you are officially an HAMA and ASS collaborator, as I just found back how to add a collaborator.... I apologise for the lateness in giving you what is well deserved with all the bugfixes you did. Congratulations!
I wish we had a chat functionnality or something. Are you on Plex forums EndOfLine369?

from hama.bundle.

EndOfLine369 avatar EndOfLine369 commented on September 26, 2024

Thank you for the promotion 👍 🎂
Now I don't have to stalk the webpages and refresh to check for issues.
And yes, I have the same id on Plex.

from hama.bundle.

EndOfLine369 avatar EndOfLine369 commented on September 26, 2024

@ZeroQI Ugh, I have the code working. but in testing it, I got banned from anidb. 😦
Seems plex does not see the cached url even though the only difference is the http/https? Any idea if there is a way we can change that? We might just have to force anidb urls to be https so they don't change and become re-pulled w/ a ban.

2016-04-29 12:51:33,338 (-b5e1470) :  DEBUG (__init__:228) - --- Update Begin -------------------------------------------------------------------------------------------
2016-04-29 12:51:33,339 (-b5e1470) :  DEBUG (__init__:609) - Update - metadata source: 'anidb', id: '10951', Title: 'Kabaneri of the Iron Fortress',([...], [...], True)
2016-04-29 12:51:33,460 (-b5e1470) :  DEBUG (__init__:633) - anidbTvdbMapping() - anidb: '10951', tvbdid: '305082', tmdbid: '', imbdid: '', studio: '', defaulttvdbseason: '1', name: 'Koutetsujou no Kabaneri'
2016-04-29 12:51:33,471 (-b5e1470) :  DEBUG (networking:166) - Requesting 'http://tvthemes.plexapp.com/305082.mp3'
2016-04-29 12:51:33,852 (-b5e1470) :  ERROR (networking:217) - Error opening URL 'http://tvthemes.plexapp.com/305082.mp3'
2016-04-29 12:51:33,853 (-b5e1470) :  DEBUG (__init__:731) - metadata_download() - metadata_download - error downloading
2016-04-29 12:51:33,854 (-b5e1470) :  DEBUG (__init__:610) - Update() - TVDB - tvdbid: '305082', url: 'http://thetvdb.com/api/A27AD9BE0DA63333/series/305082/all/en.xml'
2016-04-29 12:51:33,854 (-b5e1470) :  DEBUG (__init__:749) - xmlElementFromFile() - url: 'http://thetvdb.com/api/A27AD9BE0DA63333/series/305082/all/en.xml', filename: 'TVDB/305082.xml'
2016-04-29 12:51:33,868 (-b5e1470) :  DEBUG (networking:161) - Fetching 'http://thetvdb.com/api/A27AD9BE0DA63333/series/305082/all/en.xml' from the HTTP cache
2016-04-29 12:51:33,891 (-b5e1470) :  INFO (__init__:610) - abs_manual_placement_info = []
2016-04-29 12:51:33,891 (-b5e1470) :  INFO (__init__:610) - abs_manual_placement_status = success
2016-04-29 12:51:33,917 (-b5e1470) :  DEBUG (__init__:610) - Update() - TVDB - tvdb_table: ['s1e1', 's1e2', 's1e3']
2016-04-29 12:51:33,918 (-b5e1470) :  DEBUG (__init__:610) - Update() - TVDB - Episodes without Summary: ['s1e10', 's1e11', 's1e12', 's1e4', 's1e5', 's1e6', 's1e7', 's1e8', 's1e9']
2016-04-29 12:51:33,932 (-b5e1470) :  DEBUG (networking:161) - Fetching 'http://thetvdb.com/api/A27AD9BE0DA63333/series/305082/banners.xml' from the HTTP cache
2016-04-29 12:51:33,945 (-b5e1470) :  DEBUG (__init__:663) - getImagesFromTVDB() - Loaded picture XML: 'http://thetvdb.com/api/A27AD9BE0DA63333/series/305082/banners.xml'
2016-04-29 12:51:33,948 (-b5e1470) :  DEBUG (__init__:740) - metadata_download() - url: 'http://thetvdb.com/banners/fanart/original/305082-1.jpg', num: '1', filename: 'TVDB/fanart/original/305082-1.jpg'*
2016-04-29 12:51:33,949 (-b5e1470) :  DEBUG (__init__:740) - metadata_download() - url: 'http://thetvdb.com/banners/fanart/original/305082-2.jpg', num: '2', filename: 'TVDB/fanart/original/305082-2.jpg'*
2016-04-29 12:51:33,951 (-b5e1470) :  DEBUG (__init__:740) - metadata_download() - url: 'http://thetvdb.com/banners/posters/305082-1.jpg', num: '2', filename: 'TVDB/posters/305082-1.jpg'*
2016-04-29 12:51:33,952 (-b5e1470) :  DEBUG (__init__:740) - metadata_download() - url: 'http://thetvdb.com/banners/posters/305082-3.jpg', num: '3', filename: 'TVDB/posters/305082-3.jpg'*
2016-04-29 12:51:33,953 (-b5e1470) :  DEBUG (__init__:740) - metadata_download() - url: 'http://thetvdb.com/banners/posters/305082-2.jpg', num: '4', filename: 'TVDB/posters/305082-2.jpg'*
2016-04-29 12:51:33,955 (-b5e1470) :  DEBUG (__init__:740) - metadata_download() - url: 'http://thetvdb.com/banners/graphical/305082-g.jpg', num: '6', filename: 'TVDB/graphical/305082-g.jpg'*
2016-04-29 12:51:33,955 (-b5e1470) :  DEBUG (__init__:610) - Update() - AniDB mode - AniDB Serie XML: http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=10951, AniDB/10951.xml
2016-04-29 12:51:33,956 (-b5e1470) :  DEBUG (__init__:749) - xmlElementFromFile() - url: 'http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=10951', filename: 'AniDB/10951.xml'
2016-04-29 12:51:37,973 (-b5e1470) :  DEBUG (networking:161) - Fetching 'http://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=10951' from the HTTP cache
...
2016-04-29 12:51:38,057 (-b5e1470) :  DEBUG (__init__:613) - --- Update end -------------------------------------------------------------------------------------------------
...
(Updated HAMA preferences and checked the https option)
...
2016-04-29 12:52:19,924 (-b5e1470) :  DEBUG (__init__:228) - --- Update Begin -------------------------------------------------------------------------------------------
2016-04-29 12:52:19,925 (-b5e1470) :  DEBUG (__init__:63) - Hama will use 'https' for all URL calls
2016-04-29 12:52:19,926 (-b5e1470) :  DEBUG (__init__:609) - Update - metadata source: 'anidb', id: '10951', Title: 'Kabaneri of the Iron Fortress',([...], [...], True)
2016-04-29 12:52:20,045 (-b5e1470) :  DEBUG (__init__:633) - anidbTvdbMapping() - anidb: '10951', tvbdid: '305082', tmdbid: '', imbdid: '', studio: '', defaulttvdbseason: '1', name: 'Koutetsujou no Kabaneri'
2016-04-29 12:52:20,080 (-b5e1470) :  DEBUG (networking:166) - Requesting 'https://tvthemes.plexapp.com/305082.mp3'
2016-04-29 12:52:20,521 (-b5e1470) :  ERROR (networking:217) - Error opening URL 'https://tvthemes.plexapp.com/305082.mp3'
2016-04-29 12:52:20,522 (-b5e1470) :  DEBUG (__init__:731) - metadata_download() - metadata_download - error downloading
2016-04-29 12:52:20,522 (-b5e1470) :  DEBUG (__init__:610) - Update() - TVDB - tvdbid: '305082', url: 'https://thetvdb.com/api/A27AD9BE0DA63333/series/305082/all/en.xml'
2016-04-29 12:52:20,523 (-b5e1470) :  DEBUG (__init__:749) - xmlElementFromFile() - url: 'https://thetvdb.com/api/A27AD9BE0DA63333/series/305082/all/en.xml', filename: 'TVDB/305082.xml'
2016-04-29 12:52:20,530 (-b5e1470) :  DEBUG (networking:166) - Requesting 'https://thetvdb.com/api/A27AD9BE0DA63333/series/305082/all/en.xml'
2016-04-29 12:52:20,966 (-b5e1470) :  INFO (__init__:610) - abs_manual_placement_info = []
2016-04-29 12:52:20,966 (-b5e1470) :  INFO (__init__:610) - abs_manual_placement_status = success
2016-04-29 12:52:20,992 (-b5e1470) :  DEBUG (__init__:610) - Update() - TVDB - tvdb_table: ['s1e1', 's1e2', 's1e3']
2016-04-29 12:52:20,993 (-b5e1470) :  DEBUG (__init__:610) - Update() - TVDB - Episodes without Summary: ['s1e10', 's1e11', 's1e12', 's1e4', 's1e5', 's1e6', 's1e7', 's1e8', 's1e9']
2016-04-29 12:52:21,020 (-b5e1470) :  DEBUG (networking:166) - Requesting 'https://thetvdb.com/api/A27AD9BE0DA63333/series/305082/banners.xml'
2016-04-29 12:52:21,377 (-b5e1470) :  DEBUG (__init__:663) - getImagesFromTVDB() - Loaded picture XML: 'https://thetvdb.com/api/A27AD9BE0DA63333/series/305082/banners.xml'
2016-04-29 12:52:21,423 (-b5e1470) :  DEBUG (__init__:739) - metadata_download() - url: 'https://thetvdb.com/banners/fanart/original/305082-1.jpg', num: '1', filename: 'TVDB/fanart/original/305082-1.jpg'
2016-04-29 12:52:21,436 (-b5e1470) :  DEBUG (__init__:739) - metadata_download() - url: 'https://thetvdb.com/banners/fanart/original/305082-2.jpg', num: '2', filename: 'TVDB/fanart/original/305082-2.jpg'
2016-04-29 12:52:21,458 (-b5e1470) :  DEBUG (__init__:739) - metadata_download() - url: 'https://thetvdb.com/banners/posters/305082-1.jpg', num: '2', filename: 'TVDB/posters/305082-1.jpg'
2016-04-29 12:52:21,477 (-b5e1470) :  DEBUG (__init__:739) - metadata_download() - url: 'https://thetvdb.com/banners/posters/305082-3.jpg', num: '3', filename: 'TVDB/posters/305082-3.jpg'
2016-04-29 12:52:21,500 (-b5e1470) :  DEBUG (__init__:739) - metadata_download() - url: 'https://thetvdb.com/banners/posters/305082-2.jpg', num: '4', filename: 'TVDB/posters/305082-2.jpg'
2016-04-29 12:52:21,523 (-b5e1470) :  DEBUG (__init__:739) - metadata_download() - url: 'https://thetvdb.com/banners/graphical/305082-g.jpg', num: '6', filename: 'TVDB/graphical/305082-g.jpg'
2016-04-29 12:52:21,524 (-b5e1470) :  DEBUG (__init__:610) - Update() - AniDB mode - AniDB Serie XML: https://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=10951, AniDB/10951.xml
2016-04-29 12:52:21,525 (-b5e1470) :  DEBUG (__init__:749) - xmlElementFromFile() - url: 'https://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=10951', filename: 'AniDB/10951.xml'
2016-04-29 12:52:25,548 (-b5e1470) :  DEBUG (networking:166) - Requesting 'https://api.anidb.net:9001/httpapi?request=anime&client=hama&clientver=1&protover=1&aid=10951'

from hama.bundle.

EndOfLine369 avatar EndOfLine369 commented on September 26, 2024

@ZeroQI, see what you think. It should be easy to fix a url to either http or https.
https://gist.github.com/EndOfLine369/8ee08c812a5f3943e6cc/revisions

from hama.bundle.

ZeroQI avatar ZeroQI commented on September 26, 2024

Python has a variable called dict which is basically a dictionary of the name of every global variable (other than dict) and its value. So saying dict['foo']='bar' is the same as foo='bar

if Prefs['https']:
https_list = [
'ANIDB_TITLES', 'ANIDB_TVDB_MAPPING', 'ANIDB_COLLECTION', 'ANIDB_HTTP_API_URL', 'ANIDB_PIC_BASE_URL', 'ANIDB_SERIE_URL',
'ANIDB_TVDB_MAPPING_FEEDBACK',
'TVDB_HTTP_API_URL', 'TVDB_BANNERS_URL', 'TVDB_SERIE_SEARCH', 'TVDB_SERIE_URL',
'TMDB_MOVIE_SEARCH', 'TMDB_MOVIE_SEARCH_BY_TMDBID', 'TMDB_SEARCH_URL_BY_IMDBID', 'TMDB_CONFIG_URL', 'TMDB_IMAGES_URL',
'OMDB_HTTP_API_URL',
'THEME_URL' ]
for value, key in https_list: dict[key] = value.replace("http://", "https://" )

There is some some speed improvement + security with https but the code above however doesn't handle the failover from https to http


Question are:

  • can https fail when http does not (certificate for example) ? If not then can we not just replace http with https in URL? i would advise that to people that want https for testing in the meantime
  • if https can fail with http working we need either a placeholder function or a failover loop for every url load like: if https and url load https else url load http

from hama.bundle.

EndOfLine369 avatar EndOfLine369 commented on September 26, 2024

'dict' is just the dictionary class

>>> help(dict)
Help on class dict in module builtins:

class dict(object)
 |  dict() -> new empty dictionary
 |  dict(mapping) -> new dictionary initialized from a mapping object's
 |      (key, value) pairs
 |  dict(iterable) -> new dictionary initialized as if via:
 |      d = {}
 |      for k, v in iterable:
 |          d[k] = v
 |  dict(**kwargs) -> new dictionary initialized with the name=value pairs
 |      in the keyword argument list.  For example:  dict(one=1, two=2)
 |
 |  Methods defined here:
...

from hama.bundle.

ZeroQI avatar ZeroQI commented on September 26, 2024

__globals__ should make this possible
https://docs.python.org/2/reference/datamodel.html?highlight=__globals__

from hama.bundle.

ZeroQI avatar ZeroQI commented on September 26, 2024

Source ZeroQI/Absolute-Series-Scanner#40 (comment)
It seems it will bring some caching issues, which does make sense. changing variables in the code to https did work for all URLs, and they are conveniently located at the top of the source code

Works - http://rawgithub.com/ZeroQI/Absolute-Series-Scanner/master/tvdb4.posters.xml

2016-06-19 10:32:34,819 (-ba70470) :  DEBUG (networking:161) - Fetching 'http://rawgithub.com/ZeroQI/Absolute-Series-Scanner/master/tvdb4.posters.xml' from the HTTP cache
2016-06-19 10:32:34,832 (-ba70470) :  DEBUG (__init__:697) - getImagesFromASS() - Loaded poster XML: 'http://rawgithub.com/ZeroQI/Absolute-Series-Scanner/master/tvdb4.posters.xml'

Failed to load - https://rawgithub.com/ZeroQI/Absolute-Series-Scanner/master/tvdb4.posters.xml

2016-06-19 10:26:02,484 (-b5b7470) :  DEBUG (networking:166) - Requesting 'https://rawgithub.com/ZeroQI/Absolute-Series-Scanner/master/tvdb4.posters.xml'
2016-06-19 10:26:02,610 (-b5b7470) :  DEBUG (__init__:696) - getImagesFromASS() - Loading poster XML failed: https://rawgithub.com/ZeroQI/Absolute-Series-Scanner/master/tvdb4.posters.xml

Loads but would not cache : https://raw.githubusercontent.com/ZeroQI/Absolute-Series-Scanner/master/tvdb4.posters.xml

2016-06-19 09:57:56,638 (-b675470) :  DEBUG (networking:166) - Requesting 'https://raw.githubusercontent.com/ZeroQI/Absolute-Series-Scanner/master/tvdb4.posters.xml'
2016-06-19 09:57:56,951 (-b675470) :  DEBUG (networking:198) - Not caching 'https://raw.githubusercontent.com/ZeroQI/Absolute-Series-Scanner/master/tvdb4.posters.xml' (content type 'text/plain; charset=utf-8' not cacheable in Agent plug-ins)

from hama.bundle.

ZeroQI avatar ZeroQI commented on September 26, 2024

While there is a speed gain sometimes, it seems this prevent caching which Hama relies heavily on to avoid getting banned by AniDB. I am not ready to remove plex caching for all so a few could avoid https login issues while roaming the Plex Media Server (what's the point in a streaming server if you bring it with you ?)

There also is loading issues with some url as above, but you can edit the source and replace all http to https addresses, i did look into it and it did load each page but i did not want to add a fallback from https to http for every single URL when it works currently.

from hama.bundle.

ZeroQI avatar ZeroQI commented on September 26, 2024

Due to last issue with https and github,hama stopped for days, and https link to github was involved, i will not implement https at this time.

from hama.bundle.

EndOfLine369 avatar EndOfLine369 commented on September 26, 2024

Will you be removing the json option and the 'ValidatePrefs' string?

from hama.bundle.

ZeroQI avatar ZeroQI commented on September 26, 2024

doh! forgot i put that there
done

from hama.bundle.

Kezxo avatar Kezxo commented on September 26, 2024

Shame it didn't work out, thanks for giving it a try! 😄

Is there some ssl bug with github that might be fixed later by them?

from hama.bundle.

ZeroQI avatar ZeroQI commented on September 26, 2024

certificate issues would break it when they expire, some github url are ensured by other services adding headers for example. you can replace http with https and when i tested (prior github bug) all worked so for me. since it wend down for a week about, i prefer stability over a functionality that i can't really understand (security in downloading serie xml and posters). if you have PMS on the go and internet, why not leaving pms on a server at home, and stream the info, or do the meta refresh before leaving home??

from hama.bundle.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.