indigo-dc / flaat Goto Github PK

View Code? Open in Web Editor NEW

11.0 10.0 6.0 520 KB

FLAsk with Access Tokens - FLAAT

License: MIT License

Python 99.53% Makefile 0.47%

deep-hybrid-datacloud

flaat's Introduction

FLAAT

Use decorators for authorising access to OIDC authenticated REST APIs. Supports Flask, FastAPI and AIOHTTP.

Installation

FLAAT is available on PyPI. Install using pip:

pip install flaat

You can also install from the git repository:

git clone https://github.com/indigo-dc/flaat
pip install -e ./flaat

Documentation

The documentation is available at readthedocs.

Development

Instructions on development, testing and releasing versions can be found here.

License

FLAAT is provided under the MIT License.

flaat's People

Contributors

Stargazers

Watchers

Forkers

wetzel-desy dianagudu borjaest lburgey giosava94 vrbanecd

flaat's Issues

Error handling for expired tokens

The line below overwrites the previously logged errors, which are actually useful and more specific.

flaat/flaat/__init__.py

Line 501 in 417341d

 self.set_last_error(F"No information about user found in {str(self.get_claim_search_precedence())}") 

The example I tested with is an expired token. Without this line, the error would be more informative:

ERROR - Incoming request [127.0.0.1:56796--http://localhost:8080/user/get_status] http status: 401 - Token expired for 1540 seconds

wlcg.groups is only in AT and not in userinfo

Make sure this information can still be used for authorisation.

Avoid configuration at import time

See the following recommendation from flask main page: Configuration Best Practices

Do not write code that needs the configuration at import time. If you limit yourself to request-only accesses to the configuration you can reconfigure the object later on as needed.

As example, this functions set_client_id/secret on the flask example are implemented to run at import time. Currenty I cannot factorise my application as I have to pass the value as env variables and at "import" time are not set (for example when testing).

I would consider a more "flask" aproach, for example creating an init_app which would read "CLIENT_ID" and "CLIENT_SECRET" from the environment variables. Or even better something like this:
https://docs.authlib.org/en/stable/client/flask.html#configuration

aio_test fail: fixture 'event_loop' not found

Tests for aio are failing.

$ pytest
...
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator0-401-kwargs0]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator0-401-kwargs1]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator0-200-kwargs2]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator1-401-kwargs0]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator1-401-kwargs1]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator1-200-kwargs2]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator2-401-kwargs0]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator2-401-kwargs1]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator2-200-kwargs2]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator3-401-kwargs0]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator3-401-kwargs1]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator3-200-kwargs2]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator4-401-kwargs0]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator4-401-kwargs1]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator4-200-kwargs2]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator5-401-kwargs0]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator5-401-kwargs1]
ERROR flaat/aio/aio_test.py::test_decorator[pyloop-decorator5-200-kwargs2]
ERROR flaat/aio/overrides_test.py::test_env_override_authentication[pyloop]
ERROR flaat/aio/overrides_test.py::test_env_override_authorization[pyloop]
ERROR flaat/aio/overrides_test.py::test_env_override_authorization_without_user[pyloop]

Basically I get this error:

  @pytest.fixture
  def cli(event_loop, aiohttp_client, app):
E       fixture 'event_loop' not found

However, tests are ok when I run tox.
Am I missing something when running the tests?

Tweak cache controls

Right now cache lives too long. Check requests_cache docs.

aarc-g002 entitlement checking raises errors

[INFO] [init.py:decorated:590] Parsing entitlements
[INFO] [init.py:init:99] Input did not match (strict=False): urn:mace:egi.eu:aai.egi.eu:admins:[email protected]
[ERROR] [init.py:decorated:595] Failed to parse entitlement: Input does not seem to be an AARC-G002 Entitlement (Omitting the group authority was permitted)

FastAPI check_request_authorization() got multiple values for argument 'user_infos'

I am unable to create a first example injecting user_infos:

from fastapi import FastAPI, Request
from flaat.fastapi import Flaat

app = FastAPI()
flaat = Flaat()
flaat.set_trusted_OP_list(["https://aai.egi.eu/oidc/"])

@app.get("/info")
@flaat.inject_user_infos()  # Fail if no valid authentication is provided
def info_strict_mode(request: Request, user_infos=None):
    return user_infos.toJSON()

When calling the enpoint using a correct token, I get Internal Server Error due to check_request_authorization() got multiple values for argument 'user_infos'. Maybe I am missing something?

Strict mode off does not work

See the following code example with flask (but probably extensible to all frameworks).

from flaat.flask import Flaat
from flask import Flask

app = Flask(__name__)
flaat = Flaat()

@app.route("/info", methods=["GET"])
@flaat.inject_user_infos(strict=False)  # Doesn't fail if no user
def info(user_infos=None):
    return user_infos.toJSON() if user_infos else "No userinfo"

app.run()

Then if you GET the endpoint info:

$ curl localhost:5000/info
{"error": "Unauthenticated", "error_description": "No authorization header"}

It shows the error was raised and we get Unauthenticated where acording to the docs it should return the defined response ("No userinfo" in this case).

Check against malicious OPs

Since flaat traverses a list of OPs, one hacked OP can cause problems.

Fix:

if multiple OPs answer on the same token: Check again (but be smarter)

Too many threads created for running connections to OPs

After a while, no new threads can be started. The reason is the threads never finish their work due to waiting on a blocking get call on an empty task queue.

Flask tests fail with different oidc-account in .env

My example .env file:

### JWT ACCESS TOKEN
# the shortname depends on how you setup your oidc agent
export OIDC_AGENT_ACCOUNT="helmholtz"

# the issuer of the oidc agent account
export FLAAT_ISS="https://login.helmholtz.de/oauth2"

# These claims must point to two lists of at least two elements in the userinfo
export FLAAT_CLAIM_ENTITLEMENT="eduperson_entitlement"
export FLAAT_CLAIM_GROUP="eduperson_scoped_affiliation"

# To test token introspection we need client id / secret
#export FLAAT_CLIENT_ID="oidc-agent"
#export FLAAT_CLIENT_SECRET="" # oidc agent needs no secret
### END JWT ACCESS TOKEN

### OPTIONAL NON-JWT ACCESS TOKEN
export NON_JWT_OIDC_AGENT_ACCOUNT="google"
export NON_JWT_FLAAT_ISS="https://accounts.google.com"
### END OPTIONAL NON-JWT ACCESS TOKEN


### OPTIONAL AUD ACCESS TOKEN; OP must support setting AT audience claim
export AUD_OIDC_AGENT_ACCOUNT="wlcg"
export AUD_FLAAT_ISS="https://wlcg.cloud.cnaf.infn.it/"
### END OPTIONAL AUD ACCESS TOKEN

The following tests fail:

========================================================= short test summary info ==========================================================
FAILED flaat/flask/flask_test.py::test_authorized[ProductionConfig-ValidToken-/authorized_claim] - assert 401 == 200
FAILED flaat/flask/flask_test.py::test_authorized[ProductionConfig-ValidToken-/authorized_vo] - assert 401 == 200
FAILED flaat/flask/flask_test.py::test_authorized[ProductionConfig-ValidToken-/authorized_level] - assert 401 == 200
FAILED flaat/flask/flask_test.py::test_authorized[ProductionConfig-ValidToken-/authenticated] - assert 401 == 200
FAILED flaat/flask/flask_test.py::test_authorized[ProductionConfig-ValidToken-/authenticated_callback] - assert 401 == 200
FAILED flaat/flask/flask_test.py::test_authorized[ProductionConfig-ValidToken-/info] - assert 401 == 200
================================================= 6 failed, 89 passed, 1 skipped in 23.57s =================================================

Example output from one of the failed tests:

______________________________________ test_authorized[ProductionConfig-ValidToken-/authorized_claim] ______________________________________

client = <FlaskClient <Flask 'examples.example_flask'>>
test_authorized_path_headers = ('/authorized_claim', {'Authorization': 'Bearer eyJ0eXAiOiJhdCtqd3QiL...'})

>   ???

<makefun-gen-12>:2: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.tox/py39/lib/python3.9/site-packages/pytest_cases/fixture_parametrize_plus.py:1072: in wrapped_test_func
    return test_func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

client = <FlaskClient <Flask 'examples.example_flask'>>, path = '/authorized_claim'
headers = {'Authorization': 'Bearer eyJ0eXAiOiJhdCtqd3QiL...'}

    @parametrize_with_cases("path, headers", cases=cases.Authorized)
    def test_authorized(client, path, headers):
        response = client.get(path, headers=headers)
>       assert response.status_code == 200
E       assert 401 == 200
E        +  where 401 = <WrapperTestResponse streamed [401 UNAUTHORIZED]>.status_code

flaat/flask/flask_test.py:8: AssertionError
------------------------------------------------------------ Captured log call -------------------------------------------------------------
DEBUG    flaat:__init__.py:440 Mapping exception: User identity could not be determined

I suspect it has something to do with these lines:

flaat/flaat/flask/conftest.py

Lines 21 to 22 in 7fe6f36

 app = create_app(configuration) 

 app.config["TRUSTED_OP_LIST"] = FLAAT_TRUSTED_OPS_LIST

The TRUSTED_OP_LIST of the app contains the EGI OP (from ProductionConfig) instead of the one from the .env file.

@BorjaEst

Verify JWT Issuer against list of trusted OPs

Unit tests not working

Hi!

When I added the patch to solve our issue we also included some unit tests to issuers_test.pyto test the patch. However, we couldn't run the unit test locally- some packages don't seem to be working anymore.

make available the requests_cache backend configuration

Hi,

Thank you for Flaat!

While using it with a Flask application on Azure AppService, we encountered an Azure bug: Azure/azure-sdk-for-python#6503. This is because Flaat uses requests_cache with a default configuration using SQLite, which is the object of that bug.
Can you provide a Flaat API to set/configure the backend for requests_cache? (I can confirm it works on Azure with memory configured as backend)

Cheers,
Alexandru

PS I can do a Pull Request, if you don't have the time to add this functionality

Importing all web envs, even the unused ones

AttributeError: module 'jwt' has no attribute 'PyJWKClient'

The jwt module has moved the PyJWKClient internally.
This triggers the AttributeError

Issuer recognition fails because the address is not within defined parameters

I have created an issuer that is available in Docker Compose for testing purposes with an address:
https://keycloak:8443/realms/test-realm.

Flaat gives the error: 'Could not verify JWT: No 'iss' claim in body'.

The problem has been traced back to the fact that my issuer has a one-word name (e.g. 'keycloak' instead of 'keycloak.com').

'Flaat' object has no attribute 'set_timeout'

I have the following error:

Exception has occurred: AttributeError
'Authorization' object has no attribute 'set_timeout'

It looks it is renamed to set_request_timeout, however the example it is still outdated:

flaat/examples/example_flask.py

Line 46 in b0d7202

flaat.set_timeout(3)

Run connections to OPs in a threadpooled fashion

To make sure that one unresponsive OP will not block the whole show.

consistent status codes

Is there any documentation on what status codes are expected in different cases? Based on my testing (maybe not comprehensive list):

401:

invalid token
valid jwt but OP not supported

403:

no token provided (either no authorisation headers or no bearer token in authorisation headers)
requirements not satisfied
no sub requirements

In the first item of 403, I wonder if this should be 401?

My understanding of the http codes:

401: Unauthorized is the status code to return when the client provides no credentials or invalid credentials.
403: Forbidden is the status code to return when a client has valid credentials but not enough privileges to perform an action on a resource.

In any case, it would be nice to document this.

Originally posted by @dianagudu in #47 (comment)

Offline Verification of JWT

Needs to be implemented

flaat-userinfo requires liboidcagent

flaat-userinfo requires liboidcagent
from flaat_userinfo.py:

import liboidcagent as agent

Disable authorization when testing

I have an application using Flaat with some resources are controlled by login_required and group_required.

All works perfect, however, it took me some time to undestand how to "Mock", "Patch" or "Pass" the authentication when testing.

After looking into the code, I see there is an ENV read for 'DISABLE_AUTHENTICATION_AND_ASSUME_AUTHENTICATED_USER' which can be used to skip the authentication.

If someone interested, currently I pass the authentication in pytest using:

@fixture(scope='function')
def skip_authorization(monkeypatch):
    """Patch ENV to skip the authorization step."""
    monkeypatch.setenv(
        "DISABLE_AUTHENTICATION_AND_ASSUME_AUTHENTICATED_USER",
        "YES"
    )

However, it might be better to be able to split between different authorization grants, such as:

@fixture(scope='function')
def grant_logged(monkeypatch):
    """Patch fixture to test function as logged user."""
    monkeypatch.setenv("ASSUME_LOGGED", True)

@fixture(scope='function')
def grant_admin(monkeypatch, grant_logged):
    """Patch fixture to test function as admin user."""
    monkeypatch.setenv("ASSUME_GROUPS, ["admin1", "admin2"])

Also it is not on the documentation, but I consider it a really important feature.
Is it a feature inteded for usage?

Also note the idea is not to skip the authorization always, but control it at tearUp and tearDown. I still have some tests which return OK only if the reply was 401 UNAUTHORIZED.

Error getting issuer_config when `self.iss` is set

The function find_issuer_config_in_string sometimes returns a list, sometimes the JSON config directly (dict):

flaat/flaat/issuertools.py

Lines 81 to 97 in cabe271

 def find_issuer_config_in_string(string): 

 '''If the string provided is a URL: try several well known endpoints until the ISS config is 

  found''' 

 iss_config = None 

 if string is not None: 

 if tokentools.is_url(string): 

 iss_config = get_iss_config_from_endpoint(string) 

 if iss_config: 

 return [iss_config] 

 iss_config = get_iss_config_from_endpoint(string+'/oauth2') 

 if iss_config: 

 return [iss_config] 

 iss_config = get_iss_config_from_endpoint(string+'/.well-known/openid-configuration') 

 if iss_config: 

 return [iss_config] 

 iss_config = get_iss_config_from_endpoint(string+'/oauth2'+'/.well-known/openid-configuration') 

 return iss_config

However, when called, it is assumed to be a dict, not a list:

flaat/flaat/__init__.py

Lines 295 to 297 in cabe271

 iss_config = issuertools.find_issuer_config_in_string(self.iss) 

 if iss_config is not None: 

 return [iss_config]

This leads to the following error:

Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/home/diana/workspace/ssh-oidc/venv/lib/python3.9/site-packages/flaat/__init__.py", line 363, in thread_worker_get_userinfo
    result = issuertools.get_user_info(item['access_token'], item['issuer_config'])
  File "/home/diana/workspace/ssh-oidc/venv/lib/python3.9/site-packages/flaat/issuertools.py", line 218, in get_user_info
    logger.info('Getting userinfo from %s' % issuer_config['userinfo_endpoint'])
TypeError: list indices must be integers or slices, not str

Fix: always return iss_config in find_issuer_config_in_string.

Exception during entitlement checking with non-conforming entitlements

Improper exception handling when I have an entitlement that does not conform with the AARC G069 guideline.

My userinfo contains the following entitlements:

    "eduperson_entitlement": [
        "urn:geant:helmholtz.de:group:KIT#login-dev.helmholtz.de",
        ...
        "urn:mace:dir:entitlement:common-lib-terms"
    ]

The "urn:mace:dir:entitlement:common-lib-terms" entitlement does not contain any group authority, so parsing it using the aarc_entitlement lib fails, as expected:

import aarc_entitlement
norm_ent = aarc_entitlement.G069("urn:mace:dir:entitlement:common-lib-terms")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "python/3.10.5/lib/python3.10/site-packages/aarc_entitlement/__init__.py", line 417, in __init__
    super().__init__(entitlement)
  File "python/3.10.5/lib/python3.10/site-packages/aarc_entitlement/__init__.py", line 204, in __init__
    self._parse(self._preprocess(entitlement))
  File "python/3.10.5/lib/python3.10/site-packages/aarc_entitlement/__init__.py", line 141, in _parse
    raise ParseError(
aarc_entitlement.ParseError: Entitlement does not conform to specification (need_group_authority=False): urn:mace:dir:entitlement:common-lib-terms

Flaat does catch this exception:

flaat/flaat/requirements.py

Lines 315 to 317 in 44b4c7c

 except aarc_entitlement.Error as e: 

 logger.debug("Error parsing aarc entitlement: %s", e) 

 return None

but raises another exception due to not checking for a None parse result:

...
  File "/usr/lib/motley-cue/lib/python3.9/site-packages/flaat/__init__.py", line 526, in async_wrapper
    ((args, kwargs), error_response) = self._run_work_flow_safe(*args, **kwargs)
  File "/usr/lib/motley-cue/lib/python3.9/site-packages/flaat/__init__.py", line 507, in _run_work_flow_safe
    return (self._run_work_flow(*args, **kwargs), None)
  File "/usr/lib/motley-cue/lib/python3.9/site-packages/flaat/__init__.py", line 495, in _run_work_flow
    self.check_user_authorization(user_infos)
  File "/usr/lib/motley-cue/lib/python3.9/site-packages/flaat/__init__.py", line 406, in check_user_authorization
    check_result = req.is_satisfied_by(user_infos)
  File "/usr/lib/motley-cue/lib/python3.9/site-packages/motley_cue/mapper/authorisation.py", line 54, in is_satisfied_by
    return op_authz.get_user_requirement().is_satisfied_by(user_infos)
  File "/usr/lib/motley-cue/lib/python3.9/site-packages/flaat/requirements.py", line 135, in is_satisfied_by
    check_result = req.is_satisfied_by(user_infos)
  File "/usr/lib/motley-cue/lib/python3.9/site-packages/flaat/requirements.py", line 163, in is_satisfied_by
    check_result = req.is_satisfied_by(user_infos)
  File "/usr/lib/motley-cue/lib/python3.9/site-packages/flaat/requirements.py", line 135, in is_satisfied_by
    check_result = req.is_satisfied_by(user_infos)
  File "/usr/lib/motley-cue/lib/python3.9/site-packages/flaat/requirements.py", line 249, in is_satisfied_by
    if self.matches(self.value, self.parse(val)):
  File "/usr/lib/motley-cue/lib/python3.9/site-packages/flaat/requirements.py", line 284, in matches
    return self._matches(required, available)
  File "/usr/lib/motley-cue/lib/python3.9/site-packages/flaat/requirements.py", line 322, in _matches
    return available.satisfies(required)
AttributeError: 'NoneType' object has no attribute 'satisfies'

For reference, I use get_vo_requirement with the following list of required VOs:

[
    "urn:geant:h-df.de:group:test-vo#login-dev.helmholtz.de",
    "urn:geant:helmholtz.de:group:Helmholtz-member#login-dev.helmholtz.de",
]

Investigate authorisation if user is in no group

Either feudal or flaat fail; investigate

trusted issuers

My comment is related to flaat-userinfo.

When is use flaat-userinfo with an AT from an issuer not known to flaat, I get an Error: Issuer not trusted:

I cannot find an option to work around this, there seems to be --my-config option, but I cannot find information how that config file should look like.

I just want to display the information about the token. I don't understand why I get the error at all when using a JWT. The issuer is inside the token and is correctly obtained by flaat, but it refuses to do something.

There should be at least an command line option to skip/trust an issuer.

Outdated examples

It looks some objects used in the examples are not present in the library.
Probably those were renamed?

>>> from examples import example_flask
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/borja/Projects/flaat/examples/example_flask.py", line 6, in <module>
    from flaat.requirements import HasAARCEntitlement, HasGroup, ValidLogin
ImportError: cannot import name 'HasGroup' from 'flaat.requirements' (/home/borja/Projects/flaat/flaat/requirements.py)

Context:

$ pip show flaat
Name: flaat
Version: 1.0.1
Summary: User authorization for OIDC authenticated python web APIs.
Home-page: https://github.com/indigo-dc/flaat
Author: Marcus Hardt
Author-email: [email protected]
License: MIT
Location: /home/borja/miniconda3/envs/flaat/lib/python3.8/site-packages
Requires: liboidcagent, aarc-entitlement, cachetools, configargparse, pyjwt, humanfriendly, requests
Required-by:

Testing needs Environment example

The tests are nice. Just: I fail to give a proper environment e.g. for the CLAIM_GROUP.

Tried variants of this, without success:

$ export FLAAT_ISS=https://aai.egi.eu/oidc/; export OIDC_AGENT_ACCOUNT=egi; export CLAIM_GROUP='"{["marcus","user"]}"'
$ pytest flaat/aio/aio_test.py -v

Flask runtime error for access_levels

After extending example with access_levels, I got "runtime error" when importing configuration from env variable.

	app = create_app(configuration)
	app.config["TRUSTED_OP_LIST"] = FLAAT_TRUSTED_OPS_LIST

	def find_issuer_config_in_string(string):
	'''If the string provided is a URL: try several well known endpoints until the ISS config is
	found'''
	iss_config = None
	if string is not None:
	if tokentools.is_url(string):
	iss_config = get_iss_config_from_endpoint(string)
	if iss_config:
	return [iss_config]
	iss_config = get_iss_config_from_endpoint(string+'/oauth2')
	if iss_config:
	return [iss_config]
	iss_config = get_iss_config_from_endpoint(string+'/.well-known/openid-configuration')
	if iss_config:
	return [iss_config]
	iss_config = get_iss_config_from_endpoint(string+'/oauth2'+'/.well-known/openid-configuration')
	return iss_config

	iss_config = issuertools.find_issuer_config_in_string(self.iss)
	if iss_config is not None:
	return [iss_config]

	except aarc_entitlement.Error as e:
	logger.debug("Error parsing aarc entitlement: %s", e)
	return None