Comments (4)
Hi @glodus,
s3 can be locked pretty tightly so that it is only readable with an access_key and a secret_key and not to the rest of the world (You would have to define the appropriate security policy for the resource (bucket, or subdirectory)). This is the setup we use. Airflow is also equipped to handle IAM roles, if your setup permits.
Let me know if you have more questions about this!
from airflow.
I was more thinking about what I would put on the airflow server, about the "worklow", "pipeline" or dags. It could be very unfortunate that someone understood it.
Not really related, but still a question: Do you handle LDAP authentification or perhaps flask_login already handle it ?
from airflow.
From our perspective security is almost an all-or-nothing as far as executing pipelines. If you do have access to an Airflow box, you pretty much have full access to all the systems Airflow interfaces with. We spoke internally about using Airflow for more sensitive datasets that not all engineers should have access to, and the solution there would be to setup an alternate Airflow environment for that purpose.
The UI is mostly a read-only UI, and there are different level of access there. It's possible to create a limited access where people can just look at pipeline definitions and how they progress.
The flask_login backdoor should allow you to do pretty much anything in terms of authentication. Our setup has a reverse proxy that takes care of ssl and ldap authentication that squeezes http headers in the request, and our airflow_login
module just look for the headers to grant the right level of access. Using flask_login and ldap is probably fairly well documented. A temporary setup can be achieved with privileged users using an ssh tunnel.
from airflow.
Thanks.
from airflow.
Related Issues (20)
- Params Fail to Evaluate In Dag Body HOT 6
- The Contributor's Quick Start isn't quick enough and needs some TLC HOT 5
- Airflow 2.8.1: awscli installation hindered by version conflicts in requirements.txt HOT 2
- Bad rendering of an inline code in the documentation HOT 1
- Add median duration markline to dag run duration page
- DAG list: filter FAILED/RUNNING HOT 3
- Output tasks doc in the UI HOT 1
- GenericTransfer - Warning: "Placeholder defined in Connection..." HOT 4
- Dynamically Mapped Tasks: DB performance issues HOT 10
- KPO - get_logs - logs missing HOT 4
- EksPodOperator error with deferrable=True HOT 1
- Difference between logs which stores in files and logs in UI HOT 5
- KubernetesPodOperator with multiple containers hangs if container other than base container is still running HOT 3
- Unable to open AirFlow webserver UI HOT 1
- data to long when using s3 to gcs HOT 1
- Audit task failed-deps for later investigation HOT 2
- Executor reports task instance (...) finished (failed) although the task says it's queued HOT 4
- Deployment: Airflow 2.9.1 Installed on Kubernetes via Helm Chart - Opensearch Connection Issue (Error 406) HOT 2
- `application/octet-stream` content-type when writing to Azure Blob Storage using `ObjectStoragePath` HOT 1
- GCStoS3 Operator HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from airflow.