Comments (3)
More info here. Looks like kerberos is correctly setup in the dagster run container.
I added this code to dagster, a URL that needs windows credentials, and works correct.
`try:
# Construct the curl command
cmd = ["curl", "--negotiate", "-u", ":", "-O", url]
# Open a subprocess to execute the curl command
curl_process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
curl_output, curl_error = curl_process.communicate()
# Check for success or failure based on the return code
if curl_process.returncode == 0:
context.log.info("Curl request successful")
else:
context.log.info(f"Curl request failed. Error: {curl_error.decode()}")
except Exception as e:
context.log.info(f"An error occurred: {e}")`
Still, i have no idea what is happening with pyodbc / dagster together.
can i add more verbosity the whole process?
from dagster.
Hi, i tried to isolate the problem, and the issue is on dagster, not kerberos or pyodbc or sql provider.
I deploy a code location with Dagster Helm. i have a an asset called "Test" with a 10 minutes timeout, so i can launch the pod and connect to the pod.
Connected to the "dagster-run" pod, i check that kerberos is correct.
Then i run:
dagster asset materialize --select test-from-cmd -f assets.py
and the connection to SQL Server via Windows auth is correct!!!.
Im in a point that i dont understand whey this does work if i run the same assets from the web UI?
from dagster.
So, happily i will close the ticket.
The reason why is not working on my current setup (dagster 1.6.1, python version, kerberos libraries, OS libraries, etc) it can be multiple. But due to the nature of dagster running jobs on multiprocess.spawn, this can cause many problems in the way kerberos works and how tickets can be used between multiple threads. That can explain why pyodbc/sql driver/kerberos always worked when submitting dagster jobs from command line.
I run tests on debian and alpine python3.11 and it didnt work.
However, with redhat/ubi9:python3.11 all works perfectly.
i close the issue.
from dagster.
Related Issues (20)
- New API authentication option for Airbyte OSS HOT 4
- Attempting to show unified asset health for assets with dimension of different lengths HOT 1
- `AutomationCondition` support for `dbt_assets` HOT 1
- Logs generated during evaluation of sensor not shown in UI if they originate from within a configurable resource
- DuckDB connection_config parameter not being recognized HOT 2
- retry_on_asset_or_op_failure = false is not working HOT 2
- AssetsDefinitions underlying ops does not support "detached" outs 1.8+
- `MarkdownMetadataValue` does not properly display new lines `<br />`
- Enable max run time tracking at Op level
- AzureBlobComputeLogManager fails on missing secret_key HOT 2
- Job execution not started with `Iteration is not allowed on @records` error on Dagster 1.8 HOT 10
- Please add pandera schema to list of allowed metadata values for v2
- [dagster-polars] Upstream partitionned Polars assets cannot be optional HOT 1
- [dagster-pandera / dagster-polars] Asset with Pandera Polars schema fails to be materialized with PolarsIOManager HOT 1
- Manual backfill of partitioned dbt job spawns infinite amount of runs
- Able to report partial materialisation of graph_asset
- clicking external asset causes 'Asset must be part of at least one job'
- Add timezone argument to AutomationCondition.on_cron()
- Dagster daemon logging does not recover from error situation
- Allow passing shell=True to Popen from PipesSubprocessClient
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dagster.