Connection Support about aws-mwaa-local-runner HOT 6 OPEN

I just get errors in reference to the constraints file tellng me they're already installed...when mssql, pymssql, and s3 clearly are not.

Could you attach the logs for this as well please? I don't believe they're installed by default.

Regarding the conflict error, apache-airflow-providers-microsoft-mssql is constrained to version 3.4.2 in the constraints. Do you still have issues if you specify apache-airflow-providers-microsoft-mssql==3.4.2 in the requirements?

from aws-mwaa-local-runner.

Hi Mayushko26,

Thanks for replying; I have tried specifying the exact version, as well as just leaving the version off altogether. The only reason I had that version listed was because it was the version used in the guide, I knew it was old and just wanted to see if it would work.

Let me fire it up and I'll get the output while specifying 3.4.2. Stand by.

from aws-mwaa-local-runner.

Here is the output when I specify all the same versions listed in the constraints file.

aws-mwaa-local-runner-2_7-local-runner-1 | Installing requirements.txt
aws-mwaa-local-runner-2_7-local-runner-1 | Collecting apache-airflow-providers-snowflake==5.0.1
aws-mwaa-local-runner-2_7-local-runner-1 | Downloading apache_airflow_providers_snowflake-5.0.1-py3-none-any.whl (37 kB)
aws-mwaa-local-runner-2_7-local-runner-1 | Collecting apache-airflow-providers-mysql==5.3.1
aws-mwaa-local-runner-2_7-local-runner-1 | Downloading apache_airflow_providers_mysql-5.3.1-py3-none-any.whl (25 kB)
aws-mwaa-local-runner-2_7-local-runner-1 | Collecting apache-airflow-providers-microsoft-mssql==3.4.2
aws-mwaa-local-runner-2_7-local-runner-1 | Downloading apache_airflow_providers_microsoft_mssql-3.4.2-py3-none-any.whl (14 kB)
aws-mwaa-local-runner-2_7-local-runner-1 | Collecting apache-airflow-providers-odbc==4.0.0
aws-mwaa-local-runner-2_7-local-runner-1 | Downloading apache_airflow_providers_odbc-4.0.0-py3-none-any.whl (15 kB)
aws-mwaa-local-runner-2_7-local-runner-1 | ERROR: Cannot install pymssql==2.2.1 because these package versions have conflicting dependencies.
aws-mwaa-local-runner-2_7-local-runner-1 |
aws-mwaa-local-runner-2_7-local-runner-1 | The conflict is caused by:
aws-mwaa-local-runner-2_7-local-runner-1 | The user requested pymssql==2.2.1
aws-mwaa-local-runner-2_7-local-runner-1 | The user requested (constraint) pymssql==2.2.8
aws-mwaa-local-runner-2_7-local-runner-1 |
aws-mwaa-local-runner-2_7-local-runner-1 | To fix this you could try to:
aws-mwaa-local-runner-2_7-local-runner-1 | 1. loosen the range of package versions you've specified
aws-mwaa-local-runner-2_7-local-runner-1 | 2. remove package versions to allow pip attempt to solve the dependency conflict
aws-mwaa-local-runner-2_7-local-runner-1 |
aws-mwaa-local-runner-2_7-local-runner-1 |
aws-mwaa-local-runner-2_7-local-runner-1 | ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
aws-mwaa-local-runner-2_7-local-runner-1 |
aws-mwaa-local-runner-2_7-local-runner-1 | [notice] A new release of pip available: 22.3.1 -> 23.3.2
aws-mwaa-local-runner-2_7-local-runner-1 | [notice] To update, run: pip install --upgrade pip

from aws-mwaa-local-runner.

Thanks for following up. This doesn't look like an issue on the local runner but rather an issue of finding of the correct set of requirements and constraints for your needs. It would help if you also added the requirements.txt file you're using, and I could help unblock you.

from aws-mwaa-local-runner.

Sure, it's the bone stock requirements file that came with the docker image, I've added the few things you see, but that's it.

Here's the contents:

--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.7.2/constraints-3.11.txt"

apache-airflow-providers-snowflake==5.0.1
apache-airflow-providers-mysql==5.3.1

apache-airflow-providers-microsoft-mssql==3.4.2
apache-airflow-providers-odbc==4.0.0
pymssql==2.2.8

from aws-mwaa-local-runner.

@josborne-book4time try looking through the issues regarding platform mismatches.

from aws-mwaa-local-runner.