Comments (10)
cmarshak
commented on August 27, 2024
1
I am using rosetta on my M1 mac for this 3.9. Conda-lock is not playing nice in this environment. For the time being - I am going to close this issue as installing dask fixed this and this environment is in 3.9 to support this monolith code-base(oy)!
Pandera is fantastic and thank you for your attention/help!
from pandera.
cmarshak
commented on August 27, 2024
The issue goes away after I install dask.
from pandera.
cosmicBboy
commented on August 27, 2024
Can you provide repro instructions? A fresh install of pandera (without the dask extra) doesn't repro this issue
from pandera.
cmarshak
commented on August 27, 2024
I am using mambaforge. I saw this cropping up in our integration testing via github actions.
The reproduction instructions would be:
mamba env update -f environment.yml # https://github.com/ACCESS-Cloud-Based-InSAR/DockerizedTopsApp/blob/42038c0bce2b03fab531a9f5de10400d1766432c/environment.yml
conda activate topsapp_env
python -c "import pandera"
I reproduced the error on my M2 mac via rosetta.
It's a bigger environment and pandera is being installed via the tile-mate recipe.
from pandera.
cosmicBboy
commented on August 27, 2024
is it possible to share the environment.yml file?
from pandera.
cosmicBboy
commented on August 27, 2024
Also would you mind editing to title to be more descriptive? @cmarshak
from pandera.
cosmicBboy
commented on August 27, 2024
File ~/miniforge3/envs/topsapp_env/lib/python3.9/site-packages/pandera/typing/dask.py:9
6 from pandera.typing.pandas import DataFrameModel, GenericDtype
8 try:
----> 9 import dask.dataframe as dd
11 DASK_INSTALLED = True
12 except ImportError:
File ~/miniforge3/envs/topsapp_env/lib/python3.9/site-packages/dask/dataframe/__init__.py:87
84 except ImportError:
85 pass
---> 87 if _dask_expr_enabled():
88 import dask_expr as dd
90 # trigger loading of dask-expr which will in-turn import dask.dataframe and run remainder
91 # of this module's init updating attributes to be dask-expr
92 # note: needs reload, incase dask-expr imported before dask.dataframe; works fine otherwise
doesn't this part of the stack trace imply that dask is installed?
File ~/miniforge3/envs/topsapp_env/lib/python3.9/site-packages/dask/dataframe/__init__.py:87
Which doesn't line up with this statement:
The issue goes away after I install dask.
from pandera.
cmarshak
commented on August 27, 2024
@cosmicBboy - hope this is not taking too much of your time. I can investigate further next week. Thank you for your help thus far.
I put in the comment of the code block here it is spelled out:
name: topsapp_env
channels:
- conda-forge
dependencies:
- python>=3.9,<3.10
- pip
- affine
- asf_search>=5.0.0
- boto3
- dateparser
- flake8
- flake8-blind-except
- flake8-builtins
- flake8-import-order
- fsspec
- gdal
- geopandas
- hyp3lib>=3,<4
- ipykernel
- isce2==2.6.1
- jinja2
- joblib
- jsonschema==3.2.0
- jupyter
- lxml
- matplotlib
- netcdf4
- notebook
- numpy<1.24
- pandas
- pysolid
- papermill
- pytest
- pydantic
- pytest-cov
- pytest-mock
- rasterio
- rioxarray<0.14.0
- xarray
- scipy<1.10
- setuptools
- setuptools_scm
- shapely
- tqdm
- dem_stitcher>=2.5.5
- aiohttp # only needed for manifest and swath download
- tile_mate>=0.0.8
tile_mate is what requires pandera.
I am not as adept at navigating conda-forge as I would like. It's unclear why the program thinks dask is installed when it is clearly not. That's what I am seeing too.
from pandera.
cosmicBboy
commented on August 27, 2024
You can try creating a conda-lock file from this to see which dependency is causingdask (and its seems dask_expr) is being installed.
from pandera.
cosmicBboy
commented on August 27, 2024
friendly ping @cmarshak. with conda-lock we can figure out which of the dependencies require dask
from pandera.
Related Issues (20)
- `DataFrameModel.get_metadata` should not be typed `Optional` HOT 1
- Wrong Column name in SchemaErrors using regex Columns
- Allowing column with type list to be optional HOT 4
- Pass additional `Check` kwargs into `register_check_method`
- Bad type hint for argument unique for DataFrameSchema
- Support Pyarrow data synthesis strategies for Pandas validation engine
- `pandera.api.dataframe.model.DataFrameModel` has unexpected generic subclass behaviour
- Subclassing from `pandera.api.dataframe.model.DataFrameModel` errors on annotated but not initialized fields starting with an underscore
- Add support for polars.Enum
- Polars Enum does not work for Polars verion >=1.0.0 HOT 1
- [Bug?] Ordered: True results in alphabetically ordering
- How to properly validate a `polars.LazyFrame`? HOT 2
- Installation issues when using polars-lts-cpu HOT 6
- Dagster polars dataframe_check HOT 2
- Failed Index Uniqueness Validation for Dask Series HOT 1
- pandera.errors.SchemaErrors.failure_cases returns incorrect columns when used with regex column names HOT 1
- Polars DataFrameModel `to_json_schema()` fails on pl.Datetime column
- TypeError when calling _type_check on parameterized GenericModel in pydantic v1
- Time series integrity
- Improve `pd.Series` error message to include index / row number
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pandera.