Comments (11)
I need this for my use case of deequ and currently working on a pr for this. I should have it ready some time next week
from deequ.
Has there been any movement on this at all? Curious for a project I am currently working on.
from deequ.
Not yet unfortunately, but it should not take too much effort to implement this, would you like to give it a try?
from deequ.
@sscdotopen I would like to give it a try, I think you are right wouldn't take too much effort. Did you have any suggestion on how you would like this implemented. I am going to create a fork today and try and get started.
from deequ.
It actually looks like @paulsukow has completed almost all of the work for this enhancement but never submitted a PR.
from deequ.
I got side tracked and never got back to it.
This branch has what I started on it. I think it is almost done and just needs some tests or something
https://github.com/paulsukow/deequ/tree/enhancement/minMaxLengthAnalyzer
from deequ.
I can look into where I left off this week or @patchrick843 could take a look if he needs it sooner
from deequ.
I was able to find a work around for my current situation using
.hasPattern("last_name","""^[a-zA-Z0-9_].{0,15}$""".r)
I would be interested in helping finish up what @paulsukow has started. Let me know when you have taken a look and have an idea of what needs to be finished up. Thanks again.
from deequ.
@patchrick843 @paulsukow can you issue a PR so that we can have a look at the current state of the contribution?
from deequ.
work in progress pr: #122
from deequ.
PR merged, closing this issue.
from deequ.
Related Issues (20)
- Incremental profiling to be merged with older result
- Adding the custom constraints HOT 1
- [FEATURE] Extract failing reason when filtering records based on row-level checks
- java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.expressions.aggregate.AggregateFunction.toAggregateExpression(Z)Lorg/apache/spark/sql/catalyst/expressions/aggregate/AggregateExpression;
- Support for Snowflake Connector's query pushdown HOT 1
- Is this library can be used with other Technolgy rather than Spark, such as Flink for example? HOT 2
- [BUG] Unable to serialize Histogram with binningUdf when using them with useRepository
- Incorporate referential integrity and data synchronization checks into Deequ's VerificationSuite HOT 5
- [FEATURE] Add spark table metric repository HOT 4
- Getting Error name 'isComplete' is not defined while running deequ code in Azure Databricks HOT 4
- checks that 95% of entire table satisfy multiple conditions over different columns HOT 1
- [FEATURE] Add support for Spark 3.5 HOT 1
- [BUG] Row based output incorrect when using satisfies check and assertion with upper bound < 1 HOT 3
- [FEATURE] Exposing Anomaly Strategy Calculation Thresholds for Users
- Is Redshift supported as a data source?
- Compliance calculation result HOT 1
- numerical statistical indicators have lost precision
- [FEATURE] Supporing Aggregation metrics for a group
- [FEATURE] Filter condition is ignored when filtering records based on row-level checks HOT 5
- Anomaly checks when fails
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deequ.