Comments (1)
The current ideal state in my mind
Signature does 3 things
- Length check
- Type check
- Coercion
For length,
the common length check are
- Exact number
- Variadic (Any number)
- VariadicNonZero (Any number but at least one)
- VariadicEven (Less common, i.e. Map)
For types,
We have two style, Exact and Coercion. Exact rejects if the type mismatch, Coercion rejects if teh type is not coercible to the target type.
The combination of these are
Uniform(Vec<DataType>)
// Exact Number with Exact typeUniformCoercion(Vec<DataType>)
// Exact Number with coercionVariadic(DataType)
VariadicCoercion(DataType)
...
For non-uniform length and more then on data type signature, we could use UserDefined
.
The more tricky part is the DataType. We have many functions expect Numeric type that includes integer, float, ....
For function that expects string, there are Utf8, LargeUtf8, Utf8View.
For type checking, it would be nice to have more general Enum that includes more than one DataType to check against with.
enum ArgumentType {
Numeric
Integer
Float
List
...
}
Now, we have
Uniform(Vec<FunctionType>)
// Exact Number with Exact typeUniformCoercion(Vec<FunctionType>)
// Exact Number with coercionVariadic(FunctionType)
VariadicCoercion(FunctionType)
TypeSignature::Numeric is one of the idea that comes out from it.
For other kinds of complex type check or length check, we fall back to UserDefined
from arrow-datafusion.
Related Issues (20)
- Use GenericStringBuilder to improve performance of UDF split_part
- Document "how to read an explain plan" HOT 11
- Thread panics in SpawnedTask during shutdown. HOT 6
- SMJ full join produces a null join row for a buffered row even the buffered row is joined with streamed row and passes join filter
- Implement GroupsAccumulator for stddev and var aggregaters
- Cannot infer common string type for string concat operation Dictionary(Int32, Utf8) || Dictionary(Int32, Utf8) HOT 2
- Bug detecting datatype in VALUES tuples HOT 1
- Handle downstream impacts to union's behavioral changes. HOT 6
- Aggregation fuzz testing HOT 4
- Systematic fuzz testing for parquet predicate pushdown HOT 1
- Support protobuf serialization for `ScalarValue::Utf8View` and `ScalarValue::BinaryView` HOT 1
- Support substrait serialization for `ScalarValue::Utf8View` and `ScalarValue::BinaryView` HOT 3
- Add config flag to convert `Utf8View`/`BinaryView` --> `Utf8` / `Binary` at output HOT 4
- Add the possibility to work with plan clones directly in `inline_table_scan` HOT 3
- Parquet statistics missing when reading `Utf8` as `Utf8View`
- Panic in `substring()` scalar function (SQLancer) HOT 1
- Fuse operations in `equal_rows_arr`
- External sorting not working for (maybe only for string columns??)
- Builder style API for creating `RuntimeEnv` HOT 3
- Unchecked overflow in integer number addition HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from arrow-datafusion.