Comments (6)
Thank you for creating this ticket to facilitate the discussion.
For the general audience: The good news is that we found a way to support the old syntax simultaneously with the new options syntax -- the bad news is that it results in code complexity/maintenance burden.
Basically, the aim of this ticket is to gather information on usage so that we can decide whether it is worth to have both syntaxes supported for a while -- or should we just pull the band-aid and remove the old syntax.
from datafusion.
I agree that it makes sense to "pull the band-aid" and remove the syntax from this code.
One place where I think this might get difficult is if an project that relies on DataFusion has some sort of user/external input that relies on this syntax, that the project itself doesn't have the ability to easily change. But the nice thing about DataFusion is its extensible, so I could imagine if someone has a strong requirement to keep the old syntax, we could write a custom parser that wraps DFParser
(maybe lives in another repo, or in examples) that adds it back and just sets OPTIONS('format.has_header' 'true')
. (See an example of a custom parser at https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/sql_dialect.rs#L45)
from datafusion.
IIRC, some of this original syntax came from a desire to suppor Hive SQL, but as @phillipleblanc said, if anyone needs this then they can add it back under a specific dialect.
from datafusion.
But the nice thing about DataFusion is its extensible, so I could imagine if someone has a strong requirement to keep the old syntax, we could write a custom parser that wraps DFParser (maybe lives in another repo, or in examples) that adds it back and just sets OPTIONS('format.has_header' 'true'). (See an example of a custom parser at https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/sql_dialect.rs#L45)
Agreed, sounds like a reasonable way forward in case removing the old syntax creates challenges for certain users that can't change their SQL.
from datafusion.
If there is consensus here, given we just branched for the 38 release, maybe we can remove support on main now as part of #10404 and let that sit on main for a while 🤔
from datafusion.
Sounds good. We will update the PR tomorrow 👍
from datafusion.
Related Issues (20)
- Add an example of how to use the SQL parser/unparser API HOT 2
- Support Substrait VirtualTables
- Discussion: make it easier for specify SQL --> function translation HOT 5
- Create a DataFusion blog HOT 3
- `analysis.rs` bounds check panic HOT 5
- AggregateUDF expression API design HOT 2
- Example for building an external index for parquet files
- `array_slice` can't correctly handle NULL parameters or some edge cases
- Add an example of how to convert LogicalPlan to/from SQL Strings
- Cast from string to date with "/" HOT 2
- error: this arithmetic operation will overflow (on i386)
- REmove workaround for `COUNT(*)` in subquery decorrelation code
- Make SQL strings generated from `Expr`s "prettier" HOT 11
- Dynamic schema for custom TableProvider HOT 4
- ScalarValue serialization does not handle nested dictionary values
- Using `Expr::field` panics HOT 3
- Improve signature of `get_field` is function
- UserDefinedLogicalNode::from_template does not return a Result<...> >
- UserDefindedLogicalNode::from_template does not return a Result<...>. HOT 3
- Row groups are read out of order or with completely different values HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datafusion.