pnathan / cl-linq Goto Github PK
View Code? Open in Web Editor NEWlinq in lisp; data set manipulating done easy
linq in lisp; data set manipulating done easy
Can recursive joins be implemented without ctes?
postmodern and cl-mssql both have the option to return alists.
grouping by alist key would be kiiind of nice.
Problem Number One: Columns are unlabelled.
As our dear forefathers figured out long ago, columns of data need labelling. However, if a 2D array or list of lists is passed in; there are no labels native to the data format. It is plausible that a "header row" ala CSV can be accepted and used. Reviewing cl-linq-tests.lisp, it's completely clear that indexes into data alone will not suffice.
Problem Number Two: Database systems emit alists.
This is a good thing - it's easy to debug - and a bad thing - cl-linq doesn't understand them.
Problem Number Three: This code is slow. Postgres can run multiple times quicker than this code on data sets under 1M rows. This may be a generally unsolvable problem when taking & giving out 2D lists/vectors. Most optimizations in comp sci demand preconditions on data (sorting, etc).
Solution Number One: Data Frame object.
This takes a page from R and would require that a query take either a 2D thingie (as now) or a data frame object. A dataframe object has not only data, but labelled columns and the capacity to add indexes to columns. An interesting exploration would be to allow row/column swaps to handle OLAP-style queries quickly.
Solution Number Two: Context through QUERY. Instead of using Query as a simple sql-esque starting point, add several options to it that would significantly enhance usefulness. E.g., &key Sort-before-use
; &key list-of-columns
.
Solution Number Three: Stop attempting to mimic SQL.
Instead of using SQLish approaches, decompose the system into components:
one function to GROUP-BY, one function to WHERE, one function to SELECT-COLUMNS, etc. This would diminish the opportunity for interprocedural optimization, but would provide a flexibility of end-use.
Overall Solution Requirements : Algorithmic revisitation/restrictions.
Partial issue: lists are real handy, but vectors are faster (yet have lousy sets of functions supporting them). Should generic infrastructure be built up to allow generic ad-hoc dispatch? this will slow things down.
Example:
(defmethod head ((object vector)) (aref object 0))
(defmethod head ((object list)) (car object))
this is silly. should be easy to fix in code.
desperately needs a having clause.
A query should be "order-by"-able.
Add ASDF integration & packaging.
Add implicit parallelization if lparallel is around.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.