Git Product home page Git Product logo

datafly's Introduction

datafly

Build Status

Datafly is a lightweight database library for Common Lisp.

Quickstart

Datafly provides 3 functions which wrap CL-DBIretrieve-one, retrieve-all and execute.

They take a SxQL statement.

(use-package :sxql)

(connect-toplevel :sqlite3 :database-name #P"myapp.db")

(retrieve-one
  (select :*
    (from :user)
    (where (:= :name "nitro_idiot"))))
;=> (:ID 1 :NAME "nitro_idiot" :EMAIL "[email protected]" :REGISTERED-AT "2014-04-14T19:20:13")

(retrieve-all
  (select :*
    (from :user)))
;=> ((:ID 1 :NAME "nitro_idiot" :EMAIL "[email protected]" :REGISTERED-AT "2014-04-14T19:20:13")
;    (:ID 2 :NAME "m2ym" :EMAIL "[email protected]" :REGISTERED-AT "2014-04-15T01:03:42"))

(execute
 (insert-into :tweet
   (set= :id 1
         :user_id 1
         :body "Hi."
         :created_at (princ-to-string *now*))))

If you specify :as option with a class name to retrieval functions, they create instances of the class for each rows.

(defstruct user
  id
  name
  email
  registered-at)

(retrieve-one
  (select :*
    (from :user)
    (where (:= :name "nitro_idiot")))
  :as 'user)
;=> #S(USER :ID 1 :NAME "nitro_idiot" :EMAIL "[email protected]" :REGISTERED-AT "2014-04-14T19:20:13")

(retrieve-all
  (select :*
    (from :user))
  :as 'user)
;=> (#S(USER :ID 1 :NAME "nitro_idiot" :EMAIL "[email protected]" :REGISTERED-AT "2014-04-14T19:20:13")
;    #S(USER :ID 2 :NAME "m2ym" :EMAIL "[email protected]" :REGISTERED-AT "2014-04-15T01:03:42"))

The structure doesn't require having slots same as an existing table's in a database. It is acceptable even if the names are different. This might be convenient when you'd like to treat a JOINed table result as a structure object.

Model Definitions

Datafly provides a macro defmodel which defines a flavored structure class.

(defmodel (user (:inflate registered-at #'datetime-to-timestamp))
  id
  name
  email
  registered-at)


;; Same as the above.
(syntax:use-syntax :annot)

@model
(defstruct (user (:inflate registered-at #'datetime-to-timestamp))
  id
  name
  email
  registered-at)

(:inflate <columns> <inflation-function>) options mean inflation-function will be applied to each <columns> when creating an instance.

(defvar *user*
  (retrieve-one
    (select :*
      (from :user)
      (where (:= :name "nitro_idiot")))
    :as 'user))

;; Returns a local-time:timestamp.
(user-registered-at *user*)
;=> @2014-04-15T04:20:13.000000+09:00

defmodel also allows :has-a and :has-many options.

(use-package :sxql)

(defmodel (user (:inflate registered-at #'datetime-to-timestamp)
                (:has-a config (where (:= :user_id id)))
                (:has-many (tweets tweet)
                 (select :*
                   (from :tweet)
                   (where (:= :user_id id))
                   (order-by (:desc :created_at)))))
  id
  name
  email
  registered-at)

(defvar *user*
  (retrieve-one
    (select :*
      (from :user)
      (where (:= :name "nitro_idiot")))
    :as 'user))

(user-config *user*)
;=> #S(CONFIG :ID 4 :USER-ID 1 :TIMEZONE "JST" :COUNTRY "jp" :LANGUAGE "ja")

(user-tweets *user*)
;=> (#S(TWEET :ID 2 :USER-ID 1 :BODY "Is it working?" :CREATED-AT @2014-04-16T11:02:31.000000+09:00)
;    #S(TWEET :ID 1 :USER-ID 1 :BODY "Hi." :CREATED-AT @2014-04-15T18:58:20.000000+09:00))

Provided inflation functions

  • tinyint-to-boolean
  • datetime-to-timestamp
  • unixtime-to-timestamp
  • string-to-keyword
  • octet-vector-to-string

Tips: Getting Association List or Hash Table for each rows

retrieve-one and retrieve-all return row(s) as a property list or a list of property lists by default.

If you'd like they were other types, for example "Association List" or "Hash Table", you can do it by passing :as parameter.

(retrieve-one
  (select :*
    (from :user)
    (where (:= :name "nitro_idiot")))
  :as 'trivial-types:association-list)
;=> ((:ID . 1) (:NAME . "nitro_idiot") (:EMAIL . "[email protected]") (:REGISTERED-AT . "2014-04-14T19:20:13"))

(retrieve-one
  (select :*
    (from :user)
    (where (:= :name "nitro_idiot")))
  :as 'hash-table)
;=> #<HASH-TABLE :TEST EQL :COUNT 4 {1007AE3CD3}>

If no :as parameter is specified, *default-row-type* will be used.

(let ((*default-row-type* 'hash-table))
  (retrieve-all
    (select :*
      (from :user))))
;=> (#<HASH-TABLE :TEST EQL :COUNT 4 {100815FA03}> #<HASH-TABLE :TEST EQL :COUNT 4 {100815FE43}>)

See Also

Author

Copyright

Copyright (c) 2014 Eitaro Fukamachi ([email protected])

License

Licensed under the BSD 3-Clause License.

datafly's People

Contributors

fukamachi avatar jzumer avatar

Stargazers

nerap avatar Mateus Ryan avatar Manfred Bergmann avatar Vishal Belsare avatar Leandro Hernandez avatar Delon R. Newman avatar Dmitry Kosenkov avatar Michael McCaslin avatar Michael Kohl avatar Józef Piątkiewicz avatar  avatar Skye Freeman avatar Dubois Hugues avatar Zonoia avatar Jorge Gomez avatar Igor Sutton avatar Greg Werbin avatar Sahin Habesoglu avatar Kohei Hosoki avatar  avatar  avatar  avatar Ivo Šmerek avatar Andres Mariscal avatar Jonathan McHugh avatar w96k avatar madosuki avatar C-Entropy avatar Yufan Lou avatar Jean-Francois Parent avatar Robert Krahn avatar Gregory Mann avatar zacklocx avatar David.Gao avatar Rosso avatar kuwze avatar lagagain avatar Gabriel Aires Guedes avatar Lawrence Liu avatar Gavin Lam avatar Jyrki Gadinger avatar Mark Hudnall avatar  avatar Jack Liu Shurui avatar Bharath avatar Tamilselvan R avatar Avelino avatar chenkaiC4 avatar  avatar defunkydrummer avatar Konstantinos avatar  avatar Daniel Peters avatar Ritchie J. Latimore avatar Vikrant Sagar avatar meymao avatar huw hoover avatar Carl avatar Eddie avatar Travis avatar Artem Chernyak avatar kostafey avatar David Vázquez Púa avatar  avatar Denver Fernandes avatar Alessandro Miliucci avatar  avatar Alan D. Salewski avatar Xe Iaso avatar SuperUltimateGreatAugmentedNumerousObsolete-NoUser avatar Sergey Katrevich avatar Jordan Brown avatar Arthur M Meskelis avatar Damon Zhao avatar Alexander F. Rødseth avatar Niclas avatar ayato-p avatar Yanying Wang avatar Mark Karpov avatar Jaime Fournier avatar David Aguilar avatar Adlai avatar Olaf Merkert avatar  avatar hamlet avatar Duarte Barreira avatar Lucas Severo avatar gihnius avatar David Davidović avatar Samuel Chase avatar Javier Olaechea avatar Juanito Fatas avatar muyinliu avatar Tony Rossini avatar Katsuyoshi Ozaki avatar Martin Haesler avatar Wataru NAKANISHI avatar

Watchers

Dave Creelman avatar  avatar Sergey Katrevich avatar James Cloos avatar Manfred Bergmann avatar Emile van Raaij avatar Sasha Shipka avatar Gabriel Aires Guedes avatar  avatar

datafly's Issues

Rowid functionality missing

When inserting a row with an auto increment in sqlit3 for example, the newly created id as returned by sqlite3:last-insert-rowid (http://quickdocs.org/cl-sqlite/api) for the sqlite3 DBMS should be accessible. Sqlite3 does not support the RETURNING SQL clause. Alternatively, add possibility to use the last_insert_rowid() sql function in SxQl.

render-json fails on table row with optional relationship

render-json fails with an unbound slot error on a table row with an optional relationship, when the row has a null value for the relationship field.

The functions called are (datafly.json::convert-for-json) and (datafly.json:encode-json)

To reproduce table with mito:

(mito:deftable tweet () ((text :col-type (:varchar 64)) (author :col-type (or user :null))))

Then (render-json tweet) will fail if the user is :null with an unbound slot error.

note: I would've forked and submitted a fix / pull-request but I'm relatively new to lisp...

ros build myapp # with sbcl and datafly

I don't know if this has anything to do with datafly, roswell or log4cl.
But I don't have this problem when not including datafly.
It works with ccl-bin but not sbcl

$ cat test.ros 
#!/bin/sh
#|-*- mode:lisp -*-|#
#| <Put a one-line description here>
exec ros -Q -- $0 "$@"
|#

(ql:quickload "datafly")

(defpackage :ros.script.test.3670228186
  (:use :cl))
(in-package :ros.script.test.3670228186)
(defun main (&rest argv)
  (declare (ignorable argv)))
$ ros build test.ros 

To load "datafly":
  Load 1 ASDF system:
    datafly
; Loading "datafly"
................
ERROR - Caught SB-THREAD:JOIN-THREAD-ERROR during
        '(LOG4CL-IMPL:STOP-HIERARCHY-WATCHER-THREAD :JOIN-THREAD):
        Joining thread failed: thread #<THREAD "Hierarchy Watcher" ABORTED

                                        {1007F913C3}> did not return normally.;
        Continuing.
[undoing binding stack and other enclosing state... done]
[saving current Lisp image into test:
writing 4800 bytes from the read-only space at 0x20000000
writing 3216 bytes from the static space at 0x20100000
writing 82640896 bytes from the dynamic space at 0x1000000000
done]

sqlite3 handle unbound error

(connect-toplevel :sqlite3 :database-name #P"myapp.db")

(retrieve-one
  (select :*
    (from :user)
    (where (:= :name "nitro_idiot"))))

Using this code results in the following error, but using DBI directly works fine:

The slot SQLITE::HANDLE is unbound in the object
#<SQLITE-HANDLE {1007284013}>.
   [Condition of type UNBOUND-SLOT]
(let* ((query (dbi:prepare *conn* "SELECT * FROM users")) (result (dbi:execute query))) result)
#<DBD.SQLITE3::<DBD-SQLITE3-QUERY> {100284A883}>

possible issue using encode-json to encode a retrieve-all result consisting of a single result row

I technically found this while using caveman2 and traced it to the encode-json function imported from datafly.

If I have a table called "test" structured as such:

id foo
1 "baz"
2 "baz"
3 "bar"
4 "baz

and run the following code

(encode-json 
  (retrieve-one
    (select :*
      (from :test)
      (where (:= :foo "bar")))))

The result is as follows
"{\"id\":[1,\"foo\",\"bar\"]}"
Which is looks to be problematic. I was instead expecting something along the lines of
"[{\"id\":1,\"foo\":\"bar\"}]

After looking at encode-json and then modifying the convert-for-json function to try to trace the code's flow, I found that the function identifies the returned list as an association-list instead of a regular list of embedded plists. Running (convert-for-json '((:id 1 :foo "bar"))) yields
((:ID . #(1 :FOO "bar")))
instead of something like the following
#(((:ID . 1) (:FOO . "bar")))

While this is consistent with how association-list-p works, it causes strange problems when processing data on the receiving end when the it expects an array of objects.

I've wrote a test for the expected behavior and a copy of the modified "verbose" version of convert-for-json that I used to trace the problem and sent a pull request for your perusal.

The result of encode-json is error, when the list has only one member

My case is

 (datafly:encode-json  '((COMMENT-ID 4 COMMENT-AUTHOR liuyan COMMENT-TEXT testcomment)))

The error result is

"{\"commentId\":[4,\"commentAuthor\",\"liuyan\",\"commentText\",\"testcomment\"]}"
And the right result should be
"[{\"commentid\":4,\"commentauthor\":\"liuyan\",\"commenttext\":\"testcomment\"}]"

Is it possible to use a model for a subtable as a slot value

Is it possible to use structure model in this case:
`

CONTENT> (defstruct a
slot1
slot2
(slot3 '(1 2 3) :type cons))
A
CONTENT> (with-connection (db)
(execute (create-table :a_storage
((id :type 'string)
(slot1 :type 'integer)
(slot2 :type 'integer))))
(execute (create-table :a_slot3_values
((id :type 'string)
(value :type 'integer))))
(execute (create-table :a_slot3_links
((id_a :type 'string)
(id_slot3 :type 'string)))))
; No value

`
I use additional table for store values of slot3 and table for a links. In this case should I to create instances of 'a' directly in the code? Doesn't exist a sugar to use AS to create 'a' automatically? Sorry, if this question isn't about datafly but I using its for learning how to work with databases. Thank you.

Impossible to emit booleans using encode-json

A the moment it is impossible to emit booleans using the encode-json functions. It possible to do this using jonathan directly, by supplying the :false keyword. This does not working using datafly because the keyword gets removed in the typecase in convert-for-json.

This could be solved modifying this typecase, however it might be better to remove the typecase and use CLOS instead. This way the code is more extendable and the user could change the behavior them self.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.