xtdb / xtdb-tutorial Goto Github PK

Users should not confuse it for evict.

Issue extracted from #4

Hi, I was trying out the xtdb tutorial at nextjournal, and I've noticed there are no data returned in the first entity-history call. The author was indeed expecting all transaction data to be returned instead.

https://nextjournal.com/xtdb-tutorial/delete

How to reproduce.

1. create a test clojure file to create a node, add the transactions in the example and query about the history

(ns tutorial.test
  (:require [xtdb.api :as xt]
            ))


(def node (xt/start-node {}))

(xt/submit-tx node
              [[::xt/put {:xt/id :kaarlang/clients
                          :clients [:encompass-trade]}
                #inst "2110-01-01T09"
                #inst "2111-01-01T09"]

               [::xt/put {:xt/id :kaarlang/clients
                          :clients [:encompass-trade :blue-energy]}
                #inst "2111-01-01T09"
                #inst "2113-01-01T09"]

               [::xt/put {:xt/id :kaarlang/clients
                          :clients [:blue-energy]}
                #inst "2113-01-01T09"
                #inst "2114-01-01T09"]

               [::xt/put {:xt/id :kaarlang/clients
                          :clients [:blue-energy :gold-harmony :tombaugh-resources]}
                #inst "2114-01-01T09"
                #inst "2115-01-01T09"]])

(def res
  (xt/entity-history
   (xt/db node #inst "2116-01-01T09")
   :kaarlang/clients
   :desc
   {:with-docs true}))

res

2. Fire up a REPL and eval the file. The key here is to eval the file in one go.
3. res is [].

Is this a timing issue with db synching? if I (xt/sync node) just before the query, it works.

Thanks

At the ending of the "Departure" section of the Evict tutorial, it seems like there must be an image but all that is displayed for me is:
![ice.jpg][nextjournal#file#f0026e73-9244-45f5-ac13-bab33663707d]

commodities.txt, sample-data.txt, etc.

CIDER is giving me this:
Found lib name 'xtdb.api' containing period with prefix 'quote'. lib names inside prefix lists must not contain periods

No issue with simply doing (require [xtdb.api :as xt])

From Sean Corfield in Clojurians:

the discussion of match in the tutorial was very confusing for us and seemed... a bit pointless... since it seems it has to exactly match the most recent existing document in all fields? Specifically for the trading example, it seems like the precondition for the trade TX is that both parties have "enough" (enough credits and enough product) so an exact match there just seems weird... and it doesn't explain that the only way to tell if a match-based TX "worked" is to (await it and then?) ask if it was actually committed... at which point, wouldn't you need to re-fetch the original docs, re-compute the changes, and then re-run the TX?

Extracted from #4

See: https://clojurians.slack.com/archives/CG3AM2F7V/p1630609495040600

• Lesson 5 ("match"): s/sees the 1000 credits move from/sees the 100 credits move from/
• Lesson 7 ("evict"): There's a reference to history-descending, which I assume was changed to entity-history as used in the example code.
  via https://clojurians.slack.com/archives/CG3AM2F7V/p1660480043092479 (thanks for reporting!)

See conversation from #5 ... may involve begging for APIs from the @nextjournal folks, though. ;)

Since these messages will disappear from Slack in a matter of days, copying them here:

seancorfield 3:46 AM

@steven Deobald We did run into a couple of things -- they'll be obvious from the recording but I figure I can give you a head start: the link in the planets tutorial to the datalog section is a 404 (it has /datalog-queries instead of just /datalog); the discussion of match in the tutorial was very confusing for us and seemed... a bit pointless... since it seems it has to exactly match the most recent existing document in all fields? Specifically for the trading example, it seems like the precondition for the trade TX is that both parties have "enough" (enough credits and enough product) so an exact match there just seems weird... and it doesn't explain that the only way to tell if a match-based TX "worked" is to (await it and then?) ask if it was actually committed... at which point, wouldn't you need to re-fetch the original docs, re-compute the changes, and then re-run the TX?

I think we also struggled with how exactly a delete works, since it was pitched as helping support data privacy laws, but then there's evict which seems more suitable for things like GDPR "right to forget"?

dorab 12:49 PM

Thanks to everyone who attended and participated. Thanks to @nate for driving. Definitely a lot of subtleties around how delete works and implications on the history of each entity.

@steven Deobald In addition to what @seancorfield mentioned about the tutorial, the files (e.g., the commodities) associated with some of the tutorials seemed to have the crux ids and not the xtdb ids. Also, the manifest update at the end of the Saturn tutorial has the "BITEMP" badge missing.

dorab 12:59 PM

In any new tutorials, it would be great to have a few exercises after each (sub-)section that can be used by the viewer to better understand the topic and perhaps explore some of the edge cases.

Steven Deobald

@seancorfield @dorab Thank you! That's really helpful feedback. The Space Adventure tutorial is much older than the Learn Datalog tutorial and also grew as XT (well, Crux) grew. I think the evolutionary nature of the tutorial probably explains the confusion between delete and evict, and that's definitely something we need to correct. For the record, evict is the operation you want for non-functional things like GDPR compliance. delete is for the business domain and leaves the entity history intact (as I'm sure you saw).

Everyone is welcome to create GitHub issues (https://github.com/xtdb/xtdb-tutorial/issues) if they like, but I'm also happy to transcribe your feedback when I get a minute.

There is an insidious typo in the example code for filter-appearance.

Currently:

(defn filter-appearance
  [description]
  (xt/q (xt/db node)
        '{:find [name IUPAC]
          :where [[e :common-name name]
                  [e :IUPAC-name IUPAC]
                  [e :appearance appearance]]
          :in [apperance]}
        description))

the :in parameter is misspelled.

The usage example (filter-appearance "white solid") always returns the correct result since there is only one item with an :appearance attribute.

Should link to /datalog not /datalog-queries. See https://juxt-oss.zulipchat.com/#narrow/stream/194466-xtdb-users/topic/broken.20link.20in.20the.20tutorials/near/253900264

This doesn't seem related to the code, it hits an error before getting there.

It looks like we somehow have a badly cropped meteor image

We should replace it with the old one https://github.com/crux-labs/crux-tutorial/blob/main/images/6b-evict-meteor.jpg