Keep sample data in repo about xtdb-tutorial HOT 7 CLOSED

I think removing all references (textual and links) to the files from the Markdown, is a good approach. If we are removing all reference to the files from Markdown, I'd just remove them from the repo.

When going through the tutorial, I felt the files were somewhat of a distraction. The files contained some data, but the code cells contain some (human-modified) version of that data. Switching context to read the files created a speed-bump in the flow of the tutorial. If the files were being read by the code directly, that would have made the situation a lot better for me. If we did that, it might even work with the current Nextjournal Github Explorer.

However ...

Given that this is a tutorial, and we want to focus on the XTDB concepts and not ancillary concepts (like how to convert a data file into something XTDB can accept), my preference would be to go with @johantonelli 's suggestion of removing all reference to the files.

from xtdb-tutorial.

More context from the PR I'd butchered by bringing in all these details (#5):

Advice from @nextjournal:

Hi Steven,

In your scenario, accessing tutorial notebooks from our (experimental) github explorer https://nextjournal.com/github/xtdb/ might turn out useful for you (note I just noticed we have a routing problem and that should be accessible from https://github.nextjournal.com/xtdb and we need to fix that).

Anyway, once you visit -- say -- https://nextjournal.com/github/xtdb/xtdb-tutorial/blob/main/1-getting-started.nextjournal.md
then your notebook's "working directory" is at the repo's home (this is visible in the clojure runtime settings in the sidebar). In a clojure cell you might for instance (slurp "files/commodities.txt") and get back the files content...

There's caveat here, you'll need to also add a deps.edn in the root folder of the repo (or mount each in-notebook deps.edn at /xtdb-tutorial/deps.edn from within the runtime settings and export again) in order for the deps to be taken into account.

As an alternative, starting from a regular notebook you can achieve the same result by using a github component (https://nextjournal.com/help/github)
to mount the tutorial repo in your notebook: in this case you can access the files in the repo by traversing the runtime filesystem instead of arguing by uploads.

....

We just fixed our github subdomain, so now you can actually use https://github.nextjournal.com/xtdb (or also https://githubnj.com/xtdb :-)

Is there a way to automatically mount a GitHub component in Markdown?

A hack could work: add a github repo in a blank notebook on nextjournal and mount it in a (even bash) cell (list its content to ensure the mount work) and check how that's represented in terms of markdown references by exporting the notebook. Then try to rebuild that in your markdown source, you might need to hack the {details}-metadata at the bottom, not sure it will work if you strip that out altogether, but it could be worth a try.

Otherwise you start from a nextjournal notebook, just export it once and try to track in git the relevant changes from the first export...

... it seems like the GitHub Explorer option might be the best bet, but this issue requires some experimentation.

from xtdb-tutorial.

After trying out a few ideas, here is my email to NextJournal.

I am trying to see if the xtdb tutorials that are on Nextjournal can be hosted at Github.
Currently, some of the tutorials have a link to a file that has previously been uploaded to Nextjournal. It is a small text file. The Notebook at Nextjournal is at

https://nextjournal.com/xtdb-tutorial/put

and the sub-section is "Ticket" under the section "Assignment". Over there, you will see a file called "commodities.txt" that has been uploaded to Nextjournal. If I look at the Markdown for that sub-section that contains the file, I see

[commodities.txt][nextjournal#file#48b4711b-6c2d-4b09-9584-a15ef4f05ef2]

Instead of referring to a file uploaded to Nextjournal, how do I make a similar link to a file that is relative to the local repo on Github?

I tried

[commodities.txt](files/commodities.txt)

but that does not seem to work. I also tried

[commodities.txt][files/commodities.txt]

to match the markdown in the Nextjournal Notebook.

If you want to take a look at what I tried, see

https://github.com/bombaywalla/xtdb-tutorial/blob/expt/2-put-transactions.nextjournal.md

Would you please suggest a way to get this link to work via the Github Explorer as it does on Nextjournal with an uploaded file.

from xtdb-tutorial.

Nextjournal's response is below. Looks like the feature we'd need is not yet available.
Since it seems we cannot currently go "markdown first", I suggest we store the files under the files directory on github (even though we currently cannot use them). And, for now, manually upload to Nextjournal.

If you have thoughts for other directions to try, please let me know.

Hi Dorab,

Thank you for trying our github explorer! At present it is not possible to go "markdown-first" for importing files in nextjournal, but if you access the tutorial from within the github explorer you can read repo files directly within your clojure runtime e.g. in a clojure cell with

(slurp "files/commodities.txt")

as you can see in the sidebar settings the repo itself is mounted as the working-dir of the runtime.

What you describe above currently works for images, and we might extend it to other components (like the file upload) in the future.

Hope this helps. Let us know if you have further issues!

Andrea

from xtdb-tutorial.

Hey @bombaywalla ! Thanks for doing this research... I really appreciate it. If the GitHub Explorer doesn't satisfy our requirements, I think it makes sense to solve the problem as you describe. This unfortunately adds an extra build step, but only when the sample data text files change. That doesn't sound too bad, as long as we remember how to update the sample data files the next time we need to do it. 😃

from xtdb-tutorial.

Just a thought here - we don't actually run any of the files in the notebooks, just refer back to them. I think the most simple solution would be to remove all references to them from the md files. We could store them in the repo for reference, but then we have the problem of becoming out of sync when edits are made.

from xtdb-tutorial.

Thanks @bombaywalla - very good point on it breaking the flow. Once I've merged the reformatted md files I'll remove the data files from the repo and close this issue.

from xtdb-tutorial.