Git Product home page Git Product logo

curvenote's Introduction

curvenote

curvenote on npm MIT License CI

Create, edit, share and publish scientific documents.

Overview

curvenote is an open source library and command line interface (CLI) to create share and publish technical documents.

  • Write papers and reports in Markdown and Jupyter (docs)
  • Create and share interactive websites (docs)
  • Export to Word, LaTeX, and PDF with any template (docs)

In addition to being completely local, curvenote can optionally sync content to https://curvenote.com to allow you to work with collaborators who enjoy WYSIWYG editing, commenting & real time collaboration for technical documents.

Get Started

Curvenote is available through Node and NPM. Unless you already have this on your system you will have to install NodeJS. See full install instructions in the docs!

npm install -g curvenote
curvenote init
curvenote start
curvenote deploy

Built with Curvenote

Curvenote allows you to easily create, edit, and publish content to the web as a fast, optimized site with interactive citations, cross-referencing, math, and dynamic figures from Jupyter Notebooks.

Curvenote can be used to create all sorts of open-access content, click the links below to see some examples!

Interactive and Linked

The default website you create can have interactive Jupyter Notebook features, and live-preview of cross-references and citations.

PhD Thesis with linked references, equations, and export to PDF.

Interactive Papers with Jupyter Notebooks and interactive visualizations.

These interactive scientific sites can be easily deployed on a hosting service called curve.space or can also be hosted on your own custom domain.

Work locally with Live Reload

The client library is entirely local, and rebuilds in ~50ms for most projects. Meaning you can preview your content as you are writing!

Direct export from Curvenote

First login, see authorization docs to get an API token.

curvenote token set
> YOUR_API_TOKEN

Then you can directly export your curvenote documents to:

  • Microsoft Word (.docx)
  • Markdown (.md) - using MyST
  • LaTeX (.tex)
  • PDF (.pdf)
curvenote export docx https://curvenote.com/@curvenote/blog/communicating-science communicating-science.docx
curvenote export md https://curvenote.com/@curvenote/blog/version-control-for-scientists version-control.md
curvenote export tex https://curvenote.com/@curvenote/blog/version-control-for-scientists version-control.tex -template plain_latex
curvenote export pdf https://curvenote.com/@curvenote/blog/version-control-for-scientists version-control.pdf -template arxiv_nips

LaTeX and PDF Dependencies

Exporting to:

  • LaTeX (latex|tex) with a template option specified
  • or to PDF

Requires the jtex python package to be installed and available on the user's PATH.

With python 3.7 or greater installed, install jtex via pip:

python -m pip install jtex

Development

All dependencies for curvenote are included in this repository (a monorepo!).

What's inside?

curvenote uses npm as a package manager. It includes the following packages/apps:

Apps:

  • curvenote: the Curvenote command line interface (CLI)
  • curvespace: the web-experience for Curvenote sites, using Remix

Packages:

  • ui-providers: React providers for references and site configuration
  • myst-to-react: expose MyST content as an article, built with React
  • site: components and utilities for building React and Remix sites
  • icons: icons used throughout our projects, built for React
  • citation-js-utils: utilities for using the citation-js library
  • eslint-config-custom: eslint configurations
  • tsconfig: tsconfig.jsons used throughout the monorepo

Each package and app is 100% TypeScript.

Versioning & Publishing

Curvenote uses changesets to document changes to this monorepo, call npx changeset and follow the prompts. Later, npm run version will be called and then npm run publish.

Utilities

curvenote is built and developed using:

Build

To build all apps and packages, run the following command:

cd curvenote
npm run build

Develop

To develop all apps and packages, run the following command:

cd curvenote
npm run dev

curvenote's People

Contributors

dependabot[bot] avatar dylangrandmont avatar fwkoch avatar github-actions[bot] avatar rowanc1 avatar stevejpurves avatar yxwww avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

curvenote's Issues

Add integration tests

Add an integration test suite that

  • uses a session based on a generates token, similar to API tests
  • tests exporting to all formats
  • test exporting pre-configured content from the @curvenote/testing project
  • ...

Only tiny pandas tables will be added to exports

Curvenote store large pandas tables in files, they are not immediately available over the API, instead, a download link is provided. The cli should be updated to download the full table (HTML) content when required.

Populating Frontmatter from Config

  • Style/Design Information
    • numbering
    • hide_authors
  • Licensing, open_access, etc.
  • DOI

These should be able to be propogated from a few different places:

  • curvenote.yml
  • each folder, maybe in a frontmatter.yml?
  • frontmatter in an article or notebook

Other pieces:

  • license, add OSI licenses in addition to CC
  • split license defaults into code/text

Layout with widths

We will probably always have some content that is wider than our central page, like code cells , text outputs, large figures and tables but currently we are not making best use of space in our layout -- how can can we improve this?

An example is the remix docs which are nice and follow the same left-nav | right-outline schema that we have.

On a 4:3 screen, they make good use of all available horizontal space while not feeling cramped.

Remix  Environment Variables 2022-04-08 10-50-10

https://remix.run/docs/en/v1/guides/envvars

on a 16:9 screen, they maintain the 4:3 layout which works nicely. So this simply makes better use of available space.

BUT, other sites more aligned with scientific content take different approaches:

Quarto sticks with a narrow content layout, but their font sizes are much smaller and light, they also allow large wide or content marked as wide to bust out of the margin:

Distill - their nav areas scroll, which we don't want but they break margins to deal with wide content which is ๐Ÿ˜

Objectives:

  • What small change to basic styling / layout / gutters could be made to better accommodate content?
  • Could we create additional CSS classes allowing us to accommodate wide content like quarto / distill, e.g. busting into RHS margin? and hiding RHS menu when it collides?
  • Other ideas?

Cannot Find Image

Cannot find image, when it is a base64 image.

We should write this to the correct file.

image

Notebooks don't link properly

During web builds, curvenotejs reconciles links to other articles in a project, converting the oxa: link to a correct relative link for the build.

For some reason, this is not happening for notebooks, but should work.,

An example is here https://stevejpurves.curve.space/la-palma-earthquakes/notebooks but this can be recreated with any curvenote project, by including an article and notebook and in the article adding a cross-reference to the notebook using [[nb:

Open close LeftNav items

Current state:
image

Need to think through how to have these open/close collapsible. Take a look through firebase docs, jupyterbook as some inspiration.

Download notebook

For any page that is a notebook, we should provide a download link so the user can download that in ipynb format.

This will require some work at build time to generate the link appropriately, currently the ipynb files are only available by making a request to api.curvenote.com. e.g. the notebook at https://curvenote.com/@stevejpurves/la-palma-earthquakes/data-screening-pandas is available on the API endpoint https://api.curvenote.com/blocks/1Bk7uPlcMuaTyKEshESj/w7dhgP7jr4IrWGyDQYfZ/versions/10/download

In terms of design/UI we need somewhere to place that on the page in ourheader blocks - some design/thought is needed / need to look around?

Header blocks for notebooks could look like this if author info has been added:

image

Or like this if it has not:

image

Or the author information is potentially empty.

Table of Contents

There are many times where the conversion of our structure to JB drops some articles from the TOC.

We should ensure that no articles/notebooks are lost when we pull content.

API rejects old version of client library

  • The CLI already sends the version headers
  • Minimum version check on the minor.minor version.
  • CLI reports back to user that they need to run npm i curvenote@latest

No scroll on RHS navigation?

iyHgYz8Oge

Would it be better for the RHS navigation to be just full height, and if longer than the viewport you need to scroll the page?

curvenote init local

Using a local github repo, be able to read the paths of folders (md, notebooks) and check the contents on/off to create various sections to show in the web interface.

  • #93
  • #97
  • #94
  • #95
  • Get old init & init from curvenote working on the new config
  • Add should write project config on add
  • Watch new config files and rebuild
  • Correctly handle frontmatter from new configs
  • Unit tests!!

links with underscores get escaped and broken

when we have a link with underscores, these seem to get escaped in myst

image

and then those backslashes get escaped (URL encoded) again when curvespace is built leading to a broken link

image

Improve deployment to specific domains

There is currently a list of domains in the curvenote.yml, this should be respected with you say curvenote deploy!

There should be some handling for non-curve.space domains (simple for now).

Notebooks - follow links in markdown cells for local images

Currently during web builds, we do nothing to find and process local images that may be referenced in a markdown cell in a jupyter notebook.

We should start doing that when:

  • the image included in available on a url
  • the image included is available to us on disk relative to the notebook

Also, images can be referenced:

  • in markdown syntax ![curvenote logo](./curvenote.png)
  • using an HTML <img /> tag - (see also #55)

All these scenarios should be supported.

Currently, it seems in some cases that the build process correctly constructs links to local images when specified in markdown but the images are never made available in the public folder. e.g. https://transform.softwareunderground.org/2021-practical-python/t21-practical-seismic-with-python

Note: currently when we add notebooks to curvenote.com, images are lost :( we need more work on version control to fix that, so for this issue we have to manually make available and locally referenced image in order to test.

Admonition titles & round trip

  • Add titles to callouts in the app
  • align / standardize schema with myst / jupyterbook / sphinx
  • round trip nicely from app to myst

Notebooks render html in markdown cells (safely with limited schema?)

Currently in web builds we do nothing to handle html-in-markdown properly for notebooks, meaning that we end up rendering HTML source as plain text content.

It is common for notebook authors to use HTML in markdown cells to achieve things like this:

image

example notebook: https://github.com/gimli-org/transform2021/blob/main/1_Creating_a_subsurface_model.ipynb

Which currently renders in curvenote (and curvespace) as:

image

Objective

Modify notebook processing during the web build process to handle HTML in markdown cells.

The processNotebook function is where the specialised processing for notebooks happens at the moment. https://github.com/curvenote/curvenotejs/blob/feat/web/src/web/cache.ts#L46

When content/markdown cells are encountered we should process these (using schema/mdast) to sanitize (<script> removal) render the HTML appropriately.

Note: rendering images included in HTML, both local and via URL should be handled here ๐Ÿ‘‰๐Ÿป #56

v1 Release

  • curvenote init
  • more logging on the init pull.
  • check node version >=14
  • check that node 16 works
  • check npm version (only on web install) I think anything <8 will fail? need to test. (@Yxwww says since npm v7)
  • ensure oxa IDs are in the repo
  • check that init runs without logging in
  • Add -v as well as an option

Init

  • your.curve.space --> rowanc1.curve.space
  • Remove Would you like to add additional projects now?
  • Auto pull content
  • only say start (not install)

Emergent

  • choose a license
  • remove extra title for our renderer
  • get rid of serve
  • we are going to deploy things to this website, please confirm.
  • issue #59
  • issue #56
  • Export to docx / pdf / latex should work

Final Changes

  • grey out Import from Local curvenote init option in all cases
  • grey out Use Demo content curvenote init option in all cases
  • release v1 to npm ๐Ÿš€

Landing page for curvespace (featured & cards)

  • should be beautiful
  • can be in place at the root os a domain/site
  • can be in place at the root of a set of nested content
  • still looks good without thumbnails on the cards
  • looks even better with thumbnails

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.