jorinvo / edn-data Goto Github PK

EDN parser and generator that works with plain JS data, with support for TS and node streams

License: MIT License

JavaScript 0.30% TypeScript 99.70%

edn clojure data-format serialization parser typescript node-stream javascript

edn-data's Introduction

edn-data

edn-data is a JavaScript and TypeScript library that provides functionality to generate and parse data in the EDN format, as known from Clojure land.

Why create another library for working with EDN?

Support TypeScript with a strictly typed interface
Support the rich EDN types
Allow working with plain data in JavaScript, so everything can also be serialized to JSON
Support modern JavaScript types such as Map, Set and bigint
Support streaming EDN lists as standard Node stream in Node.js
Have a solution that works in Node.js and in browsers

Why work with EDN in JavaScript or TypeScript?

The number one reason is, your code wants to interact with data coming from Clojure.

But even outside Clojure EDN can be a compelling data format in a world where developers are stuck fighting YAML and complaining about JSON.

EDN has:

less syntax than JSON
built-in data types to represent maps, sets, keywords, dates, uuids, chars, bigints, ...
multi-line strings
tags and symbols to represent rich, custom data types
comments
streaming parsers
no relevant whitespace

Get started

Install with:

npm install edn-data

Parsing EDN

By default parsing returns JSON-compatible data structures that can represent all of the rich EDN types.

There are options to make it easier to parse simpler types.

import { parseEDNString } from 'edn-data'

parseEDNString('{:key "value" :list [1 2 3]}')
// Returns:
{
  map: [
    [{ key: 'key' }, 'value'],
    [{ key: 'list' }, [1, 2, 3]],
  ],
}

parseEDNString(
  '{:key "value" :list [1 2 3]}',
  { mapAs: 'object', keywordAs: 'string' },
)
// Returns:
{
  key: 'value',
  list: [1, 2, 3],
}

EDN lists can be streamed value by value as standard Node.js Readable streams. This is not available in the browser.

import { parseEDNListStream } from 'edn-data/stream'

const s = parseEDNListStream()
s.write('(1 2 3)')
s.read() // 1
s.read() // 2
s.read() // 3

Generating EDN

EDN is generated from plain JSON structures.

With toEDNString the same data structures parseEDNString returns can be turned to valid strings, and they represent a rich set of types.

For simple JavaScript types often toEDNStringFromSimpleObject might be the simpler use.

import { toEDNString, toEDNStringFromSimpleObject } from 'edn-data';

toEDNString({
  map: [
    [1, { key: 'keyword' }],
    [{ set: [1, 2] }, { char: 'a' }],
  ],
})
// Returns:
'{1 :keyword #{1 2} \a}'

toEDNStringFromSimpleObject({ first: 1, second: 2 })
// Returns:
'{:first 1 :second 2}'

Development

The library is developed driven by its tests.

Verify them using

npm test

For continuous development use

npm run test:watch

Ensure the code formatting with

npm run fix

CI verifies tests and creates npm releases for tags automatically.

Publish a new version

Change the version in the package.json
Push a commit to master in the following form Release <version>
A Git tag will be created and the new version will be published to NPM

License

MIT

edn-data's People

Contributors

Stargazers

Watchers

Forkers

refset cdupuis drethegreat1 mailonline prepor

edn-data's Issues

Support for unicode chars in parser

They are their escape sequence as explained here: https://github.com/edn-format/edn#characters

We would transform them to actual unicode in JS.

Encoding a nested map?

Hi,

I hope you don't mind me reaching out via github issues, but I'm lost as to how to encode a javascript structure (below), into edn (as a vector of vectors, with each inner vector containing a key/map pair):

javascript:

{"foo/bar": {message: "wibble"}}

expected edn:

[[:foo/bar {:message "wibble"]]

Experimenting with your library, I've got close with:

const encoded = toEDNString(
    [[
        {key: "foo/bar"},
        [{key: "message"}, "wibble"]
    ]]
);

to give:

[[:foo/bar: [:message "wibble"}]]]

I've tried a few things, with no joy, even trying this:

const encoded = toEDNString(
    [[
        {key: "foo/bar"},
        toEDNStringFromSimpleObject({message: "wibble"})
    ]]
);

to give:

[[:foo/bar "{:message \"wibble\"}"]]

And finally:

const encoded = toEDNString(
    [[
        {key: "foo/bar"},
        {map: [{key: "message"}, "wibble"]}
    ]]
);

throws an error:

node_modules/edn-data/dist/generate.js:69
            .map(([k, v]) => `${exports.toEDNString(k)} ${exports.toEDNString(v)}`)
                 ^

TypeError: object is not iterable (cannot read property Symbol(Symbol.iterator))

Would you have any suggestions on how this might be done?

Thank you for your time.

Support parsing bigint

As explained here: https://github.com/edn-format/edn#integers
integers ending with an N have arbitrary size. We should parse them as the JS type bigint.

Use as ES6 module in browser?

I'd like to use this library in the squint playground. It seems I'm only able to do this via esm.sh:

example

The esm.sh service rewrites all the module.exports + require stuff to proper ES6 modules.
Would it be an idea to publish ES6 modules directly, such that this can be used from unpkg.com etc?

TypeScript return type of parse function can depend on the passed options

We could further restrict the return type by parametrizing the function and using a conditional type on the options for parsing the different data types.

This would help a user of the library since they then need less checks on the returned data.

Validate characters in keywords, symbols and tags

To generate valid EDN, we need to validate what characters are allowed in keywords and symbols when generating EDN.

They also must contain not more than one slash, which is not allowed to be the first or last char.
And in the case of tags, they even must contain a slash if it's a custom tag provided by the user.

If invalid chars are given, throw an error.

lock file mismatch + unverified commits

Hello,

I am using this package in one of my projects. I noticed a new version; however, upon checking noticed that the lock file doesn't match the release. Plus, there are some unverified commits, which aren't a problem yet.

Could you kindly check and make a new release with the right lock file?