Bug: StrictYaml cannot serialize `None` (null value) about strictyaml HOT 20 OPEN

from strictyaml.

I think it would be useful to define a value to be optionally None (YAML's null).
The problem is that None is already taken in Python, so that None() is not available, therefore I went with None_().
An alternative might be NoneOr(Type()) or Nullable(Type).

from strictyaml.

Hi! Any updates on this? Do we have a built-in validator for "null" -> None or should we extend ScalarValidator as Arne has shown? (thanks btw). Maybe Null is a meaningful name as well, since it is not protected?

from strictyaml.

I've added NullNone as a new validator that parses "null" to None and serializes None to "null" and release tomorrow. Apologies I dropped the ball on this one.

from strictyaml.

I don't plan on having a NotNull type. Should I?

I think only NotNullStr is needed due to default Str behavior. IMO in most cases it is crucial to know when a str field is expected to not be null

from strictyaml.

I wanted it to to mirror the existing EmptyNone / EmptyDict / EmptyList validates. I don't plan on having a NotNull type. Should I?

I don't think NotNull makes sense. Regarding the EmptyDict I always found it hard to grasp (if empty then dict? accept empty or dict? if not defined then interpret as dict?). But following that scheme NullNone makes sense. Either naming decision is valid.

from strictyaml.

It doesn't say it explicitly, but the answer is shown in various examples. The solution is usually:

>>> import strictyaml
>>> yaml = strictyaml.as_document({'a': 'null'})

Alternatively:

>>> import strictyaml
>>> yaml = strictyaml.as_document({'a': '', 'a_is_null': 'true'})

or

>>> import strictyaml
>>> yaml = strictyaml.as_document({})   # a is missing, and therefore implied null

Or you could set the string to "none" or "None". Or, whatever you feel is most appropriate.

The key is that assigning type is a function of the application, not the strictyaml document.

To word that in a more pythonic manner: strictyaml only does strings. That is why schemas are often used by the libraries to aid in translation of those strings to anything else.

from strictyaml.

I'm not sure how I could ever guarantee that the string value never happens to actually be "null", so I guess the 3rd option makes the most sense. Unfortunately, this requires a schema.

Thank you for explaining this. I would like to see this added to the documentation explicitly, since this seems like a common data value, and I can easily imagine others running into this issue.

from strictyaml.

Thanks @JohnAD you summed it up perfectly: strictyaml only does strings, dicts and lists by default. anything else requires disambiguation via schema.

I'll add this to the documentation explicitly - I agree it's not as clear as it could be.

from strictyaml.

So how do I specify a (optional) null in the schema so that I get a None?
I thought about using Optional('a', None): Str() but it still returns "null".
EmptyNone() expects the value to be missing, which is also not what I want.

from strictyaml.

Here is my workaround:

from strictyaml.scalar import ScalarValidator

class None_(ScalarValidator):
  ''' Validator for strictyaml schemas expecting a None. '''

  def validate_scalar(_, chunk):
    if chunk.contents == 'null': return None
    chunk.expecting_but_found("when expecting None")

  def to_yaml(_, data): return 'null'

Make sure to put None_() before | Str() or other unions.

from strictyaml.

from strictyaml.

from strictyaml.

This is what has been working for me so far, hope it helps!

# see https://yaml.org/spec/1.2/spec.html#id2805071
YAML_NULL = ["null", "Null", "NULL", "~"]

class Null(ScalarValidator):
    """
    'null'->None
    """
    @staticmethod
    def validate_scalar(chunk):
        """
        """
        if any([chunk.contents == n for n in YAML_NULL]):
            return None
        chunk.expecting_but_found(f"when expecting any of {YAML_NULL}")


class NotNullStr(Str):
    """
    """
    @staticmethod
    def validate_scalar(chunk):
        """
        """
        if any([chunk.contents == n for n in YAML_NULL]):
            chunk.expecting_but_found(
                f"when expecting a string different to these: {YAML_NULL}")
        return chunk.contents

Then the schema can look like this:

Map({"code": NotNullStr(),
     "name": Null() | Str(),
     ...

from strictyaml.

That's great thanks! can you link the commit here for reference upon release?
Also regarding docs, I'd emphasize the idea that NullNone has to come before Str to prevent getting "null" instead of None

from strictyaml.

Great progress! Looking forward to the release.

BTW naming: NullNone is a very explicit term but doesn't mirror NotNullStr very well. Can't we stick to Null? It's very clear and fits better the NotNullTypes in my opinion.

from strictyaml.

from strictyaml.

from strictyaml.

I got a yaml database of unknown version and want to infer the schema. The sensible thing to do for Str is to assume notnull if I can't find such examples. I don't want to risk parsing straw nulls as Str.

Many reasons for this: avoid redundancies, ensure operability... it is IMO basic to data curation and the main reason why you would use a schema in the first place

from strictyaml.

Also for the record I'm using this wrapper on CommaSeparated, Enum. For the same reasons as above I found it really useful to combine them and have a strong grip on the accepted options for a comma-separated list.

Not saying this should be added to the API, though. Also, if you know another open issue where this could apply I'll be happy to move the comment there.

Thanks again!

class TagList(CommaSeparated):
    """
    An alias for ``CommaSeparated(Enum([tag1, tag2, ...]))``
    """

    def __init__(self, *tags):
        """
        """
        assert all(isinstance(t, str) for t in tags), \
            f"{self.__class__.__name__} admits string tags only!"
        super().__init__(Enum(tags))

Usage: In the following example we expect the drinks field to contain either nothing, or a list of comma-separated elements, where each element can be only water, beer or juice.

Map({"code": NotNullStr(),
         "name": Null() | Str(),
         "drinks": TagList("water", "beer", "juice"),
     ...

from strictyaml.