I just thought to try a search for a tool where you give it a
valid JSON document, and it returns a schema for it. Google
lead me to:

https://www.liquid-technologies.com/online-json-to-schema-converter

which looks like a useful tool.

Bob and I noticed that label can be null according to the schema. This is incorrect according to
the documentation, and should be corrected. The release of 3.2 should proceed, but this should be resolved for 3.3.

The Autoplot script which combines the separate JSON files into one file for Bob's system should be converted to a Python script. Right now there is https://github.com/hapi-server/data-specification-schema/blob/jon-jeremy-mess-3.0/misc/combineJsonSchema.jy and this should be converted to https://github.com/hapi-server/data-specification-schema/blob/jon-jeremy-mess-3.0/misc/combineJsonSchema.py. Likewise the other direction should be done. This should be fairly trivial but I need to remind myself how to work with JSON in Python.

This is checked semantically, but could be in schema.

This ticket is to verify that the 3.0 schema is complete. Items from https://github.com/hapi-server/data-specification/blob/master/hapi-3.0.0/changelog.md should be checked below as they are verified.

• Addition of a new endpoint, "about", for server description metadata.
• Ability to specify time-varying bins (#83)
• Ability to use JSON references in info response (#82)
• Ability to indicate a units schema (if one is being used for units strings) (#81)
• Added citation element

See 3.2 test that demonstrates.

• add ids
• already move things to root
• drop Refs

Version 3.1 brought the new features:

• dataset and parameters may contain unicode
• identify vector coordinate system and components 115
• extra metadata with schema 117
• maximum time range 136

The 3.2 schema changes need to be identified and the schema needs to be written.

See hapi-server/verifier-nodejs#64

This is checked semantically (search for CadenceValid, but could be in schema.

Bob, Jon, and Jeremy met to discuss the validation using JSON schema. We resolved to use the existing code within the verifier to do this, though with some changes a standard website could be used. This might be done in the future, but it's not too difficult to use nodejs.

Bob will:

• separate the validator javascript code from the verifier and move it here.
• create an area for example catalog, about and info responses to test the verifier. (https://github.com/hapi-server/data-specification-schema/tree/main/examples)
• Add the JSON used for the TestDataX.Y servers to this repo? For example

Jeremy will:

• update the 3.0 and 3.1 schemas to be correct.
• write the 3.2 schema.

Jon will:

• create a bunch of examples to test against. (This should probably be coordinated with Bob's task.)

There should be a single standard for formatting JSON documents committed here. It should
be something we can all access, and something which could be run as a git commit hook.

I propose the linux command "json_pp" which:

• indents using 3-space indents
• alphabetizes tags in a json object
• each item of array has a separate line
• each tag of an object has a separate line
• each tag of an object has a space after the tag's name

Here is an example json_pp-formatted about schema:

{
   "$schema" : "http://json-schema.org/draft-07/schema#",
   "additionalProperties" : false,
   "definitions" : {
      "HAPI" : {
         "description" : "HAPI Version",
         "id" : "#/definitions/HAPI",
         "pattern" : "^3\\.1$",
         "type" : "string"
      },
      "HAPIStatus" : {
         "additionalProperties" : false,
         "description" : "Request status",
         "id" : "#/definitions/HAPIStatus",
         "patternProperties" : {
            "x_.*" : {}
         },
         "properties" : {
            "code" : {
               "description" : "HAPI request status code",
               "enum" : [
                  1200,
                  1201,
                  1400,
                  1401,
                  1402,
                  1403,
                  1404,
                  1405,
                  1406,
                  1407,
                  1408,
                  1409,
                  1410,
                  1411,
                  1412,
                  1500,
                  1501
               ],
               "type" : "integer"
            },
            "message" : {
               "description" : "HAPI request status message",
               "type" : "string"
            }
         },
         "required" : [
            "code",
            "message"
         ],
         "title" : "",
         "type" : "object"
      }
   },
   "description" : "Server /capabilities response",
   "patternProperties" : {
      "x_.*" : {}
   },
   "properties" : {
      "$schema" : {
         "description" : "schema declaration is sometimes found in response.",
         "type" : "string"
      },
      "HAPI" : {
         "$ref" : "#/definitions/HAPI"
      },
      "outputFormats" : {
         "additionalItems" : true,
         "description" : "Output formats, which must include csv.",
         "items" : {
            "enum" : [
               "csv",
               "binary",
               "json"
            ],
            "type" : "string"
         },
         "minItems" : 1,
         "type" : "array",
         "uniqueItems" : true
      },
      "status" : {
         "$ref" : "#/definitions/HAPIStatus"
      }
   },
   "required" : [
      "HAPI",
      "outputFormats",
      "status"
   ],
   "title" : "capabilities",
   "type" : "object"
}

We need to clarify in the documentation that if there is an error, the only required elements are HAPI and status. For example, the following should be valid

{
    "HAPI": "2.1",
    "status": {
        "code": 1406,
        "message": "HAPI error 1406: unknown id"
    }
}

For < 3.0, this was tested for against a subschema that was not a part of the info schema.

We need a process to combine separate into one file. Jon and Jeremy wrote separate schemas for about, capabilities, catalog, and info responses for the branch jon-jeremy-mess-3.0. This allowed the website jsonschemavalidator.net to be used to test responses directly, plus it is easier to work four small schemas than it is one. That said, we would also like to support Bob's format where they are all combined into one schema, not to mention that having four separate schemas requires redundant definitions for each single file.

So this process will take the four separate schemas and combine them into one, indicating where redundant parts have incorrect differences. It might also be useful to go the other way as well, since this is actually easier to code since there is no redunency to worry about.

ViRES uses this

Plus semantic check that structure of data (possibly nested arrays) is consistent with size.