nijikokun / generate-schema Goto Github PK

🧞 Convert JSON Objects to MySQL, JSON Schema, Mongoose, Google BigQuery, Swagger, and more.

License: MIT License

JavaScript 100.00%

generate-schema's Introduction

Generate Schemas

Convert JSON Objects to MySQL Table Schema, JSON Schema, Mongoose Schema, ClickHouse Schema, Google BigQuery, or a Generic template for documentation, code generation, and more.

Installation
CLI
- Options
- REPL Mode
  - Example
Usage
- Example
- Methods
License

Installation

Install with npm:

$ npm i --save generate-schema

Optionally, add -g to the above if you want the generate-schema command line executable.

CLI

  Usage: generate-schema [options ...] [file]

  Common Options:

    -h, --help         output usage information
    -V, --version      output the version number
    -q, --quiet        Skip help message in program output

  Mode Options:
    -g, --generic      Generic JSON Primitives schema output
    -j, --json-schema  JSON Schema output
    -s, --mysql        MySQL Table Schema output
    -m, --mongoose     Mongoose Schema output
    -b, --big-query    Google BigQuery Schema output
    -c, --clickhouse   Clickhouse Table Schema output

REPL Mode

When no file is specified, generate-schema enters a REPL mode.

Example

$ generate-schema -b
generate-schema v2.5.1 (bigquery)
Type "exit" to quit.
Type {a:"b"} to see an example.
> {a:"b"}
[
  {
    "name": "a",
    "type": "STRING",
    "mode": "NULLABLE"
  }
]

Usage

var GenerateSchema = require('generate-schema')

Example

// Capture Schema Output
var schema = GenerateSchema.json('Product', [
    {
        "id": 2,
        "name": "An ice sculpture",
        "price": 12.50,
        "tags": ["cold", "ice"],
        "dimensions": {
            "length": 7.0,
            "width": 12.0,
            "height": 9.5
        },
        "warehouseLocation": {
            "latitude": -78.75,
            "longitude": 20.4
        }
    },
    {
        "id": 3,
        "name": "A blue mouse",
        "price": 25.50,
        "dimensions": {
            "length": 3.1,
            "width": 1.0,
            "height": 1.0
        },
        "warehouseLocation": {
            "latitude": 54.4,
            "longitude": -32.7
        }
    }
])

Outputs:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "title": "Product Set",
  "type": "array",
  "items": {
    "type": "object",
    "properties": {
      "id": {
        "type": "number"
      },
      "name": {
        "type": "string"
      },
      "price": {
        "type": "number"
      },
      "tags": {
        "type": "array",
        "items": {
          "type": "string"
        }
      },
      "dimensions": {
        "type": "object",
        "properties": {
          "length": {
            "type": "number"
          },
          "width": {
            "type": "number"
          },
          "height": {
            "type": "number"
          }
        }
      },
      "warehouseLocation": {
        "type": "object",
        "properties": {
          "latitude": {
            "type": "number"
          },
          "longitude": {
            "type": "number"
          }
        }
      }
    },
    "required": [
      "id",
      "name",
      "price",
      "dimensions",
      "warehouseLocation"
    ],
    "title": "Product"
  }
}

Methods

`g.generic(Object object)`

Generates a generic schema from object. Property types are described using primitives.

`g.mysql([String tableName,] Mixed object)`

Generates MySQL Table Schema from object.

tableName is optional, defaults to generic
object must be of type Object or Array

`g.json([String title,] Mixed object)`

Generates JSON Schema from object.

title is optional
object must be of type Object or Array

`g.mongoose(Object object)`

Generates a Mongoose Schema from object.

`g.bigquery(Object object)`

Generates a Google BigQuery schema from object.

`g.clickhouse([String tableName,] Mixed object, String dateField)`

Generates ClickHouse Table Schema from object.

tableName is optional, defaults to generic
object must be of type Object or Array
dateField Date field for ENGINE, must be of type Date

License

MIT

generate-schema's People

Contributors

Stargazers

Watchers

Forkers

siandiaz javascript-forks hihihippp inno-v nvdnkpr fbaiodias szwork2013 githubcs apeschel jpstrikesback kekee000 jlertle jamshedmelik yulingchen dondre mcraigpw sax1johno poetic dddomin3 tzechienchu zearin phra winniebear timelyportfolio buchenberg nagarajasr happy-ferret digideskio manjeshpv ldodds hybrtools dashao598 ugly-maki solobat ritz078 transform-it shanawarkhan tryeasy lmangani thepechinator reactual karayakar ampboris lgc2009sx wangchuan3533 liumorgan zettacristiano amzani hetianzhuo ykankaya fossabot sunil-lulla pxeric nick121212 rbrisita anima-os-stash nilportugues iraghumitra dhenson02 faissalmt chriscalloway chauey undeadinu iyangce dhangerkapil muz3 erreur32 rjharmon pombredanne ngdnghia28 anthonywc sumonst21 shellphon fleksin mingsai chinaczm ccinelli trinity-insight sankar4n saratadiraj hpatel2018 tykoth dasalazar1 santoshsrinivas79 allenbird jiangshicunit chinaxiaoyy dmarkrollins leandrooliveiramarino lachee sxyy wittneyr kangofchen designtestcode gabrielsouzasys odnodn twtcer tolgadevsec ecaramba 1cadumagalhaes

generate-schema's Issues

Please tag a new release

This will make it possible to include the latest fixes in Homebrew, npm, and other package managers.

Selecting json schema version

I was wondering if it would be possible tu select the json-schema version for this tool?

I need draft-06 or above for my use case

usage of "JSON Schema" is confusing

JSON Schema is a set standard:
http://json-schema.org/
http://en.wikipedia.org/wiki/JSON#JSON_Schema

yet the output produced by this library is not a valid JSON Schema, so this should be clearly stated somewhere that this is not a generator for JSON Schema

Support for Sql Server and InnoDb

hello, can we get support for Sql Server and InnoDb

thanks

Help with previous issue

Hey, I opened this issue a while ago, you responded, but I didn't see it for a long time, and you closed the issue. Wondering if I could get help with it: #30

Thanks

Null keys break schema generation

Given the test JSON:

{
 "items": [
   { "name": "foo", "details": { "a": "b" } },
   { "name": "bar", "details": null }
 ]
 
}

running the command line tool produces:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "items": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string"
          },
          "details": {
            "type": "null"
          }
        },
        "required": [
          "name",
          "details"
        ]
      }
    }
  }
}

The details key has a type of null and no items. The null value for the second object in the items array in the example is overwriting the schema generated from the first.

So there is an ordering dependency which means that null values can result in lost schema information.

For a quick local fix I've put a conditional around these lines. If the type of the item is null and there's already a key for that property, then don't overwrite it.

    if (type === 'null' && output.properties[key]) {
      //console.log(output.properties[key]);
    } else {
        output.properties[key] = {}
        output.properties[key].type = type
    }

I'm not sure that's a complete fix though, or even recommended. Just gives me a quick workaround!

Save json data to default value in json schema

Hello,

When you generate a schema based on a json, is it possible to save the values of my json inside the default value in the json schema ?

convert json schema to mysql schema?

Can this app convert a json schema to mysql schema?

there is this opendatasoft ( a french open data system) that i want to clone but they only have a json schema available.

SQL to Prisma generator

It would be very nice to have a SQL to prisma generator.

For example the following SQL QUERY

SELECT * FROM Users WHERE city = 'Copenhagen'

Should be converted to the following prisma query

await prisma.users.findMany({
  where: {
      city: "Copenhagen"
  }
})

Creating a new Schema adds empty required array

I have a JSON instance like below:

{
  "a": [
    {
      "MyName": "Ron"
    },
    {
      "lastName": "Aron"
    }
  ]
}

When I generate the new schema of this JSON above it is created as below:

import jsonSchemaGenerator from 'generate-schema';
const mySchema = jsonSchemaGenerator.json("CustomLegend1", this.sectionSchemaInstance);
console.log(" ***** JSON Schema =  " + JSON.stringify(mySchema));

  {
 	"$schema": "http://json-schema.org/draft-04/schema#",
 	"title": "CustomLegend1",
 	"type": "object",
 	"properties": {
 		"a": {
 			"type": "array",
 			"items": {
 				"type": "object",
 				"properties": {
 					"MyName": {
 						"type": "string"
 					},
 					"lastName": {
 						"type": "string"
 					}
 				},
 				"required": []
 			}
 		}
 	}
 }

Why there are no required[] fields ("required": []) ?

"required" fields only take into account the last 2

Thanks for the project! it got me started on JSON schema generation for exercising some form of contract testing.

The required fields logic seem to only take into account the last 2 examples?

How to reproduce

By providing this JSON test data:

[
    {
        "name": "leo"
    },
    {
        "name": "mike",
        "age": "30"
    },
    {
        "name": "laura",
        "age": "33"
    }
]

Run

generate-schema --json-schema data/sample-teams.json

Got

Got name and age fields as required.

    "required": [
      "name",
      "age"
    ]

Expected

Expected only the name field as required.

    "required": [
      "name"
    ]

I guess to improve it, would need to take a look at

generate-schema/src/schemas/json.js

Line 41 in edacf92

// Value is optional, it doesn't exist in A but exists in B(n)

Nested array with incomplete objects leads to type being lost

If an array contains a empty or partial object, the type inferred is lost. Example:

require("generate-schema").json(
{array:[
    {"a2":[{something:""}]},
    {"a2":[]},
]})

The object inside a2 array should have the "something" field. But the result is now

{"a2":{"type":"array","items":{}}}

require("generate-schema").json(
{array:[
    {"a2":[{something:""}]}
]})

which correctly infers

{"a2":{"type":"array","items":{"type":"object","properties":{"something":{"type":"string"}}}}}

The current code handles non nested array nicely though.

require("generate-schema").json(
{array:[
    {"a2":[{something:""}]},
    {}

]})

generates

{"a2":{"type":"array","items":{"type":"object","properties":{"something":{"type":"string"}}}

ObjectId? Error

I've taken one object from a collection and tried to run this. Im guessing objectID is not handled?
I dont get any output in console.

var GenerateSchema = require('generate-schema');
var ObjectId = require('mongodb');

// Capture Schema Output
var schema = GenerateSchema.mongoose('Review', 
	{
		"_id": ObjectId("5883cb02b768a81f86fb467e"),
		"votes": {
			"funny": 0,
			"useful": 0,
			"cool": 0
		},
		"user_id": "KZuaJtFindQM9x2ZoMBxcQ",
		"review_id": "WExNE-f93SL4D1q8s9QWKg",
		"stars": 1,
		"date": "2013-07-14",
		"text": "This place is absolute garbage...  Half of the tees are not available, including all the grass tees.  It is cash only, and they sell the last bucket at 8, despite having lights.  And if you finish even a minute after 8, don't plan on getting a drink.  The vending machines are sold out (of course) and they sell drinks inside, but close the drawers at 8 on the dot.  There are weeds grown all over the place.  I noticed some sort of batting cage, but it looks like those are out of order as well.  Someone should buy this place and turn it into what it should be.",
		"type": "review",
		"business_id": "cE27W9VPgO88Qxe4ol6y_g"
	}	
);

console error:

klik@klik ~/P/G/json-schemas-yelp> node reviews.js                                           14:52:18
(node:14514) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error:
invalid schema, expected mongodb
(node:14514) DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise
rejections that are not handled will terminate the Node.js process with a non-zero exit code.

If I remove the objectID and try running in the cli i get Invalid Object or JSON see following:

> { "id": "5883cb02b768a81f86fb467e", "votes": { "funny": 0, "useful": 0, "cool": 0 }, "user_id": "KZuaJtFindQM9x2ZoMBxcQ", "review_id": "WExNE-f93SL4D1q8s9QWKg", "stars": 1, "date": "2013-07-14", "text": "This place is absolute garbage...  Half of the tees are not available, including all the grass tees.  It is cash only, and they sell the last bucket at 8, despite having lights.  And if you finish even a minute after 8, don't plan on getting a drink.  The vending machines are sold out (of course) and they sell drinks inside, but close the drawers at 8 on the dot.  There are weeds grown all over the place.  I noticed some sort of batting cage, but it looks like those are out of order as well.  Someone should buy this place and turn it into what it should be.", "type": "review", "business_id": "cE27W9VPgO88Qxe4ol6y_g" }
Invalid Object or JSON
>

I double checked the json. Its valid.

This works if I dont let the cli go to repl by specifying the file, by the way.

generate schema just displays data

Hi. Not sure if I'm running this correctly but I input a JSON data file and run generate-schema -g <file.json> but all it does is display the data file back to me. Am I missing something here?

Thanks
Michael

mongoose schema generator can't detect arrays

Hello,I was trying to convert the a json object using your generator and I noticed that it can't detect arrays.
You can try to recreate a problem on using the following JSON
{
"id": 2,
"name": "An ice sculpture",
"price": 12.50,
"tags": ["cold", "ice"],
"dimensions": {
"length": 7.0,
"width": 12.0,
"height": 9.5
},
"warehouseLocation": {
"latitude": -78.75,
"longitude": 20.4
}
}
as for me I had the following output
{
id: 'Number',
name: 'Number',
price: 'Number',
tags: 'Number',
dimensions: 'Number',
warehouseLocation: 'Number'
}

NB: On posting this issue I also detected it didn't also detect name as it typed it as Number instead of String.
Great work by the way!!

Output valid JSON Schema

Migrating topic discussion from #2

Create a method for generating json schema output.

http://json-schema.org/
http://en.wikipedia.org/wiki/JSON#JSON_Schema
Update repl to support both generic, and json schema

mongoose schema generator throws on null value (patch)

Hi Nijiko,

Thanks for this useful module.

I was trying it on a rather big json output from a web API, but it failed when hitting null values.

Here's a sample output

> data
{ a: 1, b: 2, x: null }
> GenerateSchema.json(data)
{ '$schema': 'http://json-schema.org/draft-04/schema#',
  type: 'object',
  properties: 
   { a: { type: 'number' },
     b: { type: 'number' },
     x: { type: 'null' } } }
> GenerateSchema.mongoose(data)
TypeError: Cannot read property 'toString' of null
    at Object.Process [as mongoose] (/home/mich/.nvm/versions/node/v7.4.0/lib/node_modules/generate-schema/src/schemas/mongoose.js:49:21)
    at repl:1:16
    at realRunInThisContextScript (vm.js:22:35)
    at sigintHandlersWrap (vm.js:98:12)
    at ContextifyScript.Script.runInThisContext (vm.js:24:12)
    at REPLServer.defaultEval (repl.js:346:29)
    at bound (domain.js:280:14)
    at REPLServer.runBound [as eval] (domain.js:293:12)
    at REPLServer.onLine (repl.js:544:10)
    at emitOne (events.js:101:20)

I'm just starting with mongoose and json schemas so I don't know yet what should be the correct behavior, but I can suggest this simple patch to fix the issue (though a bit of refactoring may be needed):

--- src/schemas/mongoose.js	2017-01-08 16:47:22.301455645 +0100
+++ /home/mich/.nvm/versions/node/v7.4.0/lib/node_modules/generate-schema/src/schemas/mongoose.js	2017-01-08 17:56:16.659882008 +0100
@@ -46,7 +46,7 @@ module.exports = function Process (objec
       type = 'buffer'
     }
 
-    if (typeof value.toString !== 'undefined' && value.toString().match(/^[0-9a-fA-F]{24}$/)) {
+    if (value!=null && typeof value.toString !== 'undefined' && value.toString().match(/^[0-9a-fA-F]{24}$/)) {
       type = 'objectid'
     }

Here's how it goes once the change applied:

 > var GenerateSchema = require('generate-schema');
undefined
> data = { a: 1, b: 2, x: null }
{ a: 1, b: 2, x: null }
> GenerateSchema.mongoose(data)
{ a: 'Number', b: 'Number', x: 'Mixed' }

Compatibility with Ajv ? Geojson

Context : Validate input data with Ajv

Request.Body

{
   "type":"FeatureCollection",
   "features":[
      {
         "type":"Feature",
         "geometry":{
            "type":"Point",
            "coordinates":[
               102,
               0.5
            ]
         },
         "properties":{
            "prop0":"value0"
         }
      },
      {
         "type":"Feature",
         "geometry":{
            "type":"LineString",
            "coordinates":[
               [
                  102,
                  0
               ],
               [
                  103,
                  1
               ],
               [
                  104,
                  0
               ],
               [
                  105,
                  1
               ]
            ]
         },
         "properties":{
            "prop0":"value0",
            "prop1":0
         }
      },
      {
         "type":"Feature",
         "geometry":{
            "type":"Polygon",
            "coordinates":[
               [
                  [
                     100,
                     0
                  ],
                  [
                     101,
                     0
                  ],
                  [
                     101,
                     1
                  ],
                  [
                     100,
                     1
                  ],
                  [
                     100,
                     0
                  ]
               ]
            ]
         },
         "properties":{
            "prop0":"value0",
            "prop1":{
               "this":"that"
            }
         }
      }
   ]
}```

**Schema generated from the request.body** 
```json
{
   "$id":"http://json-schema.org/draft-04/schema#",
   "$schema":"http://json-schema.org/draft-04/schema#",
   "title":"Product",
   "type":"object",
   "properties":{
      "type":{
         "type":"string"
      },
      "features":{
         "type":"array",
         "items":{
            "type":"object",
            "properties":{
               "type":{
                  "type":"string"
               },
               "geometry":{
                  "type":"object",
                  "properties":{
                     "type":{
                        "type":"string"
                     },
                     "coordinates":{
                        "type":"array",
                        "items":{
                           "oneOf":[
                              {
                                 "type":"number"
                              },
                              {
                                 "type":"number"
                              },
                              {
                                 "type":"number"
                              },
                              {
                                 "type":"number"
                              },
                              {
                                 "type":"number"
                              }
                           ],
                           "type":"array"
                        }
                     }
                  }
               },
               "properties":{
                  "type":"object",
                  "properties":{
                     "prop0":{
                        "type":"string"
                     },
                     "prop1":{
                        "type":"object",
                        "properties":{
                           "this":{
                              "type":"string"
                           }
                        }
                     }
                  }
               }
            },
            "required":[
               "type",
               "geometry",
               "properties"
            ]
         }
      }
   }
}

Schema validation with Ajv

[
  {
    keyword: 'type',
    dataPath: '.features[0].geometry.coordinates[0]',
    schemaPath: '#/properties/features/items/properties/geometry/properties/coordinates/items/type',
    params: { type: 'array' },
    message: 'should be array'
  }
]

Why Ajv detects an issue ?

Thx

Null type in mysql throws error. Possible solution.

The following json (partial from the github api) :

{
  "message": "Out-of-srcdir builds: AM_CPPFLAGS=-I${srcdir}/src",
  "tree": {
    "sha": "5b9f0e1342264d1465fa7001faec3902c89ca944",
    "url": "https://api.github.com/repos/stedolan/jq/git/trees/5b9f0e1342264d1465fa7001faec3902c89ca944"
  },
  "url": "https://api.github.com/repos/stedolan/jq/git/commits/0bc77083ba711e1961db1cb2ebc9de7c71d1a746",
  "comment_count": 0,
  "verification": {
    "verified": false,
    "reason": "unsigned",
    "signature": null,
    "payload": null
  }
}

throws the following error when converting to mysql :

src/utils.js:12 Uncaught (in promise) TypeError: Cannot read property 'length' of null
    at Object.exports.isTimestamp (src/utils.js:12)
    at processObject (schemas/mysql.js:93)
    at processObject (schemas/mysql.js:103)
    at processObject (schemas/mysql.js:103)
    at processObject (schemas/mysql.js:103)
    at Object.Process [as mysql] (schemas/mysql.js:162)

as Utils.isTimestamp() doesn't like nulls instead of strings. Changing line 92 of mysql.js from :

       if (type !== 'undefined') {

if (type !== 'undefined' && type !== 'null') {

seems to fix this, and the type for null fields like this (signature and payload in the example json above) becomes string, which in my case is the correct handling.

I haven't submitted a pull request as I'm not sure that this is the correct handling in all cases / the generic case, and I don't know enough about the code to know if it will cause any other unintended side effects.

Thanks for a great library.

Enhacements For JSON Schema

This is in regard to JSON Schema file generation.

More Flexible Command Line

It would be great if the draft schema could be set for the generated file.

Testing integers and numbers

It seems that numeric data is always "number", which is used to denote floats. JSON Schema also has integers and would help in data verification.

converting to mongoose gives odd results

try this JSON:

{"address_components":[{"long_name":"Cologne","short_name":"Cologne","types":["locality","political"]},{"long_name":"Cologne","short_name":"K","types":["administrative_area_level_2","political"]},{"long_name":"North Rhine-Westphalia","short_name":"NRW","types":["administrative_area_level_1","political"]},{"long_name":"Germany","short_name":"DE","types":["country","political"]}],"formatted_address":"Cologne, Germany","geometry":{"bounds":{"northeast":{"lat":51.08496299999999,"lng":7.1620628},"southwest":{"lat":50.8295269,"lng":6.7725819}},"location":{"lat":50.937531,"lng":6.9602786},"location_type":"APPROXIMATE","viewport":{"northeast":{"lat":51.08496299999999,"lng":7.1620628},"southwest":{"lat":50.8295269,"lng":6.7725819}}},"place_id":"ChIJ5S-raZElv0cR8HcqSvxgJwQ","types":["locality","political"]}

converts to

{
  "address_components": [
    "Mixed"
  ],
  "formatted_address": {
    "0": "String",
    "1": "String",
    "2": "String",
    "3": "String",
    "4": "String",
    "5": "String",
    "6": "String",
    "7": "String",
    "8": "String",
    "9": "String",
    "10": "String",
    "11": "String",
    "12": "String",
    "13": "String",
    "14": "String",
    "15": "String"
  },
  "geometry": {
    "bounds": {
      "northeast": {
        "lat": "Number",
        "lng": "Number"
      },
      "southwest": {
        "lat": "Number",
        "lng": "Number"
      }
    },
    "location": {
      "lat": "Number",
      "lng": "Number"
    },
    "location_type": {
      "0": "String",
      "1": "String",
      "2": "String",
      "3": "String",
      "4": "String",
      "5": "String",
      "6": "String",
      "7": "String",
      "8": "String",
      "9": "String",
      "10": "String"
    },
    "viewport": {
      "northeast": {
        "lat": "Number",
        "lng": "Number"
      },
      "southwest": {
        "lat": "Number",
        "lng": "Number"
      }
    }
  },
  "place_id": {
    "0": "String",
    "1": "String",
    "2": "String",
    "3": "String",
    "4": "Date",
    "5": "Date",
    "6": "Date",
    "7": "Date",
    "8": "Date",
    "9": "Date",
    "10": "Date",
    "11": "Date",
    "12": "Date",
    "13": "Date",
    "14": "Date",
    "15": "Date",
    "16": "Date",
    "17": "Date",
    "18": "Date",
    "19": "Date",
    "20": "Date",
    "21": "Date",
    "22": "Date",
    "23": "Date",
    "24": "Date",
    "25": "Date",
    "26": "Date"
  },
  "types": {
    "0": "String",
    "1": "String"
  }
}

[Issue] Nested object under an array returns wrong number of items

Basic Object

const sampleObj = {
  randomnumber: 2,
  arr: [
    1, 
    2,
    "hello"
  ]
};

console.log(JSON.stringify(GenerateSchema.json(sampleObj), null, 2));

Above returns wrong validation for the array, I have 3 items in the array and it returns 4... am I missing something obvious here?

Output of the above:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "randomnumber": {
      "type": "number"
    },
    "arr": {
      "type": "array",
      "items": {
        "oneOf": [
          {
            "type": "number"
          },
          {
            "type": "number"
          },
          {
            "type": "number"
          },
          {
            "type": "string"
          }
        ]
      }
    }
  }
}

Thanks

[question] can I convert between schemas?

Hi! I would like to know if it is possible to convert from one schema to another with this project.
if I already have json schema and I want to convert it to mongo or mysql... is it possible with this project?

DATE inference for BigQuery is incorrect

Given an object such as

{
  dateField: "2019-03-04 12:13:14"
}

A BigQuery schema is generated that considers that a DATE. If you then attempt to insert into the table with that same object, the insert fails because BigQuery doesn't parse the field as a date

Required field not properly set

Required is wrongly set for arrays of items

> var generator = require('generate-schema')
> generator.json([{a: 1, b: 2}, {a: 2, b: 3}, {b: 5}])
{
  '$schema': 'http://json-schema.org/draft-04/schema#',
  type: 'array',
  items: {
    type: 'object',
    properties: { a: [Object], b: [Object] },
    required: [ 'a', 'b' ]
  }
}

a cannot be required since it is not in the last item.
It might be related to #32

Thanks

running from cli, not outputting schema

I have a dir where I have my json files - I did the following steps:

npm install generate-schema
ln -s node_modules/generate-schema/bin/generate-schema
./generate-schema post.json

it outputs the json of the file but does not output a schema or any error message. I would expect that 1) it wouldn't print out the original json file and 2) would output the schema to stout so I could capture it. Or at least print an error message if there was error.

I'm using node 0.12

Big Query schema is wrong

Expected:

generate-schema v2.5.0 (bigquery)
Type "exit" to quit.
Type {a:"b"} to see an example.
> {a:"b"}
[
  {
    "name": "a",
    "type": "STRING",
    "mode": "NULLABLE"
  }
]

But got:

generate-schema v2.5.0 (bigquery)
Type "exit" to quit.
Type {a:"b"} to see an example.
> {a:"b"}
[
  {
    "name": "0",
    "type": "RECORD",
    "mode": "NULLABLE",
    "fields": [
      {
        "name": "name",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "type",
        "type": "STRING",
        "mode": "NULLABLE"
      },
      {
        "name": "mode",
        "type": "STRING",
        "mode": "NULLABLE"
      }
    ]
  }
]

how to set required key as optional for some key in main json

Dear contributors,

once to generate schema some fields are optional it means if they exist the schema will validate the type & etc and if it is not exist it just ignore in validation phase, now i have a json to create the schema, Is there any way to made some JSON fields as optional, i mean it does not need to put in required field list ?

[question] josnschema to mysql

just a question, this repo can convert a json schema to mysql? or only from object to any of them?

mongoose mode doesn't descend into array definitions

Mongoose mode doesn't descend into arrays. An object within an array of will have type mixed, unlike in the default JSON mode which will infer the type of the object within the array.
Is this intentional?

Repro:
Using example.json:

{
  "array":[{
    "value1": "a",
    "value2": "b"
    }]
}

With the default mode (json):

⋊> ~/m/p/so-app generate-schema example.json                                                                                                  
{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "array": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "value1": {
            "type": "string"
          },
          "value2": {
            "type": "string"
          }
        }
      }
    }
  }
}

With mongoose-mode:

⋊> ~/m/p/so-app generate-schema -m example.json                                                                                               
{
  "array": {
    "type": [
      "Mixed"
    ]
  }
}

I noticed this while idly investigating the asker's problem in this old StackOverflow question: https://stackoverflow.com/questions/39362065/how-to-create-a-mongoose-schema-from-json.

motd should not be displayed when input is piped from another command

i am trying to pipe a JSON string through generate-schema cli, like this:
echo '{a: "b"}' | generate-schema
the output includes the 'motd' welcome and usage message which makes it inconvenient to pipe the output to another command. i would like to do this:
echo '{a: "b"}' | generate-schema | do_something_useful_with_schema

could you please add a cli option (say, '-q' for quiet) so that generate-schema can be used seamlessly in a pipeline?

thanks

SQLite support

I had a quick look at adding SQLite support, based on MySQL method, and it turns out MySQL method returns valid SQLite syntax already. Perhaps it would be worth exporting sqlite as an alias of mysql method, so in the future if they diverge they could be separated..?

Bugs? + Additional documentation for mysql.

I want to use the mysql function to generate SQL schema from JSON data.
However, I can not find any documentation and executing an example did not provide an intuitive result.

I would like to have one JSON file to generate multiple mysql table schema.
Just basic schema would be enough(primary id, boolean, date, etc.).

Please could you provide simple documentation on possible inputs, outputs and how to achieve what I have stated above.
Regards.

The information I did find:
#25

*One thing I did notice was that the output of the command below is wrapped in double quotes and \n characters are displayed. Also, it is not clear if a table name parameter can be passed.
generate-schema --mysql data.json > output.sql

Change api to allow better extensibility

Current: g.json(String name, Object model)

Proposal: g(Object model).json(String name)

Creating objects from schema

So I want to offer my users the ability to upload CSV and from that generate a mongoose schema, that i store in the DB against that user. When the user logs in, they can create a collection according to their personal schema. Using generate-schema I am able to create a json schema which looks like:

{
    "_id" : ObjectId("596a872cd1e59c6135fa7b2e"),
    "title" : "Product Set",
    "type" : "array",
    "items" : {
        "type" : "object",
        "properties" : {
            "booktitle" : {
                "type" : "string"
            },
            "bookid" : {
                "type" : "string"
            },
            "bookauthor" : {
                "type" : "string"
            }
        },
        "required" : [ 
            "booktitle", 
            "bookid", 
            "bookauthor"
        ],
        "title" : "Product"
    },
    "＄schema" : "http://json-schema.org/draft-04/schema#"
}

and store that in my schema collection. All good...

When I want to create a collection according to that schema, and store data in it using mongoose, I have tried to retrieve the schema from the database (which works) and then do

var generatedSchema = GenerateSchema.mongoose(response)

I then create a model from that with:

var Model = db.models.Product || db.model('Product', generatedSchema);

and create an item from that model

        var item = new Model({
            "_id": new ObjectID(),
          booktitle: 'The Godfather',
          bookid: 'abc123',
          bookauthor: 'Mario Puzo' 
        });

and save it:

item.save(function(err, response) { ... })

I don't get any errors but when I save it, in the collection I just see:

{
    "_id" : ObjectId("5970b1a584d396d7a2241eba"),
    "items" : {
        "required" : []
    },
    "__v" : 0
}

Can you point me in the right direction as to why this isn't working? My suspicion is I am using the wrong type of schema to create the model.

Thanks

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be one of type string, Buffer, or URL. Received type object

generate-schema-2.6.0 is failing tests on macOS as part of Homebrew with node 11.3.0. The error is the following:

18:34:53 ==> /usr/local/Cellar/generate-json-schema/2.6.0/bin/generate-schema
18:34:53 internal/fs/utils.js:454
18:34:53     throw err;
18:34:53     ^
18:34:53 
18:34:53 TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be one of type string, Buffer, or URL. Received type object
18:34:53     at Object.open (fs.js:416:3)
18:34:53     at ReadStream.open (internal/fs/streams.js:125:6)
18:34:53     at new ReadStream (internal/fs/streams.js:114:10)
18:34:53     at Object.createReadStream (fs.js:1759:10)
18:34:53     at Object.<anonymous> (/usr/local/Cellar/generate-json-schema/2.6.0/libexec/lib/node_modules/generate-schema/bin/generate-schema:110:15)
18:34:53     at Module._compile (internal/modules/cjs/loader.js:722:30)
18:34:53     at Object.Module._extensions..js (internal/modules/cjs/loader.js:733:10)
18:34:53     at Module.load (internal/modules/cjs/loader.js:620:32)
18:34:53     at tryModuleLoad (internal/modules/cjs/loader.js:560:12)
18:34:53     at Function.Module._load (internal/modules/cjs/loader.js:552:3)
18:34:53 Error: generate-json-schema: failed
18:34:53 An exception occurred within a child process:
18:34:53   Test::Unit::AssertionFailedError: <0> expected but was
18:34:53 <1>.

Does it calculate probability?

Thanks for the great work. I wonder if it's possible to provide multiple JSON docs, and it will conclude the field types based on all of them. For example, if the field "zip" contains number for most document but one of them has a value with string "4334-23" then the inferred type will be string and not number

Child objects without conversion

//code
var GenerateSchema = require('generate-schema')

var schema = GenerateSchema.json('Product', [
{
  "test": [
    {
      "testchild1": "s",
      "child2": [
        1
      ]
    }
  ],
  "c": {
    "a": [
      "as.wYYYYY",
      2,
      {
        "s": "s"
      }
    ]
  }
}
])

console.log(JSON.stringify(schema,null,2))
//result

              "oneOf": [
                {
                  "type": "string"
                },
                {
                  "type": "number"
                },
                {
                  "s": {
                    "type": "string"
                  }
                }
              ]
            }
          }
        }
      }
    }
"s" is a object but do not have properties and type

Update node version on travis.yml

The travis config file, is dated.
The node version 0.12 on travis.yml, not run the tests.
I believe the version 10 only is sufficient

Incorrect Schema generation for Objects embedded in array inside array for version 2.6.0

PFB the following snippet of json

  "Persons": [
    [
      {
        "Person": {
          "Confidence": 52,
          "Id": "XXXXY",
          "Urls": [],
          "Face": {
            "Confidence": 99.9940185546875,
            "Quality": {
              "Brightness": 69.32221984863281,
              "Sharpness": 46.177310943603516
            },
            "BoundingBox": {
              "Left": 0.4975961744785309,
              "Top": 0.25,
              "Height": 0.2094016969203949,
              "Width": 0.11778845638036728
            },
            "Pose": {
              "Pitch": -4.6145734786987305,
              "Roll": -5.172172546386719,
              "Yaw": 24.621706008911133
            },
            "Landmarks": [
              {
                "Type": "eyeLeft",
                "X": 0.5388702750205994,
                "Y": 0.32651984691619873
              },
              {
                "Type": "eyeRight",
                "X": 0.5758471488952637,
                "Y": 0.3228190243244171
              },
              {
                "Type": "nose",
                "X": 0.5691379308700562,
                "Y": 0.3715713620185852
              },
              {
                "Type": "mouthLeft",
                "X": 0.5409747958183289,
                "Y": 0.4054807424545288
              },
              {
                "Type": "mouthRight",
                "X": 0.5757172703742981,
                "Y": 0.39944666624069214
              }
            ]
          },
          "Name": "John Doe"
        },
        "Timestamp": 4000
      },
      {
        "Person": {
          "Confidence": 50,
          "Id": "XXXXY",
          "Urls": [],
          "Face": {
            "Confidence": 99.99772644042969,
            "Quality": {
              "Brightness": 78.21239471435547,
              "Sharpness": 32.335453033447266
            },
            "BoundingBox": {
              "Left": 0.4663461744785309,
              "Top": 0.279914528131485,
              "Height": 0.21581196784973145,
              "Width": 0.12139423191547394
            },
            "Pose": {
              "Pitch": -1.8032464981079102,
              "Roll": -3.601317882537842,
              "Yaw": 18.598796844482422
            },
            "Landmarks": [
              {
                "Type": "eyeLeft",
                "X": 0.5119572877883911,
                "Y": 0.35759350657463074
              },
              {
                "Type": "eyeRight",
                "X": 0.5483719110488892,
                "Y": 0.355496883392334
              },
              {
                "Type": "nose",
                "X": 0.5384360551834106,
                "Y": 0.40178537368774414
              },
              {
                "Type": "mouthLeft",
                "X": 0.5115289092063904,
                "Y": 0.43868288397789
              },
              {
                "Type": "mouthRight",
                "X": 0.5490919351577759,
                "Y": 0.4344561994075775
              }
            ]
          },
          "Name": "John Doe"
        },
        "Timestamp": 4480
      }
    ]
  ],

generates schema as below

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "Persons": {
      "type": "array",
      "items": {
        "type": "array"
      }
    }
}

Please advise.

AdditionalProperties are missing and how to add enum properties?

I am trying to create json schema for an array of object. I need to include enum property which contains some enum values used for array creation. And need to add " 'additionalProperties':false " also in generated schema.

Nested Object Field doesn't have "required" property

Hello,

I am trying to generate a schema using this JSON below:

[{"customField_A":"hello"},{"customField_B":"hello"},{"customField_C":"hello"},{"hello":{"hi":"testing"}}]

I want to set property "hi" as required for "hello" object, but it doesn't get set.

Can anyone provide assistance?

Thanks in advance!

Add a command line tool that reads JSON from files or stdin

Would be great to have that included!

If you don't have time to do it, I may be able to and could send a pull request.

What is current stable version?

I'm not sure what's considered the current stable version for this software. Is it the 2.2.1 tag here on GitHub or the 2.1.1 on registry.npmjs.org or something else?

How to use the module in the browser?

Hi, can you show how to include all the code in one js file to be used in the browser? Thanks for your help.

Integrated in transform

Hi,

Thank you for this awesome library. Just wanted to tell you that I have integrated this functionality in https://transform.now.sh

BigQuery: Look at more objects of an array for type inference

When generating a schema for BigQuery, only the first member of the array is looked at for schema generation. If this object contains optional fields, looking at one member may not be sufficient.

If you generate a schema from the following object, and then try to insert that object with the options {ignoreUnknownValues: false}, it fails.

{
  arrayField: [ { snafu: true }, { snafu: true, foobar: true } 
}

generateSchema.json does not create 'required' field on objects or array of 1

I'm not sure if there is an option switch on this. I can't duplicate the example in the readme that shows the 'required' property on the schema.
I guess it requires multiple items in an array to find what is required. But shouldn't the smart default be that for objects or arrays of 1, that the keys all be required?

currently this:

 var schema = GenerateSchema.json('Product', { b:1 })

outputs:


schema {
  "$schema": "http://json-schema.org/draft-04/schema#",
  "title": "Product Set",
  "type": "array",
  "items": {
    "type": "object",
    "properties": {
      "b": {
        "type": "number"
      }
    },
    "title": "Product"
  }
}

I'd expect to see required: [ 'a' ]

Any thoughts on making that change?

requirement section missing

After generating a JSON, there are no required keys.

...
"required": [
"id",
"name",
"price",
"dimensions",
"warehouseLocation"
],
"title": "Product"
}
}

is missing

nijikokun / generate-schema Goto Github PK

generate-schema's Introduction

Generate Schemas

Table of Contents

Installation

CLI

REPL Mode

Example

Usage

Example

Methods

g.generic(Object object)

g.mysql([String tableName,] Mixed object)

g.json([String title,] Mixed object)

g.mongoose(Object object)

g.bigquery(Object object)

g.clickhouse([String tableName,] Mixed object, String dateField)

License

generate-schema's People

Contributors

Stargazers

Watchers

Forkers

generate-schema's Issues

How to reproduce

By providing this JSON test data:

Run

Got

Expected

More Flexible Command Line

Testing integers and numbers

Recommend Projects

Recommend Topics

Recommend Org

`g.generic(Object object)`

`g.mysql([String tableName,] Mixed object)`

`g.json([String title,] Mixed object)`

`g.mongoose(Object object)`

`g.bigquery(Object object)`

`g.clickhouse([String tableName,] Mixed object, String dateField)`