Git Product home page Git Product logo

json-typedef-codegen's Introduction

jtd-codegen: Generate code from JSON Typedef schemas

JSON Type Definition, aka RFC8927, is an easy-to-learn, standardized way to define a schema for JSON data. You can use JSON Typedef to portably validate data across programming languages, create dummy data, generate code, and more.

jtd-codegen is a CLI tool that generates code bindings in many different programming languages from JSON Typedef schemas. For example, from this JSON Typedef schema:

{
    "properties": {
        "name": { "type": "string" },
        "isAdmin": { "type": "boolean" },
        "favoriteNumbers": { "elements": { "type": "float64" }}
    }
}

You can generate this TypeScript interface:

export interface User {
    favoriteNumbers: number[];
    isAdmin: boolean;
    name: string;
}

Or this Go type:

package user

type User struct {
    FavoriteNumbers []float64 `json:"favoriteNumbers"`
    IsAdmin         bool      `json:"isAdmin"`
    Name            string    `json:"name"`
}

Or types/classes/structs for any of the following programming languages:

  • C# with System.Text.Json as the JSON backend
  • Golang
  • Java with Jackson as the JSON backend
  • Python
  • Rust
  • TypeScript

With many more on the way. If you'd like a particular programming language included, please open an issue on this repo!

What is JSON Type Definition?

JSON Type Definition is a schema format for JSON data. A JSON Type Definition schema describes what is and isn't a "valid" JSON document. JSON Type Definition is easy to learn, portable (there are functionally-identical implementations across many programming languages) and standardized (the spec is set in stone as IETF RFC 8927).

Here's an example of a JSON Type Definition schema:

{
    "properties": {
        "name": {
            "type": "string"
        },
        "isAdmin": {
            "type": "boolean"
        }
    }
}

This schema considers any object with a name property (whose value must be a string), an isAdmin property (whose value must a boolean), and no other properties, to be valid.

To learn more about JSON Type Definition, check out the online documentation at jsontypedef.com.

Installation

Go to the latest jtd-codegen release on GitHub, and then install the file for your platform.

Usage

To use jtd-codegen, you first need to have a JSON Typedef schema to generate data from. Let's say you have this example data already in place:

$ cat user.jtd.json
{
    "properties": {
        "name": { "type": "string" },
        "isAdmin": { "type": "boolean" },
        "favoriteNumbers": { "elements": { "type": "float64" }}
    }
}

Then you can invoke jtd-codegen as:

# make sure you've already created the "user" directory before running this
$ jtd-codegen user.jtd.json --typescript-out user
๐Ÿ“ Writing TypeScript code to: user
๐Ÿ“ฆ Generated TypeScript code.
๐Ÿ“ฆ     Root schema converted into type: User

In that example, we generated TypeScript code. If you want to generate something else, use the appropriate "out" parameter for your desired language. For specific instructions for each programming languages, check out the documentation for:

  • C# with System.Text.Json as the JSON backend
  • Golang
  • Java with Jackson as the JSON backend
  • Python
  • TypeScript

You can produce code for multiple programming languages at once. Just pass all of the relevant parameters in the jtd-codegen invocation. For example:

$ jtd-codegen user.jtd.json --typescript-out ts-user --python-out py-user
๐Ÿ“ Writing Python code to: py-user
๐Ÿ“ฆ Generated Python code.
๐Ÿ“ฆ     Root schema converted into type: User
๐Ÿ“ Writing TypeScript code to: ts-user
๐Ÿ“ฆ Generated TypeScript code.
๐Ÿ“ฆ     Root schema converted into type: User

Advanced Usage: Adding descriptions to generated code

If you'd like to add a commented description to generated code -- for example, JavaDocs for Java code, or a docstring for Python -- jtd-codegen supports those via the description and enumDescription fields in any schema's metadata.

For example, this schema:

{
    "properties": {
        "name": {
            "metadata": {
                "description": "The user's name"
            },
            "type": "string"
        },
        "status": {
            "metadata": {
                "description": "The user's account status",
                "enumDescription": {
                    "UNVERIFIED": "The user's email has not yet been verified",
                    "VERIFIED": "The user's email has been verified",
                    "DISABLED": "The user's account was terminated"
                }
            },
            "enum": ["UNVERIFIED", "VERIFIED", "DISABLED"]
        }
    }
}

Generates into this TypeScript:

/**
 * The user's account status
 */
export enum UserStatus {
    /**
     * The user's account was terminated
     */
	Disabled = "DISABLED",

    /**
     * The user's email has not yet been verified
     */
	Unverified = "UNVERIFIED",

    /**
     * The user's email has been verified
     */
	Verified = "VERIFIED",
}

export interface User {
    /**
     * The user's name
     */
    name: string;

    /**
     * The user's account status
     */
    status: UserStatus;
}

Advanced Usage: Customizing jtd-codegen output

If you'd like to force jtd-codegen to use a particular type/class for some subset of your schema, you can use a "type override" property to do this. For example, if you generate TypeScript from this schema:

{
    "properties": {
        "name": {
            "metadata": {
                "typescriptType": "MyCustomNameType"
            },
            "type": "string"
        },
        "isAdmin": { "type": "boolean" },
        "favoriteNumbers": { "elements": { "type": "float64" }}
    }
}

You'll get:

export interface User {
    favoriteNumbers: number[];
    isAdmin: boolean;
    name: MyCustomNameType;
}

Each language supported by jtd-codegen supports a different set of overrides:

  • C# with System.Text.Json as the JSON backend
    • csharpSystemTextType overrides the entire outputted type
    • csharpSystemTextContainer overrides IList<T> or IDictionary<string, T> in favor of a different container type
  • Golang
    • goType overrides the entire outputted type
  • Java with Jackson as the JSON backend
    • javaJacksonType overrides the entire outputted type
    • javaJacksonContainer overrides List<T> or Map<String, T> in favor of a different container type
  • Python
    • pythonType overrides the entire outputted type
  • Rust
    • rustType overrides the entire outputted type
  • TypeScript
    • typescriptType overrides the entire outputted type

Advanced Usage: Using jtd-codegen in a larger build process

If you're using jtd-codegen as part of a larger build process (for example: if you're building an OpenAPI-like format on top of JSON Typedef), you may find it useful to programmatically get back the names of jtd-codegen-generated types. jtd-codegen supports this use-case via the --log-format CLI option.

By default, jtd-codegen uses --log-format pretty, which outputs human-friendly text to stdout. This is an example of pretty output:

๐Ÿ“ Writing TypeScript code to: user
๐Ÿ“ฆ Generated TypeScript code.
๐Ÿ“ฆ     Root schema converted into type: User
๐Ÿ“ฆ     Definition "name" converted into type: Name

If instead you use --log-format minimal, then jtd-codegen outputs startup diagnostic information to stderr, and information about generated data structures to stdout:

TypeScript: writing to: user
TypeScript: root: User
TypeScript: definition: name: Name

(The first line above is to stderr, the subsequent two lines are to stdout.)

Finally, --log-format json outputs information about generated data structures to stdout as JSON. No startup information is produced:

{
  "TypeScript": {
    "out_dir": "user",
    "root_name": "User",
    "definition_names": {
      "name": "Name"
    }
  }
}

Typically speaking, --log-format minimal is easier to process in simple bash scripts. --log-format json is often easier to use from anything that's not a shell-like programming language.

json-typedef-codegen's People

Contributors

ucarion avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

json-typedef-codegen's Issues

Supports optional properties in C# < 8.0?

I have a project where I'm generating types for C# and Typescript. In C# I'm stuck on .NET Framework, thus on C# < 8.0. I'd like to be able to use optional properties (for Typescript), but they are turned into nullable reference types in C#, which are only supported in 8.0.

Here's an example. I have this json, which includes an optional reference property.

{
  "definitions": {
    "jobState": {
      "properties": {
        "jobId": {
          "type": "string"
        },
      }
    },
  },
  "properties": {
    "isReady": {
      "type": "boolean"
    }
  },
  "optionalProperties": {
    "jobState": {
      "metadata": { "description": "Current job." },
      "ref": "jobState"
    }
  }
}

It generates this Typescript, which is great (note the question mark after jobState):

export interface Status {
    isReady: boolean;

    /**
     * Current job.
     */
    jobState?: JobState;
}

export interface JobState {
    jobId: string;
}

and this C#, which is only compilable in 8.0. In 7.3 I get this error:

CS8370: Feature 'nullable reference types' is not available in C# 7.3. Please use language version 8.0 or greater.

using System.Text.Json.Serialization;

namespace MyProject
{
    public class Status
    {
        [JsonPropertyName("isReady")]
        public bool IsReady { get; set; }

        /// <summary>
        /// Current job.
        /// </summary>
        [JsonPropertyName("jobState")]
        [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
        public JobState? JobState { get; set; }
    }
}

Can there be a toggle to support C# < 8.0? Thanks!

TypeScript output doesn't put quotes around snake-case property names

{
  "properties": {
    "snake-case": { "type": "boolean" }
  }
}

Generates the following TypeScript output:

// Code generated by jtd-codegen for TypeScript v0.2.0

export interface Blah {
  snake-case: boolean;
}

snake-case has to be wrapped in a string to be a valid type definition.

echo '{"properties":{"snake-case": {"type":"boolean"}}}' | jtd-codegen - --typescript-out ~ --root-name "blah"

Typescript type should be null, not undefined

Given MyType.jtd.json:

{
    "properties": {
        "myProp": { "type": "string", "nullable": true }
    }
}

jtd-codegen generates:

export interface MyType {
  myProp: (string | undefined);
}

which describes JSONs {"myProp": "foo"} and {},

while IMO it should generate:

export interface MyType {
  myProp: (string | null);
}

which would describe JSONs {"myProp": "foo"} and {"myProp": null}.

Arm64 support for jtd-codegen

Hello

It seems that there is no support for jtd-codegen for arm64 but only x86. Is it planned to be released at some point? We work a lot on Jetsons and using cargo build can be quite slow.

Feature Request/Discussion: ProtoBuf definition file codegen backend

Heyo. First of all great job on building this tool, it's been a ton of fun for me in the last few days reading the code and playing around with it! We have some pretty high hopes for this becoming a much simpler and more ergonomic alternative to json schemas!

I am currently investigating whether it might be worth it to implement a mapping between the json type schemas and protobuffer files. The reason for this is that at SOUNDBOKS, we have a bunch of internal APIs that would be well fit to utilize jsontypedef for their interfaces, but we also have some firmware code that as to occasionally take data from these services, which currently requires an awkward conversion step, as parsing json in an embedded context is no-bueno. ๐Ÿ˜›

From a technical point of view I think this is actually fairly doable, I added a target for protobuf on a fork (https://github.com/SOUNDBOKS/json-typedef-codegen) and it already passes more than half the test suite and with that 80+% of the use cases I want. But there is still some annoying kinks to work out, in terms of finding a good 1-to-1 mapping between the two formats + with the current structure of the AST walker implementing Ref is not straight forward as far as I can tell, because proto does not natively support type aliases + I think I need to find a way to generate certain type definitions of enums and structs inside of other ones to prevent name collisions, which I also didn't immediately see an easy solution for (You can look at this file for an example of such a name clash. RootFooBar should ideally be generated as a sub-type of the RootFoo message and just be called Bar).

Anyway let me know if this at all interests you. I could also see the argument for this being out of scope, in which case I would prob. take some inspiration and develop it as a separate tool or something.

Cheers, Mark@SOUNDBOKS ๐Ÿ˜„

cant generate types from this jsonld schema

i clone this repo: https://github.com/redaktor/ActivityPubSchema

and did:
jtd-codegen type/Accept.json --typescript-out ts/

and I got

Error: Failed to parse input as JSON

Caused by:
unknown field `$id`, expected one of `definitions`, `nullable`, `ref`, `type`, `enum`, `elements`, `properties`, `optionalProperties`, `additionalProperties`, `values`, `discriminator`, `mapping`, `metadata` at line 2 column 8

jtd-codegen generates invalid typescript if the empty string is allowed as an enum value

Given the following input JTD:

{
  "properties": {
    "a": {"type":  "string"},
    "b": {"enum":  ["", "x", "y"]}
  }
}

jtd-codegen with the --typescript-out flag generates the following output:

// Code generated by jtd-codegen for TypeScript v0.2.0

export enum SimpleB {
   = "",
  X = "x",
  Y = "y",
}

export interface Simple {
  a: string;
  b: SimpleB;
}

However, the line = "" with nothing before the equals sign is rejected by the typescript compiler as a syntax error.

jtd-codegen should either reject the input if I'm misunderstanding something and this is not valid JTD, or it should emit typescript which does not cause the typescript compiler to complain.

Rust: Naming convention for fields is incorrect [Patch]

As per Rust RFC430

FIELD_NAMING_CONVENTION should be snake case

From f0e1a04cb88bd7ffcb66b80ddad15f8d6851cb15 Mon Sep 17 00:00:00 2001
From: Adam Leyshon <[email protected]>
Date: Thu, 15 Jul 2021 18:15:01 +0100
Subject: [PATCH] Fix Rust casing

---
 crates/target_rust/src/lib.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/crates/target_rust/src/lib.rs b/crates/target_rust/src/lib.rs
index b6215ce..4890e64 100644
--- a/crates/target_rust/src/lib.rs
+++ b/crates/target_rust/src/lib.rs
@@ -19,7 +19,7 @@ lazy_static! {
     static ref FIELD_NAMING_CONVENTION: Box<dyn inflect::Inflector + Send + Sync> =
         Box::new(inflect::KeywordAvoidingInflector::new(
             KEYWORDS.clone(),
-            inflect::TailInflector::new(inflect::Case::camel_case())
+            inflect::TailInflector::new(inflect::Case::snake_case())
         ));
     static ref ENUM_MEMBER_NAMING_CONVENTION: Box<dyn inflect::Inflector + Send + Sync> =
         Box::new(inflect::KeywordAvoidingInflector::new(
-- 
2.30.2.windows.1

Publish as WASM Module

I would like to use jtd-codegen via node. It should be possible to generate the wasm bindings with wasm-pack and publish them to npm.

generate multiple types at once

can we generate types from a directory of jtd.json files ?
for example let's say I have these two files in my schema directory "user.jtd.json" and "company.jtd.json".
can I generate ts types for both at once ? something like "jtd-codegen schemas/**/*.jtd.json --typescript-out src/types.ts"

Convert Typescript to JTD

Apologies as this is more of a feature request than an issue. As a developer with apps that already have a number of Typescript types / interfaces written, I would like the ability to convert these into JTD via the codegen, so that I may be able to perform JSON โ€œschemaโ€ validation against these without having to rewrite the apps to make the JTD defs authoritative.

Thanks for your consideration!

codegen for c or c++

Hello, great work on this project! Opening an issue as requested on readme page. I was searching for C or C++ codegen. Is this something that might already be in progress or has it been discussed? The primitive type support in RFC8927 seems to map somewhat nicely to those languages. thanks!

Rust: tests broken by shifting dependency versions

Problem

Cloning down the repository and utilizing the current default branch, tests are currently not passing.

$ cd ./crates/target_rust
$ cargo test

The error that occurs is

executor failed running [/bin/sh -c cargo build]: exit code: 101
[+] Building 0.0s (10/14)
 => [internal] load build definition from Dockerfile                     0.0s
 => => transferring dockerfile: 372B                                     0.0s
 => [internal] load .dockerignore                                        0.0s
 => => transferring context: 2B                                          0.0s
 => [internal] load metadata for docker.io/library/rust:1.49             0.0s
 => [internal] load build context                                        0.0s
 => => transferring context: 1.20kB                                      0.0s
 => [ 1/10] FROM docker.io/library/rust:1.49                             0.0s
 => CACHED [ 2/10] WORKDIR /work                                         0.0s
 => CACHED [ 3/10] COPY /Cargo.toml /work/Cargo.toml                     0.0s
 => CACHED [ 4/10] RUN mkdir /work/src                                   0.0s
 => CACHED [ 5/10] RUN echo 'fn main() {}' > /work/src/main.rs           0.0s
 => ERROR [ 6/10] RUN cargo build                                       51.0s
------
 > [ 6/10] RUN cargo build:
#10 0.906     Updating crates.io index
#10 50.29  Downloading crates ...
#10 50.82   Downloaded time v0.1.44
#10 50.86   Downloaded unicode-ident v1.0.3
#10 50.87   Downloaded quote v1.0.21
#10 50.89   Downloaded iana-time-zone v0.1.46
#10 50.94   Downloaded ryu v1.0.11
#10 50.96   Downloaded serde v1.0.143
#10 50.99 error: failed to parse manifest at `/usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.143/Cargo.toml`
#10 50.99
#10 50.99 Caused by:
#10 50.99   feature `resolver` is required
#10 50.99
#10 50.99   this Cargo does not support nightly features, but if you
#10 50.99   switch to nightly channel you can add
#10 50.99   `cargo-features = ["resolver"]` to enable this feature

Cause

It seems that the problem is caused by a couple things. The Dockerfile uses rust at version 1.49, but the dependencies in the Cargo.toml aren't specific enough and the automatically pulled newer versions require a newer version of cargo

chrono = { version = "0.4", features = ["serde"] }
serde_json = "1"
serde = { version = "1.0", features = ["derive"] }

Rust code generated can't be compiled when there is recursive type

example:

{
    "definitions": {
        "node": {
            "properties": {
                "child": {"ref": "node"}
            }
        }
    },
    "properties": {
        "root": {
            "ref": "node"
        }
    }
}

generated code

use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize)]
pub struct Jtd {
    #[serde(rename = "root")]
    pub root: Node,
}

#[derive(Serialize, Deserialize)]
pub struct Node {
    #[serde(rename = "child")]
    pub child: Node,
}

should be boxed

pub child: Box<Node>

Permit specifying/overriding field names

I have a handwritten struct that is shared between services in Python, TypeScript, C#, and Go.

The Go version has a field like this (where Date is a custom date type):

    EffectiveDate Date `json:"effective"`

It looks like it's currently impossible to generate a drop-in replacement because the field name doesn't match the JSON field name. Would it be possible to add the equivalent of metadata.<lang>Type but for field-naming? I.e., I'd like the schema to be able to generate the field like this:

    "effective": {
      "metadata": {
        "description": "The date that this transaction takes effect",
        "goField": "EffectiveDate"
      },
      "type": "Date"
    },

Presumably the TypeScript generator wouldn't offer this option for the reasons explained here, but I think it makes sense to offer it for all other supported languages.

Panic on empty discriminator mapping

A minimal example is as follows:

{
    "properties": {
        "tag": {
            "enum": [
                "foo"
            ]
        },
        "bar": {
            "discriminator": "tag",
            "mapping": {
                "foo": {}
            }
        }
    }
}

As I understand the spec, this should be a valid Typedef. When I try to run it by

$ RUST_BACKTRACE=full jtd-codegen foo.jtd.json --log-format minimal --typescript-out .

I get the following panic and backtrace.

TypeScript: writing to: .
thread 'main' panicked at 'internal error: entered unreachable code', /project/crates/core/src/codegen/ast.rs:377:30
stack backtrace:
   0:           0x73717b - std::backtrace_rs::backtrace::dbghelp::trace::h61b20ba2180b2411
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\..\..\backtrace\src\backtrace\dbghelp.rs:98:5
   1:           0x73717b - std::backtrace_rs::backtrace::trace_unsynchronized::h83cb15607b6aabb2        
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66:5
   2:           0x73717b - std::sys_common::backtrace::_print_fmt::h1db737bace97a9a3
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\sys_common\backtrace.rs:67:5
   3:           0x73717b - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h34d7951d049896a6
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\sys_common\backtrace.rs:46:22
   4:           0x7a2a9b - core::fmt::write::h869094fcfa18bcd2
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\core\src\fmt\mod.rs:1078:17
   5:           0x7286e1 - std::io::Write::write_fmt::h09af653157e460eb
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\io\mod.rs:1517:15
   6:           0x73af08 - std::sys_common::backtrace::_print::h1e43b04f309c2b98
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\sys_common\backtrace.rs:49:5
   7:           0x73af08 - std::sys_common::backtrace::print::hf10619d2eb24b971
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\sys_common\backtrace.rs:36:9
   8:           0x73af08 - std::panicking::default_hook::{{closure}}::hba2be3522cb11021
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\panicking.rs:208:50
   9:           0x73aa2d - std::panicking::default_hook::hae3332b244cce04c
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\panicking.rs:225:9
  10:           0x73b789 - std::panicking::rust_panic_with_hook::h591ddcf8b9c5a5c0
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\panicking.rs:591:17
  11:           0x73b2f8 - std::panicking::begin_panic_handler::{{closure}}::hdeb48e5ef966eb2f
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\panicking.rs:495:13
  12:           0x737b2f - std::sys_common::backtrace::__rust_end_short_backtrace::h30bf946dcb572b09    
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\sys_common\backtrace.rs:141:18
  13:           0x73b289 - rust_begin_unwind
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\panicking.rs:493:5
  14:           0x79eb40 - core::panicking::panic_fmt::h2a0fa684de22a764
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\core\src\panicking.rs:92:14
  15:           0x79ea8c - core::panicking::panic::h731c7b783f4684bb
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\core\src\panicking.rs:50:5
  16:           0x44fdf2 - jtd_codegen::codegen::ast::Ast::new::h4bf1d5cbf223dbd0
  17:           0x44e5f2 - jtd_codegen::codegen::ast::Ast::new::h4bf1d5cbf223dbd0
  18:           0x44ad20 - jtd_codegen::codegen::ast::Ast::new_top_level::hd15b14dd193b5373
  19:           0x45f87b - jtd_codegen::codegen::ast::SchemaAst::new::hb66fafe858a2aa60
  20:           0x434388 - jtd_codegen::codegen::codegen::h823d9544d601a5f1
  21:           0x473e90 - jtd_codegen::main::hafe47fad1009e448
  22:           0x43d066 - std::sys_common::backtrace::__rust_begin_short_backtrace::hf52320e7b3bae508  
  23:           0x43d0bd - std::rt::lang_start::{{closure}}::h71c4337b36217baa
  24:           0x73ba0c - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h4082d3e330be1195
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\library\core\src\ops/function.rs:259:13
  25:           0x73ba0c - std::panicking::try::do_call::h780e3a0d8897887c
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\panicking.rs:379:40
  26:           0x73ba0c - std::panicking::try::h2567bf242b4fa6c6
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\panicking.rs:343:19
  27:           0x73ba0c - std::panic::catch_unwind::h494a7b02b9feffd3
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\panic.rs:396:14
  28:           0x73ba0c - std::rt::lang_start_internal::he12bde4ee8419433
                               at /rustc/cb75ad5db02783e8b0222fee363c5f63f7e2cf5b\/library\std\src\rt.rs:51:25
  29:           0x475848 - main
  30:           0x4013f8 - __tmainCRTStartup
                               at /./mingw-w64-crt/crt/crtexe.c:334
  31:           0x40151b - mainCRTStartup
                               at /./mingw-w64-crt/crt/crtexe.c:212
  32:     0x7ffb3aab7034 - <unknown>
  33:     0x7ffb3bc22651 - <unknown>

Version is 0.4.0.

$ jtd-codegen --version
jtd-codegen 0.4.0

I feel that this is a bug.

Cheers

Edit: I tried with the latest release 0.4.1, and the problem remains the same.

Remote resource support

I thought it would be very useful to support URLs for specifying resources and being able to target remote schema.

What do you think about this feature?

Python datetime serialisation/deserialisation not working correctly without timezone

There seems to be an issue with serialising and deserialising datetimes for python generated dataclasses.
This example fails with ValueError: ('Invalid RFC3339 date/time', '2023-01-01T00:00:00'):

some_dataclass = SomeDataclass(some_time=datetime(2023, 1, 1, 0, 0, 0))
json_data = some_dataclass.to_json_data()
some_dataclass = SomeDataclass.from_json_data(json_data)

It seems to be because the serialising to json for datetime is just datetime.isoformat(), which will return '2023-01-01T00:00:00' for the mentioned value. This will then fail the check in the generated dataclass' _parse_rfc3339 function.

Go: Use private interfaces instead of generating structs for discriminating types

Proposal

This issue proposes that for the following JSON type definition:

{
  "Thing": {
    "discriminator": "type",
    "mapping": {
      "a": {
        "properties": {}
      },
      "b": {
        "properties": {}
      },
      "c": {
        "properties": {}
      }
    }
  }
}

jtd-codegen should generate the following Go code:

type Thing struct {
	Type string `json:"type"`

	// Value can be the following types:
	//
	//    - [A]
	//    - [B]
	//    - [C]
	//
	Value thingValue `json:"-"`
}

func (t *Thing) UnmarshalJSON(data []byte) error
func (t *Thing) MarshalJSON() ([]byte, error)

type thingValue interface {
	isThing()
}

type ThingA struct{}
type ThingB struct{}
type ThingC struct{}

func (ThingA) isThing() {}
func (ThingB) isThing() {}
func (ThingC) isThing() {}

The user would consume the API like so:

var t Thing

switch v := t.Value.(type) {
case ThingA:
	log.Println("thing contains type A")
case ThingB:
	log.Println("thing contains type B")
case ThingC:
	log.Println("thing contains type C")
default:
	log.Println("thing contains unknown type")
}

If the user makes a mistake in the type-switch, the compiler will complain:

var t Thing

switch v := t.Value.(type) {
case ThingA:
	log.Println("thing contains type A")
case string:
	// does not even compile
}

This issue is an alternative to issue #49.

Rationale

Pros:

  • Using interfaces instead of flat structs can sometimes takes up less space in
    memory if the mapping is large.
  • The user can use type-switches to determine the type of the value, which is
    more idiomatic and safer in Go than using a string field.

Cons:

  • The magic field's name Value is not ideal. This is done because we can't
    have custom marshalers and unmarshalers on an interface.
  • The magic field has an unexported type. Showing an unexported type in an
    exported type is not ideal.
  • It's not explicit which types actually satisfy TValue outside a single
    comment. This might be confusing to users.

Binary type support

Would be great to see binary type support. Pure JSON could utilize base64 for that, but also besides that same definition could be used to play with MessagePack, that supports binary data natively.

npm package?

Are there any plans to make this available via npm? I'd really like to use this in a code repository as one of our developer tools without adding additional requirements beyond npm install

I wouldn't mind scripting the brew install, but I'd also like to use the tool in our CI/CD environment, and we use linux there...

Typedefs with reserved Rust names require qualification

There's a type in my schema called "option" that results in Option structs being generated, which causes a collision when compiling with the Rust compiler. For now, a workaround is to manually rename references to the Option trait to "std::option::Option" but it would be better if the codegen could handle it.

How to generate classes part of build.gradle?

Hello,

I am trying to use json-typedef to generate POJO classes part of build process. Are there any plugins that I can use to generate them?

Appreciate your help.

Thanks
Prasanna Balaraman

[FEATURE REQUEST] Online codegen

I tried jtd-codegen and found it very useful !
On the other hand, I felt that it was too much of a psychological burden to install and use it.

If possible, I would like to perform this operation in the browser. In other words, I want to have an online generator.

Could you please packaging jtd-codegen in npm as well as jtd? Or could you tell me a way to make use of it from js as wasm ?

p.s.
I have already created an online validator powered by jtd.
If we have a generator here, I convince it would make JTD development very easy and attractive more and more.

Python 3.12 Datetime Regex, SyntaxWarning

With the example schema from the tutorial here:
https://jsontypedef.com/docs/python-codegen/

{
  "properties": {
    "id": { "type": "string" },
    "createdAt": { "type": "timestamp" },
    "karma": { "type": "int32" },
    "isAdmin": { "type": "boolean" }
  }
}

Python 3.11 is happy:

$ conda activate py311
$ python --version
Python 3.11.8
$ python py/__init__.py 
# (Quiet; prints nothing)

Python 3.12 issues this SyntaxWarning about the regex used for the datetime:

$ conda activate py312
$ python --version
Python 3.12.2
$ python py/__init__.py 
.../py/__init__.py:58: SyntaxWarning: invalid escape sequence '\d'
  datetime_re = '^(\d{4})-(\d{2})-(\d{2})[tT](\d{2}):(\d{2}):(\d{2})(\.\d+)?([zZ]|((\+|-)(\d{2}):(\d{2})))$'

Rust: Allow specifying of Custom `use` in header and `#[derives]` on Structs

This would enable people to use the generated Structs with other libraries such as Diesel that require us to use #[derive] macros on Structs that have complicated trait implementations and others such as Default

Example:

#[derive(FromSqlRow, AsExpression, Serialize, Deserialize, Debug, Default)]
#[serde(deny_unknown_fields)]
pub struct ApiConfigData {
    #[serde(rename = "inventory")]
    inventory: Inventory,

    #[serde(rename = "maintenance")]
    maintenance: Maintenance,
}

Is it possible to use it as a crate from the build.rs?

Hi! I just discovered jdt and this crate and it looks really great! Thanks for all the hard work here.

I was wondering, is it possible to install this package from crates.io and generate code on the build.rs of a project? I saw that it's available from brew which is nice, but I was looking for a more portable way, where it's a dependency attached to the project and reads a json and auto generate rust code when I do cargo build/run on my project.

Thanks!

Inconsistent Python property names with JSON Type Definition

Symptoms

JSON Type Definitions generated code is not consistent across languages of (known so far) python and typescript.

Minimal reproduction

{
  "properties": {
    "_id": {},
    "name": { "type": "string" }
  },
  "optionalProperties": {
    "_tid": {},
    "__ttid": {}
  }
}

Output of --python-out using jtd-codegen cli...

@dataclass
class TmpTest:
    id: 'Any'
    name: 'str'
    ttid: 'Any'
    tid: 'Any'

Output of --typescript-out using jtd-codegen cli...

export interface TmpTest {
  _id: any;
  name: string;
  __ttid?: any;
  _tid?: any;
}

OS, Environment Details

  • Ubuntu 20
  • Bash
  • jtd-codegen: v0.4.1

Question:

Is this somehow intentional? The .from_json_data and .to_json_data methods of the python dataclass include the underscores. I'm no python super-star, but I don't know of a reason why this tool should enforce this "opinionated" code generation for python alone.

NPM packaging request

Right now in a CI/CD respect the current workflow of downloading separate release binaries is setting off red flags for usage. It would be great if there was some NPM package similar to https://github.com/timostamm/protobuf-ts/tree/master/packages/protoc where its downloading the proper binaries and setting on path. It also means the binaries are pasted through our artifactory instance to track versioning when related to generated containers.

Discriminator variant disapears when type name is overridden

This schema with custom type name in metadata field:

{
  "discriminator": "eventType",
  "mapping": {
    "USER_CREATED": {
      "metadata": {
        "typescriptType": "UserCreatedEvent",
        "pythonType": "UserCreatedEvent"
      },
      "properties": {
        "id": {
          "type": "string"
        }
      }
    },
    "USER_DELETED": {
      "properties": {
        "id": {
          "type": "string"
        },
        "softDelete": {
          "type": "boolean"
        }
      }
    }
  }
}

does not generate discriminated variant USER_CREATED:

export type Test = TestUserCreated | TestUserDeleted;

export interface TestUserDeleted {
  eventType: "USER_DELETED";
  id: string;
  softDelete: boolean;
}
@dataclass
class Test:
    event_type: 'str'

    @classmethod
    def from_json_data(cls, data: Any) -> 'Test':
        variants: Dict[str, Type[Test]] = {
            "USER_CREATED": TestUserCreated,
            "USER_DELETED": TestUserDeleted,
        }
        return variants[data["eventType"]].from_json_data(data)

    # ...

@dataclass
class TestUserDeleted(Test):
    id: 'str'
    soft_delete: 'bool'
    # ...

Output to stdout as json

I want to be able to use jtd-codegen as an part of a larger generator, right now you are dumping to a folder (at least in TS as an index.ts) which is not ideal. Would like to have something like

jtd-codegen --typescript-out test req.json --log-format json --export-to-logs

Which would no write out files but the output would contain the results ala

{
  "TypeScript": {
    "out_dir": "test",
    "root_name": "Req",
    "definition_names": {},
    "output":"// Code generated by jtd-codegen for TypeScript v0.2.1\n\nexport type..."
}

json schema codegen

I suggest adding --json-schema-out to the codegen. What do you guys think?

TypeScript: `any` vs `unknown`

Currently, empty forms are generated as any type.

In TypeScript, there are few situations where the any type is more suitable than the unknown type.

Both are tolerant of assignments.
unknown is more type-safe than any because it is strictly for reference.

I think the unknown type is better suited for the role of a code generator.

Feature Request/Discussion: Golang string types for discriminator

A definition of

 "foo": {
      "discriminator": "type",
      "mapping": {
        "a": {
          "properties": {}
        },
        "b": {
          "properties": {}
        },
        "c": {
          "properties": {}
        }
      }
    }

Generates a Go struct of

type Foo struct {
	Type string

	A FooA

	B FooB

	C FooC
}

Where it should be more acccurately be the same as an enum and look more like...

type FooTypeDiscriminator string

const (
	FooTypeDiscriminatorA FooTypeDiscriminator = "a"

	FooTypeDiscriminatorB FooTypeDiscriminator = "b"

	FooTypeDiscriminatorC FooTypeDiscriminator = "c"
)

type Foo struct {
	FooTypeDiscriminator string

	A FooA

	B FooB

	C FooC
}

This would improve auto-completion and validation.

Certain type names causes errors for Python codegen

Having user defined types with names like Optional will make Python unable to use the typing.Optional type since it is from imported.

Not importing typing.Optional but instead using it with the full module prefix would fix this.

The same would happen with Enum, Any, or List.

Shame that this is unmaintained, I would not mind making a PR for this since it's a simple fix.
Creating this issue so that others will realize their issue faster.


With the following schema:

{
    "properties": {
        "opt": {
            "ref": "optional"
        }
    },
    "optionalProperties": {
        "t": {"type":"string"}
    },
    "definitions": {
        "optional": {}
    }
}

Running jtd-codegen --python-out . schema.json creates the following:

# Code generated by jtd-codegen for Python v0.3.1

import re
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from typing import Any, Dict, Optional, Union, get_args, get_origin


@dataclass
class Schema:
    opt: 'Optional'
    t: 'Optional[str]'

    @classmethod
    def from_json_data(cls, data: Any) -> 'Schema':
        return cls(
            _from_json_data(Optional, data.get("opt")),
            _from_json_data(Optional[str], data.get("t")),
        )

    def to_json_data(self) -> Any:
        data: Dict[str, Any] = {}
        data["opt"] = _to_json_data(self.opt)
        if self.t is not None:
             data["t"] = _to_json_data(self.t)
        return data

@dataclass
class Optional:
    value: 'Any'

    @classmethod
    def from_json_data(cls, data: Any) -> 'Optional':
        return cls(_from_json_data(Any, data))

    def to_json_data(self) -> Any:
        return _to_json_data(self.value)

def _from_json_data(cls: Any, data: Any) -> Any:
    if data is None or cls in [bool, int, float, str, object] or cls is Any:
        return data
    if cls is datetime:
        return _parse_rfc3339(data)
    if get_origin(cls) is Union:
        return _from_json_data(get_args(cls)[0], data)
    if get_origin(cls) is list:
        return [_from_json_data(get_args(cls)[0], d) for d in data]
    if get_origin(cls) is dict:
        return { k: _from_json_data(get_args(cls)[1], v) for k, v in data.items() }
    return cls.from_json_data(data)

def _to_json_data(data: Any) -> Any:
    if data is None or type(data) in [bool, int, float, str, object]:
        return data
    if type(data) is datetime:
        return data.isoformat()
    if type(data) is list:
        return [_to_json_data(d) for d in data]
    if type(data) is dict:
        return { k: _to_json_data(v) for k, v in data.items() }
    return data.to_json_data()

def _parse_rfc3339(s: str) -> datetime:
    datetime_re = '^(\d{4})-(\d{2})-(\d{2})[tT](\d{2}):(\d{2}):(\d{2})(\.\d+)?([zZ]|((\+|-)(\d{2}):(\d{2})))$'
    match = re.match(datetime_re, s)
    if not match:
        raise ValueError('Invalid RFC3339 date/time', s)

    (year, month, day, hour, minute, second, frac_seconds, offset,
     *tz) = match.groups()

    frac_seconds_parsed = None
    if frac_seconds:
        frac_seconds_parsed = int(float(frac_seconds) * 1_000_000)
    else:
        frac_seconds_parsed = 0

    tzinfo = None
    if offset == 'Z':
        tzinfo = timezone.utc
    else:
        hours = int(tz[2])
        minutes = int(tz[3])
        sign = 1 if tz[1] == '+' else -1

        if minutes not in range(60):
            raise ValueError('minute offset must be in 0..59')

        tzinfo = timezone(timedelta(minutes=sign * (60 * hours + minutes)))

    second_parsed = int(second)
    if second_parsed == 60:
        second_parsed = 59

    return datetime(int(year), int(month), int(day), int(hour), int(minute),
                    second_parsed, frac_seconds_parsed, tzinfo)            

Running it with

import model
import json
data = json.loads("{}")
t = model.Schema.from_json_data(data)

causes

Traceback (most recent call last):
  File "/media/asd/96B4-13F2/jtdc/t.py", line 7, in <module>
    t = model.Schema.from_json_data(data)
  File "/media/asd/96B4-13F2/jtdc/model.py", line 18, in from_json_data
    _from_json_data(Optional[str], data.get("t")),
TypeError: 'type' object is not subscriptable

Emit errors as json when `--log-format json` is set.

In wrapping the cli programmatically, it's easy to process std out with --log-format json. It would be convienent to also be able to parse errors in the same way.

For example, when parsing the invalid jtd:

{ "enum": [1, 2, 3] }

with --log-format json this is output to stderr

Error: Failed to parse input as JSON
      
Caused by:
      invalid type: integer `1`, expected a string at line 1 column 10

Something like this would be what I would expect:

{
    "error": "Failed to parse input as JSON",
    "cause": "invalid type: integer `1`, expected a string at line 1 column 10"
}

[Suggestion] Release/publish jtd-codegen to npm

Hi,

First, thank you for your work with this project! ๐ŸŽ‰

I'm using the jtd-codegen CLI on one of my project to generate some TypeScript types from a JTD schema generated using a custom Serializer writing in Ruby. The serializers are used to pass some data to the frontend written in TypeScript. The jtd-codegen is used in a CI environment to ensure consistency between the frontend and the backend.

It would be really useful to be able to install the codegen using a package manager like npm/yarn, because it is what a lot of people are using to install their dependencies and it would allow us to add simple script commands in a package.json to easily run the codegen in any environment.

{
  "dependencies": {
    "jtd-codegen": "x.x.x",
    // any other deps
  },
  "scripts": {
    "generate-typings": "jtd-codegen --typescript-out path/to/schema.json path/to/types/folder"
  }
}

and then in any environment we could simply run:

npm install
npm run generate-typings

Here are some documentation/examples on how to achieve this ๐Ÿ™‚

If you need any help do not hesitate to ask me. I can help and propose a PR if you are open to it. ๐Ÿ™‚

Thank you again for this project! ๐ŸŽ‰

Name JSON key when erroring

When running jtd-codegen, I get an error like:

thread 'main' panicked at 'no entry found for key', /project/crates/core/src/codegen/mod.rs:123:44

But there's no hint of what "key" was being used at the time of the error.

For this error, can you provide a hint as to which JSON key has no entry?

Rust: Generated structs do not have public fields

Please allow an option to make the fields public during code generation.
This would allow people to instantiate the types and impl traits such as Default.

Instead of:

#[derive(Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Maintenance {
    #[serde(rename = "start_time")]
    start_time: u32,
}

Should yield:

#[derive(Debug, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Maintenance {
    #[serde(rename = "start_time")]
    pub start_time: u32,
}

Go: support for explicit embedded structs

I would like to generate a Go struct where some of the fields are "embedded" in another struct:

type Inner struct {
    Foo string `json:"foo"`
    Bar string `json:"bar"`
}

type SharedType struct {
    Inner
    
    Baz string `json:"baz"`
}

This is marshaled and unmarshaled as:

{
    "foo": "<data>",
    "bar": "<data>",
    "baz": "<data>"
}

Would it be feasible to add Go-specific medadata to enable codegen like this? Something like:

{
  "properties": {
    "foo": {
      "metadata": {
        "goEmbedded": "Inner"
      },
      "type": "string"
    },
    "bar": {
      "metadata": {
        "goEmbedded": "Inner"
      },
      "type": "string"
    },
    "baz": {
      "type": "string"
    },
  }
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.