Git Product home page Git Product logo

yaml-validator's Introduction

yaml-validator

Validate Yaml files and enforce a given structure

Ubuntu Build Status Windows build status code coverage Code Smells

Yaml files are parsed via js-yaml and the structure defined in the configuration options is enforced with check-type.

Getting Started

Please note that the minimum supported version of Node.js is 18.12.0, which is the active Long Term Support (LTS) version.

This tool can be used in two ways, either via Node.js script, or as a command line tool. Note that when used via command line, custom structure cannot be validated.

Installation when used via Node.js script:

npm install yaml-validator --save-dev

Installation when used as a command line tool:

npm install --global yaml-validator

Usage as a part of a Node.js script:

const YamlValidator = require('yaml-validator');

// Default options
const options = {
  log: false,
  structure: false,
  onWarning: null,
  writeJson: false
};

const files = [
  'file paths',
  'that exists',
  'somewhere',
  'and are Yaml files'
];

const validator = new YamlValidator(options);
validator.validate(files);
validator.report();

Using via command line tool, the only argument would be the Yaml file which should be validated:

yaml-validator random_file.yml

The available options for command line use, can be seen with the help command yaml-validator -h, which results in output similar to:

yaml-validator [options] <files>

  -h, --help             Help and usage instructions
  -V, --version          Version number
  -w, --write-json       Write the contents of the Yaml file to a JSON file next to it
  -l, --log-file String  Log file where errors are written

Version 5.0.0

When used from the command line, the process exits with the number of invalid files.

Configuration options

All options are false by default which disables their use.

options.log

Type: string

Default value: false

In case the value is not false, the given string will be used as log file where all the task output is written.

options.structure

Type: object

Default value: false

The most complex style of checking validity.

options.onWarning

Type: function

Default value: null

One of the options passed to load method of js-yaml.

Please note that the onWarning callback is being used by this library and any method written for it, will be run after the one implemented in this library. The callback get called with two parameters, of which the first is the error in question, while the second is the file path of the given Yaml file.

options.writeJson

Type: boolean

Default: false

Write the given Yaml file as pretty printed JSON in the same path, just by changing the file extension to json.

Please note that any existing JSON files will be cruelly overwritten.

Typescript Support

YamlValidator ships with its own typing definition in the library, no need to use @types.

Examples

Structure validation options

In case an array is found, all its members are assumed to have the given structure. This can be seen in the classRooms property, which according to the configuration below, should be an array, for which all items are objects, which all should have a name and id properties, with the given types.

The teachers array is made of strings, thus all items in that array must be a string.

const options = {
  structure: {
    school: {
      'description?': 'string', //Optional, won't show in invalid array
      code: 'number',
      principal: {
        name: 'string'
      },
      classRooms: [
        {
          name: 'string',
          id: 'number',
          'location?':{
            floor: "string",
            building: "string",
          }
        }
      ],
      teachers: [
        'string'
      ]
    }
  }
};

Warning callback in Yaml parsing options

Using the options.onWarning callback, the possible parsing errors can be retrieved.

const options = {
  onWarning: function (error, filepath) {
    console.log(filepath + ' has error: ' + error);
  }
};

Write a JSON file option

It is possible to use the options.writeJson to have all the files processed, to be saved in JSON format, in the same file path as the original Yaml files.

const options = {
  writeJson: true
};

Contributing

"A Beginner's Guide to Open Source: The Best Advice for Making your First Contribution".

Also there is a blog post about "45 Github Issues Dos and Don’ts".

Linting is done with ESLint and can be executed with npm run lint. There should be no errors appearing after any JavaScript file changes.

Please note that any features or changed will not be merged without working unit tests.

Unit tests are written with tape and can be executed with npm test. Code coverage is inspected with nyc and can be executed with npm run coverage after running npm test. Please make sure it is over 90% at all times.

Version History

Changes happening across different versions and upcoming changes are tracked in the CHANGELOG.md file.

License

Copyright (c) Juga Paazmaya [email protected]

Licensed under the MIT license.

yaml-validator's People

Contributors

ben-polinsky avatar bilby91 avatar dependabot[bot] avatar gtagmodss avatar marclipovsky avatar mbad0la avatar paazmaya avatar rbanffy avatar renovate-bot avatar renovate[bot] avatar river0825 avatar snyk-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

yaml-validator's Issues

Other useful validations

The discussion about regexp keys got me thinking about two other things that would be really useful:

Regexp validator

Add /regexp/ as a matching criteria, with the intended meaning that:

key: /regexp/

permits any string value whose undelimited form matches the regexp.

Procedure validator

Permit a lambda expression (val) => boolean as a matchable type, with the intended meaning that the boolean, number, [undelimited] string, list, or structure value read is passed to the function, and is accepted exactly if the function returns true. This enables some very interesting filtering without adding much complexity to the core implementation. In particular, it would enable a sophisticated matcher to implement things like:

  • Accept only specific number values
  • Check a structure using a (nested) yaml-validator, but then confirm that co-required keys or mutually-exclusive keys are not present.

Unfortunately, I suspect that either of these takes us past what check-type can do.

Thoughts?

filepath and error.message don't get consoled out or logged together

Expected behaviour

Error output and log should show both the filepath and error.message.

Actual behaviour

When validator is ran, error.message is consoled out but the filepath is not. When you generate a report only the filepath is logged and not the error.message

Versions and environment

  • Operating system: macOS 10.12.5
  • Node.js: any version
  • npm: any version
  • yaml-validator: 0.3.0

Thanks.

New release ?

Hello!

Was wondering when there will be a new release ? Want to use the new typescript definitions :)

TypeError on run

Expected behaviour

Should have validated yaml file.

Actual behaviour

/usr/local/lib/node_modules/yaml-validator/index.js:129
    const lineNumber = error.message.match(/line (\d+)/)[1];
                                                        ^

TypeError: Cannot read property '1' of null
    at YamlValidatore.loadData (/usr/local/lib/node_modules/yaml-validator/index.js:129:57)
    at YamlValidatore.loadFile (/usr/local/lib/node_modules/yaml-validator/index.js:158:15)
    at YamlValidatore.checkFile (/usr/local/lib/node_modules/yaml-validator/index.js:167:20)
    at mapFiles (/usr/local/lib/node_modules/yaml-validator/index.js:201:18)
    at Array.map (<anonymous>)
    at YamlValidatore.validate (/usr/local/lib/node_modules/yaml-validator/index.js:200:34)
    at Object.<anonymous> (/usr/local/lib/node_modules/yaml-validator/bin/yaml-validator.js:112:11)
    at Module._compile (internal/modules/cjs/loader.js:722:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:733:10)
    at Module.load (internal/modules/cjs/loader.js:620:32)

Versions and environment

  • Operating system: Mac OS X
  • Node.js: v11.2.0
  • npm: 6.4.1
  • yaml-validator: 1.3.0

Empty yaml file produces error, but no logs about the error

Setup: run validate() on a long list of files, with one being empty (or just a comment)

Expected behaviour

report() will tell me which file has an error
or
report() will return success.

Actual behaviour

the validator.report() will report 1 file with an error, but the logs will not say which file has an error.

Unit testing

Currently there is just a single test case which does not bring any value.

Test all methods and different use cases.

onWarning callBack doesn´t trigger

Hi there,

I wrote a little test app where I wanted to test your library, but it seems that the callback won´t trigger.

Content of testfile

test:
  structure:
    myValue: "1"

Code to load:


const yamlContent = fs.readFileSync('bin/test.yaml', {
    encoding: 'utf-8'
})

Content of schema

{
    "test": {
        "structure":{
            "myValue": "string",
            "myValue2": "string"
        }
    }
}

Validator Options:

const structure = fs.readFileSync('bin/schema.json', { encoding: 'utf-8' })
.....

const options: IYamlValidatorOptions = {
    log: 'validator.log',
    onWarning: (err, file) => {
        console.log("File:",file)
        throw new Error(err.message)
    },
    structure: JSON.parse(structure),
    writeJson: false

};

Call validate


const validator = new YamlValidator(options);
validator.validate(files);
validator.report();

logfile content:

bin/test.yaml is not following the correct structure, missing:
test.structure.myValue2
Yaml format related errors in 1 files
Total of 1 structure validation error(s)

But the onWarning function doesn´t get executed (log console logs, no error thrown) :-(

Any hints for me ?

Thanks man!

Typescript definitions

Hello!

First of all thanks for this awesome tool! I needed exactly this tool :)

Having said that, I was wondering if your open on hosting typescript definitions inside the repository. I can get them done and push a PR if you are fine with it.

Thanks!

Optional fields with null values should be counted as valid

Expected behaviour

Optional fields with a null value should be counted as valid if the structure matches.

Actual behaviour

Optional fields with a null value are counted as invalid.

Verify a yaml with this field/value:
middle name: null
Against this structure:
'middle name?': 'string'.

Versions and environment

  • Operating system: Windows 10
  • Node.js: 8.2.1
  • npm: 5.3
  • yaml-validator: 1.0.0

Thank you and have some 🍇.

Type boolean not supported

yaml-validator does not currently provide a way to indicate that a key should have a boolean value. This seems like something that could easily be added.

Note that the acceptable boolean values changed between YAML 1.1 and YAML 1.2 in a way that was not backwards compatible. I'm not sure whether js-yaml makes it possible to specify which version should be parsed, but if it does, I think it would be helpful to be able to specify that in the yaml-validator options structure.

Bug: Missing log when file is empty

Expected behaviour

We should have a log added to output (same as other errors)

Actual behaviour

  1. Try to validate an empty YAML file
  2. You can see that the checkFile function return 1 so it means that inValidFilesCount will return 1
  3. But not log was written to the output

Versions and environment

  • Operating system: Mac
  • Node.js: 18.13
  • npm: 8.19.3
  • yaml-validator:4.0.1

Thank you and have some 🍇.

Restricted to Node >=6.9.5

Looks like this package is restricted to Node >=6.9.5, however nothing about it relies on >=6.9.5. I've just installed it using --ignore-engines flag when I install it because I'm using 6.7.0. But I'm sure others may not know to do that. Any chance the node engine could be a bit looser. Thanks!

Git Hub Actions

I had a github action that was running yamllint to handle linting of YAML files, but a couple yaml files had syntax errors that were being detected in vscode by the redhat yaml parser, but not detected in the linting.

Anyways, I was looking for a tool that supported syntax chacking and it looks like yaml-validator does it.

So I was playing around with a github action for yaml-validator


---
name: Yaml Syntax Validation
on:
  pull_request:
    paths: ['**.yaml']
jobs:
  run-npm-install:
    name: Yaml Syntax Validation Install
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: npm install
        run: |
          npm install --global yaml-validator
      - name: validate
        run: |
          find . -iname '*.yaml' -exec yaml-validator "{}" +
...

it works alright, but I was wondering if others had built up their own github actions ? Could those share their examples.

Structure Validation does not work

Expected behaviour

options.structure should get considered by validator
see bottom of this ticket, change e.g. description from string to number -> no effect

Actual behaviour

using options.structure with a structure object always works

report() always returns 1 and no warning gets printed out

Versions and environment

  • Operating system: MacOSx
  • Node.js: 10.15.3
  • npm: 6.4.1
  • yaml-validator: ^2.2.0

Thank you and have some 🍇.

api.yml

globals:
  consumer:
    - event: "#/definitions/Register"
    - event: "#/definitions/Next"
    - event: "#/definitions/Previous"
    - event: "#/definitions/Destroy"
  producer:
    - event: "#/definitions/Initialize"

apps:
  - name: "login"
    consumer:
      - event: "#/definitions/Register"
      - event: "#/definitions/Next"
      - event: "#/definitions/Destroy"
    producer:

definitions:
  - event: "Register"
    name: "[Registration] register"
    description: "MF registers itself at the CL"

  - event: "Next"
    name: "[Routing] next"
    description: "MF tells the CL to route to the next MF"

  - event: "Previous"
    name: "[Routing] previous"
    description: "MF tells the CL to route to the previous MF"

  - event: "Initialize"
    name: "[Lifecycle] initialize"
    description: "CL may send the MF a configuration and/or a state (e.g. previous state - for-/backwards-navigation)"
    payload:
      - name: "config"
        type: "any"
      - name: "state"
        type: "any"

  - event: "Destroy"
    name: "[Lifecycle] destroy"
    description: "MF tells the CL that it has been destroyed and may pass the (latest) state"
    payload:
      - name: "state"
        type: "any"

api-schemal.validator.ts

import * as YamlValidator from 'yaml-validator';

export class ApiSchemaValidator {
  static structure = {
    globals: {
      consumer: [{ event: 'string' }],
      producer: [{ event: 'string' }],
    },
    apps: [
      {
        name: 'string',
        consumer: [{ event: 'string' }],
        producer: [{ event: 'string' }],
      },
    ],
    definitions: [
      {
        event: 'string',
        name: 'string',
        description: 'string',
        'payload?': [
          {
            name: 'string',
            type: 'string',
          },
        ],
      },
    ],
  };

  options: YamlValidator.IYamlValidatorOptions = {
    log: false,
    structure: ApiSchemaValidator.structure,
    onWarning: (error, filepath) => {
      console.warn(filepath + ' has error: ' + error);
    },
    writeJson: false,
  };

  validate(file: string): number {
    console.log(`Validating ${file}...`);

    // console.log(JSON.stringify(this.options));

    const validator = new YamlValidator(this.options);
    validator.validate([file]);
    return validator.report();
  }
}

Prototypes

I was so hoping this would fit into my project, but I can't see, either in your readme or tests, if it supports a prototype schema.

Given the following yaml:

app:
  services:
    foo:
        verb: "Give"
        quantifier: 10
        noun: "slaps"
    bar:
        verb: "Accept"
        quantifier: "any"
        noun: "women"

Each service must conform to the same schema, but it's unknown how many there are or what keys they will use.

Looking at the sources, I don't believe it is possible to read the file myself and apply a schema against an existing object. If it were, I could have done a multi stage validation.

dynamic key support?

Does yaml-validator support dynamic keys? I have object with the following structure:

dynamicKey1:
  staticKeyA: value
  staticKeyB: value
dynamicKey2:
  staticKeyA: value
  staticKeyB: value
...

Is it possible to check this structure?

I know I can convert the data model to:

- name: dynamicKey1
  staticKeyA: value
  staticKeyB: value
- name: dynamicKey2
  staticKeyA: value
  staticKeyB: value
...

then it's easy to be validated. But changing the data model introduces cost.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Warning

These dependencies are deprecated:

Datasource Name Replacement PR?
npm codecov Unavailable

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

npm
package.json
  • check-type 0.4.11
  • js-yaml 4.1.0
  • optionator 0.9.4
  • @types/node 20.14.11
  • c8 9.1.0
  • codecov 3.8.3
  • eslint 8.57.0
  • eslint-config-paazmaya 9.0.1
  • eslint-plugin-n 14.0.0
  • sinon 17.0.1
  • tape 5.8.1
  • typescript 5.5.3
  • node >=18.12.0

  • Check this box to trigger a request for Renovate to run again on this repository

Errored function is called when validation is ok

Expected behaviour

Errored function is called when an error occurs.

Actual behaviour

Writing Unit Tests (running on Jenkins with mocha), i override errored function to throw Error instead logging error in console.

But now Report function throw an error: "Total of 0 structure validation error(s)", that is definitely not an error.

I suggest to manage error and logs with options object, so anyone can handle errors and logs by his own (as fallback errored implementation will remain the same) like onWarning property of "options".

This is a simple test set to reproduce issue

const path = require("path");
const YamlValidator = require("yaml-validator");
const options = {
  log: false,
  structure: false,
  onWarning: (err, path) => {
    console.log("WARN ERROR", path, err);
  },
  writeJson: false
};
const validator = new YamlValidator(options);
validator.errored = err => {
  throw new Error(err);
};

describe("validate yaml file", function() {
  it("Should validate gateway.config", done => {
    let file = [path.join(__dirname, "./../config/gateway.config.yml")];
    validator.validate(file);
    validator.report();
    done();
  });

  it("Should validate system.config", done => {
    let file = [path.join(__dirname, "./../config/system.config.yml")];
    validator.validate(file);
    validator.report();
    done();
  });
});

I will submit a PR as soon as possible if this idea can be interesting.

Versions and environment

  • Operating system: Windows 10
  • Node.js: 8.10.0
  • npm: 5.6.0
  • yaml-validator: 2.0.0

Thank you and have some 🍇.

Work with content and not files

It's possible to add some feature to work just with plain text and not files... I want to use this to validate my custom editor control

Command line tool always exits as successful (exit code 0)

Expected behaviour

When invalid files are encountered, an exit code other than 0 should be returned.

Actual behaviour

Running the command line tool on an invalid file results in a success (exit code 0).

$ yaml-validator valid.yaml
$ echo $?
0
$ yaml-validator invalid.yaml
end of the stream or a document separator is expected at line 8, column 1:
    - {"name":"crm_id","type":"STRIN ...
    ^
$ echo $?
0

What should happen is:

$ yaml-validator invalid.yaml
end of the stream or a document separator is expected at line 8, column 1:
    - {"name":"crm_id","type":"STRIN ...
    ^
$ echo $?
1

With a non-zero result (1 in this example because it's a single file).

Versions and environment

  • Operating system: macOS
  • Node.js: v8.15.0
  • npm: 6.5.0
  • yaml-validator: fresh checkout from master

Thank you and have some 🍇.

writeJson option doesn't create a JSON file

Expected behaviour

With the writeJson option enabled, it should create a JSON file with the same name of the YAML being validated.

Actual behaviour

The JSON contents are being written in the YAML file that was read, overwriting it completely.

const Validator = require('yaml-validator');

const valid = new Validator({writeJson: true});
valid.validate(['person1.yaml']);
valid.report();

After that, person1.yaml has, actually, all the JSON contents that should be in a person1.json.

Versions and environment

  • Operating system: OS X High Sierra
  • Node.js: 8.9.4
  • npm: 5.7.1
  • yaml-validator: 1.2.0

Thank you and have some 🍇.

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: Cannot find preset's package (github>whitesource/merge-confidence:beta)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.