I would like to keep newlines in block scalars for better readability.
But yamlfmt (v0.5.0)'s basic formatter changes block scalar with > into one line.

e.g.

commands: >
  [ -f "/usr/local/bin/foo" ] &&
  echo "skip install" ||
  go install github.com/foo/foo@latest

yamlfmt make this file like below

commands: >
  [ -f "/usr/local/bin/foo" ] && echo "skip install" || go install github.com/foo/foo@latest

Description

yamlfmt convert emoji to Unicode codepoints.
This is breaking change to yaml.

Reproduce steps

1. prepare yaml.

ascii: this is acsii
# https://emojipedia.org/smiling-face-with-smiling-eyes/ https://codepoints.net/U+1F60A
# https://emojipedia.org/party-popper/ https://codepoints.net/U+1F389
emoji: 😊 🎉

2. Run yamlfmt.
3. See changed. Emoji convert to it's Unicode.

ascii: this is acsii
# https://emojipedia.org/smiling-face-with-smiling-eyes/ https://codepoints.net/U+1F60A
# https://emojipedia.org/party-popper/ https://codepoints.net/U+1F389
emoji: "\U0001F60A \U0001F389"

-dry diff

$ ls
foo.yaml

$ yamlfmt -dry
foo.yaml:
  ascii: this is acsii
  # https://emojipedia.org/smiling-face-with-smiling-eyes/ https://codepoints.net/U+1F60A
  # https://emojipedia.org/party-popper/ https://codepoints.net/U+1F389
- emoji: 😊 🎉
+ emoji: "\U0001F60A \U0001F389"

Version Info

• yamlfmt v0.3.0
• OS: Windows 10 and 11

Sometimes yaml blocks will have a line break between them for human readability, for example:

block_a:
  b: "some stuff"

block_b:
  a: "some more info"

Today, this will end up being formatted like this:

block_a:
  b: "some stuff"
block_b:
  a: "some more info"

It would be great not to remove these if possible.

When running the tool, it will fail if there is a folder with .yml or .yaml in the name.

Maybe we should we skip folders when attempting for format yaml files?

Many projects has its own rule specifying a maximum line length of yaml file (e.g. 80), and, for example, yamllint supports a feature to configure it via this feature.
There are cases that a line has to be split into multiple lines li this in order to comply with the rule

      KEY: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\
        xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

, but yamlfmt automatically formats it to

      KEY: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

, then the line violates the rule, and yamllint fails.
It would be helpful if yamlfmt can support a feature to specify the maximum line length in config file.

Right now I'm simply return erring everywhere. I'd like to do some better error wrapping so it's easier to tell where the error is coming from.

Hi there! I came across your project when looking for a yaml formatter to use in a project and I encountered some unexpected behaviour that might be useful to you.

The project I'm contributing to the yaml is parsed using Ruby, so there are tags in the yaml to help Ruby map yaml to classes. The problems below aren't caused by those tags, but they make small problems more painful than they'd be otherwise (problem 2 below in particular).

Problem 1 - inconsistent handling of comments in files containing multiple yaml documents

If a file contains multiple yaml documents starting with --- then the comments are reordered differently at the start of the file versus elsewhere. This seems to be due to the first --- of the file being prioritised as line 1 of the output.

This file has two documents with a consistent format:

# Set.new([1,2]).to_yaml - first
--- !ruby/object:Set
hash:
  1: true
  2: true

# Set.new([1,2]).to_yaml - second
--- !ruby/object:Set
hash:
  3: true
  4: true

(Adapted from https://rhnh.net/2011/01/31/yaml-tutorial/ - example shows yaml output from Ruby code)

The formatter introduces differences between the comment positions in the first and subsequent entries:

---
!ruby/object:Set
# Set.new([1,2]).to_yaml - first
hash:
  1: true
  2: true

# Set.new([1,2]).to_yaml - second
---
!ruby/object:Set
hash:
  3: true
  4: true

Problem 2 - tags being moved above comments at start of file

In the example above you can see that the first !ruby/object:Set tag is 'pulled' up to the start of the file, above the comment which was originally line 1. This can be an issue if the yaml file starts with a copyright notice (or any other comment that needs to be at the top of the file) because the tag is separated from the yaml and this obscures how the file works.

Original file:

# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http:#www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Set.new([1,2]).to_yaml - first
--- !ruby/object:Set
hash:
  1: true
  2: true

# Set.new([1,2]).to_yaml - second
--- !ruby/object:Set
hash:
  3: true
  4: true

A yamlfmt moves the tag away from the yaml:

---
!ruby/object:Set
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http:#www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Set.new([1,2]).to_yaml - first
hash:
  1: true
  2: true

# Set.new([1,2]).to_yaml - second
---
!ruby/object:Set
hash:
  3: true
  4: true

Ideally (blocks of) comments at the start of a file would be given a special status and would still be the first lines of the file after formatting.

I appreciate you're a sole maintainer and the project is in v0.x.x at the moment, so I'm not expecting a fix in response! However I thought these examples and info about reproducing problems would be helpful to you.

Happy new year! 👋

Starting with v0.7.0, yamlfmt prints an extra hyphen when called with the - argument (read from stdin, output to stdout):

Is that intentional or is that a regression?

Unfortunately, this leads to things breaking downstream. See jose-elias-alvarez/null-ls.nvim#1331 for an example.

When integrating this into our repo we run a wrapped yamlfmt as part of a suite of misc checks. In our case the diff piped to stdout can be quite large and not very interesting in my opinion, the more helpful approach in our case would be to just provide the path of the files with incorrect formatting. Would it be possible to add this behavior under some flag?

Hello, thanks for a really neat and fast tool! Seems to work fantastic in our mono-repo.

I noticed something strange though when formatting this minimal example:

Foobar:
  baz: >
    Lorem Ipsum is simply dummy text of the printing_and_typesetting industry.

Foobaz:
  baz: "foobar"

which will get formatted into:

Foobar:
  baz: >
    Lorem Ipsum is simply dummy text of the printing_and_typesetting industry. #magic___^_^___line
Foobaz:
  baz: "foobar"

with the following config:

formatter:
  type: basic
  indent: 2
  retain_line_breaks: true

I guess there is some problem in internal/hotfix/retain_line_break.go but could not see where.

I'd like to be able to ensure a YAML document doesn't include any anchors or aliases: https://yaml.org/spec/1.2.2/#3222-anchors-and-aliases, e.g.:

definitions: 
  steps:
    - step: &build-test
        name: Build and test
        script:
          - mvn package
        artifacts:
          - target/**

pipelines:
  branches:
    develop:
      - step: *build-test
    main:
      - step: *build-test

(from here: https://support.atlassian.com/bitbucket-cloud/docs/yaml-anchors/)

It doesn't look like that's currently possible with this tool, but would you accept a PR to add it?

Practically speaking I think this would look like checking for Node.Kind == yaml.AliasNode, or Node.Anchor != "", defined in the Node type. The option would probably default to false for compatibility.

I am very interested in this tool.

As written in the README.md, .yamlfmt configuration file is automatically loaded from the working directory, but it doesn't seem to have a feature to optionally specify the path of the config file to be loaded. For example, other tool such as yamllint has an option -c like this:

yamllint -c path/to/yamllint.yaml foo.yaml

https://github.com/adrienverge/yamllint#usage

yamlfmt seems to have the following three options only at this moment.

$ ./yamlfmt --help
Usage of ./yamlfmt:
  -dry
    	Perform a dry run; show the output of a formatting
    	operation without performing it.
  -in
    	Format yaml read from stdin and output to stdout
  -lint
    	Check if there are any differences between
    	source yaml and formatted yaml.

I know my sudden post and request might be impolite, but may I ask a favor of you to add a feature to optionally specify the path of config file to be loaded?

I did the v0.1.0 release manually, but it would be much preferred if the releases could be automated through GitHub Actions using the existing goreleaser configuration.

First of, thanks for the amazing tool! :D

In many CLI programs is usual that stdin is configured automatically for input if no arguments were added. But, in many others (were no arguments means print help) it is (almost) standard to have - mean read from stdin.

I know is not important, but I propose follow the semantics of others, as I think allow for a (a little bit) more seamless experience. :)

I'm using some formatters like black, stylua,.
I can use 'global' configurations for them, stored in a home-directory as fallback, like below.

• ~/.config
  • black/pyproject.toml
  • stylua/styula.toml

I can manage them in one repository (it holds a part of ~/.config) conveniently.
Would you consider supporting such a global configuration file?

It would be nice to be able to support reading from stdin, which would enable the yamlfmt command to be used with | operators. Could open up some interesting uses with editor integration or yq.

The diff reporter always reports the left and right side of every diff, however sometimes the same line is the right side of one diff and the left side of another. Need to think of another way to report the diffs that doesn't require significant mental untangling.

The Unicode parser hotfix can be removed when go-yaml/yaml#737 is closed.

go get github.com/google/yamlfmt/cmd/yamlfmt@latest
build github.com/google/yamlfmt/cmd/yamlfmt: cannot load io/fs: malformed module path "io/fs": missing dot in first path element

Go version 13.

I use yamlfmt in my NVIM setup. I want to avoid creating a config.yaml and instead configure the formatting behaviour through the CLI args instead. Most other formatters already support this.

For example, to set the max_line_length, I just wish to do the following:

yamlfmt --max-line-length=80 in.yaml

Yaml support indented or not array.

array:
  - a
  - b

array:
- a
- b

It would be great if the basic formater had an option to remove spaces before hyphens.

While investigating #84 I realized that the yaml library parses weirdly when there is whitespace at the end of lines in a folded block scalar.

When scan_folded_as_literal: false, you get the original bug shown in issue #84.

When scan_folded_as_literal: true, you get the following with the same input:

Foobar:
  baz: "Lorem Ipsum is simply dummy text of the printing_and_typesetting industry.
    \n#magic___^_^___line\n"
Foobaz:
  baz: "foobar"

Will need to figure out why whitespace at the end of the line causes the library to think it's not printable.

I've just started using yamlfmt since v0.4.0, so I don't know how it worked before, but I think this issue was introduced in v0.4.0 because of the new diff library.

https://github.com/RageCage64/multilinediff seems to always return non-empty string but

Similarly, -dry always outputs all contents.

When I launched this repo a few days ago, I did not expect this amount of attention (that is to say I didn't expect any at all). If you're reading this, thank you very much for your interest! I apologize for being unprepared for this amount of attention, and I'm going to expedite the work on some basic housekeeping, starting with releasing v0.1 so the tool can be used without go install.

PRs should pass all available tests* along with the already existing CLA check. Some other things should be checked as well, such as go mod tidy, goimports, and go vet (there's a good chance some go vet checks are failing right now).

• As of creating this issue there are very few tests but more will be added soon

I didn't realize that the gopkg.in/yaml.v2 package stripped comments. I'll probably swap libraries for the basic formatter to github.com/goccy/go-yaml which provides a way to preserve comments.

I already have things like indent size configured in .editorconfig. I'd love to have yamlfmt use those settings rather than duplicate them in .yamlfmt.

Would you be open to a PR that includes editorconfig-core-go and uses .editorconfig values as defaults?

I have the following YAML:

---
Resources:
  ElasticBeanstalk:
    Type: AWS::CloudFormation::Stack
    Properties:
      #NotificationARNs: 
      #  - String
      #Parameters: 
      #  Key : Value
      #Tags: 
      #  - Tag
      TemplateURL: https://examplebucket.s3.ap-southeast-1.amazonaws.com/example-file.json
      TimeoutInMinutes: 5

after formatting, each commented line will have a blank line inserted, becoming:

Resources:
  ElasticBeanstalk:
    Type: AWS::CloudFormation::Stack
    Properties:
      #NotificationARNs: 

      #  - String

      #Parameters: 

      #  Key : Value

      #Tags: 

      #  - Tag
      TemplateURL: https://examplebucket.s3.ap-southeast-1.amazonaws.com/example-file.json
      TimeoutInMinutes: 5

I don't see any option to configure the formatter so that it does not include a blank line after each comment.

Thank you.

I've made the decision to fork go-yaml. This will enable me to make the patches I need available for yamlfmt, as the upstream yaml.v3 maintainers seem uninterested in them. This will enable me to actually have some control over the yaml serialization in the basic formatter and opens up numerous opportunities for features and fixes.

The fork is at https://github.com/braydonk/yaml

Hi, I have seen some configs around the web using compact_sequence_indent. For example, in this issue : #27.

However, I cannot find this field in the basic formatter config, in the go code. Using it in my file also does not keep the sequence indentation compact.

Has this been removed?

If the basic formatter is configured like so:

formatter:
  include_document_start: true

The default value of 2 for Indent is lost, and the indentation will end up being the yaml.v3 default of 4.

When running yamlfmt without any .yamlfmt, the --formatter flag may lead to a panic :

$ rm .yamlfmt

$ yamlfmt --formatter retain_line_breaks=true
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x119228f]
goroutine 1 [running]:
main.makeCommandConfigFromData(0x12156e0?)
	github.com/google/yamlfmt/cmd/yamlfmt/config.go:194 +0x18f
main.run()
	github.com/google/yamlfmt/cmd/yamlfmt/main.go:55 +0x17b
main.main()
	github.com/google/yamlfmt/cmd/yamlfmt/main.go:27 +0x19

This also happens when .yamlfmt contains no override, but with a different panic message :

$ cat .yamlfmt
formatter:
  type: basic

$ yamlfmt --formatter retain_line_breaks=true
panic: assignment to entry in nil map
goroutine 1 [running]:
main.makeCommandConfigFromData(0xc000024300?)
	github.com/google/yamlfmt/cmd/yamlfmt/config.go:194 +0x19f
main.run()
	github.com/google/yamlfmt/cmd/yamlfmt/main.go:55 +0x17b
main.main()
	github.com/google/yamlfmt/cmd/yamlfmt/main.go:27 +0x19

No issue when .yamlfmt contains at least one override :

$ cat .yamlfmt
formatter:
  type: basic
  retain_line_breaks: false

$ yamlfmt --formatter retain_line_breaks=true

When running yamlfmt basic formatting with a config file and targeting several paths in a repository, the formatter spuriously deletes backslashes and thereby breaks the semantics of those YAML files. For example

outer:
    - inner: "python3 -m some.module \\${VARIABLE_:-}" 

becomes

outer:
    - inner: "python3 -m some.module ${VARIABLE_:-}" 

and

outer:
  - name: something
    regex: '\(\d+, "Unknown database .*'

becomes

outer:
  - name: something
    regex: '(d+, "Unknown database .*'

which are not semantically identical. Strangely enough, targeting these individual files with the formatter doesn't change them this way. Only running the formatter over a glob of YAML paths in my directory has this effect.

Unfortunately, I can't attach the entire directory as it's proprietary. But I'm hoping you have some insights as to why this might happen. I'll try to create a sanitized reproduction if possible.

I have this:

doublestar: true
exclude:
  - **/templates/**
formatter:
  type: basic
  indent: 2
  include_document_start: true

And I am doing this to prevent the yamlfmt from formatting my templates/ folder that are in my helm charts, since it messes everything up.

However, it keeps complaining that:
2023/03/24 14:38:02 yaml: line 3: did not find expected alphabetic or numeric character.

So I put line 3 in quotes "- **/templates/**" and now the error goes away. However, it doesn't respect my excludes and continues to format things in the templates/ folder.

When the upstream bugs go-yaml/yaml#891 and go-yaml/yaml#893 are resolved, I will revisit the CRLF hotfix and use the fixed features instead.

Right now calling the tool with -h prints the flag package default help message. This is okay, but it doesn't include any information about the usage with the argument functionality, only flags. An improved help message would be an improvement to new user experience so they don't have to come back to this repo to figure out how to use the tool.

Pull Request and Issue templates should be created featuring some of the basic information requirements.

I have the following block:

- name: Configure and restart kubelet
  when: >
    ( _allow_disruption | default(False)
        or do_upgrade | default(False)
        or _init_cluster | default(False))
    and not k8s_kubelet_disable_customizations | default(False)

After each run of yamlfmt, a new line is added like so:

- name: Configure and restart kubelet
  when: >
    ( _allow_disruption | default(False)

        or do_upgrade | default(False)
        or _init_cluster | default(False))
    and not k8s_kubelet_disable_customizations | default(False)

- name: Configure and restart kubelet
  when: >
    ( _allow_disruption | default(False)


        or do_upgrade | default(False)
        or _init_cluster | default(False))
    and not k8s_kubelet_disable_customizations | default(False)

- name: Configure and restart kubelet
  when: >
    ( _allow_disruption | default(False)



        or do_upgrade | default(False)
        or _init_cluster | default(False))
    and not k8s_kubelet_disable_customizations | default(False)

I'm not sure if this might be related to #63

I use yamllint --strict to check my YAML files, and it checks one's YAML fits into 80-chars.

It looks like yamlfmt is wrapping my lines to be longer than 80-chars. Looking at the Basic Formatter, I don't see a config option to change this line wrap.

It would be nice to add line_length as a config option to the formatter. Cheers!

I have a file, namespace/collection/meta/runtime.yml that is generated by the Ansible Galaxy collection init step, that starts off like so:

# Collections must specify a minimum required ansible version to upload
# to galaxy
# requires_ansible: '>=2.9.10'

# Content that Ansible needs to load from another location or that has
# been deprecated/removed
# plugin_routing:
#   action:

And continues with only blank or commented lines.

When I run yamlfmt -dstar '**/*yml' from the top of the directory tree, yamlfmt deletes all of the content in the file, resulting in a 0 byte file.

If I manually add a --- separator at the start of the file, then yamlfmt -dstar '**/*yml' does not strip all of the content. It does, however, strip the --- so the next run empties the file.

If I manually add a --- to the file and use this config file:

formatter:
  type: basic
  include_document_start: true

then the file is not emptied.

I see two things that might be addressed:

• fix the yaml parser/writer so that it doesn't not strip comments from empty files; and
• change the default for include_document_start to true (this gives me a work around).

Otherwise it's a little too easy to run it in the wrong directory and not even know what's changed.

Right now the -dry and -lint diff outputs are the direct result from cmp.Diff. This gets the message across but it's very hard to read. The most important first step would be removing all the string array display elements, and the double quotes in each line. It's not going to be trivial, since the yaml could contain double quotes, and I'm not entirely sure what happens in that scenario.

Following the currently suggested install instructions: go install github.com/google/yamlfmt/cmd/yamlfmt@latest

> go install github.com/google/yamlfmt/cmd/yamlfmt@latest
go: downloading github.com/google/yamlfmt v0.5.0
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading github.com/RageCage64/multilinediff v0.2.0
go: downloading github.com/mitchellh/mapstructure v1.5.0
go: downloading github.com/bmatcuk/doublestar/v4 v4.2.0
go: downloading github.com/RageCage64/go-utf8-codepoint-converter v0.1.0
go: downloading github.com/google/go-cmp v0.5.9
> yamlfmt
zsh: command not found: yamlfmt

The request is to have working install instructions in the README. Excited to start using yamlfmt!

Providing the following yaml file:

a:
 b: true

Results in the following lint output:

.\tmp\a\y.yaml:
2023/03/11 13:00:55 The following formatting differences were found:

.\tmp\a\y.yaml:
- a:
 b: true  a:
+                b: true
+

The new option is actually indentless_arrays according to the source code, not the indentless_array in config.md

I come in peace, from the Python world 👽

If you could somehow get this to work with pre-commit, it would be sick! You could get lots of new adopters...

The default Unmarshal behaviour in yaml.v3 will only read and output the first document in a multi-document yaml file. I was unaware that this was possible, let alone common in Kubernetes so I did not test it initially.

The way to make yaml.v3 parse these properly is to to make a Decoder for the whole document, and decode them one at a time in a loop. This behaviour might diverge heavily from the basic formatter, so it might need to be an entirely new one, but I will see if I can make it work by default.

When a backslash is used in a value, yamlfmt basic formatter will strip it away.

Before:

a: \*

After:

b: *

So when you set emoji_support to true, adding a test case to the unicode_test.go:

		{
			name: "literal string",
			yamlStr: `a: |
  hello 😄\n`,
			expectedStr: `a: |
  hello 😄\n`,
		},

Which results:

parsed string does not match:
        expected: a: |
          hello 😄\n
        got: a: "hello 😄n"

I suppose the literal string turns one line due to the go-yaml library. However, the tail n is unacceptable.