Similar project about junit-annotate-buildkite-plugin HOT 13 CLOSED

I have wondered if we should just support both .yml and .json?

Meh. IMO yaml > json any day if I have to manually write the thing. But I wouldn't fight someone who was determined to write json.

from junit-annotate-buildkite-plugin.

@micheletest: Is this roughly what you were thinking bugcrowd/test-summary-buildkite-plugin#3? If you do end up trying out the bugcrowd one, I'd be nice to hear how it goes. I'm curious to see how it holds up in something other than the project I built it for.

@toolmantim: So I've found a couple bugs with the annotation server-side stuff. Where would be the best place to report them? So far I have a bug with the markdown renderer, #<job_uuid> links don't work from annotations and there's a CSS padding issue with nested <details> elements.

from junit-annotate-buildkite-plugin.

@tessereth excellent, that looks great. I'll report back how it goes when I get a chance to try out the bugcrowd plugin.

from junit-annotate-buildkite-plugin.

@tessereth if you could email support@buildkite with those bugs, that’d be great!

from junit-annotate-buildkite-plugin.

Wow @tessereth — that plugin is super impressive! It's really well written, tested and documented. It's awesome. Would you be interested in moving it to the buildkite-plugins org, and deprecating this one?

I noticed you're using a larger Ruby image, where I tried to stick with the Alpine version here. But it's hard to use nokogiri and friends, so it's probably worth just using the Ruby one in the end.

Let me know if you'd like anything clarified, or would like to collab. I'll send you a PR that adds the new https://github.com/buildkite-plugins/plugin-linter-buildkite-plugin and plugin.yml.

from junit-annotate-buildkite-plugin.

I'm glad you liked it 🙂

Would you be interested in moving it to the buildkite-plugins org, and deprecating this one?

In the back of my mind, that was my optimal end-goal for this project so yeah, I'm definitely down. We should probably add some recognition of Bugcrowd in the readme or something though, given I built it on work time for a hackathon.

I noticed you're using a larger Ruby image

I'm using that image and nokogiri just because it got the job done and I'm pretty new to docker. Junit isn't exactly a complicated XML format so I could get behind using something smaller.

I'll send you a PR

More than happy to collaborate on this. Anything that makes this a real thing used by real people rather than just "a thing I built for a hackathon once" sounds great to me.

from junit-annotate-buildkite-plugin.

So plugin.yml and plugin-linter... Both of these seem like a great idea. Just a couple things from my playing around:

1. The docs still refer to plugin.json. You might want to fix that.
2. If the configuration is not a valid json schema, plugin-linter exits with no output. It probably wouldn't hurt to log exceptions
3. The linter doesn't detect the example in my readme because it's not in an explicit steps:. This may or may not be something you care about.
4. Is the actual configuration used compared against the json schema in plugin.yml at runtime? Or is it going to be at some point? Because that would be sweet. It doesn't check it for me but we're running an old beta version of the agent (for reasons I don't remember) so I can't actually tell.

from junit-annotate-buildkite-plugin.

The docs still refer to plugin.json. You might want to fix that.

This is some very fresh paint (as of the last 7 days) so I'm still waiting to lock it down 100%. But we're close, and then we'll get those docs updated. You're the first to adopt it (we haven't even rolled it out across all our plugins yet)

If the configuration is not a valid json schema, plugin-linter exits with no output. It probably wouldn't hurt to log exceptions

Hrmm that's no good. I'll add a test for that! If I can't recreate it, I might ask for an example.

The linter doesn't detect the example in my readme because it's not in an explicit steps:. This may or may not be something you care about.

Interesting! Yeah, probably not a big deal, because I like the readability of what you've done.

In time we're hoping to have a plugin directory, and have the params documented there, so no need to add all that to the readme, you can just link to the plugin directory.

The JSON spec can contain the description, examples and defaults in it too (see buildkite-plugins/docker-compose-buildkite-plugin@bf75a21). I removed it in the end, because it's a pain to maintain both that, and the readme version.

Is the actual configuration used compared against the json schema in plugin.yml at runtime? Or is it going to be at some point? Because that would be sweet. It doesn't check it for me but we're running an old beta version of the agent (for reasons I don't remember) so I can't actually tell.

We don't use it for that, but that's the idea! First step was getting some usable configuration schema's in there. Next step: agent runtime validation support.

from junit-annotate-buildkite-plugin.

The JSON spec can contain the description, examples and defaults in it too

Hm, interesting. But I get your point. I kinda like the flexibility to document it how I like in the readme rather than auto-generating it or just pointing to the yml file. And then maintaining both would suck. I think leaving the descriptions out of the json-schema makes sense.

(Random aside: do you ever feel weird talking about a json schema when you're writing it in yaml to verify yaml? I mean, it works fine, right, because it's all the same data structures but there's no actual json involved...)

Next step: agent runtime validation support

Okay, makes sense. But having agent runtime validation support would be a nice feature. Currently if you give a bad data structure to mine, you get undefined method [] for nil:NilClass or whatever. Not super helpful but adding that validation myself is somewhat tedious.

from junit-annotate-buildkite-plugin.

Hm, interesting. But I get your point. I kinda like the flexibility to document it how I like in the readme rather than auto-generating it or just pointing to the yml file. And then maintaining both would suck. I think leaving the descriptions out of the json-schema makes sense.

Ok cool, that makes me comfortable with where we ended up.

I wonder if I could make the linter validate your readme based on the configuration still, especially given it's nested? 🤔 We found with the docker plugins it was easy to forget to add docs for a configuration option, especially if it was a third-party PR.

(Random aside: do you ever feel weird talking about a json schema when you're writing it in yaml to verify yaml? I mean, it works fine, right, because it's all the same data structures but there's no actual json involved...)

YES! 100%. But it reads so much nicer. I have wondered if we should just support both .yml and .json? 🤔

But having agent runtime validation support would be a nice feature.

100%. We just have to get @lox excited enough to have a stab (😉😉😉 @lox)

from junit-annotate-buildkite-plugin.

I was just looking to use this buildkite provided plugin today and ran across this issue. I like the bugcrowd plugin, looks great, especially the ability to label and group different types of tests separately. My two cents, but it would be nice to port over the job-uuid-file-pattern setting, or something similar, to the bugcrowd one so that the job link annotation works.

from junit-annotate-buildkite-plugin.

Hm, yeah I do like the idea. It makes a lot of sense when you have parallel builds. I'm still trying to think of a good way to do handle it though when there's only one build for a given group. It seems rather unnecessary to include a link to the build from every failure when obviously they're all going to the same place.

from junit-annotate-buildkite-plugin.

After so long, the purpose and features of the plugin listed here have diverged quite a lot so I don't think that a merge is possible at this time. I also believe that they serve different purposes.

Will close this issue as there is no immediate action to take on the matter. Whatever plugin users prefer to use is their choice and we would encourage opening issues in this repository to suggest missing features or functionality if they are using this one :)

from junit-annotate-buildkite-plugin.