Comments (13)
I have wondered if we should just support both .yml and .json?
Meh. IMO yaml > json any day if I have to manually write the thing. But I wouldn't fight someone who was determined to write json.
from junit-annotate-buildkite-plugin.
@micheletest: Is this roughly what you were thinking bugcrowd/test-summary-buildkite-plugin#3? If you do end up trying out the bugcrowd one, I'd be nice to hear how it goes. I'm curious to see how it holds up in something other than the project I built it for.
@toolmantim: So I've found a couple bugs with the annotation server-side stuff. Where would be the best place to report them? So far I have a bug with the markdown renderer, #<job_uuid>
links don't work from annotations and there's a CSS padding issue with nested <details>
elements.
from junit-annotate-buildkite-plugin.
@tessereth excellent, that looks great. I'll report back how it goes when I get a chance to try out the bugcrowd plugin.
from junit-annotate-buildkite-plugin.
@tessereth if you could email support@buildkite with those bugs, thatβd be great!
from junit-annotate-buildkite-plugin.
Wow @tessereth βΒ that plugin is super impressive! It's really well written, tested and documented. It's awesome. Would you be interested in moving it to the buildkite-plugins org, and deprecating this one?
I noticed you're using a larger Ruby image, where I tried to stick with the Alpine version here. But it's hard to use nokogiri and friends, so it's probably worth just using the Ruby one in the end.
Let me know if you'd like anything clarified, or would like to collab. I'll send you a PR that adds the new https://github.com/buildkite-plugins/plugin-linter-buildkite-plugin and plugin.yml.
from junit-annotate-buildkite-plugin.
I'm glad you liked it π
Would you be interested in moving it to the buildkite-plugins org, and deprecating this one?
In the back of my mind, that was my optimal end-goal for this project so yeah, I'm definitely down. We should probably add some recognition of Bugcrowd in the readme or something though, given I built it on work time for a hackathon.
I noticed you're using a larger Ruby image
I'm using that image and nokogiri just because it got the job done and I'm pretty new to docker. Junit isn't exactly a complicated XML format so I could get behind using something smaller.
I'll send you a PR
More than happy to collaborate on this. Anything that makes this a real thing used by real people rather than just "a thing I built for a hackathon once" sounds great to me.
from junit-annotate-buildkite-plugin.
So plugin.yml and plugin-linter... Both of these seem like a great idea. Just a couple things from my playing around:
- The docs still refer to
plugin.json
. You might want to fix that. - If the
configuration
is not a valid json schema,plugin-linter
exits with no output. It probably wouldn't hurt to log exceptions - The linter doesn't detect the example in my readme because it's not in an explicit
steps:
. This may or may not be something you care about. - Is the actual configuration used compared against the json schema in
plugin.yml
at runtime? Or is it going to be at some point? Because that would be sweet. It doesn't check it for me but we're running an old beta version of the agent (for reasons I don't remember) so I can't actually tell.
from junit-annotate-buildkite-plugin.
The docs still refer to plugin.json. You might want to fix that.
This is some very fresh paint (as of the last 7 days) so I'm still waiting to lock it down 100%. But we're close, and then we'll get those docs updated. You're the first to adopt it (we haven't even rolled it out across all our plugins yet)
If the configuration is not a valid json schema, plugin-linter exits with no output. It probably wouldn't hurt to log exceptions
Hrmm that's no good. I'll add a test for that! If I can't recreate it, I might ask for an example.
The linter doesn't detect the example in my readme because it's not in an explicit steps:. This may or may not be something you care about.
Interesting! Yeah, probably not a big deal, because I like the readability of what you've done.
In time we're hoping to have a plugin directory, and have the params documented there, so no need to add all that to the readme, you can just link to the plugin directory.
The JSON spec can contain the description, examples and defaults in it too (see buildkite-plugins/docker-compose-buildkite-plugin@bf75a21). I removed it in the end, because it's a pain to maintain both that, and the readme version.
Is the actual configuration used compared against the json schema in plugin.yml at runtime? Or is it going to be at some point? Because that would be sweet. It doesn't check it for me but we're running an old beta version of the agent (for reasons I don't remember) so I can't actually tell.
We don't use it for that, but that's the idea! First step was getting some usable configuration schema's in there. Next step: agent runtime validation support.
from junit-annotate-buildkite-plugin.
The JSON spec can contain the description, examples and defaults in it too
Hm, interesting. But I get your point. I kinda like the flexibility to document it how I like in the readme rather than auto-generating it or just pointing to the yml file. And then maintaining both would suck. I think leaving the descriptions out of the json-schema makes sense.
(Random aside: do you ever feel weird talking about a json schema when you're writing it in yaml to verify yaml? I mean, it works fine, right, because it's all the same data structures but there's no actual json involved...)
Next step: agent runtime validation support
Okay, makes sense. But having agent runtime validation support would be a nice feature. Currently if you give a bad data structure to mine, you get undefined method [] for nil:NilClass
or whatever. Not super helpful but adding that validation myself is somewhat tedious.
from junit-annotate-buildkite-plugin.
Hm, interesting. But I get your point. I kinda like the flexibility to document it how I like in the readme rather than auto-generating it or just pointing to the yml file. And then maintaining both would suck. I think leaving the descriptions out of the json-schema makes sense.
Ok cool, that makes me comfortable with where we ended up.
I wonder if I could make the linter validate your readme based on the configuration still, especially given it's nested? π€ We found with the docker plugins it was easy to forget to add docs for a configuration option, especially if it was a third-party PR.
(Random aside: do you ever feel weird talking about a json schema when you're writing it in yaml to verify yaml? I mean, it works fine, right, because it's all the same data structures but there's no actual json involved...)
YES! 100%. But it reads so much nicer. I have wondered if we should just support both .yml and .json? π€
But having agent runtime validation support would be a nice feature.
100%. We just have to get @lox excited enough to have a stab (πππ @lox)
from junit-annotate-buildkite-plugin.
I was just looking to use this buildkite provided plugin today and ran across this issue. I like the bugcrowd plugin, looks great, especially the ability to label and group different types of tests separately. My two cents, but it would be nice to port over the job-uuid-file-pattern setting, or something similar, to the bugcrowd one so that the job link annotation works.
from junit-annotate-buildkite-plugin.
Hm, yeah I do like the idea. It makes a lot of sense when you have parallel builds. I'm still trying to think of a good way to do handle it though when there's only one build for a given group. It seems rather unnecessary to include a link to the build from every failure when obviously they're all going to the same place.
from junit-annotate-buildkite-plugin.
After so long, the purpose and features of the plugin listed here have diverged quite a lot so I don't think that a merge is possible at this time. I also believe that they serve different purposes.
Will close this issue as there is no immediate action to take on the matter. Whatever plugin users prefer to use is their choice and we would encourage opening issues in this repository to suggest missing features or functionality if they are using this one :)
from junit-annotate-buildkite-plugin.
Related Issues (20)
- Bump MAX_SIZE HOT 2
- fail-build-on-error not respected if annotation is too large
- issue with using junit annotate with bk local run HOT 1
- Docker is seg faulting on Google Compute agents when running junit-annotate-buildkite-plugin HOT 2
- feature request: mode to post number of tests successful when all are HOT 2
- bug: glob should fail if 0 files are returned? HOT 1
- ruby: No such file or directory -- /src/bin/annotate (LoadError) HOT 8
- Feature Request: Ability to specify Ruby docker image to use HOT 1
- Sometimes, at some point s3 hits a lot of 404s and task takes 10+ minutes HOT 1
- Don't show failed tests from previous run, if the current run has passed successfuly HOT 1
- Dependency Dashboard
- Update annotation as parallel jobs complete commands HOT 1
- only list the failed test methods only for large junit reports
- Error while Processing the junits: docker: invalid reference format. HOT 1
- junit-annotate#v2.2.0 docker rate limit issue with ruby:2.7-alpine. HOT 1
- How to prevent "The failures are too large to create a build annotation. Please inspect the failed JUnit artifacts manually."
- Feature request: Ability to show also skipped tests in annotations
- Escape HTML entities in failure messages, not just the body
- Feature Request: Support running the plugin without docker HOT 1
- FEATURE - Supporting JSON buildkite annotations HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from junit-annotate-buildkite-plugin.