Noticed we mention projectId in some places, whereas it should be projectID

For reference https://medium.com/cloudstory/running-load-testing-at-scale-on-shoestring-budget-b7ea34e4d5db

When the summary export was written no one noticed that the boolean used to say whether a threshold has passed or not is true when it failed and false when it has passed, which confusing to users.

I think this should be documented better and we should have an example of a threshold

links:
original PR
community forum link
k6 issue

After we merge this k6 WIP PR: grafana/k6#1396 (grafana/k6#1353), but before we release k6 v0.28.0, someone has to write a docs page that explains plugins, and maybe has a very short and simple tutorial on how to write one. We should then link to that page from the v0.28.0 release notes.

Also prompted by grafana/k6#1595

Edit: as I've said many times, it's very important to stress in that docs page that this is a beta-quality feature. The current plugin API (and k6 APIs they will depend on) are not guaranteed to be stable, and plugins may need changes when we update k6.

We'd like to have k6 as a part of the load testing page on Wikipedia. The page maintainer would like to have a dedicated page per https://en.wikipedia.org/w/index.php?title=Load_testing&action=history. Once this happens, we should be able to add k6 to the load testing page and resolve https://github.com/loadimpact/k6/issues/301

https://k6io.slack.com/archives/C3E688J8Z/p1597785761016100 made me realize we don't have any place in the docs we can point users to when they are looking for this type of info...

And that https://www.npmjs.com/package/@types/k6 is old, 0.26.1 😞 I think we only have to add scenarios in https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/k6/options.d.ts , but I've never written a line of TypeScript and have no idea how to encode the complicated situations with different option keys possible in each scenario, based on the executor type 😭

A user commented on the approach we suggest for using k6 in GitLab CI - our examples (https://gitlab.com/loadimpact/k6-gitlab-example, https://loadimpact.com/integrations/performance-testing-gitlab/#configuration) use wget to download a very old (v0.21.1) k6 version instead of the official docker image.

cc @ppcano, @robingustafsson, @markjmeier

We've committed the cardinal sin of forums, blogs and docs - there's no way to get back to the main site...
The link on the top left of https://k6.io/docs/ leads to https://k6.io/docs/, not https://k6.io/
Same, the top left link in https://community.k6.io/ leads to https://community.k6.io/

There used to be a detailed description of how k6 options are "consolidated" at the top of the options page in the old docs, which I now see has disappeared...

In essence, we need to explain that the end options are the result of {option defaults + JSON config options + the script exported options + environment variables + CLI flags}, in increasing order of precedence. And that this result is saved back into the script options before the actual test run starts.

It seems that the link of the log tab page that comes out by searching with log is broken.

I created an issue because I don't know the correct URL.

dead links: https://k6.io/docs/cloud/analyzing-results/logs-tab

Prompted by this forum thread: https://community.k6.io/t/how-to-get-failure-statistics/1247

Whether these should be part of the docs or a tutorial on the blog is unclear, but we probably should have things like that:

• an example of generating an HTML report with the new handleSummary() function after grafana/k6#1768
• and example script that parses our raw JSON and/or CSV metrics and generates a richer HTML or PDF report than the one above
• a grafana dashboard like the one in https://github.com/loadimpact/open-source-load-testing-stack#test-result-dashboard, but with InfluxDB

https://en.wikipedia.org/wiki/Load_testing

Seems like we have an OAuth 2 example, but not OAuth 1: https://community.k6.io/t/how-to-implement-oauth-1-0a-in-k6/1235/2

We received a very detailed report from a user on Slack about various discrepancies in the docs: https://k6io.slack.com/archives/C5C0281HB/p1598167979001200

Pasting a screenshot here, since the public k6 slack doesn't keep messages forever:

There used to be a big warning in the old docs that k6 doesn't support all, or even most, node modules. This seems to have disappeared in the new docs, or at least I don't see it in https://k6.io/docs/using-k6/modules#bundling-node-modules, and it should be very, very prominent... Currently the only prominent thing there is the "Using the k6-es6 starter" section, giving the opposite impression. k6 is probably never going to be able to support node modules that directly or indirectly use the os package, or a bunch of other stdlib packages that make external network calls (at least for the foreseeable future).

grafana/k6#1067

use the rest API to stop the test

Feature Description

The K6 rest API can adjust the VU and pause the test, but it cannot stop the test. If I want to stop the test manually, I have to switch to cli again,Otherwise, I can't stop it remotely.

Suggested Solution (optional)

It would be very convenient to update this interface(PATCH /v1/status) so that it can stop testing.

It was very depressing that we merged a PR and I saw the master build fail because of a bunch of typos 😅

It is fairly common to get questions in the vain of:

1. I changed this variable here but this other VU didn't see it
2. I did some changes to this variable in setup/default but couldn't see it in default/teardown
3. Init code is executed multiple times
4. I want to do X in init code

Examples:
https://github.com/loadimpact/k6/issues/1008
grafana/k6#784
grafana/k6#1710
grafana/k6#693 <- why setup isn't modifiable between VUs
https://community.k6.io/t/token-is-not-passed-from-setup-to-general/495
https://community.k6.io/t/using-auth0-authentication/854

The https://k6.io/docs/using-k6/test-life-cycle does explain some of those but not directly enough in my opinion and given the number of questions we get it is probably better for this to be explained with examples of why it would not work and how to get what the user wants given that this approach doesn't work

Node v11.10.1 stopped working for me (not sure why).
Node v13.7 doesn't work due to one of the dependencies not working.

It would be good to specify which Node version is supported. The easiest way is to create a .nvmrc file like this
node -v > .nvmrc

The error codes list is missing information about error code 1213.

The module containing that code is here.

There may or may not be others missing; that's the one I noticed.

Thanks!

As discussed internally, we should add a high-level page in the docs that explains xk6, what it is, how to use it, etc., without going too much into details about developing an extension or showing any Go code for that matter.

This should include a large visible warning about the experimental stage of the project, no API stability guarantees, etc., and maybe a list of trusted extensions (the same ones mentioned in the upcoming article?) and link to https://github.com/topics/xk6.

This is a low priority issue because it appears to be benign.

Example page where the error appears: https://k6.io/docs/javascript-api/k6-http/batch-requests

Hi Josh,

Thank you for the suggestion! We've decided to keep the dollar signs in the scripts for now. However, we would like to proceed with @bolotskydev's idea about omitting them when copying. I'm creating a separate issue for that.

Closing this PR without merging. With that said, I'd like to thank you again for the excellent contribution. It definitely got us thinking through the UX.

Best,
Simme

Originally posted by @simskij in #146 (comment)

I think you are most suited to decide the best path forward, @bolotskydev, so I'm happy to leave this in your hands.

Similar to how the the CSV output supports it (grafana/k6#1550) and mentions it in the docs, the JSON output also supports directly producing gzipped files since grafana/k6#1114.

Requested in grafana/jslib.k6.io#4
Implemented in https://community.k6.io/t/http-get-query-params/120

I'm thinking that we need an example of this in the examples section.
@legander and @allansson; would you mind assisting with this?

Does k6 support HTTP/2 multiplexing? I am using k6 to showcase the advantage of HTTP/2 over HTTP/1.1 for non-browser flow (REST API).
One of the major benefits of HTTP/2 for non-browser flow is its ability to multiplex multiple requests on a single connection.

If in case this feature is not supported, is it in future version roadmap? Would love to see this feature.

Prompted by grafana/k6#1623 (comment), we should make sure to explain that the arrival-rate executors are going to cycle over all of the initialized VUs, given enough time, even if they only need a small fraction of them at any given point to run their allotted iterations per second.

The only current documentation about the aggregation algorithm and options resides here. It has to be expanded with better descriptions, rationale and examples in it's own section of the k6 docs.

https://k6.io/docs/results-visualization/influxdb-+-grafana#influxdb-options doesn't contain a bunch of minor options. All of the possible options can be found in here.

It's quite likely that other output options were also omitted...

As demonstrated by https://community.k6.io/t/warn-0000-error-getting-terminal-size/767 / grafana/k6#1559, k6 has some troubles when running in "git bash" on Windows. We will fix the panic, but the user experience of k6 will never be quite as good there as on other terminals, so we probably should document something like the the following usage guidelines for windows:

If you're using git bash or mintty as your terminal on Windows, you might not get the best user experience out of k6. Consider using a different terminal like Windows Terminal, PowerShell or Cmder. Alternatively, to work around the issues, you can run k6 through winpty, which should already be preinstalled in your Git Bash environment: winpty k6 run script.js.

If you're using the Windows Subsystem for Linux (WSL), you are probably going to get better experience by using the Linux k6 binary above.

For all other cases of running k6 on Windows, the normal k6 Windows binary / .msi package should work fine.

(With the understanding that as soon as we have an official signed package for the windows package manager, that will be the recommended way to install k6.)

cc @imiric

If you use the setup function to perform login steps - for example, logging in and automatically storing an SSO cookie - those cookies that are stored will not be automatically available to the main function (please contact me for a specific reproduction, if needed).

https://community.k6.io/t/k6-http-header-anomaly-how-to-config-fiddler-for-k6/137/6 made me look this up, and we don't seem to mention proxy support at all...

k6 uses Go's http.ProxyFromEnvironment, so we can pretty much copy and adapt this documentation: https://golang.org/pkg/net/http/#ProxyFromEnvironment

@mstoykov mentioned these in the chat:

https://k6.io/docs/cloud/analyzing-results/test-comparison#phase-2---baseline-testing-scaling-your-tests-and-complex-cases from the same page this is the 1st link on the page with text baseline
https://k6.io/docs/testing-guides/load-testing-websites#what-k6-does-not the link is on the bottom of the same page

also, I noticed that the "Cloud support chat" from the little help popup in the bottom right is broken, leading to https://k6.io/docs/contact instead of presumably https://k6.io/contact?

To simplify the workflow, we should change the branching strategy to trunk-based + short-lived feature branches instead of the current feature -> dev (and staging) -> master (and prod) approach.

The new branching strategy means that features will be merged directly to master, which in turn will be released in staging. Tagging a commit in master with a version number will release it to production. For instance:

master will be released to staging, but not feature/whatever.
v1.2.3 will be released to production, but not an-arbitrary-tag.

The page should have a disclaimer that this are not officially supported and might not be up to date

In an internal discussion around grafana/k6#1439 it was proposed the messages get smaller, but link to specific documentation pages that explain possible reasons and solutions to the problems.

Similar thing in SqlAlchemyErrors and AWS solution pages.

Provoked by grafana/k6#1781. We've supported that since k6 v0.26.0 (grafana/k6#1143), but we didn't document it properly in https://k6.io/docs/using-k6/options#summary-trend-stats (or maybe we did, in the old docs...), only in the v0.26.0 release notes.

Problem

Currently, stages are discussed in the Options page, which is not where I'd expect to find them. When attempting to replicate a load profile, I went straight for the Scenarios page.

Suggested solution

I think it would be worth mentioning and linking to our documentation on Stages from the Scenarios page to make it easier to find.

Expected behaviour

Enabling throw exits the specific test iteration as soon as an request error occurs: connection issues, status 4xx, status 5xxx.

I hoped that with this option I wouldn't have to write checks for every request.

Actual behaviour

Status errors don't exit the iteration, or show anything in the log.

Documented behaviour

A boolean, true or false, specifying whether to throw errors on failed HTTP requests or not.

The documented api is CookieJar.set(name, value, [options]). However, there is an example

jar.set('https://httpbin.org/cookies', 'my_cookie', 'hello world', {
    domain: 'httpbin.org',
    path: '/cookies',
    secure: true,
    max_age: 600,
});

which suggests the api is CookieJar.set(url, name, value, [options]). This is correct. Calling CookieJar.set with cookie name as the first parameter results in no cookies set.

In the course of contributing to #144, I noticed that the contents of the https://k6.io/docs/javascript-api page (or, rather, https://mdr-ci.staging.k6.io/docs/refs/pull/144/merge/javascript-api) were not being updated to include gRPC.

This seems like a bug, since from what I can see, they are generated by this query, which should include the new gRPC JS page: https://github.com/loadimpact/k6-docs/blob/d36c6d4c01a21e198fa4b37969108daea8f039c8/src/templates/docs/javascript-api.js#L97-L100

The latest develop build failed during gatsby build --prefix-paths with:

Error: ENOENT: no such file or directory, stat '.cache/google-fonts/docs/fonts

I was able to reproduce this locally, after a fresh npm install.

It seems that this plugin has been deprecated (and hidden behind a private repo?), and others are having similar issues, so we should probably abandon it. Not sure why this is only failing now, though... Probably a change on Google's side.

A suggested alternative in that issue is to use gatsby-plugin-google-fonts, but that one doesn't download fonts but links to Google's CDN in the CSS (e.g. src: url(https://fonts.gstatic.com...)). Which we might want to consider doing anyway.

As mentioned in grafana/k6#644 it is actually possible to use crypto-js with some tweaks and this might be useful substitute for people who trully need more than what k6/crypto can provide currently.

As indicated by this forum comment, people might have a wrong expectation of how VUs are scheduled and used by the new executors. Since this changed considerably in v0.27.0, we should explain this in more detail in the documentation.

Instead of this being another page filled with text, I think a few simple animations in the style of the current PixelPoint ones that show some examples of how this works in practice would be very helpful. Showing how VUs are initialized, retrieved from and returned to the global VU pool would be good to have for each executor, although seeing how VUs are reused between executors in multi-scenario tests might be more informative.

I think i've detected a sentence that doesn't flow properly.

Scenarios allow us to make in-depth configurations to how VUs and iterations are scheduled, allowing k6 scenarios enable configuring how VUs and iterations are scheduled, making it possible to model diverse traffic patterns in load tests.

Like asdf, aur, appimage, snap among others.

Should also include a disclaimer about these methods not being official

While there is currently a page on what arrival rate is and why it is better, it isn't really recommended through the documentation, so a lot of users will be unlikely to use it.

I would argue that also there should be an even more in-depth documentation/blog post/examples showing how arrival rate makes the difference between testing the SUT and playing along with it.