KATT
Docs
KATT (Klarna API Testing Tool) is an HTTP-based API testing tool for Erlang.
Use for shooting HTTP requests in a sequential order and verifying the response. Any relevant difference between expected and actual responses will cause a failure.
Install from git or https://hex.pm/packages/katt .
An example is worth a 1000 words. Have a look!
If some values are static (constants) and you want to reuse them across multiple requests, you can add one or more params like below
PARAM a_string="with some text"
PARAM a_boolean=true
PARAM a_null=null
PARAM a_float=1.1
PARAM an_integer=1
The builtin validator supports
- basic text validation
- more advanced validation of HTTP headers
- JSON validation
application/json
,application/*+json
- URL-encoded validation
application/x-www-form-urlencoded
The validator makes use of a few tags with special meaning:
"{{_}}"
Match anything including undefined (i.e. no real validation).
"{{expected}}"
Match anything but undefined (i.e. no real validation, only check existence).
"{{unexpected}}"
Match nothing (i.e. no real validation, only check lack of existence)
"{{>key}}"
Store value of the whole string (key must be unique within testcase)
"{{<key}}"
Recall stored value.
The "{{_}}"
tag can also be used as a JSON object's property in order to
validate any other additional properties.
By default, the builtin validator will allow additional properties in an object
structure, or additional items in an array structure. To counteract that
default, one can do {..., "{{_}}": "{{unexpected}}"}
or
[..., "{{unexpected}}"]
, effectively making a rule that no properties/items
are expected beyond the ones defined.
A request can also be configured via HTTP request headers:
x-katt-content-type
would set a request content-type, without sending acontent-type
HTTP headerx-katt-description
would take precedence over the transaction's descriptionx-katt-request-timeout
would take precedence over therequest_timeout
paramx-katt-request-sleep
would delay the request for a specific amount of millisecondsx-katt-transform
would call thetranform
callback with its value asid
A response can also be configured via HTTP response headers:
x-katt-content-type
would set a response (expected and actual) content-type, without expecting/receiving acontent-type
HTTP headerx-katt-transform
would call thetranform
callback with its value asid
set
will ignore the order of an array's items, and just check for existence:
{
"some_array": {
"{{type}}": "set",
"value": [1, 2, 3]
}
}
So the above would validate against JSON instances such as
{"some_array": [1, 3, 2]}
, or {"some_array": [3, 2, 1]}
,
or even {"some_array": [4, 3, 2, 1]}
unless we add {{unexpected}}
.
runtime_value
would just run code (only erlang
and shell
supported for now),
while having access to ParentKey
, Actual
, ItemsMode
and Callbacks
,
and return the expected value and matched against the actual one.
{
"rfc1123": {
"{{type}}": "runtime_value",
"erlang": "list_to_binary(httpd_util:rfc1123_date(calendar:now_to_datetime(erlang:now())))"
}
}
or in array format
{
"rfc1123": {
"{{type}}": "runtime_value",
"erlang": ["list_to_binary(",
" httpd_util:rfc1123_date(",
" calendar:now_to_datetime(",
" erlang:now()",
")))"
]
}
}
runtime_validation
would just run code (only erlang
and shell
supported for now),
while having access to ParentKey
, Actual
, ItemsMode
and Callbacks
,
and return
{pass, [{"Key", "Value"}]}
i.e. validation passed, store new param "Key" with value "Value"{not_equal, {Key, Expected, Actual}}
{not_equal, {Key, Expected, Actual, [{"more", "info"}]}}
{
"rfc1123": {
"{{type}}": "runtime_validation",
"erlang": "Expected = httpd_util:rfc1123_date(calendar:now_to_datetime(erlang:now())), case Actual =:= Expected of true -> {pass, []}; false -> {not_equal, {ParentKey, Expected, Actual}} end"
}
}
or in array format
{
"rfc1123": {
"{{type}}": "runtime_validation",
"erlang": ["Expected = httpd_util:rfc1123_date(calendar:now_to_datetime(erlang:now())),",
"case Actual =:= Expected of",
" true ->",
" {pass, []};",
" false ->",
" {not_equal, {ParentKey, Expected, Actual}}",
"end"
]
}
}
You can also fire up katt
from the CLI, with
bin/katt base_url=http://httpbin.org my_name=Joe your_name=Mike -- doc/example-httpbin.apib
If you want non-string params, use :=
as a separator e.g. my_int:=123
.
You can also output the result in JSON format, with --json
, and beautify it e.g. with python
bin/katt --json base_url=http://httpbin.org my_name=Joe your_name=Mike -- doc/example-httpbin.apib | python -m json.tool
Check bin/katt --help
for a full list of arguments.
A simple example that will make requests to a third party server:
ERL_LIBS=_build/default/deps erl $(for f in _build/default/lib/*/ebin; do echo "-pa $f"; done) -noshell -eval '
application:ensure_all_started(katt),
BlueprintFile = "doc/example-httpbin.apib",
Params = [{base_url, "http://httpbin.org"}, {my_name, "Joe"}, {your_name, "Mike"}],
io:format("~p~n", [katt:run(BlueprintFile, Params)]).
' -s init stop
... or run the code passed to -eval from the Erlang shell, assuming that you
have started the Erlang shell from the repo's root directory with
ERL_LIBS=_build/default/deps erl $(for f in _build/default/lib/*/ebin; do echo "-pa $f"; done)
.
katt:run
is to be called with
filename
params
(optional)base_url
, alternatively you can use the legacyprotocol
hostname
port
base_path
request_timeout
scenario_timeout
callbacks
(optional)ext
to be called withscope
(recall_body, parse, validate_body, validate_type)recall
to be called withsyntax
,text
,params
,callbacks
parse
to be called withheaders
,body
,params
,callbacks
request
to be called withrequest
,params
,callbacks
validate
to be called withexpected
,actual
,params
,callbacks
progress
to be called withtransaction_result
text_diff
to be called withtext
,text
transform
to be called withid
,katt_request
or{katt_response, actual_response}
,params
,callbacks
The HTTP Archive format or HAR, is a JSON-formatted archive file format for logging of a web browser's interaction with a site, standardized by the Web Performance Working Group of the World Wide Web Consortium (W3C).
For example, to convert doc/example-teapot.har into doc/example-teapot.apib, run:
bin/katt from-har --apib -- doc/example-teapot.har > doc/example-teapot.apib
OnlyText = fun(_Scope) -> [] end,
katt:run("text_only_scenario.apib", [], [{ext, OnlyText}]).
PlusXml =
fun(recall_body) ->
[ fun custom_callbacks_xml:recall_body/4
] ++ katt_callbacks:ext(recall_body);
fun(parse) ->
[ fun custom_callbacks_xml:parse/5
] ++ katt_callbacks:ext(parse);
fun(validate_body) ->
[ fun custom_callbacks_xml:validate_body/3
] ++ katt_callbacks:ext(validate_body),
fun(validate_type) ->
[ fun custom_callbacks_xml:validate_type/7
] ++ katt_callbacks:ext(validate_type),
katt:run("xml_scenario.apib", [], [{ext, PlusXml}]).
See src/katt_callbacks_json.erl to see how your
custom_callbacks_xml
module should be implemented.
export KATT_BARE_MODE=true
# or
touch _build/BARE_MODE
A pull-request is most welcome. Please make sure that the following criteria are fulfilled before making your pull-request:
- Include a description regarding what has been changed and why.
- Make sure that the changed or added functionality (if you modify code) is covered by unit tests.
- Make sure that all unit tests pass.
* Despite the "Klarna" mention, this repository is not affiliated with Klarna AB. KATT was indeed born at Klarna, and Klarna AB holds copyright for parts of the code, but it is now being maintained outside the company, by its original authors and new contributors.