tomatool / tomato Goto Github PK
View Code? Open in Web Editor NEWfunctional testing tools - keep your application healthy with 🍅
Home Page: https://tomatool.github.io/tomato
License: Apache License 2.0
functional testing tools - keep your application healthy with 🍅
Home Page: https://tomatool.github.io/tomato
License: Apache License 2.0
add feature to be able to check contains
rather than exact table view
Redis is frequently used cache storage, so it will be good to have it as one of the resources.
for simplicity, rather than having multiple duplicated Dockerfile, merge both dockerfile into 1, and use docker build stage to separate test image & production image
for e. g:
FROM golang AS builder
FROM golang AS test
# copy test binary from builder
FROM golang AS tomato
# copy production binary from builder
Is your feature request related to a problem? Please describe.
Define database schema using Gherkin clauses.
Describe the solution you'd like
Support for sql schema definitions inside the feature test files.
Describe alternatives you've considered
Migrations as part of the database setup/teardown process. Support inside the database resource
Is your feature request related to a problem? Please describe.
In a lot of cases integration test that involving external services, are using wiremock. Even though, tomato built in http/server also able to solve this problem, having support for wiremock integration would makes it easier for people to integrate.
Describe the solution you'd like
Implement wiremock client api that implement http/server interface, and use it as we're driver for another resource
- name: tomato-http-server
type: http/server
ready_check: true
params:
driver: wiremock
host: [wiremock_endpoint]
Would be nice if we can have empty table, or even empty all tables feature for steps in scenario
example :
Then "tomato-mysql" empty table "customers"
Then "tomato-postgres" empty table "*"
add debug mode when running the test, show verbose log of what tomato is doing.
Level 1 : handler
Level 2 : handler, resource
examples :
resource "psql" connected to "127.0.0.1:5432" using driver "postgres"
"psql" exec query "INSERT INTO"
"psql" query row "SELECT COUNT(*) FROM"
Is your feature request related to a problem? Please describe.
Currently there is no version transparency when installing tomato. The last acknowledged version number is v1.1.0
https://pkg.go.dev/github.com/tomatool/tomato?tab=overview. This makes readability around what version we have harder.
As you can see here, it picks up the last acknowledged version, then appends the date and commit for the latest release.
Describe the solution you'd like
I could be lost, but I believe v
prefixing the tags will make this small problem go away.
Describe alternatives you've considered
I have not considered any, happy to hear if there are.
Additional context
N/A
due to user experience, error message from tomato should give just enough information of
error message when action is not available at the moment looks like:
2 scenarios (2 failed, 1 undefined)
5 steps (2 failed, 1 undefined, 2 skipped)
8.374339ms
You can implement step definitions for undefined steps with these snippets:
func responseHeaderShouldBe(arg1, arg2, arg3 string) error {
return godog.ErrPending
}
func FeatureContext(s *godog.Suite) {
s.Step(`^"([^"]*)" response header "([^"]*)" should be "([^"]*)"$`, responseHeaderShouldBe)
}
we'd like to have more brief & informative error message that tells the user which action is wrong, e.g:
Step action is unavailable :
Given "cli" response headers "Content-Type" should be "application/json"
would even nicer if there's suggestion
It could be helpful to generate timeout/disconnection behavior for apps using tomato. There seem to be a few libraries that we could use as a proxy between the connections and then manipulate them using tomato.
github.com/Shopify/toxiproxy seems promising and has an easy to use go client
Is your feature request related to a problem? Please describe.
When the RabbitMQ resource is called with New(), it executes an amqp dial call, while all other resources do not attempt a connection until Ready() or an action has been called on the resource.
Describe the solution you'd like
For consistency, I think we should define the resource interface and keep all logic similar across all resources.
New - construct a configuration for a given resource
Open - build and access resource implementation
Ready - attempt action execution to verify resource readiness
Reset - clean up state
Close - destroy any relation with given resource
Describe the bug
Unable to explicitly set a varchar field to an empty string. When setting to an empty string, the value is converted to null.
To Reproduce
Steps to reproduce the behavior:
Schema
CREATE TABLE table (
"id" SERIAL PRIMARY KEY,
"email" VARCHAR(255),
)
Feature
Given set "db" table "table" list of content
| id | email |
| 1 | |
Expected behavior
Defined field with empty string value.
Additional context
Add any other context about the problem here.
refers to discussion on #42
to be able to have consistent and good documentation, would be nice to have docs generator that reflected directly to the app
Is your feature request related to a problem? Please describe.
Writing go code based/generated from yml file feels wrong, since it's a go application. Go (executable binary) should be the source of truth.
Describe the solution you'd like
It would be nice to have dictionary.yml
generated from handler
package itself, maybe consider using how godoc.org
steal from code comments to get detailed documentation.
Describe alternatives you've considered
Re-structure the handler packages to be godoc friendly, that way, the handler/action documentation can come from godoc
Describe the bug
When sending request with payload, httpclient
is not trimming the request body, so
server will be able to read this properly
Given "http-cli" send request to "POST /api/v1" with body
"""
myparams=booyah
"""
but not this
Given "http-cli" send request to "POST /api/v1" with body
"""
myparams=booyah
"""
Expected behavior
those 2 example above should behave the same way
For before each Scenario is started, we call Reset()
func for each resources (https://github.com/alileza/tomato/blob/master/handler/handler.go#L17), and seems like at the moment Reset()
function get called after Background:
gets called, which makes everything on Background feature not valid.
Is your feature request related to a problem? Please describe.
There is a lot of overlap between the tomato.yml file and docker-compose file
Describe the solution you'd like
Maybe we could explore a way to integrate the two with the tomato.yml outputting a docker-compose (or k8s cluster configuration, or any other dependency orchestrator) file whenever you update it so you only have to maintain one file
Is your feature request related to a problem? Please describe.
I would like to use wiremock to check for request body patterns and provide an XML response body defined in a file.
At the moment, wiremock has a rigid setup only allowing request counting (no matching possible) and only inline JSON body. Wiremock itself provides many options via its JSON api (http://wiremock.org/docs/stubbing/) which are also easy to use and have powerful documentation. I see no reason to limit it for Tomato users.
Describe the solution you'd like
I would like to be able to define, in my feature file, the JSON that would define the whole mapping for wiremock, like this:
"request": {
"method": "GET",
"url": "/some/thing"
},
"response": {
"status": 200,
"body": "Hello world!",
"headers": {
"Content-Type": "text/plain"
}
}
}
Describe alternatives you've considered
I considered building tests in pure Go and using wiremock separately, but I like how Tomato integrates with rabbitmq and I would like to keep using it. I also tried setting up mappings for wiremock separately, by putting JSON files in mapping directory, but as Tomato resets wiremock after each scenario, this did not work out.
Additional context
I think one possible solution would be to add an additional step for wiremock, something along the lines of Given wiremock with JSON mapping
or similar. Then, this whole JSON is POSTed to wiremock JSON api.
I think this addition would be very powerful for users who use Tomato with wiremock as this would enable full stubbing features of wiremock and require little coding in the Tomato itself, as wiremock has stubbing nicely built in already.
After this changes 4b4da9e codecov report stop working
In order to extend tomato functionality, it would be nice to have a concrete example of how to write a custom resource.
Is your feature request related to a problem? Please describe.
Currently http/server using wiremock is hardcoded to GET requests, when we should support all http methods.
Describe the solution you'd like
Support all HTTP methods such as POST, PUT, etc.
Is your feature request related to a problem? Please describe.
Unable to interact with MySQL bit column type
Describe the solution you'd like
Add a function that handle bit column type something like
b'1'
& b'0'
or bit::0
&bit::1
Using localstack I think we could implement functional testing of applications that depend on AWS resources
Is your feature request related to a problem? Please describe.
There is no way we can check exit code from executed command in shell resource
Describe the solution you'd like
Would be nice to add a new function to get latest exit code from executed command
https://github.com/tomatool/tomato/blob/master/handler/shell/shell.go#L10
something like
type Resource interface {
resource.Resource
Exec(command string, arguments ...string) error
ExitCode() (int, error)
Stdout() (string, error)
Stderr() (string, error)
}
Is your feature request related to a problem? Please describe.
At the moment we have some config passed as a cli flag, and some in config file.
Describe the solution you'd like
Move all cli flag config to be able to be configured also from config file
Is your feature request related to a problem? Please describe.
At the moment each time using shell
resource, you'd need to specify full shell command, for example
Scenario 1: check pod
Given "bash" execute "kubectl get pods"
when most likely we'd use the same command over and over for the scenario (in this case kubectl
)
Describe the solution you'd like
add prefix
params that automatically prepended to all command executed by the shell resource, like :
resources:
- name: kubectl
type: shell
params:
prefix: "kubectl"
this way, you would use the resource this way instead
Scenario 1: check pod
Given "kubectl" execute "get pods"
I find that github.com/nsqio/go-nsq
and github.com/bitly/go-nsq
coexist in this repo:
https://github.com/tomatool/tomato/blob/main/go.mod (Line 9&22)
github.com/bitly/go-nsq v1.0.7
github.com/nsqio/go-nsq v1.0.7 // indirect
That’s because the "github.com/bitly/go-nsq" has already been redirected to "github.com/nsqio/go-nsq". When you use the old path "github.com/bitly/go-nsq" to import the go-nsq
, github automatically directed to github.com/nsqio/go-nsq
, returned the repo github.com/nsqio/go-nsq
, and reintroduces github.com/nsqio/go-nsq
through the import statements "import github.com/nsqio/go-nsq" in the go source file of nsqio/go-nsq
.
https://github.com/nsqio/go-nsq/blob/v1.0.7/config_flag_test.go
package nsq_test
import (
"flag"
"github.com/nsqio/go-nsq"
)
This will work in isolation, bring about potential risks and problems.
Replace all the old import paths, change "github.com/bitly/go-nsq"
to "github.com/nsqio/go-nsq"
.
Where did you import it: https://github.com/tomatool/tomato/search?q=github.com%2Fbitly%2Fgo-nsq&unscoped_q=github.com%2Fbitly%2Fgo-nsq
Then "psql" table "plan" should look like
| id | name |
| 1 | antoni |
on table should look like, when rows not found. would be nice to show what's the table looks like on that point.
In the meantime, comparing JSON column for db/sql
postgres driver would throw an error. It would be nice to have a way, to make it possible.
Then "tomato-mysql" table "customers" should look like
| customer_id | name | metadata |
| 1 | cembri | {"ip_address":"127.0.0.1"} |
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.