tomatool / tomato Goto Github PK

add feature to be able to check contains rather than exact table view

Redis is frequently used cache storage, so it will be good to have it as one of the resources.

for simplicity, rather than having multiple duplicated Dockerfile, merge both dockerfile into 1, and use docker build stage to separate test image & production image

for e. g:

FROM golang AS builder
FROM golang AS test
# copy test binary from builder
FROM golang AS tomato
# copy production binary from builder

Is your feature request related to a problem? Please describe.
Define database schema using Gherkin clauses.

Describe the solution you'd like
Support for sql schema definitions inside the feature test files.

Describe alternatives you've considered
Migrations as part of the database setup/teardown process. Support inside the database resource

Is your feature request related to a problem? Please describe.
In a lot of cases integration test that involving external services, are using wiremock. Even though, tomato built in http/server also able to solve this problem, having support for wiremock integration would makes it easier for people to integrate.

Describe the solution you'd like
Implement wiremock client api that implement http/server interface, and use it as we're driver for another resource

    - name: tomato-http-server
      type: http/server
      ready_check: true
      params:
         driver: wiremock
         host: [wiremock_endpoint]

Would be nice if we can have empty table, or even empty all tables feature for steps in scenario

example :

Then "tomato-mysql" empty table "customers"
Then "tomato-postgres" empty table "*"

add debug mode when running the test, show verbose log of what tomato is doing.

Level 1 : handler
Level 2 : handler, resource

examples :

resource "psql" connected to "127.0.0.1:5432" using driver "postgres"
"psql" exec query "INSERT INTO"
"psql" query row "SELECT COUNT(*) FROM"

Is your feature request related to a problem? Please describe.
Currently there is no version transparency when installing tomato. The last acknowledged version number is v1.1.0 https://pkg.go.dev/github.com/tomatool/tomato?tab=overview. This makes readability around what version we have harder.

As you can see here, it picks up the last acknowledged version, then appends the date and commit for the latest release.

Describe the solution you'd like
I could be lost, but I believe v prefixing the tags will make this small problem go away.

Describe alternatives you've considered
I have not considered any, happy to hear if there are.

Additional context
N/A

due to user experience, error message from tomato should give just enough information of

• what is happening?
• what cause it?
• how can you fix it?
• and some context

error message when action is not available at the moment looks like:

2 scenarios (2 failed, 1 undefined)
5 steps (2 failed, 1 undefined, 2 skipped)
8.374339ms

You can implement step definitions for undefined steps with these snippets:

func responseHeaderShouldBe(arg1, arg2, arg3 string) error {
        return godog.ErrPending
}

func FeatureContext(s *godog.Suite) {
        s.Step(`^"([^"]*)" response header "([^"]*)" should be "([^"]*)"$`, responseHeaderShouldBe)
}

we'd like to have more brief & informative error message that tells the user which action is wrong, e.g:

Step action is unavailable : 
    Given "cli" response headers "Content-Type" should be "application/json"

would even nicer if there's suggestion

It could be helpful to generate timeout/disconnection behavior for apps using tomato. There seem to be a few libraries that we could use as a proxy between the connections and then manipulate them using tomato.

github.com/Shopify/toxiproxy seems promising and has an easy to use go client

Is your feature request related to a problem? Please describe.
When the RabbitMQ resource is called with New(), it executes an amqp dial call, while all other resources do not attempt a connection until Ready() or an action has been called on the resource.

Describe the solution you'd like
For consistency, I think we should define the resource interface and keep all logic similar across all resources.

New - construct a configuration for a given resource
Open - build and access resource implementation
Ready - attempt action execution to verify resource readiness
Reset - clean up state
Close - destroy any relation with given resource

Describe the bug
Unable to explicitly set a varchar field to an empty string. When setting to an empty string, the value is converted to null.

To Reproduce
Steps to reproduce the behavior:
Schema

CREATE TABLE table (
	"id" SERIAL PRIMARY KEY,
	"email" VARCHAR(255),
)

Feature

  Given set "db" table "table" list of content
      | id    | email |  
      |  1    |           |

Expected behavior
Defined field with empty string value.

Additional context
Add any other context about the problem here.

refers to discussion on #42

to be able to have consistent and good documentation, would be nice to have docs generator that reflected directly to the app

Is your feature request related to a problem? Please describe.
Writing go code based/generated from yml file feels wrong, since it's a go application. Go (executable binary) should be the source of truth.

Describe the solution you'd like
It would be nice to have dictionary.yml generated from handler package itself, maybe consider using how godoc.org steal from code comments to get detailed documentation.

Describe alternatives you've considered
Re-structure the handler packages to be godoc friendly, that way, the handler/action documentation can come from godoc

Describe the bug
When sending request with payload, httpclient is not trimming the request body, so

server will be able to read this properly

Given "http-cli" send request to "POST /api/v1" with body
        """
myparams=booyah
        """

but not this

Given "http-cli" send request to "POST /api/v1" with body
        """
               myparams=booyah
        """

Expected behavior
those 2 example above should behave the same way

For before each Scenario is started, we call Reset() func for each resources (https://github.com/alileza/tomato/blob/master/handler/handler.go#L17), and seems like at the moment Reset() function get called after Background: gets called, which makes everything on Background feature not valid.

Is your feature request related to a problem? Please describe.
There is a lot of overlap between the tomato.yml file and docker-compose file

Describe the solution you'd like
Maybe we could explore a way to integrate the two with the tomato.yml outputting a docker-compose (or k8s cluster configuration, or any other dependency orchestrator) file whenever you update it so you only have to maintain one file

Is your feature request related to a problem? Please describe.
I would like to use wiremock to check for request body patterns and provide an XML response body defined in a file.

At the moment, wiremock has a rigid setup only allowing request counting (no matching possible) and only inline JSON body. Wiremock itself provides many options via its JSON api (http://wiremock.org/docs/stubbing/) which are also easy to use and have powerful documentation. I see no reason to limit it for Tomato users.

Describe the solution you'd like
I would like to be able to define, in my feature file, the JSON that would define the whole mapping for wiremock, like this:

    "request": {
        "method": "GET",
        "url": "/some/thing"
    },
    "response": {
        "status": 200,
        "body": "Hello world!",
        "headers": {
            "Content-Type": "text/plain"
        }
    }
}

Describe alternatives you've considered
I considered building tests in pure Go and using wiremock separately, but I like how Tomato integrates with rabbitmq and I would like to keep using it. I also tried setting up mappings for wiremock separately, by putting JSON files in mapping directory, but as Tomato resets wiremock after each scenario, this did not work out.

Additional context
I think one possible solution would be to add an additional step for wiremock, something along the lines of Given wiremock with JSON mapping or similar. Then, this whole JSON is POSTed to wiremock JSON api.

I think this addition would be very powerful for users who use Tomato with wiremock as this would enable full stubbing features of wiremock and require little coding in the Tomato itself, as wiremock has stubbing nicely built in already.

After this changes 4b4da9e codecov report stop working

In order to extend tomato functionality, it would be nice to have a concrete example of how to write a custom resource.

Is your feature request related to a problem? Please describe.
Currently http/server using wiremock is hardcoded to GET requests, when we should support all http methods.

Describe the solution you'd like
Support all HTTP methods such as POST, PUT, etc.

Is your feature request related to a problem? Please describe.
Unable to interact with MySQL bit column type

Describe the solution you'd like
Add a function that handle bit column type something like

b'1' & b'0' or bit::0 &bit::1

Using localstack I think we could implement functional testing of applications that depend on AWS resources

Is your feature request related to a problem? Please describe.
There is no way we can check exit code from executed command in shell resource

Describe the solution you'd like
Would be nice to add a new function to get latest exit code from executed command

https://github.com/tomatool/tomato/blob/master/handler/shell/shell.go#L10

something like

type Resource interface {
	resource.Resource

	Exec(command string, arguments ...string) error
	ExitCode() (int, error)
	Stdout() (string, error)
	Stderr() (string, error)
}

Is your feature request related to a problem? Please describe.
At the moment we have some config passed as a cli flag, and some in config file.

Describe the solution you'd like
Move all cli flag config to be able to be configured also from config file

Is your feature request related to a problem? Please describe.
At the moment each time using shell resource, you'd need to specify full shell command, for example

Scenario 1: check pod
   Given "bash" execute "kubectl get pods"

when most likely we'd use the same command over and over for the scenario (in this case kubectl)

Describe the solution you'd like
add prefix params that automatically prepended to all command executed by the shell resource, like :

resources:
  - name: kubectl
    type: shell
    params:
        prefix: "kubectl"

this way, you would use the resource this way instead

Scenario 1: check pod
   Given "kubectl" execute "get pods"

Background

I find that github.com/nsqio/go-nsq and github.com/bitly/go-nsq coexist in this repo：
https://github.com/tomatool/tomato/blob/main/go.mod （Line 9&22）

github.com/bitly/go-nsq v1.0.7
github.com/nsqio/go-nsq v1.0.7 // indirect

That’s because the "github.com/bitly/go-nsq" has already been redirected to "github.com/nsqio/go-nsq". When you use the old path "github.com/bitly/go-nsq" to import the go-nsq, github automatically directed to github.com/nsqio/go-nsq, returned the repo github.com/nsqio/go-nsq, and reintroduces github.com/nsqio/go-nsq through the import statements "import github.com/nsqio/go-nsq" in the go source file of nsqio/go-nsq.

https://github.com/nsqio/go-nsq/blob/v1.0.7/config_flag_test.go

package nsq_test
import (
	"flag"
	"github.com/nsqio/go-nsq"
)

This will work in isolation, bring about potential risks and problems.

Solution

Replace all the old import paths, change "github.com/bitly/go-nsq" to "github.com/nsqio/go-nsq".
Where did you import it: https://github.com/tomatool/tomato/search?q=github.com%2Fbitly%2Fgo-nsq&unscoped_q=github.com%2Fbitly%2Fgo-nsq

Then "psql" table "plan" should look like
      | id | name    |
      | 1  | antoni  |

on table should look like, when rows not found. would be nice to show what's the table looks like on that point.

In the meantime, comparing JSON column for db/sql postgres driver would throw an error. It would be nice to have a way, to make it possible.

Then "tomato-mysql" table "customers" should look like
        | customer_id | name    | metadata                   |
        | 1           | cembri  | {"ip_address":"127.0.0.1"} |