Git Product home page Git Product logo

Comments (10)

Stranger6667 avatar Stranger6667 commented on June 12, 2024 1

Thank you! I can reproduce it now.
Looks like it starts failing if Hypothesis is >=6.84.3, I'll fix it soon

from schemathesis.

dkbarn avatar dkbarn commented on June 12, 2024 1

Confirmed that the error is now resolved with 3.26.1 - thanks for the super fast turnaround!

from schemathesis.

Stranger6667 avatar Stranger6667 commented on June 12, 2024

Hi! This is weird, I think we should have tests for it.

What is your pytest version?

from schemathesis.

Stranger6667 avatar Stranger6667 commented on June 12, 2024

And pytest-subtests version would be useful to know too

from schemathesis.

dkbarn avatar dkbarn commented on June 12, 2024

Sure thing, the test venv has:
pytest 8.1.1
pytest-subtests 0.7.0

(venv) bash-4.2$ poetry show pytest
 name         : pytest                                      
 version      : 8.1.1                                       
 description  : pytest: simple powerful testing with Python 

dependencies
 - colorama *
 - exceptiongroup >=1.0.0rc8
 - iniconfig *
 - packaging *
 - pluggy >=1.4,<2.0
 - tomli >=1

required by
 - pytest-asyncio >=7.0.0
 - pytest-benchmark >=3.8
 - pytest-cov >=4.6
 - pytest-datadir >=5.0
 - pytest-mock >=6.2.5
 - pytest-regressions >=6.2.0
 - pytest-subtests >=7.0
 - schemathesis >=4.6.4,<9
(venv) bash-4.2$ poetry show pytest-subtests
 name         : pytest-subtests                                 
 version      : 0.7.0                                           
 description  : unittest subTest() support and subtests fixture 

dependencies
 - pytest >=7.0

required by
 - schemathesis >=0.2.1,<0.8.0

from schemathesis.

Stranger6667 avatar Stranger6667 commented on June 12, 2024

I tried to reproduce it with a simpler version but wasn't successful. I also noted that graphql_schema_from_asgi has no @pytest.fixture decorator - its a typo in the original comment, right?

from schemathesis.

Stranger6667 avatar Stranger6667 commented on June 12, 2024

Generally, I don't expect that the exact Python version / Platform would matter here, but will take a closer look.

import hypothesis
import pytest
from hypothesis import HealthCheck
import schemathesis
from test.apps import _graphql


@pytest.fixture
def graphql_schema():
    app = _graphql._fastapi.create_app()
    return schemathesis.graphql.from_asgi("/graphql", app)


schema = schemathesis.from_pytest_fixture("graphql_schema")


@schema.parametrize()
@hypothesis.settings(
    deadline=None,
    suppress_health_check=[HealthCheck.too_slow, HealthCheck.filter_too_much],
)
def test_graphql_schema(case):
    response = case.call_asgi()
    case.validate_response(response)
❯ pytest sample.py
======================================= test session starts =======================================
platform darwin -- Python 3.11.7, pytest-8.1.1, pluggy-1.4.0
codspeed: 2.2.0 (callgraph: not supported)
rootdir: /tmp/schemathesis
configfile: pyproject.toml
plugins: syrupy-4.6.1, schemathesis-3.26.0, codspeed-2.2.0, hypothesis-6.98.10, mock-3.10.0, asyncio-0.21.1, trio-0.8.0, doctestplus-0.12.1, anyio-3.7.1, xdist-3.3.1, forked-1.6.0, subtests-0.7.0, httpserver-1.0.6
asyncio: mode=Mode.STRICT
collected 1 item                                                                                  

sample.py ,,,,.

======================================== 1 passed in 3.24s ========================================

How do you run your tests? Maybe there is some CLI arg, that could interfere?

from schemathesis.

dkbarn avatar dkbarn commented on June 12, 2024

You're right, I had left out a key detail from my example code which it turns out is the cause of the error. It seems that the error happens when the schema fixture is parametrized. Here is an example to demonstrate:

import copy
import falcon
import hypothesis
from hypothesis import HealthCheck
import itertools
import pytest
import schemathesis

from .util import GRAPHQL_HEADERS, create_database

INTERFACES = ["v1"]
DB_SCHEMAS = ["db0"]

@pytest.fixture(params=itertools.product(INTERFACES, DB_SCHEMAS))
def graphql_schema_from_asgi(request, db_setup_factory):
    interface, db_schema = request.param
    setup = db_setup_factory(schema=db_schema)
    # this slows things down but we need this fixture to not be async
    # so schemathesis.from_pytest_fixture can use it correctly
    falcon.async_to_sync(create_database, setup.sql_engine_manager)

    endpoint = f"/{interface}/graphql"
    return schemathesis.graphql.from_asgi(endpoint, setup.app, headers=GRAPHQL_HEADERS)

graphql_schema = schemathesis.from_pytest_fixture("graphql_schema_from_asgi")

@graphql_schema.parametrize()
@hypothesis.settings(
    deadline=None,
    suppress_health_check=[HealthCheck.too_slow, HealthCheck.filter_too_much],
)
def test_graphql_schema(case):
    response = case.call_asgi()
    case.validate_response(response)

from schemathesis.

Stranger6667 avatar Stranger6667 commented on June 12, 2024

Fixed in 3.26.1

from schemathesis.

Stranger6667 avatar Stranger6667 commented on June 12, 2024

Awesome! Happy to help

from schemathesis.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.