Concretizer: test dependencies affect concretization even when not testing about spack HOT 6 OPEN

@tgamblin said this is by design, but that sounds incorrect to me as the following two things aren't equivalent, but I would expect them to be

I said basically that the dependency types are requirements, not conditions. When you say:

depends_on("x", type=("link", "run"))

You don't have a choice of whether link or run dependencies are enabled. You're saying that you require X as a link and run dependency -- unconditionally. The only (intended) conditional thing is the when= clause. That's the trigger. So in @haampie's second example above, the when=@1 is the condition and it says "apply the rest of this information when @1 is true".

I don't think I said that much was "by design" with test dependencies. The only thing we "designed" here is when=, and we do not consider (that I know of) that certain deptypes might or might not be enabled. They're all considered to be requirements.

Test deps are kind of grafted on, and "test" doesn't mean the same thing as other deptypes. "test" is really more of an environment than a deptype. If we did this right, I think it would make sense to have build/link/run deptypes just for a test environment. But we do not have that.

I think the solution here is maybe not to do this:

Could fix it by working with (deptype, when) -> constraint instead of when -> constraint

But to have dependencies and test_dependencies dictionaries on the package, and to use the latter to store constraints on the test run environment. I think you probably need a separate directive for this so you could express build/link/run deptypes for the test env.

I haven't thought this through entirely, but an advantage of this is that you would not have to put test "run" dependencies (most test deps) on the spec, but you could co-concretize them with the spec (and its dependencies') run requirements. If you really had build-time test dependencies you could add "build" or "link" deps to the package at build time, but I think that is rare.

Or maybe instead of a new directive, something like:

with tests():
    depends_on(...)
    require()
    conflict()

etc. I think this is more general. Basically the idea and enables any of our constraints to be used to define the test env, not just a few deptypes.

Thoughts?

from spack.

just ran into this in a concretization that also built poppler for graphviz (requires freetype >= 2.10)

from spack.

Spack merges

depends_on("x", type="build")
depends_on("x@1", type="link")

into

depends_on("x@1", type=("build", "link"))

and type="test" is no different.

Could fix it by working with (deptype, when) -> constraint instead of when -> constraint (@tgamblin).

Or you have to come up with a unique when condition like when="@0:" ;)

from spack.

Spack merges...

Pip doesn't work this way, so we may encounter packages with different version requirements at build time and at run time: #31511

and type="test" is no different.

It sounds like we just need to handle test differently. Can we exclude it from the clingo solve entirely when --test is not requested? Seems like that would also make the solve faster/simpler.

from spack.

depends_on(...) immediately merges constraints on the same package for the same when condition, so when setting up the solver that information is already gone.

@tgamblin said this is by design, but that sounds incorrect to me as the following two things aren't equivalent, but I would expect them to be:

# constrains `x@1` when tests disabled
version("1.0")
depends_on("x", type="link")
depends_on("x@1", type="dep")

# does not constrain `x` when tests disabled
version("1.0")
depends_on("x", type="link")
depends_on("x@1", type="test", when="@1")

Seems like that would also make the solve faster/simpler.

It doesn't

from spack.

I haven't thought this through entirely, but an advantage of this is that you would not have to put test "run" dependencies (most test deps) on the spec, but you could co-concretize them with the spec (and its dependencies') run requirements. If you really had build-time test dependencies you could add "build" or "link" deps to the package at build time, but I think that is rare.

I'm still trying to parse this, but I'll also take this opportunity to bring up a related comment about PythonPackage.

The primary way of testing Python libraries is to import the library after it is installed. In Spack, we auto-detect all installed .py files and attempt to import them. Currently, this is supported by both spack install --test and spack test run. However, we run into a few issues with this:

1. Auto-detection often picks up testing files that require test-only dependencies like pytest, so we have to add fake run-time dependencies to get spack test run to work
2. Run-time dependencies are not loaded by spack install, so we have to add fake build-time dependencies to get spack install --test to work

I've brought these issues up with @tldahlgren before. @haampie and @alalazo have been eager to remove these fake dependencies to make concretization faster and easier. It would also allow us to support circular run-time dependencies, which are common in the Python ecosystem.

We haven't figured out a good solution for 1). My suggestion for 2) would be to only run the import tests for PythonPackage in spack test run. However, this is an extra step in a lot of people's workflows. Would it be possible for spack install --test to also result in spack test run automatically?

from spack.