Dry run about e2e-framework HOT 13 CLOSED

from e2e-framework.

I think the parallel issue doesn't bother the counts much; it makes it a bit unpredictable which will finish when, but otherwise if you keep incrementing the counts AfterEachFeature or AfterEachTest you'd be fine.

from e2e-framework.

Sorry, been a bit behind on this PR. Will have this ready for review by EOW.

from e2e-framework.

I'll take a stab at this once my other PRs merge which will make pre/post hooks easier as well as some writing the test for it.

from e2e-framework.

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

from e2e-framework.

@johnSchnake Are you still looking into this or I am free to take this up ?

from e2e-framework.

As a side note for this, my hope would be that this work would also enable the total number of tests to be run to be determined before any run.

This is important when sending progress updates so you can say things like: 3/99 tests completed. Otherwise you have no knowledge of how many tests will end up being targeted.

Ginkgo makes this available in a suiteWillBegin hook; hopefully we can get that same data available in one of our hooks.

from e2e-framework.

@johnSchnake Sounds good. Let me put up a draft PR with the changes so that you and @vladimirvivien can confirm if it is on the right path. Thanks

// H

from e2e-framework.

@johnSchnake / @vladimirvivien Do we want to consider each assessment as a test and provide a current/totalCount output or the same output for Testxxxx of testing.T instead ?

We should be able to extract the total number of tests in testing.M using a simple hack and pass that around in the context.

e.ctx = context.WithValue(e.ctx, "totalTests", reflect.ValueOf(m).Elem().FieldByName("tests").Len())

Let me try to identify an easy way to do the same for current test being executed.

from e2e-framework.

So I think this is one possible complication because different people may consider different things 'tests'.

Some people may see each TestFoo as a test, but each of those can include multiple features. Each feature may have multiple steps.

Ideally we could surface each of these but that definitely makes it harder. But now we do have the ability to filter tests based on test name and feature tag, so I think we have to surface those.

So I think it would be best to have something like:

TestsToBeRun: 8
FeaturesToBeRun: 11
AssessmentsToBeRun: 30

from e2e-framework.

@johnSchnake All right. I do like that idea. Let me get that changes sorted.

The other complication comes like you mentioned in #101 with TestInParallel when it comes to tracking the completed tests/features/assessments. Do we have a preference on how we want to track the number of completed items for corresponding bucket ?

from e2e-framework.

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

from e2e-framework.

/remove-lifecycle rotten

from e2e-framework.