gardener / test-infra Goto Github PK
View Code? Open in Web Editor NEWTest machinery for orchestration of integration/e2e/smoke style tests
License: Apache License 2.0
Test machinery for orchestration of integration/e2e/smoke style tests
License: Apache License 2.0
How to categorize this issue?
/area testing
/kind bug
/priority normal
What happened:
When promoting the integration test should rotate kubeconfig for a shoot cluster
in gardener/gardener#3300 from beta to default, we missed to understand that this tests will "invalidate" the kubeconfigs used by subsequent tests.
The attempt to overcome this with gardener/gardener#3326 unfortunately doesn't work, as the kubeconfigs from TestMachinery are currently not meant to be "updated" via intermediate steps, hence all tests will only get the kubeconfigs from the create-shoot step, see e.g. this generated argo task for the e2e tests:
- arguments:
artifacts:
- from: '{{tasks.create-shoot-create-testflow.outputs.artifacts.kubeconfigs}}'
name: kubeconfigs
...
dependencies:
- shoot-default-serial-test-suite-tests-default-testflow
name: e2e-fast-e2e-testflow
What you expected to happen:
Maybe kubeconfigs should be allowed to be overwritten by intermediate steps.
What happened:
Following the docs I found some issues when I tried to deploy the testmachinery.
make install-controller
was renamed to deploy-controller
.make deploy-controller
the testmachinery-controller runs into an ErrImageNeverPull
error because the pull policy in the local-values.yaml is set to Never
.ImagePullBackOff
because it can't find the version 0.192.0-dev
. Do I have to push it by myself?latest
, the pod at least pulls the right image but then an error occurs in the controller:"msg":"error during ensureDependencies","error":"[elasticsearchConfiguration: Required value: elastic search config is required if collector is enabled, s3Configuration.server.endpoint: Required value: endpoint or minio has to be defined]","errorCauses":[{"error":"elasticsearchConfiguration: Required value: elastic search config is required if collector is enabled"},{"error":"s3Configuration.server.endpoint: Required value: endpoint or minio has to be defined"}]
What you expected to happen:
The deployment setup is properly documented and works.
How to reproduce it (as minimally and precisely as possible):
make deploy-controller
Environment:
AWS k8s version: 1.18.14
test-infra repo: commit
How to categorize this issue?
/area testing
/kind flake
/priority 3
Which test(s)/suite(s) are flaking:
https://github.com/gardener/test-infra/blob/f494cd53f4571bc4e4d3d0bbc9f81dc379420668/pkg/testmachinery/controller/ttl/controller_test.go
Reason for failure:
A test namespace is actually composed of a rand int from 0 to 1000, hence with some tests it collides here and there.
However upon checking I noticed that the AfterEach
test-infra/pkg/testmachinery/controller/ttl/controller_test.go
Lines 64 to 72 in f494cd5
Terminating
state as this won't throw any error...TestRun
.
@schrodit do I miss another intent? If not, either testenv can be setup with a gc controller or we remove the resp. wait for ns deletion and instead increase the randomness greatly.
Anything else we need to know:
Currently, the testmachinery can run garden-setup on one host cloudprovider. I'd like to be able to test whether creating a gardener works on multiple infrastructures. I can do that already, by starting one test for each infrastructure after the previous one is finished, but doing this sequentially takes a lot of time
Therefore I'd like to spin up a gardener on multiple infrastructures in parallel within one test.
Find below the logs
fatal error: concurrent map read and map write goroutine 617 [running]: runtime.throw(0x1e07ae8, 0x21) /usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc000a41b38 sp=0xc000a41b08 pc=0x42e5c2 runtime.mapaccess2_faststr(0x1ab3160, 0xc0007a8690, 0xc0065cf640, 0x18, 0x3, 0xc0065cf640) /usr/local/go/src/runtime/map_faststr.go:116 +0x48f fp=0xc000a41ba8 sp=0xc000a41b38 pc=0x412c6f github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00248de50, 0x7, 0xc00248de40, 0x10, 0xc002a40a50, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:63 +0x160 fp=0xc000a41d10 sp=0xc000a41ba8 pc=0x1690b00 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc0004dbb00, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 fp=0xc000a41ea8 sp=0xc000a41d10 pc=0x1693c40 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff fp=0xc000a41f68 sp=0xc000a41ea8 pc=0x169587f runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc000a41f70 sp=0xc000a41f68 pc=0x45b9a1 created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 1 [semacquire]: sync.runtime_Semacquire(0xc00105cd78) /usr/local/go/src/runtime/sema.go:56 +0x42 sync.(*WaitGroup).Wait(0xc00105cd70) /usr/local/go/src/sync/waitgroup.go:130 +0x64 github.com/gardener/test-infra/pkg/testrunner.RunList.Run(0xc00065a000, 0x2e, 0x30, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb, 0xc00013c1c0, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:103 +0x330 github.com/gardener/test-infra/pkg/testrunner.ExecuteTestruns(0x212a420, 0xc00105b9e0, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0x7ffdf61e7a5e, 0xb, 0xc00013c1c0, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/testrunner.go:40 +0x266 github.com/gardener/test-infra/cmd/testrunner/cmd/run_template.glob..func1(0x30e0ca0, 0xc000239e40, 0x0, 0x16) /go/src/github.com/gardener/test-infra/cmd/testrunner/cmd/run_template/run_template.go:117 +0x59c github.com/spf13/cobra.(*Command).execute(0x30e0ca0, 0xc000239ce0, 0x16, 0x16, 0x30e0ca0, 0xc000239ce0) /go/src/github.com/gardener/test-infra/vendor/github.com/spf13/cobra/command.go:830 +0x2aa github.com/spf13/cobra.(*Command).ExecuteC(0x30dfb20, 0x30e11a0, 0xc000793f20, 0x43c30a) /go/src/github.com/gardener/test-infra/vendor/github.com/spf13/cobra/command.go:914 +0x2fb github.com/spf13/cobra.(*Command).Execute(...) /go/src/github.com/gardener/test-infra/vendor/github.com/spf13/cobra/command.go:864 github.com/gardener/test-infra/cmd/testrunner/cmd.Execute() /go/src/github.com/gardener/test-infra/cmd/testrunner/cmd/cmd.go:50 +0x31 main.main() /go/src/github.com/gardener/test-infra/cmd/testrunner/main.go:22 +0x20 goroutine 6 [chan receive]: k8s.io/klog.(*loggingT).flushDaemon(0x30f43e0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/klog/klog.go:1010 +0x8b created by k8s.io/klog.init.0 /go/src/github.com/gardener/test-infra/vendor/k8s.io/klog/klog.go:411 +0xd6 goroutine 68 [syscall]: os/signal.signal_recv(0x45b9a6) /usr/local/go/src/runtime/sigqueue.go:147 +0x9c os/signal.loop() /usr/local/go/src/os/signal/signal_unix.go:23 +0x22 created by os/signal.init.0 /usr/local/go/src/os/signal/signal_unix.go:29 +0x41 goroutine 69 [select]: go.opencensus.io/stats/view.(*worker).start(0xc0006c7c20) /go/src/github.com/gardener/test-infra/vendor/go.opencensus.io/stats/view/worker.go:154 +0x100 created by go.opencensus.io/stats/view.init.0 /go/src/github.com/gardener/test-infra/vendor/go.opencensus.io/stats/view/worker.go:32 +0x57 goroutine 639 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0025518c9, 0x7, 0xc0025518b0, 0x10, 0xc00293b440, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000d60900, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 633 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0027c4919, 0x7, 0xc0027c4900, 0x10, 0xc002553230, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000d61ac0, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 626 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002867809, 0x7, 0xc0028677f0, 0x10, 0xc002869a70, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000e61440, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 28 [IO wait]: internal/poll.runtime_pollWait(0x7fc18ed90f08, 0x72, 0xffffffffffffffff) /usr/local/go/src/runtime/netpoll.go:184 +0x55 internal/poll.(*pollDesc).wait(0xc00041e498, 0x72, 0xd100, 0xd198, 0xffffffffffffffff) /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Read(0xc00041e480, 0xc000d8c000, 0xd198, 0xd198, 0x0, 0x0, 0x0) /usr/local/go/src/internal/poll/fd_unix.go:169 +0x1cf net.(*netFD).Read(0xc00041e480, 0xc000d8c000, 0xd198, 0xd198, 0x203000, 0x0, 0x62b6) /usr/local/go/src/net/fd_unix.go:202 +0x4f net.(*conn).Read(0xc0002b2008, 0xc000d8c000, 0xd198, 0xd198, 0x0, 0x0, 0x0) /usr/local/go/src/net/net.go:184 +0x68 crypto/tls.(*atLeastReader).Read(0xc002b003c0, 0xc000d8c000, 0xd198, 0xd198, 0xefb, 0x20ca060, 0xc000edb970) /usr/local/go/src/crypto/tls/conn.go:780 +0x60 bytes.(*Buffer).ReadFrom(0xc0004b0958, 0x20c9e60, 0xc002b003c0, 0x409e05, 0x1b2ce80, 0x1d5cf00) /usr/local/go/src/bytes/buffer.go:204 +0xb4 crypto/tls.(*Conn).readFromUntil(0xc0004b0700, 0x20cc920, 0xc0002b2008, 0x5, 0xc0002b2008, 0x9) /usr/local/go/src/crypto/tls/conn.go:802 +0xec crypto/tls.(*Conn).readRecordOrCCS(0xc0004b0700, 0x0, 0x0, 0x0) /usr/local/go/src/crypto/tls/conn.go:609 +0x124 crypto/tls.(*Conn).readRecord(...) /usr/local/go/src/crypto/tls/conn.go:577 crypto/tls.(*Conn).Read(0xc0004b0700, 0xc000835000, 0x1000, 0x1000, 0x0, 0x0, 0x0) /usr/local/go/src/crypto/tls/conn.go:1255 +0x161 bufio.(*Reader).Read(0xc0004c2f60, 0xc00080c1f8, 0x9, 0x9, 0xc000edbd10, 0x0, 0x87e875) /usr/local/go/src/bufio/bufio.go:226 +0x26a io.ReadAtLeast(0x20c9c00, 0xc0004c2f60, 0xc00080c1f8, 0x9, 0x9, 0x9, 0xc000094060, 0x0, 0x20ca060) /usr/local/go/src/io/io.go:310 +0x87 io.ReadFull(...) /usr/local/go/src/io/io.go:329 golang.org/x/net/http2.readFrameHeader(0xc00080c1f8, 0x9, 0x9, 0x20c9c00, 0xc0004c2f60, 0x0, 0x0, 0xc00293b020, 0x0) /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/frame.go:237 +0x87 golang.org/x/net/http2.(*Framer).ReadFrame(0xc00080c1c0, 0xc00293b020, 0x0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 golang.org/x/net/http2.(*clientConnReadLoop).run(0xc000edbfb8, 0xc00046d7a8, 0x867c08) /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/transport.go:1732 +0x8c golang.org/x/net/http2.(*ClientConn).readLoop(0xc00025a300) /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/transport.go:1660 +0xa3 created by golang.org/x/net/http2.(*Transport).newClientConn /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/transport.go:673 +0x62f goroutine 643 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0024c49d0, 0x7, 0xc0024c49c0, 0x10, 0xc00293a8d0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000fad140, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 279 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1(0x20e3840, 0xc0007a8630, 0xc0007a8600) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/record/event.go:219 +0xc5 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/record/event.go:217 +0x6e goroutine 96 [sync.Cond.Wait]: runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:310 sync.runtime_notifyListWait(0xc000762188, 0x26) /usr/local/go/src/runtime/sema.go:510 +0xf8 sync.(*Cond).Wait(0xc000762178) /usr/local/go/src/sync/cond.go:56 +0x9d k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop(0xc000762160, 0xc000332210, 0x0, 0x0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/delta_fifo.go:423 +0xa6 k8s.io/client-go/tools/cache.(*controller).processLoop(0xc00022ae00) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/controller.go:150 +0x40 k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc000797e00) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152 +0x5e k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000797e00, 0x3b9aca00, 0x0, 0xc00053b601, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153 +0xf8 k8s.io/apimachinery/pkg/util/wait.Until(...) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 k8s.io/client-go/tools/cache.(*controller).Run(0xc00022ae00, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/controller.go:124 +0x2fd k8s.io/client-go/tools/cache.(*sharedIndexInformer).Run(0xc0008394d0, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/shared_informer.go:227 +0x47f created by sigs.k8s.io/controller-runtime/pkg/cache/internal.(*specificInformersMap).Start.func1 /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:126 +0x122 goroutine 625 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0025514e9, 0x7, 0xc0025514d0, 0x10, 0xc002b02990, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000373100, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 622 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00270e8c9, 0x7, 0xc00270e8b0, 0x10, 0xc00293b5f0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000a7bc00, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 685 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000d61f00, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 689 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000ef9600, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 693 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000a7ba80, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 764 [chan receive]: k8s.io/client-go/tools/cache.(*sharedProcessor).run(0xc000132540, 0xc0007f7b00) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/shared_informer.go:425 +0x49 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1() /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:54 +0x2e k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc0024c4d50, 0xc0024df580) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:71 +0x59 created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:69 +0x62 goroutine 620 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00255c490, 0x7, 0xc00255c480, 0x10, 0xc00293aa80, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000751fc0, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 698 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000ef8600, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 280 [chan receive]: k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1(0x20e3840, 0xc0007a8660, 0xc0002f9fd0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/record/event.go:219 +0xc5 created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/record/event.go:217 +0x6e goroutine 660 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0026cb140, 0x7, 0xc0026cb130, 0x10, 0xc0024e52f0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc001088800, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 694 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000bd9bc0, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 683 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000373100, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 95 [chan receive]: sigs.k8s.io/controller-runtime/pkg/cache/internal.(*specificInformersMap).Start(0xc00080c2a0, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:132 +0x4c created by sigs.k8s.io/controller-runtime/pkg/cache/internal.(*InformersMap).Start /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/cache/internal/deleg_map.go:61 +0x80 goroutine 378 [IO wait]: internal/poll.runtime_pollWait(0x7fc18ed90e38, 0x72, 0xffffffffffffffff) /usr/local/go/src/runtime/netpoll.go:184 +0x55 internal/poll.(*pollDesc).wait(0xc00022b798, 0x72, 0xce00, 0xceb5, 0xffffffffffffffff) /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Read(0xc00022b780, 0xc000b68000, 0xceb5, 0xceb5, 0x0, 0x0, 0x0) /usr/local/go/src/internal/poll/fd_unix.go:169 +0x1cf net.(*netFD).Read(0xc00022b780, 0xc000b68000, 0xceb5, 0xceb5, 0x203000, 0x0, 0xb82c) /usr/local/go/src/net/fd_unix.go:202 +0x4f net.(*conn).Read(0xc0002b20e0, 0xc000b68000, 0xceb5, 0xceb5, 0x0, 0x0, 0x0) /usr/local/go/src/net/net.go:184 +0x68 crypto/tls.(*atLeastReader).Read(0xc000822f80, 0xc000b68000, 0xceb5, 0xceb5, 0x2000, 0x400, 0xc000799970) /usr/local/go/src/crypto/tls/conn.go:780 +0x60 bytes.(*Buffer).ReadFrom(0xc0008f4258, 0x20c9e60, 0xc000822f80, 0x409e05, 0x1b2ce80, 0x1d5cf00) /usr/local/go/src/bytes/buffer.go:204 +0xb4 crypto/tls.(*Conn).readFromUntil(0xc0008f4000, 0x20cc920, 0xc0002b20e0, 0x5, 0xc0002b20e0, 0x9) /usr/local/go/src/crypto/tls/conn.go:802 +0xec crypto/tls.(*Conn).readRecordOrCCS(0xc0008f4000, 0x0, 0x0, 0x0) /usr/local/go/src/crypto/tls/conn.go:609 +0x124 crypto/tls.(*Conn).readRecord(...) /usr/local/go/src/crypto/tls/conn.go:577 crypto/tls.(*Conn).Read(0xc0008f4000, 0xc000941000, 0x1000, 0x1000, 0x0, 0x0, 0x0) /usr/local/go/src/crypto/tls/conn.go:1255 +0x161 bufio.(*Reader).Read(0xc000a394a0, 0xc00080c9d8, 0x9, 0x9, 0xc000799d10, 0x0, 0x87e875) /usr/local/go/src/bufio/bufio.go:226 +0x26a io.ReadAtLeast(0x20c9c00, 0xc000a394a0, 0xc00080c9d8, 0x9, 0x9, 0x9, 0xc000094060, 0x0, 0x20ca060) /usr/local/go/src/io/io.go:310 +0x87 io.ReadFull(...) /usr/local/go/src/io/io.go:329 golang.org/x/net/http2.readFrameHeader(0xc00080c9d8, 0x9, 0x9, 0x20c9c00, 0xc000a394a0, 0x0, 0x0, 0xc000bda180, 0x0) /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/frame.go:237 +0x87 golang.org/x/net/http2.(*Framer).ReadFrame(0xc00080c9a0, 0xc000bda180, 0x0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/frame.go:492 +0xa1 golang.org/x/net/http2.(*clientConnReadLoop).run(0xc000799fb8, 0xc0002a87a8, 0x867c08) /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/transport.go:1732 +0x8c golang.org/x/net/http2.(*ClientConn).readLoop(0xc000538d80) /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/transport.go:1660 +0xa3 created by golang.org/x/net/http2.(*Transport).newClientConn /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/transport.go:673 +0x62f goroutine 646 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002866c90, 0x7, 0xc002866c80, 0x10, 0xc0024e5440, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc00040e300, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 618 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0026cad00, 0x7, 0xc0026cacf0, 0x10, 0xc00293b620, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000bd84c0, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 641 [runnable]: k8s.io/apimachinery/pkg/types.NamespacedName.String(...) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/types/namespacedname.go:42 github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00255dd30, 0x7, 0xc00255dd20, 0x10, 0xc00293b6b0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:69 +0x189 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000e60a80, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 585 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000750c80, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 94 [chan receive]: sigs.k8s.io/controller-runtime/pkg/cache/internal.(*specificInformersMap).Start(0xc00080c000, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:132 +0x4c created by sigs.k8s.io/controller-runtime/pkg/cache/internal.(*InformersMap).Start /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/cache/internal/deleg_map.go:60 +0x50 goroutine 632 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0028de829, 0x7, 0xc0028de810, 0x10, 0xc00273f260, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc00040f940, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 687 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000ae9440, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 688 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000ef8ac0, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 658 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002867079, 0x7, 0xc002867060, 0x10, 0xc002869fb0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000372380, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 584 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000373e00, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 670 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000ef6840, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 729 [chan receive]: sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start(0xc0008ae000, 0xc00005a300, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:163 +0x3dc sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).startLeaderElectionRunnables.func1(0xc0001948c0, 0x20cce80, 0xc0008ae000) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/manager/internal.go:321 +0x42 created by sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).startLeaderElectionRunnables /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/manager/internal.go:320 +0x102 goroutine 634 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002550140, 0x7, 0xc002550130, 0x10, 0xc002b02510, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000ae9440, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 690 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc00040f5c0, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 642 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0028dec79, 0x7, 0xc0028dec60, 0x10, 0xc00293af90, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000efee80, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 728 [sync.Cond.Wait]: runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:310 sync.runtime_notifyListWait(0xc000979380, 0xc00000000a) /usr/local/go/src/runtime/sema.go:510 +0xf8 sync.(*Cond).Wait(0xc000979370) /usr/local/go/src/sync/cond.go:56 +0x9d golang.org/x/net/http2.(*pipe).Read(0xc000979368, 0xc002514001, 0x3dff, 0x3dff, 0x0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/pipe.go:65 +0xa6 golang.org/x/net/http2.transportResponseBody.Read(0xc000979340, 0xc002514001, 0x3dff, 0x3dff, 0x0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/transport.go:2005 +0xac encoding/json.(*Decoder).refill(0xc000979760, 0xc002b2a30a, 0x9) /usr/local/go/src/encoding/json/stream.go:161 +0xeb encoding/json.(*Decoder).readValue(0xc000979760, 0x0, 0x0, 0x1acf360) /usr/local/go/src/encoding/json/stream.go:136 +0x1dc encoding/json.(*Decoder).Decode(0xc000979760, 0x1af5c00, 0xc002b2a300, 0x0, 0x0) /usr/local/go/src/encoding/json/stream.go:63 +0x79 k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Read(0xc000a68540, 0xc00252a000, 0x4000, 0x4a80, 0x20dda80, 0x0, 0x38) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/framer/framer.go:150 +0x1a1 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode(0xc002342500, 0x0, 0x20e3240, 0xc0053d5e40, 0x0, 0x0, 0x0, 0xc0007f6f58, 0x455940) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/runtime/serializer/streaming/streaming.go:77 +0x89 k8s.io/client-go/rest/watch.(*Decoder).Decode(0xc005942fc0, 0xc0006b9fa8, 0x5, 0x20dda80, 0xc002acfb00, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/rest/watch/decoder.go:49 +0x7c k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc000a68570) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:93 +0x175 created by k8s.io/apimachinery/pkg/watch.NewStreamWatcher /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0x9f goroutine 627 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002ad7329, 0x7, 0xc002ad7310, 0x10, 0xc002ab0d80, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000d61f00, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 674 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000a7a680, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 667 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000efee80, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 681 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc00005ce00, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 288 [chan receive]: k8s.io/client-go/util/workqueue.(*Type).updateUnfinishedWorkLoop(0xc0004c3860) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/util/workqueue/queue.go:198 +0xe0 created by k8s.io/client-go/util/workqueue.newQueue /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/util/workqueue/queue.go:58 +0x132 goroutine 624 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002550539, 0x7, 0xc002550520, 0x10, 0xc002b02690, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000bd8e40, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 619 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00255c0b9, 0x7, 0xc00255c0a0, 0x10, 0xc00293aa50, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000d60140, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 477 [sync.Cond.Wait]: runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:310 sync.runtime_notifyListWait(0xc000740810, 0xc000000024) /usr/local/go/src/runtime/sema.go:510 +0xf8 sync.(*Cond).Wait(0xc000740800) /usr/local/go/src/sync/cond.go:56 +0x9d k8s.io/client-go/util/workqueue.(*Type).Get(0xc0004c3860, 0x0, 0x0, 0x1ba4200) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/util/workqueue/queue.go:145 +0xa0 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0008ae000, 0x0) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:178 +0x43 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc0008ae000) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:171 +0x2b k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc001a025a0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152 +0x5e k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc001a025a0, 0x3b9aca00, 0x0, 0x201, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153 +0xf8 k8s.io/apimachinery/pkg/util/wait.Until(0xc001a025a0, 0x3b9aca00, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 +0x4d created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:157 +0x32e goroutine 640 [select]: k8s.io/apimachinery/pkg/util/wait.WaitFor(0xc00639c560, 0xc000709968, 0xc00005a300, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:403 +0x12e k8s.io/apimachinery/pkg/util/wait.PollUntil(0x5f5e100, 0xc000709968, 0xc00005a300, 0x8, 0xc000709938) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:356 +0x56 k8s.io/client-go/tools/cache.WaitForCacheSync(0xc00005a300, 0xc000709a08, 0x1, 0x1, 0x7) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/shared_informer.go:108 +0x7c sigs.k8s.io/controller-runtime/pkg/cache/internal.(*specificInformersMap).Get(0xc00080c000, 0x1de23dd, 0xa, 0x1ddeb70, 0x7, 0x18f1564, 0x7, 0x20e1b00, 0xc000554f00, 0x0, ...) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:166 +0x13b sigs.k8s.io/controller-runtime/pkg/cache/internal.(*InformersMap).Get(0xc0003238a0, 0x1de23dd, 0xa, 0x1ddeb70, 0x7, 0x18f1564, 0x7, 0x20e1b00, 0xc000554f00, 0x0, ...) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/cache/internal/deleg_map.go:85 +0x105 sigs.k8s.io/controller-runtime/pkg/cache.(*informerCache).Get(0xc0004a8020, 0x211aa80, 0xc00004e090, 0x1ddded5, 0x7, 0x1dddca5, 0x7, 0x20e1b00, 0xc000554f00, 0x0, ...) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/cache/informer_cache.go:53 +0xe4 sigs.k8s.io/controller-runtime/pkg/client.(*DelegatingReader).Get(0xc000323b80, 0x211aa80, 0xc00004e090, 0x1ddded5, 0x7, 0x1dddca5, 0x7, 0x20e1b00, 0xc000554f00, 0xc0003fc701, ...) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/client/split.go:51 +0x13e github.com/gardener/test-infra/pkg/testrunner.GetHostURLFromIngress(0x2130720, 0xc000789860, 0x1ddded5, 0x7, 0x1dddca5, 0x7, 0x0, 0x0, 0x0, 0xc0019eae70) /go/src/github.com/gardener/test-infra/pkg/testrunner/util.go:97 +0xc3 github.com/gardener/test-infra/pkg/testrunner.GetArgoHost(...) /go/src/github.com/gardener/test-infra/pkg/testrunner/util.go:47 github.com/gardener/test-infra/pkg/testrunner.GetArgoURL(0x2130720, 0xc000789860, 0xc0003e98c0, 0x0, 0x0, 0x0, 0x21) /go/src/github.com/gardener/test-infra/pkg/testrunner/util.go:33 +0x63 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000740d00, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:56 +0x2e7 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 630 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0028de389, 0x7, 0xc0028de370, 0x10, 0xc0028697d0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000750c80, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 695 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc00005db00, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 638 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002a4a0b9, 0x7, 0xc002a4a0a0, 0x10, 0xc002a401e0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000bd9800, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 696 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000e617c0, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 684 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000e61440, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 586 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000bd9380, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 761 [sync.Cond.Wait]: runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:310 sync.runtime_notifyListWait(0xc0019eaf48, 0x1) /usr/local/go/src/runtime/sema.go:510 +0xf8 sync.(*Cond).Wait(0xc0019eaf38) /usr/local/go/src/sync/cond.go:56 +0x9d k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop(0xc0019eaf20, 0xc005502c50, 0x0, 0x0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/delta_fifo.go:423 +0xa6 k8s.io/client-go/tools/cache.(*controller).processLoop(0xc0024e8800) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/controller.go:150 +0x40 k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc000edce00) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152 +0x5e k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000edce00, 0x3b9aca00, 0x0, 0xc00077fe01, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153 +0xf8 k8s.io/apimachinery/pkg/util/wait.Until(...) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 k8s.io/client-go/tools/cache.(*controller).Run(0xc0024e8800, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/controller.go:124 +0x2fd k8s.io/client-go/tools/cache.(*sharedIndexInformer).Run(0xc00241e510, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/shared_informer.go:227 +0x47f created by sigs.k8s.io/controller-runtime/pkg/cache/internal.(*specificInformersMap).addInformerToMap /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:204 +0x3db goroutine 628 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002867fb9, 0x7, 0xc002867fa0, 0x10, 0xc002869d10, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000ef7740, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 691 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000facb40, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 631 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002867bd9, 0x7, 0xc002867bc0, 0x10, 0xc002869bc0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000bd9380, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 686 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000ef7740, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 644 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00248da69, 0x7, 0xc00248da50, 0x10, 0xc002b026c0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc00005c580, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 645 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00265ed70, 0x7, 0xc00265ed60, 0x10, 0xc002a8e180, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000d61f80, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 386 [chan receive]: k8s.io/client-go/tools/cache.(*sharedProcessor).run(0xc0006c5810, 0xc0007f6120) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/shared_informer.go:425 +0x49 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1() /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:54 +0x2e k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc00027c390, 0xc00000c680) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:71 +0x59 created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:69 +0x62 goroutine 679 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000ae8640, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 370 [select]: k8s.io/client-go/util/workqueue.(*delayingType).waitingLoop(0xc0004c39e0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:209 +0x344 created by k8s.io/client-go/util/workqueue.newDelayingQueue /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/util/workqueue/delaying_queue.go:55 +0x1d4 goroutine 791 [runnable]: k8s.io/apimachinery/pkg/util/wait.poller.func1.1(0xc00248e6c0, 0x5f5e100, 0x0, 0xc00248e660) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:450 +0x18e created by k8s.io/apimachinery/pkg/util/wait.poller.func1 /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:433 +0x8c goroutine 676 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000bd84c0, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 621 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00270e4f0, 0x7, 0xc00270e4e0, 0x10, 0xc00293b650, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000ae8640, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 371 [select]: sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).Start(0xc0001948c0, 0xc00005b0e0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/manager/internal.go:282 +0x170 github.com/gardener/test-infra/pkg/testrunner.StartWatchController.func1(0x2143f60, 0xc0001948c0, 0xc00005b0e0, 0x212a420, 0xc000138fc0) /go/src/github.com/gardener/test-infra/pkg/testrunner/testrunner.go:75 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.StartWatchController /go/src/github.com/gardener/test-infra/pkg/testrunner/testrunner.go:74 +0x490 goroutine 629 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00265e589, 0x7, 0xc00265e570, 0x10, 0xc002a40240, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000373e00, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 636 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00248d649, 0x7, 0xc00248d630, 0x10, 0xc00293a900, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000ef9600, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 223 [select]: k8s.io/client-go/tools/cache.(*processorListener).pop(0xc0009dbf00) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/shared_informer.go:517 +0x18d k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc0006c5870, 0xc0001ae400) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:71 +0x59 created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:69 +0x62 goroutine 637 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002550929, 0x7, 0xc002550910, 0x10, 0xc002b026f0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc00040f5c0, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 692 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc00005c580, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 697 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000750080, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 623 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0026b0409, 0x7, 0xc0026b03f0, 0x10, 0xc00273f0e0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc00005ce00, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 659 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00265e990, 0x7, 0xc00265e980, 0x10, 0xc0024ebf80, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000ef6840, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 278 [chan receive]: k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc000740200) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/watch/mux.go:207 +0x66 created by k8s.io/apimachinery/pkg/watch.NewBroadcaster /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/watch/mux.go:75 +0xcc goroutine 682 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000bd8e40, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 92 [chan receive]: sigs.k8s.io/controller-runtime/pkg/cache/internal.(*InformersMap).Start(0xc0003238a0, 0xc00005a300, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/cache/internal/deleg_map.go:62 +0x97 sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).waitForCache.func1(0xc0001948c0) /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/manager/internal.go:338 +0x39 created by sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).waitForCache /go/src/github.com/gardener/test-infra/vendor/sigs.k8s.io/controller-runtime/pkg/manager/internal.go:337 +0x5a goroutine 222 [chan receive]: k8s.io/client-go/tools/cache.(*processorListener).run.func1.1(0xc0009e2dc8, 0x41742e, 0x7fc18ed92558) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/shared_informer.go:549 +0x54 k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff(0x989680, 0x3ff0000000000000, 0x3fb999999999999a, 0x5, 0x0, 0xc000794dd8, 0xc0009e2ee8, 0xc0009e2de8) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:265 +0x51 k8s.io/client-go/tools/cache.(*processorListener).run.func1() /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/shared_informer.go:548 +0x79 k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc0009e2f40) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152 +0x5e k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000794f40, 0xdf8475800, 0x0, 0x1, 0xc00057c000) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153 +0xf8 k8s.io/apimachinery/pkg/util/wait.Until(...) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 k8s.io/client-go/tools/cache.(*processorListener).run(0xc0009dbf00) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/shared_informer.go:546 +0x9b k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc0006c5870, 0xc0001ae3f0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:71 +0x59 created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:69 +0x62 goroutine 635 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00270e110, 0x7, 0xc00270e100, 0x10, 0xc00293b5c0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000ef8ac0, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 387 [chan receive]: k8s.io/client-go/tools/cache.(*controller).Run.func1(0xc00005a300, 0xc00022ae00) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/controller.go:103 +0x34 created by k8s.io/client-go/tools/cache.(*controller).Run /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/controller.go:102 +0xae goroutine 388 [select]: k8s.io/client-go/tools/cache.(*Reflector).watchHandler(0xc000836140, 0x20e3800, 0xc000a68570, 0xc000563c00, 0xc001056420, 0xc00005a300, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/reflector.go:316 +0x1ec k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch(0xc000836140, 0xc00005a300, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/reflector.go:287 +0x833 k8s.io/client-go/tools/cache.(*Reflector).Run.func1() /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/reflector.go:125 +0x33 k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc00007def0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152 +0x5e k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000563ef0, 0x3b9aca00, 0x0, 0x1, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153 +0xf8 k8s.io/apimachinery/pkg/util/wait.Until(...) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 k8s.io/client-go/tools/cache.(*Reflector).Run(0xc000836140, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/reflector.go:124 +0x169 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1() /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:54 +0x2e k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc00027c7e0, 0xc00000c760) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:71 +0x59 created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:69 +0x62 goroutine 726 [select]: k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch.func2(0xc000836140, 0xc00005a300, 0xc0007f6de0, 0xc001056420) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/reflector.go:227 +0x178 created by k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/reflector.go:221 +0x260 goroutine 657 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00255d969, 0x7, 0xc00255d950, 0x10, 0xc00293b680, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000facb40, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 590 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc00040e300, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 528 [sync.Cond.Wait]: runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:310 sync.runtime_notifyListWait(0xc0027ff0c0, 0xc000000000) /usr/local/go/src/runtime/sema.go:510 +0xf8 sync.(*Cond).Wait(0xc0027ff0b0) /usr/local/go/src/sync/cond.go:56 +0x9d golang.org/x/net/http2.(*pipe).Read(0xc0027ff0a8, 0xc002505200, 0x200, 0x200, 0x0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/pipe.go:65 +0xa6 golang.org/x/net/http2.transportResponseBody.Read(0xc0027ff080, 0xc002505200, 0x200, 0x200, 0x0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/golang.org/x/net/http2/transport.go:2005 +0xac encoding/json.(*Decoder).refill(0xc0027286e0, 0xc0025e23c0, 0x40be26) /usr/local/go/src/encoding/json/stream.go:161 +0xeb encoding/json.(*Decoder).readValue(0xc0027286e0, 0x0, 0x0, 0x1acf360) /usr/local/go/src/encoding/json/stream.go:136 +0x1dc encoding/json.(*Decoder).Decode(0xc0027286e0, 0x1af5c00, 0xc0025e23c0, 0xc000a09500, 0x10d2a6a) /usr/local/go/src/encoding/json/stream.go:63 +0x79 k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Read(0xc002711e30, 0xc002770c00, 0x400, 0x400, 0x0, 0x203000, 0x38) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/framer/framer.go:150 +0x1a1 k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode(0xc00272b310, 0x0, 0x20e3240, 0xc00190b340, 0x7fc190ffdd98, 0x0, 0xc000082700, 0xc000443440, 0x437a6e) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/runtime/serializer/streaming/streaming.go:77 +0x89 k8s.io/client-go/rest/watch.(*Decoder).Decode(0xc002719240, 0xc00005db00, 0xc00057c3c0, 0x0, 0xc000a09758, 0x169587f, 0x872796) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/rest/watch/decoder.go:49 +0x7c k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc002711e60) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:93 +0x175 created by k8s.io/apimachinery/pkg/watch.NewStreamWatcher /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0x9f goroutine 705 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000ef8000, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 675 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc0004dbb00, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 654 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0027c4110, 0x7, 0xc0027c4100, 0x10, 0xc002a40900, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000e617c0, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 651 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00255cca9, 0x7, 0xc00255cc90, 0x10, 0xc002a40690, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000a7ba80, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 680 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000a7bc00, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 652 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002ad6f29, 0x7, 0xc002ad6f10, 0x10, 0xc002ab0ba0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000bd9bc0, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 647 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0024889a0, 0x7, 0xc002488990, 0x10, 0xc002711fb0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000ae8000, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 648 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002550d09, 0x7, 0xc002550cf0, 0x10, 0xc00293b470, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc0004db940, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 649 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc00255c8b9, 0x7, 0xc00255c8a0, 0x10, 0xc00293aab0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000ef8000, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 765 [chan receive]: k8s.io/client-go/tools/cache.(*controller).Run.func1(0xc00005a300, 0xc0024e8800) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/controller.go:103 +0x34 created by k8s.io/client-go/tools/cache.(*controller).Run /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/controller.go:102 +0xae goroutine 655 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002867449, 0x7, 0xc002867430, 0x10, 0xc002869920, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000750080, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 656 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0028df069, 0x7, 0xc0028df050, 0x10, 0xc002b02660, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000ef8600, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 673 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000bd9800, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 583 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000d61ac0, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 677 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000d60140, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 587 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc00040f940, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 653 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002866870, 0x7, 0xc002866860, 0x10, 0xc002869e60, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc00005db00, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 589 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000d61f80, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 592 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc0004db940, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 678 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000751fc0, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 669 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000372380, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 661 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0025510f9, 0x7, 0xc0025510e0, 0x10, 0xc002b02840, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc0010598c0, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 671 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc001088800, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 662 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc0010598c0, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 666 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000e60a80, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 668 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000fad140, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 616 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc0026b0100, 0x7, 0xc0026b00f0, 0x10, 0xc00273e9f0, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000a7a680, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 650 [select]: github.com/gardener/test-infra/pkg/testmachinery/controller/watch.(*watch).WatchUntil(0xc0007a86c0, 0x274a48a78000, 0xc002a4a4b9, 0x7, 0xc002a4a4a0, 0x10, 0xc002a40420, 0x0, 0x0) /go/src/github.com/gardener/test-infra/pkg/testmachinery/controller/watch/watch.go:83 +0x371 github.com/gardener/test-infra/pkg/testrunner.(*Run).Exec(0xc000efe380, 0x212a420, 0xc00105bc00, 0x30f2940, 0x7ffdf61e7a5e, 0xb) /go/src/github.com/gardener/test-infra/pkg/testrunner/run.go:61 +0x420 github.com/gardener/test-infra/pkg/testrunner.RunList.Run.func1(0xc00105cd70, 0x30f2940, 0xc00065a000, 0x2e, 0x30, 0xc000ff5260, 0x24, 0xc00013c1c0, 0x1, 0x1, ...) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:81 +0x1ff created by github.com/gardener/test-infra/pkg/testrunner.RunList.Run /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:75 +0x302 goroutine 591 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000ae8000, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 588 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000efe380, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 665 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000740d00, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 664 [chan send]: github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent.func1(0xc000d60900, 0xc00057c3c0) /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:150 +0x3f created by github.com/gardener/test-infra/pkg/testrunner.triggerRunEvent /go/src/github.com/gardener/test-infra/pkg/testrunner/list.go:149 +0x56 goroutine 766 [select]: k8s.io/client-go/tools/cache.(*Reflector).watchHandler(0xc00168f9a0, 0x20e3800, 0xc002711e60, 0xc0006bbc00, 0xc0004f9e00, 0xc00005a300, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/reflector.go:316 +0x1ec k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch(0xc00168f9a0, 0xc00005a300, 0x0, 0x0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/reflector.go:287 +0x833 k8s.io/client-go/tools/cache.(*Reflector).Run.func1() /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/reflector.go:125 +0x33 k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc0005f8ef0) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152 +0x5e k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0006bbef0, 0x3b9aca00, 0x0, 0x1, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153 +0xf8 k8s.io/apimachinery/pkg/util/wait.Until(...) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 k8s.io/client-go/tools/cache.(*Reflector).Run(0xc00168f9a0, 0xc00005a300) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/reflector.go:124 +0x169 k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1() /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:54 +0x2e k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc0024c4d90, 0xc0024df640) /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:71 +0x59 created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start /go/src/github.com/gardener/test-infra/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:69 +0x62 goroutine 718 [select]: k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch.func2(0xc00168f9a0, 0xc00005a300, 0xc002694ae0, 0xc0004f9e00) /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/reflector.go:227 +0x178 created by k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch /go/src/github.com/gardener/test-infra/vendor/k8s.io/client-go/tools/cache/reflector.go:221 +0x260
Add some webhook support github <-> testmachinery internal
What would you like to be added:
Some repos like https://github.com/gardener/gardener-extension-provider-openstack want to run integration tests via the tm-bot that target non-public facing networks.
For tests targeting non-public networks we have a testmachinery installation residing in said non-public networks (let's call it tm-internal).
However the tm-bot just reacts to github webhook events, hence public github cannot push events to the tm-bot from tm-internal.We should add some mechanism that allows a tm-bot from tm-internal to react on public github events.
Possible options include:
- webhook queue (like gardener bot)
- tm-public to store received webhook events targetting internal networks; tm-internal to watch these received webhook events in tm-public and act accordingly
Why is this needed:
Allow public github repos to trigger integration tests targetting a non-public network.Add support for artifactFrom
Problem
As a user a want to specify specific steps to get the input artifact.
This feature give the opportunity to create shoot --> update gardener to a new version --> test the same shoot again.Description:
Add a
artifactFrom
parameter to the step configuration so that specific steps can be referenced as input sources.
⚠️ Only steps withdefinition.name
are allowed asdefinition.labels
could include multiple steps.testflow: - name: step1 definition: name: test - name: step2 definition: label: default artifactFrom: step1Support more machine image version patterns in flavors
In flavors we can today specify either a fixed machine image version or
latest
.
For certain tests we want to test all non-deprecated versions of a certain machine image.
It would be helpful if the flavors would support such a version specification similar to how it was done for the k8sVersions (were we supportlatest
and semver).Example:
machine: type: n1-standard-2 image: name: gardenlinux version: "*" filterDeprecatedVersions: true
Workflow failed with duplicate input parameter
Description
Workflow created by the TestMachinery failed with the error:
'invalid spec: templates.create-shoot-create-testflow.inputs.parameters[8].name ''env-r5h'' is not unique'
This issue can occur if the random generated parameter names contain duplicates.
Remove testgrid sections for gardener e2e tests
Remove testgrid sections for gardener e2e tests (presumably located in https://github.com/kubernetes/test-infra/tree/master/config/testgrids/gardener) as we only want to report/upload conformance tests.
Passing secret to test-def : Format is changed when I access inside the container
I have a use-case where I want to pass a yaml file as a environment variable inside the container. When I make it as a secret and access it inside the container. The yaml format getting changed.
Testrun was unable to run if a step finds no testrun
What happened:
Testmachinery silently errors in the workflow with the following error message
message: 'invalid spec: templates.testrun.tasks.delete-shoot-delete-testflow templates.delete-shoot-delete-testflow inputs.artifacts.repo was not supplied'
The problem with the testrun is that the step before
delete-shoot
does not match a test-def.steps: - name: create dependsOn: [ generate-provider ] definition: name: create-shoot - name: tests dependsOn: [ create ] definition: label: no-test continueOnError: true - name: delete dependsOn: [ tests ] definition: name: delete-shootWhat you expected to happen:
test should ignore the step if no tests can be found and only execute the found ones.
We may want to define min required 1 on labels.Refactor Alerting
We need a proper alerting for failing testmachinery test steps.
Required information:
- which test steps have failing
- links to kibana dashboard, grafana logs, argo workflow, concourse step
- mention of corresponding people in the alert (owner and change commiters)
- environment context (provider, gardener version, k8s version, landscape)
The alert should be preferably be implemented as a Slack Alert. Alternative would be alerting via email.
replace "component-spec" language bindings w/ OCM-Lib
Context / Motivation
component-cli was deprecated (superseded by ocm-cli and removed from gardener-cicd (pipeline template and all pipelines) as part of cleanup and consolidation activity.
component-cli
as well as testmachinery uses golang bindings from component-spec repository for retrieving and deserialising OCM-Component-Descriptors. Said language-bindings were also superseded by a re-implementation (in golang) in ocm-repository.The re-implemented ocm-lib supports handling of different versions of component-descriptors w/ a stable API (which is promised to be kept stable also for future spec-versions of OCM; there are already two spec-versions: v2 and v3 (gardener-cicd currently uses v2).
Adopting the re-implemented ocm-lib in testmachinery will allow us to archive the (currently unmaintained) "component-spec"-language-bindings and (hopefully) reduce related maintenance efforts in the future.
Implementation Proposal
Remove dependencies from testmachinery towards https://github.com/gardener/component-spec, and port it to using ocm-lib instead.
add direct grafana/loki links to argo workflow resource
Argo 2.7.0 introduces the possibility to directly link to external logging facitilies.
Add the respective grafana/loki links to the argo resources (workflow).
See argoproj/argo-workflows@14478bcWrong additionalLocations in flavors lead to rejected Testruns
A flavor snippet like this:
- provider: gcp additionalLocations: - type: git allowPrivilegedContainers: false
Leads to a rendered Testrun that contains a location without any repo and revision and is therefore consequently rejected by the admission webhook.
admission webhook \"validate-testrun.tm.garden.cloud\" denied the request: [spec.locationSets[0].locations[84].repo: Required value: repo has to be defined for git TestDefinition locations, spec.locationSets[0].locations[84].revision: Required value: revision has to be defined for git TestDefinition locations]"
The flavor parsing should instead immediately reject meaningless
additionalLocations
fields (that have no repo and revision but just a type).Add PDBs to test pods
Currently our test pods get terminated when cluster nodes are rolling (i.e. due to OS updates).
However we'd prefer to let the tests pass and roll the node only after the last test pod terminated on its own.Hence some kind of PDBs are needed for the test pods to let mcm drain the nodes by waiting for PDBs to be fulfilled. Unfortunately we cannot specify
.spec.maxUnavailable
as the testpods are not supervised by a deployment/replicaset/... (see https://kubernetes.io/docs/tasks/run-application/configure-pdb/#arbitrary-controllers-and-selectors)
Hence we could only use.spec.minAvailable
which is however pretty dynamic (as the number of currently active testpods changes all the time), so this would need to be applied continuously by some controller.Configuration Namespace
Problem
When writing testruns where we have a lot of nested workflows global config (
spec.config
) to share configuration is not enough as subtrees share a lot of configuration.example:
When running a full gardener test configuration like shootname, secrets, provider, etc is shared across one sub shoot test but parallel subtree's need different configuration.
Description
To save duplicated configuration configuration of steps should be shared within its subtree.
Therefore config should be able to be inherited.could not construct Shoot client: unsupported kubernetes version \"v1.19.4\"
Checking the logs from a
conformance-testgrid-parallel
step execution, I am able to find events from the Seed but I think the Shoot related events are not being fetched because ofcould not construct Shoot client: unsupported kubernetes version \"v1.19.4\"
:time="2020-11-16T18:04:33Z" level=info msg="[SEED foo] [NAMESPACE shoot--it--tmau7-mxu] [EXTENSIONS] [OS]" 2020-11-16 20:04:33 time="2020-11-16T18:04:33Z" level=info msg="[SEED foo] [NAMESPACE shoot--it--tmau7-mxu] [EXTENSIONS] [CONTROLPLANE]" 2020-11-16 20:04:33 time="2020-11-16T18:04:33Z" level=info msg="[SEED foo] [NAMESPACE shoot--it--tmau7-mxu] [EXTENSIONS] [INFRASTRUCTURE]" 2020-11-16 20:04:33 time="2020-11-16T18:04:33Z" level=info msg="[SEED foo]" 2020-11-16 20:04:33 time="2020-11-16T18:04:33Z" level=error msg="retry failed with context deadline exceeded, last error: could not construct Shoot client: unsupported kubernetes version \"v1.19.4\"" 2020-11-16 19:59:33 time="2020-11-16T17:59:33Z" level=error msg="<nil>" 2020-11-16 19:59:33 time="2020-11-16T17:59:33Z" level=info msg="END: dump kubetest stdout last 5000 bytes" 2020-11-16 19:59:33 time="2020-11-16T17:59:33Z" level=info msg=" Test Suite Failed" 2020-11-16 19:59:33 time="2020-11-16T17:59:33Z" level=info msg=" Ginkgo ran 1 suite in 1m40.92987864s" time="2020-11-16T17:59:33Z" level=info msg=" " 2020-11-16 19:59:33 time="2020-11-16T17:59:33Z" level=info msg=" FAIL! -- 4 Passed | 1 Failed | 0 Pending | 5229 Skipped" 2020-11-16 19:59:33 time="2020-11-16T17:59:33Z" level=info msg=" Ran 5 of 5234 Specs in 89.982 seconds"
I believe the
github.com/gardener/gardener
dependency needs to be updated to include gardener/gardener@8076764.GitHub Bot command to run testruns from repo
What would you like to be added:
We need a mechanism to run predefined testruns as PR tests with the bot.In addition, a default test or location for the testruns should be customizable.
Support more version patterns in flavor configs
Today we support things like
kubernetes: pattern: "latest"
Sometimes we want to express a flavor that should select one minor version older than the latest one, something like
latest - 0.1
.
If arithmetic is too complex, we could also simply introduce a few fixed named patterns likeoneMinorBeforeLatest
,twoMinorBeforeLatest
, ... (as we usually anyway only support a limited set of minor K8s versions at a time).Support selecting the network type calico<->cilium
Testmachinery should support switching the network type via the shoot spec to allow tests to be created that don't target the default (calico) but can target cilium as well.
- Testmachinery flavors need to support this
- CreateShoot step in g/g needs to support this
Update integration step image
Currently the integration step fails to compile given test because it contains [email protected] features - ref https://concourse.ci.gardener.cloud/teams/gardener/pipelines/test-infra-master/jobs/master-head-update-job/builds/6.
Currently the used image is
eu.gcr.io/gardener-project/cc/job-image-golang:0.10.0
$ docker run eu.gcr.io/gardener-project/cc/job-image-golang:0.10.0 go version go version go1.12.6 linux/amd64
The image needs to be updated to
eu.gcr.io/gardener-project/cc/job-image-golang:0.12.0
$ docker run eu.gcr.io/gardener-project/cc/job-image-golang:0.12.0 go version go version go1.13.9 linux/amd64
PS: I am opening an issue as I don't have rights to push to
refs/meta/ci
or create a PR about it :(Use Telemetry Controller Under Test
What would you like to be added:
@dkistner has implemented a "telemetry controller" that keeps track of the control plane availability. It would make sense to have it observing the state of clusters under reconciliation/maintenance/test and report this metric to alert about poor shoot cluster control plane availability and eventually break the release/transport if KPIs are not met. Or shall this be part of the specific Gardener tests instead?Why is this needed:
We sometimes miss issues here and lack repeating test results of this most important metric (it is the only metric relevant in our SLO).Reserve Excess capacity pods are monitored in the TM health check
What happened:
The reserve-excess-capacity deployment is monitored via the argo mr.
If the cluster currently scales up the health check fails which causes the tm to be unhealthyWhat you expected to happen:
The access capacity pods are not considered for healthinessLog Collector and Log Evaluation (for excessive logs and errors) Under Test
What would you like to be added:
We don't really collect and look into pod logs regularly, but should do that actually automatically, so that we do not miss excessive (=useless) logs and errors that under normal circumstances should not appear in logs of a healthy system.Why is this needed:
We sometimes miss excessive (=useless) logs and errors that have otherwise no side effects that the current tests would catch.Automation of Synthetic Load Tests
What would you like to be added:
Measure CPU and memory footprint, measure disk and network throughput and latency, measure API server and ETCD throughput and latency of our shoot clusters automatically, so that we do not miss regressions.Why is this needed:
We sometimes miss regressions that the current tests do not catch.
⚠️ Note: Avoid duplicating the Kubernetes performance tests: https://github.com/kubernetes/perf-testsBinary files in TM configs
When TM
file
config is used with binary base64 data, the file is corrupted and the size of the file at stage input is larger than the original file encoded with thebase64
utility.I'm not a UTF-8 expert or read all the code paths, but I presume it's due to usage of
string()
typecasting and possibly due to protobufs internal to Argo. It seems likely there is some code point expansion going on as a result.Add support for global artifacts.
Issue
As a user I want to use global artifacts from serial steps as input source.
Description
Currently serial steps write their output to the global artifact registry.
To consume this kind of artifacts a special argo reference is needed.
Therefore another parameter (likeglobalInput
) needs to be introduced that specifies the usage of a global artifact.This is also needed to share files between the
testflow
and theonExit
flow.testflow: - name: demo-step definition: name: demo - name: label-step globalInput: true dependsOn: [ demo-step ] definition: label: defaultAdd support for `adminKubeconfig` subresource to tests
As of today,
TestDefinitions
and their respective code in this repo expect akubeconfig
to be presented at runtime (https://github.com/gardener/test-infra/blob/master/.test-defs/conformanceTestgrid.yaml#L56). While this is still valid for local test executions, we should also support requesting/using short-lived kubeconfigs.Gardener issues short-lived admin kubeconfigs as an alternative to static tokens: https://gardener.cloud/docs/gardener/api-reference/authentication/#adminkubeconfigrequest
Ideally, the test framework used here can be adapted to:
- accept an existing kubeconfig, if it is presented with such a file
- request an
adminkubeconfig
, if a Gardener kubeconfig is presentedPR check Pending for successful test run
/kind bug
What happened:
On gardener/gardener-extension-provider-aws#325, I triggered a test run via
/test
, which finished successfully, but the PR check is stillPending
:What you expected to happen:
The PR check status to reflect the correct test result?
Probably, there is some retry mechanism missing.Deploy prerequisiteis at runtime
What would you like to be added:
The prerequisites chart (containing argo workflows, crds, minio, ..) should be automatically installed during runtime so it can be also automatically updated and checked if all needed components are available.
Therefore we should have a look at https://github.com/gobuffalo/packr to include the complete helmchart.
We could also think of reusing the gardener resource manager(https://github.com/gardener/gardener-resource-manager) to ensure everything is up and running and gets reconciled otherwise we would have to manage leaked resource on our own.as a result I want to move all configuration to a dedicated configuration file that then also includes the helm values for the additional deployed charts.
WDYT @dguendisch @OlegLoewen
Allow filtering of testruns
What would you like to be added:
It would be nice if in the tm dashboard it is possible to filter test runs by their status, details (cloud provider, Kubernetes version, OS), time range, etc.Why is this needed:
Currently it is hard to correlate failed integration test cluster with the respective testrun to check the actual failure reason.Add semver pattern support for gardener extensions
Instead of using hard coded values here it may be a good idea to use semver ranges. Gardener/Gem offers great support for this use case.
Add support to specify semver ranges in the test configuration of the tm bot like it is implemeneted in the g/gems.
It should be possible to specify the following exmaple:
# configure default values for pull request tests test: # gardener extensions to be deployed # will be merged with the default extensions defined in garden setup. gardener-extensions: value: | os-coreos: repo: "https://github.com/gardener/gardener-extensions.git" version": "1.0.x"
Implementation
Reuse the extension gems to resolve th correct version.
We would need to replace the git tolling with GitHub api requests.Add fallback to e2e test execution
Recommend Projects
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
Recommend Topics
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web
Some thing interesting about web. New door for the world.
server
A server is a program made to process requests and deliver data to clients.
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization
Some thing interesting about visualization, use data art
Game
Some thing interesting about game, make everyone happy.
Recommend Org
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.