Git Product home page Git Product logo

fabric8-jenkins-proxy's Introduction

fabric8-jenkins-proxy Build Status Build Status

What is it?

fabric8-jenkins-proxy (Jenkins Proxy) is the sister project to fabric8-jenkins-idler(Jenkins Idler). Its task is to run a HTTP Proxy which sits in between an openshift.io user and its Jenkins instance within openshift.io. For more information refer to the Idler README.

Architectural Diagram

Data flow diagrams

The following diagrams describe the data flow within the proxy for a received GitHub webhook respectively a direct user interaction with the Jenkins service:

GitHub Webhook

Jenkins UI

How to build?

The following paragraphs describe how to build and work with the code source.

Prerequisites

The project is written in Go, so you will need a working Go installation (Go version >= 1.9.1).

The build itself is driven by GNU Make which also needs to be installed on your systems.

Last but not least, you need a running Docker daemon, since the final build artifact is a Docker container.

Make usage

Compile the code

$ make build

Build the container image

$ make image

Run the tests

$ make test

Format the code

$ make fmt

Check commit message format

$ make validate_commits

Clean up

$ make clean

More help is provided by make help.

Dependency management

The dependencies of the project are managed by Dep. To add or change the current dependencies you need to delete the Dep lock file (Gopkg.lock), update the dependency list (Gopkg.toml) and then regenerate the lock file. The process looks like this:

$ make clean
$ rm Gopkg.lock
# Update Gopkg.toml with the changes to the dependencies
$ make build
$ git add Gopkg.toml Gopkg.lock
$ git commit

Continuous Integration

At the moment Travis CI and CentOS CI are configured. Both CI systems build all merges to master as well as pull requests.

CI System
CentOS CI master, pr
Travis CI master, pr

Running locally

The repository contains a script setupLocalProxy.sh which can be used to run the Proxy locally. A prerequisite for this is access to https://console.dsaas-stg.openshift.com/. To run the script you need to export your OpenShift access token for console.dsaas-stg.openshift.com as DSAAS_PREVIEW_TOKEN. Note, In order to port forward you need to edit permissions on the dsaas-preview namespace. You need to have jq installed to run these commands. For fedora use sudo dnf install jq

Usage: ./scripts/setupLocalProxy.sh [start|stop|env|unset]

This script is used to run the Jenkins Proxy on localhost. As a prerequisite OPENSHIFT_API_TOKEN needs to be exported. In your shell (from the root of fabric8-jenkins-proxy):

To start proxy and other required services

[user@localhost ~]$ export DSAAS_PREVIEW_TOKEN=<dsaas-preview-token>
[user@localhost ~]$ ./scripts/setupLocalProxy.sh start

Run command below this in a separate terminal so that we can see logs of the above command.

[user@localhost ~]$ export DSAAS_PREVIEW_TOKEN=<dsaas-preview-token>
[user@localhost ~]$ eval $(./scripts/setupLocalProxy.sh env)
[user@localhost ~]$ fabric8-jenkins-proxy

After you stop fabric8-jenkins-proxy, you would want to stop all the dependency services as well To remove postgres container and stop port-forwarding to prod-preview's idler service and tenant service

[user@localhost ~]$ ./scripts/setupLocalProxy.sh stop

Services running as a part of this local setup:

  • idler on 9001
  • tenant service on 9002
  • postgres on 5432

This will run proxy over HTTPS on port 8080.

Testing webhooks

You can trigger local webhook delivery like so. Go to a GitHub repository generated by the OpenShift.io launcher. Find the webhook settings under Settings->Webhooks. There you can see the recent deliveries. Copy the payload of a webhook delivery into a file webhook-payload.json. Then execute the following curl command:

$ curl https://localhost:8080/github-webhook/ \
-H "Content-Type: application/json" \
-H "User-Agent: GitHub-Hookshot/c494ff1" \
-H "X-GitHub-Event: status" \
-d @webhook-payload.json

Testing Through UI

Any request that is made to proxy(i.e., port 8080) regardless of the path, proxy will send a request to idler to unidle jenkins, if it is idled.

curl https://localhost:8080/*

This would show a spinning wheel until jenkins is idle. On running locally the html page might not exist so, it will show a message on not finding the html page.

APIs

This project opens three ports 9091, 9092 and 8080. Proxy service running on 8080 is exposed at route https://jenkins.openshift.io and Jenkins API router running on 9092 is exposed on https://jenkins.api.openshift.io. API router running on 9091 is not exposed.

9091

The unexposed API router(9091) has only one API, which is info API. An example is as follows

Request: GET https://localhost:9091/api/info/ksagathi-preview

Response: {"namespace":"ksagathi-preview","requests":0,"last_visit":0,"last_request":0}

Apart from this we have Prometheus running at /metrics

9092

Jenkins API has only one API, which gets us current state of the Jenkins instance and triggers its unidling.

Request: curl -X POST -H "Authorization: Bearer $ACCESS_TOKEN" https://jenkins.api.openshift.io/api/jenkins/start -k

Response: {
                Data: { State: idled|starting|running},
                Errors : []ResponseError {
                                Code: httpStatusCode,
                                Description: "Description",
                                }
        }

fabric8-jenkins-proxy's People

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fabric8-jenkins-proxy's Issues

Remove Tenant.GetTenantInfoByNamespace

This method is not used anymore and obsolete. Let's remove it.

It also would need some sanity checks around the master_url parameter which needs to end in a '/'.

Error: http: multiple response.WriteHeader calls

{"level":"info","msg":"Redirecting to remove token from URL","ns":"vpavlin-jenkins","time":"2018-02-08T11:52:57Z"}
  | 2018/02/08 11:52:57 http: multiple response.WriteHeader calls

This prevents token removal from the URL

Create interfaces for all interaction with external services

ATM, the code creates direct connections to OpenShift or the Idler. This creates a tight coupling between the proxy and external services, making it hard to test outside of an actual fully running configuration.

We should create interfaces/facades for these external services. This will partly make the API between proxy and other services transparent and it will also allow to plug in mock/dummy implementations when needed.

Restructure source

The source should be restructured as in fabric8-jenkins-idler, creating a clearer separation between Go source code (internal vs public) and other resources.

Update README

  • Add overview section
  • Link doc images
  • Add build information

Support multi-cluster

The current Proxy/Idler solution is designed for a single cluster use case. In particular, the Idler only watches a single OpenShift instance for build and config changes. A multi-cluster solution needs to monitor multiple clusters. There are two main ways to go about this:

  • 1 Proxy, multiple Idlers (one per cluster) - in this case, the Proxy needs to determine the target cluster of the request and contact the right Idler. Effectively the Proxy needs to maintain a map of target clusters to Idler instances
  • 1 Proxy, 1 Idler where the single Idler monitors multiple OpenShift instances

Right now, I am in favour of the first option. I think it is easier to implement and architecturally "nicer"

Introduce a router component in the proxy

We should introduce a router component (interface + impl) which dispatches incoming URLs to Proxy. Handle to set of handlers.

The idea is to split up the current Proxy.Handle method, partly for readability and maintainability, but it should also allow us to unit test just the routing part (aka the proxy is able to distinguish the various incoming request types and passes them to the right type of handler)

Update GitHub webhook URLs

GitHub webhooks triggering Jenkins build have the form https://jenkins-<user>-jenkins.8a09.starter-us-east-2.openshiftapps.com/github-webhook/

In order for the Jenkins Proxy to work, the webhook URL needs to be changed when the idling is enabled for a given user.

The proxy should on startup iterate the list users and determine which user has the idler feature enabled. For these users, the webhook URL needs to be checked and potentially updated.

The idea would be to delegate the change of the URL to the launchpad (generator-backend) which should already have the required code to set/modify the webhook. @gastaldi WDYT?

Update deployment config template for Proxy

Let's set the default value for GODEBUG to empty string or 'gctrace=1' and add a new variable JC_LOG_LEVEL which seems to be missing.

Might be worth doing a overall review of template and config map to ensure the variables and defaults match the code.

Gateway Timeout

For a user who's not been enabled, going to jenkins.openshift.io results in a 504 timeout error

Tenant ID cannot be empty string error

I created a new codebase on prod-preview and when looking at the first delivery of the newly generated webhook I see:

{"Errors":[{"code":"500","detail":"Tenant ID cannot be empty string"}]}

When I then re-deliver the webhook request, I get a status of 200.

Make existing unit test compile and pass

At the moment most of the tests are not even compiling:

make test
go test -v github.com/fabric8-services/fabric8-jenkins-proxy github.com/fabric8-services/    fabric8-jenkins-proxy/api github.com/fabric8-services/fabric8-jenkins-proxy/clients github.com/    fabric8-services/fabric8-jenkins-proxy/configuration github.com/fabric8-services/    fabric8-jenkins-proxy/proxy github.com/fabric8-services/fabric8-jenkins-proxy/storage github.com/    fabric8-services/fabric8-jenkins-proxy/testutils
?   	github.com/fabric8-services/fabric8-jenkins-proxy	[no test files]
?   	github.com/fabric8-services/fabric8-jenkins-proxy/api	[no test files]
# github.com/fabric8-services/fabric8-jenkins-proxy/clients_test
clients/idler_test.go:11:35: not enough arguments in call to testutils.IdlerData1
	have ()
	want (string)
clients/tenant_test.go:12:36: not enough arguments in call to testutils.TenantData1
	have ()
	want (string)
FAIL	github.com/fabric8-services/fabric8-jenkins-proxy/clients [build failed]
?   	github.com/fabric8-services/fabric8-jenkins-proxy/configuration	[no test files]
# github.com/fabric8-services/fabric8-jenkins-proxy/proxy_test
proxy/proxy_test.go:16:36: not enough arguments in call to testutils.TenantData1
	have ()
	want (string)
proxy/proxy_test.go:18:35: not enough arguments in call to testutils.IdlerData1
	have ()
	want (string)
proxy/proxy_test.go:25:26: not enough arguments in call to proxy.NewProxy
	have (clients.Tenant, clients.WIT, clients.Idler, string, string, string)
	want (clients.Tenant, clients.WIT, clients.Idler, string, string, string,     *storage.DBService, string, int)
FAIL	github.com/fabric8-services/fabric8-jenkins-proxy/proxy [build failed]
?   	github.com/fabric8-services/fabric8-jenkins-proxy/storage	[no test files]
?   	github.com/fabric8-services/fabric8-jenkins-proxy/testutils	[no test files]
make: *** [test] Error 2
(fabric8-jenkins-proxy)
[hardy@nineveh ~/work/developer-platform/cdk-3/go/src/github.com/fabric8-services/    fabric8-jenkins-proxy (master)]$ make test
go test -v github.com/fabric8-services/fabric8-jenkins-proxy github.com/fabric8-services/    fabric8-jenkins-proxy/api github.com/fabric8-services/fabric8-jenkins-proxy/clients github.com/    fabric8-services/fabric8-jenkins-proxy/configuration github.com/fabric8-services/    fabric8-jenkins-proxy/proxy github.com/fabric8-services/fabric8-jenkins-proxy/storage github.com/    fabric8-services/fabric8-jenkins-proxy/testutils
?   	github.com/fabric8-services/fabric8-jenkins-proxy	[no test files]
?   	github.com/fabric8-services/fabric8-jenkins-proxy/api	[no test files]
# github.com/fabric8-services/fabric8-jenkins-proxy/clients_test
clients/idler_test.go:11:35: not enough arguments in call to testutils.IdlerData1
	have ()
	want (string)
clients/tenant_test.go:12:36: not enough arguments in call to testutils.TenantData1
	have ()
	want (string)
FAIL	github.com/fabric8-services/fabric8-jenkins-proxy/clients [build failed]
?   	github.com/fabric8-services/fabric8-jenkins-proxy/configuration	[no test files]
# github.com/fabric8-services/fabric8-jenkins-proxy/proxy_test
proxy/proxy_test.go:16:36: not enough arguments in call to testutils.TenantData1
	have ()
	want (string)
proxy/proxy_test.go:18:35: not enough arguments in call to testutils.IdlerData1
	have ()
	want (string)
proxy/proxy_test.go:25:26: not enough arguments in call to proxy.NewProxy
	have (clients.Tenant, clients.WIT, clients.Idler, string, string, string)
	want (clients.Tenant, clients.WIT, clients.Idler, string, string, string,     *storage.DBService, string, int)
FAIL	github.com/fabric8-services/fabric8-jenkins-proxy/proxy [build failed]
?   	github.com/fabric8-services/fabric8-jenkins-proxy/storage	[no test files]
?   	github.com/fabric8-services/fabric8-jenkins-proxy/testutils	[no test files]
make: *** [test] Error 2    

For now we should just get existing tests to compile, disabling or all deleting all tests which are obsolete. We should be able to add "make test && make fmt" to the list of targets which are getting called as part of the CI build.

Proxy fails for users on 2a cluster

Proxy now returns error for all tenants on our second OSO cluster

Got status 404 Not Found (404) from https://api.starter-us-east-2.openshift.com/oapi/v1/namespaces/jchevret-osiotest3-jenkins/deploymentconfigs/jenkins

This is due to the fact that Idler only knows about one cluster and thus asks wrong cluster for non-existing user

Temporary workaround is to return false in IsIdle call when a response StatusCode is 404. In other words, if we cannot verify whether the Jenkins is idled, just carry on with actual proxying.

Side effect might be that a user who has never logged into OSIO and does not have any spaces (i.e. does not have OSO namespaces initilased) will get proxied to a non-existent route - which results in "Application Not Available" or some 5xx error - which is the same as if he would access the Jenkins route directly

Refactor Proxy.Handle

The method is too long and has too many nested if/for statements. In its current form it is hard to track what's going on in this method.

Use Goa for defining the Proxy REST API

Currently, the REST API is defined and served by httprouter. We should switch to Goa to design and serve the API. Partly because it aligns with the other services and partly since this allows for automatic client generation on the caller side.

Remove dependency to fabric8-jenkins-idler

At the moment there is a dependency to idler in order to reuse the OpenShift and OpenShiftController structs. Instead of this dependency, the router should call the respective REST API endpoint of the Idler.

Improve Proxy logging

  • Proxy.Handle should log incoming requests. At least debug level, maybe even info.
  • Proxy.HandleError should provide context information about the caller

Change papi to api in the API endpoint URL

The use of papi is a relict of some old code. Really it should be api. The change is trivial, but it requires a change on the Idler at the same time and then a synced re-deploy.

Create performance test harness

Before enabling Proxy for all users, we need to run some basic performance validation. One thing we can do is to create GitHub webbhook requests and see how the system behaves. Another thing we can do is to simulate some "more than normal" usage by a selected number of users.

Update Jenkins build log URLs from OpenShift console pipeline view

On the pipeline page w/i the OpenShift console, there are links to the Jenkins build logs. These links refer directly to the user-specific Jenkins instance/route. These links needs to point to the Jenkins Proxy URL, so that in case of an idled Jenkins instance, the service can be first scaled up again.

This issue is a placeholder to find out where code needs to change to make this happen.

Un-idling Jenkins via directly accessing jenkins.openshift.io results in HTTP 500 error

"code": "500", "detail": "Could not login to Jenkins in manderse-jenkins namespace on behalf of the user [email protected]"

This happens when trying to directly access jenkins.openshift.io when Jenkins is idled. The Jenkins pod gets scaled up, but then the actual login and forwarding does not work and results in the above error.

If one tries to access jenkins.openshift.io after the pod is up, one gets successfully forwarded.

Allow for resetting the stats database

It should be possible to reset the stats database of the proxy. Either by setting an environment variable or by accessing an API endpoint. There should be the ability to purge the database as well as deleting the schema altogether.

Request to Proxy timeouts before request to upstream

With checks of route being available added because of #99, it sometimes happens that request to jenkins route does not finish before the request to proxy hits the timeout which then results in unexpected 504 to the loading page.

To fix this, we can use lower timeouts for the requests to Jenkins route

Stats recorded multiple times

When I tried to access jenkins.openshift.io for the first time, I eventually got redirected to Jenkins, but I found all of the following in the logs originating basically from the same login. In the best case it is a bit too verbose logging. Alternatively something gets executed a a bit too often:

January 11th 2018, 21:21:22.352 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:22.342 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:17.348 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:17.336 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:12.352 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:12.339 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:08.611 jenkins-proxy-2-b2bq1 Redirecting to https://auth.openshift.io/api/login?redirect=https:%2F%2Fjenkins.openshift.io%2Fopensearch.xml
January 11th 2018, 21:21:08.192 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:08.154 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.725 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.687 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.672 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.647 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.628 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.566 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.560 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.535 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.513 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.494 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.473 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.409 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.392 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.380 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.352 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.334 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.317 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.221 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.221 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.208 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.193 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.177 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:07.159 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.992 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.987 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.973 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.921 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.889 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.869 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.809 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.788 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.755 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.714 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.695 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.682 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.649 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.620 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.541 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.506 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.494 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.491 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.454 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.431 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.382 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.336 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.333 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.298 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.268 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.175 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.170 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.083 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:06.074 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:05.877 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:05.866 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:05.604 jenkins-proxy-2-b2bq1 New user hardyferentschik-jenkins
January 11th 2018, 21:21:05.602 jenkins-proxy-2-b2bq1 Recording stats for hardyferentschik-jenkins
January 11th 2018, 21:21:05.462 jenkins-proxy-2-b2bq1 Cached Jenkins route jenkins-hardyferentschik-jenkins.8a09.starter-us-east-2.openshiftapps.com in node015jelxkdzfpaw1xto3ah7cr3ir738.node0
January 11th 2018, 21:21:05.462 jenkins-proxy-2-b2bq1 Redirecting to https://jenkins.openshift.io/
January 11th 2018, 21:21:05.353 jenkins-proxy-2-b2bq1 Loaded OSO token
January 11th 2018, 21:21:05.353 jenkins-proxy-2-b2bq1 Logging in https://jenkins-hardyferentschik-jenkins.8a09.starter-us-east-2.openshiftapps.com/
January 11th 2018, 21:21:05.307 jenkins-proxy-2-b2bq1 Extracted Tenant Info: hardyferentschik-jenkins
January 11th 2018, 21:21:05.298 jenkins-proxy-2-b2bq1 Found token info in query
January 11th 2018, 21:21:05.298 jenkins-proxy-2-b2bq1 Extracted JWT token
January 11th 2018, 21:21:05.298 jenkins-proxy-2-b2bq1 Extracted UID from JWT token
January 11th 2018, 21:21:05.298 jenkins-proxy-2-b2bq1 Tenant by id: http://f8tenant/api/tenants/63aaadbe-e126-481a-8dfb-b871d3274311
January 11th 2018, 21:21:04.899 jenkins-proxy-2-b2bq1 Cached Jenkins route jenkins-hardyferentschik-jenkins.8a09.starter-us-east-2.openshiftapps.com in node0xy0bst1tttxtatbcvtxl4uow737.node0
January 11th 2018, 21:21:04.899 jenkins-proxy-2-b2bq1 Redirecting to https://jenkins.openshift.io/
January 11th 2018, 21:21:04.787 jenkins-proxy-2-b2bq1 Loaded OSO token
January 11th 2018, 21:21:04.787 jenkins-proxy-2-b2bq1 Logging in https://jenkins-hardyferentschik-jenkins.8a09.starter-us-east-2.openshiftapps.com/
January 11th 2018, 21:21:04.739 jenkins-proxy-2-b2bq1 Extracted Tenant Info: hardyferentschik-jenkins
January 11th 2018, 21:21:04.733 jenkins-proxy-2-b2bq1 Extracted UID from JWT token
January 11th 2018, 21:21:04.733 jenkins-proxy-2-b2bq1 Tenant by id: http://f8tenant/api/tenants/63aaadbe-e126-481a-8dfb-b871d3274311
January 11th 2018, 21:21:04.731 jenkins-proxy-2-b2bq1 Found token info in query
January 11th 2018, 21:21:04.731 jenkins-proxy-2-b2bq1 Extracted JWT token
January 11th 2018, 21:21:04.029 jenkins-proxy-2-b2bq1 Cached Jenkins route jenkins-hardyferentschik-jenkins.8a09.starter-us-east-2.openshiftapps.com in node0gaig4w9ez6pe1jbudzy2tccx8736.node0
January 11th 2018, 21:21:04.029 jenkins-proxy-2-b2bq1 Redirecting to https://jenkins.openshift.io/
January 11th 2018, 21:21:03.857 jenkins-proxy-2-b2bq1 Loaded OSO token
January 11th 2018, 21:21:03.857 jenkins-proxy-2-b2bq1 Logging in https:// jenkins-hardyferentschik-jenkins.8a09.starter-us-east-2.openshiftapps.com/
January 11th 2018, 21:21:03.806 jenkins-proxy-2-b2bq1 Extracted Tenant Info: hardyferentschik-jenkins
January 11th 2018, 21:21:03.798 jenkins-proxy-2-b2bq1 Extracted JWT token
January 11th 2018, 21:21:03.798 jenkins-proxy-2-b2bq1 Extracted UID from JWT token
January 11th 2018, 21:21:03.798 jenkins-proxy-2-b2bq1 Tenant by id: http://f8tenant/api/ tenants/63aaadbe-e126-481a-8dfb-b871d3274311
January 11th 2018, 21:21:03.797 jenkins-proxy-2-b2bq1 Found token info in query
January 11th 2018, 21:21:03.037 jenkins-proxy-2-b2bq1 Cached Jenkins route jenkins-hardyferentschik-jenkins.8a09.starter-us-east-2.openshiftapps.com in node0145jcvp6c9ujon8wunsu0p6hk735.node0
January 11th 2018, 21:21:03.037 jenkins-proxy-2-b2bq1 Redirecting to https://jenkins.openshift.io/
January 11th 2018, 21:21:02.929 jenkins-proxy-2-b2bq1 Logging in https:// jenkins-hardyferentschik-jenkins.8a09.starter-us-east-2.openshiftapps.com/
January 11th 2018, 21:21:02.928 jenkins-proxy-2-b2bq1 Loaded OSO token
January 11th 2018, 21:21:02.882 jenkins-proxy-2-b2bq1 Extracted Tenant Info: hardyferentschik-jenkins
January 11th 2018, 21:21:02.868 jenkins-proxy-2-b2bq1 Found token info in query
January 11th 2018, 21:21:02.868 jenkins-proxy-2-b2bq1 Extracted JWT token
January 11th 2018, 21:21:02.868 jenkins-proxy-2-b2bq1 Extracted UID from JWT token
January 11th 2018, 21:21:02.868 jenkins-proxy-2-b2bq1 Tenant by id: http://f8tenant/api/ tenants/63aaadbe-e126-481a-8dfb-b871d3274311
January 11th 2018, 21:21:02.131 jenkins-proxy-2-b2bq1 Cached Jenkins route jenkins-hardyferentschik-jenkins.8a09.starter-us-east-2.openshiftapps.com in node0da054fquhs742ut5qup40rqm734.node0
January 11th 2018, 21:21:02.131 jenkins-proxy-2-b2bq1 Redirecting to https://jenkins.openshift.io/
January 11th 2018, 21:21:00.754 jenkins-proxy-2-b2bq1 Loaded OSO token
January 11th 2018, 21:21:00.754 jenkins-proxy-2-b2bq1 Logging in https:// jenkins-hardyferentschik-jenkins.8a09.starter-us-east-2.openshiftapps.com/
January 11th 2018, 21:21:00.101 jenkins-proxy-2-b2bq1 Extracted Tenant Info: hardyferentschik-jenkins

Create a StatisticRecorder

The idea is that the StatisticRecorder is a component running in a separate goroutine listening on a stats channel and updating the stats in the DB on each channel read.

This relates to issue #75

Unexpected calls to ProxyAPI.Info

When turning on debug trace, we get a whole bunch of:

{"level":"debug","msg":"Did not find data for none","time":"2018-02-09T11:00:21Z"}

Looking at the underlying code I am also wondering a bit about the implementation:

func (api *ProxyAPI) Info(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
	ns := ps.ByName("namespace")
	s, notFound, err := api.storageService.GetStatisticsUser(ns)
	if err != nil {
		if notFound {
			log.Debugf("Did not find data for %s", ns)
		} else {
			log.Error(err) //FIXME
			fmt.Fprintf(w, "{'error': '%s'}", err)
			return
		}
	}
	c, err := api.storageService.GetRequestsCount(ns)
	if err != nil {
		log.Error(err) //FIXME
		fmt.Fprintf(w, "{'error': '%s'}", err)
		return
	}
	resp := APIResponse{
		Namespace:   ns,
		Requests:    c,
		LastRequest: s.LastBufferedRequest,
		LastVisit:   s.LastAccessed,
	}

	json.NewEncoder(w).Encode(resp)
}

If there is no record found, will there be also an error? That seems odd.

Create a Replayer component

Extract a component for Proxy.ProcessBuffer which acts as "Replayer" (name to be discussed). The idea to refactor Proxy.ProcessBuffer easier to understand (less nested levels).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.