browserup / browserup-proxy Goto Github PK

View Code? Open in Web Editor NEW

163.0 163.0 42.0 5.09 MB

BrowserUp Proxy is a free utility to watch, test, and manipulate web application network traffic and performance.

Home Page: https://browserup.com

License: Apache License 2.0

Java 59.10% Groovy 37.11% Python 3.63% Dockerfile 0.06% JavaScript 0.04% Ruby 0.06%

browserup-proxy's People

Contributors

Stargazers

Watchers

browserup-proxy's Issues

Fix broken path references/tests pointing to /lightbody/ url references

Prior to release, I searched our codebase for stale references to "lightbody" URLs that pointed to Patrick Lightbody, the original author's github. I found some references that look like we should both fix them, and also make sure the tests are being run and working. It looks like they should fail. If they should be ignored, we/I should understand why.

https://github.com/browserup/browserup-proxy/blob/master/browserup-proxy-mitm/src/test/groovy/com/browserup/bup/mitm/KeyStoreFileCertificateSourceTest.groovy#L29-L30

https://github.com/browserup/browserup-proxy/blob/master/browserup-proxy-mitm/src/test/groovy/com/browserup/bup/mitm/PemFileCertificateSourceTest.groovy#L36-L38

https://github.com/browserup/browserup-proxy/blob/master/browserup-proxy-mitm/src/test/groovy/com/browserup/bup/mitm/TrustSourceTest.groovy#L98

As part of this ticket, please search the project for "lightbody" and make sure those references are fixed and understood. It is ok to point to the old BrowserMob tickets for reference to a github issue reported there. What we should avoid is pointing to invalid URLs that replace the name BrowserMob with browserup but points to locations that don't exist.

REST API for asserting on content of all URLs matching a regular expression pattern

Similar to #16, use #13 to implement content assertions methods for all URLs that match a given regular expression pattern accessible via REST.

assertAnyUrlContentLengthWithin(Pattern.compile("."), 100000 / bytes /);
assertAnyUrlContentContains(".js", "Copyright");
assertAnyUrlContentDoesNotContain("", "ERROR");
assertAnyUrlResponseHeaderContains("/api/"), "_sessionCookie");
assertAnyUrlStatusEquals(".js", 200);

See https://github.com/browserup/browserup-driver/tree/new-api#url-content-and-performance

java.lang.ClassCastException error

[ERROR] 21:05:19.534 org.littleshoot.proxy.impl.ProxyToServerConnection.exceptionCaught(ProxyToServerConnection.java:465) - (AWAITING_INITIAL) [id: 0x7c523e00, L:/172.20.10.3:61563 - R:hz-apush20.aliexpress.com/203.119.213.192:443]: Caught an exception on ProxyToServerConnection
java.lang.ClassCastException: io.netty.buffer.PooledUnsafeDirectByteBuf cannot be cast to io.netty.handler.codec.http.HttpObject
	at org.littleshoot.proxy.impl.ProxyConnection.read(ProxyConnection.java:122)
	at org.littleshoot.proxy.impl.ProxyToServerConnection.read(ProxyToServerConnection.java:229)
	at org.littleshoot.proxy.impl.ProxyConnection.channelRead0(ProxyConnection.java:582)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
	at org.littleshoot.proxy.impl.ProxyConnection$ResponseReadMonitor.channelRead(ProxyConnection.java:732)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:328)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:302)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
	at org.littleshoot.proxy.impl.ProxyConnection$BytesReadMonitor.channelRead(ProxyConnection.java:686)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at java.lang.Thread.run(Thread.java:748)

API to return most recent entry for most recent HTTP request that matches a URL regular expression pattern

Similar to #13, but only returns the most recently recorded matching entry.

[enhancement] Generate static HTML OpenAPI Documentation and add it to GitHub

Is your feature request related to a problem? Please describe.
As a developer or tester, I have no way to view the new openapi/swagger definitions.

Describe the solution you'd like
We should provide an online, hosted static output of the swagger UI documentation as HTML, so that our output is always up to date. We should regenerate this at build time automatically so it is always "correct." It should not be necessary to run the proxy to see the openapi doc--it should be available on github.

Swagger UI can consume an openapi/swagger definition or endpoint (for example, an openapi.yaml, or openapi.json or the URL from our REST api if we have that stood up) and use that to generate static HTML output documentation pages. I'd like us to do that, link to those pages from the README, and remove the large table of HTML definitions from the readme. We can have a docs subfolder to hold these and link to it, or use github pages if that is necessary.

https://swagger.io/resources/articles/documenting-apis-with-swagger/

https://swagger.io/tools/swagger-ui/download/

As an alternative, if we can't come up with good build steps or a gradle step that generates the open api HTML automatically, in the JS / node world, the commands below can work with a swagger.yml created from our annotations:

npx api2html -o dist/index.html swagger.yml
git add dist/index.html
git commit -m 'Update rebuild all documentation'
git push

As a related matter, we will also need to use swagger gen to generate REST clients, so we need to get a handle on capturing the yml file, and using it in build steps to spit out output.

NumberFormatException when specifying upstream proxy

Describe the bug
browserup-proxy throws me 500 Server Error when I specify an upstream proxy. I'm switching away from browsermob-proxy which worked with upstream proxies just fine, but hasn't updated for a while and lacks brotli support.

To Reproduce
Steps to reproduce the behavior:

$ docker run -p 28700:8080 autifyhq/browserup-proxy:2.0.1

$ curl -X "POST" "http://127.0.0.1:28700/proxy?httpProxy=ip:port&proxyUsername=username&proxyPassword=password"

<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 500 Server Error</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /proxy. Reason:
<pre>    Server Error</pre></p><h3>Caused by:</h3><pre>java.lang.ExceptionInInitializerError
        at org.mvel2.PropertyAccessor.set(PropertyAccessor.java:249)
        at org.mvel2.PropertyAccessor.set(PropertyAccessor.java:135)
        at org.mvel2.MVEL.setProperty(MVEL.java:1088)
        at com.google.sitebricks.MvelEvaluator.write(MvelEvaluator.java:64)
        at com.google.sitebricks.binding.MvelRequestBinder.bind(MvelRequestBinder.java:79)
        at com.google.sitebricks.routing.WidgetRoutingDispatcher.bindAndReply(WidgetRoutingDispatcher.java:97)
        at com.google.sitebricks.routing.WidgetRoutingDispatcher.dispatch(WidgetRoutingDispatcher.java:88)
        at com.google.sitebricks.DebugModeRoutingDispatcher.dispatch(DebugModeRoutingDispatcher.java:56)
        at com.google.sitebricks.SitebricksFilter.doFilter(SitebricksFilter.java:62)
        at com.google.sitebricks.HiddenMethodFilter.doFilter(HiddenMethodFilter.java:70)
        at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:121)
        at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:133)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1591)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1581)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1307)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:482)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1549)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1204)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:494)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:374)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:268)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:782)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:918)
        at java.base/java.lang.Thread.run(Thread.java:830)
Caused by: java.lang.NumberFormatException: For input string: &quot;14-&quot;
        at java.base/jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2054)
        at java.base/jdk.internal.math.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
        at java.base/java.lang.Double.parseDouble(Double.java:549)
        at org.mvel2.util.ParseTools.&lt;clinit&gt;(ParseTools.java:62)
        ... 34 more
</pre>
<h3>Caused by:</h3><pre>java.lang.NumberFormatException: For input string: &quot;14-&quot;
        at java.base/jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2054)
        at java.base/jdk.internal.math.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
        at java.base/java.lang.Double.parseDouble(Double.java:549)
        at org.mvel2.util.ParseTools.&lt;clinit&gt;(ParseTools.java:62)
        at org.mvel2.PropertyAccessor.set(PropertyAccessor.java:249)
        at org.mvel2.PropertyAccessor.set(PropertyAccessor.java:135)
        at org.mvel2.MVEL.setProperty(MVEL.java:1088)
        at com.google.sitebricks.MvelEvaluator.write(MvelEvaluator.java:64)
        at com.google.sitebricks.binding.MvelRequestBinder.bind(MvelRequestBinder.java:79)
        at com.google.sitebricks.routing.WidgetRoutingDispatcher.bindAndReply(WidgetRoutingDispatcher.java:97)
        at com.google.sitebricks.routing.WidgetRoutingDispatcher.dispatch(WidgetRoutingDispatcher.java:88)
        at com.google.sitebricks.DebugModeRoutingDispatcher.dispatch(DebugModeRoutingDispatcher.java:56)
        at com.google.sitebricks.SitebricksFilter.doFilter(SitebricksFilter.java:62)
        at com.google.sitebricks.HiddenMethodFilter.doFilter(HiddenMethodFilter.java:70)
        at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:121)
        at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:133)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1591)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1581)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1307)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:482)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1549)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1204)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:494)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:374)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:268)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:782)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:918)
        at java.base/java.lang.Thread.run(Thread.java:830)
</pre>
<hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.20.v20190813</a><hr/>

</body>
</html>

Expected behavior
It should work just as fine as it does without an upstream proxy:

$ curl -X "POST" "http://127.0.0.1:28700/proxy"                                                                           
{"port":8081}%

Full server log

$ docker run -p 28700:8080 autifyhq/browserup-proxy:2.0.1 
Running BrowserUp Proxy, powered by LittleProxy.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/browserup-proxy-2.0.1/lib/guice-4.2.2.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[INFO  2020-02-19T14:39:56,360 com.browserup.bup.proxy.Main] (main) Starting BrowserUp Proxy version ${project.version} 
[INFO  2020-02-19T14:39:56,417 org.eclipse.jetty.util.log] (main) Logging initialized @1108ms to org.eclipse.jetty.util.log.Slf4jLog 
[INFO  2020-02-19T14:39:56,491 org.eclipse.jetty.server.Server] (main) jetty-9.4.20.v20190813; built: 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm 14-ea+15 
[INFO  2020-02-19T14:39:56,526 org.eclipse.jetty.server.session] (main) DefaultSessionIdManager workerName=node0 
[INFO  2020-02-19T14:39:56,526 org.eclipse.jetty.server.session] (main) No SessionScavenger set, using defaults 
[INFO  2020-02-19T14:39:56,529 org.eclipse.jetty.server.session] (main) node0 Scavenging every 600000ms 
Feb 19, 2020 2:39:57 PM org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
WARNING: A provider io.swagger.v3.jaxrs2.integration.resources.OpenApiResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider io.swagger.v3.jaxrs2.integration.resources.OpenApiResource will be ignored. 
[INFO  2020-02-19T14:39:57,171 org.hibernate.validator.internal.util.Version] (main) HV000001: Hibernate Validator 6.0.17.Final 
[INFO  2020-02-19T14:39:57,351 org.eclipse.jetty.server.handler.ContextHandler] (main) Started o.e.j.s.ServletContextHandler@707ca986{/,null,AVAILABLE} 
[INFO  2020-02-19T14:39:57,369 org.eclipse.jetty.server.AbstractConnector] (main) Started ServerConnector@44afefd5{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} 
[INFO  2020-02-19T14:39:57,369 org.eclipse.jetty.server.Server] (main) Started @2061ms 
[WARN  2020-02-19T14:40:23,356 org.eclipse.jetty.server.HttpChannel] (qtp654740048-18) /proxy java.lang.ExceptionInInitializerError: null
        at org.mvel2.PropertyAccessor.set(PropertyAccessor.java:249) ~[mvel2-2.1.3.Final.jar:?]
        at org.mvel2.PropertyAccessor.set(PropertyAccessor.java:135) ~[mvel2-2.1.3.Final.jar:?]
        at org.mvel2.MVEL.setProperty(MVEL.java:1088) ~[mvel2-2.1.3.Final.jar:?]
        at com.google.sitebricks.MvelEvaluator.write(MvelEvaluator.java:64) ~[sitebricks-0.8.11.jar:0.8.11]
        at com.google.sitebricks.binding.MvelRequestBinder.bind(MvelRequestBinder.java:79) ~[sitebricks-0.8.11.jar:0.8.11]
        at com.google.sitebricks.routing.WidgetRoutingDispatcher.bindAndReply(WidgetRoutingDispatcher.java:97) ~[sitebricks-0.8.11.jar:0.8.11]
        at com.google.sitebricks.routing.WidgetRoutingDispatcher.dispatch(WidgetRoutingDispatcher.java:88) ~[sitebricks-0.8.11.jar:0.8.11]
        at com.google.sitebricks.DebugModeRoutingDispatcher.dispatch(DebugModeRoutingDispatcher.java:56) ~[sitebricks-0.8.11.jar:0.8.11]
        at com.google.sitebricks.SitebricksFilter.doFilter(SitebricksFilter.java:62) ~[sitebricks-0.8.11.jar:0.8.11]
        at com.google.sitebricks.HiddenMethodFilter.doFilter(HiddenMethodFilter.java:70) ~[sitebricks-0.8.11.jar:0.8.11]
        at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:121) ~[guice-servlet-4.2.2.jar:?]
        at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:133) ~[guice-servlet-4.2.2.jar:?]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1591) ~[jetty-servlet-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) ~[jetty-servlet-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1581) ~[jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1307) ~[jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:482) ~[jetty-servlet-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1549) ~[jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1204) ~[jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.Server.handle(Server.java:494) ~[jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:374) [jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:268) [jetty-server-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) [jetty-io-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) [jetty-io-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:782) [jetty-util-9.4.20.v20190813.jar:9.4.20.v20190813]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:918) [jetty-util-9.4.20.v20190813.jar:9.4.20.v20190813]
        at java.lang.Thread.run(Thread.java:830) [?:?]
Caused by: java.lang.NumberFormatException: For input string: "14-"
        at jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2054) ~[?:?]
        at jdk.internal.math.FloatingDecimal.parseDouble(FloatingDecimal.java:110) ~[?:?]
        at java.lang.Double.parseDouble(Double.java:549) ~[?:?]
        at org.mvel2.util.ParseTools.<clinit>(ParseTools.java:62) ~[mvel2-2.1.3.Final.jar:?]
        ... 34 more

[INFO  2020-02-19T14:43:15,391 com.browserup.bup.proxy.bricks.ProxyResource] (qtp654740048-18) POST / 
[INFO  2020-02-19T14:43:15,391 com.browserup.bup.proxy.bricks.ProxyResource] (qtp654740048-18) {} 
[INFO  2020-02-19T14:43:15,992 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp654740048-18) Starting proxy at address: 0.0.0.0/0.0.0.0:8081 
[INFO  2020-02-19T14:43:16,042 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp654740048-18) Proxy listening with TCP transport 
[INFO  2020-02-19T14:43:16,093 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp654740048-18) Proxy started at address: /0.0.0.0:8081

Please complete the following information:

Docker image from https://hub.docker.com/r/autifyhq/browserup-proxy
It uses OpenJDK 14
browserup-proxy versions: 2.0.1, 2.0.0, 1.2.1

Fix ignored tests

testWaitForQuiescenceInterruptedBySecondRequestSuccessful (QuiescenceTest)
ImpersonationPerformanceTests

API to return all entries for HTTP requests that match a URL regular expression pattern

Primarily to be used internally, create an API that returns the HAR entries for all recorded requests whose URL matches the regular expression provided.

Describe REST API with OpenAPI definition

https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md

Bad Request due to allowRequestToOriginServer == false

Describe the bug
I'm using browserup-proxy embedded in a spring boot application. I can successfully create and start the proxy server:

@Bean
public BrowserUpProxy proxy() throws IOException {
    BrowserUpProxy proxy = new BrowserUpProxyServer();
    proxy.setMitmManager(impersonatingMitmManager());
    proxy.start(proxyPort);
    return proxy;
}

Starting proxy at address: 0.0.0.0/0.0.0.0:8401 : MDC[]
Initializing thread pools for TCP with 2 acceptor threads, 8 incoming worker threads, and 8 outgoing worker threads : MDC[]
Proxy listening with TCP transport : MDC[]
Proxy started at address: /0:0:0:0:0:0:0:0:8401 : MDC[]

When I make a REST call to create a proxy I get a Bad Request:

$ curl -X POST http://127.0.0.1:8401/proxy
Bad Request to URI: /

Setting a break point in ClientToProxyConnection I see the bad request is coming because !proxyServer.isAllowRequestsToOriginServer() && isRequestToOriginServer(httpRequest) is true.

How do I allow requests to the origin server?

Getting io.netty.util.ResourceLeakDetector leak exception

browserup proxy is so far looks good for us. During local testing I'm seeing this exception periodically though:

14:41:28.972 [LittleProxy-13-ClientToProxyWorker-5] ERROR io.netty.util.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
	io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:349)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
	io.netty.buffer.CompositeByteBuf.allocBuffer(CompositeByteBuf.java:1837)
	io.netty.buffer.CompositeByteBuf.copy(CompositeByteBuf.java:1482)
	io.netty.buffer.AbstractByteBuf.copy(AbstractByteBuf.java:1187)
	io.netty.handler.codec.http.HttpObjectAggregator$AggregatedFullHttpRequest.copy(HttpObjectAggregator.java:412)
	org.littleshoot.proxy.impl.ClientToProxyConnection.copy(ClientToProxyConnection.java:1040)
	org.littleshoot.proxy.impl.ClientToProxyConnection.doReadHTTPInitial(ClientToProxyConnection.java:210)
	org.littleshoot.proxy.impl.ClientToProxyConnection.readHTTPInitial(ClientToProxyConnection.java:186)
	org.littleshoot.proxy.impl.ClientToProxyConnection.readHTTPInitial(ClientToProxyConnection.java:54)
	org.littleshoot.proxy.impl.ProxyConnection.readHTTP(ProxyConnection.java:144)
	org.littleshoot.proxy.impl.ProxyConnection.read(ProxyConnection.java:122)
	org.littleshoot.proxy.impl.ProxyConnection.channelRead0(ProxyConnection.java:582)
	io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
	org.littleshoot.proxy.impl.ProxyConnection$RequestReadMonitor.channelRead(ProxyConnection.java:709)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
	io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
	org.littleshoot.proxy.impl.ProxyConnection$BytesReadMonitor.channelRead(ProxyConnection.java:686)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:682)
	io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:617)
	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:534)
	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
	io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906)
	io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	java.lang.Thread.run(Thread.java:748)

Is this something you've seen before?

WebSocket Network Recording

Collect WebSockets traffic in Proxy (extension to HAR?) for access via APIs for assertions from browserup/browserup-driver.

[enhancement] Write HAR from server to File

Hi all,

I am writing an Arquillian Extension using your Proxy-Server to create a har file.

Problem: you do not yet have a method for this.

I tried using https://github.com/SmartBear/har-java, which turns out to be kinda tricky.

as a workaround/quick-fix until you can provide such method:
could you use the annotations @JsonPropertyOrder and @JsonProperty in your HarLog and other Har-classes? it would be much easier to use the JsonGenerator to write a file.

Review and Import PR's against BrowserMob Proxy

I'd like to grab some of the low-hanging PR fruit from the browsermob proxy, but only if they seem 1-fairly predictable, and 2-fairly well implemented. Please review these and bring them in if they seem solid.

Credentials leak fix:
https://github.com/lightbody/browsermob-proxy/pull/756/files

Brotli support:
lightbody/browsermob-proxy#783

Content Type Recognition:
lightbody/browsermob-proxy#731

use proxy.setChainedProxy(...) the error

why this error about?
err code

[WARN ] 21:47:18.740 io.netty.util.internal.logging.Slf4JLogger.warn(Slf4JLogger.java:151) - An exception was thrown by org.littleshoot.proxy.impl.ConnectionFlow$2.operationComplete()
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:91)
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:79)
	at io.netty.handler.codec.http.HttpObjectAggregator$AggregatedFullHttpMessage.release(HttpObjectAggregator.java:417)
	at org.littleshoot.proxy.impl.ProxyToServerConnection.connectionSucceeded(ProxyToServerConnection.java:938)
	at org.littleshoot.proxy.impl.ConnectionFlow.succeed(ConnectionFlow.java:168)
	at org.littleshoot.proxy.impl.ConnectionFlow.advance(ConnectionFlow.java:88)
	at org.littleshoot.proxy.impl.ConnectionFlowStep.onSuccess(ConnectionFlowStep.java:83)
	at org.littleshoot.proxy.impl.ConnectionFlow$2.operationComplete(ConnectionFlow.java:149)
	at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
	at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
	at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
	at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
	at io.netty.handler.ssl.SslHandler.setHandshakeSuccess(SslHandler.java:1392)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1235)
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1087)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1122)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:491)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:430)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1302)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:581)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
	at java.lang.Thread.run(Thread.java:748)

Debugging UI html page or JSON route [enhancement]

Is your feature request related to a problem? Please describe.
As a developer or end-user, debugging the proxy is sometimes challenging. I find on stack overflow, that there's lots of people who get stuck debugging the proxy and can't tell why their code fails.

Describe the solution you'd like

I'd like a status page served on the root proxy address that lets you know what proxies have been started on what ports, and the current har for the ports. The page should indicate whether any traffic (or connection) has come through via https/https for a given port and protocol. It should provide a link to see the current har.

If no proxies have been started, it should display instructions.

WebSocket Support

Ensure WebSocket connections are proxied properly.

Release 1.0.0!

Add build instructions to README.md
Create the binary
Create a releases page via github like so: https://help.github.com/en/articles/creating-releases
Add link to release location where to download our .jar to the README.md

[enhancement] Generate Rest Clients with Swagger CodeGen

Is your feature request related to a problem? Please describe.

Right now, as a user, I cannot use the new API enhancements with clients in other languages.

Describe the solution you'd like

https://swagger.io/tools/swagger-codegen/

Now that we have swagger, we should be able to automatically generate clients using swagger codegen. I'd like to use it to generate Python, Ruby, and Javascript proxy clients. Initially, they can be stored in a /clients/(Ruby|Python|JavaScript) subfolder of this project. After that, we need to create a strategy to push them to RubyGems, NPM, PyPi etc.

The HAR output is incompatible with the original HAR-Viewer

Describe the bug
The output HAR for a proxy does not work with the original HAR-Viewer. When uploading the HAR output from browserup-proxy, the following output is observed:


log.entries[0].response.redirectURL |   |   | is missing and it is not optional
log.entries[0].response.content.size |   |   | is missing and it is not optional
log.entries[1].response.redirectURL |   |   | is missing and it is not optional
log.entries[1].response.content.size |   |   | is missing and it is not optional
log.entries[2].response.redirectURL |   |   | is missing and it is not optional

To Reproduce
Steps to reproduce the behavior:

Create a proxy
Enabled HAR logging for the proxy
Start a client browser using the proxy and navigate to any page
Export the HAR file from the proxy
Upload the HAR file to the online HAR Viewer: http://www.softwareishard.com/har/viewer/

Expected behavior
The HAR file is parsed and rendered

Screenshots

Please complete the following information:

OS: macOS 10.14.5
Browser Chrome
Version 75.0.3770.142

Additional context
After monkeying about with the code, it appears that the issue is rooted in the model classes that were ported from the original project. There are some default values being set that are not valid HAR output for the Viewer when they are not fully populated. After removing some of these default values, the HAR file can be parsed. Of note are the following fields:

HarResponse.redirectURL <-- This should default to an empty String value instead of null
HarContent.size <-- This should default to 0 instead of null
HarLog.browser <-- This should default to null instead of an empty HarCreatorBrowser

There are likely many more, but these are the ones I noticed in my testing.

Error with MITM + Selenium

Describe the bug
I have this example code:

`BrowserUpProxy browserUpServer = new BrowserUpProxyServer();
//browserUpServer.addFirstHttpFilterFactory(new RequestFilterAdapter.FilterSource(requestFilter, 83886080));
//browserUpServer.addLastHttpFilterFactory(new ResponseFilterAdapter.FilterSource(responseFilter, 83886080));
browserUpServer.start();
System.out.println("Iniciado proxy en puerto: "+ browserUpServer.getPort());
Proxy seleniumProxy = ClientUtil.createSeleniumProxy(new InetSocketAddress("localhost", browserUpServer.getPort()));
System.setProperty("webdriver.chrome.driver","chromedriver.exe");
System.setProperty("webdriver.firefox.marionette","geckodriver.exe");
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.setAcceptInsecureCerts(true);
chromeOptions.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
chromeOptions.setCapability(CapabilityType.PROXY, seleniumProxy);

chromeOptions.setProxy(seleniumProxy);
SeleniumDrivers.primitiveDriver = new ChromeDriver(chromeOptions);`

And when i open a https web i have this error:

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than or equal to IV size (8) + tag size (16) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) [netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) [netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) [netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) [netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) [netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:677) [netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:612) [netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:529) [netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:491) [netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) [netty-all-4.1.34.Final.jar:4.1.34.Final] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_231] Caused by: javax.net.ssl.SSLException: Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than or equal to IV size (8) + tag size (16) at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[na:1.8.0_231] at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709) ~[na:1.8.0_231] at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:970) ~[na:1.8.0_231] at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896) ~[na:1.8.0_231] at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766) ~[na:1.8.0_231] at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[na:1.8.0_231] at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) ~[netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1330) ~[netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1225) ~[netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1272) ~[netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-all-4.1.34.Final.jar:4.1.34.Final] at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-all-4.1.34.Final.jar:4.1.34.Final] ... 15 common frames omitted Caused by: javax.crypto.BadPaddingException: Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than or equal to IV size (8) + tag size (16) at sun.security.ssl.CipherBox.applyExplicitNonce(CipherBox.java:936) ~[na:1.8.0_231] at sun.security.ssl.CipherBox.applyExplicitNonce(CipherBox.java:993) ~[na:1.8.0_231] at sun.security.ssl.InputRecord.decrypt(InputRecord.java:157) ~[na:1.8.0_231] at sun.security.ssl.EngineInputRecord.decrypt(EngineInputRecord.java:177) ~[na:1.8.0_231] at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:963) ~[na:1.8.0_231] ... 24 common frames omitted

Expected behavior
I want to use a request filter for modify request when X conditions are true but this part of code is commented.

Why i get this error on console?

Please complete the following information:

OS: Windows 10
Browser: Chrome
Version: Versión 79.0.3945.88 (Build oficial) (64 bits)

assertNoBrokenLinks

Perform a HEAD request (following redirects) for all links on the rendered page, confirm that all return successful responses.

We need to consider whether these requests should be proxied and included in the HAR, because if so they will impact other assertions and performance metrics of the actual scripted pages. Perhaps we tag them in some way so that they don't, or record them into a different HAR, etc. Let's discuss prior to implementation.

Multiple clients support and can not limit memory question

I want to ask again here, nice to see you forked the formal project.
I have 2 questions:

Does one proxy server support multiple clients recording HAR? It's supported from my testing, but I want to confirm it as I see someone is asking this question and trying to spawn separate proxy for each client. lightbody/browsermob-proxy#655
I tried to set xxm to limit memory, but the proxy is stuck when memory reaches the limit, it does not accept new requests any more. The proxy is expected to be long-time-running.

Java 8?

Is there anything specific that requires Java 11? I'm limited to Java 8 at the moment. Would be as simple as changing build.gradle?

Set entries[].time on HAR

Currently the Har does not have the time field populated on entries. The specifications for this field are:

time [number] - Total elapsed time of the request in milliseconds. This is the sum of all timings available in the timings object (i.e. not including -1 values) .

The time value for the request must be equal to the sum of the timings supplied in this section (excluding any -1 values).

Following must be true in case there are no -1 values (entry is an object in log.entries) :

entry.time == entry.timings.blocked + entry.timings.dns +
entry.timings.connect + entry.timings.send + entry.timings.wait +
entry.timings.receive;

REST API for asserting on timings of most recent URL matching a regular expression pattern

Using #14, implement performance assertions methods for most recent URLs that match a given regular expression pattern accessible via REST.

assertUrlResponseTimeWithin - asserts on log.entries[].time

See https://github.com/browserup/browserup-driver/tree/new-api#url-content-and-performance

Performance Assertions API

Provide API for evaluating performance of recorded network requests

Wrong scope of public api's dependencies

Describe the bug
All public available api's dependencies should be placed in api scope

Note: If I want to implement ResponseFilter I should manually add io.netty:netty-codec:4.1.43.Final library to my project. And it can produce library version incompatibility.

To Reproduce
Steps to reproduce the behavior:

Try to implement ResponseFilter

Expected behavior
io.netty.handler.codec.http.HttpResponse is available for use

Screenshots
Class isn't in Compile scope. By default it has Runtime scope.

Please complete the following information:

OS: macOS 10.15.1 (19B88)

Additional context
To fix this issue place all public available api and dependencies in api scope instead of implementation

browserup-proxy CI/CD

Configure AWS CodePipeline/CodeBuild for continuous builds and deploy of successful builds of master branch to Maven repository.

Produce zip distribution including shell scripts and dependencies

Is there an easy way to download all the binaries and instructions on how to use them?

BrowserMob Proxy had a zip file on the releases with some extra files (sh and bat) and ssl certificates.

Add ability to limit maximum number of active proxies at a given time.

This is a cross post from browsermob-proxy.

Is your feature request related to a problem? Please describe.
I currently have an application that utilizes BMP 2.1.4 to capture request/response data from a Selenium webpage load. My application is essentially and API wrapper that consists of many other abilities, but my current problem is that there's no method to limit the maximum number of proxies allowed at a given time. I could refactor a few pieces of code to limit this, but I thought it might make a neat addition to BMP/BUP.

Describe the solution you'd like
Add new CLI option of -max-proxies or similar where the value would be an integer representing the maximum number proxies allowed at a given time.

Please let me know if any additional information is needed.

assertNoBrokenImages, assertNoBrokenJavascriptLinks etc.

I'd like add assertNoBrokenLinks() and assertNoBrokenImages() to our assertions library. For this to work, we need to check every item of the appropriate type. Also perhaps assertNoBrokenJavascriptLinks(), assertNoBrokenStyleLinks(), and finally assertNoBrokenResources(), which asserts that everything on the page loaded.

https://www.guru99.com/find-broken-links-selenium-webdriver.html

REST Interface to return all entries for HTTP requests that match a URL regular expression pattern

REST interface to #13

[enhancement] - Proxy chain exceptions URLs

I want to define a list of URLs that will not go through the defined chain proxy.

One solution could be adding an exception list when setting up a chained proxy.
void setChainedProxy(InetSocketAddress chainedProxyAddress, List<String> exceptionsUrls);

Support PKI auth against upstream proxy server [enhancement]

Is your feature request related to a problem? Please describe.

In our use case, we'd like to be able to authenticate with an upstream https proxy server using PKI, and providing a client side certificate (which we'd generate on our end).

Describe the solution you'd like

It would be great if, while specifying the URL of an optional upstream proxy during har creation, we could also specify paths to a client side certificate/private key files (or just a single .jks file) which browserup would then provide as authentication to an upstream proxy.

Would something like this be feasible?

the module suport for proxy ip?

can the browserup-proxy suport for proxy ip ?

Rename BrowserMob to BrowserUp

Java Packages, Maven Artifact Name, project name, etc. all in the browserup-proxy repo should be renamed to browserup, not browsermob.

Ensure build reliability

Ensure all builds pass reliably, locally and in CI. @ignore any particularly tests to fix and create an issue to resolve.

Fix unit test - com.browserup.bup.proxy.WhitelistTest

com.browserup.bup.proxy.WhitelistTest > testWhitelistCannotShortCircuitCONNECT is failing at the moment on master branch locally and on CI.

Images are still saved to HAR if captureBinaryContent is false

Describe the bug
When I setup new HAR capture with "captureBinaryContent": false, images (jpg, png, ...) are still saved to output HAR.

To Reproduce
Steps to reproduce the behavior:

Add Proxy with "captureBinaryContent": false
Go to 'https://www.google.com'
Look in HAR
See images with content
Check HAR size

Expected behavior
I expect images not to be captured when I setup HAR capture wiht "captureBinaryContent": false

Please complete the following information:

OS: macOS Mojave 10.14.6
Browser chrome
Version 2.0.1

integer overflow on HarTiming.getWait()

I noticed below overflow occurs while running browserup after 12 hours. As per below exception, the time spent waiting for a response from the server (in milliseconds) may be larger than max integer on extreme network conditions.

java.lang.ArithmeticException: integer overflow
at java.lang.Math.toIntExact(Math.java:1011)
at com.browserup.harreader.model.HarTiming.getWait(HarTiming.java:233)
at com.browserup.bup.util.BrowserUpProxyUtil.getTotalElapsedTime(BrowserUpProxyUtil.java:152)
at com.browserup.bup.filters.HttpConnectHarCaptureFilter.proxyToServerConnectionFailed(HttpConnectHarCaptureFilter.java:185)
at com.browserup.bup.filters.BrowserUpHttpFilterChain.lambda$proxyToServerConnectionFailed$8(BrowserUpHttpFilterChain.java:229)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at com.browserup.bup.filters.BrowserUpHttpFilterChain.proxyToServerConnectionFailed(BrowserUpHttpFilterChain.java:227)
at org.littleshoot.proxy.impl.ProxyToServerConnection.become(ProxyToServerConnection.java:401)
at org.littleshoot.proxy.impl.ConnectionFlow.lambda$fail$2(ConnectionFlow.java:216)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:514)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:488)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:427)
at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:170)
at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:93)
at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:28)
at org.littleshoot.proxy.impl.ConnectionFlow.fail(ConnectionFlow.java:207)
at org.littleshoot.proxy.impl.ConnectionFlow.lambda$doProcessCurrentStep$1(ConnectionFlow.java:161)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:514)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:507)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:486)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:427)
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:129)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:269)
at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
at java.lang.Thread.run(Thread.java:748)

Errors while building

I'm typing gradle build -x test && gradle test in the root of 2.0.1, getting following errors:

java.lang.AssertionError: Expected exception: org.apache.http.conn.HttpHostConnectException
	at org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:32)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at com.github.tomakehurst.wiremock.junit.WireMockRule$1.evaluate(WireMockRule.java:73)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at sun.reflect.GeneratedMethodAccessor84.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
	at sun.reflect.GeneratedMethodAccessor83.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError
	at org.junit.Assert.fail(Assert.java:86)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.junit.Assert.assertTrue(Assert.java:52)
	at org.junit.Assert$assertTrue$8.callStatic(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:55)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:196)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:208)
	at com.browserup.bup.proxy.NewHarTest.testHttpsConnectTimeoutCapturedInHar(NewHarTest.groovy:798)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at com.github.tomakehurst.wiremock.junit.WireMockRule$1.evaluate(WireMockRule.java:73)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at sun.reflect.GeneratedMethodAccessor84.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
	at sun.reflect.GeneratedMethodAccessor83.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.lang.Thread.run(Thread.java:748)

Gradle 6.1.1

No response received / exit code 0

Describe the bug
Rarely, there are requests with exit code 0 in the har response, with the error given being "No response received", and no response gets sent to the web browser.

This seems to be the same bug in upstream browsermob-proxy reported here: lightbody/browsermob-proxy#713

We initially hit this with browsermob-proxy, and part of the reason why we switched to browserup-proxy was to see if this problem went away, but it didn't.

To Reproduce
Steps to reproduce the behavior:
If you create enough new har proxies and send enough requests through them, you may eventually see exit code 0's

Expected behavior
Expected that these errors do not occur.

Please complete the following information:

OS: RHEL7 / JDK8
Browser: Firefox 52

[enhancement] Create “Verify” (non-throwing) assert-like capability in the BrowserUp Proxy that maps 1-1 with Assert (assert can just throw if verify fails)

Is your feature request related to a problem? Please describe.

Create “Verify” (non-throwing) assert-like capability in the BrowserUp Proxy that maps 1-1 with Assert (assert can just throw if verify fails), This pattern is followed in several Selenium tools. A user may want to check these items without failing an assertion necessarily.

Describe the solution you'd like
For every assert method we offer, we should have a verify version. The assert implementation can rely on the verify, and then raise assertion failures as it does now.

NTLM login fails with 'AuthType NTLM is not supported for HTTP Authorization'

I'm trying to automate NTLM login using browserup-proxy with intergration of Selenium WebDriver and it results into 'AuthType NTLM is not supported for HTTP Authorization'

Steps to reproduce:
Following is code snippet to achieve the same -

        String path = "src/test/resources";
        File file = new File(path);
        String absolutePath = file.getAbsolutePath() + "\\chromedriver.exe";
        System.setProperty("webdriver.chrome.driver", absolutePath);

        BrowserUpProxy browserUpProxyServer = new BrowserUpProxyServer();
        browserUpProxyServer.autoAuthorization("the-myapp.com", "admin", "admin", AuthType.NTLM);
        browserUpProxyServer.start();

        Proxy seleniumProxy = ClientUtil.createSeleniumProxy(browserUpProxyServer);

        ChromeOptions chromeOptions = new ChromeOptions();
        chromeOptions.setProxy(seleniumProxy);

        driver = new ChromeDriver(chromeOptions);
        driver.get("https://the-myapp.com/bnp");

When this code is run, following error is thrown for autoAuthorization() method -

AuthType NTLM is not supported for HTTP Authorization

Expected behavior
NTLM auto authorization is expected to succeed

OS: Windows 10
Browser: Chrome
Version: 79

Increase/Config Max Response Buffer Size

Awesome work taking on the bmp proxy support!
One nice enhancement would be to allow for users to specify their own max response sizes:

private static final int DEFAULT_MAXIMUM_RESPONSE_BUFFER_SIZE = 2097152;

Currently, the 2Meg cap is too light for video and similarly large responses. My team has been building BMP from source for a while now and increasing this to 20Megs but ideally it'd be great if the user could set this via a flag.

SSL Certificate - certificate_unknown - Android

Describe the bug
Hi,
I'm trying to run the BrowserMobProxyServer with an Android device to monitor the traffic.

The problem, it's that I can't monitor the traffic that runs over HTTPS, ONLY for the apps.
The Chrome in my Android device, works fine with the BrowserUp Proxy, but I when I try to access to any app (like Play Store) it won't work, and it throws me a "javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown" exception.

I'm using Selenium to automate the tests and I'm using the BrowserUpProxy version 1.2.1.

I've already installed the ca-certificate-rsa.cer on my Android Device. Still won't work.

To Reproduce
Steps to reproduce the behavior:

Create the Embedded BrowserUpProxy with setTrustAllServers to true.
Start the Proxy, connect Android to the proxy.

Screenshots
Throws me the error:
[LittleProxy-0-ClientToProxyWorker-0] ERROR org.littleshoot.proxy.impl.ClientToProxyConnection - (NEGOTIATING_CONNECT) [id: 0x53e1bf22, L:0.0.0.0/0.0.0.0:8080 ! R:/10.10.40.77:46362]: Caught an exception on ClientToProxyConnection io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:682) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:617) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:534) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.base/java.lang.Thread.run(Thread.java:835) Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131) at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:307) at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:285) at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:180) at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164) at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:681) at java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:636) at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:454) at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:433) at java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:634) at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1332) at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1227) at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1274) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ... 16 more [LittleProxy-0-ProxyToServerWorker-3] INFO org.littleshoot.proxy.impl.ProxyToServerConnection - (HANDSHAKING) [id: 0x74048cdf, L:0.0.0.0/0.0.0.0:63811]: Connection to upstream server failed javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131) at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:307) at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:285) at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:180) at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164) at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:681) at java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:636) at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:454) at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:433) at java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:634) at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1332) at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1227) at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1274) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:682) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:617) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:534) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.base/java.lang.Thread.run(Thread.java:835)

Running the BrowserUp Proxy 1.2.1 from Gradle with Java8
Android 9.0

REST API for asserting on content of most recent URL matching a regular expression pattern

Using #14, implement content assertions methods for most recent URL that matches a given regular expression pattern accessible via REST.

assertUrlContentContains
assertUrlContentDoesNotContain
assertUrlResponseHeaderContains
assertUrlResponseHeaderDoesNotContain
assertUrlStatusEquals
assertUrlContentLengthWithin
assertUrlContentLengthWithin

See https://github.com/browserup/browserup-driver/tree/new-api#url-content-and-performance

Hi all! A couple of questions about your project

What about removing the Via header from requests? Original BMP does not know how to do it.
What about displaying CONNECT method in HAR? Original BMP does not know how to do it too.

REST API for asserting on timings of all URLs matching a regular expression pattern

Similar to #15, use #13 to implement performance assertions methods for all URLs that match a given regular expression pattern accessible via REST.

assertAllUrlsResponseTimeWithin - asserts on log.entries[].time

See https://github.com/browserup/browserup-driver/tree/new-api#url-content-and-performance

browserup / browserup-proxy Goto Github PK

browserup-proxy's People

Contributors

Stargazers

Watchers

Forkers

browserup-proxy's Issues

Recommend Projects

Recommend Topics

Recommend Org