Comments (8)
Version Info
package:cronet
Version: 0.0.1+1
Dart SDK: 2.12.0
Cronet Dynamic Library Version: 86.0.4240.198 Debug
Report 1
Environment
Server: Flask 2.0.1 with Waitress 2.0.0 on Ubuntu 20.04 LTS
Hosted: Local Server
HTTP Protocol: HTTP/1.1
TLS: No
Payload: Lorem Ipsum plain text appended with the url query parameter sent from the client. For each query, different URL query parameter is used.
Performance (JIT Mode)
Cronet: 1807.8428 us
dart:io: 1801.662115336 us
So, we can see that package:cronet
and dart:io
performs on par with each other in JIT Mode.
Note: These are average values derived by running the benchmarking script 5 times.
Performance (AOT Mode)
Cronet: 1441.286689427 us
dart:io: 1049.144274589 us
So, we can see that dart:io
takes a slight edge over package:cronet
in AOT Mode.
Note: These are average values derived by running the benchmarking script 5 times.
from cronet.dart.
Report 4 (Throughput)
Environment
Server: Caddy v2.4.3 (golang version 1.15.13
) on Ubuntu 20.04 LTS
Hosted: Local Server
HTTP Protocol: HTTP/2 with Experimental HTTP3 support
TLS: Yes
Compression: gzip
Payload: example.org's index.html
Target Time Duration: 1 second.
Performance (JIT)
Cronet: Total Spawned: 4096, In Time Returns: 2982
. This is the best result. After that, no. of successful request dipped slowly.
dart:io: Total Spawned: 512, In Time Returns: 512.
This is the best result. After that, numbers dipped to 1 successful request
. So, 512 concurrent requests are the limit for dart:io
in this case.
Performance (AOT)
Cronet: Total Spawned: 4096, In Time Returns: 2883.
This is the best result. After that, no. of successful request dipped slowly.
dart:io: Total Spawned: 512, In Time Returns: 512.
This is the best result. After that, numbers dipped to 1 successful request
. So, 512 concurrent requests are the limit for dart:io
in this case.
So, in terms of concurrent requests, package:cronet
surpassed dart:io
in both the modes with a large margin (5.7x
more request approx.)
Report 5 (Throughput)
Environment
We fetched data from example.com.
Target Time Duration: 1 second.
Performance (JIT)
Cronet: Total Spawned: 512, In Time Returns: 178
. This is the best result. After that, no. of successful request dipped slowly.
dart:io: Total Spawned: 128, In Time Returns: 39
. This is the best result. Then it dipped rapidly and eventually hit 0 as we reach 512
concurrent requests.
Performance (AOT)
Cronet: Total Spawned: 512, In Time Returns: 214.
. This is the best result. After that, no. of successful request dipped slowly.
dart:io: Total Spawned: 128, In Time Returns: 49.
. This is the best result. Then it dipped rapidly and eventually hit 0 as we reach 512
concurrent requests.
So, by pinging a remote server we observe that, package:cronet
served more than 4x
more requests compared to dart:io
within a second.
from cronet.dart.
We may reduce benchmarking boilerplate by using package:benchmark_harness.
Edit: Can't use this package as async
support isn't there yet. dart-lang/benchmark_harness#44
Should we bump up the cronet version to the latest for a better QUIC support?
I was playing around with 2 different versions of chromium (v83 & v90) and I've noticed that v83 rarely uses QUIC even on the enabled sites. v90 does a far better job (though, in some cases it used QUIC after multiple visits to a site). Currently v86 is being used.
from cronet.dart.
Report 2
I'm surprised (and confused) by the outcome.
Environment
Server: Caddy v2.4.3 (golang version 1.15.13
) on Ubuntu 20.04 LTS
Hosted: Local Server
HTTP Protocol: HTTP/2 with Experimental HTTP3 support
TLS: Yes
Compression: gzip
Payload: Visual Studio CE iso file. Size: 7.1GB
Performance (JIT Mode)
Cronet: 202.690879 sec
dart:io: 167.378507 sec
dart:io
got the lead here. (A little surprised)
Performance (AOT Mode)
Cronet: 270.522353 sec
dart:io: 155.367736 sec
dart:io
got the lead here too.
I tested this again. And, almost similar result came up. Shocked. Not because of dart:io
. But, package:cronet
's performance decreased in AOT mode? Weird. The only way I can reduce the shock is, we're using a debug build. Or, is my machine acting funny?
from cronet.dart.
Well, time to dig into what is going on.
- Is it slow because of computations or because of waiting on I/O or async. (E.g. is the CPU fully in use or not.)
- If the CPU is fully in use, we can use tools such as Linux
perf
to see what is being used.
Sometimes in JIT we can do speculative optimizations that the AOT cannot do. But 202 -> 270 seconds is surprising.
Another hypothesis is that we copy data multiple times in the package:cronet implementation but not in the dart:io implementation.
All the surprise means we're going to learn something! 🦀
from cronet.dart.
Is it slow because of computations or because of waiting on I/O or async. (E.g. is the CPU fully in use or not.)
No. CPU isn't getting fully utilized. I tested again. For Cronet AOT, time got reduced to 258 sec
but other observations stayed the same.
Now, there's something interesting going on when I observed resource consumption.
For Cronet, in JIT mode, the CPU utilization is 32%
while in AOT, it is 21%
. Memory consumption remained the same, around 20MB
.
For dart:io, it is even more interesting. In both the modes, CPU utilization remained 16%
. However, for memory, in JIT mode it is 30MB
. While, in AOT it is 61MB
of utilization.
from cronet.dart.
Sometimes in JIT we can do speculative optimizations that the AOT cannot do.
I want to hear more about this. Tell me about this when you'll have less workload.
Another hypothesis is that we copy data multiple times in the package:cronet implementation but not in the dart:io implementation.
Can be a reason.
All the surprise means we're going to learn something! 🦀
Excited 😄
from cronet.dart.
Report 3
Here we go again ᕕ( ᐛ )ᕗ with another report.
Environment
I tried to download Google Chrome Deb Package. Size: 79MB (approx).
Performance (JIT)
Cronet: 51.5 sec
dart:io: 47.02 sec
Performance (AOT)
Cronet: 47.003 sec
(This time, cronet acted a bit "rational")
dart:io: 47.75 sec
This time, cronet
actually crossed dart:io
in AOT mode by a negligible margin. Though, this time, dart:io
performed very slightly worse? But, the difference maybe due to the uncertainties come along with networking. Nonetheless, we can see that, in practical workload, they seem to perform on par in both the modes in this benchmarking round.
from cronet.dart.
Related Issues (20)
- Flutter support HOT 1
- Cache and Cookie Store
- error: No net_fetcher for performing AIA chasing. HOT 1
- Sudden hiccups during benchmarking HOT 1
- Use package:args
- fix: throughput benchmark
- Support sending files through POST request HOT 1
- 32 Bit system support
- Adding user specified headers in http request
- Add separate set methods to set common http headers in http requests
- minimising Flutter build size
- Xx
- QUIC not working? HOT 1
- When I use 32-bit dynamic library, there is an error HOT 4
- Investigate using the Cronet shipped in Android
- Force Close Release APK HOT 1
- Unknown CMake command "add_compile_definitions"
- Webtransport support
- Security Policy violation Binary Artifacts HOT 146
- flutter pub run cronet:setup failed with error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cronet.dart.