Comments (6)
Magic, but the failure does not occur anymore. I think we can close this issue for now.
from s3mock.
@tashoyan
which test did you run in a loop?
How did you determine that S3Mock suddenly respond with HTTP to an HTTPS request? We are relying on Docker, Jetty, Java and Spring for communication, S3Mock does not implement HTTPS communication.
Maybe you just overloaded the S3Mock, it may have run out of memory, it may be a problem with your Docker runtime or TestContainers...
Are the errors evenly distributed or are all errors occurring towards the end of the run?
We are using a self-signed certificate, which does not pass curl
s SSL certificate inspection.
$ curl http://localhost:9090/
<ListAllMyBucketsResult><Owner><ID>79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be</ID><DisplayName>s3-mock-file-store</DisplayName></Owner><Buckets/></ListAllMyBucketsResult>
$ curl https://localhost:9191/
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
$ curl --insecure https://localhost:9191/
<ListAllMyBucketsResult><Owner><ID>79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be</ID><DisplayName>s3-mock-file-store</DisplayName></Owner><Buckets/></ListAllMyBucketsResult>
Also: when using the S3Mock locally, it's unnecessary to use an SSL connection.
from s3mock.
My test does the following:
- create a bucket
- list all buckets
- put a file in the bucket
- list objects in the bucket
An instance of S3Mock is created before each test and shut down after each test.
A loop is implemented as a shell script which repeatedly runs a Maven command to execute the test. For sure it is not a load test.
Nevertheless I get failures in approximately 30% of runs.
I checked the failure with the debugger. Indeed the failure occurs in SSLSocketInputRecord.decode(), where either HADSHAKE or ALERT is expected, but a string "HTTP/" appears in the header instead:
// header is actually "HTTP/"
if ((header[0] != ContentType.HANDSHAKE.id) &&
(header[0] != ContentType.ALERT.id)) {
plaintext = handleUnknownRecord();
}
I also checked the response from the S3Mock endpoint with curl command, as described above. Indeed, in case of the failure it responses with plain HTTP rather than HTTPS - see the examples above.
from s3mock.
Regarding your remark:
Also: when using the S3Mock locally, it's unnecessary to use an SSL connection.
We use S3Mock for integration tests on some application, that expects HTTPS endpoint. That's why we need S3Mock with HTTPS.
from s3mock.
And this is how S3Mock endpoint responds in case of success:
# Try HTTPS
$> curl --insecure https://localhost:49650
<ListAllMyBucketsResult><Owner><ID>79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be</ID><DisplayName>s3-mock-file-store</DisplayName></Owner><Buckets><Bucket><Name>b1</Name><CreationDate>2023-10-04T19:16:28.391Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>
# Try HTTP
$> curl http://localhost:49650
curl: (1) Received HTTP/0.9 when not allowed
from s3mock.
# Try HTTP
$> curl http://localhost:49650
curl: (1) Received HTTP/0.9 when not allowed
This is expected - you tried to connect to an HTTPS port using HTTP. This would not work in any application...
Not sure what you want me to do here...
Are there logs from one of the failed runs? Any errors that are actually in the S3Mock code?
I added a test that does the same thing as your test, ran it 1000 times.
Locally, all is fine.
On the PRB on Github, all is fine.
https://github.com/adobe/S3Mock/actions/runs/6418197863/job/17425509251?pr=1398
#1398
Try checking out the branch and run a mvn build (this will take hours).
from s3mock.
Related Issues (20)
- Copying object with new storage class does not appear to save storage class HOT 5
- initialBuckets is ignored if at least one bucket exists with a bucketMetadata.json
- Service: Amazon S3; Status Code: 404; Error Code: 404 HOT 2
- org.eclipse.jetty.server.Request : java.lang.IllegalStateException: Form is larger than max length 200000 HOT 1
- while reading large number of record from s3mock getting Exception( java.io.IOException: Attempted read on closed stream) HOT 2
- S3Mock is incompatible with Spring Boot 3.2.x and Spring Framework 6.x HOT 3
- createBucket fails using S3AsyncClient with NoSuchBucketException HOT 1
- Support S3AsyncClient from software.amazon.awssdk:s3 HOT 2
- Cannot put object acl without a policy HOT 5
- When upgrading to Spring 3.2.2: ClassNotFoundException: javax.xml.bind.annotation.XmlElement HOT 1
- When using async uploads, S3Mock incorrectly stores unsigned, chunked uploads without removing meta (chunk, checksum) information. HOT 14
- Some key names lead to request paths that do not comply to RFC3986, leading to failing requests in S3Mock HOT 7
- When starting S3Mock in Docker, root folder is not accessible from outside of the container HOT 4
- Synchronisation issues HOT 5
- missing namespace xml header. HOT 4
- When using async uploads, S3Mock incorrectly stores unsigned, chunked uploads without removing meta (chunk, checksum) information when the file is >16KB HOT 2
- Validate checksums for uploads and parts
- NumberFormatException when uploading files HOT 5
- Content-MD5 does not match object md5 HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from s3mock.