Git Product home page Git Product logo

corona-warn-app / cwa-log-upload Goto Github PK

View Code? Open in Web Editor NEW
5.0 9.0 7.0 615 KB

The log upload service is the counterpart of the log upload in the app. It enables the App developers to analyze the log data uploaded to the CWA infrastructure to identify the root cause of bugs that only occur in rare conditions are not easy to reproduce.

License: Apache License 2.0

Dockerfile 0.24% Java 62.40% CSS 8.70% JavaScript 22.81% HTML 5.86%

cwa-log-upload's Introduction

Corona-Warn-App Log Upload

DevelopmentDocumentationSupportContributeContributorsRepositoriesLicensing

The goal of this project is to develop the official Corona-Warn-App for Germany based on the exposure notification API from Apple and Google. The apps (for both iOS and Android) use Bluetooth technology to exchange anonymous encrypted data with other mobile phones (on which the app is also installed) in the vicinity of an app user's phone. The data is stored locally on each user's device, preventing authorities or other parties from accessing or controlling the data. This repository contains the log upload for the Corona-Warn-App.

Status

ci-master quality gate coverage bugs

About this component

The log upload service is the counterpart of the log upload in the app. It enables the App developers to analyse the log data uploaded to the CWA infrastructure to identify the root cause of bugs that only occur in rare conditions are not easy to reproduce. The log upload and viewer service follows the privacy preserving paradigm of the corona warn app, only allowing authorized personnel to access the logs. The means to restrict access and control who can access the data. The logs are kept inside the application and are only accessible trough the portal. The access to the portal is restricted trough the IAM component with a separate role for this access.

Development

This component can be built locally in order to test the functionality of the interfaces and verify the concepts it is build upon. There are two ways to build:

  • Maven build - to run this component as spring application on your local machine
  • Docker build - to run it as docker container build from the provided docker build file

Prerequisites

Open JDK 11
Maven (optional): Docker

Build

Whether you cloned or downloaded the 'zipped' sources you will either find the sources in the chosen checkout-directory or get a zip file with the source code, which you can expand to a folder of your choice.

In either case open a terminal pointing to the directory you put the sources in. The local build process is described afterwards depending on the way you choose.

Maven based build

For actively take part on the development this is the way you should choose.
Please check, whether following prerequisites are fulfilled

is installed on your machine.
You can then open a terminal pointing to the root directory of the log upload and do the following:

mvn package
java -jar target/cwa-log-upload-0.0.1-SNAPSHOT.jar  

The log upload will start up and run locally on your machine available on port 8081. Please keep in mind, that you need another component [cwa-verification-iam] the get this running in a sensable manner.

Docker based build

We recommend that you first check the prerequisites to ensure that

is installed on your machine.

On the command line do the following:

docker build -f|--file <path to dockerfile>  -t <imagename>  <path-to-log-upload-root>
docker run -p 127.0.0.1:8085:8085/tcp -it <imagename>

or simply

docker build --pull --rm -f "Dockerfile" -t cwa-log-upload "."
docker run -p 127.0.0.1:8085:8085/tcp -it cwa-log-upload

if you are in the root of the checked out repository.
The docker image will then run on your local machine on port 8085 assuming you configured docker for shared network mode.

Code of Conduct

This project has adopted the Contributor Covenant in version 2.0 as our code of conduct. Please see the details in our CODE_OF_CONDUCT.md. All contributors must abide by the code of conduct.

Working Language

We are building this application for Germany. We want to be as open and transparent as possible, also to interested parties in the global developer community who do not speak German. Later on this application might also serve as a template for other projects outside of Germany. For these reasons, we decided to apply English as the primary project language.

Consequently, all content will be made available primarily in English. We also ask all interested people to use English as language to create issues, in their code (comments, documentation etc.) and when you send requests to us. The application itself, documentation and all end-user facing content will - of course - be made available in German (and probably other languages as well). We also try to make some developer documentation available in German, but please understand that focussing on the Lingua Franca of the global developer community makes the development of this application as efficient as possible.

Documentation

The full documentation for the Corona-Warn-App can be found in the cwa-documentation repository. The documentation repository contains technical documents, architecture information, and white papers related to this implementation.

Support and Feedback

The following channels are available for discussions, feedback, and support requests:

Type Channel
General Discussion
Concept Feedback
Log Upload Issue
Other Requests

How to Contribute

Contribution and feedback is encouraged and always welcome. For more information about how to contribute, the project structure, as well as additional contribution information, see our Contribution Guidelines. By participating in this project, you agree to abide by its Code of Conduct at all times.

Contributors

The German government has asked SAP AG and Deutsche Telekom AG to develop the Corona-Warn-App for Germany as open source software. Deutsche Telekom is providing the network and mobile technology and will operate and run the backend for the app in a safe, scalable and stable manner. SAP is responsible for the app development, its framework and the underlying platform. Therefore, development teams of SAP and Deutsche Telekom are contributing to this project. At the same time our commitment to open source means that we are enabling -in fact encouraging- all interested parties to contribute and become part of its developer community.

Repositories

A list of all public repositories from the Corona-Warn-App can be found here.

Licensing

Copyright (c) 2020-2023 Deutsche Telekom AG.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.

You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an " AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the LICENSE for the specific language governing permissions and limitations under the License.

cwa-log-upload's People

Contributors

ascheibal avatar daniel-eder avatar ein-tim avatar f11h avatar mschulte-tsi avatar roesslerj avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cwa-log-upload's Issues

implement TOTP based verification

For validating data upload, the origin has to be proved to be a real phone.
The phone will send a TOTP Token in the request header to indicate that.
The log upload service must verify this token against the cwa infrastructure for validity.

[BSI][20210511][v2.2] ZIP Processing Resource Exhaustion

Rating: Informational

The JavaScript that is used to process the ZIP archives is vulnerable to a resource exhaustion attack. More specifically,
it is possible to upload a specially crafted ZIP file which leads to an endless loop while decompressing the file.
As the file decompression is done in a separate worker, this does not affect the main process of the log upload browser
tab. Therefore, the implications of this issue are rather low. On the tester’s machine the endless loop resulted in a very
high CPU usage, however, as the main process of the tab is not affected, it can simply be closed to stop the loop.
Due to time restrictions, the cause of this issue has not been further investigated. The issue has been tested in a setup
isolated from the cwa-log-upload application to rule out any possible influence by the application itself. This test showed
that the issue, in fact, stems from the library.

Proof of Concept
The screenshot below shows that the file is struck while decompressing and the processor utilization of the worker
goes up to about 100%:
image

Below is a hexdump of the offending ZIP file. The file is a partially corrupted ZIP file which contains two files (a and P)
with a size of 34 and 19. It was created during randomly corrupting ZIP files, therefore the corruption does not follow a
specific format.

00000000 50 4b 03 04 14 00 08 00 08 00 00 00 00 00 00 00 |PK..............|
00000010 00 00 00 00 00 00 00 00 00 00 01 00 00 00 61 4a |..............aJ|
00000020 84 82 24 9c 00 10 00 00 ff ff 50 4b 07 08 6f b2 |..$.......PK..o.|
00000030 bf e3 0b 00 00 00 22 00 00 00 50 4b 03 04 14 00 |......"...PK....|
00000040 08 00 08 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000050 00 00 00 00 01 00 00 00 62 4a 4c 4a 4e 49 4d 4b |........bJLJNIMK|
00000060 cf 30 30 34 32 36 ff 7f 33 b7 b0 34 00 04 00 00 |.00426..3..4....|
00000070 ff ff 50 4b 07 08 da c9 f3 fc 19 00 00 00 13 00 |..PK............|
00000080 00 00 50 4b 01 02 14 00 14 00 08 00 08 00 00 00 |..PK............|
00000090 00 00 6f b2 bf e3 0b 4c 00 00 22 00 00 00 01 00 |..o....L..".....|
000000a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000b0 61 50 4b 01 02 14 00 14 00 08 00 08 00 00 00 00 |aPK.............|
000000c0 00 da c9 f3 fc 19 00 00 00 13 00 00 00 01 00 00 |................|
000000d0 00 00 00 00 00 00 00 00 00 00 00 3a 00 00 00 50 |...........:...P|
000000e0 4b 05 06 00 00 00 00 02 00 02 00 5e 00 00 00 82 |K..........^....|
000000f0 00 00 00 00 00 |.....|
000000f5

CWA log upload is broken

Describe the bug

Uploading logs from the app to the server fails with an error message.

Expected behaviour

Uploading an error log should be possible.

Steps to reproduce the issue

Differ between iOS & Android, should be clear.

Possible Fix

@MikeMcC399 found that https://logupload.coronawarn.app has a new SSL certificate issued today. Maybe this is the root cause?

Additional context

This issue was originally reported by @diablodale via corona-warn-app/cwa-app-android#5089.

Split features

The features Upload API and Log-Viewer-Portal need to be seperated by spring profiles.

AK:

  • Define 2 profiles
  • prifle "portal": Only Portal and Download API
  • profile "api": Only Upload API
  • Cleanup Job runs only in "api" profile
  • Add Shedlock to Cleanup Job

[BSI][20210511][v2.2] Log Injection

Rating: Medium

The cwa-log-upload server is vulnerable to a log injection attack at two individual endpoints. This enables an unauthenticated
(as well as an authenticated) attacker to violate the integrity of the cwa-log-upload server logs by injecting
arbitrary log messages. Note that this does not affected the user-supplied log file ZIP archives, but the actual server
logging output.
The injected log entries are not easily distinguishable to legitimate entries that have been appended by the cwa-logupload
server itself.
This issues could be abused for social engineering attacks on administrative personnel by injecting malicious messages
into log files. These could include false error conditions that instruct administrators to, for example, contact someone
or interact with services or systems by restarting them.
In this specific case, the endpoints /api/logs and /portal/search (which requires an authenticated portal user) are
affected.

Proof of Concept:

The following screenshot shows a successfully injected log entry via the /api/logs endpoint:
image

Request:

POST /api/logs HTTP/1.1
Host: localhost:8085
User-Agent: curl/7.64.1
Accept: */*
Content-Type: multipart/form-data; boundary=11D989FA-25A8-4082-A4D2-703A4A27D392
Cwa-Otp: 357CC67D-3D97-4E4F-8F5B-E4730B71B4BD
Content-Length: 881
Connection: close
--11D989FA-25A8-4082-A4D2-703A4A27D392
Content-Disposition:form-data; name="file"; filename="filename.txt
[2m2021-05-07 17:10:17.277 [0;39m [32m INFO [cwa-log-upload,,] [0;39m [35m50287 [0;39m [2m--- [0;39m
,→ [2m[ main] [0;39m [36ma.c.logupload.LogUploadApplication [0;39m [2m: [0;39m Injected Log Line
Content-Type:application/zip
Content-Length: 458
PK [...]
--11D989FA-25A8-4082-A4D2-703A4A27D392--

Response:

HTTP/1.1 201
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
Content-Type: application/json
Date: Fri, 07 May 2021 15:14:58 GMT
Connection: close
Content-Length: 63
{"id":"C0B1DBCE400C8610C2FE","hash":"PthK38bUN7JLVgyyk8PmpQ=="}

The following screenshot shows a successfully injected log entry via the /portal/search endpoint, which requires an
authenticated user.
image

It was created using the following request:

Request

POST /portal/search HTTP/1.1
Host: localhost:8085
Accept-Encoding: gzip, deflate
Connection: close
Cache-Control: max-age=0
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="90", "Google Chrome";v="90"
sec-ch-ua-mobile: ?0
Upgrade-Insecure-Requests: 1
Origin: http://localhost:8085
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko)
,→ Chrome/90.0.4430.93 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q
,→ =0.8,application/signed-exchange;v=b3;q=0.9
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Referer: http://localhost:8085/portal/start?
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8,de;q=0.7
Cookie: JSESSIONID=0DD040DF4AD43C531993DD103AE948BE
Content-Length: 357
logId=%0a%1B%5B2m2021-05-07+17%3A10%3A17.277%1B%5B0%3B39m+%1B%5B32m+INFO+%5Bcwa-log-upload%2C%2C%5D%1B
,→ %5B0%3B39m+%1B%5B35m50287%1B%5B0%3B39m+%1B%5B2m---%1B%5B0%3B39m+%1B%5B2m%5B+++++++++++main%5D
,→ %1B%5B0%3B39m+%1B%5B36ma.c.logupload.LogUploadApplication++++++%1B%5B0%3B39m+%1B%5B2m%3A%1B%5
,→ B0%3B39m+Injected+Log+Line&_csrf=6a896d20-cdb6-42b5-bd2f-f94e49752a67

Response:

HTTP/1.1 200
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
Content-Type: text/html;charset=UTF-8
Content-Language: en-GB
Date: Fri, 07 May 2021 15:25:38 GMT
Connection: close
Content-Length: 1458
<!DOCTYPE html>
<html lang="de">
<head>
<meta charset="utf-8"/>
<meta content="default-src 'self' style-src 'unsafe-inline'" http-equiv="Content-Security-Policy">
<meta content="width=device-width, initial-scale=1.0" name="viewport"/>
<link href="/portal/static/css/cwa.css" rel="stylesheet" type="text/css"/>
<title>Corona Warn App - Log Upload</title>
<link href="/portal/static/img/c-19_logo.png" rel="icon"/>
<script src="/portal/static/js/logout.js"></script>
</head>
<body>
[...]

Document 'Overview' is checked-in duplicated

Describe the bug

The document 'Overview.md' is available at two different positions in the repo:

  1. at doc/Overview.md
  2. at src/doc/Overview.md

Note: This needs to be fixed asap.
The two versions are already out of synch.

Expected behaviour

Steps to reproduce the issue

Technical details

  • Host Machine OS (Windows/Linux/Mac):

Possible Fix

Merge the two versions, and then delete one.

Additional context

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.