"# streaming-anonymous-algorithm"

This a dummy platform to implement a streaming anonymous algorithm to protect InputRecord.

I felt like I need to do some research into knowledge and techniques with the General Data Protection Regulation (GDPR). I want to find a easy way that could be made more effective, especially when we want to use those data in machine learning and artificial intelligence.

1. Application Structure

   

   

2. K-anonymisation Algorithm

   1.1 sorted and fix-size aggregation algorithm

   • sort records by rawValue.
   • split all records into diff group, make sure that there are K records in each group
   • If the number of records is not a multiple of K, those records aggregate into the last group.

   

3. Stream Windows Strategy

   In this version, window's size fixed in the config file.

   Next step, dynamic calculation of the size of the window

   2.1 Kafka Stream Windows

   2.2 Stream Windows in Apache Flink

1. calculate the mean of the InputRecord's rawValue
2. calculate Mean Variances as A
3. calculate Anonymised Mean Variances as B
4. Distortion = B / A

Distortion Algorithm Java File

Analysis tool Java File

1. What is [K-anonymity]

1. JDK 1.8 or later
2. Maven 3.2+

If you are using Maven, execute:

 mvn package spring-boot:run

Then Open Test Page

 http://localhost:8080