1. Support Vector Machine
   https://en.wikipedia.org/wiki/Decision_tree

• [X ] DONE

3. Decision Tree Classifier -

• [X ] DONE

https://en.wikipedia.org/wiki/Decision_tree

Some of our key classes still need further JUnit tests

Description

Implement the classical method of matrix multiplication. This method will use the straightforward algorithm of computing the dot product between rows of the first matrix and columns of the second matrix.

Task Details

• Objective: Implement a function for the classical matrix multiplication of two 2D arrays of doubles.
• Function Signature: double[][] multiplyMatricesClassical(double[][] matrixA, double[][] matrixB)

Acceptance Criteria

• Function correctly multiplies two matrices of compatible dimensions.
• Appropriate error handling for matrices of incompatible sizes.
• Comprehensive unit tests that cover various scenarios including edge cases.

We need to create a comprehensive GitHub Wiki page that provides clear instructions on how to import Edux into an Integrated Development Environment (IDE). This documentation will help our users and contributors get started with Edux development more efficiently.

Expected Behavior:

A well-structured GitHub Wiki page should be created under our project's Wiki tab with detailed instructions on how to import Edux into popular IDEs.

Add some Images

IDE to cover

• Eclipse
• VSCode
• IntelliJ IDEA

Summary: Create a RandomAugmentation class, that apply randomly one of existing Augmentations.

If a choosen Effect expect constructor parameter, these need to be randomized too.

We need to change the parameter in the getColumnDataOf method within DataProcessor. Specifically, I suggest altering the parameter from String columnName to int columnIndex due to identified design shortcomings.
Additionally, to maintain code coherence, it's needed to remove all methods associated with the field columnNames.

class location : lib/src/main/java/de/edux/data/provider/DataProcessor.java

Create a Social Preview Card

We need a social media preview image for our repository.

• .png File
• Images should be at least 640×320px (1280×640px for best display).

Template

Summary:

Develop a class to perform multiplication of 2D matrices using virtual threads to improve performance.

Description:

With the increasing need to process large datasets and matrices, leveraging concurrency mechanisms like virtual threads can lead to significant performance improvements. Implementing matrix multiplication using virtual threads can not only enhance performance but also serve as a blueprint for other matrix operations in the future.

Requirements:

1. Create an interface named ConcurrentMatrixMultiplication - double[][] multiplyMatrices(double[][] a, double[][] b)
2. Create a class MathMatrix that implement ConcurrentMatrixMultiplication
3. Utilize virtual threads for matrix multiplication.
4. Ensure proper thread synchronization to avoid race conditions.
5. Add proper error handling, especially for matrices that can't be multiplied due to dimension mismatches.

Issue Description:

We currently have examples in our library that utilize the IRIS dataset. To diversify and showcase the capabilities of our ML Java Library, it would be great to introduce an example using the Seaborn Penguin dataset.

Additional Context

The Seaborn Penguin dataset is quite popular and provides a good balance of categorical and numerical data. It would be a good addition to our example suite and potentially attract more users familiar with this dataset.

• Add the dataset csv to the project
• Write a DatasetProvider
• Implement an example. Use a multilayer neural network.

Dataset: https://github.com/mwaskom/seaborn-data/blob/master/penguins.csv

Summary: Develop advanced geometric transformations for EDUX's image augmentation toolset.

Details:

• Perspective Transformations: Implement methods for perspective changes.
• Elastic Transformations: Add elastic deformation functions.

Java Features: Advanced use of AffineTransform and custom algorithms.

Documentation: Comprehensive Javadoc and practical examples in README.

Tests: Extensive unit and integration tests for transformation accuracy.

public class CudaMatrixArithmetic implements IMatrixArithmetic

• convert to Singleton
• edit all tests

Implement class MatrixVectorProduct

Issue Description:

We currently have examples in our library that utilize the IRIS dataset. To diversify and showcase the capabilities of our ML Java Library, it would be great to introduce an example using the Seaborn Penguin dataset.

Additional Context

The Seaborn Penguin dataset is quite popular and provides a good balance of categorical and numerical data. It would be a good addition to our example suite and potentially attract more users familiar with this dataset.

• Add the dataset csv to the project
• Write a DatasetProvider
• Implement an example. Use k-nearest neighbors algorithm.

Dataset: https://github.com/mwaskom/seaborn-data/blob/master/penguins.csv
See #16

We need a classic multithreaded Math Class for Matrix multiplication. We will compare it perfomance with #12

We will test ist with matrix sizes of >4000x4000

A classificator interface is to be built that is used by all ML algorithms.

Goal: A unified intuitive API.

java.lang.IllegalStateException: Module entity with name: edux.lib.main should be available

• adapt - rootProject.name = 'Edux' to 'edux'

Description:

As our library grows and tackles more complex ML tasks, the need for leveraging the power of GPU becomes vital to ensure optimal performance and resource utilization. The goal of this ticket is to discuss and track the implementation of Nvidia CUDA support, enabling the library to conduct parallel computations on Nvidia GPUs, thus drastically enhancing data processing and training times for ML models.

Performance:

Implementing CUDA support should exhibit a noticeable performance improvement in model training and predictions, especially for computationally intensive tasks.

Tasks

• Implement Matrix Multiplications Feature that runs on GPU

Testing:

Adequate testing frameworks need to be in place to ensure stability and performance of CUDA implementation across different GPUs and operating systems.

Targeting CUDA/cudnn Version:

• CUDA Toolkit 11.8
• cuDNN 8.9.6

Summary: Develop scaling and cropping capabilities for the EDUX image augmentation library.

Details:

• Scaling: Methods to scale images with options for smooth scaling.
• Cropping: Functions to crop to a region or to a specific size.

Java Features: Utilize java.awt.Image for quality scaling.

Documentation: Update Javadoc and README with usage samples.

Tests: Create unit tests for boundary conditions and typical use cases.

do it

Update Project to Gradle 8.3

Description

Gradient Boosting is a prominent ensemble learning method used widely in regression, classification, and ranking tasks. Given its importance and frequent demand in various ML applications, there's a need to incorporate a Gradient Boosting implementation in our Java ML Library. This ticket aims at the development and optimization of the Gradient Boosting algorithm in pure Java.

Acceptance Criteria:

• The algorithm should be implemented in pure Java without relying on external dependencies.
• The implementation should support both regression and classification tasks.
• Incorporate decision trees as weak learners, ensuring the option to define the depth of the tree.
• A provision should be there to define the number of boosting rounds.
• Allow tunable parameters such as learning rate, loss function, and regularization.
• The solution needs to be performant and scalable, capable of handling larger datasets efficiently.
• Boundary and edge cases, such as data inconsistencies or parameter mismatches, should be handled gracefully with appropriate exceptions and error messages.
• Unit tests should be written to verify the functionality, performance, and reliability of the implemented algorithm.
• Integration examples or demos illustrating how to use the implemented class in real-world scenarios.

We need to go with JDK21.

It looks like we need Virtual Threads with Executor Service, so we need to update the project to 21.

• Update JDK Version in Gradle File to 21

Description

We've identified a bug in our Decision Tree implementation where the currentLeafNodes counter, which is supposed to track the number of leaf nodes, does not update during the tree construction. This issue leads to an inability to correctly enforce the maxLeafNodes constraint, potentially allowing the tree to grow beyond the intended limits.

Expected Behavior

The currentLeafNodes variable should increment whenever a leaf node is created, ensuring that the total number of leaf nodes never exceeds maxLeafNodes. The tree construction should halt when the maximum number of leaf nodes is reached.

Actual Behavior

The currentLeafNodes remains at its initial value (0) throughout the execution, causing the maxLeafNodes constraint to be ignored. This behavior can lead to overfitting if the tree grows too complex, as it doesn't stop at the designed constraint.

Steps to Reproduce

1. Initialize a Decision Tree with a specific maxLeafNodes value.
2. Train the Decision Tree with a dataset large enough to potentially exceed the maxLeafNodes if unconstrained.
3. Observe the size of the constructed tree, noting that it can exceed the intended maximum leaf nodes.

Suggested Fix

We need to adjust the tree construction logic to correctly increment currentLeafNodes each time we create a leaf node. Additionally, before creating new branches, we should check whether adding them would exceed maxLeafNodes and, if so, stop branching further. These changes should enforce the correct tree size and complexity as originally intended by the maxLeafNodes parameter.

Additional Information

• Problematic behavior identified in the buildTree method of the DecisionTree class.
• This issue might be contributing to overfitting in our model predictions due to the increased complexity of the tree.

Please, any assistance or further insights into how we can resolve this issue would be greatly appreciated.

There is an open discussion for this ticket. #62
In our data processing logic, we currently use a boolean named filterIncompleteRecords to decide whether to filter out incomplete records or not. To provide more flexibility and future extensibility in handling missing data, we propose replacing this boolean with an enum named Imputation.

Let see on this example tiger dataset.

With filterIncompleteRecords = true, our Data Processing Class will auto remove Amur Tiger from the dataset, because the size value is missed.

The new Imputation enum would have various strategies such as MEDIAN, MODE, FILTEROUT, and AVERAGE. For the scope of this ticket, we'll focus on implementing FILTEROUT and AVERAGE, MODE.

Advantages of this new Feature

A user can auto fill missed values.

Hint

How filterIncompleteRecords works you can see in the Seaborn Pinguin k-nearest neighbors algorithm (knn) example.
https://github.com/Samyssmile/edux/blob/main/example/src/main/java/de/example/knn/KnnSeabornExample.java

https://journalofbigdata.springeropen.com/articles/10.1186/s40537-021-00516-9

In our example subfolder, we need a Class that show how to use RandomForest

Write examples for all of our ML Algorithms.

• Use Seaborn Dataset
• Use the new Imputation Feature

This Ticket is blocked by: #109
This Ticket will profit from: #36 and #44

CNNs are state of the art approach for Image & Audio recognition.

As machine learning and artificial intelligence evolve, the ability of EDUX to solve complex problems, especially those related to image recognition and processing, is vital. Currently, the library lacks a fundamental functionality in the domain of deep learning, which is the Convolutional Neural Network (CNN). The implementation of CNNs would dramatically broaden the scope and capability of EDUX, making it feasible to address issues in image processing, video analysis, and various computer vision applications.

CNN Development Roadmap

1. Designing the CNN Architecture

• Define the CNN architecture.
  • Determine the number and types of layers:
    • Convolutional layers
    • Dense layers
    • MaxPooling layers
    • Flatten layers
    • Dropout layers
  • Decide on the activation function (e.g., ReLU for hidden layers, Softmax for output).

2. Implementing the Core Components

• Develop or integrate the Matrix3D class for matrix operations.
• Implement the Convolutional layer operations in Matrix.
• Implement MaxPooling, Flatten, and Dropout layers using Matrix.
• Implement Dense layers with matrix support.
• Integrate Softmax functionality for output layer.

3. Loss Function and Optimization

• Implement the CrossEntropyLoss class.
• Integrate gradient-based learning mechanisms.
• Implement the ADAM optimizer.

4. Building the Training Infrastructure

• Create data preprocessing utilities.
• Develop a data loader for feeding the network.
• Implement a training loop.
  • Batch processing
  • Forward pass
  • Loss computation
  • Backward pass (gradient calculation)
  • Parameters update (using ADAM)
  • Implement CPU based Multithreading
  • Make Initial Weight Algorithm chnagable
  • Build Augmentation Tools

5. Testing and Evaluation

• Develop a testing routine to evaluate the model.
• Implement performance metrics (e.g., accuracy, loss).

6. Debugging and Optimization

• Perform code reviews and debugging.
• Optimize performance (focus on matrix operations in Matrix3D).

7. Documentation and Reporting

• Document the code and architecture.
• Create a report on model performance and learn

Description

Optimize the classical matrix multiplication method using multi-threading. The implementation should parallelize the computation to utilize system resources more efficiently.

Task Details

• Objective: Develop a multi-threaded approach for matrix multiplication.
• Function Signature: double[][] multiplyMatricesMultiThreaded(double[][] matrixA, double[][] matrixB)

Acceptance Criteria

• Multi-threading is used to parallelize the multiplication process.
• The function ensures thread safety when accessing and modifying shared resources.
• Unit tests validate the correctness of the multiplication across multiple threads.

Goal

We currently lack an implementation for the Random Forest algorithm. This algorithm is a fundamental component of modern machine learning techniques and has been in demand by many users. The objective of this ticket is to implement the Random Forest algorithm in Java for our ML Library.

https://en.wikipedia.org/wiki/Random_forest

Acceptance Criteria:

• - The Random Forest algorithm must be implemented in pure Java without any external dependencies.
• - The algorithm should support both classification and regression functionalities.
• - The algorithm must be compatible with our existing data structures.
• - Unit tests must ensure the correct operation of the algorithm.
• - The implementation needs to be performant and scalable, capable of handling larger datasets effectively.

This is blocked by: #43

Add Image Pre-processing Tools for Augmentation and Resizing

Description

In the context of developing and optimizing Convolutional Neural Networks (CNNs) #43 it is essential to implement a suite of image pre-processing tools that can facilitate streamlined and effective model training. Specifically, we require utility classes/methods that address common tasks like image augmentation and resizing.

Image Augmentation enriches the dataset, allowing the model to be trained on more diverse data without collecting new images. Consequently, it helps improve the model's robustness and ability to generalize, thus reducing overfitting.

Requirements:

1. Image Augmentation: Implement tools or utilities that support various image augmentation techniques, such as:
   • Rotation
   • Flipping
   • Shearing
   • Zooming
   • Color variations (brightness, saturation, etc.)
2. Image Resizing: Provide functionality for resizing images, which should support:
   • Specifying target size
   • Maintaining the aspect ratio or optionally ignoring it
   • Different interpolation methods (e.g., nearest-neighbor, bilinear, etc.)

Acceptance Criteria:

• Implement augmentation tools that provide at least the above-mentioned functionalities.
• Implement resizing tools with the specified features.
• Ensure that the tools are adaptable and can be easily integrated into the existing workflow with CNNs.
• Provide documentation for each tool/utility, detailing the usage and potential application scenarios.
• Create unit tests to verify the functionality and reliability of the implemented tools.

Additional Context:

Ensuring robust and versatile image pre-processing tools can significantly enhance the model training phase, enabling the CNN to generalize better through exposure to varied and transformed data. This enhancement will bolster our ongoing work in developing CNN models for our project.

Tasks:

• Define the API for image augmentation and resizing tools.

• Develop the image augmentation tools.

• Develop the image resizing tools.

• Create documentation and usage examples.

• Implement unit tests.

• Conduct a review and testing phase to ensure functionality and integration capability.

• https://albumentations.ai/docs/introduction/image_augmentation/#:~:text=Image%20augmentation%20is%20a%20process,slightly%20change%20the%20original%20image.

As @acsolle66 noticed, its a mistake to do NORMALIZE in such an early stage.

var dataset = seabornDataProcessor.loadDataSetFromCSV(csvFile, ',', SHUFFLE, NORMALIZE);

because it will have to much impact with potential outliners on the data. Instead we want to use a separate method for it.

    var dataset = seabornDataProcessor.loadDataSetFromCSV(csvFile, ',', SHUFFLE);
    dataset = seabornDataProcessor
            .imputation("Subspecies", Imputation.MODE)
            .imputation("1", Imputation.MEAN)
            .imputation("2", Imputation.MEAN)
            .imputation("3", Imputation.AVERAGE)
            .normalize();

Feature Request: Integration of JMH for Benchmarking Performance

Summary

EDUX, as a growing Java Machine Learning Library, has reached a stage where performance optimization can yield significant benefits. To ensure our library remains efficient and scalable, integrating Java Microbenchmark Harness (JMH) for performance and memory benchmarking is proposed.

Expected Outcome

The integration of JMH will allow contributors to:

• Identify performance bottlenecks.
• Ensure code changes do not degrade performance.
• Profile memory usage alongside execution time.
• Establish a benchmark suite for ongoing performance evaluation.

Action Items

• Set up JMH within the EDUX project. (Library and Examples subprojects)
• Document how to run benchmarks.

Additional Context

The ability to measure performance and memory utilization is critical as we optimize our algorithms for both speed and efficiency. This addition will be instrumental in maintaining EDUX's competitive edge as a user-friendly and performant Java ML library.

Summary: Implement rotation and translation functions in EDUX's image augmentation module.

Details:

• Rotation: Add a method to rotate images by specific angles using AffineTransform in Java.
• Translation: Implement translation along x and y axes leveraging BufferedImage.

Java Features: Use java.awt.geom for geometric transformations.

Documentation: Include Javadoc and README examples.

Tests: Unit tests for each transformation.

Description:

To bolster the foundational components of our Java ML Library, there's a need to introduce classes that cater to mathematical operations commonly used in machine learning. Specifically, this ticket aims at the development of classes for matrices, vectors, and scalar multiplications in Java. These classes should allow users to perform basic and advanced matrix-vector operations seamlessly.

Acceptance Criteria:

• Develop a Matrix class that supports basic operations such as addition, subtraction, and matrix multiplication.
• Implement a Vector class that supports operations like dot product, vector addition, and vector subtraction.
• Both the Matrix and Vector classes should support scalar multiplication.
• All classes must be written in pure Java without relying on external dependencies.
• The implementation should ensure that operations are efficient, even for larger matrices and vectors.
• Boundary cases, such as matrix-vector dimension mismatches, should be handled gracefully with appropriate error messages.
• Comprehensive documentation of the code, methods, and the API must be provided.
• Unit tests should be written to verify the accuracy and reliability of these operations.

Perspective

Later on we will refactor existing code and use this Math Classes to perform better.

Description:

In our codebase, there are numerous instances where one-line comments are used. While comments can be helpful, excessive usage can reduce the readability of the code. This ticket proposes the removal of these one-line comments and, where appropriate, extracting code blocks into self-explanatory methods.

Extract Code into Self-Explaining Methods:

For code blocks that are accompanied by comments to explain their logic, consider extracting them into separate methods.
Ensure that the new methods have descriptive names, effectively removing the need for the original comment.
For example, if there's a comment like // Check if user is valid, you can create a method named isValidUser().

Summary: Integrate noise injection and filtering techniques into EDUX image augmentation.

Details:

• Noise Injection: Add random noise to simulate real-world imperfections.
• Filtering: Implement Gaussian blur, sharpen, and edge detection filters.

Java Features: Consider java.awt.image.ConvolveOp for filtering.

Documentation: Document methods and include usage in the README.

Tests: Write tests for expected output of noise and filter applications.

Description

This ticket outlines the creation of a Antora documentation site for the EDUX. The goal is to provide users with a comprehensive guide on integrating the library into their Java projects and utilizing its machine learning algorithms.

Objectives

• Set up a basic Antora site structure.
• Document the process of including the EDUX library in a Java project.
• Provide code examples for each supported ML model.

Additional Notes

• Include a section for hardware acceleration requirements.
• Provide a benchmark section linking to the discussion on algorithm comparison.

The code examples can be found in the example directory of the EDUX repository.

• - Update all documentation
• - Update GitHub Page
• - Add Junit tests - Ticket
• - Reformat CodeBase with Google Formatter
• - All IRIS examples are there and using new API
• - Seaborn Dataset examples using the new Imputation Feature Solve this

I trained several models using your examples, and they all perform well. However, your Readme was not updated, which caused some initial confusion for me.

Feature Request: OpenCL-based Matrix Multiplication

Summary

We are looking to expand the computational backend options of EDUX by integrating OpenCL for matrix multiplication. This will provide an alternative to CUDA, allowing users with non-Nvidia GPUs to leverage the power of their hardware for matrix operations.

Motivation

While CUDA offers robust solutions for Nvidia GPUs, the user base with different GPU vendors is substantial. By incorporating OpenCL, we can make EDUX more accessible and versatile, catering to a wider audience.

Description

The goal is to implement the existing IMatrixArithmetic interface using OpenCL to perform matrix multiplication. The interface is as follows:

public interface IMatrixArithmetic {
  double[][] multiply(double[][] matrixA, double[][] matrixB);
}

Summary: Create comprehensive documentation and example use cases for the EDUX image augmentation library.

Details:

• Javadoc for all public classes and methods.
• README with clear, executable examples of all features.

Java Features: N/A (Documentation-related).

Tests: Verify that all examples in documentation are working as intended.

Background/Objective

Our 'Edux' library currently lacks an official landing page. As we aim to make our Java ML library accessible and easy to understand for new visitors and potential contributors, having a well-designed landing page is critical. This landing page will serve as the first point of contact where users can learn about the features, installation processes, and how to get involved with 'Edux.'

• Describe what is edux
• How to install part
• How to use part
• Clean & Simple design

We need a reliable test that shows that the MultiLayerPerceptron's training and predictions work correctly.

• use seaborn dataset
• train and check the accuracy value of the model.
• Test should be repeatable 3x.

Summary: Add flipping and color augmentation features to EDUX's image augmentation suite.

Details:

• Flipping: Horizontal and vertical flipping functions.
• Color Augmentation: Adjustments for brightness, contrast, saturation, and hue.

Java Features: Explore java.awt.image.RescaleOp for color operations.

Documentation: Extend Javadoc and provide README examples.

Tests: Include unit tests for color integrity and flip correctness.

Delete

• DataProcessor
• Records

We dont want to support Record based data preparation.

Add github pages action to the build pipeline

Description

Implement the Strassen algorithm for matrix multiplication. This algorithm is an advanced divide and conquer method that is more efficient for large matrices.

Task Details

• Objective: Implement the Strassen algorithm to multiply two matrices.
• Function Signature: double[][] multiplyMatricesStrassen(double[][] matrixA, double[][] matrixB)

Acceptance Criteria

• The implementation must adhere to the Strassen algorithm's divide and conquer approach.
• Proper handling of base cases and recursive subdivision of matrices.
• Unit tests to ensure the algorithm's correctness and to compare its efficiency with classical multiplication.