tum-i4 / aletheia Goto Github PK

View Code? Open in Web Editor NEW

4.0 9.0 1.0 2.12 MB

C# 21.82% HTML 65.89% CSS 5.67% JavaScript 6.62%

aletheia's Introduction

Aletheia

A framework for failure diagnosis

1. Requirements

a) Matlab Runtime 2016a
b) OpenCppCoverage tool, [has to be added in the System path variable]

2. Getting Aletheia

Run command git clone https://github.com/tum-i22/Aletheia.git
Then cd Aletheia
Find the visual studio solution file, open it and build the solution.
Or copy the exe file along with dll files. Aletheia is portable. Find the binaries in bin folder. dll files are needed.

3. Components

Alethia has three modules

Data Generation: to generte hit/count spectra
Failure Clustering: to cluster failing tests with respect to hypothesized faults
Fault Localization: to localize the faults

Help can be obtained by executing:

Aletheia.exe do=getHelp

A sample command to generate Hit Spectra:

Aletheia.exe do=GenerateHitSpectra separator=; project_path=path\to\*.vcxproj file source_directory=path\to\source degreeofparallelism=12 gtest_path=path\to\gtest.exe

For Clustering and Fault Localization, the first column of spectrum csv file must be labeled as Index and the last column must be labeled as Result

Sample command to do clustering on a hit spectrum:

Aletheia.exe do=cluster separator=, input_path=path\to\your\data.csv clustering_method=maxclust linkage_method=average linkage_metric=euclidean

Sample command to do fault localization:

Aletheia.exe do=faultLocalization separator=; input_path=path\to\your\data.csv ranking_metric=Tarantula

Different ranking metrics can be used for fault localization, Our tool allows the following ranking metrics: Dstar1, Dstar2, Dstar3, Dstar4, Jaccard, Tarantula and Ochiai. If you do not choose any ranking metric, Jaccard will be used as the default.

4. Plugin / VS Extension

There is a seperate Visual Studio Extension project located in https://github.com/tum-i22/AletheiaPlugin .

5. Contact

In case of question, simply email us at

Mojdeh Golagha ([email protected])
Abu Mohammed Raisuddin ([email protected])

aletheia's People

Contributors

Stargazers

Watchers

Forkers

csampaio26

aletheia's Issues

ML_SVMSGD test suit failure after injecting bug but HitSpectra failed to recognize the injected file.

Injected a bug for failing the ML_SVMSGD test suite. The test cases have failed.
Newly failed test cases in the ML_SVMSGD test suit are :

// N.B, Snapshot from the test run in command line ...

[ FAILED ] ML_SVMSGD.trainSameScale2
[ FAILED ] ML_SVMSGD.trainSameScale5
[ FAILED ] ML_SVMSGD.trainSameScale100
[ FAILED ] ML_SVMSGD.trainDifferentScales2
[ FAILED ] ML_SVMSGD.trainDifferentScales5
[ FAILED ] ML_SVMSGD.trainDifferentScales100
[ FAILED ] ML_SVMSGD.twoPoints

Bug injected in data.cpp file under opencv_ml module.

Ptr<TrainData> TrainData::create(InputArray samples, int layout, InputArray responses,
                                 InputArray varIdx, InputArray sampleIdx, InputArray sampleWeights,
                                 InputArray varType)
{
    CV_TRACE_FUNCTION_SKIP_NESTED();
    Ptr<TrainDataImpl> td = makePtr<TrainDataImpl>();
    layout = -100; //  Injected bug in this line
    td->setData(samples, layout, responses, varIdx, sampleIdx, sampleWeights, varType, noArray());
    return td;
}

Expected Behaviour: Hitspectra should recognize the data.cpp file in the matrix coverage.

Locale issue

In some country a comma is used as decimal separator. While exporting hit spectra in comma separated value format, the data might be corrupted.

Low speed of Spectra Calculation

An parallel implementation of spectra generation can highly reduce the spectra generation time. CUDA driver can be a useful too here.

Dependency on Matlab

One version of Matlab library is not compatible with another version. Therefore depending on Matlab Runtime is not a good idea. Moreover, while clustering, every time the matlab libraries are loaded into RAM killing some time (few minutes). Replacing matlab library with something else might speed up Aletheia a bit.

HitSpectra failed to recognize bug injected file in opencv_videostab module

Injected bug in the global_motion.hpp file in opencv_videostab module. Code snippet is given below :

class CV_EXPORTS MotionEstimatorRansacL2 : public MotionEstimatorBase
{
public:
    MotionEstimatorRansacL2(MotionModel model = MM_AFFINE);

    //void setRansacParams(const RansacParams &val) { ransacParams_ = val; }

    // @Injected Bug, Added by : Reza
    void setRansacParams(const RansacParams &val)
    {
            ransacParams_ = val;
            ransacParams_.size = -1 * val.size;
    }

    RansacParams ransacParams() const { return ransacParams_; }

    void setMinInlierRatio(float val) { minInlierRatio_ = val; }
    float minInlierRatio() const { return minInlierRatio_; }

    virtual Mat estimate(InputArray points0, InputArray points1, bool *ok = 0);

private:
    RansacParams ransacParams_;
    float minInlierRatio_;
};

// Snapshot from the test run in command line ...

[----------] Global test environment tear-down
[==========] 6 tests from 1 test case ran. (15 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 6 tests, listed below:
[ FAILED ] Regression.MM_TRANSLATION
[ FAILED ] Regression.MM_TRANSLATION_AND_SCALE
[ FAILED ] Regression.MM_ROTATION
[ FAILED ] Regression.MM_RIGID
[ FAILED ] Regression.MM_SIMILARITY
[ FAILED ] Regression.MM_AFFINE

Expected Behaviour: Hitspectra should recognize the global_motion.hpp file in the matrix coverage.

Unit Test for Aletheia

Write unit tests for Aletheia

Dependency on OpenCppCoverage

OpenCppCoverage does not provide coverage for many of the projects of our consideration. As a result we don't have spectra matrix of many projects. The correctness of some coverage also not pure.