domohuhn / mutation-test Goto Github PK
View Code? Open in Web Editor NEWAutomated mutation testing for any programming language
License: BSD 3-Clause "New" or "Revised" License
Automated mutation testing for any programming language
License: BSD 3-Clause "New" or "Revised" License
Having a command line flag that takes a string as argument for the verification command would users a quick start.
For most CI tools, YAML files are used as inputs. The binary should switch from xml files to YAML files as inputs, as its syntax is easier for humans to read and write.
Specifying (flutter test test/some_test_file.dart) as command in the input files (ex: config.xml) is throwing the following Error / Exception :
"Error while processing:
ProcessException: Das System kann die angegebene Datei nicht finden.
Command: flutter test test/example_test.dart"
Are "flutter test" commands not supported ?
The reports currently only show the undetected mutations in the files. Showing all mutations gives a better picture in the reports.
The name flag for commands serves no purpose- remove it.
Users should be able to choose if they want the builtin rules or not.
Use list of files from the command line arguments as input files if the file extension is not .xml. This allows users to run tests without having to create a xml file with the inputs.
Having the ability to combine multiple rule documents gives users more flexibility how they use the application.
Creating output will fail with an exception if the path to the input file does not contain a /.
The output on the command line should be improved by including a progress bar and an remaining estimate.
The configuration should contain thresholds that describe the percent of undetected mutations for a run to be considered as failed.
For the given String 'a+b'
the Regex will be a match and it will be executed, it's a false positive since it's not an arithmetic operation.
For String templates, only mutates if it's using variables such as '$a+$b'
, '${a + b}'
, etc.
The tests currently do not run code in several files (and codecov doesnot see them either):
The deliveries for linux and mac are missing the executable permissions and should not have the .exe extension. It is probably better to use .tgz for linux as archive format.
Ideally, this should be automated using a github action.
What could be causing such an Error ?
PS C:\Users\M\Desktop\my_app> dart run mutation_test mutation_testing/config.xml --rules mutation_testing/small_test.xml -d
Error while processing:
RangeError (start): Invalid value: Not in inclusive range 0..176: 179
My config.xml :
<?xml version="1.0" encoding="UTF-8"?>
<mutations version="1.0">
<!-- License: BSD-3-Clause -->
<directories>
<directory recursive="true">lib
<matching pattern="\.dart$"/>
</directory>
</directories>
<exclude>
<!-- lines can also be globally excluded -->
<lines begin="1" end="2"/>
</exclude>
<commands>
<!-- <command group="test" expected-return="0" working-directory="./">dart //version</command> -->
<command group="test" expected-return="0" working-directory="./">C:\scr\flutter\bin\flutetr.bat test</command>
</commands>
</mutations>
my small_test.xml :
<?xml version="1.0" encoding="UTF-8"?>
<mutations version="1.0">
<!-- License: BSD-3-Clause -->
<!-- The rules element describes all mutations done during a mutation test -->
<!-- The following children are parsed: literal and regex -->
<!-- A literal element matches the literal text -->
<!-- A regex element mutates source code if the regular expression matches -->
<!-- Each of them must have at least one mutation child -->
<rules>
<!--this replaces (x op y,x) and (x op y,y)-->
<regex pattern="([A-Za-z0-9._]+(?:\(\)|\s*))\s+([\*\/\+\-\~])\s+([A-Za-z0-9._]+(?:\(\)|\s*))" dotAll="false">
<mutation text="$1"/>
<mutation text="$3"/>
</regex>
</rules>
<!-- This element creates a blacklist, allowing you to exclude parts from the mutations -->
<exclude>
<!-- excludes anything between two tokens -->
<token begin="//" end="\n"/>
<token begin="#" end="\n"/>
<token begin="import" end=";"/>
<!-- excludes anything that matches a pattern -->
<regex pattern="/[*].*?[*]/" dotAll="true"/>
<!-- exclude loops to prevent infinte tests -->
<regex pattern="[\s]for[\s]*\(.*?\)[\s]*{" dotAll="true"/>
<regex pattern="[\s]while[\s]*\(.*?\)[\s]*{.*?}" dotAll="true"/>
<!-- lines can also be globally excluded -->
<!-- line index starts at 1 -->
<lines begin="1" end="2"/>
</exclude>
</mutations>
We are trying to run a mutation test for our project daily nightly. The estimated time to complete the test shown by the tool is ~21h 12m 17s, which makes it hard to run the mutation test for the whole project.
As far as I understood, this tool generates the mutants for the code regardless of whether the test suites cover it. If so, it will be better to generate mutants only for the code covered by the test suites.
I'd also like to know how mutants are subjected to test suites. Is every mutant subjected to the whole test suite or only the test that covers the mutant itself?
Running without any commandline options should assume the standard dart layout and glob all files in ./lib/ as inputs and use dart test as test command.
Maybe all OS licenses should be included in releases.
Similar to https://github.com/muter-mutation-testing/muter#muter-can-be-run-in-your-ci
Mutation tests are usually slow to run, I'd like to test only the modified files in my PR via CI.
The mutated lines should be enclosed in div elements and their style should be set via css instead of inline styles.
Positional and Optional positional parameters may have bugs if misplaced, but I don't see how changing the order of named parameters would help.
I am currently working on a Flutter testing project to implement effective mutation testing techniques. However, these techniques require a kill matrix that can illustrate which test kills which mutant. I was wondering if there is any way to detect this using built-in mechanisms of the tool
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.