Perhaps have it mark a file when it stops, or hold a lock while it's running.

Perhaps the easiest-to-use structure is missed.txt, caught.txt, etc.

If cargo mutants is run from a subdirectory of a Rust source tree, with no -d, it errors. It could instead find the enclosing project, perhaps using cargo locate-project.

These options need to be passed consistently to all of check, build, and test.

For example, time-rs/time can't build its tests unless --all-features is specified. time-rs/time#497

More generally people might want to test some crates with specific features.

#59 is about having custom options specifically for the build, but these seem important enough that they might be worth handling specifically.

--release would be another good one to add, either for crates where it significantly helps the test suite speed, or for crates like syn where the test suite insists on --release. (Tangentially, it might actually be a good idea in general, depending on the balance of time between linking and running the test suite...)
This could be tested with a specific testdata tree that fails to build unless a given feature is given.

Some trees might have configured e.g. unused variables to be an error, for that real Go-in-Rust feeling.

These are likely to fail to build when mutated, but this isn't a useful signal. It'd be better to turn off these warnings for mutated builds.

There seem at least two options:

• Insert #[allow:all] in the mutated function?
• Set RUSTFLAGS=-Aall
• Set RUSTFLAGS=--cap-lints=allow

See also https://stackoverflow.com/a/25877389/243712

Perhaps, cargo mutants should only list mutants that were missed, as that's really the actionable output. -v could show everything that was tested.

So for my project I have to run with cargo test -- --test-threads 1 as the project uses unix APIs that aren't thread-safe within a process so running the integration tests in parallel causes failures. This means my unmutated run fails and I can't try cargo-mutants on my project. Not sure if there's a way to pass this that I've missed, but if there's not it feels like a useful thing to add (and I'd be willing to help 😄 )

This is a really nice crate and works pretty great!

I have one issue: I really don't want mutants to be a non-dev dependency. Do you think it's possible to define an attribute which can be checked, so for example the following example would work? This probably won't need the mutants crate at all.

#[cfg_attr(mutants, mutants::skip)]
fn function_to_skip(&mut self);

Another possibility would be to set the test-attribute when creating mutations, so this would work (this would only need a dev-dependency):

#[cfg_attr(test, mutants::skip)]
fn function_to_skip(&mut self);

Fuzz testing randomly varies the input to the program and sees if the program will crash. Mutation testing deterministically varies the program (in ways that will probably still compile) and sees if your tests detect the change.

Probably pseudo-random edits to the program are introducing bugs that should be caught by tests, and so if no tests fail that might be a gap in test coverage. (Or, it might be that the edit is semantically harmless, or it changes something that is very hard to test hermetically, such as making something slower…)

fuzz testing is generally used to tell you that your program will crash/hang/spin on bad input (i.e. there are bugs in your code, if you can’t trust the input); mutation testing tells you that some presumably-bad edits to your code would not be caught — i.e. if a hapless contributor proposed some patches they would not be caught by CI

fuzz testing e.g. with cargo-fuzz for Rust requires that you have an interface to “run the code on this blob of bytes”, e.g. “parse this blob of bytes as an MP3 file”; mutation testing doesn’t make any assumptions about the structure and doesn’t need you to write a harness: you can just run it on any existing crate with hermetic tests.

fuzz testing is necessarily sampling-based because there are so many possible input files once you get beyond a couple of bytes (256 << n_bytes); mutation testing can in principle test every mutation it can generate. (Although for large crates that might be a large number, it’s still probably only in the thousands-millions, not astronomical.)

Feature request

It would be handy to have a mutation to remove statements that call a single function or method. I see many times in bigger projects (written without TDD) that such things are not covered by tests.

For example, let's say I have the following code:

impl EventBus {

    pub fn deposit_event(event_id: u32) {
        print!("event with id {} is emitted", event_id);
    }

    pub fn do_logic(event_id: u32) {
        //some logic
        Self::deposit_event(3);
    }
}

It would be nice to have a mutation of this code where we would delete the following line in the do_logic method:

    Self::deposit_event(3);

What do you think, it is feasible to do such a thing?

Note

If I run cargo mutants, the following mutant is generated

src/main.rs:12: replace EventBus::deposit_event with () ... NOT CAUGHT in 0.248s

It rules out many cases, but does not rule out the situation where we have tests for public API deposit_event, but not for calling it within other public API do_logic

Reproduction

Steps

1. Clone the following test repo
2. Navigate to subproject ./delete-statement
3. Execute cargo mutants

Expected behaviour:

There is a mutant generated for removing a statement calling a specific function, something like:

src/main.rs:15: remove Self::deposit_event(3) ... NOT CAUGHT in 0.248s

Actual behaviour:

No such mutation is supported by command cargo mutants

Currently, if I want to use #[mutants(skip)], I need to add mutants to Cargo.toml as a dependency. The readme at least promises that there is no effect on the generated code by this. I think it should be sufficient to add it as a dev-dependency.

Minor issue, thanks for the tool!

It seems like ecryptfs has a limit of 143 chars for file name. That causes cargo mutants to fail sometimes.
Would you accept a PR with some limit imposed here

It would sometimes be useful to check for mutants in just one or a few files, e.g.

cargo mutants -f mutate.rs

Probably this should match a whole path from the root if it contains slashes, and just the filename if it does not.

Is there a way to get some descriptive text into the list? It might not be possible for plugin commands.

While there are issues with killing the test process while in progress, for some repositories this isn't an issue, and timeouts could be enabled by the command line

Block merging changes that don't have canonical formatting.

Hi me again 😅, I have the following test which works always outside cargo-mutants but failed in the unmutated baseline test on the final assert. The manifest dir stuff is all correct in the run as it works for other tests of a similar pattern. The only thing that stands out as unusual in this test is the modifying of the CARGO_HOME environment variable. I'm going to ignore it for now to see if I can get through to running mutants and if I can I'll look into it more afterwards. But just making this issue in case you can see anything that stands out initially.

#[test]
fn cargo_home_filtering() {
    let new_home =
        Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/data/HttptestAndReqwest/new_home");
    let previous = env::var("CARGO_HOME");

    let mut config = Config::default();
    config.test_timeout = Duration::from_secs(60);
    config.set_clean(false);
    let restore_dir = env::current_dir().unwrap();
    let test_dir = get_test_path("HttptestAndReqwest");
    env::set_current_dir(&test_dir).unwrap();
    config.manifest = test_dir;
    config.manifest.push("Cargo.toml");

    env::set_var("CARGO_HOME", new_home.display().to_string());
    let run = launch_tarpaulin(&config, &None);
    match previous {
        Ok(s) => env::set_var("CARGO_HOME", s),
        Err(_) => {
            let _ = Command::new("unset").args(&["CARGO_HOME"]).output();
        }
    }
    let (res, _) = run.unwrap();

    env::set_current_dir(restore_dir).unwrap();

    assert_eq!(res.iter().count(), 1);
}

Also repo link: https://github.com/xd009642/tarpaulin

cargo-mutants currently assumes that src/**/*.rs will find all the source files in a tree, which is typical but not necessarily true for Rust source trees. https://github.com/Nukesor/pueue is one example where it is not true.

Perhaps a better approach here is to look at the output of cargo metadata to find out where the source is? Or, possibly to look at the commands that are run during a build in the source tree...

list_mutants_in_all_trees runs --list on everything under testdata/tree, but this easily gives false negatives when switching between branches, because untracked target files can be left behind, causing the directory to still exist when the expected data does not.

One way would be to simply remove this test.

Another smaller step would be to run it only on directories that contain a Cargo.toml and so that seem to "really" exist.

For example Option<Result<String>> could generate various values within the tree of possibilities.

Conceivably this might generate too many mutations for some functions in which case we might need a cap.

Clap 3 seems to have addressed some of the weird API overgrowth in earlier versions, and also to be more actively maintained than Argh. In particular this would allow syntax like --cargo-arg=--release for #59: Argh insists on --cargo-arg --release.

Description

cargo mutants command does not generate mutants for workspace members on a workspace project. Let's say I have a workspace project which contains 2 binary and 1 lib cargo crates. When I execute the cargo mutants on the root, there are no mutants generated, so the workspace member crates are not included in the generation of the mutants.

I prepared a GitHub repo for reproducing the problem:

Reproduction

Prerequisite

Clone the following workspace test project test-repo-for-cargo-mutants

Steps:

• Go to the root folder of the cloned repo
• Execute cargo mutants

Expected behavior:

Mutants are generated for all the projects in the workspace.

Note: I would expect this to happen because when for example I execute cargo test, then all the tests of workspace member crates are executed. The same for cargo build.

Actual behavior:

No mutants are generated

==PS==
Anyway, awesome job with this crate, thank you!

Is there an option somewhere to disable running doctests? I am attempting to run this on a fairly large project with over 50 doctests, but it causes each mutant to take a long time to run. That, combined with a lot of mutations (over 300) makes for a long run-time.

For comparison, running normally takes ~15 seconds for each mutation, which ends up being about an hour and a half. However, if I modify my library's Cargo.toml to disable doctests (setting doctests = false), the time for each mutant reduces to about 1-1.5 seconds each. That ends up being only approximately 7.5 minutes, which is much more desirable, especially if I want to run this in CI.

Is there a way a flag can be added to disable doctests? Or perhaps one already exists? In general, my doctests tend to test things already covered in my unit tests, and serve more as examples in the documentation, so I don't think much is lost by excluding them.

Right now, the entire function path is highlighted with a single color in the terminal output. This makes it difficult to read, especially when the path is deeply nested or contains complex <impl Trait for T> parts.

I propose using the basic set of ANSI escape code colors to highlight different parts of the path, similar to how an IDE would. I believe the most important parts to distinguish by color are the brackets (<>) and double colons (::). Nice "could haves" would be different colors for keywords (impl, for), module names, type names and function names.

As far as I know, it should be possible to identify these parts of the path context-free, with the exception of type names and function names (here its possible to make some pretty good guesses though).

I believe that we could improve the names of our automated tests, and setting up a guideline for it, to

• make them more readable
• be able to capture more business/functional knowledge in the test names
• having them "standardized"

The ShouldWhen and the GivenWhenThen template are the most popular ones, personally I am a big fan of ShouldWhen template as it is quite flexible in most of the cases. But I am open to any other suggestions :)

Of course just as a guideline, and not a rigid rule. But eventually we could add this guideline to CONTRIBUTING testing section

Just to pick some examples:

list_mutants_well_tested_exclude_folder_filter 

would become

exclude_option_should_exclude_folder_when_specified_with_slash

Or

copy_testdata_doesnt_include_build_artifacts

would become

build_artifcats_should_not_be_included_when_testdata_is_copied

Or

error_when_no_mutants_found

would become

mutants_generation_should_return_error_when_no_mutants_are_found

What do you think?

For example, in a PR, it would be interesting to see new mutants that are missed relative to the baseline, or mutants that are newly caught.

This could be done by reading a file from mutants.out in the old tree and doing some kind of intelligent diff. The line numbers are likely to have changed but we could probably match on the function name and mutation.

So I've been running mutants on my code base all day, and while showing only the failing mutants and current one does keep the output nice and terse it does mean I have no idea how many more are left to run before it's finished. Something like N/total on the in-progress line would helpful for gauging if it's worth waiting for it to finish or whether I should consider killing it and adding some ignores in the codebase 😅

General interpretation of cfg conditions seems difficult but we could at least skip #[cfg(windows)] on Unix and so on. At the moment they are mutated and reported as missed.

This could perhaps use https://docs.rs/cargo-platform/0.1.2/cargo_platform/.

Cargo also seems to expose cfg variables to build scripts, but perhaps not otherwise. This might not be usable. https://doc.rust-lang.org/cargo/reference/environment-variables.html

https://docs.rs/target-spec/latest/target_spec/ exists and might be useful.

Although cargo and the test framework try to use many cores, they often go through phases of being single-threaded.

The overall mutation tests would go faster if we ran multiple cargo builds in parallel.

Conceivably up to NCPUs builds could be run in parallel. But on some machines that might cause thrashing or too much memory pressure. Possibly we should also look at available memory. There should at least be an option to limit it. As a starting point perhaps NCPUs/2 is reasonable.

This requires making multiple scratch directories. However, we should not make more than the number of discovered mutants.

• Factor out a CargoTask with a poll method
• Launch a thread for each worker
• Add a command-line option --jobs, -j and thread it through.
• For now, still default to 1 job?
• Make multiple build directories
• Give the progress model a concept of multiple cargo tasks being underway
• In the lab code:
  • While there's a build dir free and still work to be done, start a new job in it
  • Sleep a fraction of a second
  • Update the progress bar for all running jobs
  • Poll all jobs to see if any have finished
• Update readme docs
• Add at least a smoke test that -j2 works.
• Test Mutant::unapply.
• Clean up self-mutants.

Deferred until later:

• Consider increasing the auto-set timeout when there are multiple jobs?
• #94

In some situations where there are many possible mutants or they're slow to test, it might be good to say "test at many as you can in n minutes".

After the time expires cargo mutants should allow any current tests to finish and then return a result on what it found so far.

It needs to coordinate with or be routed through Nutmeg so that it doesn't scramble the progress bar.

Split out from #20:

It would be good to have --no-copy-target or similar, to omit copying /target.

This would be helpful in isolating any test dependencies on the original build directory, and make more sure that the builds are all hermetic. The downside is that at least the initial build may be slower.

Also in that case, there's probably no point doing a build in the source tree.

• Tests for this (perhaps just looking at the log output)
• Update the readme doc
• Update the design doc?

Following on from #58 it would be nice to have options to control which functions are mutated.

Perhaps a regexp against the entire generated function name.

For example you might want to exclude all impl Debug.

• Add tests

cargo-mutants has limited dependencies on Cargo and could in principle be used with any other build tool.

This bug is primarily blocked on someone providing a realistic example of a Rust tree that builds with something other than Cargo. A minimized example would be good, but it should correspond to some real tree to give confidence that any solution actually works at scale. Preferably there would be an open source tree as a real example on which this could be tested.

• Add a command-line option or configuration option saying which build system to use: or maybe auto-detect.
• Add some kind of build system trait.
• Find the root of the tree to copy. For cargo this is cargo locate-project.
• Find the sources for non-test targets within a directory. Currently this is from cargo metadata. For Bazel we could potentially look at bazel query?
• Work out how to adjust any relative dependency paths in the scratch tree.
• Generate commands to build and test the targets. It may be less clear, in Bazel, how to run the tests for changes to some given package, since tests can be in a different directory.

#72 adds some knowledge of cargo workspaces. These very loosely correspond to Bazel packages.

cc @kupiakos

If someone's iterating on trying to improve coverage it might be helpful to only try mutants that were not caught on the last run, maybe by automatically looking in mutants.out.

However you can get close to this today by narrowing it with -f.

Closely related to #57.

Implementation approach

I'll pick this up, based on ideas for cl3joly but in a separate branch.

This could be done in either a positive or negative sense:

• Skip mutants that passed on a previous run: will pick up newly added mutants (probably good); will test mutants that were not yet known to have passed; does require remembering which one are skipped or they will be included in a third run
• Only test mutants that failed on a previous run

Basic idea is to:

• Add an --iterate command-line option (it would not make sense to have in the config.)
• Add the concept of an "previous output directory", by default the same path that will be used for output.
• Read the previously caught mutants from caught.txt in that output directory
• (It would help here to also have a list of mutants that were still pending and untested when the previous run finished, #290.)
• If those lists are all empty, print an info message that there's no iterative work to do.
• After generating all the mutants, filter down to only the names that occur in the list. (There's an opportunity here to only generate mutants from files that occur in the lists, but that could be left for later.)

In this approach if you also change the filters, for example focussing on one file at a time, then after that change it will test everything, which seems OK.

https://github.com/kupiakos/papermario-solver/blob/main/Cargo.toml builds a WASM application as a cdylib. Cargo-mutants 0.2.11 doesn't treat cdylib targets as containing sources that should be mutated, so it finds no files and no mutants.

• Walk through *lib targets
• Add a tree with an example of this and test that mutants are generated
• Mention in https://github.com/sourcefrog/cargo-mutants/wiki/Trials#papermario-solver

Discussed in #23

Yeah, good idea. mutagen optionally does something live this according to its documentation, but I have not looked at the implementation.

We could take it a step further by understanding which tests run which function under test. Functions not reached by any test we know are just apparently not tested. Functions that are reached by some tests, we can mutate and then run only the relevant tests. This would potentially be dramatically faster on some trees.

That said, I think there are a few things that might make this annoying to implement reliably, but perhaps my preconceptions are out of date. In the past, getting coverage files out of Rust historically to be a bit platform-dependent and fiddly to set up in my experience. And, historically the output was in a platform-dependent format that required external preprocessing. Both of these are in tension with my goal for a very easy start with cargo-mutants.

However there is now https://blog.rust-lang.org/inside-rust/2020/11/12/source-based-code-coverage.html providing -Z instrument-coverage, which is moving towards stabilization as -C instrument-coverage.

So if this ends up with a way to just directly get a platform-independent coverage representation out of cargo this might be pretty feasible.

Coverage may still raise some edge cases if the test suite starts subprocesses, potentially in different directories, as both cargo-mutants and cargo-tarpaulin seem to do. Will we still collect all the aggregate coverage info? But, we could still offer it for trees where it does work well. And maybe it will be fine.

There might also be a hairy bit about mapping from a function name back to the right cargo test invocation to hit it. But that also can probably be done: if nothing else perhaps by just running the test binary directly...

Possibly this could be done with https://github.com/taiki-e/cargo-llvm-cov

I believe that it would be handy to exclude specified paths from the mutants generation.

Let's say that I want to exclude things like configuration logic or some test helper logic files, located within the src directory. Right now the default behaviour of cargo mutants command is that it tries to generate mutants for every file in the src directory and in their child directories.

I could envision having a command line parameter as follows:

cargo mutants --exclude <regexes-for-paths-to-be-excluded>

Use case scenario:

Prerequisite:

Git clone test repo

Steps:

1. Go to the main directory of the cloned repo
2. Execute the following command

cargo mutants --exclude "(\/src\/tests)" "(\/src\/config)"

Expected behaviour:

Every file which is located in either /src/tests or /src/config directories are excluded from mutants generation.

Actual behaviour:

Because of the default behaviour of cargo-mutants (and due to the non-existency of this command line parameter) there are mutants generated for the files both in /src/tests and /src/config

Requires changing cp_r to let a callback tell it to stop (sourcefrog/cp_r#1).

cargo mutants does many incremental builds and test runs. In most trees, most of the time is spent running the tests.

Tests will typically run faster with --release; in some crates it will be much faster.

So that should at least be an option (#59) but according to the principle of "make it easy" perhaps --release should be the default.

There are at least two potential downsides:

1. Conceivably in some trees the performance hit from doing the initial and incremental builds in release mode will outweigh the benefit of the tests being faster. However, even if that is true sometimes it's probably a win across most trees.
2. Some projects might have test suites that are less rigorous, or simply don't work properly, in release mode. For example they might rely on debug_assert to catch some bugs. I think this would be an unidiomatic use of Rust, but it might well happen.

I think those could both reasonably be handled by having an option to go back to debug builds.

Hi.

I created a simple fizz buzz program as example.
Executing 'cargo mutants' leads to the result

$ cargo mutants
Freshen source tree ... ok in 0.184s
Copy source and build products to scratch directory ... 7 MB in 0.326s 
Unmutated baseline ... ok in 1.892s   
Auto-set test timeout to 20.0s   
Found 2 mutations to test   

after executing

git add .
git commit -m "Example"

in a git Bash, executing 'cargo mutants' leads to s different result.

$ cargo mutants
Freshen source tree ... ok in 0.154s
Copy source and build products to scratch directory ... failed
error copying source tree . to C:/Users/Foo~1/AppData/Local/Temp/.tmpfEmxQg: copy source tree to lab directory

Caused by:
    0: writing file: C:\Users\Foo~1\AppData\Local\Temp\.tmpfEmxQg\.git\objects\07\4cf47c98083e4f871d3da4bc11050869ce2ec2: Zugriff verweigert (os error 5)
    1: Zugriff verweigert (os error 5)
Error: copy source tree to lab directory

Caused by:
    0: writing file: C:\Users\Foo~1\AppData\Local\Temp\.tmpfEmxQg\.git\objects\07\4cf47c98083e4f871d3da4bc11050869ce2ec2: Zugriff verweigert (os error 5)
    1: Zugriff verweigert (os error 5)

I tried it with different projects, but the result is always the same. Before committing everything is fine, after committing I get Access Denied (os error 5).

Sometimes tests fail in GitHub actions due to timeouts, e.g. https://github.com/sourcefrog/cargo-mutants/runs/7932363158?check_suite_focus=true.

It's not always the same test. It might be more common on Windows but it does happen elsewhere.

I think this is because the CI machines are over-committed and sometimes slow down.

Options in order of preference:

• Add more debug logging about timeouts that's visible in CI, to help understand the cause. (Maybe send trace logs to stdout?)
• Take the minimum timeout from an environment variable that can be overridden in CI.
• Set a longer timeout specifically during cargo-mutants' own test suite?
• Increase the default minimum timeout from 20s (although that already seems like a very long time for these simple tests.)
• Just ignore the flakes and live with it (not great).

It's possible that this is not just slow but actually somehow getting deadlocked?

failures:

---- cargo_mutants_in_replace_dependency_tree_passes stdout ----
thread 'cargo_mutants_in_replace_dependency_tree_passes' panicked at 'Unexpected failure.
code-3
stderr=```""```
command=`"D:\\a\\cargo-mutants\\cargo-mutants\\target\\debug\\cargo-mutants.exe" "mutants" "--no-times" "--no-copy-target" "--no-shuffle" "-d" "testdata/tree/replace_dependency"`
code=3
stdout="Copy source to scratch directory ... done\nUnmutated baseline ... ok\nFound 2 mutants to test\nsrc/lib.rs:6: replace is_even -> bool with false ... TIMEOUT\n2 mutants tested: 1 caught, 1 timeouts\n"
stderr=""
', /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f\library\core\src\ops\function.rs:248:5
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library\std\src\panicking.rs:584
   1: core::panicking::panic_fmt
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library\core\src\panicking.rs:142
   2: core::panicking::panic_display<assert_cmd::assert::AssertError>
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f\library\core\src\panicking.rs:72
   3: assert_cmd::assert::AssertError::panic<assert_cmd::assert::Assert>
             at C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\assert_cmd-2.0.4\src\assert.rs:1036
   4: core::ops::function::FnOnce::call_once<assert_cmd::assert::Assert (*)(assert_cmd::assert::AssertError),tuple$<assert_cmd::assert::AssertError> >
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f\library\core\src\ops\function.rs:248
   5: enum$<core::result::Result<assert_cmd::assert::Assert,assert_cmd::assert::AssertError> >::unwrap_or_else<assert_cmd::assert::Assert,assert_cmd::assert::AssertError,assert_cmd::assert::Assert (*)(assert_cmd::assert::AssertError)>
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f\library\core\src\result.rs:1484
   6: assert_cmd::assert::Assert::success
             at C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\assert_cmd-2.0.4\src\assert.rs:156
   7: cli::cargo_mutants_in_replace_dependency_tree_passes
             at .\tests\cli.rs:906
   8: cli::cargo_mutants_in_replace_dependency_tree_passes::closure$0
             at .\tests\cli.rs:901
   9: core::ops::function::FnOnce::call_once<cli::cargo_mutants_in_replace_dependency_tree_passes::closure_env$0,tuple$<> >
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f\library\core\src\ops\function.rs:248
  10: core::ops::function::FnOnce::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library\core\src\ops\function.rs:248
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


failures:
    cargo_mutants_in_replace_dependency_tree_passes

test result: FAILED. 54 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in [143](https://github.com/sourcefrog/cargo-mutants/runs/7932363158?check_suite_focus=true#step:6:144).12s

e.g.

Unmutated baseline ... ok in 30 + 40 + 10 s

So a pattern I've seen a number of places in rust code (and have in my own code in places) is:

impl SomeStruct {
	pub fn new() -> Self {
	    Self::default()
    }
}

In cargo-mutants it will take this function and apply the Default mutagen to it creating code that is equivalent to the source code and then register as not caught. This could be solved by #[mutants::skip] so it's not a must have but I figured it might be a reasonably simple/nice optimisation for working out the number of mutants. Similarly if someone has a function which returns a bool but is just a single literal the only valid mutagen is the negation of that constant not the constant itself

Description

The cargo mutants command fails when there is a local dependency for a binary project.

This happens in a bigger project of us, but I created a test repo for the sake of easy reproduction.

Reproduction

Steps:

1. Clone the following repo
2. Go to the ./main directory
3. Execute cargo mutants command

Expected behavior:

• Some mutations are generated (if possible)

Actual behavior:

• The command fails with the following log:

Freshen source tree ... ok in 0.015s
Copy source and build products to scratch directory ... 14 MB in 0.006s
Unmutated baseline ... FAILED in 0.007s

*** baseline
*** run /home/dmoka/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo check --tests
*** cargo result: Exited(101) in 0.007sency of package `main v0.1.0 (/tmp/cargo-mutants--eL1pHT.tmp)`

Caused by:
  failed to load source for dependency `utils`

Caused by:
  Unable to update /tmp/utils

Caused by:
  failed to read `/tmp/utils/Cargo.toml`

Caused by:
  No such file or directory (os error 2)

Note: when cargo run is executed in the ./main directory, the binary is executed properly with the local utils dependency included. So I assume that it should have something to do with the cargo-mutants crate itself.

System: Ubunt 22.04

If you need any more information, let me know, thank you!

Any extra arguments to cargo mutants are passed to cargo test, but it would be helpful in some cases to also pass custom arguments to cargo build (or cargo check if that's what's used.)

What you probably actually want to debug is target/debug/cargo-mutants mutants --exclude ..., after building it.

Note you must have mutants as argv[1] because that's the protocol for cargo subcommands.

This will run your new code as the direct child process and so the debugger should work.

I have used the VS Code debugger although I probably don't reach for it as often as would be optimal because I find the UI of configuring targets in json a bit weird. But this should work with anything that can launch and debug a child.

Originally posted by @sourcefrog in #65 (comment)