Add the SonarSearch project as a source, and consider removing RapidDns. Most data would be duplicated?

Some duplicates where slipping through if they where formatted like:
subdomain.example.com\nsubdomain.example.com because the split wasn't done until after the filtering:

The split is now done before any of the filtering.

With the addition of SonarSearch it seems redundant to use another data source which uses the same dataset as SonarSearch.

Add RiskIQ PassiveTotal as a source

Being able to specify connection timeout gives the user the ability to skip sources which take a longer amount of time to return results. For example, Wayback Archive

Currently if you pass in a large number of root domains(~7000) into vita, the compiled regex exceeds the size limits and causes a panic.

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: CompiledTooBig(10485760)', src/bin/main.rs:36:23
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

To fix this we should add an increase to that size limit using:
https://docs.rs/regex/1.3.9/regex/struct.RegexSetBuilder.html#method.size_limit

Hello! From my understanding of the code, in here, if I call vita -d a.b.com, the tool will run on b.com pulling all sorts of domains I'm not interested in.

Would you consider a feature to toggle that behavior something interesting for this tool?

Add Intelx as a data source.

Some changes have been made to the AnubisDB API that is causing a panic on the handling of the response. This can be triggered by passing in a domain which AnubisDB has no results for.

thread 'tokio-runtime-worker' panicked at 'called `Option::unwrap()` on a `None` value', src/sources/anubisdb.rs:20:9

Add Chaos as a data source.

Vita currently panics if the binaryedge API returns 401 unauthorised, fix it to handle the error not panic.

Add Github as a data source.

Add RapidDNS as a data source.

Make sure the enumeration results filter domains which would be considered out of scope. For example, if given the domain example.com currently the output would contain subdomains which include other country extensions:

somesubdomain.example.com.au
somesubdomain.example.com

but the desired output would be:

somesubdomain.example.com

The reason we want to apply this fix is due to program scopes. We don't want to include subdomains in the results which would be out of scope. For example, given the root domain: example.com as in scope, but the domain example.com.au as out of scope. Therefore, we wouldn't want to include those in the output.

Currently if we receive an item in the response which doesn't match the length of the majority of the other responses, it would cause a panic because the index would be out of bounds.

thread 'async-std/runtime' panicked at 'index out of bounds: the len is 2 but the index is 2', src/sources/bufferover.rs:38:17

In the previous version we had no connection timeouts which would cause vita, in some instances to run infidelity and never finish.

thread 'tokio-runtime-worker' panicked at 'BINARYEDGE_TOKEN must be set in order to use Binaryedge as a data source: NotPresent

Async-std didn't cause a panic if the environment variables were not set, but tokio does.

Currently we use a hardcoded value to limit how many domains we can fetch data for at one time, but I feel like the user should have control of this value. For example, if I have a list of root domains which has over 10k entries, having a hardcoded value setting the limit to 200 might be capping me for no reason.

Instead we should add a flag with a default value set to something reasonable.

Hey cool tool! I was curious to what the benefits of this tool are over other similar tools i.e. subfinder, assetfinder, findomain, etc.

Currently the tool fetches all subdomains and at the end prints them all to stdout. When running again targets with very large amounts of subdomains, this allocates a very large vector for subdomains before calling cleaner.clean(subdomains). This is made worse by the fact that I'm running Vita on a 2 GB VPS which simply can't handle it and crashes.

vita -d comcast.net results in memory allocation of 1610612736 bytes failed I don't know if there's something going wrong with the error message, but that's quite a bit of memory!

Flushing results to output as they are fetched would solve that, however it might make it difficult to output only unique results. Personally I wouldn't mind a --flush switch that outputs duplicated results.

Instead of implementing this inside the run methods, we should instead make a struct and implement a method to read the credentials from environment variables. For example:

struct Creds {
  token: String,
}

impl Creds {
   pub fn read_env() -> Result<Self> {
      // returns credentials or errors
   }
}

Add a flag that enables the user to print useful information for debugging purposes or just because they're interested.

Users should have the ability to exclude sources they don't want to collect data from, so they don't have to resort to workarounds like the one mentioned here:
#33 (comment)