While theoretically possible, in practice I've never found anyone using C# regexes via Regex.Matches. A standard practice here is to use var reg = new Regex(@"[^a-zA-Z&-]+", RegexOptions.Compiled);. Another case is the fact, that there seems to be no warmup phase for JITer compiler.

Hi guys 👋. Can I use your code for my research article?)

It might be worth using RegexOptions.ECMAScript in C# benchmark, or just add another bench for c# with this option. It drops some 'advanced' features like Unicode support, which increases performance in ~1.5-2 times.

I understand that it's a questionable decision, and I could mention some pros and cons here:

• We are competing with different regex engines, and some of them, like JS one, as far as I'm aware, does not support UTF. So it's a bit unfair to compare motorbike with a truck.
• It looks like we compare default regex usage patterns, and I have never seen anyone use RegexOptions.ECMAScript in a real life.

My results (Windows 10, dotnet core 2.1.4):

Original

1824.7035 - 92
1560.8753 - 5301
109.0058 - 5

With ECMAScript

1072.2842 - 92
908.0764 - 5301
110.9607 - 5

TCL regex engine is different from other regex engines. It will be nice to see here in benchmark. More about TCL regex engine: https://www.princeton.edu/~mlovett/reference/Regular-Expressions.pdf

I'd like to suggest that BenchmarkDotNet be used to perform the C# benchmark. There's a lot of overhead to running the C# executable once, as the JIT is doing a lot of upfront work coupled with the RegexOptions.Compiled flag.

Once the pump is primed, you'll notice that performance is better. I suppose it depends on what is trying to be measured, though. If it's to show the performance of running an executable that is trying to match the 3 regexes once and terminate, then the current approach works. However, this doesn't represent more typical usage in an application.

BenchmarkDotNet=v0.12.0, OS=Windows 10.0.18362
Intel Core i5-8500 CPU 3.00GHz (Coffee Lake), 1 CPU, 6 logical and 6 physical cores
.NET Core SDK=3.1.101
  [Host] : .NET Core 3.1.1 (CoreCLR 4.700.19.60701, CoreFX 4.700.19.60801), X64 RyuJIT

void Main()
{
    BenchmarkRunner.Run<RegexBenchmark>();
}

[MemoryDiagnoser]
public class RegexBenchmark
{
    private string _data;
    
    [GlobalSetup]
    public void GlobalSetup()
    {
        _data = File.ReadAllText(@"D:\Benchmarks\input-text.txt");
    }
    
    [Benchmark]
    public int Email()
    {
        MatchCollection matches = Regex.Matches(_data, @"[\w\.+-]+@[\w\.-]+\.[\w\.-]+");
        return matches.Count;
    }
    
    [Benchmark]
    public int Uri()
    {
        MatchCollection matches = Regex.Matches(_data, @"[\w]+://[^/\s?#]+[^\s?#]+(?:\?[^\s#]*)?(?:#[^\s]*)?");
        return matches.Count;
    }
    
    [Benchmark]
    public int Ip()
    {
        MatchCollection matches = Regex.Matches(_data, @"(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])");
        return matches.Count;
    }

    [Benchmark]
    public int EmailCompiled()
    {
        MatchCollection matches = Regex.Matches(_data, @"[\w\.+-]+@[\w\.-]+\.[\w\.-]+", RegexOptions.Compiled);
        return matches.Count;
    }

    [Benchmark]
    public int UriCompiled()
    {
        MatchCollection matches = Regex.Matches(_data, @"[\w]+://[^/\s?#]+[^\s?#]+(?:\?[^\s#]*)?(?:#[^\s]*)?", RegexOptions.Compiled);
        return matches.Count;
    }

    [Benchmark]
    public int IpCompiled()
    {
        MatchCollection matches = Regex.Matches(_data, @"(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", RegexOptions.Compiled);
        return matches.Count;
    }
}

I've just tried CTRE on these benchmarks. On my machine, it beats rust.

Here's the code:

#include "ctre.hpp"
#include <array>
#include <chrono>
#include <fstream>
#include <iostream>
#include <numeric>

template<size_t N>
void measure(const std::string& data) {
  using clock = std::chrono::high_resolution_clock;
  const auto start = clock::now();
  unsigned count = 0;
  if constexpr ( N == 0 )
    // Email
    for(auto match : ctre::range<"[\\w.+\\-]+@[\\w.\\-]+\\.[\\w.\\-]+">(data)) count++;
  else if constexpr ( N == 1 )
    // URI
    for(auto match : ctre::range<"[\\w]+:\\/\\/[^\\/\\s?#]+[^\\s?#]+(?:\\?[^\\s#]*)?(?:#[^\\s]*)?">(data)) count++;
  else
    // IP
    for(auto match : ctre::range<"(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])">(data)) count++;

  const auto end = clock::now();
  const double elapsed = std::chrono::duration_cast<std::chrono::nanoseconds>(end - start).count() * 1e-6;
  std::cout << elapsed << " - " << count << "\n";
}

int main(int argc, char** argv) {
  if (argc != 2) {
    std::cerr << "Usage: benchmark <filename>\n";
    return 1;
  }

  std::ifstream file{argv[1]};
  if (!file) {
    std::cerr << "unable to open " << argv[1] << "\n";
    return 1;
  }

  const std::string data{std::istreambuf_iterator<char>{file}, std::istreambuf_iterator<char>{}};

  measure<0>(data);
  measure<1>(data);
  measure<2>(data);
}

CTRE is different from other C++ regex libraries in that it computes the state machine at compile time. The hard part about submitting a CTRE pull request for this repo is that it isn't packaged for Ubuntu. On my machine I have just downloaded the header straight from raw.githubuserconent.com with wget. The API is also completely different, so I don't know if I should make a new file for the PR or...

Also, it needs GCC 9.

A compiled regular expression is almost always used rather than recompiling each time.
And the measurement should be an average of multiple times.
A benchmark is only useful if it is close enough to real applications.

There isn't any note in the README pointing to the used php version

.NET 5 has been released for several months, but the readme is still showing data from .NET Core 3.1. Could it be updated with numbers for .NET 5 instead? Thanks!

Henry Spencer wrote multiple regex libraries which were historically important and are still relevant today: https://garyhouston.github.io/regex/

The one that survives in production code today are the Tcl library which is used in both Tcl and Postgres.

It would be very interesting to know how they fare as they have a number of different approaches from the other historical lineages of regex library, many of which have not been picked up by modern implementations.

The Postgres regex engine supports a lot of modern features like being able to turn on/off greediness per capture group and supporting unicode (well whatever the OS puts in wchar_t).

https://www.regular-expressions.info/email.html

https://dart.dev/tools/dart-compile

Instead of dart2native the following command should be used:

   mkdir -p dart/bin && dart compile exe dart/benchmark.dart -o dart/bin/benchmark

A Regexp literal in Ruby, e.g. /foo/ results in a call to Regexp::compile, which means the Regexp is already created by the time measure is invoked.

Instead, pass a String pattern that is compiled with Regexp::compile, which makes the test more analogous to the Rust one.

The csharp code in the benchmark is incorrect: RegexOptions.Compiled Flags should not be set.
LINK: csharp/Benchmark.cs#L33

Because:
In many other languages, Compile is used as the Parser process for Regex. But on .NET, a Compiled option is provided to dynamically create IL objects, and the vast majority of cases should not used. It is easy to be used incorrectly.

In addition, the javascript code used is unfair.
The /...regex.../ syntax in javascript creates a Regex instance object that has been Parser, but the Parser consumption in other languages Benchmark is counted.

Hello everyone,

Language/engine features seem to be very controversial and give us a lot of different opinions. Because of that, I have decided to use default settings for every language/engine but I also decided to create a new branch where code and settings optimizations will be allowed. It includes disabling some features (Unicode support for example) or enabling specific mechanics(JIT for example), but the measure, style, and regex patters will be the same.

At this point, we have two benchmarks, the first for comparing default settings and the second to push the benchmark until the limit for this specific scenario.

Results for both benchmarks will be in master readme.

I will work on it the next days but I accept PR for the new branch optimized from now.

Python is a dynamic language. Go is a static language. There is no way that Go is slower than Python.

And C# is similar to Java in the runtime. So C# should at least faster than Python too.

This repository came up in a recent C++ discussion in /r/cpp and once thing was immediately pointed out as wrong.

https://github.com/mariomka/regex-benchmark/blob/master/cpp/benchmark.cpp#L8-L16

You're measuring the time it takes to create the regex too, not just the loop.

Second, std::regex is known to be terribly slow and boost::regex is API compatible with much better performance. On top of that CTRE is even faster.

Benchmarks on my computer, with C/PCRE for reference:

PCRE
bstaletic@Gallifrey c  (git)-[master]-% ./bin/benchmark ../input-text.txt
31.998263 - 92
26.271349 - 5301
7.516541 - 5

std::regex
bstaletic@Gallifrey cpp  (git)-[master]-% ./bin/benchmark ../input-text.txt
631.659 - 92
512.522 - 5301
391.858 - 5

boost::regex
bstaletic@Gallifrey cpp  (git)-[master]-% ./bin/benchmark ../input-text.txt
84.5532 - 92
86.0452 - 5301
34.0546 - 5

CTRE
bstaletic@Gallifrey cpp  (git)-[master]-% ./bin/benchmark ../input-text.txt
62.5399 - 92
52.8931 - 5301
12.9586 - 5

--> perl perl/benchmark.pl input-text.txt
213.690996 - 92
165.706158 - 5301
29.313803 - 5

--> perl perl/benchmark_bytes.pl input-text.txt
95.394135 - 92
62.829971 - 5301
24.369955 - 5

--> perl perl/benchmark_bytes_ascii.pl input-text.txt
57.080984 - 92
62.375784 - 5301
24.430990 - 5

--> perl perl/benchmark_bytes_ascii_nocapture.pl input-text.txt
57.039022 - 92
62.216997 - 5301
24.269104 - 5

--> uname -v
FreeBSD 11.1-RELEASE-p1 #0: Wed Aug  9 11:55:48 UTC 2017     [email protected]:/usr/obj/usr/src/sys/GENERIC

--> perl -E 'say $^V'
v5.26.1

--> diff -u perl/benchmark.pl perl/benchmark_bytes.pl
--- perl/benchmark.pl   2017-10-22 14:08:18.000000000 +0900
+++ perl/benchmark_bytes.pl     2017-10-22 14:30:27.000000000 +0900
@@ -18,7 +18,7 @@

 my ($filename) = @ARGV;

-open my $fh, '<:encoding(UTF-8)', $filename or die 'Could not open file.';
+open my $fh, '<', $filename or die 'Could not open file.';
 my $text;
 read $fh, $data, -s $filename;
 close $fh;

--> diff -u perl/benchmark_bytes.pl perl/benchmark_bytes_ascii.pl
--- perl/benchmark_bytes.pl     2017-10-22 14:30:27.000000000 +0900
+++ perl/benchmark_bytes_ascii.pl       2017-10-22 14:30:33.000000000 +0900
@@ -5,7 +5,7 @@

     my $start = Time::HiRes::gettimeofday();

-    my $count = () = $data =~ /$pattern/g;
+    my $count = () = $data =~ /$pattern/ga;

     my $elapsed = (Time::HiRes::gettimeofday() - $start) * 1e3;


--> diff -u perl/benchmark_bytes_ascii.pl perl/benchmark_bytes_ascii_nocapture.pl
--- perl/benchmark_bytes_ascii.pl       2017-10-22 14:30:33.000000000 +0900
+++ perl/benchmark_bytes_ascii_nocapture.pl     2017-10-22 14:52:44.183470000 +0900
@@ -5,7 +5,7 @@

     my $start = Time::HiRes::gettimeofday();

-    my $count = () = $data =~ /$pattern/ga;
+    my $count = () = $data =~ /$pattern/gan;

     my $elapsed = (Time::HiRes::gettimeofday() - $start) * 1e3;

For Nim, you should also use the -d:danger flag to get the fastest speed.

@BurntSushi

fn measure(pattern: &str, data: &str) {
    let start = Instant::now();

    let regex = Regex::new(pattern);
    match regex {
        Ok(re) => {
            let count = re.find_iter(data).count();

            let elapsed = Instant::now().duration_since(start);
            println!("{} - {}", elapsed.as_secs_f64() * 1e3, count);
        }
        Err(err) => {
            println!("{:?}", err);
        }
    }
}

prints

regex parse error:
    .* x-x[[0-9]+] time="[0-9\.\:\-T]+" level=error msg="x x x x ([a-z0-9]{64}): x to x device ([a-z0-9]{64})-init:devicemapper: x x x x failed""
                                           ^^
error: unrecognized escape sequence

while other languages and online regex compile it just fine
without changing the regex