Git Product home page Git Product logo

csfastfloat's Introduction

csFastFloat : a fast and accurate float parser

.NET

C# port of Daniel Lemire's fast_float fully ported from C++ to C#. With older .NET frameworks (5.0 and older) it is up to 9 times faster than the standard library in some cases while providing exact results.

Note

An important portion of our work had been merged to .NET Runtime within version 7.0 of the .NET Framework.

Benchmarks

We use the realistic files in /data. The mesh.txt data file contains numbers that are easier to parse whereas the canada.txt data file is representative of a more challenging scenario. Synthetic.txt contains 150 000 random floats. We compare the Double.Parse() function from the runtime library with our FastFloat.ParseDouble() function. The ParseNumberString() only function parses the string itself without any float computation: it might represent an upper bound on the possible performance.

BenchmarkDotNet=v0.12.1, OS=ubuntu 20.04 (container)
AMD EPYC 7262, 1 CPU, 16 logical and 8 physical cores
.NET Core SDK=5.0.102
  [Host]        : .NET Core 5.0.2 (CoreCLR 5.0.220.61120, CoreFX 5.0.220.61120), X64 RyuJIT
  .NET Core 5.0 : .NET Core 5.0.2 (CoreCLR 5.0.220.61120, CoreFX 5.0.220.61120), X64 RyuJIT

Job=.NET Core 5.0  Runtime=.NET Core 5.0

|                         Method |           FileName |      Mean |       Min | Ratio | MFloat/s |     MB/s |
|------------------------------- |------------------- |----------:|----------:|------:|---------:|---------:|
|     FastFloat.TryParseDouble() |    data/canada.txt |  4.586 ms |  4.559 ms |  0.12 |    24.38 |   458.00 |
|     'ParseNumberString() only' |    data/canada.txt |  2.472 ms |  2.411 ms |  0.07 |    46.10 |   866.13 |
|                 Double.Parse() |    data/canada.txt | 37.537 ms | 37.159 ms |  1.00 |     2.99 |    56.19 |
|                                |                    |           |           |       |          |          |
|     FastFloat.TryParseDouble() |      data/mesh.txt |  1.834 ms |  1.834 ms |  0.27 |    39.81 |   338.05 |
|     'ParseNumberString() only' |      data/mesh.txt |  1.168 ms |  1.164 ms |  0.17 |    62.71 |   532.43 |
|                 Double.Parse() |      data/mesh.txt |  6.850 ms |  6.788 ms |  1.00 |    10.76 |    91.34 |
|                                |                    |           |           |       |          |          |
|     FastFloat.TryParseDouble() | data/synthetic.txt |  5.147 ms |  5.118 ms |  0.11 |    29.31 |   551.44 |
|     'ParseNumberString() only' | data/synthetic.txt |  2.655 ms |  2.653 ms |  0.05 |    56.54 |  1063.78 |
|                 Double.Parse() | data/synthetic.txt | 48.744 ms | 48.283 ms |  1.00 |     3.11 |    58.45 |

In this repo FastFloatTestBench we demonstrate a concrete performance gain obtained with FastFloat.ParseDouble() with the CSVHelper library. This is one of the fastest CSV parser available.

Requirements

.NET Standard 2.0 or newer. Under .NET 5 framework or newer, the library takes advantage of the new Math.BigMul() and Sse4.1 (SIMD) functions.

Compile and testing

As this library targets multiple framework, you can specify the target framework version with -f parameter :

dotnet build -c Release -f net8.0
dotnet test -f net8.0

If you omit the target framework and you don't have both .net 8.0 and dotnetcore 3.1 SDKs installed you may experience an error when building or running tests.

The set of unit tests in /TestcsFastFloat project combines unit tests from many libraries. It includes tests used by the Go Team. Additional info on Nigel Tao's work can be found here. It also include strtod test file that can be found here.

Some unit tests are based on Approvals.net library. They require a diff tool installed on your computer. Tests will be automatically skiped if no diff tool is found.

Usage

Two functions are available: FastDoubleParser.ParseDouble and FastFloatParser.ParseFloat. Since v3.0, TryParse pattern is supported for each function.

String, char * and ReadOnlySpan<char> are supported inputs.

using csFastFloat;


double x;
float y;
double z;
double answer = 0;
foreach (string l in lines)
{
        x = FastDoubleParser.ParseDouble(l);
        FastDoubleParser.TryParseDouble(l, out z);
        
        y = FastFloatParser.ParseFloat(l);
}

Input strings are expected to be valid UTF-16.

Trailing content in the string is ignored. You may pass an optional out int characters_consumed parameter FastDoubleParser.ParseDouble(l, out int characters_consumed) if you wich to check how many characters were processed. Some users may want to fail when the number of characters consumed does not match the string length.

For UTF-8 or ASCII inputs, you may pass a ReadOnlySpan<byte> argument. You can also pass an optional out int characters_consumed parameter to track the number of characters consumed by the number pattern.

Credit

This library was the main project in my master's degree in 2021 under the direction of Professor Daniel Lemire at TELUQ University. A special thanks to Egor Bogatov and all contributors for their really meaningful contribution.

Reference

csfastfloat's People

Contributors

carlverret avatar lemire avatar gfoidl avatar hterb avatar buybackoff avatar mburbea avatar egorbo avatar jnyrup avatar lordjz avatar

Stargazers

Olivier Giniaux avatar  avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.