Git Product home page Git Product logo

effcee's Introduction

Effcee

Linux and OSX Build Status

Effcee is a C++ library for stateful pattern matching of strings, inspired by LLVM's FileCheck command.

Effcee:

  • Is a library, so it can be used for quickly running tests in your own process.
  • Is largely compatible with FileCheck, so tests and test-writing skills are transferable.
  • Has few dependencies:
    • The C++11 standard library, and
    • RE2 for regular expression matching.
    • Abseil utilities for C++ (via RE2).

Example

The following is from examples/main.cc:

    #include <iostream>
    #include <sstream>

    #include "effcee/effcee.h"

    // Checks standard input against the list of checks provided as command line
    // arguments.
    //
    // Example:
    //    cat <<EOF >sample_data.txt
    //    Bees
    //    Make
    //    Delicious Honey
    //    EOF
    //    effcee-example <sample_data.txt "CHECK: Bees" "CHECK-NOT:Sting" "CHECK: Honey"
    int main(int argc, char* argv[]) {
      // Read the command arguments as a list of check rules.
      std::ostringstream checks_stream;
      for (int i = 1; i < argc; ++i) {
        checks_stream << argv[i] << "\n";
      }
      // Read stdin as the input to match.
      std::stringstream input_stream;
      std::cin >> input_stream.rdbuf();

      // Attempt to match.  The input and checks arguments can be provided as
      // std::string or pointer to char.
      auto result = effcee::Match(input_stream.str(), checks_stream.str(),
                                  effcee::Options().SetChecksName("checks"));

      // Successful match result converts to true.
      if (result) {
        std::cout << "The input matched your check list!" << std::endl;
      } else {
        // Otherwise, you can get a status code and a detailed message.
        switch (result.status()) {
          case effcee::Result::Status::NoRules:
            std::cout << "error: Expected check rules as command line arguments\n";
            break;
          case effcee::Result::Status::Fail:
            std::cout << "The input failed to match your check rules:\n";
            break;
          default:
            break;
        }
        std::cout << result.message() << std::endl;
        return 1;
      }
      return 0;
    }

For more examples, see the matching tests in effcee/match_test.cc.

Status

Effcee is mature enough to be relied upon by third party projects, but could be improved.

What works:

  • All check types: CHECK, CHECK-NEXT, CHECK-SAME, CHECK-DAG, CHECK-LABEL, CHECK-NOT.
  • Check strings can contain:
    • fixed strings
    • regular expressions
    • variable definitions and uses
  • Setting a custom check prefix.
  • Accurate and helpful reporting of match failures.

What is left to do:

  • Add an option to define shorthands for regular expressions.
    • For example, you could express that if the string %% appears where a regular expression is expected, then it expands to the regular expression for a local identifier in LLVM assembly language, i.e. %[-a-zA-Z$._][-a-zA-Z$._0-9]*. This enables you to write precise tests with less fuss.
  • Better error reporting for failure to parse the checks list.
  • Write a check language reference and tutorial.

What is left to do, but lower priority:

  • Match full lines.
  • Strict whitespace.
  • Implicit check-not.
  • Variable scoping.

Licensing and contributing

Effcee is licensed under terms of the Apache 2.0 license. If you are interested in contributing to this project, please see CONTRIBUTING.md.

This is not an official Google product (experimental or otherwise), it is just code that happens to be owned by Google. That may change if Effcee gains contributions from others. See the CONTRIBUTING.md file for more information. See also the AUTHORS and CONTRIBUTORS files.

File organization

  • effcee/ : library source code, and tests
  • third_party/: third party open source packages, downloaded separately
  • examples/: example programs

Effcee depends on:

  • the RE2 regular expression library.
  • the Abseil utility library for C++.

Effcee tests depend on Googletest and Python 3.

In the following sections, $SOURCE_DIR is the directory containing the Effcee source code.

Getting and building Effcee

  1. Check out the source code:
git clone https://github.com/google/effcee $SOURCE_DIR
cd $SOURCE_DIR/third_party
git clone https://github.com/google/googletest.git
git clone https://github.com/google/re2.git
git clone https://github.com/abseil/abseil-cpp.git
cd $SOURCE_DIR/

Note: There are two other ways to manage third party sources:

  • If you are building Effcee with Bazel (https://bazel.build), you do not need to clone the repositories for googletest and re2. They will be automatically downloaded by Bazel during build. Bazel will suggest adding sha256 attributes to each repository rule to get hermetic builds (these notices are safe to ignore if you are not interested in hermetic builds).
  • If you are building Effcee as part of a larger CMake-based project, add the RE2 and googletest projects before adding Effcee.
  • Otherwise, you can set CMake variables to point to third party sources if they are located somewhere else. See the Build options below.
  1. Ensure you have the requisite tools -- see the tools subsection below.

  2. Decide where to place the build output. In the following steps, we'll call it $BUILD_DIR. Any new directory should work. We recommend building outside the source tree, but it is also common to build in a (new) subdirectory of $SOURCE_DIR, such as $SOURCE_DIR/build.

4a) Build and test with Ninja on Linux or Windows:

cd $BUILD_DIR
cmake -GNinja -DCMAKE_BUILD_TYPE={Debug|Release|RelWithDebInfo} $SOURCE_DIR
ninja
ctest

4b) Or build and test with MSVC on Windows:

cd $BUILD_DIR
cmake $SOURCE_DIR
cmake --build . --config {Release|Debug|MinSizeRel|RelWithDebInfo}
ctest -C {Release|Debug|MinSizeRel|RelWithDebInfo}

4c) Or build with MinGW on Linux for Windows: (Skip building threaded unit tests due to Googletest bug 606)

cd $BUILD_DIR
cmake -GNinja -DCMAKE_BUILD_TYPE={Debug|Release|RelWithDebInfo} $SOURCE_DIR \
   -DCMAKE_TOOLCHAIN_FILE=$SOURCE_DIR/cmake/linux-mingw-toolchain.cmake \
   -Dgtest_disable_pthreads=ON
ninja

4d) Or build with Bazel on Linux:

cd $SOURCE_DIR
bazel build -c opt :all

After a successful build, you should have a libeffcee library under the $BUILD_DIR/effcee/ directory (or $SOURCE_DIR/bazel-bin when building with Bazel).

The default behavior on MSVC is to link with the static CRT. If you would like to change this behavior -DEFFCEE_ENABLE_SHARED_CRT may be passed on the cmake configure line.

Tests

By default, Effcee registers two tests with ctest:

  • effcee-test: All library tests, based on Googletest.
  • effcee-example: Executes the example executable with sample inputs.

Running ctest without arguments will run the tests for Effcee as well as for RE2.

You can disable Effcee's tests by using -DEFFCEE_BUILD_TESTING=OFF at configuration time:

cmake -GNinja -DEFFCEE_BUILD_TESTING=OFF ...

The RE2 tests run much longer, so if you're working on Effcee alone, we suggest limiting ctest to tests with prefix effcee:

ctest -R effcee

Alternately, you can turn off RE2 tests entirely by using -DRE2_BUILD_TESTING=OFF at configuration time:

cmake -GNinja -DRE2_BUILD_TESTING=OFF ...

Tools you'll need

For building, testing, and profiling Effcee, the following tools should be installed regardless of your OS:

  • A compiler supporting C++11.
  • CMake: for generating compilation targets.
  • Python 3: for a test script.

On Linux, if cross compiling to Windows: - MinGW: A GCC-based cross compiler targeting Windows so that generated executables use the Microsoft C runtime libraries.

On Windows, the following tools should be installed and available on your path:

  • Visual Studio 2015 or later. Previous versions of Visual Studio are not usable with RE2 or Googletest.
  • Git - including the associated tools, Bash, diff.

Build options

Third party source locations:

  • EFFCEE_GOOGLETEST_DIR: Location of googletest sources, if not under third_party.
  • EFFCEE_RE2_DIR: Location of re2 sources, if not under third_party.
  • EFFCEE_THIRD_PARTY_ROOT_DIR: Alternate location for googletest and re2 subdirectories. This is used if the sources are not located under the third_party directory, and if the previous two variables are not set.

Compilation options:

  • DISABLE_RTTI. Disable runtime type information. Default is enabled.
  • DISABLE_EXCEPTIONS. Disable exceptions. Default is enabled.
  • EFFCEE_ENABLE_SHARED_CRT. See above.

Controlling samples and tests:

  • EFFCEE_BUILD_SAMPLES. Should Effcee examples be built? Defaults to ON.
  • EFFCEE_BUILD_TESTING. Should Effcee tests be built? Defaults to ON.
  • RE2_BUILD_TESTING. Should RE2 tests be built? Defaults to ON.

Bug tracking

We track bugs using GitHub -- click on the "Issues" button on the project's GitHub page.

What uses Effcee?

References

effcee's People

Contributors

alan-baker avatar antiagainst avatar dneto0 avatar dnovillo avatar ehsannas avatar kraj avatar s-perron avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

effcee's Issues

Compiling with MSVC

There is no a single windows DLL export, it would be good to add
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON) in CMakeLists.txt

Integrating with OSS-Fuzz

Greetings effcee developers and contributors,

We’re reaching out because your project is an important part of the open source ecosystem, and we’d like to invite you to integrate with our fuzzing service, OSS-Fuzz. OSS-Fuzz is a free fuzzing infrastructure you can use to identify security vulnerabilities and stability bugs in your project. OSS-Fuzz will:

  • Continuously run all the fuzzers you write.
  • Alert you when it finds issues.
  • Automatically close issues after they’ve been fixed by a commit.

Many widely used open source projects like OpenSSL, FFmpeg, LibreOffice, and ImageMagick are fuzzing via OSS-Fuzz, which helps them find and remediate critical issues.

Even though typical integrations can be done in < 100 LoC, we have a reward program in place which aims to recognize folks who are not just contributing to open source, but are also working hard to make it more secure.

We want to stress that anyone who meets the eligibility criteria and integrates a project with OSS-Fuzz is eligible for a reward.

To help you getting started, we can provide an internal fuzzer for your project that you are welcome to use directly, or to use it as a starting point.

If you're not interested in integrating with OSS-Fuzz, it would be helpful for us to understand why—lack of interest, lack of time, or something else—so we can better support projects like yours in the future.

If we’ve missed your question in our FAQ, feel free to reply or reach out to us at [email protected].

Thanks!

Tommy
OSS-Fuzz Team

Assert when subtracting stringpiece iterators

When using the latest RE2, the windows debug builds give an assert.

input->remove_prefix(captured->end() - input->begin());

This subtraction with two iterators from different stringpieces gives an assert:

C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\include\xstring(1086) : Assertion failed: cannot subtract incompatible string_view iterators

This is caused by the definition of the re2 StringPiece class being replaced with std::string_view.

Is there another way to do what we want to do in effcee?

Migrate to GitHub Actions

It's annoying to maintain both Travis and AppVeyor configurations, and this is a small enough project that GitHub Actions is a better fit than Kokoro.

fix compilation of tests for MSVC

From the bot run for #59

FAILED: effcee/CMakeFiles/effcee-test.dir/cursor_test.cc.obj
C:\PROGRA2\MIB0551\2019\COMMUN1\VC\Tools\MSVC\14291.301\bin\Hostx64\x64\cl.exe /nologo /TP -DGTEST_HAS_COMBINE=1 -IT:\src\github\effcee\effcee.. -IT:\src\github\effcee\third_party\re2 -IT:\src\github\effcee\third_party\abseil_cpp -external:IT:\src\github\effcee\third_party\googletest\googlemock\include -external:IT:\src\github\effcee\third_party\googletest\googletest\include -external:IT:\src\github\effcee\third_party\googletest\googlemock -external:IT:\src\github\effcee\third_party\googletest\googletest -external:W0 /DWIN32 /D_WINDOWS /GR /EHsc /Zi /O2 /Ob1 /DNDEBUG -MD /wd4800 -std:c++17 /showIncludes /Foeffcee\CMakeFiles\effcee-test.dir\cursor_test.cc.obj /Fdeffcee\CMakeFiles\effcee-test.dir\ /FS -c T:\src\github\effcee\effcee\cursor_test.cc
T:\src\github\effcee\effcee\cursor_test.cc(139): error C2664: 'std::basic_string_view<char,std::char_traits>::basic_string_view(const char *const ,const std::basic_string_view<char,std::char_traits>::size_type) noexcept': cannot convert argument 1 from 'std::_String_view_iterator<_Traits>' to 'const char *const '
with
[
_Traits=std::char_traits
]

CHECK-NOT looks too far ahead.

If I add the following test to effcee, it fails. It seems like CHECK-NOT is checking right to the end of the string stead of checking just to the line that matches the following CHECK.

TEST(Match, AppearsAfterNextCheck) {
  const auto result =
      Match("Hello\nWorld\nBorg", "CHECK: Hello\nCHECK-NOT: Borg\nCHECK: World");
  EXPECT_TRUE(result);
}

Don't use StringPiece.as_string()

absl::string_view and std::string_view do not have a as_string() operator. This makes porting effcee to other environments a bit difficult. It would be nice to change such code to use an explicit std::string conversion, e.g. std::string(StringPiece), or to use a helper function.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.