Git Product home page Git Product logo

two-face's Introduction

two-face

build status Crates.io codecov

Dedicated to chasing the bat man

Extra syntax and theme definitions for syntect including many common ones that are missing from the default set like TOML, TypeScript, and Dockerfile. Curated by the bat Project

Example

The following

$ cargo add two-face --features syntect-default-onig
use two_face::re_exports::syntect;

const TOML_TEXT: &str = "\
[section]
key = 123
";

fn main() {
    let syn_set = two_face::syntax::extra_newlines();
    let theme_set = two_face::theme::extra();

    let syn_ref = syn_set.find_syntax_by_extension("toml").unwrap();
    let theme = theme_set.get(two_face::theme::EmbeddedThemeName::Nord);
    let htmlified = syntect::html::highlighted_html_for_string(
        TOML_TEXT,
        &syn_set,
        syn_ref,
        theme
    ).unwrap();

    println!("{htmlified}");
}

will print this

<pre style="background-color:#2e3440;">
<span style="color:#d8dee9;">[section]
</span><span style="color:#81a1c1;">key </span><span style="color:#d8dee9;">= </span><span style="color:#b48ead;">123
</span></pre>

Feature Flags

The feature flags are divided by syntect's underlying regex implementation with Oniguruma aka onig being the default and fancy-regex aka fancy as an alternative pure-Rust implementation. fancy: however, doesn't support all of the features used by some of the syntax definitions, so some of the defintions are excluded when fancy is selected* to keep the regex compilation infallible. This means that it's important to match whichever regex implementation syntect is using

* This is also why fancy's bundled syntax definitions are smaller than onig's

default: syntect-onig

Feature Desc.
syntect-onig / syntect-fancy Enables the minimal feature set that we require from syntect
syntect-default-onig / syntect-default-fancy The mimimal feature sets along with syntect's default feature set (useful when using the syntect re-export)

Embedded Asset Sizes

This crate embeds some reasonably large assets in the final binary in order to work. Luckily the linker is smart enough to discard unused assets, so you generally only pay for what you use

For reference here are the sizes associated with their different functions

function two-face (KiB) syntect (KiB)
acknowledgement::listing() 10 -
syntax::extra_newlines() (onig) 859 360
^^ (fancy) 804 360
syntax::extra_no_newlines() (onig) 858 359
^^ (fancy) 803 359
theme::extra() 45 5

In short the syntax definitions are the real chonky part, and if you're switching from syntect to two-face, then you can expect a ~0.5MiB increase in binary size from them (in exchange for a lot of syntax definitions)

Syntaxes

The full listing of all syntaxes included in two_face::syntax

  • * Excluded when using the fancy-regex implementation
  • † Included in syntect's bundled defaults
Syntax Definition
A ActionScript†, Ada, Apache Conf, AppleScript†, AsciiDoc, ASP†, ARM Assembly*, Assembly (x86_64), AWK
B Bash†, Batch File†, BibTeX†
C C†, C#†, C++†, Cabal, Clojure†, CMake, CoffeeScript, Crontab, Crystal, CSS†, CSV†
D D†, Dart, Dockerfile, DotENV, Diff†
E Elixir, Elm, Email, Erlang†
F F#, Fish, Fortran
G Git (commit, config, ignore, etc.)†, GLSL, Go†, GraphQL, Graphviz (DOT)†, Groff/troff†, Groovy†
H Haskell†, HTML†
I INI
J Java†, Javadoc†, Java Server Page (JSP)†, JavaScript†, JavaScript (Babel)*, Jinja2, JQ, JSON†, Julia
K Kotlin
L LaTeX†, LaTeX Log†, Lean, LESS, Lisp†, Literate Haskell†, LiveScript*, LLVM, Lua†
M Makefile†, Manpage, Markdown†, MATLAB†, Mediawiki, MutliMarkdown†
N NAnt Build File†, Nginx, Nim, Ninja, Nix
O Objective-C†, Objective-C++†, OCaml†, OCamllex†, OCamlyacc†, Org Mode
P Pascal†, Perl†, PHP†, PowerShell*, Protobuf, Puppet, PureScript, Python†
Q QML
R R†, Racket, Rd†, Rego, Regular Expression†, Requirements.txt, reStructuredText†, Robot Framework, Ruby†, Ruby Haml†, Ruby on Rails†, Ruby Slim, Rust†
S Sass*, Scala†, SCSS, Salt State SLS*, SML, Solidity, SQL†, Strace, Stylus, Svelte, Swift, SystemVerilog
T Tcl†, Terraform, TeX†, Textile†, Todo.txt, TOML, TypeScript, TypescriptReact
V Varlink, Verilog, VimL, Vue, Vyper
X XML†
Y YAML†
Z Zig

Themes

Note: For visual examples of all of the embedded themes look at the docs for two_face::theme::EmbeddedThemeName

The full listing of themes provided by two_face::theme. Many of these themes only make sense situationally, so you'll likely want to only expose a subset

  • † Included in syntect's bundled defaults
Theme
1 1337 (aka leet)
A Ansi
B Base16, Base16-256, Base16-Eighties (dark)†, Base16-Mocha (dark)†, Base16-Ocean (light/dark)†
C Coldark (cold/dark aka light/dark)
D DarkNeon, Dracula
G GitHub, gruvbox (light/dark)
I InspiredGitHub†
M Monokai Extended (plain, bright, light, and origin)
N Nord
O One Half (light/dark)
S Solarized (light/dark)†, Sublime Snazzy
T TwoDark
V Visual Studio Dark+
Z Zenburn

Legal

Most of the code for generating the syntax and theme dumps along with curating said syntax and themes is taken from bat. Because of this we also mirror bat's licenses by being dual licensed under MIT and Apache-2.0.

See the LICENSE-APACHE and LICENSE-MIT files for license details.

The embedded syntax definitions and assets also have their own licenses which are compiled into this markdown file along with programmatic access in the acknowledgement module.

bat's NOTICE

Copyright (c) 2018-2021 bat-developers (https://github.com/sharkdp/bat).

bat is made available under the terms of either the MIT License or the Apache License 2.0, at your option.

See the LICENSE-APACHE and LICENSE-MIT files for license details.

two-face's People

Contributors

cosmichorrordev avatar dependabot[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

open-runtime

two-face's Issues

Short term

Dumping here since I'm too busy to work on this for the next few weeks

  • Setup CI
    • Test over a decent set of feature flags
    • Could track code cov
    • Allow cutting a release from CI (on second thought, nah)
    • Make sure to also test just the crate, so that feature unification from xtask doesn't foul things up
  • Expand tests heavily
    • Would be cool to add a test that keeps track of embedded asset sizes
  • Have newline and no newline variants for syntaxes (separate feature for each)
  • Add an enum that enumerates all the themes contained in the provided set
    • You should be able to infallibly access a theme by enum variant
  • Add a release checklist since releases are non-trivial

Add luau support

Can you add luau support here? or maybe a way to be able to use .add_from_folder from syntect? Thanks!

Approaching goofy levels of testing

There's a few unconventional things that I'd like to add some equally unconventional tests for. Namely:

  • Test that the linker strips away unused embedded assets
    • Have different integration tests that make use of embedded assets. Do an optimized build then inspect the .rodata section of the binary to verify whether or not they're included
  • Add a watchdog test to keep track of our syntect version and manually enter in relevant info for the assets sizes and an identifying fingerprint for testing if they're in the .rodata
  • Add a release workflow that ensures the assets are up-to-date. The asset generation seems stable now that the Cargo.lock file is committed (on linux at least, but that's what we run in CI, so no problem there)
    • Specifically this would be a release workflow instead of a regular CI run because I don't want to have a ton of churn that winds up bloating our git history

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.