Git Product home page Git Product logo

omar-azmi / fbicodec_ts Goto Github PK

View Code? Open in Web Editor NEW
0.0 1.0 0.0 250 KB

A Forward and Backward Invertible binary serializer TypeScript library built on modularity and composition. When you hear the words `FBI OPEN UP!`, you'll be able to prove your innocence with any data encoded with this library.

Home Page: https://omar-azmi.github.io/fbicodec_ts/

License: Other

TypeScript 100.00%
binary-serialization codec decoder deno encoder es6 file-parser invertible modular oop

fbicodec_ts's Introduction

FBIcodec

FBIcodec is so cleanly invertible, it's the last thing the FBI would investigate!

This is a typescript library for building bidirectional (encoding and decoding) codecs using composable steps. The "FBI" part in the name stands for "Forward-Backward-Interfaced", and Codec stands for an invertible data transformation.

The main building block of the library is its general purpose Step interface, which ensures the ability for a forward transformation to be reversed. The forward and backward methods take a single argument, allowing Steps to be composed into more complex Codecs.

Features

  • Composable steps for encoding and decoding binary data
  • Steps for primitive types like numbers, strings, bytes
  • Steps for composing records, arrays, sequences
  • Invertible - encoding and decoding are symmetric
  • Type safe

Overview

The main abstractions are:

  • Step - encodes forward and decodes backward between a source and target type.
  • Binarystep - a Step for encoding/decoding binary data.

To work with binary data of primitive types, such as: numbers, strings, bytes etc..., you can use the the BinaryPureSteps inside the file binary_primitive_steps.ts.

  • BinaryNumberStep - supports signed/unsigned integers of 8, 16, 32 and 64 bits, and floats of 32 and 64 bits, in both little-endian and big-endian formats.
  • BinaryStringStep - for strings with varying lengths (based on the args.length parameter).
  • BinaryCStringStep - for NUL ("\x00") terminated strings.
  • BinaryBytesStep - for sequence of bytes (based on the args.length parameter).
  • BinaryNumberArrayStep - for numeric array of numbers (based on the args.length parameter).

To work with binary data of composite types, such as: records, arrays, sequences etc..., you can use the the BinaryPureSteps inside the file binary_composition_steps.

  • BinaryRecordStep - encodes a record with named fields, when provided with a sequence of Steps for each field.
  • BinaryArrayStep - encodes an array of items, when provided with a single Step for the item type.
  • SequentialSteps - chains multiple Steps.

For example, here is a step that decode and then repack PNG image file's metadata blocks:

class PNG_Codec_Step extends PureStep<Uint8Array, GeneralChunk_schema[]> {
	forward(input: Uint8Array): GeneralChunk_schema[] {
		// parse input into chunks
	}
	backward(input: GeneralChunk_schema[]): Uint8Array {
		// encode chunks into binary
	}
}

Theory

The key components that enable building robust bidirectional transformations are:

  • Step<FROM, TO, LOST>

    • Defines symmetric forward and backward methods between source FROM and target TO types. Any lost information should be stored in the LOST private property.
  • PureStep<FROM, TO> extends Step<FROM, TO, never>

    • This is a Step that is guaranteed to not lose information during forward transformation, allowing backward transformation to fully reconstruct the original value. It also means that one may reuse a single instance of this kind of step for multiple forward and backward transformations, in whatever order it is deemed convenient. Hence, these kind of steps are suitable for being applied repeatdly by larger compositional steps.
  • BinaryStep<OUT, ARGS, LOST> extends Step<{ bin: Uint8Array, pos: number, args: ARGS }, { val: OUT, len: number }, LOST>

    • A step that specialized for parsing binary data. backward encodes the wrapped OUT value back into a wrapped Uint8Array object.
  • BinaryPureStep<OUT, ARGS> extends PureStep<{ bin: Uint8Array, pos: number, args: ARGS }>

    • Similar to BinaryStep, but is guaranteed not to lose data during forward transformation. This is the class that is basically used by all implementations of BinarySteps.

These simple interfaces allow Steps to be composed in a type-safe way.

Moreover, the compositions form a tree structure that preserves invertibility - as long as each Step inside implements their forward and backward methods correctly, the entire composition will be robustly reversible.

It also makes diagnosing issues easier - if there is a decoding error, it must be within one of the composed Steps. The composition tree narrows down where the issue lies.

fbicodec_ts's People

Contributors

actions-user avatar omar-azmi avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.