Git Product home page Git Product logo

goruntime's Introduction

Table of Contents generated with DocToc

Goruntime

Overview

Goruntime is a Go client for Runtime application level feature flags and configuration.

Installation

go get github.com/lyft/goruntime

Usage

In order to start using goruntime, import it to your project with:

import "github.com/lyft/goruntime"

Intended Use

The runtime system is meant to support small amounts of data, such as feature flags, kill switches, regional configuration, experiment settings, etc. Individual files should typically contain a single key/value pair (filename as key, content as value).

Components

The runtime system is composed of a Loader interface, Runtime interface and a Snapshot interface. The Snapshot holds a version of the runtime data from disk, and is used to retrieve information from that data. The Loader loads the current snapshot, and gets file system updates when the runtime data gets updated. The Loader also uses the Refresher to watch the runtime directory and refreshes the snapshot when prompted.

Refresher

The Refresher interface is defined like this:

// A Refresher is used to determine when to refresh the runtime
type Refresher interface {
	// @return The directory path to watch for changes.
	// @param runtimePath The root of the runtime path
	// @param appDirPath Any app specific path
	WatchDirectory(runtimePath string, appDirPath string) string

	// @return If the runtime needs to be refreshed
	// @param path The path that triggered the FileSystemOp
	// @param The Filesystem op that happened on the directory returned from WatchDirectory
	ShouldRefresh(path string, op FileSystemOp) bool
}

The Refresher determines what directory to watch for file system changes and if there are any changes when to refresh.

Two refreshers are provided out of the box

  • Symlink Refresher : Watches the runtime directory as if it were a symlink and prompts a refresh if the symlink changes.
  • Directory Refresher : Watches the runtime directory as a regular directory and prompts a refresh if the content of that directory change (not its subdirectories).

Loader

The Loader interface is defined like this:

type IFace interface {
	// @return Snapshot the current snapshot. This reference is safe to use forever, but will grow
	//         stale so should not be stored beyond when it is immediately needed.
	Snapshot() snapshot.IFace

	// Add a channel that will be written to when a new snapshot is available. "1" will be written
	// to the channel as a sentinel.
	// @param callback supplies the callback to add.
	AddUpdateCallback(callback chan<- int)
}

To create a new Loader:

import (
        "github.com/lyft/goruntime/loader"
        "github.com/lyft/gostats"
)

// for full docs on gostats visit https://github.com/lyft/gostats
store := stats.NewDefaultStore()
runtime, err := loader.New2("runtime_path", "runtime_subdirectory", store.Scope("runtime"), &DirectoryRefresher{}, opts ...Option)
if err != nil {
	// Handle error
}

The Loader will use filesystem events to update the filesystem snapshot it has.

NOTE: The old loader.New(...) function is deprecated in favor of loader.New2(...) which returns an error instead of panicking.

Loader Options

New2 is a variadic function that takes in arguments of type Option. These arguments are of the type func(l *loader) and are used to configure the loader being constructed. Dave Cheney wrote an article explaining this pattern.

Currently the loader package provides the following Options:

  1. AllowDotFiles: the loader will take into account dot files when it builds a snapshot.
  2. IgnoreDotFiles: the loader will ignore dot files when it builds a snapshot.

Snapshot

The Snapshot interface is defined like this:

type IFace interface {
	FeatureEnabled(key string, defaultValue uint64) bool

	// Fetch raw runtime data based on key.
	// @param key supplies the key to fetch.
	// @return const std::string& the value or empty string if the key does not exist.
	Get(key string) string

	// Fetch an integer runtime key.
	// @param key supplies the key to fetch.
	// @param defaultValue supplies the value to return if the key does not exist or it does not
	//        contain an integer.
	// @return uint64 the runtime value or the default value.
	GetInteger(key string, defaultValue uint64) uint64

	// Fetch all keys inside the snapshot.
	// @return []string all of the keys.
	Keys() []string

	Entries() map[string]*entry.Entry

	SetEntry(string, *entry.Entry)
}

A Snapshot is composed of a map of Entrys. Each entry represents a file in the runtime path. The Snapshot can be used to Get the value of an entry (or GetInteger if the file contains an integer).

Keys are built by joining paths with . relative to the runtime subdirectory. For example if this is your filesystem:

/runtime/
└── config
    ├── file1
    └── more_files
        ├── file2
        └── file3

And the runtime loader is setup like so:

store := stats.NewDefaultStore()
runtime, err := loader.New2("/runtime", "config", stats.Scope("runtime"), AllowDotFiles)
if err != nil {
	// Handle error
}

The values in all three files can be obtained the following way:

s := runtime.Snapshot()
s.Get("file1")
s.Get("more_files.file2")
//Supposed file3 contains an integer, or you want to use a default integer if file3 does not contain one
s.GetInteger("more_files.file3", 8)

goruntime's People

Contributors

ameliariely avatar benpope avatar charlievieth avatar danielmmetz avatar gitsen avatar jameswang14 avatar junr03 avatar m-rcl avatar rodaine avatar scode avatar usman-lyft avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

goruntime's Issues

Failure on kubernetes due to walking hidden subdirectories

I've tried to deploy https://github.com/lyft/ratelimit to Kubernetes, but it fails to find the correct configuration because goruntime first traverses a hidden directory, and then fails with a duplicate domain on the configuration it should actually load.

When Kubernetes mounts a configMap, it does symlink trickery to allow for atomic swapping; e.g.,:

bash-4.4$ ls -al 
total 32
drwxrwxrwx 3 root root 4096 May  9 17:03 .
drwxr-xr-x 3 root root 4096 May  9 17:03 ..
drwxr-xr-x 2 root root 4096 May  9 17:03 ..2018_05_09_17_03_25.201237914
lrwxrwxrwx 1 root root   31 May  9 17:03 ..data -> ..2018_05_09_17_03_25.201237914
lrwxrwxrwx 1 root root   18 May  9 17:03 config.yaml -> ..data/config.yaml
bash-4.4$ ls -al ..2018_05_09_17_03_25.201237914 
total 24
drwxr-xr-x 2 root root 4096 May  9 17:03 .
drwxrwxrwx 3 root root 4096 May  9 17:03 ..
-rw-r--r-- 1 root root  210 May  9 17:03 config.yaml

goruntime first loads ..2018_05_09_17_03_25.201237914/config.yaml, and then ./config.yaml is rejected with a duplicate domain. Due to the way goruntime forms a hierarchical key, ratelimit is unable to find suitable configuration.

Would it make sense to prevent walking directories that start with a dot? I guess they don't make a lot of sense given that the hierarchical key separator is also a dot.

I'm happy to make a PR if there is general agreement.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.