Git Product home page Git Product logo

nfx's Introduction

NFX - .NET Standard Unistack Framework








NFXv5 ANNOUNCEMENT

The current development of NFX HAS MOVED HERE is moving. This repository contains the current NFX (v5) project which targets .NET 4.7.1 and Net Stadard 2/ Core 2. We have moved from legacy NFX v3 repository

NFX(v5 and beyond) is based on .Net Standard 2, having all of cli tools targeting both .NET FX 4.7.1+ and .NET Core 2+. Future NFX will not officially support targets below .NET 4.7.1 (though possible) and .NET Core 2.

The legacy NFXv3* repo will continue to target .NET 4.5/Mono and may be supported for major bug fixes however all new feature development is going to happen here in NFXv5+.

Logo

Server UNISTACK (unified full software stack avoiding dependencies on 3rd party libs) framework.

License: Apache 2.0 Runs: .NET 4.7.1+, Core 2+

Project Badge

Documentation: http://nfxlib.com

Framework/Platform Support

NFX is now based on .NET Standard which works on different runtimes. Officially we support .NET Core and .NET Framework:

  • .NET Standard 2 - supported
  • .NET Core 2 - supported
  • .NET Framework 4.7.1 - supports classic .NET Framework 4.7.1+

NFX supports cross-platform development and is tested on:

  • Windows Win 2010 Core 2 and Net Fx
  • Linux Ubuntu 16 LTS using Core 2
  • Mac OS 14 using Core 2

NFX Builds on:

  • Windows / MSBuild 15 / VS 2017
  • in process - *nix (need scripts and build process refinements)

NFX abstacts platform-specific functions via PAL (Platform Abstraction Layer) which implements features such as: 2D graphics (drawing and image compression), CPU/RAM performance counters acquisition, Security functions (EnsureAccessibleDirectory etc.). PAL is implemented for every runtime differently and gets injected into NFX at process entry point.

All tools get compiled in .NET Core (out/{Config}/run-core) and .NET Framework (out/{Config}/run-netf) variants all targeting the same NFX.dll which is linked against .NET Standard 2.

About NFX

NFX is a modern .NET full stack framework designed for building cloud and on-premises apps. It is written in C# and runs on a CLR machine linking against .NET Stadard 2. NFX supports app containers, configuration, big memory heaps, IPC, and functions that significantly simplify the development of large distributed systems. It boosts performance and simplifies the development (such as services/web). The majority of the achievements are possible because of the following key features:

promoting:

  • Distributed systems: clusters/hierarchies
  • IPC - contract-based object-oriented high performance services
  • Stateful/stateless programming (web/service)
  • Full utilization of big RAM capacities in-proc (i.e. 128 Gb resident) without GC stalls

good for:

  • Linux-style prоgramming in .NET. Minimalistic, elegant.
  • Vendor de-coupling
  • General Scalability (i.e. hybrid DataStores with virtual queries and CQRS)
  • Distributed Macro/micro/nano services, REST or RPC
  • Actor-model systems with message passing and supervisors
  • In-memory processing, for hundreds of millions of objects in-RAM with full GC < 20ms
  • Supercomputer/Cluster/Farm applications (i.e. bulk rendering cluster, social graphs etc.)
  • High-performance business apps (i.e. serving 50,000+ BUSINESS WEB requests a second (with logic) on a general server looking up data in a 300,000,000 business object cache in-RAM)
  • non-trivial CLR cases like: structs with read-only fields, arrays of structs of structs, custom streamers like Dictionary<> with comparers etc. In the past 7 years, teleportation mechanism has moved trillions of various CLR object instances

Guides/Documentation

All Guides and Docs/Samples/Tutorials are on the NFXLIB site

NFXLIB - Documentation/Guides/Tutorials

NUGET

UNDER CONSTRUCTION As of 2018-01-14 you can use NFXv5 only if you get it here and build it. We are still working on publishing Nuget packages

NFX Packages TO BE RELEASED for NFXv5

Older (NFXv3) Package home:

cmd Description
pm> install-package NFX NFX Core Package (App, Pile, Glue, Log, Instr etc.)
pm> install-package NFX.Web NFX Web (Amazon S3, Google Drive, SVN, Stripe, Braintree, PPal, FB, Twtr etc.)
pm> install-package NFX.Wave NFX Wave Server + MVC
pm> install-package NFX.MsSQL NFX Microsoft SQL Server Provider
pm> install-package NFX.MySQL NFX MySQL RDBMS Provider (CRUD etc.)
pm> install-package NFX.MongoDB NFX MongoDB Proivder (CRUD etc.) + Native Client
pm> install-package NFX.WinForms NFX WinForms (for legacy)
pm> install-package NFX.Azure NFX Azure IaaS Provider WIP/pre-release
pm> install-package NFX.Erlang NFX Erlang Language + OTP Node

Resources

Big Memory Object Pile + Cache

NFX/ApplicationModel/Pile

Various Demo Projects

https://github.com/aumcode/nfx-demos

https://github.com/aumcode/howto()

NFX Provides

  • Unified App Container

    • Unified app models: console, web, service, all have: user, session, security, volatile state
    • Configuration: file, memory, db, vars, macros, structural merges, overrides, scripting
    • Dep injection: inject dependencies
    • Logger: async file, debug, db destinations with graphs, SLA rules, filtering and routing
    • Security: declr and/or imperative permission model, strong password manager, virtual credentials
  • Big Memory Pile

    • Pile memory manager: keeps hundreds of millions of CLR objects in memory without GC pauses
    • Distributed Pile (objects stored on cluster nodes)
    • Pile Cache: materialize 2,000,000 CLR objects/sec in-memory on a 4 core machine
  • Full Web Stack

    • Web server
    • Rule-based network Gate (business firewall)
    • MVC: filters, attributes, complex model binding, security, web API, MVVM binding
    • Template
    • Client JS lib + MVVM
  • Hybrid data access layer

    • RDBMS (we use: MySQL, MsSQL)
    • NoSQL (we use: MongoDB, Elastic search, Erlang OTP)
    • Native ultra-fast MongoDB driver (socket based achieving 200K+reads/sec)
  • Full Instrumentation Suite

    • Gauges and Events keyed on business enities
    • Instrumentation buffers
    • Injectable instrumentation sinks (i.e. telemetry receiver)
    • Cluster real-time map:reduce on zones and regions
  • High-level service oriented stack "Glue"

    • Contract based design with security
    • Injectable bindings: i.e. "Async TCP"
    • 150,000 ops/sec two-way calls using business objects (not byte array) on a 4 core machine
    • Unistack payload teleportation (no need to decorate various classes, teleport as-is)
  • Serialization Suite

    • Slim: the fastest general purpose CLR serializer, very well tested and proven processing
    • Teleportation: moving CLR objects as-is without any extra metadata between processes
    • BSON: an efficient BSON implementation
    • JSON: includes multi-language selective serialization of large graphs
  • Erlang Support: including types, serialization and full OTP node

  • Virtual File Systems: Amazon S3, SVN, Local, (Google Drive and DropBox created by others)

  • Virtual payment processing: Braintree, Stripe, PayPal

  • Code Analysis (building lexers/parsers)

  • Type Conversion Accessors (i.e. object.AsInt(dflt)....)

IMPORTANT!

NFX uses the very Base-Class-Lib of .NET:

  • Basic/primitive types: string, ints, doubles, decimal, dates, +Math
  • Parallel task library: 25% of features - create, run, wait for completion, Task, Parallel.For/Each
  • Collections: List, Dictionary, ConcurrentDictionary, HashSet, Queue
  • Threading: Thread, lock()/Monitor, Interlocked*, AutoresetEvent
  • Various: Stopwatch, Console, WinForms is used for SOME interactive tests(not needed for operation)
  • Some ADO references (Reader/SQLStatement) in segregated data-access components
  • Reflection API
  • Drawing 2D (Graphics)

NFX does not use any 3rd party components. (with the exception of system packages which are not in the CLR/sys "core" yet, such as System.Buffers, ValueTask(T) etc...)

nfx's People

Contributors

agnibos avatar itadapter avatar saleyn avatar shalokshalom avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nfx's Issues

Cover NFX.Graphics with unit tests

Such as (some rough ideas):

  • create various sized images, Image.Of(......)
  • Setpixel to preset colors
  • Save to file, stream, byte[], read back
  • Aver that read back image color are the same as the drawn presets
    etc...

BISAC codes increase assembly size by 300kb due to string repetition

Need to use prefix tree, as the whole structure consists of many repetitions, OR
we can store the data as JSON/laconic compressed include file, decompress it on run from resource segment.
We can also use SealedStrings not to keep all of the crap in memory

 private static readonly IDictionary<string, string> s_ANT
      = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
      {
        { "ANT000000", "ANTIQUES & COLLECTIBLES / General" },
        { "ANT001000", "ANTIQUES & COLLECTIBLES / Americana" },
        { "ANT002000", "ANTIQUES & COLLECTIBLES / Art" },
        { "ANT003000", "ANTIQUES & COLLECTIBLES / Autographs" },
        { "ANT005000", "ANTIQUES & COLLECTIBLES / Books" },
        { "ANT006000", "ANTIQUES & COLLECTIBLES / Bottles" },
        { "ANT007000", "ANTIQUES & COLLECTIBLES / Buttons & Pins" },
        { "ANT008000", "ANTIQUES & COLLECTIBLES / Care & Restoration" },
        { "ANT009000", "ANTIQUES & COLLECTIBLES / Transportation" },
        { "ANT010000", "ANTIQUES & COLLECTIBLES / Clocks & Watches" },
        { "ANT011000", "ANTIQUES & COLLECTIBLES / Coins, Currency & Medals" },
        { "ANT012000", "ANTIQUES & COLLECTIBLES / Comics" },
        { "ANT015000", "ANTIQUES & COLLECTIBLES / Dolls" },
        { "ANT016000", "ANTIQUES & COLLECTIBLES / Firearms & Weapons" },
...

Start fixing Warnings

The project must have Zero warnings most of which accumulated during conversion to V5

Get Child Module by type/name

/// <summary>
    /// Gets a child module of the specified TModule type optionally applying a filter.
    /// If module is not found then exception is thrown. Contrast with TryGet()
    /// </summary>
    TModule Get<TModule>(Func<TModule, bool> filter = null) where TModule : class, IModule;

    /// <summary>
    /// Tries to get a child module of the specified TModule type optionally applying a filter.
    /// If module is not found then returns null. Contrast with Get()
    /// </summary>
    TModule TryGet<TModule>(Func<TModule, bool> filter = null) where TModule : class, IModule;

    /// <summary>
    /// Gets a child module of the specified TModule type with the specified name.
    /// If module is not found then exception is thrown. Contrast with TryGet()
    /// </summary>
    TModule Get<TModule>(string name) where TModule : class, IModule;

    /// <summary>
    /// Tries to get a child module of the specified TModule type with the specified name.
    /// If module is not found then returns null. Contrast with Get()
    /// </summary>
    TModule TryGet<TModule>(string name) where TModule : class, IModule;

F#

Hi there :)

How is your F# support?

Why is MMF slower than byte[] Pile?

Hello,

I read in your benchmarks that MMF Pile is slower than byte[] Pile. Why is this?

Is it only for cases where you'll be converting from MMF format to CLR format? For example, if you have a MMF Pile Object representing a UTF16 String and you have to convert it to CLR string it is somehow worse when using pointer math to generate a CLR string? Versus copying it from a byte[]?

I would have thought that MMF Pile vs byte[] Pile should be 100% equivalent in perf.

UnauthorizedAccessException if multiple pile instances are used on one server

We experienced an issue with 2 separate pile instances are used in different appPools (IIS). Both applications are using different pile locations.
This seems to be caused by not unique names of the pile segments. The method MMFMemory.MakeFilename should generate unique filenames.
When we change the MMF_PREFIX manually to a random string there aren't any issues anymore.
That's why we think map-name of the memory-mapped-file with a static suffix+prefix causes this problem. A map-name should be used only once on the whole server.

Best regards,
Peter

Pile.Cache generating an avalanche of "Object reference not set to an instance of an object."

This happens on table.sweep and on table.fethExistingEntry - one the memory corrupts the error keeps generating on subsequent sweeps:

Sweep exception:

@20180120-022702|8216d599-fcbc-41ca-a4ff-45c4a68590fa|......zw02||
Critical|Data.Cache|LocalCache('MDBDataStore::GraphSystem').threadSpin().foreach.Sweep|0
Leaked exception while sweeping table 'GraphSystemService.Node': [System.NullReferenceException] Object reference not set to an instance of an object.'

  +-Exception 
  | Type      System.NullReferenceException
  | Source    NFX
  | Target    sweep
  | Message   Object reference not set to an instance of an object.
  | Stack     
     at NFX.ApplicationModel.Pile.LocalCacheTable`1._bucket.sweep()
     at NFX.ApplicationModel.Pile.LocalCacheTable`1.Sweep(Stopwatch timer, Int32 maxTimeMs)
     at NFX.ApplicationModel.Pile.LocalCache.threadSpin()

Access/Get Exception:

@20180120-022703|9602f37a-26cc-462b-95cd-d9fe04d05431|.......zw02||
Error|AppMgmt|Agni.Social.Graph.Server.GraphSystemService.GetNode|0
[System.NullReferenceException] Object reference not set to an instance of an object.

  +-Exception 
  | Type      System.NullReferenceException
  | Source    NFX
  | Target    fetchExistingEntry
  | Message   Object reference not set to an instance of an object.
  | Stack     
     at NFX.ApplicationModel.Pile.LocalCacheTable`1.fetchExistingEntry(_bucket bucket, TKey key, Int32 hashCode)
     at NFX.ApplicationModel.Pile.LocalCacheTable`1.Get(TKey key, Int32 ageSec)
     at NFX.ApplicationModel.Pile.CacheExtensions.FetchThrough[TKey,TResult](ICache cache, TKey key, String tblCache, ICacheParams caching, Func`2 fFetch, Func`3 fFilter)
     at Agni.Social.Graph.Server.GraphSystemService.DoGetNode(GDID gNode, ICacheParams cacheParams)
     at Agni.Social.Graph.Server.GraphSystemService.GetNode(GDID gNode)

Add NFX.Scripting

Add NFX.Scripting - ability to write C# script invocable from the outside
as facilitated by Runnable, Run attributes

Revise MemoryStream use cases - consider something like RecyclableMemoryStream or base it on Pile

MemoryStream allocates arrays in LOH, and it is inefficient.
Along the lines what MSFT did with RecyclablememoryStream, we could use Pile as memory manager or use RecyclableMemoryStream by MSFT as a drop-in replacement.

This deals with perf critical code where MemoryStream gets allocated/released many times.
See:

https://github.com/Microsoft/Microsoft.IO.RecyclableMemoryStream

The design needs to be re-evaluated in the light of: Pile, and ArrayPool/ Memory no to create code duplication

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.