manofstick / cistern.valuelinq Goto Github PK

There are circumstances such as in Fork where pull can't be applied, so could control this through splitting off the push functionality. But this would mean I'd have to have a limited functionality container, the equivalent of ValueEnumerable. Not totally out of the question, I was doing something similar with OrderBy before changing tack there, so have the t4 template in git history.

Getting there... A few flavours of each, but this is all that remains...

• Append
• AsEnumerable
• DefaultIfEmpty
• Distinct
• Except
• GroupBy
• GroupJoin
• Intersect
• Join
• LongCount
• Prepend
• SelectMany
• SequenceEqual
• SkipLast
• TakeLast
• ToLookup
• Union
• Zip

Investigate Integer overflow on Average; as it occurs in here, but not on System.Linq, so maybe they are converting to double for summation?

Now that I have implemented overflow handling, SIMD version should be default for Sum.

Fallback to logic from ToListViaArrayPool, which will need to have an adjusted start index (i.e. prior to which the stack based collected data can be dumped)

Can't do this directly because it give a different result on floating point numbers, and doesn't handle overflow for integers, so should add an enum (something like the options "Compatibility", "SIMD fast", "SIMD safe" - handling overflow) as an argument

• Github main page

• Architectural overview

• Push/Pull

• Optimizations pipeline

• SIMD

• Benchmarks summary

• Additional functionality

• Fork

As per https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/proposals/csharp-9.0/nullable-reference-types-specification

@stephentoub, @MadsTorgersen

Probably not that you would be that interested, but personally I think this project has some interesting ideas (possibly none that you would use in anger! But...) but given that I have tried to contact people whom I would of thought might be interested, yet not managed to spark any interest I thought I might as well tag you guys...

(Who am I? .net core's System.Lazy implementation is mine, and a couple of other minor things)

Anyway, Cistern.ValueLinq was an attempt to bridge some of my previous ideas from Cistern.Linq with a Value Based Linq such as the one done a few years back by Kevin Montrose. Unlike that implementation, this version is quite trim (IL wise), relying on a novel stack folding to generate enumerators so that the underlying enumerable doesn't necessarily need to leak to the outer generic signature (which means that it's not needed to be provided for functions where the type isn't carried with function arguments).

It also provides dual access to the underlying enumerable - one for pull - standard MoveNext/Current semantics, and one for push, where the control of the iteration is from the enumerable object down.

Another "feature" is a quite novel use (!) of generics to allow pseudo-interface like behaviour, such that stream optimizations, such as those in the standard System.Linq implementation, could be implemented on top of a stack of value-based objects.

Also some extensions to the standard Linq surface were also added such that "value-based lamdas" could be used. When used these could (under great strain for the JIT I'm sure!!) achieve basically "handcoded" performance (i.e. that zero-cost - ignoring JIT/code bloat - abstraction goal).

(there are other bits and pieces there as well, such as SIMD for math functions where the same results as System.Linq could be achieved, it handling Memory (and hence Span) containers, some extensions like Fork which would evaluate further on the pipeline without multiple iterations on the source, ...)

Anyway, given that I appear to be the only developer these days who writes long running, computationally intensive code (and given that the "value-based lamdas" don't have pretty syntatic sugar) I think this whole thing, for all it's ... umm... glory (?), is basically dead in the water, but maybe it could live on in:

• testing generics; bloat,performance
• testing general code generation
• testing inlining
• some fun exploring a strange codebase :-)
• or even cause you to be inspired as to a valuable use of "value-based Lambdas"!!

(The test suite in the project was lifted from a oldish version of the System.Linq test suite from .net core; there are a couple of failures which I didn't get back to, and you get a whole bunch of obsolete warnings for the System.Linq APIs that I hadn't finished porting)

Anyway, if you are interested I'm more than happy to explain, expand on info, provide more architectural info, etc. But otherwise I'm just going to let this fade away...

Have a good day! Thanks for reading (if you got this far :-)

Can then handle Skip/Take whilst building stack

Hi anyone!

Looking for anyone else who wants to:

• help complete the whole surface area (commenting out functions as I do them here)
• write more benchmarks
• expand testing to ensure both up and down pipes are covered
• think of any further optimizations
• offer better naming
• organise/cleanup project
• whatever else

@reegeek Iooking at you! :-) I think my underlying architecture here offers greater benefits, but obviously I'm not as far down the path as you.

• Set construction from IEnumerable should check INode
• Value Types
• DefaultComparer
• Add optional ArrayPool