Proposal: array function to return a unique list of values about jsonata HOT 8 CLOSED

Is anyone progressing this proposal?

from jsonata.

I've been thinking for a while about how to best implement this. For starters, I need to upgrade the equality operator to work for non-atomic types. Currently [1,2] = [1,2] returns false which is clearly wrong. This needs fixing anyway. The bigger challenge is eliminating duplicate values in an efficient way, without comparing every value with every other (n-squared efficiency - terrible!). Hash sets normally come to the rescue here, but I would need to come up with a hash function that copes with all data types including hierarchies of maps and arrays.

from jsonata.

This seems pretty ambitious to me without external dependencies. There are whole libraries out there to handle things like equality testing. Getting into implementing uniqueness as well and you are opening to door to reimplementing something like lodash. I'm not saying it is necessarily a bad idea. But I would be cautious.

Also note that we've discussed in the past the idea of having a "core" language and "extensions". If structured that way, the core could be dependency free. But an extension might bring something like lodash in and be able to provide a lot of additional functionality with very little implementation cost.

from jsonata.

Here is my solutions to the problem using current functions. One thing to note is since I do sort you do lose order.

(
    $unique := function($v, $i, $a){$i != 0 ? (( $v = $a[$i-1] ) ? "": $v) : $v };
    $notEmpty := function($v){$v  !=""};
	
    {
	"productIDs": **.ProductID,
        "uniqProductIDs" :$filter($map($sort( **.ProductID),$unique),$notEmpty)
        
    }
)

from jsonata.

This probably only works for first-order objects as it relies on the in operator, but it's succinct and clear:

(
	$dupReduce := function($acc, $el) {
    	$el in $acc ? $acc : $append($acc,$el)
    };

	$removeDups := function ($a) {
    	$reduce($a,$dupReduce)
    };

	$removeDups(input)

)

The only part you'd need to change for more complex comparisons would be the test at the start of the $dupReduce higher-order function.

Alternately you wait for @andrew-coleman to extend in to do more complex comparisons.

from jsonata.

$distinct(array) +1

that would be good

from jsonata.

I found @shrickus suggestion useful if the array is a selection of strings, for example a query:

devices.*.itemSelected

Then you can use the strings temporarily as object keys to eliminate duplicates, then return the keys:

$keys(devices.*{itemSelected:""})

from jsonata.

If the result contains only one object, $keys returns a string so you need to include all in brackets to always return an array.

[$keys(devices.*{itemSelected:""})]

from jsonata.