yurigor / deepdash Goto Github PK

<--- Last few GCs --->

[10820:00000178C3D42B80] 15484 ms: Mark-sweep 1959.0 (2095.8) -> 1959.0 (2095.8) MB, 470.1 / 0.0 ms (average mu = 0.076, current mu = 0.000) allocation failure scavenge might not succeed
[10820:00000178C3D42B80] 15786 ms: Mark-sweep 1959.0 (2095.8) -> 1959.0 (2095.8) MB, 302.8 / 0.0 ms (average mu = 0.044, current mu = 0.000) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x00789f1408d1
0: builtin exit frame: concat(this=0x036ca3d71271 <JSArray[6552]>,0x019ed21f20a9 <JSArray[1]>,0x019ed21fcb59 <JSArray[0]>,0x036ca3d71271 <JSArray[6552]>)

1: getOwnChildren(aka getOwnChildren) [000002FB79D26571] [P:\DEV\CC\CC\node_modules\deepdash\private\getIterate.js:~211] [pc=0000019BD302EF67](this=0x01909d2804b1 <undefined>,0x02fb79d0bd51 <Object map = 000002306D5E2C19>,0x007db7dff45...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Just forgot it

I was thinking about writing a library like this myself :) I'm the author of object-scan and was thinking it might be a great fit to improve key matching and efficiency of this library (I'm not a big fan of regex matching).

Take a look and let me know what you think! It would certainly be more in line with the existing lodash selector syntax.

Would be very happy to help with any questions! Cheers, L~

Thanks for enabling synk to protect your deps, but I'm guessing it should rather be added as a devDependency, i.e. since it's never used at runtime all your package consumers shouldn't be forced to download it into their node_modules.

https://github.com/YuriGor/deepdash/blob/master/package.json#L85

I've encountered performance issues in my code (which uses filterDeep) and after some research using Chrome DevTools performance tab, it seems that the getIterate.js file causes performance issues.

Here are 2 images displaying the results of the performance test recordings:

I understand that there must be a deep scan to search for a path which might be relevant for the user (was specified in childrenPath).
but maybe one of these suggestions can help:

1. Add a maxScanDepth (number) option, which will limit the depth of the search for cases where the user knows his deep object's structure and doesn't want to scan all the paths in the scanned object.
2. Add a isExactChildrenPath (boolean) for cases where only the paths specified by the user in the childrenPath parameter are relevant and there is no need to search other paths (Ofcourse he must supply the full path).
3. As mentioned in this stackoverflow question and was tested here
   it is possible to use spread instead of concat for better performance.
   like this:
   currentParents = [...parents, currentObj];
   instead of currentParents = parents.concat(currentObj);
   or
   childPath = [...path, childKey];
   instead of childPath = (path || []).concat([childKey]);

https://github.com/lodash/lodash/issues/4738
Once (if) it will be fixed - I need to bump the deps in Deepdash

Your require syntax seems weird. Is there a reason that lodash is not a peer dependency?

This discussion #16 led me to the understanding of importance 'tree' mode in these methods, so let it be.

Because Lodash's omit doesn't

Is it possible to use a function in omitDeep? I would like to exclude an item and its children from an array but only if the id of an item matches. Please correct me if this isn't the correct function for this but with filterDeep I got an infinite loop (idk why).

This is what I expect:

omitDeep(
      items,
      (value) => {
          return value.id === itemId
      },
      {
          childrenPath: ['children'],
      }
 )

PS: Thanks for the great lib. This is a huge time saver for me in my current project.

Hey @YuriGor nice work! Any chance we could export separate modules? Like lodash has:

import _omit from 'lodash/omit'

for smaller bundle size on the browser?

selectDeep - like findDeep, but returns an array of all the matches, not the first one only.
opposite to filterDeep this method returns just a flat array of chosen meta-values, without preserving the structure of the original data source object.
also shorthands similar to find**:
selectValuesDeep
selectPathsDeep

Same like in filterDeep we have cloneDeep option to let the user specify his own cloning method,
we need also to review all the deepdash methods where flat _.clone method used and also let user replace it.

People ask for it.
Consider Lodash types as a starting point

Now in the tree mode deepdash walks over all the nested fields and checks each path of it matches given 'children' criteria, because children path may be configured as a regexp, and it's impossible to say beforehand which nested value is a children collection, until we build and tested it's path.

Regexp support in the 'children' option should be dropped as an overkill. If only constant paths will be allowed, we will be able to go and get children collection directly, without bruteforcing all the possible variants.

Based on:
https://lodash.com/docs/4.17.15#pickBy
https://lodash.com/docs/4.17.15#omitBy

Thanx!

• Consider to use lerna instead of manual package management.
• check if something can be done for solving npms issue.

I received good feedback, I agree with.
Iteratee should have less arguments, now it's hard to remember right order, also code looks unreadable when last argument needed.
value, key, may be source collection will be enough, same as in Lodash.
Rest of arguments such as path, depth, parent* should be passed as fields of single object context or properties.
It will be breaking change, so I need to think twice.

It's a common case when object has regular structure with predictable children array name.
If it's defined beforehand, we can process nodes rather than separate fields.
This way filtering becomes much easier.
I think I need to implement eachTree and filterTree methods, as a higher level alternatives for eachDeep and filterDeep.
This methods will accept childrenField argument, which can be string path, regex or array of any of this.
Default value will be children

Now rootIsChildren option, if set to true, supposes source object is a top-level collection of children.
But no options exist to declare source object as a single root tree item, to make iterator pass it to the iteratee/predicate.
For this case, we have to wrap such an object into an array before iterating, it looks not so good.

I used deep map to integrate some object node with extra properties and deepMap works perfectly returning new value where I want. The structure of the mapped object remains the same.

Let's suppose I need to transform array of strings node in a leaf with those string concatenated altering the structure:

{ arr: ['a', 'b', { c: ['y', 'w', 'z'] }, 'd', 'e'] }

{ arr: ['a', 'b', { c: "ywz" }, 'd', 'e'] }

I was no able to do that with deepMap. Is it possible?

In the following example I pass an array of paths to pickDeep and after execution it becomes an array of array! have a look to the console

https://codepen.io/masciugo/pen/MNjgra?editors=1111

As found in #62 - it makes sense to support childrenPath option in condenseDeep method. This way we can improve the performance of filterDeep in tree mode because condenseDeep will skip not children props.

`const tree = {
id: 1,
rootKey: 'rootVal'
};

let cleaned = _.filterDeep(tree, (k, v) => k === 'rootKey' || v === 'rootVal', {includeRoot: true});

console.log(cleaned);
// get empty object
// expected result: { rootKey: 'rootVal' }`

Now all the methods mixed into Lodash with default chain=true.
Need to clarify the difference, adjust as needed and write corresponding tests.

Hi,
I'm having an issue when building and running on IE11, a project using deepdash 4.2.10.
The error comes from getHasChildren.js which use an arrow function and cause a 'SCRIPT1002' error in Internet Explorer 11.

Can you fix this issue ?
Thank you !

lodash methods has / hasIn return true for the path pointing to the empty array slot (in sparse array)
In case of deep works considering empty slots is pretty important, so deepdash needs its own alternative implementation for this methods.
Let it be _.exists

I feel the possibility of incorrect path generation.

Just filter children by empty 'children' field before callback.

Maybe just index or buildIndex?

any possible to provide such feature?

Could just be how I've interpreted it, but:

_.filterDeep(arr, Boolean, { leavesOnly: true, onFalse: { keepIfEmpty: true } })

still removes empty objects

filterDeep convert type function (for example: String) to empty object.
You can check this pen: https://codepen.io/gabortorma/pen/OJyBWXP
You can see on the console. omitByDeep keep the function, filterDeep convert it to empty object.

Hi

I found some interesting things in mapValuesDeep funtcion (and maybe on the other map function...):
If a change an array value to the object, the array values remaining in the object with index key.

You can check it on my pen:
https://codepen.io/gabortorma/pen/zYvgXaG

Thanx!
Gábor

selectDeep - like findDeep, but returns an array of all the matches, not the first one only.
opposite to filterDeep this method returns just a flat array of chosen meta-values, without preserving the structure of the original data source object.
also shorthands similar to find**:
selectValuesDeep
selectPathsDeep

mapDeep - returns an array of deep values processed by specified iteratee
opposite to mapValuesDeep(gonna be renamed) this method also returns just a flat array of chosen values, without preserving the structure of the original data source object.

Iterate deeply over the object and return first value\path iteratee will agree with.

Trying to break out of loop if key==2 is matched, but it does not work. What would be the right way to break out of loop

_.eachDeep({ id: 1, children: [ {id: 2, children: [ { id: 3, children: []}]}]},(k, v, parent, context) => {
      if(k == 2) {
        return false;
      }
      console.log(k)

    });

Now, results are returned from top to bottom,
so I need to add option order and if 'bottom-to-top' was specified - I will use the 'callbackAfterIterate' option internally.

Ok, it's time to do this:

• avoid recursion - do everything in the plain loop instead, by pushing and popping current context into some array(=stack) depending on the iteration depth.
• there is an internal creation of function heavily coupled with surrounding closure - that's also subject of future refactoring.
• make path format array by default
  (oh noes, breaking changes, v5 to v6 jump without minor versions in between, what a shame)
• make code readable to let the community contribute easily if anybody would like to
• Don't forget to measure performance before and after.

How can I deeply transform keys with deepdash?

Looks like some methods ignore it, need to cover such cases by tests.
Also need tests for the case when in tree mode children is not an array but an object

Keep old option as working alias, to make no breaking changes

deepFilter will 'trim' the objects it filters removing parent keys that don't have children matching your predicate function. I wanted to propose the option to retain the full objects that have children that match your predicate function. So for example

const myList = [ { a: {b: 'hello'}, c: 'world' } ];
const filtered = deepFilter(myList, (value) => { return `${value}`.includes('world') };
// filtered is [ { a: {b: 'hello'}, c: 'world' } ]
// rather than [ {c: 'world' } ];

subj

I created a simple demo project and included the dist bundle within it to show my problem. I am using deepdash very much as described in the docs, like this:

import _ from 'lodash';
import deepdash from 'deepdash';

deepdash(_);

(reference)

However, webpack seems to have some kind of problem with this, since it creates this line in the bundle:

deepdash__WEBPACK_IMPORTED_MODULE_1___default()(lodash__WEBPACK_IMPORTED_MODULE_0___default.a);

(reference)

which then leads to the following error in the browser:

Now I have to put a disclaimer here: I am not entirely sure this is either a deepdash or webpack problem, so I am happy to report it over at webpack if you feel deepdash is doing everything correctly. I am also not sure that I am not just a bit stupid and screw up something myself - in that case, sorry!

I hope the minimal project I set up helps debugging a bit. If I can do anything to help debug, I am happy to do so.

P.S.: This works absolutely fine in node.js where I run all my tests with mocha and such, but webpack just won't take it.

E.g. omiting field named "__typename" from an nested object?

[default] \node_modules\deepdash-es\filterDeep.d.ts:12:5
Type expected.
[default] Checking finished with 1 errors

This seems to be related to the extra pipe before void
) =>
| void
| boolean
| {
skipChildren: boolean;
cloneDeep: boolean;
keepIfEmpty: boolean;
},

There are no errors when changed to
) =>
void
| boolean
| {
skipChildren: boolean;
cloneDeep: boolean;
keepIfEmpty: boolean;
},

And then post new examples into #16

We are seeing code which are incompatible with "use strict"

ERROR in js/vendors~ui.8f578b013cec78bc290e.js from UglifyJs
In strict mode code, functions can only be declared at top level or immediately within another function. [./node_modules/deepdash/private/getIterate.js:101,8][js/vendors~ui.8f578b013cec78bc290e.js:186542,8]

at deepdash/private/getIterate.js:101

      !options['break'] &&
      res !== false &&
      !isCircular &&
      _.isObject(value)
    ) {
      if (options.childrenPath !== undefined) {
        function forChildren(children, cp, scp) { // DECLARING A FUNCTION WITHIN AN EXPRESSION
          if (children && _.isObject(children)) {
            _.forOwn(children, function(childValue, childKey) {
              var childPath = (path || []).concat( (cp || []), [childKey]);
              var strChildPath =
                options.pathFormat == 'array'
                  ? pathToString([childKey], strPath || '', scp || '')
                  : undefined;
              iterate({
                value: childValue,
                callback: callback,