Sync is slow for large amount of content about unicorn HOT 3 CLOSED

And while I would question the need to check in so much into Unicorn there's several valid reasons for this:

1. For content migrations there's often a need to script global fixes, so it is nice to have this in source to run the scripts to fix a large amount of content rather than an environment somewhere you'd need to backup/restore.
2. Makes environment promotion part of CI, which I guess you could do it other ways but is nice to know pre-production that you don't have multiple ways of promoting environments.

Talking to some colleagues it was brought up that Unicorn does some consistency/dependency checks and it might be possible to do a "fast" option, so if we know that there does not need to be validation we can turn it off. Thinking out loud at this point as I wait for the content to update on my local machine.

from unicorn.

1gb, that's a lot :)

I would guess there are two bottlenecks:

1. The local I/O subsystem. There are a lot of disk reads to sync
2. The database. Sitecore has to read every item to compare against

Reading a single item file is pretty fast. But 1gb of anything small will
be less so.

Unicorn completely supports plugging an arbitrary serialization provider.
It mimics but is not the stock inflexible sitecore static class.

Alex Shyba once tweeted about extreme deserialization performance using an
experimental write-only SQL bulk update. I don't know the details but you
might bug him if you're doing massive stuff like this with no need to
compare values just overwrite.
On Wed, Dec 10, 2014 at 8:53 AM Geoffrey Smith [email protected]
wrote:

And while I would question the need to check in so much into Unicorn
there's several valid reasons for this:

For content migrations there's often a need to script global fixes, so
it is nice to have this in source to run the scripts to fix a large amount
of content rather than an environment somewhere you'd need to
backup/restore.
2.

Makes environment promotion part of CI, which I guess you could do it
other ways but is nice to know pre-production that you don't have multiple
ways of promoting environments.

Talking to some colleagues it was brought up that Unicorn does some
consistency/dependency checks and it might be possible to do a "fast"
option, so if we know that there does not need to be validation we can turn
it off. Thinking out loud at this point as I wait for the content to update
on my local machine.

—
Reply to this email directly or view it on GitHub
#35 (comment).

from unicorn.

One further note: though it's probably non-trivial on this huge amount of data, some previous performance benchmarking I did indicated that the actual comparison of field values is quite cheap compared to reading the item from Sitecore API and from the .item file - so your idea of a fast predicate may help slightly but won't be miraculous - unless I'm totally wrong.

from unicorn.