grpc DEADLINE_EXCEDDED error when syncing about mirror HOT 9 CLOSED

Hey, thanks for reaching out!

I was initially thinking maybe your server's sync_workspace had a ton of files in it, so was taking longer than ~3 minutes to respond, e.g. there are 3 minutes between these two lines:

2017-05-01 11:02:45 INFO   Client has 1 paths
2017-05-01 11:05:45 SEVERE Error from incoming server stream
io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED

But there being 5 minutes between these two lines is really odd:

2017-05-01 11:02:46 INFO   Starting new session 1 for + /home/<myusername>/./sync_workspace
^C2017-05-01 11:07:56 INFO   Watchman not found, using WatchService instead

Is that control-c from you hitting control-c to kill the server? Which is fine, but it's very curious it just hung between the "Starting new session" and "watchman not found" ... let me poke around.

from mirror.

Can you run watchman get-sockname on your remote dev machine and see what it outputs?

from mirror.

Ah yeah the last 2 lines appeared after I hit control-C to try to kill the server.

When I first encountered this error there's only 1 directory in sync_workspace and mirror says "Client has 545 paths", but after I removed that directory from my laptop (directory and file are still there on the remote machine, essentially trying to re-sync), I still see the error above.

And when I run watchman get-sockname on my remote dev machine, it just hangs there with no output...weird.

from mirror.

And when I run watchman get-sockname on my remote dev machine, it just hangs there
with no output...weird.

Okay, this is what's causing the error then...mine returns right away with:

$ watchman get-sockname
{
    "version": "4.7.0",
    "sockname": "/usr/local/var/run/watchman/shaberma-state/sock"
}

I'll look around at why this might be happening; in theory watchman should at least error out right away, e.g. with a -1, so that mirror can continue and fallback on the Java 8 WatchService.

from mirror.

Hm, sounds like there was a known issue when installing/upgrading watchman via brew, but given this happens on your remote machine, I assume that's Linux:

facebook/watchman#371

Maybe do a ps aux | grep watchman, see if there is a watchman daemon running, if so do a killall watchman to kill it, and then see if watchman get-sockname works.

from mirror.

There are indeed several processes of watchman get-sockname hanging there. I killed all of them and then watchman get-sockname worked, as well as mirror.

Thanks a lot for helping! Btw, I'm definitely interested in making this tool better, but I feel I'm not very familiar with network applications, etc. Where could I start to take a look?

from mirror.

Great, I'm glad that was it, because otherwise I would have been out of ideas.

Awesome to hear you'd like to help out, that would be really great.

In terms of where to start, the guts of the "am I out of sync?" logic is UpdateTree and UpdateTreeDiff and tests. The network stuff is mostly in MirrorClient/MirrorServer.

Here is a presentation I did for a local user's group a few months back, it would also be a good intro:

https://docs.google.com/presentation/d/1PQ1LH13rxTQhiyy1DVV4AfJ7-i-WIZB0je13MK2NAW8/edit#slide=id.p

from mirror.

FWIW, if you wanted to try and write some code that detected what you ran in to, e.g. somewhere in WatchmanChannelImpl.createIfAvailable try and run like ps aux | grep watchman (not sure the best way to do that) and if there are N > 1 watchmans around, do either a log.warn/log.error/fail to let the user know, that would be pretty handy.

(I have no idea how many people have run into the watchman bug that you did; you might be the only one, or you might have been the only one to actually file an issue about it. :-))

from mirror.

Huh, looks like this might be easy to drop in:

https://github.com/profesorfalken/jProcesses

from mirror.