mhallin / notify-forwarder Goto Github PK

This is great for using something like this with the intent of one client where the server sends notification to one NFS client in one to one, many to one, or one to many configs, but in a larger environments, the NFS servers will be accommodating many hosts in a many to many arrangement. If there is more than one NFS server, I'm not sure, but does that mean there needs to be a fully meshed notify-forwarder mesh from where each NFS server needs to notify each NFS client. Managing a large number of notify-forwarder would get unwieldy and the problem of updating all the new servers. We support min 50 to easily over NFS servers that are in a continual being upgraded and changed. There is support for multiple paths from a single remote server,Feature request is to possibly auto identify all clients during client registration with NFS server, and notify all possible clients with ATTRIB event. On the flip side, I wonder with NFS server redundancy or load balancing (possibly balancer, DNS round robin, heartbeats, automounting or any number of other tricks where NFS Server IP is not necessarily fixed), does "watch -c " support hosts and/or DNS hostnames?

Figure this is probably an apple issue and perhaps a bit of a shot in the dark.

I'm using notify-forwarder via https://github.com/mhallin/vagrant-notify-forwarder

My specific usecase involves a linux guest (ubuntu 14.04) though I don't think the guest matters all that much beyond "it's linux".

The host:

$ uname -a
Darwin Macbook-Pro 17.4.0 Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64 x86_64 i386 MacBookPro11,4 Darwin
$ system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: macOS 10.13.3 (17D47)
      Kernel Version: Darwin 17.4.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: asottile-mbp142
      User Name: Anthony Sottile (asottile)
      Secure Virtual Memory: Enabled
      System Integrity Protection: Enabled
      Time since boot: 1 day 5:25

The "large number of files" in question involves removing a shared node_modules folder from either the host or the guest (it is much more likely to reproduce when deleted on the host, though it has also reproduced while deleting on the guest). (rm -rf node_modules)

The kernel panic in question ends with:

*** Panic Report ***
panic(cpu 7 caller 0xffffff800b6869f1): "vnode_dropiocount(0xffffff80607863e0): v_iocount < 1"@/BuildRoot/Library/Caches/com.apple.xbs/Sources/xnu/xnu-4570.41.2/bsd/vfs/vfs_subr.c:4782

The Vagrantfile in question is part of a closed source application though it's pretty simple. Let me know if you'd like additional information for reproducing this and I can attempt to anonymize it.

I would love to see a way to ignore files, similar to inotifywait's --exclude flag.

• Avoids hitting the max watch limit
• Reduces overall RAM usage, as inotifywait uses 1kB of RAM per file watched on a 64bit system.

Either that or perhaps a way to pipe inotifywait output directly into the forwarder's STDIN.

I set up a vagrant vm with your vagrant notify-forwarder plugin, host is a Mac OS X (10.10.5), guest is debian jessie. In the vm there's an app running, waiting for some fs events by calling inotifywait => https://github.com/synrc/fs/blob/master/src/sys/inotifywait.erl#L10
If I open/edit some file on the host nothing happens (should compile something).
I played a bit, started inotifywait /vagrant and edited, deleted, created files, but all I can see is

vagrant@db24-phoenix /vagrant ❯❯❯ while true; do inotifywait /vagrant -r; done                                                               
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/vagrant/ ATTRIB mix.exs
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/vagrant/ ATTRIB mix.exs
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/vagrant/ ATTRIB mix.lock
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/vagrant/script/ ATTRIB .vagrant_provision.sh.swp
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/vagrant/.git/ ATTRIB FETCH_HEAD
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/vagrant/.git/ ATTRIB FETCH_HEAD
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.

If I do something inside the vm in /vagrant inotifywait shows

Watches established.
/vagrant/ ATTRIB,ISDIR test
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/vagrant/ CREATE foo
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/vagrant/ DELETE foo
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/vagrant/ ATTRIB foo

So what's happening here? Is notify-forwarder not able to forward the correct event types?

There's another app running (https://github.com/brunch/brunch), which responds to events from notify-forwarder, but I did not dig deeper since it's doing its job ;)

The following error occurs when there's a dangling symlink in the tree being watched:

terminate called after throwing an instance of 'std::logic_error'
what(): Could not stat file of unknown type
Aborted

This PR fixes it for me:
#9

The repo seems quite inactive so maybe this issue/PR will help someone else :)

Microsoft recently added inotify support to their Windows Subsystem for Linux (see: https://blogs.msdn.microsoft.com/commandline/2016/10/07/wsl-adds-inotify-filesystem-change-notification-support/).

Happy to report running notify-forwarder watch via bash on the windows host and notify-forwarder receive on the linux guest picks up file & directory changes made on the windows host (confirmed by running while true; do inotifywait /vagrant/gitrepos -r; done on the guest).

Only caveats: I had to run notify-forwarder as root on the guest and DELETE events are not picked up (only creation/modification, albeit shown as ATTRIB events by inotifywait).

So you can add Windows 10 to the supported platforms list :)

Notifier-forwarder isn't loading the complete directory tree, it will load the first 6 paths at start up and only watch a directory if its created..
example:

/media/NAS/video will be seen but /media/NAS/video/movies
yet any in for example
/media/NAS/Audio will go one level deeper and see /media/nas/audio/music and /media/nas/audio/podcasts and such

Same as this issue on your related project:
mhallin/vagrant-notify-forwarder#5

terminate called after throwing an instance of 'std::runtime_error'
what(): Could not open directory
Aborted

I tried the proposed fix in that issue report but still get the error after setting /proc/sys/fs/inotify/max_user_watches = 1410065407 and the number of files in the watched folder is only 1438187.

Please let me know if there is any other information that could be useful.