Added feature to poll files for help with virtual machines/docker about pytest-watch HOT 8 CLOSED

FYI Released pytest-watch 3.0.0 today, which includes --poll.

Be sure to pip install --upgrade pytest-watch 😃

from pytest-watch.

Hi, @carsongee. Perhaps.

Actually, do you think this is a watchdog bug? If the files are indeed changing, watchdog should pick it up regardless if it's running in a container or not. Or am I misunderstanding?

from pytest-watch.

I just checked their repo, and found a very similar issue: gorakhargosh/watchdog#283. So it isn't really designed to handle events on remote file systems like we have with mounted container volumes since it is just using kernel events, so if you run watchdog in the container, since the kernel in the container isn't firing the event, the one in the host OS is, the watchdog can't react to it.

from pytest-watch.

Good find!

Looks like watchdog provides a decent workaround with PollingObserver.

It'd be great it watchdog was able to detect whether inotify was reliable or not, so you wouldn't need an application-level concern. This is, in my opinion, a leaky abstraction that watchdog could potentially plug up. (Interested in opening a new issue asking for them to look into this? Docker is indeed growing in popularity.)

We could look at other libraries to see if they work inside of Docker out-of-the-box so pytest-watch users don't have to waste any time with this surprise. In the short-term, adding an option to use polling should be ok. I'd happily pull in a PR if you wanted to add a --poll option.

from pytest-watch.

I added the PR. As to reporting it upstream, are you thinking the issue would be to have the standard observer figure out if it needs to use polling? That would be pretty cool. I don't think there is any other way to address it in watchdog since the nature of remote filesystems is not to generate kernel events, that would have to be a fix in Mac and Linux kernels I think, but I'm a little out of my depth on this.

from pytest-watch.

are you thinking the issue would be to have the standard observer figure out if it needs to use polling

Exactly. Then all tools that use watchdog would just work under this scenario instead of having to implement their own custom solutions.

I believe it's possible. For instance, one (bad) way would be to have watchdog create a .watchdog file and delete it right away in the directory it's observing. If no kernel events happen, turn on polling.

from pytest-watch.

gorakhargosh/watchdog#316

from pytest-watch.

@carsongee Thank you!

from pytest-watch.