Migrate VFS API support into VFSForGit repo about libprojfs HOT 10 CLOSED

Will take a look tomorrow, @kivikakk! Sorry not to get to this today!

from libprojfs.

This is fantastic; thank you for pushing the envelope on this @kivikakk!

And I was 100% wrong about my dim, fuzzy recollections of putting C in C#; as you said, it's not possible. (There's an unsafe keyword and matching .csproj file declaration, but you were totally correct, that doesn't let you embed unsafe code the way I'd mis-remembered.)

So I guess that leads me to think we have a range of options here, each with pros and cons, so I made up this chart -- please amend and correct as you think best!

Having written this out, and thought about your point that indeed there's no C-in-C# (unlike C-in-C++), I confess I'm leaning slightly toward the rewrite option -- but only after we fine-tune the API some more. (For instance, we can likely eliminate the CompleteCommand stuff if we don't have any async operations, the way the Windows projectedfslib does.) And I may well be overlooking some important advantage of one of the other options!!

What say you, having done this first pass of work? Thoughts, comments, criticism?

from libprojfs.

See github/VFSForGit#2 for a spike of the rewrite option.

from libprojfs.

Upstream PR is at microsoft/VFSForGit#848.

from libprojfs.

A question to ponder before our chat tomorrow: what should we do with t[57]*.t? It seems like, for the most part, these should all be removed from libprojfs, and all relevant ones reimplemented against PrjFSLib.Linux.Managed (which is absorbing the API).

from libprojfs.

Indeed -- to be honest, I think we should just remove them and all the --enable-vfs-api tooling. (I'm very slightly verklempt to see it go, but mostly delighted; this is far better! :-) And I have a pretty good working knowledge of how I wired that stuff in, so pulling it out should be something I can tackle as a quick item.

As for re-implementing tests (or more generally, implementing new more appropriate ones), I think we're on our own to some degree. The Mac kext and user-space library are getting their own unit tests in microsoft/VFSForGit#561 and microsoft/VFSForGit#562, but those either don't exist yet or don't help us on Linux. What might be a better framework to study is the stuff in GVFS/GVFS.NativeTests which is used, IIUC, to perform native Windows API tests within the larger GVFS functional (integration?) unit tests. But again, it's not an exact parallel, and we likely don't need or want the C++. Maybe even better is the mock filesystem framework in GVFS/GVFS.UnitTests/Mock/FileSystem.

I suppose the interesting question is, what to we actually want to test? I would think things like the process name lookup (GetProcCmdline()) and error code translation (ConvertErrnoToResult()) and of course the actual event callbacks, especially the file hydration and use of WriteFileContents().

Does it then make sense to build a dedicated set of C# mock filesystem and handler tests just for ProjFS.Linux? Or integrate into the larger framework? I think we may need advice from our Microsoft colleagues on this, to be honest.

from libprojfs.

Indeed -- to be honest, I think we should just remove them and all the --enable-vfs-api tooling. (I'm very slightly verklempt to see it go, but mostly delighted; this is far better! :-)

Yes, I was hesitant to suggest it if only because I know how much work went into it! But I do think it's moreorless the only thing that makes sense. We'll implement new (appropriate) tests against the C# stuff.

Now! I was on my way to write that email to John et al. when I dug around and found GVFS.UnitTests.Platform.Mac. I copied it across to a .Linux variant and fixed it up — you can see this at github/VFSForGit@2a97cfe...vfs-api-tests. (I merged my VFS API work and your Linux GVFS initial build work at github/VFSForGit@2a97cfe (no conflicts!). Hopefully we can get them both actually merged upstream soon and then this compare URL will be a lot more sensible.)

Moreover, the tests actually run and pass on Linux:

$ dotnet ../BuildOutput/GVFS.UnitTests/bin/x64/Debug/netcoreapp2.1/linux-x64/GVFS.UnitTests.dll --test=GVFS.UnitTests.Platform.Linux.LinuxFileSystemVirtualizerTests
NUnitLite 3.10.1 (.NET Standard 2.0)
Copyright (c) 2018 Charlie Poole, Rob Prouse

Runtime Environment
   OS Version: Linux 4.9.0-7-amd64 #1 SMP Debian 4.9.110-3+deb9u2 (2018-08-13)
  CLR Version: 4.0.30319.42000

Test Files
    /home/kivikakk/BuildOutput/GVFS.UnitTests/bin/x64/Debug/netcoreapp2.1/linux-x64/GVFS.UnitTests.dll

Test Filters
    Test: GVFS.UnitTests.Platform.Linux.LinuxFileSystemVirtualizerTests

Run Settings
    Number of Test Workers: 4
    Work Directory: /home/kivikakk/VFSForGit
    Internal Trace: Off

Test Run Summary
  Overall result: Passed
  Test Count: 11, Passed: 11, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2019-02-27 07:10:42Z
    End time: 2019-02-27 07:10:42Z
    Duration: 0.288 seconds

Results (nunit3) saved as /home/kivikakk/VFSForGit/TestResult.xml
Completed test pass in 00:00:00.4906198

$

So this is an excellent start! I don't actually have anything for an email to MS yet — I can use this as prior art to cook up test suites for other bits, as it turns out to be simple (and not awful like I remember).

from libprojfs.

I suppose the interesting question is, what to we actually want to test? I would think things like the process name lookup (GetProcCmdline()) and error code translation (ConvertErrnoToResult()) and of course the actual event callbacks, especially the file hydration and use of WriteFileContents().

Does it then make sense to build a dedicated set of C# mock filesystem and handler tests just for ProjFS.Linux? Or integrate into the larger framework? I think we may need advice from our Microsoft colleagues on this, to be honest.

Finishing up my workday, but this does still need to be addressed. I plan to dig more tomorrow to see what best I can come up with, and then maybe email to inquire about what they think the best route might be — or open an issue, to keep the discussion public!

from libprojfs.

This is fantastic, as usual, @kivikakk! I'm sorry I missed MacFileSystemVirtualizerTests.cs on my first spelunking for prior art; it looks precisely right for this purpose!

It's fantastic that this actually runs and passes. ✅ 😃

from libprojfs.

Closed by microsoft/VFSForGit#848 and #53.

from libprojfs.