falk-werner / webfuse Goto Github PK
View Code? Open in Web Editor NEWwebsocket filesystem based on libfuse
License: GNU Lesser General Public License v3.0
websocket filesystem based on libfuse
License: GNU Lesser General Public License v3.0
The C provider API does not support authentication yet. Since adapter (server) is aware of authentication, C provider should support this to allow some real world usage.
Describe some use cases which covers the goals of the project.
Problem
Only one client can be connected simulationously.
Solution
More than one client should be able to connect.
Problem to solve
inodes must be manages somehow.
Currently, application is running in websockets server mode, in order to allow clients to connect remotely. This makes sence in cases where a human user will connect to a remote device an provides some files to the device, e.g. via web browser.
However, for IoT devices it might be a use case to connect to a remote server to request files.
First of all, I'd like to thank you for this wonderful project! Can't wait to start building using your software!
Unfortunately, I ran into this issue while trying to build it. I installed all the required prerequisites, but still seeing the issue given below.
OS: Ubuntu 20.04
elza@aspire3:~/projects/webfuse$ cmake -B build
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1")
-- Checking for module 'fuse3'
-- Found fuse3, version 3.9.0
-- Checking for module 'libwebsockets'
-- Found libwebsockets, version 3.2.0
-- Checking for module 'gtest_main'
-- Found gtest_main, version 1.10.0
-- Checking for module 'gmock'
-- Found gmock, version 1.10.0
-- Configuring done
-- Generating done
-- Build files have been written to: /home/elza/projects/webfuse/build
elza@aspire3:~/projects/webfuse$ cmake --build build
Scanning dependencies of target webfuse_static
[ 1%] Building CXX object CMakeFiles/webfuse_static.dir/version.cpp.o
[ 3%] Building CXX object CMakeFiles/webfuse_static.dir/src/webfuse/webfuse.cpp.o
[ 5%] Building CXX object CMakeFiles/webfuse_static.dir/src/webfuse/provider.cpp.o
[ 6%] Building CXX object CMakeFiles/webfuse_static.dir/src/webfuse/fuse.cpp.o
[ 8%] Building CXX object CMakeFiles/webfuse_static.dir/src/webfuse/request_type.cpp.o
[ 10%] Building CXX object CMakeFiles/webfuse_static.dir/src/webfuse/response_type.cpp.o
[ 11%] Building CXX object CMakeFiles/webfuse_static.dir/src/webfuse/util/commandline_args.cpp.o
[ 13%] Building CXX object CMakeFiles/webfuse_static.dir/src/webfuse/util/commandline_reader.cpp.o
[ 15%] Building CXX object CMakeFiles/webfuse_static.dir/src/webfuse/util/authenticator.cpp.o
/home/elza/projects/webfuse/src/webfuse/util/authenticator.cpp:37:9: error: use of undeclared identifier 'closefrom' [clang-diagnostic-error]
closefrom(1);
^
543 warnings and 1 error generated.
Error while processing /home/elza/projects/webfuse/src/webfuse/util/authenticator.cpp.
Suppressed 543 warnings (543 in non-user code).
Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
Found compiler error(s).
make[2]: *** [CMakeFiles/webfuse_static.dir/build.make:167: CMakeFiles/webfuse_static.dir/src/webfuse/util/authenticator.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:192: CMakeFiles/webfuse_static.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
Please help ๐
Description
Webfuse2 authentication exposes the authentication token via command line, We have seen that modern AV solutions like MS Defender grabs the token so an attacker might do the same.
Steps To Reproduce
Use webfuse2 authentication.
Expected behavior
It should not be that easy to grab the authentication token.
Additional context
PAM's exec module uses stdin instead of command line options to perform the job (see https://linux.die.net/man/8/pam_exec).
An example implementation of a webfuse2 provider should be added as native executable.
Description
Reading large files fails since webfuse is unable to handle parallel fuse requests.
Steps To Reproduce
> md5sum /tmp/wf_bench/file.bin
md5sum: /tmp/wf_bench/file.bin: Datei oder Verzeichnis nicht gefunden
Expected behavior
webfuse should be able to read large files.
Additional context
A detailed documentation of websocket serices should be provided, which allows to write own client applications. This requires a more or less stable API.
I would like to create a tag 0.1.0 to state that all relevant feature for a first release are implemented by now. What do you think, @supermajor , @nosamad ?
We should proceed our way towards 1.0.0 by creating tests, providing code coverage and fixing some bugs (if any). But we should avoid to introduce a new feature.
I also expect some API changes, especially for provider stuff, since we should review our API, create some more examples and change it, when necessary.
We should add a a JavaScript example of a webfuse2 provider.
We need to evaluate state-of-the-art websocket authentication mechanisms (e.g. HTTP-Header, Cookies, protocol-specific solutions).
Websocket clients should authenticate themselves, before using the API.
Add the webfuse2 protocol specification as markdown file.
I want to get rid of this ugly pthread_kill trigger, currently used for communication between fuse and websockets threads. Since fuse low level API provides access of fd, it is even not necessary to use multiple threads.
Once webfuse2 is ready for it's first release, webfuse can be made legacy (including the fellow repositories).
Describe the bug
The functions wf_server_service and wfp_client_service specify a timeout in milliseconds as their second argument. This timeout is directly passed to libwebsockets lws_service() function. Since libwebsockets 3.2 the behavior of lws_service changed an the timeout is ignored.
Because of this, the *_service calls in libwebfuse will not return after timeout, but with the next lws event.
Steps To Reproduce
Expected behavior
Since timeout_ms should is ignored in libwebsockets, it should be ignored in libwebfuse too.
To cancel a service, a new pair of functions should be introduces: wf_server_interrupt and wfp_client_interrupt.
Additional context
@nosamad, @supermajor:
Yet another inquiry: I thought about a major change in API, in order to ease implementation of file system providers and to allow more than one connection per time.
Currently, wsfs (name subject to change) adopts a part of fuse library to websockets protocol. This API is kind of chatty, since lookup, getattr and readdir are communicated to the remote peer. Another problem is to scale the API to more than one client, since inode numbers are maintained by the provider. This is okay for just one client, but there must be a kind of separation, when more clients should be allowed.
Therefore I propose to change API dramatically:
A provider should register files, e.g. by a newly added add_file method. The provider is only responsible to answer read requests, all other stuff will be handled by wsfs (name subject to change) adapter.
API proposal: (provider --> adapter)
--> {"method": "add_file", "params": {"path": <path>, "size": <size>, "mode": <mode>}, "id": <id>}
<-- {"result": {"inode": <inode>}, "id": <id>}
--> {"method": "remove_file", "params": {"inode": <inode>}, "id": <id>}
<-- {"result": true, "id": <id>}
--> {"method": "make_topdir", "params": {"prefix": <prefix>}, "id": <id>}
<-- {"result": {"path": <path>}, "id": <id>}
API proposal: (adapter --> provider)
--> {"method": "read", "params": {"inode": <inode>, "offset": <offset>, "size": <size>}, "id": <id>}
<-- {"result": {"inode": <inode>}, "id": <id>}
Description:
As a web developer I'd like to use the client app as a library in my own projects.
Therefore, I propose to make the JavaScript client a node package and publish it on the official npm registry.
@nosamad , @supermajor
I would like to rename the project, since the current name is not pronouncable.
I'm in favor of webfs or something alike, but any suggestions are welcome.
What's your opion?
Is your feature request related to a problem? Please describe.
For authenticated communication, it should be possible to specify the authentication token.
Describe the solution you'd like
There should be at least two ways to specify the token:
Describe alternatives you've considered
n.a.
Additional context
n.a.
Description
Unit tests crash on arm32v7 (see Travis).
It seems that the bug is bot 100% reproducible, since different tests fail. But all failures were in context of IntegrationTest.
Steps To Reproduce
make -j4 DISTRIB_ID=ubuntu VARIANT=minsize MARCH=arm32v7
make -j1 DISTRIB_ID=ubuntu VARIANT=minsize MARCH=arm32v7 check
Expected behavior
All unit tests
Additional context
[----------] 4 tests from IntegrationTest
[ RUN ] IntegrationTest.HasMountpoint
--- stderr ---
fatal: unknown param_type '?'
fatal: unknown param_type '?'
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
-------
Currently, the project consists only of one executable, which is not even installable.
The user should be able to use wsfs as (shared) library, so it can be included in custom applications. User applications might or might not provide a custom main loop and / or own websocket services.
I would like to switch the build system from CMake to Meson. Currently, both build systems are supported. But I do not plan to support two or more build systems in future. Therefore, I intend to remove CMake support.
Please repond to this issue, if this might be a problem in you use case.
Webfuse supports a low level API to integrate servers and clients into existing libwebsockets applications. This is true for adapter (server), but it is not possible for provider (client), since there is no war for users to create a wfp_provider instance.
Provide unit tests for webfuse2
Add some project documentation:
Is your feature request related to a problem? Please describe.
It's not related to an actual problem.
Describe the solution you'd like
The timeout of webfuse server should be configurable. The current implementation uses a fixed value of 10 seconds.
Describe alternatives you've considered
n.a.
Additional context
n.a.
TLS should be supported an it should be configurable.
Port and local interface should be configurable too.
Description
webfuse_server blocks, when reading a file larger that 4 KByte.
Steps To Reproduce
fix_read_large_file
server.read_large_file
Expected behavior
Files should be read, independently from their size.
Additional context
N/A
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.