Git Product home page Git Product logo

curlftpfs's Introduction

CurlFtpFS
=========

CurlFtpFS is a filesystem for acessing ftp hosts based on FUSE and
libcurl. It automatically reconnects if the server times out. 

My motivation to start this project was to learn how to program using
Curl (http://curl.haxx.se) and Fuse (http://fuse.sourceforge.net). I
also wanted to maintain my website (http://www.golaberto.com.br) using
the tools that I was used to, like cd, mv, cp and vim.

As the FTP protocol is not very feature rich, this filesystem does not
fulfill every constraint of a real filesystem, but it should be usable
for simple tasks like copying and editing files.

Requirements
============

glib-2.0
libcurl >= 7.17.0


Compilation and Installation
============================

./configure
make
make install

Usage
=====

curlftpfs <ftpsite> <mountpoint>

ex.:
curlftpfs ftp://ftp.sunet.se/ sunet/


Debugging
=========

curlftpfs -f -v -o debug,ftpfs_debug=3 <ftpsite> <mountpoint> 

(runs the curlftpfs in foreground and shows libcurl verbose debug output)  


Known Problems
==============

1) Several GUI application (gedit, leafpad,...) use open(O_RDWR) + seek mode for saving files 
which cannot be supported by the FTP protocol. Therefore saving files might throw an error.
Hopefully future kernels will provide special errno's to make it easier to 
deal with less capable file-systems.

2) There seems to be a bug in libcurl 7.18 which sometimes causes problems reading files. 
Should be fixed in libcurl >= 7.18.2

http://sourceforge.net/tracker/index.php?func=detail&aid=1951588&group_id=976&atid=100976




curlftpfs's People

Contributors

gregnietsky avatar

Stargazers

Murat Dindar avatar

Watchers

James Cloos avatar  avatar  avatar

curlftpfs's Issues

Constant network traffic - Not a curlftpfs flaw but recursive access by Finder - Which options one needs to change

Environment

  • macOS 11.7.1
  • macfuse: 4.4.1
  • curlftpfs 0.9.2 libcurl/7.64.1 fuse/2.9

Cyberduck — Observation with Wireshark

  • In Cyberduck I use FTP-SSL (port 21) with username/password inside the SSL tunnel.
  • There is traffic when I login and for displaying the first directory.
  • Then there is network silence.
  • When I go into another directory, there's short network traffic.
  • Then there's again absolute network silence until my next action.

✅ That's a frugal network usage how it should be in my opinion.

curlftps — Observation with Wireshark

  • Connecting as FTP-SSL (port 21) via curlftps in FUSE
    curlftpfs -o ssl,uid=501,gid=20
    ftp://mycustomerid-backup:[email protected]/
    ~/Documents/Web/mydomain/backup-remote
    
  • Ca 10 seconds of network traffic (SSL handshake, listing root directory, done; 246 ethernet packets).
  • Then network silence.
  • In Finder browse inside your locally mounted remote FTP server.
    • ❌ The network traffic starts and continues for minutes, never stops…
      • On a low level, ca 20-30 KB/sec, but nevertheless, a constant data stream.
      • Initially I had no idea why.

Finder and Spotlight can access quite recursively - Check these settings

  1. System Preferences > Spotlight > Privacy:
    • Add the folder that acts as the mount point, e.g.
      ~/Documents/Web/mydomain/backup-remote
    • As soon as CURL mounts the volume as "macFUSE Volume 1 (curlftpfs)" that filepath is gone from Spotlight's exclusion list.
    • When you then try to drag'n'drop "macFUSE Volume 1 (curlftpfs)" onto the exclusion list again, it fails with an error message:
      Privacy List Error: The item couldn't be added or removed because of an unknown error.
    • ❌ An indication that FUSE and or curlftpfs do not properly integrate into the Mac filesystem mechanisms and its extensions light Spotlight (the search engine indexing mechanism of macOS)
      • ℹ️ Spotlight access anyhow seems not to be the reason for the constant network traffic. It's Finder. See next point.
  2. Finder: Show folder size (disk usage) with the "Calculate all sizes" option causes heavy recursive FTP access
  • Open any folder outside the FTP mount, e.g. your home directory, doesn't matter which one.
  • Menu > View > View Options (⌘ J): [√] Calculate all sizes
    • That option gives you the size of a directory.
    • Finder needs to recurse and sum up all the file sizes below your folder of interest.
    • This is fine on your local fast SSD. But not so fine for remote disks (which Finder may not aware of to be remote disks when using an alternative filesystem driver such as FUSE).
    • Disable "Calculate all sizes". Click "Use as defaults". Now you may re-activate it, if you need it in the particular folder you had opened.
    • ✅ From now on network traffic is decent:
      • Access a folder, there's 2-15secs of network traffic (depending on the amount of items in it)
      • Then there's network silence until the next user action.
      • But still, a lot of interactions still cause traffic, Finder is quite data-hungry:
        • If you only select a folder (one mouse click) and not open it (double click) this still causes some lookups.
        • Noticeable traffic. But least it ends after 2-3secs.
      • So for frugal browsing a true FTP/S client such as Cyberduck may be better for you.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.