Comments (3)
modem7
commented on July 18, 2024
It seems that copying anything from a non-greyhole/samba share into a greyhole/samba share causes the slowdowns.
Currently we're looking at around 1-5kb/s average throughput rate.
I have reinstalled Greyhole, removed mysql, and replaced it with MariaDB (far lower resource usage too), and I've also reinstalled Samba, same issues unfortunately.
from greyhole.
gboudreau
commented on July 18, 2024
No much Greyhole-related is used, when copying files onto a share : Samba handles this by itself, and will log some data for the Greyhole daemon to work with, once the file is finished being written to.
Some Samba version will log a "lot" of "write" logs, and this is why those logs are stored in /var/spool/greyhole/mem, which is a tmpfs (a filesystem stored in RAM), so writing in there would be very fast, and thus not cause any slowdown.
Maybe look into this ; check that this folder in indeed a tmpfs, and maybe try to stop the Greyhole daemon, and write some files in there, to make sure it works fast enough. (It's a 4MB FS, so you won't be able to write a lot of data, maybe try to rapidly create a lot of small files; it should be very fast.) Make sure to remove those test files before your restart the Greyhole damon ; not sure it would be happy to find files with content it does not expect in there.
Also : did you try to just comment out, temporarily, the vfs objects = greyhole line, for the target share?
Try to copy a file manually, make sure it is slow, then comment out that line for the target share, restart samba, and try the same file copy. If it is now slow anymore, then you will know the VFS module is indeed the culprit. (De-comment the line and restart Samba.)
from greyhole.
modem7
commented on July 18, 2024
File is going from /var/hda/files/drives/drive12/downloads (non-Greyhole) to /var/hda/files/drives/drive12/Greyhole/requests (Greyhole).
Without vfs objects (and restarted GH + Samba)
[Requests]
path = /var/hda/files/drives/drive12/Greyhole/requests
create mask = 0775
dfree command = /usr/bin/greyhole-dfree
directory mask = 0775
force create mode = 0664
force directory mode = 0775
valid users = alex
#vfs objects = greyhole
write list = alex
Looks like the spool folder is barely being used in this particular case:
And GH logs seem to be acceptable:
Oct 04 01:04:57 DEBUG read_smb_spool: Processing Samba spool...
Oct 04 01:04:57 DEBUG sleep: Nothing to do... Sleeping.
Oct 04 01:05:01 DEBUG read_smb_spool: Processing Samba spool...
Oct 04 01:05:07 DEBUG read_smb_spool: Processing Samba spool...
Oct 04 01:05:07 DEBUG read_smb_spool: Processing Samba spool...
Oct 04 01:05:07 INFO write: Now working on task ID 192641: write Requests/Movies/FileInQuestion.mkv
Oct 04 01:05:07 INFO write: File created: Requests/Movies/FileInQuestion.mkv - 412MB
Oct 04 01:05:07 DEBUG write: Loading metafiles for Requests/Movies/FileInQuestion.mkv ...
Oct 04 01:05:07 DEBUG write: Got 0 metadata files.
Oct 04 01:05:08 DEBUG write: File Requests/Movies/FileInQuestion.mkv is locked by another process (smbd, PID 2843598). Will wait until it's unlocked to work on any file in this share.
Oct 04 01:05:08 DEBUG write: Deleting 20 future 'write' tasks that are duplicate of this one.
Oct 04 01:05:08 DEBUG read_smb_spool: Processing Samba spool...
Oct 04 01:05:08 DEBUG sleep: Nothing to do... Sleeping.
Oct 04 01:05:12 DEBUG read_smb_spool: Processing Samba spool...
Oct 04 01:05:17 DEBUG read_smb_spool: Processing Samba spool...
Oct 04 01:05:18 DEBUG read_smb_spool: Processing Samba spool...
Oct 04 01:05:18 INFO write: Now working on task ID 192641: write Requests/Movies/FileInQuestion.mkv
Oct 04 01:05:18 INFO write: File created: Requests/Movies/FileInQuestion.mkv - 426MB
Oct 04 01:05:18 DEBUG write: Loading metafiles for Requests/Movies/FileInQuestion.mkv ...
Oct 04 01:05:18 DEBUG write: Got 0 metadata files.
Oct 04 01:05:19 DEBUG write: File Requests/Movies/FileInQuestion.mkv is locked by another process (smbd, PID 2843598). Will wait until it's unlocked to work on any file in this share.
Oct 04 01:05:19 DEBUG write: Deleting 18 future 'write' tasks that are duplicate of this one.
Oct 04 01:05:19 DEBUG read_smb_spool: Processing Samba spool...
Oct 04 01:05:19 DEBUG sleep: Nothing to do... Sleeping.
Oct 04 01:05:22 DEBUG read_smb_spool: Processing Samba spool...
Oct 04 01:05:28 DEBUG read_smb_spool: Processing Samba spool...
Samba config is pretty generic: https://paste.modem7.com/ZrSl8AyXLB
Syslog: https://paste.modem7.com/44rCXnXodM
20221004010055.mp4
from greyhole.
Related Issues (20)
- MySQL server has gone away HOT 2
- No access to files from shares. HOT 11
- Change fsck cron timing HOT 1
- Debian 12, samba 4.17 and VFS not building HOT 2
- VFS errors HOT 9
- logrotate is already installed HOT 6
- daemon: Greyhole VFS module warning HOT 2
- Greyhole won't start after update - SQL error Can't DROP 'uniqness'; HOT 4
- Makefile error Samba 4.18.6 HOT 5
- DEBUG read_smb_spool: Directory created: / WARN read_smb_spool: Failed to create directory HOT 1
- Can't add drive to drive selection group! HOT 16
- Ownership of share-folders inside gh folder HOT 6
- Accidentally Copied Lots of Small Files - Recovering from a Blown Out log and Spool. HOT 5
- fsck parallelism HOT 1
- Failed file copy (using rename) HOT 3
- No Files shown after reinstall Greyhole HOT 10
- Greyhole won't start HOT 5
- greyhole service won't start with containerd mariadb (greyhole.service) HOT 1
- Feature Request: Make Read_Smb_Spool a separate task running concurrent HOT 7
- Balance Seems to Ignore Available Drive HOT 11
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from greyhole.