Comments (6)
You can ignore the "housekeeper not running" message. This is not fatal but it isn't correct behavior as well, I'm working on the fix.
The problem is with "FATAL ERROR: operation not supported". It looks like there is something wrong with filesystem and it doesn't support some operation. Right now, it's enigmatic where this error originates from and why this happens in the first place. I'm right now working on the fix to make sure that the error has correct file and line shown. But in meantime, do you know what is underlying filesystem in your environment? - you can run lsblk -f
to check that
from aistore.
Hey @benjha I've pushed some new fixes/commits into master branch. If you have a chance to build from that and run it, would be awesome. Let me know what error message you are getting.
from aistore.
Thanks @VirrageS,
I don't see the housekeeper error anymore. I think the FATAL ERROR: operation not supported
is something I forced to happen when commenting the line that verifies setfattr
in deploy/dev/local/deploy.sh
given that the command is not installed.
This is the output of lsblk -f
in the compute node.
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
bb-cache 253:0 0 104.3G 0 lvm /var/cache/fscache
bb-bb1 253:1 0 1.4T 0 lvm /mnt/bb/benjha
nvme0n1 259:1 0 1.5T 0 disk
Once I figure out the issue with the extended attributes, I'd like to launch AIStore on /mnt/bb/benjha
which is the mount point of nvme0n1
(NVMe's parition uses XFS). I think this should be done in one of the configuration files, right ? Where can I find documentation about this ?
On the other hand, is aisfs
an optional requirement ?
from aistore.
Once I figure out the issue with the extended attributes
Yeah, I think it this can be connected the extended attributes as AIStore uses them broadly and requires them to be enabled. AIStore requires following packages to be installed: gcc
, sysstat
and attr
(see: https://github.com/NVIDIA/aistore/blob/master/docs/getting_started.md#prerequisites).
I'd like to launch AIStore on /mnt/bb/benjha which is the mount point of nvme0n1 (NVMe's parition uses XFS). I think this should be done in one of the configuration files, right ? Where can I find documentation about this ?
Yes, you can check out here https://aiatscale.org/docs/configuration. Basically, what you probably want to do is modify the content of deploy/dev/local/aisnode_config.sh
(assuming you are doing make deploy
). The thing which can be interesting for you are:
confdir
- you can also change this withAIS_CONF_DIR
env variablelog_dir
- you can also change this withAIS_LOG_DIR
env variablefspaths
- you can also change this withAIS_FS_PATHS
env variabletest_fspaths
- you can also change this withTEST_FSPATH_ROOT
env variable
On the other hand, is aisfs an optional requirement ?
Yes, this is totally optional. This is the tool that lets you mount the AIStore as the directory.
from aistore.
Ok, looks like none of the kernel modules needed by AIStore are loaded in the system.
Thanks for your help.
from aistore.
Sounds good :) Closing the issue. If you have any further problems, let us know!
from aistore.
Related Issues (20)
- Hotplugging of volumes in an air-gapped deployment HOT 3
- Terraform Kubernetes deployment on GCP unable to start aistorage/aisnode:latest image HOT 6
- aistore.sdk.errors.AISError: STATUS:404, MESSAGE:x-[emL1D2Xudc] not found (jhNp8080(*)) This exception is thrown by __iter__ of AISFileListerIterDataPipe(length=-1, source_datapipe=['ais://OB_static']) HOT 1
- [question] may I know what is main design difference with standard azure object storage and minio for AI workload HOT 1
- Use more reliable way to validate backend disk of mountpath HOT 3
- the proper way to restart the cluster HOT 1
- Python sdk put content times out HOT 2
- Promoting an object using python sdk fails HOT 1
- test
- Extreme delays while reading objects for the first time HOT 3
- Migration of data between deployments HOT 1
- Got a lot of object lost in minimal production-ready standalone docker deployment HOT 1
- Question: List of Production Adopters HOT 1
- Documentation on nodes with different storage capacity HOT 5
- Build for arm64 HOT 3
- [CLOSED] HOT 1
- Docker image exits HOT 2
- AuthN support for S3 Compat Clients HOT 7
- GO Api examples/tests HOT 3
- AiStore.org HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aistore.