Comments (6)
Depending on your data it could be OK or not.
If you're writing many time-series and each individual series gets small number of datapoints no writes to the storage volumes will be made. Your data will be stored in memory and WAL. But WAL capacity is not reflected by the stats endpoint. You can restart akumulid
and the /stats will be updated.
If the input file is ill-formed you won't see any error message while running this command time cat tsdb_mock.txt > /dev/tcp/127.0.0.1/4242
. In case of error in the file OpenTSDB endpoint will log the error and close the connection. I believe that bash won't print any warning or error message in this case.
from akumuli.
I restarted docker container, stats endpoint now shows volume_0 free space changed.
Does this mean Akumuli used 175.5 MB (524288000 total space on volume_0 - 352784384 free space)
of disk space to store the time-series?
Where is WAL stored? How do I check the disk space used by WAL? Can WAL size be configured?
{
"volume_0": {
"free_space": "352784384",
"file_name": "/Akumuli/db_0.vol"
},
"volume_1": {
"free_space": "524288000",
"file_name": "/Akumuli/db_1.vol"
},
"volume_2": {
"free_space": "524288000",
"file_name": "/Akumuli/db_2.vol"
},
"volume_3": {
"free_space": "524288000",
"file_name": "/Akumuli/db_3.vol"
},
"volume_4": {
"free_space": "524288000",
"file_name": "/Akumuli/db_4.vol"
},
"volume_5": {
"free_space": "524288000",
"file_name": "/Akumuli/db_5.vol"
}
}
This is what I see under /tmp/akumuli:
ls -lR /tmp/akumuli/
/tmp/akumuli/:
total 175380
-rw-r--r-- 1 root root 524288000 Aug 4 09:46 db_0.vol
-rw-r--r-- 1 root root 524288000 Aug 3 15:41 db_1.vol
-rw-r--r-- 1 root root 524288000 Aug 3 15:41 db_2.vol
-rw-r--r-- 1 root root 524288000 Aug 3 15:41 db_3.vol
-rw-r--r-- 1 root root 524288000 Aug 3 15:41 db_4.vol
-rw-r--r-- 1 root root 524288000 Aug 3 15:41 db_5.vol
-rw-r--r-- 1 root root 8085504 Aug 4 09:46 db.akumuli
drwxr-xr-x 2 root root 53 Aug 3 22:33 logs
/tmp/akumuli/logs:
total 4052
-rw-r--r-- 1 root root 4138135 Aug 4 09:47 akumuli.log
-rw-r--r-- 1 root root 5692 Aug 3 15:50 akumuli.log.2020-08-03
Looks like timeseries tags and other metadata are stored in db.akumuli which is a SQLite db. Will the metadata be auto deleted if the associated time-series expires and no new series for that object is added?
from akumuli.
WAL is configured in the config file. There is a WAL section in the automatically generated config and it has reasonalbe defaults. It also has a path to WAL storage. But normally you won't be able to locate any files if the database is stopped. You will be able to see the files if the database is running or crashed.
Will the metadata be auto deleted if the associated time-series expires and no new series for that object is added?
No. As for now it's not deleted.
from akumuli.
WAL is disabled by default. I'm using docker version "0.8.72-skylake". I looked at the config file ~/.akumulid
docker but didn't see any config related to WAL.
This is from the log:
Command line: /usr/bin/akumulid
2020-08-05 17:21:33,888 main [INFO] WAL is disabled in configuration
2020-08-05 17:21:33,888 akumuli-storage [INFO] Open database at: /Akumuli/db.akumuli
from akumuli.
I see. The config in docker image is hardcoded and it doesn't have a wal section. I probably should enable it by default in docker. I'll look into that. You can try to run akumulid
locally if you need to use WAL. Generally, it's preferable because Akumuli provides better persistency guarantees with WAL, also, it requires less RAM if you have a lot of empty stats.
from akumuli.
Thanks @Lazin, I will try to run akumulid locally with WAL settings.
from akumuli.
Related Issues (20)
- Errors while running grafana-server with akumuli-datasource when !events not found HOT 1
- Performance advice HOT 1
- Akumuli crashing on write HOT 1
- Are only positive integers supported? HOT 2
- "akumulid init" doesn't work without a pre-existing config file HOT 1
- Announcement: Akumuli Java Support HOT 1
- Akumuli advice HOT 6
- all VALUES must have the same number of terms HOT 5
- Memory leak? HOT 2
- Group-Aggregate Algorithm HOT 1
- POST on the HTTP server is probably truncated HOT 1
- Custom Panic Handler
- never ending "Main [ERROR] Empty rescue points list found, leaf-node data was lost" HOT 2
- crash at recovery HOT 2
- Abandonware?
- Data loss after recovery
- Missing headers in eval.cpp and storage2.cpp HOT 1
- CMakeLists use C++11, but std::shared_lock and std::shared_mutex are only available in C++14 and C++17, respectively
- Angular is deprecated - please migrate to React
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from akumuli.