stats endpoint about akumuli HOT 6 OPEN

Depending on your data it could be OK or not.
If you're writing many time-series and each individual series gets small number of datapoints no writes to the storage volumes will be made. Your data will be stored in memory and WAL. But WAL capacity is not reflected by the stats endpoint. You can restart akumulid and the /stats will be updated.

If the input file is ill-formed you won't see any error message while running this command time cat tsdb_mock.txt > /dev/tcp/127.0.0.1/4242. In case of error in the file OpenTSDB endpoint will log the error and close the connection. I believe that bash won't print any warning or error message in this case.

from akumuli.

I restarted docker container, stats endpoint now shows volume_0 free space changed.
Does this mean Akumuli used 175.5 MB (524288000 total space on volume_0 - 352784384 free space) of disk space to store the time-series?

Where is WAL stored? How do I check the disk space used by WAL? Can WAL size be configured?

{
    "volume_0": {
        "free_space": "352784384",
        "file_name": "/Akumuli/db_0.vol"
    },
    "volume_1": {
        "free_space": "524288000",
        "file_name": "/Akumuli/db_1.vol"
    },
    "volume_2": {
        "free_space": "524288000",
        "file_name": "/Akumuli/db_2.vol"
    },
    "volume_3": {
        "free_space": "524288000",
        "file_name": "/Akumuli/db_3.vol"
    },
    "volume_4": {
        "free_space": "524288000",
        "file_name": "/Akumuli/db_4.vol"
    },
    "volume_5": {
        "free_space": "524288000",
        "file_name": "/Akumuli/db_5.vol"
    }
}

This is what I see under /tmp/akumuli:

ls -lR /tmp/akumuli/
/tmp/akumuli/:
total 175380
-rw-r--r-- 1 root root 524288000 Aug  4 09:46 db_0.vol
-rw-r--r-- 1 root root 524288000 Aug  3 15:41 db_1.vol
-rw-r--r-- 1 root root 524288000 Aug  3 15:41 db_2.vol
-rw-r--r-- 1 root root 524288000 Aug  3 15:41 db_3.vol
-rw-r--r-- 1 root root 524288000 Aug  3 15:41 db_4.vol
-rw-r--r-- 1 root root 524288000 Aug  3 15:41 db_5.vol
-rw-r--r-- 1 root root   8085504 Aug  4 09:46 db.akumuli
drwxr-xr-x 2 root root        53 Aug  3 22:33 logs

/tmp/akumuli/logs:
total 4052
-rw-r--r-- 1 root root 4138135 Aug  4 09:47 akumuli.log
-rw-r--r-- 1 root root    5692 Aug  3 15:50 akumuli.log.2020-08-03

Looks like timeseries tags and other metadata are stored in db.akumuli which is a SQLite db. Will the metadata be auto deleted if the associated time-series expires and no new series for that object is added?

from akumuli.

WAL is configured in the config file. There is a WAL section in the automatically generated config and it has reasonalbe defaults. It also has a path to WAL storage. But normally you won't be able to locate any files if the database is stopped. You will be able to see the files if the database is running or crashed.

Will the metadata be auto deleted if the associated time-series expires and no new series for that object is added?
No. As for now it's not deleted.

from akumuli.

WAL is disabled by default. I'm using docker version "0.8.72-skylake". I looked at the config file ~/.akumulid docker but didn't see any config related to WAL.

This is from the log:

Command line: /usr/bin/akumulid


2020-08-05 17:21:33,888 main [INFO] WAL is disabled in configuration
2020-08-05 17:21:33,888 akumuli-storage [INFO] Open database at: /Akumuli/db.akumuli

from akumuli.

I see. The config in docker image is hardcoded and it doesn't have a wal section. I probably should enable it by default in docker. I'll look into that. You can try to run akumulid locally if you need to use WAL. Generally, it's preferable because Akumuli provides better persistency guarantees with WAL, also, it requires less RAM if you have a lot of empty stats.

from akumuli.

Thanks @Lazin, I will try to run akumulid locally with WAL settings.

from akumuli.