Comments (4)
You can use the bash script below as a temporal solution.
It takes an oldest table and delete it if the value of TARGET_PERCENT more than current percent of use /var/lib/clickhouse directory.
#!/bin/bash
TARGET_PERCENT=95
POD=$(/usr/local/bin/kubectl get po -n loghouse |awk '/clickhouse-server/{ print $1}')
CURRENT_PERCENT=$(/usr/local/bin/kubectl -n loghouse exec -ti $POD -- /bin/df -h |awk '/\/var\/lib\/clickhouse/{print $5}')
CURRENT_PERCENT=${CURRENT_PERCENT//%/}
if [ "$CURRENT_PERCENT" -gt "$TARGET_PERCENT" ]; then
CL_PASS=$(/usr/local/bin/kubectl -n loghouse get po $POD -o yaml |grep CLICKHOUSE_PASS -A1 | awk '/value/{print $2}')
echo "CURRENT_PERCENT $CURRENT_PERCENT -gt TARGET_PERCENT $TARGET_PERCENT"
TBL=$(/usr/local/bin/kubectl -n loghouse exec -ti $POD -- clickhouse-client -h 127.0.0.1 -d logs --password $CL_PASS -q "show tables" |grep -E "logs[0-9]{10}" |sort |head -1)
echo "drop table $TBL"
/usr/local/bin/kubectl -n loghouse exec -ti $POD -- clickhouse-client -h 127.0.0.1 -d logs --password $CL_PASS -q "drop table $TBL"
fi
You can also use it in crontab.
Example:
*/15 * * * * root /opt/scripts/loghouse.cleaner.sh >> /var/log/loghouse.cleaner.log 2>&1
from loghouse.
@zimmermann-russia you are right, but this patch makes you script better:
--- loghouse.cleaner.sh 2018-10-12 18:21:34.046754060 +0300
+++ loghouse.cleaner_new.sh 2018-10-12 18:22:27.719205599 +0300
@@ -4,10 +4,12 @@
POD=$(/usr/local/bin/kubectl get po -n loghouse |awk '/clickhouse-server/{ print $1}')
CURRENT_PERCENT=$(/usr/local/bin/kubectl -n loghouse exec -ti $POD -- /bin/df -h |awk '/\/var\/lib\/clickhouse/{print $5}')
CURRENT_PERCENT=${CURRENT_PERCENT//%/}
-if [ "$CURRENT_PERCENT" -gt "$TARGET_PERCENT" ]; then
+while [ "$CURRENT_PERCENT" -gt "$TARGET_PERCENT" ]; do
CL_PASS=$(/usr/local/bin/kubectl -n loghouse get po $POD -o yaml |grep CLICKHOUSE_PASS -A1 | awk '/value/{print $2}')
echo "CURRENT_PERCENT $CURRENT_PERCENT -gt TARGET_PERCENT $TARGET_PERCENT"
TBL=$(/usr/local/bin/kubectl -n loghouse exec -ti $POD -- clickhouse-client -h 127.0.0.1 -d logs --password $CL_PASS -q "show tables" |grep -E "logs[0-9]{10}" |sort |head -1)
echo "drop table $TBL"
/usr/local/bin/kubectl -n loghouse exec -ti $POD -- clickhouse-client -h 127.0.0.1 -d logs --password $CL_PASS -q "drop table $TBL"
-fi
+ CURRENT_PERCENT=$(/usr/local/bin/kubectl -n loghouse exec -ti $POD -- /bin/df -h |awk '/\/var\/lib\/clickhouse/{print $5}')
+ CURRENT_PERCENT=${CURRENT_PERCENT//%/}
+done
from loghouse.
@broken-ufa Thanks!
from loghouse.
Closed because of loghouse is no longer being actively developed.
from loghouse.
Related Issues (20)
- Clickhouse doesn't respect cpu and memory limits HOT 1
- hierarchical JSON in log HOT 2
- Display multiline values with "pre" attribute HOT 1
- Helm chart: Clickhouse should be managed by StatefulSet instead of Deployment HOT 2
- Persistent volumes should be initialized
- Export in CSV file doesn't work HOT 3
- jessie-updates: 404 Not Found HOT 1
- Logs are not displayed on the dashboard
- Is there docker only (without kubernetes) installation method ? HOT 1
- Error deploying loghouse to GKE HOT 4
- Propose to add docker-compose file HOT 1
- Migrate release process to Github Actions
- Example configuration for external clickhouse? HOT 7
- Can Fluentd we swapped out for Fluent Bit by just changing the values.yaml image? HOT 1
- Helm chart is not compatible with the current cert-manager stable release HOT 1
- Unable to filter by label containing slash / symbol
- loghouse support containerd and fluent bit ?
- Logs UI
- PVC settings can't parse HOT 1
- Clickhouse still not respecting CPUShares in the latest version HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from loghouse.