Comments (5)
macobo
commented on July 22, 2024
1
This was indeed completely broken. It is now mostly solved, but not yet accessible in a posthog release: read #279 context.
Closing this ticket in favor of that one for now.
from charts-clickhouse.
macobo
commented on July 22, 2024
Indeed, this isn't well-documented.
How is your clickhouse cluster named? You could set an env variable at the top level of values.yaml like so:
env:
- name: CLICKHOUSE_CLUSTER
value: YOURCLUSTERNAMEfrom charts-clickhouse.
fuziontech
commented on July 22, 2024
Hey @tpoxa ! You are one of the first people to be using their own ClickHouse cluster to host PostHog. Nice.
You can find where this is used in the code here:
https://github.com/PostHog/posthog/blob/06bdd1d4860a351e61a376ae595c7f046fe4618b/posthog/settings.py#L182
We use this because we somewhat manage replicated tables this way and run distributed DDLs this way as well. Please let us know how this goes. You are in relatively uncharted territory :)
from charts-clickhouse.
tpoxa
commented on July 22, 2024
You are in relatively uncharted territory :)
I've felt that :)
I added that env with the cluster I have in my clusters lists. Haven't worked with this feature of Clickhouse.
env:
- name: CLICKHOUSE_CLUSTER
value: test_cluster_two_shards
plugins:
env:
- name: CLICKHOUSE_CLUSTER
value: test_cluster_two_shards
The good news its definitely changed something. Bad news - migrations not coming through
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 89, in wrapped
res = handle_func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/migrate.py", line 75, in handle
self.check(databases=[database])
File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 419, in check
all_issues = checks.run_checks(
File "/usr/local/lib/python3.8/site-packages/django/core/checks/registry.py", line 76, in run_checks
new_errors = check(app_configs=app_configs, databases=databases)
File "/usr/local/lib/python3.8/site-packages/django/core/checks/model_checks.py", line 34, in check_all_models
errors.extend(model.check(**kwargs))
File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 1290, in check
*cls._check_indexes(databases),
File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 1682, in _check_indexes
connection.features.supports_covering_indexes or
File "/usr/local/lib/python3.8/site-packages/django/utils/functional.py", line 48, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/features.py", line 93, in is_postgresql_11
return self.connection.pg_version >= 110000
File "/usr/local/lib/python3.8/site-packages/django/utils/functional.py", line 48, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 329, in pg_version
with self.temporary_connection():
File "/usr/local/lib/python3.8/contextlib.py", line 113, in __enter__
return next(self.gen)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 603, in temporary_connection
with self.cursor() as cursor:
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 259, in cursor
return self._cursor()
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 235, in _cursor
self.ensure_connection()
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
self.connect()
File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
self.connect()
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 200, in connect
self.connection = self.get_new_connection(conn_params)
File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 187, in get_new_connection
connection = Database.connect(**conn_params)
File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: FATAL: database "posthog" does not exist
System check identified some issues:
WARNINGS:
axes.AccessAttempt: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the AppConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
axes.AccessLog: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the AppConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
ee.License: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the EnterpriseConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.Action: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.ActionStep: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.Annotation: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.Cohort: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.Dashboard: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.DashboardItem: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.Element: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.ElementGroup: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.Event: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.FeatureFlag: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.FeatureFlagOverride: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.Person: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.PersonDistinctId: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.Plugin: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.PluginAttachment: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.PluginConfig: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.PluginStorage: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.SessionRecordingEvent: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.SessionRecordingViewed: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.SessionsFilter: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.Team: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
posthog.User: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostHogConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
rest_hooks.Hook: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'.
HINT: Configure the DEFAULT_AUTO_FIELD setting or the AppConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'.
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute
output = self.handle(*args, **options)
File "/home/posthog/code/ee/management/commands/migrate_clickhouse.py", line 38, in handle
self.migrate(CLICKHOUSE_HTTP_URL, options)
File "/home/posthog/code/ee/management/commands/migrate_clickhouse.py", line 74, in migrate
database.migrate(MIGRATIONS_PACKAGE_NAME, options["upto"], replicated=CLICKHOUSE_REPLICATION)
File "/usr/local/lib/python3.8/site-packages/infi/clickhouse_orm/database.py", line 349, in migrate
operation.apply(self)
File "/usr/local/lib/python3.8/site-packages/infi/clickhouse_orm/migrations.py", line 271, in apply
database.raw(item)
File "/usr/local/lib/python3.8/site-packages/infi/clickhouse_orm/database.py", line 292, in raw
return self._send(query, settings=settings, stream=stream).text
File "/usr/local/lib/python3.8/site-packages/infi/clickhouse_orm/database.py", line 371, in _send
raise ServerError(r.text)
infi.clickhouse_orm.database.ServerError: There is no Zookeeper configuration in server config (version 21.3.16.5 (official build)) (139)
from charts-clickhouse.
hazzadous
commented on July 22, 2024
Hi @tpoxa , it looks like the liwenhe1993 helm chart doesn't include zookeeper, so you'll likely need to deploy a separate zookeeper cluster. Something like:
helm install zookeeper bitnami/zookeeper
helm install clickhouse liwenhe/clickhouse --set clickhouse.imageVersion=21.6 --set clickhouse.configmap.zookeeper_servers.enabled=true --set clickhouse.configmap.zookeeper_servers.config[0].host=zookeeper --set clickhouse.configmap.zookeeper_servers.config[0].port=2181
might do the trick, although I haven't confirmed.
from charts-clickhouse.
Related Issues (20)
- Cleanup old `posthog-grafana-annotation` jobs
- Add additional configuration to externalKafka
- Read-only tables in clickhouse
- CI - Ingress: validate traffic routing
- Alertmanager - switch `RedisTooManyConnections` to be % based
- Support nodeSelector, tolerations, affinity for "events" HOT 1
- Plugin server is not starting
- Unable to change clickhouse log level
- Backups error HOT 1
- Trying to install second instance HOT 3
- Only 23.5.1 available for Helm Chart for k8 HOT 1
- Support for external Grafana HOT 3
- External Zookeeper seems ignored - references built-in
- Support GCP CloudSQL Proxy for connecting to Postgres database in GCP deployment HOT 1
- Migration failed to run HOT 1
- Service does not have load balancer ingress IP address: posthog/posthog-ingress-nginx-controller HOT 3
- Add slow lane consumer for ingestion
- Bump version of kafka
- Kafka error: 'posthog' is not in the form of topic-partition or topic-partition.uniqueId-delete (if marked for deletion). HOT 1
- Unable to reset password using SMTP.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from charts-clickhouse.