GCP: NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:postgresql.psycopg exception about superset-on-gcp-cloud-run HOT 14 CLOSED

@bono-mp I encountered this error and I had to drop the +psycopg part from the connection string, then the host name couldn't be resolved, so I went into the GCP web console and found the private IP for my DB instance and replaced the whole superset?host=<long string> part of the end of the connection string with the private IP and that got me passed the error.

from superset-on-gcp-cloud-run.

@bono-mp did you get resolving this?
Having the same issue.

from superset-on-gcp-cloud-run.

@bono-mp did you get resolving this? Having the same issue.

I had no succes, although I haven't had any time to really look into it.

from superset-on-gcp-cloud-run.

Having the same issue.

from superset-on-gcp-cloud-run.

Same issue

from superset-on-gcp-cloud-run.

Hello friends. After much pounding of sand I was able to get passed this error. I finally noticed that the error was loading sqlalchemy.dialects:postgresql.psycopg, but psycopg2 was what was being pip installed in the requirements.txt file. So I updated the connection string to start with postgres+psycopg2:// instead of postgres+psycopg://.

Changing that actually got me a net new error, but I was at least making progress. Next it started barking at me about not being able to resolve the host in the example uri .

I then went a googling and found that the connection string was malformed a tiny bit.

Here is an updated version:

postgres+psycopg2://<un>:<pwd>@/<db>?host=/cloudsql/<project>:<region>:<db>

I was still getting an error, but this time it was something about a unix_sock and I went back to googling. Eventually I ran across an this stackoverflow post. I read through this and gave it a shot.

I commented out this line and set an environment variable in my container for SQLALCHEMY_DATABASE_URI as the updated connection sting and bingo.

My next step is to update the gcloud run deploy command to insert the connection string as an environment variable named SQLALCHEMY_DATABASE_URI and retest deploying to cloud run. I'll update this post once I have finished testing.

Hope this helps someone.

thx @lord-skinner

from superset-on-gcp-cloud-run.

Welp it didn't give me errors when running locally in a container, but once I deployed I noticed it was attempting to work off a sqlite database so it looks like it's just defaulting to that with no explicit var in the setup_config.py 🤔

from superset-on-gcp-cloud-run.

@lord-skinner , thx for looking into this. I'm picking this up again. I'll report one I've made progress.

from superset-on-gcp-cloud-run.

I've done as you said @ericdrosas87 , and it's working , just want to mention that I've used vpc serverless connection to connect to the private cloud sql instance .

from superset-on-gcp-cloud-run.

Just want to note that @ericdrosas87's solution works great, but a VPC network is necessary to get it going!

from superset-on-gcp-cloud-run.

Thank you all for looking into this. I'm sorry I haven't had time to return to this for so long. As some folks have noted the root of the error was in a type in this line. I've updated the repository with a correction.
SUPERSET_CONNECTION_SECRET=postgresql+psycopg2://superset:<SQL_PASSWORD>@/superset?host=/cloudsql/apache-superset-350214:us-west1:superset-configs

from superset-on-gcp-cloud-run.

Hi all, I'm still facing the issue. Could anyone help me troubleshoot? thank you very much.

from superset-on-gcp-cloud-run.

@vkt1414 The error you're showing is caused by a separate issue. It appears to be a problem with the actual database you've set up to drive Superset. I recommend attempting to run the superset db upgrade command again from your VS Code dev container as in this step.

from superset-on-gcp-cloud-run.

thanks so much for your response. I think I was blindly following the setup instructions. I was not able to make it work from this repo yet (i'm going to give more tries and try to fix some dependencies), but when you mentioned db upgrade, it got me thinking that we are using the dev container just to make sure the superset metadata database is populated and is ready to go when we deploy the same image in cloud run. So I used my bread-and-butter mysql database as backend, and deployed a container locally and did superset db upgrade and init. I then used exact same config to build an image on gcp, then used it run on cloud run and it worked! thanks again for sharing the knowledge and making this repo!

from superset-on-gcp-cloud-run.