Git Product home page Git Product logo

pgsu's Introduction

Build Status Coverage Status PyPI version GitHub license

pgsu

Connect to an existing PostgreSQL cluster as a PostgreSQL SUPERUSER and execute SQL commands.

psycopg2 has a great API for interacting with PostgreSQL, once you provide it with the connection parameters for a given database. However, what if your desired database and database user do not yet exist? In order to create them, you will need to connect to PostgreSQL as a SUPERUSER.

Features

  • autodetects postgres setup, tested on
  • uses psycopg2 to connect if possible
  • can use sudo to become the postgres UNIX user if necessary/possible (default Ubuntu PostgreSQL setups)

Usage

Python API

from pgsu import PGSU
pgsu = PGSU()  # On Ubuntu, this may prompt for sudo password
pgsu.execute("CREATE USER newuser WITH PASSWORD 'newpassword'")
users = pgsu.execute("SELECT usename FROM pg_user WHERE usename='newuser'")
print(users)

While the main point of the package is to guess how to connect as a postgres superuser, you can also provide partial or all information abut the setup using the dsn parameter. These are the default settings:

from pgsu import PGSU
pgsu = PGSU(dsn={
    'host': None,
    'port': 5432,
    'user': 'postgres',
    'password': None,
    'database': 'template1',  # Note: you cannot drop databases you are connected to
})

Command line tool

The package also comes with a very basic pgsu command line tool that allows users to execute PostgreSQL commands as the superuser:

$ pgsu "SELECT usename FROM pg_user"
Trying to connect to PostgreSQL...
Executing query: SELECT usename FROM pg_user
[('aiida_qs_leopold',),
 ('postgres',)]

Tests

Run the tests as follows:

pip install -e .[testing]
pytest

pgsu's People

Contributors

chrisjsewell avatar danielhollas avatar jbweston avatar ltalirz avatar

Watchers

 avatar  avatar  avatar

Forkers

jbweston

pgsu's Issues

Support Ubuntu 22.04

When trying to update the CI tests to the latest runner image based on Ubuntu 22.04 the tests were failing while trying to connect to the Postgres socket.

fix windows CI build via conda

The windows CI build has started failing with the error
https://github.com/aiidateam/pgsu/runs/1914573646#step:7:69

psycopg2.OperationalError: FATAL:  role "postgres" does not exist

(while the MacOS build using otherwise identical instructions still succeeds)

This sounds like, for some reason, on windows there is no postgresql superuser.

However, on April 5th 2020 the tests still passed, and there hasn't been an update in the anaconda postgresql package for windows since then
https://anaconda.org/anaconda/postgresql/files

the only difference I can see between macos and windows is

MacOS

initdb: warning: enabling "trust" authentication for local connections
syncing data to disk ... ok
You can change this by editing pg_hba.conf or using the option -A, or

--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    pg_ctl -D test_db -l logfile start

waiting for server to start....2021-02-16 23:09:22.255 UTC [1487] LOG:  starting PostgreSQL 12.2 on x86_64-apple-darwin14.5.0, compiled by clang version 4.0.1 (tags/RELEASE_401/final), 64-bit
2021-02-16 23:09:22.259 UTC [1487] LOG:  listening on IPv6 address "::1", port 5432
2021-02-16 23:09:22.259 UTC [1487] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-02-16 23:09:22.271 UTC [1487] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2021-02-16 23:09:22.312 UTC [1488] LOG:  database system was shut down at 2021-02-16 23:09:22 UTC
2021-02-16 23:09:22.321 UTC [1487] LOG:  database system is ready to accept connections
 done
server started

Windows

initdb: warning: enabling "trust" authentication for local connections
syncing data to disk ... ok
You can change this by editing pg_hba.conf or using the option -A, or

--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    C:/Miniconda/envs/test/Library/bin/pg_ctl -D test_db -l logfile start

waiting for server to start....2021-02-16 23:09:36.914 UTC [6712] LOG:  starting PostgreSQL 12.2, compiled by Visual C++ build 1916, 64-bit
2021-02-16 23:09:36.917 UTC [6712] LOG:  listening on IPv6 address "::1", port 5432
2021-02-16 23:09:36.917 UTC [6712] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-02-16 23:09:36.960 UTC [4936] LOG:  database system was shut down at 2021-02-16 23:09:35 UTC
2021-02-16 23:09:36.972 UTC [6712] LOG:  database system is ready to accept connections
 done
server started

don't specify host for local psycopg2 connection

When specifying a host (even localhost) in the arguments to psycopg2, the connection will be made over method host rather than local.

If the postgresql configuration pg_hba.conf looks like this, then this will cause an OperationalError: fe_sendauth: no password supplied:

local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            md5
# IPv6 local connections:
host    all             all             ::1/128                 md5

I.e. unless the host is explicitly provided by the user and does not equal localhost, it should be None.

add command line interface

this should have a very minimal command line interface, at least to test whether a connection can be established, so that one can ask users with connection issues to try

pgsu-connect --host ...

potentially incorrect order of connection trials

Iterating over sets will not necessarily produce the same order as the one used by the original lists:

pgsu/pgsu/__init__.py

Lines 137 to 144 in c75a1f4

# Try to connect as a postgres superuser via psycopg2 (equivalent to using psql).
LOGGER.debug('Trying to connect via "psycopg2"...')
for pg_user in set([self.dsn.get('user'), None]):
dsn['user'] = pg_user
# First try the host specified (works if 'host' has setting 'trust' in pg_hba.conf).
# Then try local connection (works if 'local' has setting 'trust' in pg_hba.conf).
# Then try 'host' localhost via TCP/IP.
for pg_host in set([self.dsn.get('host'), None, 'localhost']):

Warning: Could not change directory for `sudo` route

When pgsu tries to become the postgres user and run a psql command, psql prints a warning that it is not able to change directory:

WARNING  pgsu:__init__.py:368 could not change directory to "/anfhome/a-ctalirz/software/aiida-core": Permission denied
psql: error: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.41871"?

Since the current implementation routes this to a python logger, this is typically not seen by the user unless an exception is raised (or when running tests).
It would be good to fix nevertheless.

pgsu always writes to stderr, and this cannot be suppressed

When using functionality from pgsu in a Jupyter notebook I always get messages like the following printed to stderr:

Trying to become 'postgres' user. You may be asked for your 'sudo' password.                                              
could not change directory to "..../code/aiida-dynamic-workflows": Permission denied

These messages cannot be suppressed with contextlib.redirect_stderr, and there is no pgsu setting that I am aware of to suppress these messages.

AFAIK this happens here: https://github.com/aiidateam/pgsu/blob/master/pgsu/__init__.py#L294

we explicitly set the stderr of a subprocess to subprocess.STDOUT, which does not respect any reassignments to sys.stdout (e.g. by the redirect_stderr context manager). This is expected and documented behaviour of the subprocess module.

Would it be possible to have stderr redirected to a log, so that it may be managed with standard logging infra?

Ubuntu environment on WSL is not detected

Related to aiidateam/aiida-core#4748. The the sudo is not only needed for quicksetup but also other things like verdi profile delete etc.

Because WSL does not have version keys containing Ubuntu, the use of sudo can not be set.

Perhaps there the package can use an environmental variable to force the use of sudo a workaround.

platform.version gives

  • '#488-Microsoft Mon Sep 01 13:43:00 PST 2020' on my WSL1 system
  • '#1 SMP Wed Nov 13 11:44:37 UTC 2019' on my WSL2 system

fix azure pipeline

  • figure out how to create DB dir & start cluster when running tests with conda
    (essentially look into what pgtest does and copy that)
  • make the homebrew build with MacOS work

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.