Git Product home page Git Product logo

Comments (7)

corydolphin avatar corydolphin commented on August 20, 2024 2

@stevenschlansker is there an easier way to adopt some of the Postgres suggested configurations? I'm interested to see how disabling fsync and synchronous_commit impacts perf.

Also, it would be nice to disable auto vacuum to reduce the number of processes booted.

from otj-pg-embedded.

athimel avatar athimel commented on August 20, 2024

+1

from otj-pg-embedded.

athimel avatar athimel commented on August 20, 2024

I think it's a good idea, even if you may just configure your runtime to put files in a tmpfs folder

from otj-pg-embedded.

stevenschlansker avatar stevenschlansker commented on August 20, 2024

Unfortunately the solution linked requires root privileges. I don't think this component can assume that the user has root privilege.

Are you aware of any portable way to do this without elevating privileges? Unfortunately I am not.

from otj-pg-embedded.

archenroot avatar archenroot commented on August 20, 2024

Well, there are 2 ways of how to let PostgreSQL working in RAM:

  1. The more easier way is (from my perspective) to mount tmpfs by asking root user to add entry into /etc/fstab
  2. This is unsecure solution. Access /dev/shm or /run/shm if available on your system. Create a directory in this destination and symlink it somewhere, init the database by something like (I am on Gentoo linux, not sure how this works on Ubuntu or Centos):
# mkdir -p /run/pgsql-{version}-data
# chown -R postgres:postgres /run/pgsql-9.4.5-data
# chmod 700 /run/pgsql-{version}-data
# su - postgres
$ /usr/local/pgsql-{version}/bin/initdb \
    --pgdata=/run/pgsql-{version}-data \
    --encoding=UTF8 \
    --no-locale

3.But there are some other "preffered" ways of doing this with Postgres:

Durability is a database feature that guarantees the recording of committed transactions even if the server crashes or loses power. However, durability adds significant database overhead, so if your site does not require such a guarantee, PostgreSQL can be configured to run much faster. The following are configuration changes you can make to improve performance in such cases. Except as noted below, durability is still guaranteed in case of a crash of the database software; only abrupt operating system stoppage creates a risk of data loss or corruption when these settings are used.

Place the database cluster's data directory in a memory-backed file system (i.e. RAM disk). This eliminates all database disk I/O, but limits data storage to the amount of available memory (and perhaps swap).

Turn off fsync; there is no need to flush data to disk.

Turn off synchronous_commit; there might be no need to force WAL writes to disk on every commit. This setting does risk transaction loss (though not data corruption) in case of a crash of the database.

Turn off full_page_writes; there is no need to guard against partial page writes.

Increase max_wal_size and checkpoint_timeout ; this reduces the frequency of checkpoints, but increases the storage requirements of /pg_xlog.

Create unlogged tables to avoid WAL writes, though it makes the tables non-crash-safe.

from otj-pg-embedded.

archenroot avatar archenroot commented on August 20, 2024

So I suggest doing changes in this file:
repack-postgres.sh

from otj-pg-embedded.

stevenschlansker avatar stevenschlansker commented on August 20, 2024

Right, notice how both of those approaches require root privilege. I'm happy to add support for this but it has to be opt in as you cannot assume most users have root privileges.

from otj-pg-embedded.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.