Git Product home page Git Product logo

pg_auto_reindexer's People

Contributors

ivan avatar vitabaks avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar

Forkers

crztssr dbigunyak tuananh

pg_auto_reindexer's Issues

add retry after the reindex is canceled due to lock_timeout

At the moment, the script stops if it fails to take the lock during the specified lock_timeout

  • 1) it is necessary to add several retries (for example, after 10, 30, 60 sec. I think 3 attempts will be enough).
  • 2) do not stop the script if there is an error in the first index, try maintenance of the next index. And terminate database maintenance if we get an error for N indexes in a row.
  • 3) do not stop the script if there is an error in the first database, continue working with the next database (relevant if there are several databases).

Invalid indexes leftovers on retries 

2022-12-05 14:15:46 INFO:   completed repack index public.access_person_user_id_key (size after: 1 MB)
2022-12-05 14:15:49 INFO:   repack index public.idx_to_repack (size: 3 MB)
2022-12-05 14:15:50 WARN:   WARNING: Error creating index "public"."index_17767": ERROR:  canceling statement due to lock timeout
2022-12-05 14:16:00 INFO:   [retry 1/3] repack index public.idx_to_repack (size: 3 MB)
2022-12-05 14:16:00 WARN:   WARNING: Cannot create index "public"."index_17767", already exists
2022-12-05 14:16:30 INFO:   [retry 2/3] repack index public.idx_to_repack (size: 3 MB)
2022-12-05 14:16:30 WARN:   WARNING: Cannot create index "public"."index_17767", already exists
2022-12-05 14:17:30 INFO:   [retry 3/3] repack index public.idx_to_repack (size: 3 MB)
2022-12-05 14:17:30 WARN:   WARNING: Cannot create index "public"."index_17767", already exists
2022-12-05 14:17:30 WARN:   failed to repack index "public.idx_to_repack" after all attempts. Skipping
2022-12-05 14:17:30 INFO:   repack index public.another_idx_to_repack (size: 3 MB)
2022-12-05 14:17:30 INFO:   completed repack index public.another_idx_to_repack (size after: 2 MB)
2022-12-05 14:17:30 WARN: A temporary index apparently created by pg_repack has been left behind, and we do not want to risk dropping this index ourselves. If the index was in fact created by an old pg_repack job which didn't get cleaned up, you should just use DROP INDEX CONCURRENTLY and try the pg_auto_reindexer again.
2022-12-05 14:17:30 WARN: invalid indexes: public.index_17767
2022-12-05 14:17:30 INFO: Total amount released during maintenance: 0 MB

Code seems to work without issues as expected, but probably need to change behaviour in case of pg_repack.
Options:

  1. Drop invalid index, in case it matches pattern indexrelname like 'index%'"_, but as the code say, it might be dangerous. In worst case scenario system may contain invalid index which matches the pattern, but is not related to pg_repack.
  2. Prohibit retries in case we are using pg_repack and got lock timeout error
  3. Leave it as it is, since script is working, just spending a bit more time (making excessive tries, when repack will not be possible anyway).

Connect via unix socket

I have my pg_hba.conf configured in the way that superuser can only connect via peer auth and local unix socket. So that feature is highly desirable.

Ignore pg_catalog indexes

pg_auto_reindexer attempts to process pg_catalog indexes:

root@hostname:~# pg_auto_reindexer --dbname=dbname --index_bloat=80                                                                                      
2021-04-27 23:54:16 INFO: Start index maintenance for database: dbname
2021-04-27 23:54:17 INFO: repack index pg_catalog.pg_attribute_relid_attnum_index                                                                            
2021-04-27 23:54:19 INFO: repack index pg_catalog.pg_depend_reference_index
2021-04-27 23:54:21 INFO: repack index pg_catalog.pg_depend_depender_index
...

I suppose they should be ignored.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.