Git Product home page Git Product logo

pgsql_admin_script's Introduction

pgsql_admin_script

常用管理脚本

  1. generate_report.sh # 用来生成数据库巡检报告, 注意可以使用lock_timeout和statement_timeout来避免长时间等待。
  2. pgq.md # 用来实现数据逻辑增量复制。
  3. pgq_case.md # 在线数据逻辑复制。
  4. pgsql_perf_tuning.md # PostgreSQL TPC-C 极限优化。

pgsql_admin_script's People

Contributors

digoal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pgsql_admin_script's Issues

可选参数 -t

可以通过在psql 命令行中添加 -t 参数让查询结果中仅显示数据信息而不显示字段信息,方便在psql执行语句返回值传入shell等脚本进行判断和比较用。

sshd_config

MaxStartups 1000:30:3000
防止GP大量使用ssh连接时出现问题

PG10中《数据库XLOG, 流复制状态分析》这部分的几个SQL有改动

psql --pset=pager=off -q -c 'select name,setting from pg_settings where name in ($$archive_mode$$,$$autovacuum$$,$$archive_command$$)'
echo "建议: "
echo " 建议开启自动垃圾回收, 开启归档. "
echo -e "\n"
echo "----->>>---->>> 归档统计信息: "
psql --pset=pager=off -q -c 'select pg_xlogfile_name(pg_current_xlog_location()) now_xlog, * from pg_stat_archiver'
echo "建议: "
echo " 如果当前的XLOG文件和最后一个归档失败的XLOG文件之间相差很多个文件, 建议尽快排查归档失败的原因, 以便修复, 否则pg_xlog目录可能会撑爆. "
echo -e "\n"
echo "----->>>---->>> 流复制统计信息: "
psql --pset=pager=off -q -x -c 'select pg_xlog_location_diff(pg_current_xlog_location(),flush_location), * from pg_stat_replication'
echo "建议: "
echo " 关注流复制的延迟, 如果延迟非常大, 建议排查网络带宽, 以及本地读xlog的性能, 远程写xlog的性能. "
echo -e "\n"
echo "----->>>---->>> 流复制插槽: "
psql --pset=pager=off -q -c 'select pg_xlog_location_diff(pg_current_xlog_location(),restart_lsn), * from pg_replication_slots'

这里用到的3个SQL在PG10中有改动

-- PG9
select pg_xlogfile_name(pg_current_xlog_location()) now_xlog, * from pg_stat_archiver;

-- PG10
select pg_walfile_name(pg_current_wal_lsn()) now_xlog, * from pg_stat_archiver;
--PG9
select pg_xlog_location_diff(pg_current_xlog_location(),flush_location), * from pg_stat_replication

--PG10
select pg_wal_lsn_diff(pg_current_wal_lsn(),flush_lsn), * from pg_stat_replication;
--PG9
select pg_xlog_location_diff(pg_current_xlog_location(),restart_lsn), * from pg_replication_slots;

--PG10
select pg_wal_lsn_diff(pg_current_wal_lsn(),restart_lsn), * from pg_replication_slots;

完善简单密码检查

完善简单密码检查 .
cracklib 中的密码为$simple_pwd
数据库用户名为$username
检查pg_authid.rolpassword 是否和 'md5'||md5($simple_pwd||$username) 匹配,如果匹配,说明用户使用了简单密码。

检查文件系统快照

如果是btrfs或者zfs,检查一下是否有通过快照来备份,
以及检查快照的有效性(pg_start_backup()后产生的快照)
快照的可恢复性(快照后产生的wal全部保留)

表膨胀检测SQL中hdr和ma的疑问?

(SELECT current_setting($$block_size$$)::numeric) AS bs,
CASE WHEN SUBSTRING(SPLIT_PART(v, $$ $$, 2) FROM $$#"[0-9]+.[0-9]+#"%$$ for $$#$$)
IN ($$8.0$$,$$8.1$$,$$8.2$$) THEN 27 ELSE 23 END AS hdr,
CASE WHEN v ~ $$mingw32$$ OR v ~ $$64-bit$$ THEN 8 ELSE 4 END AS ma
FROM (SELECT version() AS v) AS foo
) AS constants

德哥我在分析你给出的表膨胀SQL的时候,看到内部这个SQL中会输出:

  bs  | hdr | ma 
------+-----+----
 8192 |  23 |  8
(1 row)

SQL逻辑也比较简单,bs我能看出来是块大小,hdr是根据版本直接给出的,ma根据version 64bit给出的。这里的hdr和ma涉及到后面SQL的很多计算式,比如

...
(datawidth+(hdr+ma-(case when hdr%ma=0 THEN ma ELSE hdr%ma END)))::numeric
...
(maxfracsum*(nullhdr+ma-(case when nullhdr%ma=0 THEN ma ELSE nullhdr%ma END)))
...

想问一下这个hdr和ma是什么的缩写,代表什么意思?

ps. 我在分析表膨胀的长SQL,看到这个hdr和ma后面分析不下去了 :(
ps. 文章链接:https://github.com/mutex73/blog/blob/master/src/pgsql-sql-bloatsql.md

@digoal

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.