Comments (5)
if a client is taken out of the collection pool it can't be used until it is put back into it. If you're creating a massive amount of concurrent requests then the pool is constantly empty and so new connections need to be made. You're then hitting postgres' limit for connections.
You can try increasing the maximum connection limit from postgres if you know what kind of traffic to expect in additional to adding rate limiting in your nginx config.
In your specific example, I recommend caching a connection across the request, instead of pulling one for every query.
We currently don't have a programmatic way of dealing with the "sorry, too many clients already" error. One option would be to sleep and try again later until we can get a new connection or pull one from the pool.
from pgmoon.
@leafo By saying so new connections need to be made
, do you mean if the pool is constantly empty in my example (50 maxium), pgmoon will still try to make a new connection to database instead of simply refusing to do that?
Does the pool size
of pgmoon's keepalive function have to (or the recommend way )be smaller than the max_connections
of postgres?
test=# show max_connections;
max_connections
-----------------
100
(1 row)
from pgmoon.
correct, nginx cosockets do not block you from creating a new connection even if the pool is empty. pgmoon doesn't have it's own pooling mechanism, it's directly calling https://github.com/openresty/lua-nginx-module#tcpsocksetkeepalive
the amount of connections you're going to need in postgres depnds on concurrent traffic, but you'll want minimally the pool size you set in lua
from pgmoon.
Ok, thanks.
from pgmoon.
@leafo I suddenly realized I should use disconnect
when error happens in query
. Now I won't get too many clients already error. But I'm not sure whether disconnect
on error is OK
local function query(statement)
local db, res, ok, err
db = pg_driver.new(CONNECT_TABLE) -- always success
db:settimeout(CONNECT_TIMEOUT)
res, err = db:connect()
if not res then
return nil, err
end
local a, b, c, d, e = db:query(statement)
if a ~= nil and db.sock_type == 'nginx' then
ok, err = db:keepalive(max_idle_timeout, pool_size)
if not ok then
return nil, 'fail to keepalive:'..err
else
return a
end
else
ok, err = db:disconnect() -- non-openresty error
if not ok then
return nil, 'fail to disconnect:'..err
else
return a, b, c, d, e
end
end
end
from pgmoon.
Related Issues (20)
- support ipv6 by using luasocket HOT 1
- What should I do if error happens when using pg:query ? HOT 1
- Implement extended query protocol HOT 1
- attempt to call method 'keepalive' (a nil value) HOT 1
- [PostgreSQL 14.1] Why is the connection wrong and the encryption method is scram-sha-256? HOT 2
- It works in neovim!
- unable to read data from frontend HOT 4
- bad request on repeated requests HOT 3
- multi-hosts HOT 1
- pgmoon support automatic master-slave switch HOT 1
- Add locking to query functions HOT 1
- authentication exchange unsuccessful in postgresql 15 scram-sha-256 in scram_sha_256_auth
- lua entry thread aborted: runtime error: attempt to yield across C-call boundary HOT 2
- multiple statements in one query HOT 2
- Support all array types
- postgres:query() does not return error as 2. parameter
- Socket pool getreusedtimes function HOT 1
- Pool violation after v1.13 HOT 2
- Connection pooling outside of OpenResty?
- which pgmoon version can connect to postgresql 15 ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pgmoon.