Would you consider adding a non-blocking version of Acquire to this API? Instead of blocking until a resource becomes available, the API would fail if there aren't any available and there is no headroom to construct a new resource. Perhaps it could be named TryAcquire, AcquireNow, or AquireOrFail.

This behavior could be approximated by supplying a context with a short lifespan, but that feels clunky. And, since the context is passed into the resource constructor, it could potentially cancel an otherwise valid resource construction.

The use case I'm designing for is a scheduler that schedules work based on what resources are available. So, it would be ideal for my application to do something else if a requested resource is unavailable.

I would be happy to submit a pull request if you have any interest in adding something like that to this library. Just figured I'd ask before replicating this elsewhere.

constructor := func(ctx context.Context) (interface{}, error) {
	return net.DialTimeout("tcp", addr, 5*time.Second)
}

When you want to create a network connection, but the network is unavailable.
DialTimeout will be blocked 5 seconds,and TryAcquire will be blocked 5 seconds also.

Go 1.18 with generics is released.
They can improve library usage experience (no type casting needed, etc).
Supporting them require a bit of changes (I can send PR if needed) but breaks backwards compatibility (v2 needed).

UPD: Found way to save backwards compatibility. See in PR.

Hi could you do a new release with your fixes?

package main

import (
	"context"
	"log"
	"os/signal"
	"syscall"

	"github.com/jackc/pgx/v5/pgxpool"
	"golang.org/x/sync/errgroup"
)

func main() {
	ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
	defer stop()

	db, err := pgxpool.New(ctx, "postgresql://XXX")
	if err != nil {
		log.Panicln(err)
	}
	g, ctx := errgroup.WithContext(ctx)
	g.SetLimit(8)

	loop := func() {
		for {
			select {
			case <-ctx.Done():
				return
			default:
			}
			g.Go(func() error {
				log.Println("Going to db.Exec")
				_, err := db.Exec(ctx, "select pg_sleep(0.2);")
				return err
			})
		}
	}

	loop()
	db.Close()
}

When running this script with go run main.go, if I hit ctrl+C, the program will sometimes hang on the db.Close() and more especially on the p.destructWG.Wait() line. It's pretty reliable, though you may want to try something like 10 times (go run + ctrl+c).

I couldn't debug it more :( I am on Linux go 1.19.

Edit: I noticed that if I hit ctrl+c quite early after the program started, there's a higher chance to reproduce the deadlock.

Hi.
pool.go

func (p *Pool) Acquire(ctx context.Context) (*Resource, error) {
                        ...

                        **p.allResources** = append(p.allResources, res)
			**p.destructWG.Add(1)**
			p.cond.L.Unlock()
			value, err := p.constructResourceValue(ctx)
			p.cond.L.Lock()
			if err != nil {
				p.allResources = removeResource(p.allResources, res)
				**p.destructWG.Done()**
                        ...
}

func (p *Pool) Close() {
    ...
    for _, res := range **p.idleResources** {
		p.allResources = removeResource(p.allResources, res)
		go p.destructResourceValue(res.value)
   }
  // Different WaitGroup counter
    ...
    **p.destructWG.Wait()**
    // process freezes because
    //  len(p.idleResources) != len(p.allResources)
}

The easiest way to solve:

func (p *Pool) Close() {
    for _, res := range **p.idleResources** {
		p.allResources = removeResource(p.allResources, res)
		go p.destructResourceValue(res.value)
   }
   for _, res := range p.allResources {
		p.allResources = removeResource(p.allResources, res)
		p.destructWG.Done()
    }

...
}

after running "make run" in the terminal I get the error "github.com/jackc/puddle/v20v2.2.1/pool.go:142:30: undefined: atomic.Int64", please can someone assist?

I am using Go 1.22 and running it on an Ubuntu 23.10 VM

Thank you

Repro:

func TestDeadlock(t *testing.T) {
	var pool *puddle.Pool

	pool = puddle.NewPool(
		func(ctx context.Context) (res interface{}, err error) {
			t.Log("closing pool")
			pool.Close()
			t.Log("closed pool")
			return 1, nil
		}, func(res interface{}) {
			t.Log("releasing resource")
		}, 10,
	)
	t.Log("creating resource before closing pool")
	err := pool.CreateResource(context.Background())
	if !errors.Is(err, puddle.ErrClosedPool) {
		panic(fmt.Errorf("unxpected error %v", err))
	}

	t.Log("creating resource after closing pool")
	pool.CreateResource(context.Background())
}

Example: https://play.golang.org/p/EaWMf76kzLk

Why:

	p.cond.L.Lock()
	// If closed while constructing resource then destroy it and return an error
	if p.closed {
		go p.destructResourceValue(res.value)
		return ErrClosedPool // <-- forgot to release lock
	}

I want to know, are you open to an MR to give the possibility to resize the pool size on the fly?

If you're up for it, I can create a MR. Thanks!

We rotate credentials on the fly, this would include credentials for AWS, databases, google cloud platform, etc.

is there a way to automatically delete the resources that were created more than a certain number of minutes ago? Or is that a bad idea? I was also thinking, we could Destroy IF the resource results in an error, but that'd mean if there's a genuine error happening, then we'd drain our pool of resources.

Do you have any recommendations on how we would go about this? :), currently, I'm using a resource, then checking if it was created more than a certain number of minutes ago, then destroying it.

in file pool.go

			select {
			case <-ctx.Done():
				// Allow goroutine waiting for signal to exit. Re-signal since we couldn't
				// do anything with it. Another goroutine might be waiting.
				go func() {
					<-waitChan
					p.cond.Signal()
					p.cond.L.Unlock()
				}()

				p.cond.L.Lock()
				p.canceledAcquireCount += 1
				p.cond.L.Unlock()
				return nil, ctx.Err()
			case <-waitChan:
			}

when ctx Done, it will try to Lock before return.
Consider such a situation, when the maxSize is exceeded, the Acquire function is called again. If no resources have been released and ctx has timed out, the above logic is triggered. If other resources have not been released, it will wait until a new resource is released, triggering the unlocking.
I personally believe that in practical applications, most of the data statistics are completed by the monitoring system. Maybe the Stat here is not very useful, but it has increased the complexity of locking and logic

Hey,

the recent commit

86ffb01

introduces 1.19 requirement, however, pgx itself still requires 1.18:

https://github.com/jackc/pgx/blob/master/go.mod

But it pulls puddle/v2 via replace:

jackc/pgx@c00fb5d

This breaks the builds for Go 1.18:

../../go/pkg/mod/github.com/jackc/puddle/[email protected]/pool.go:142:30: undefined: atomic.Int64
note: module requires Go 1.19

While I understand how hard is to maintain compatibility and we are trying hard to catchup with Go upstream as quickly as possible, it's not always ideal (e.g. our infra team requires a specific Go version/build). So currently we are on Go 1.18. We'd really appreciate support for last two major Go versions, which is also the Google's policy:

https://endoflife.date/go

When this can't be done, well, then I think pgx needs a bump of minimum Go version then. This was done in a minor release (5.0.4) thus I am creating this issue to make sure this was not accidental.

Thanks!