Comments (12)
I don't resolve it. Just found this issue yesterday when moving to a new laptop. Decided to continue work in my old environment so far.
from embedded-postgres.
When you see this message, you should do waht @Minsoo-Shin did just above. You should hand run the same initdb command and see what the actual error message is. We also saw the shmget failure. We've seen this error after successive builds using this package. In all cases, we were failing to shutdown the postgres embedded instance (calling stop). You can check ps
to see if you have a bunch of zombie postgres instances running and kill them. You can also use ipcs -a
on OSX to see what shmem segments might be outstanding. If that is your problem, you need to fix the places where you aren't shutting this down.
Note that this is not a problem in container build envs because zombie processes are just thrown away when the container is. However, if you're running on a machine without a container (e.g. OSX), failing to shutdown the Postgres instance will cause many times will ultimately result in depletion of available shared memory and shmem failures. A reboot fixing the problem is also a good indicator of this.
from embedded-postgres.
Hey @technicallyty we've just cut a release v1.20.0 which adds better support for Apple M1/2. I'd be interested to see if you fair any better with this and Postgres v14.2+ which should offer native support.
We also have some upcoming changes to improve logging, including displaying errors a little more clearly in this case.
from embedded-postgres.
Hey @technicallyty. I've not seen this before but I am aware of some issues running with Apple M1.
Could you post the Go code you used to produce this error?
Have you also been able to try this on a non Apple M1 device?
from embedded-postgres.
here's the code which setups up the embedded pg
package dbtest
import (
"fmt"
"io"
"testing"
embeddedpostgres "github.com/fergusstrange/embedded-postgres"
"github.com/phayes/freeport"
"github.com/rs/zerolog"
"github.com/stretchr/testify/require"
"github.com/regen-network/toucan-bridge-service/db"
)
// NewTestDatabase creates an embedded postgres database for testing purposes.
func NewTestDatabase(t *testing.T) db.Database {
port, err := freeport.GetFreePort()
require.NoError(t, err)
logger := zerolog.New(zerolog.NewConsoleWriter())
cfg := embeddedpostgres.DefaultConfig().
Port(uint32(port)).
Database("postgres").
Logger(logger).
RuntimePath(t.TempDir())
postgres := embeddedpostgres.NewDatabase(cfg)
require.NoError(t, postgres.Start())
t.Cleanup(func() {
require.NoError(t, postgres.Stop())
})
customDbWrapper, err := db.NewDatabase(
fmt.Sprintf("host=localhost port=%d user=postgres password=postgres dbname=postgres sslmode=disable", port),
logger,
)
require.NoError(t, err)
return customDbWrapper
}
Have you also been able to try this on a non Apple M1 device?
I have not
edit: i have never seen this in our CI system / github actions, though. i think it runs it in a linux container.
from embedded-postgres.
I have similar issue on M2 chip:
unable to init database using '/var/folders/j2/_f8t7nhd1zx2ycjy6nymcddc0000gn/T/embedded_postgres_test_lib_2003912407/bin/initdb -A password -U postgres -D /var/folders/j2/_f8t7nhd1zx2ycjy6nymcddc0000gn/T/embedded_postgres_test_data_2418798707 --pwfile=/var/folders/j2/_f8t7nhd1zx2ycjy6nymcddc0000gn/T/embedded_postgres_test_runtime_1981438580/pwfile': exit status 1
same code on x86_64 platform works without any issues
from embedded-postgres.
I have similar issue on M2 chip:
unable to init database using '/var/folders/j2/_f8t7nhd1zx2ycjy6nymcddc0000gn/T/embedded_postgres_test_lib_2003912407/bin/initdb -A password -U postgres -D /var/folders/j2/_f8t7nhd1zx2ycjy6nymcddc0000gn/T/embedded_postgres_test_data_2418798707 --pwfile=/var/folders/j2/_f8t7nhd1zx2ycjy6nymcddc0000gn/T/embedded_postgres_test_runtime_1981438580/pwfile': exit status 1
same code on x86_64 platform works without any issues
how do you usually resolve this issue? Do you just restart your laptop as well? Hoping you may have found another way so I don't have to keep restarting my machine to run tests 😆
from embedded-postgres.
+1, having issues on the M1 Max
as well as in github action CI using ubuntu-latest
. Any logs I can provide to help debug?
embeddedDB := embeddedpostgres.NewDatabase(embeddedpostgres.DefaultConfig().
Username(dbConfig.GetUser()).
Password(dbConfig.GetPassword()).
Database(dbConfig.GetDBName()).
Port(uint32(dbConfig.GetPort())),
)
if err := embeddedDB.Start(); err != nil {
return nil, err
}
from embedded-postgres.
/Users/ms/.embedded-postgres-go/extracted/bin/initdb -A password -U acloset -D /Users/ms/.embedded-postgres-go/extracted/data --pwfile=/Users/ms/.embedded-postgres-go/extracted/pwfile
The files belonging to this database system will be owned by user "ms".
This user must also own the server process.
The database cluster will be initialized with locale "ko_KR.UTF-8".
The default database encoding has accordingly been set to "UTF8".
initdb: could not find suitable text search configuration for locale "ko_KR.UTF-8"
The default text search configuration will be set to "simple".
Data page checksums are disabled.
creating directory /Users/ms/.embedded-postgres-go/extracted/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 20
selecting default shared_buffers ... 400kB
selecting default time zone ... Asia/Seoul
creating configuration files ... ok
running bootstrap script ... 2022-11-25 16:12:44.266 KST [4737] FATAL: could not create shared memory segment: Cannot allocate memory
2022-11-25 16:12:44.266 KST [4737] DETAIL: Failed system call was shmget(key=16783864, size=56, 03600).
2022-11-25 16:12:44.266 KST [4737] HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMALL parameter. You might need to reconfigure the kernel with larger SHMALL.
The PostgreSQL documentation contains more information about shared memory configuration.
child process exited with exit code 1
initdb: removing data directory "/Users/ms/.embedded-postgres-go/extracted/data"
i think this is because that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMALL parameter
we have to configuring shared memory in macOS is to create a file named /etc/sysctl.conf,
https://www.postgresql.org/docs/13/kernel-resources.html
from embedded-postgres.
Facing the same issue on Windows 11:
unable to init database using: <MY_PATH>\.embedded-postgres-go\extracted\bin\initdb.exe -A password -U postgres -D <MY_PATH>\.embedded-postgres-go\extracted\data --pwfile=<MY_PATH>\.embedded-postgres-go\extracted\pwfile exit status 1
On Windows 10 and Linux still working.
UPDATE
Trying to debug: Executing the bin\initdb.exe under the compressed folder I got this error: VCRUNTIME140.dll not found.
So I download this Visual C++ package from Microsoft:
https://www.microsoft.com/pt-BR/download/details.aspx?id=48145
Then my problem on Windows 11 was solved.
from embedded-postgres.
Release now made for improved logging at v1.21.0. I'm going to close this now. Please reopen should you continue to have issues.
from embedded-postgres.
Release now made for improved logging at v1.21.0. I'm going to close this now. Please reopen should you continue to have issues.
late to reply, but this version works for me on m1 👍🏾
from embedded-postgres.
Related Issues (20)
- Unexported field 'version' usage HOT 3
- Error when DataBase is Specified in Config HOT 1
- Expose an interface to retrieve binaries from non-network sources HOT 5
- List of all supported versions? - From Maven? HOT 2
- twirp error internal: conn closed HOT 1
- Getting 'creating directory /Users/MyUser/.embedded-postgres-go/extracted/data ... initdb: error: could not create directory "/Users/MyUser/.embedded-postgres-go/extracted/data": File exists' when running tests that use embedded postgres in parallel HOT 1
- How to shutdown an existing PG instance? HOT 1
- Absolutely not concurrency-safe HOT 3
- CLI release for PostgreSQL version manager [embedding your project] HOT 1
- Process already running in port when test panics using embedded postgres HOT 6
- how to change r/w permission? HOT 7
- Metadata tables missing HOT 1
- Jar unzip fails when server does not set ContentLength HOT 1
- syntax error at or near "-" (SQLSTATE 42601) HOT 1
- Support for dynamic port allocation HOT 4
- postgres version 13.11 has multiple high CVEs: CVE-2023-39417, CVE-2023-5869, CVE-2024-0985 HOT 1
- unable to init database FATAL: could not create shared memory segment: No space left on device HOT 1
- Run as a different user (non-root) HOT 11
- Race in remote fetcher can cause parallel tests to fail HOT 11
- Get error "unable to extract postgres archive: xz: data is truncated or corrupt" in GitLab pipeline. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from embedded-postgres.