Git Product home page Git Product logo

postgres's People

Contributors

0marq avatar a631807682 avatar alessandrosechi avatar alidevhere avatar asmeikal avatar black-06 avatar dependabot[bot] avatar dino-ma avatar izoyo avatar jasonchuan avatar jeffry-luqman avatar jimmyduang avatar jinzhu avatar kainonly avatar long2ice avatar luckylhb90 avatar moredure avatar mrchaos1993 avatar notofir avatar pierre-borckmans avatar ppanphper avatar ruanlang avatar rwrz avatar s-takehana avatar saeidee avatar shedyfreak avatar toctan avatar warren830 avatar x1nchen avatar yuxinyin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

postgres's Issues

UUID Package Introduces Vulnerability

Playground link - N/A

The package github.com/satori/[email protected] introduces a vulnerabiltiy to the postgres driver for gorm, meaning we cannot use the default postgres driver and would need to write our own driver to get past this. Wondering if it would be possible to have the dependency github.com/satori/[email protected] updated or removed.

Details of vulnerability:
Affected versions of this package are vulnerable to Insecure Randomness producing predictable UUID identifiers due to the limited number of bytes read when using the g.rand.Read function.

Use postgres double precision for Golang float64

It looks like fields declared as float without a scale or precision are migrated/created as SQL decimal:

postgres/postgres.go

Lines 140 to 147 in e552243

case schema.Float:
if field.Precision > 0 {
if field.Scale > 0 {
fmt.Sprintf("numeric(%d, %d)", field.Precision, field.Scale)
}
return fmt.Sprintf("numeric(%d)", field.Precision)
}
return "decimal"

Which is an arbitrary precision type that is identical to numeric, according to the docs:

The types decimal and numeric are equivalent. Both types are part of the SQL standard.

Using numeric for fields with defined scale/precision makes sense - those are the only arbitrary precision types - but it creates a disconnect between Golang floats and their database analogs in other cases. Namely, floating point numbers such as SQL real and double precision, which are analogous to Go's float32 and float64, respectively, are defined by IEEE 754 - as is their behavior.

  • Floating point numbers don't always translate exactly to arbitrary precision (the opposite is obviously also true) - sometimes it's nice to not have an extra .00000007 appended to our values
  • Arbitrary precision types in Postgres have nonstandard mathematical behavior (eg with rounding) - it's nice to know that moving simple maths to Postgres won't change the result of the equation
  • Arbitrary precision math is extremely slow, which is one of the main advantages of fixed precision - do you use Go's big.Float every time you want to add a couple of real numbers?

That all being said, is there a reason for this behavior? It would be nice to change it, but I understand that would have far-reaching consequences - you can't alter a column from arbitrary to fixed precision!

gorm/postgres driver1.4.6 and pgx v5 in parallel panics

Using Gorm with the Postgres driver 1.4.6 and pgx@v5 in parallel within a program leads to a panic (see below). With version 1.4.5 or lower parallel using works.

`panic: sql: Register called twice for driver pgx

goroutine 1 [running]:
database/sql.Register({0x17c8c7e, 0x3}, {0x18bb180, 0xc00012b098})
/usr/local/opt/go/libexec/src/database/sql/sql.go:51 +0x13d
github.com/jackc/pgx/v5/stdlib.init.0()
/Users/DrKalb/.go/pkg/mod/github.com/jackc/pgx/[email protected]/stdlib/sql.go:88 +0x85

Process finished with the exit code 2
`

fail to alter column

GORM Playground Link

go-gorm/playground#475

Description

If we want to change column from varchar to int, it will report error like below:
[2022-05-09 11:57:51] INFO [db] /opt/homebrew/Cellar/[email protected]/1.17.9/pkg/mod/gorm.io/driver/[email protected]/migrator.go:270 ERROR: column "xxx" cannot be cast automatically to type bigint (SQLSTATE 42804)
[1.840ms] [rows:0] ALTER TABLE "xxx" ALTER COLUMN "xxx" TYPE bigint

We only need to append USING xxx::bigint in func AlterColumn in [email protected]/migrator.go

Panic with `uint8` column types

Sorry for skipping the playground PR, but I think the below snippet should explain the issue well enough.

Example

package main

import (
	"gorm.io/driver/postgres"
	"gorm.io/gorm"
	"gorm.io/gorm/logger"
)

type Foo struct {
	ID     uint  `gorm:"primary_key"`
	Field1 uint8 `gorm:"not null; index:idx_1"`
}

func main() {
	db, err := gorm.Open(postgres.New(postgres.Config{
		DSN: "host=localhost port=5432 user=gorm_debug dbname=gorm_debug password=gorm_debug",
	}), &gorm.Config{
		Logger: logger.Default.LogMode(logger.Info),
	})
	if err != nil {
		panic(err)
	}

	if err := db.AutoMigrate(&Foo{}); err != nil {
		panic(err)
	}
}

Here's the output:

2022/04/03 18:00:11 /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:145
[2.766ms] [rows:1] SELECT count(*) FROM information_schema.tables WHERE table_schema = CURRENT_SCHEMA() AND table_name = 'foos' AND table_type = 'BASE TABLE'

2022/04/03 18:00:11 /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:20
[0.381ms] [rows:1] SELECT CURRENT_DATABASE()

2022/04/03 18:00:11 /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:330
[9.333ms] [rows:-] SELECT c.column_name, c.is_nullable = 'YES', c.udt_name, c.character_maximum_length, c.numeric_precision, c.numeric_precision_radix, c.numeric_scale, c.datetime_precision, 8 * typlen, c.column_default, pd.description FROM information_schema.columns AS c JOIN pg_type AS pgt ON c.udt_name = pgt.typname LEFT JOIN pg_catalog.pg_description as pd ON pd.objsubid = c.ordinal_position AND pd.objoid = (SELECT oid FROM pg_catalog.pg_class WHERE relname = c.table_name AND relnamespace = (SELECT oid FROM pg_catalog.pg_namespace WHERE nspname = c.table_schema)) where table_catalog = 'gorm_debug' AND table_schema = CURRENT_SCHEMA() AND table_name = 'foos'

2022/04/03 18:00:11 /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:461
[0.457ms] [rows:-] SELECT * FROM "foos" LIMIT 1

2022/04/03 18:00:11 /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:400
[9.246ms] [rows:-] SELECT c.column_name, constraint_type FROM information_schema.table_constraints tc JOIN information_schema.constraint_column_usage AS ccu USING (constraint_schema, constraint_name) JOIN information_schema.columns AS c ON c.table_schema = tc.constraint_schema AND tc.table_name = c.table_name AND ccu.column_name = c.column_name WHERE constraint_type IN ('PRIMARY KEY', 'UNIQUE') AND c.table_catalog = 'gorm_debug' AND c.table_schema = CURRENT_SCHEMA() AND c.table_name = 'foos'

2022/04/03 18:00:11 /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:430
[0.852ms] [rows:-] SELECT a.attname as column_name, format_type(a.atttypid, a.atttypmod) AS data_type
                FROM pg_attribute a JOIN pg_class b ON a.attrelid = b.relfilenode AND relnamespace = (SELECT oid FROM pg_catalog.pg_namespace WHERE nspname = CURRENT_SCHEMA())
                WHERE a.attnum > 0 -- hide internal columns
                AND NOT a.attisdropped -- hide deleted columns
                AND b.relname = 'foos'

2022/04/03 18:00:11 /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:218
[5.948ms] [rows:0] SELECT description FROM pg_catalog.pg_description WHERE objsubid = (SELECT ordinal_position FROM information_schema.columns WHERE table_schema = CURRENT_SCHEMA() AND table_name = 'foos' AND column_name = 'id') AND objoid = (SELECT oid FROM pg_catalog.pg_class WHERE relname = 'foos' AND relnamespace = (SELECT oid FROM pg_catalog.pg_namespace WHERE nspname = CURRENT_SCHEMA()))

2022/04/03 18:00:11 /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:20
[0.200ms] [rows:1] SELECT CURRENT_DATABASE()

2022/04/03 18:00:11 /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:330
[5.446ms] [rows:-] SELECT c.column_name, c.is_nullable = 'YES', c.udt_name, c.character_maximum_length, c.numeric_precision, c.numeric_precision_radix, c.numeric_scale, c.datetime_precision, 8 * typlen, c.column_default, pd.description FROM information_schema.columns AS c JOIN pg_type AS pgt ON c.udt_name = pgt.typname LEFT JOIN pg_catalog.pg_description as pd ON pd.objsubid = c.ordinal_position AND pd.objoid = (SELECT oid FROM pg_catalog.pg_class WHERE relname = c.table_name AND relnamespace = (SELECT oid FROM pg_catalog.pg_namespace WHERE nspname = c.table_schema)) where table_catalog = 'gorm_debug' AND table_schema = CURRENT_SCHEMA() AND table_name = 'foos'

2022/04/03 18:00:11 /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:461
[0.167ms] [rows:-] SELECT * FROM "foos" LIMIT 1

2022/04/03 18:00:11 /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:400
[7.104ms] [rows:-] SELECT c.column_name, constraint_type FROM information_schema.table_constraints tc JOIN information_schema.constraint_column_usage AS ccu USING (constraint_schema, constraint_name) JOIN information_schema.columns AS c ON c.table_schema = tc.constraint_schema AND tc.table_name = c.table_name AND ccu.column_name = c.column_name WHERE constraint_type IN ('PRIMARY KEY', 'UNIQUE') AND c.table_catalog = 'gorm_debug' AND c.table_schema = CURRENT_SCHEMA() AND c.table_name = 'foos'

2022/04/03 18:00:11 /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:430
[0.376ms] [rows:-] SELECT a.attname as column_name, format_type(a.atttypid, a.atttypmod) AS data_type
                FROM pg_attribute a JOIN pg_class b ON a.attrelid = b.relfilenode AND relnamespace = (SELECT oid FROM pg_catalog.pg_namespace WHERE nspname = CURRENT_SCHEMA())
                WHERE a.attnum > 0 -- hide internal columns
                AND NOT a.attisdropped -- hide deleted columns
                AND b.relname = 'foos'
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x81bdb7]

goroutine 1 [running]:
database/sql.(*ColumnType).DatabaseTypeName(...)
        /opt/go/src/database/sql/sql.go:3172
gorm.io/gorm/migrator.ColumnType.DatabaseTypeName(...)
        /home/ferdinand/go/pkg/mod/gorm.io/[email protected]/migrator/column_type.go:44
gorm.io/driver/postgres.Migrator.AlterColumn.func1.1(0xc0002aa680?)
        /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:251 +0xf7
gorm.io/gorm.(*DB).Connection(0x865960?, 0xc0001cf360)
        /home/ferdinand/go/pkg/mod/gorm.io/[email protected]/finisher_api.go:539 +0x102
gorm.io/driver/postgres.Migrator.AlterColumn.func1(0xc0003a4700)
        /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:249 +0x26b
gorm.io/gorm/migrator.Migrator.RunWithValue({{0xc0?, 0xc00038cdb0?, {0x97e320?, 0xc00028ae40?}}}, {0x843460?, 0xc000298ae0}, 0xc0001cf7a0)
        /home/ferdinand/go/pkg/mod/gorm.io/[email protected]/migrator/migrator.go:52 +0x13c
gorm.io/driver/postgres.Migrator.AlterColumn({{{0x0?, 0xc00038cdb0?, {0x97e320?, 0xc00028ae40?}}}}, {0x843460?, 0xc000298ae0?}, {0x83a1be?, 0x0?})
        /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:237 +0xb1
gorm.io/gorm/migrator.Migrator.MigrateColumn({{0xcb?, 0xc00038cbd0?, {0x97e320?, 0xc00028ae40?}}}, {0x843460, 0xc000298ae0}, 0xc0002965a0, {0x97e458, 0xc0002abd40})
        /home/ferdinand/go/pkg/mod/gorm.io/[email protected]/migrator/migrator.go:464 +0x689
gorm.io/driver/postgres.Migrator.MigrateColumn({{{0x0?, 0xc00038cbd0?, {0x97e320?, 0xc00028ae40?}}}}, {0x843460, 0xc000298ae0}, 0xc0002965a0, {0x97e458?, 0xc0002abd40?})
        /home/ferdinand/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:205 +0x95
gorm.io/gorm/migrator.Migrator.AutoMigrate.func1(0xc0002bcc40)
        /home/ferdinand/go/pkg/mod/gorm.io/[email protected]/migrator/migrator.go:120 +0x17b
gorm.io/gorm/migrator.Migrator.RunWithValue({{0x30?, 0xc00029b830?, {0x97e320?, 0xc00028ae40?}}}, {0x843460?, 0xc000298ae0}, 0xc0001cfe10)
        /home/ferdinand/go/pkg/mod/gorm.io/[email protected]/migrator/migrator.go:52 +0x13c
gorm.io/gorm/migrator.Migrator.AutoMigrate({{0x0?, 0xc00029b830?, {0x97e320?, 0xc00028ae40?}}}, {0xc0002b6a20?, 0x0?, 0x0?})
        /home/ferdinand/go/pkg/mod/gorm.io/[email protected]/migrator/migrator.go:101 +0x1ce
gorm.io/gorm.(*DB).AutoMigrate(0x97e168?, {0xc0002b6a20, 0x1, 0x1})
        /home/ferdinand/go/pkg/mod/gorm.io/[email protected]/migrator.go:28 +0x43
main.main()
        /home/ferdinand/dev/gorm-bug/main.go:24 +0x1ba

Process finished with the exit code 2

Please bring back WithoutReturning functionality

Describe the feature

In a recent question, there was an issue using Gorm with AWS Redshift because of the unsupported RETURNING keyword. This was resolved later, and then removed again. The option in the postgres.Config struct is still there, but without effect.

Motivation

Using Redshift with Gorm is still a valid use case. Don't know if the removal was an accident.

Related Issues

#20
70e3bdf
5992594

Would be cool if that can be supported again :-)

Dynamically-computed field is ignored or generates an error on row insertion

Your Question

Using packages gorm.io/gorm v1.23.8 gorm.io/driver/postgres v1.3.9

I'm trying to run the following SQL command using GORM:
SELECT *, (SELECT SUM(count) FROM crash_counts WHERE crash_id = crashes.id) AS total FROM crashes;
and have the result stored in an array of Crash structs.

Here, "total" is a dynamically-computed field, i.e. not an actual column in the table.
"total" is not in the table because it must be computed on-the-fly from data in another table.
Then the info is passed along to be displayed in HTML and discarded.

There are two rows in the DB, and the SQL command returns 12 & 3 for the totals.

I tested the 3 documented "ignore this field" permissions:

Totalx	uint `gorm:"-"`
Totaly	uint `gorm:"-:all"`
Totalz	uint `gorm:"-:migration"`

I run:

query = query.Debug().Select("*, (?) AS totalx, (?) AS totaly, (?) AS totalz",
	database.DB.Table("crash_counts").Select("SUM(count)").Where("crash_id = crashes.id"),
	database.DB.Table("crash_counts").Select("SUM(count)").Where("crash_id = crashes.id"),
	database.DB.Table("crash_counts").Select("SUM(count)").Where("crash_id = crashes.id"))
query.Find(&Crashes)
fmt.Printf("%v\n", Crashes)

And I get 0 0 12 0 0 3 for totalx totaly totalz
I don't understand why only totalz is correct and why totalx & totaly are 0?

Related issue: when I try to insert a new row, I get the following error:
ERROR: column "totalz" of relation "crashes" does not exist (SQLSTATE 42703)

So totalz gets updated correctly for Find(), but it generates an error on row insertion.

How would I define the "total" field to have both use-cases functional?

The document you expected this should be explained

https://gorm.io/docs/models.html

Advanced
Field-Level Permission
  Name string `gorm:"-"`            // ignore this field when write and read with struct
  Name string `gorm:"-:all"`        // ignore this field when write, read and migrate with struct
  Name string `gorm:"-:migration"`  // ignore this field when migrate with struct

Expected answer

I think the gorm:"-:all" tag should work in my use-case.

  1. Never create the field in the table
  2. Update the field in the Find query
  3. Ignore the field in row insertion or reads.

What am I doing wrong? :(

Migrator().ColumnTypes() not working inside Transaction

issue

I found AutoMigrate no longer works when put inside Transaction after recent updates of postgres driver. Then I found it is caused by Migrator().ColumnTypes() not work when it is put inside Transaction. example code below (sorry about the playground thing)

env

PostgreSQL 12.9 (Ubuntu 12.9-0ubuntu0.20.04.1)
gorm.io/driver/postgres v1.3.1
gorm.io/gorm v1.23.2

code

	type Email struct {
		gorm.Model
		Email string
	}

	result, ok := db.Migrator().ColumnTypes(&Email{})
	log.Printf("db:")
	log.Printf("ColumnTypes: %v", result)
	log.Printf("ColumnTypes error: %v", ok)

	// transaction
	if err := db.Transaction(func(tx *gorm.DB) error {
		result, ok := tx.Migrator().ColumnTypes(&Email{})
		log.Printf("Transaction:")
		log.Printf("ColumnTypes: %v", result)
		log.Printf("ColumnTypes error: %v", ok)
		return nil
	}); err != nil {
		return err
	}

output

2022/03/09 01:55:22 db:
2022/03/09 01:55:22 ColumnTypes: [{0xc0001fa230 {id true} {int8 true} { false} {true true} {false true} {true true} {64 true} {64 true} {0 true} {false true} <nil> { false} { false}} {0xc0001fa280 {created_at true} {timestamptz true} { false} {false true} {false true} {false false} {64 true} {6 true} {0 false} {true true} <nil> { false} { false}} {0xc0001fa2d0 {updated_at true} {timestamptz true} { false} {false true} {false true} {false false} {64 true} {6 true} {0 false} {true true} <nil> { false} { false}} {0xc0001fa320 {deleted_at true} {timestamptz true} { false} {false true} {false true} {false false} {64 true} {6 true} {0 false} {true true} <nil> { false} { false}} {0xc0001fa370 {email true} {text true} { false} {false true} {false true} {false false} {0 false} {0 false} {0 false} {true true} <nil> { false} { false}}]
2022/03/09 01:55:22 ColumnTypes error: <nil>

2022/03/09 01:55:22 /go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:331 driver: bad connection
[0.000ms] [rows:-] SELECT * FROM "emails" LIMIT 1
2022/03/09 01:55:22 Transaction:
2022/03/09 01:55:22 ColumnTypes: []
2022/03/09 01:55:22 ColumnTypes error: driver: bad connection

1.4.6 is a breaking change! "W10=" instead of "[]"

Don't have time to create a playground link. its just too much work. updating 1.4.5 to 1.4.6 broke our system.

Description

we have a table with a column of type jsonb in postgres. in go, we represent this column with json.RawMessage . in version 1.4.5 we can save an empty array into jsonb column and it would save [] in to the table. now, in the new version (1.4.6), it saves the value "W10=" instead of [] which can not be unmarshalled to a known object in go.

How can I disable WithReturning config before insert queries?

Your Question

I want to use this package for RedShift database. It works very well for simple queries. But the redshift db does not support RETURNING part on insert queries and it fails with ERROR: syntax error at or near "RETURNING" (SQLSTATE 42601) error.

func (dialector Dialector) Initialize(db *gorm.DB) (err error) {
	// register callbacks
	callbacks.RegisterDefaultCallbacks(db, &callbacks.Config{
		WithReturning: true,
	})
        //.....
}

Expected answer

How can I disable WithReturning: true config as a false before the create operation?

Vulnerability introduced

Snyk reports a vulnerability in the current version of this package introduced by github.com/jackc/pgproto3/v2

The current version used is 2.0.7 and the vulnerability is fixed in 2.1.1.

Please upgrade as soon as possible with #54.

Snyk output:

✗ Medium severity vulnerability found in github.com/jackc/pgproto3/v2
  Description: Denial of Service (DoS)
  Info: https://snyk.io/vuln/SNYK-GOLANG-GITHUBCOMJACKCPGPROTO3V2-1316251
  Introduced through: gorm.io/driver/[email protected]
  From: gorm.io/driver/[email protected] > github.com/jackc/pgx/v4/[email protected] > github.com/jackc/pgx/[email protected] > github.com/jackc/pgproto3/[email protected]
  From: gorm.io/driver/[email protected] > github.com/jackc/pgx/v4/[email protected] > github.com/jackc/pgx/[email protected] > github.com/jackc/pgconn/[email protected] > github.com/jackc/[email protected] > github.com/jackc/pgproto3/[email protected]
  Fixed in: 2.1.1

Complex pgx configuration not supported

Describe the feature

For some use cases (for example around TLS in dockerized dynamic environments) just setting up the pgx configuration from the dsn string or connection string is not enough.

Motivation

In the particular case I am running into I need to set up a TLS context but the certificates and key are not available in the file system but retrieved from a vault source (think AWS secret manager and others). In this environment the certificates cannot be written to the disk as they are running as docker containers from scratch and there are not permissions to write to the file system... even if there were I do not want to expose the certs and keys to a potential snoop.

I can think of other configuration sets where it would be better to just pass the pgx configuration directly and just have the adapter perform the connection

Related Issues

gorm.io/driver/postgres/migrator.go:337:38: Error

ERROR:

go run main.go server
# gorm.io/driver/postgres
vendor/gorm.io/driver/postgres/migrator.go:337:38: cannot use column (variable of type Column) as type gorm.ColumnType in argument to append:
        Column does not implement gorm.ColumnType (missing AutoIncrement method)
# gorm.io/driver/sqlserver
vendor/gorm.io/driver/sqlserver/create.go:55:34: not enough arguments in call to field.ValueOf
        have (reflect.Value)
        want (context.Context, reflect.Value)
vendor/gorm.io/driver/sqlserver/create.go:61:36: not enough arguments in call to field.ValueOf
        have (reflect.Value)
        want (context.Context, reflect.Value)

I update the latest gorm module with the following command:

rm -rf go.mod go.sum
go mod init "panel"
go mod tidy
go mod vendor

After running the error, the replacement version is fine.

go mod edit -require=gorm.io/driver/[email protected]
go mod edit -require=gorm.io/[email protected]
go mod vendor

uniqueIndex Cockroach incompatibility resulting from non-idiomatic column query

The Issue

I am using the postgres driver with a cockroachDB cluster (which maintains compatibility with PostgreSQL). However, the migrator fails to update the following model (after initial table creation):

type User struct {
	Model
	Name           string
	Email          string `gorm:"uniqueIndex;not null"`
	ProviderUserId string `gorm:"uniqueIndex; not null"`
}

With the following error message:

2022/03/19 14:47:22 /home/geno/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:419
[24.911ms] [rows:-] SELECT a.attname as column_name, format_type(a.atttypid, a.atttypmod) AS data_type
                FROM pg_attribute a JOIN pg_class b ON a.attrelid = b.relfilenode AND relnamespace = (SELECT oid FROM pg_catalog.pg_namespace WHERE nspname = CURRENT_SCHEMA())
                WHERE a.attnum > 0 -- hide internal columns
                AND NOT a.attisdropped -- hide deleted columns
                AND b.relname = 'users'

2022/03/19 14:47:22 /home/geno/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:271 ERROR: duplicate index name: "idx_users_email" (SQLSTATE 42P07)
[18.032ms] [rows:0] ALTER TABLE "users" ADD CONSTRAINT "idx_users_email" UNIQUE("email")
2022/03/19 14:47:22 ERROR: duplicate index name: "idx_users_email" (SQLSTATE 42P07)
panic: ERROR: duplicate index name: "idx_users_email" (SQLSTATE 42P07)

Most Likely Cause

This fails because of the following query in migrator.go:

postgres/migrator.go

Lines 415 to 419 in 5156d7e

dataTypeRows, err := m.DB.Raw(`SELECT a.attname as column_name, format_type(a.atttypid, a.atttypmod) AS data_type
FROM pg_attribute a JOIN pg_class b ON a.attrelid = b.relfilenode AND relnamespace = (SELECT oid FROM pg_catalog.pg_namespace WHERE nspname = ?)
WHERE a.attnum > 0 -- hide internal columns
AND NOT a.attisdropped -- hide deleted columns
AND b.relname = ?`, currentSchema, table).Rows()

As documented in cockroachdb/cockroach#28309:

pg_class is supported, but most if it's values pre-set
--   reltype, relfilenode, reltablespace and reltoastrelid are all set to an OID of 0
--   relowner, relam, relpages, reltuples, relacl and reloptions are all set to NULL
--   relallvisible and relfrozenxid are both set to 0
--   relisshared, relistemp, relhasoids, relhasrules, relhastriggers and relhassubclass are all set to FALSE
--   relPersistencePermanent is always set to 'p'
-- The more idiomatic way of looking these up should be using the information_schema

Potential Fix

So it would provide greater compatibility, be more idiomatic SQL, and resolve this migration issue, to instead use a query like the following:

dataTypeRows, err := m.DB.Raw(`SELECT column_name, data_type FROM information_schema.columns
WHERE table_schema = ?
AND table_name = ?`, currentSchema, table).Rows()

New version is using gorm side branch

Hi,

The new driver version v1.5.0 is using the gorm side branch "gorm.io/gorm v1.24.7-0.20230306060331-85eaf9eeda11".
You can see it in the go.mod file.
I think that this commit was already merged to the main banch. Is it possible to fix it?
Thanks
Dina.

Breaking change since 1.4.6: nil-Slices with custom driver.Valuer / sql.Scanner

This is a follow-up to #152, because the proposed "solution" / workaround doesn't work. The ticket was already closed though.

In PostgreSQL, the column has type JSONB.

go (simplified):

type ArgumentList []interface{}

func (a ArgumentList) Value() (driver.Value, error) {
	js, _ := a.MarshalJSON() // Returns a json-array. Nil-slices return "[]".
	return datatypes.JSON(js).Value()
}

func (a *ArgumentList) Scan(value interface{}) error {
	var js datatypes.JSON
	js.Scan(value)
	return json.Unmarshal(js, a)
}

func (ArgumentList) GormDataType() string {
	return datatypes.JSON{}.GormDataType()
}

func (ArgumentList) GormDBDataType(db *gorm.DB, field *schema.Field) string {
	return datatypes.JSON{}.GormDBDataType(db, field)
}

I'm already using datatypes.JSON in my custom serializer / scanner, as proposed by the other ticket.
However, they are never called for nil-slices.

Before 1.4.6, my serializers converted nil-slices to empty json-arrays.
Now, they are ignored, nil gets converted into a PostgreSQL null, and my not-null-constraints are triggered.
So this is definitely a breaking change.

连接失败无法返回错误导致panic

db, err: = gorm.Open(postgres.Open(dsn), &gorm.Config{})

fmt.Println(db, err)
fmt.Println(db == nil, err == nil)

输出

&{0xc0007d46c0 <nil> 0 0xc0000ae820 1} <nil>

在连接失败或者没有开启数据库的情况下 不会返回错误

AutoMigrate always alters columns of "bigint"

GORM Playground Link

go-gorm/playground#500

Description

The problem is rather simple: every run of AutoMigrate will execute a redundant ALTER COLUMN for int types (int64).

The reason?: The GORM Postgres driver uses "bigint" as the type for go's int64. However, in postgres, bigint is not the type name internally - it's just an alias to int8:

SELECT * FROM pg_type WHERE typname IN ('int8', 'bigint');

typname
int8

In practice, this means that AutoMigrate will always detect a discrepancy ("int8" != "bigint") and run an ALTER COLUMN.

The bad: You need to disable AutoMigrate by default to avoid these redundant queries.
The ugly: The ALTER COLUMN int8 to bigint actually breaks conditional indexes which depend on the column and requires VACUUM ANALYZE to fix (a Postgres issue, not a problem for here, I'll report when I have time). However, suffice to say, the redundant ALTER COLUMNs are not benign.

Happy to create a pull request.

JWT-go package is vulnerable

Playground link - N/A

The package github.com/dgrijalva/jwt-go introduces a vulnerabiltiy to the postgres driver for gorm

Details of vulnerability:

github.com/dgrijalva/jwt-go is a go implementation of JSON Web Tokens.

Affected versions of this package are vulnerable to Access Restriction Bypass if m["aud"] happens to be []string{}, as allowed by the spec, the type assertion fails and the value of aud is "". This can cause audience verification to succeed even if the audiences being passed are incorrect if required is set to false.

Remediation
Upgrade github.com/dgrijalva/jwt-go to version 4.0.0-preview1 or higher.

v1.3.8 has a compile time error

	gormdb, err := gorm.Open(postgres.New(postgres.Config{
		DSN:        connectionString,
	}), &gorm.Config{})
	if err != nil {
		log.Fatal(err)
	}

results in:

# gorm.io/driver/postgres
../../../go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:617:57: undefined: gorm.Index
../../../go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:618:25: undefined: gorm.Index
../../../go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:628:25: undefined: migrator.Index

Question - Can the tags be released?

go get gorm.io/driver/postgres installs v1.0.0 as the default version, which is incompatible with the latest version of gorm.io/gorm.
can the the latest tag be released? very confusing for migrating users such as me

Latest version v1.5.0 depends on unpublished gorm version

Description

Latest version v1.5.0 depends on unpublished gorm version.

go get -u gorm.io/driver/postgres
go: upgraded gorm.io/driver/postgres v1.4.8 => v1.5.0
go: upgraded gorm.io/gorm v1.24.6 => v1.24.7-0.20230306060331-85eaf9eeda11

Panic will be caused when the column's comment value is one string

Description

Main problem is the function MigrateColumn in file migrator.go ,

func (m Migrator) MigrateColumn(value interface{}, field *schema.Field, columnType gorm.ColumnType) error {
    //....
    //....
    comment := field.Comment
    if comment != "" {
	    comment = comment[1 : len(comment)-1]
    }
    //....
}

excample,when the comment value is 'h' or one space, comment = comment[1 : len(comment)-1] become comment = comment[1 : 0] ,so it wuill be caused panic

sql: unknown driver "postgres"

Your Question

import (
  "database/sql"

  "gorm.io/gorm"
  "gorm.io/gorm/logger"
  "gorm.io/driver/postgres"
)

func abc() {
  ... ... ...
  sqlDB, err = sql.Open("postgres", dsn)

  db, err = gorm.Open(postgres.New(postgres.Config{
    Conn: sqlDB,
     }), &gorm.Config{})
... ... ...
}

It fails to establish a connection with the PostgreSQL database.

Error message:
sql: unknown driver "postgres" (forgotten import?)

The document you expected this should be explained

official documents site

Expected answer

With MySQL and SQLite3 drivers gorm.io/driver/mysql, gorm.io/driver/sqlite, everything works perfectly fine.

Temporary workaround

By importing _ "github.com/jinzhu/gorm/dialects/postgres" along with "gorm.io/driver/postgres", connection with PostgreSQL database works smoothly.

So, why isn't "gorm.io/driver/postgres" works without importing _ "github.com/jinzhu/gorm/dialects/postgres" ? Am I missing something?

Thank you in advance.

incorrect gorm version dependency

In v1.4.5, go.mod requires gorm.io/gorm v1.24.1-0.20221019064659-5dd2bb482755, that is not a stable release.
I suppose it should require gorm.io/gorm v1.24.0.

I cannot use v1.4.5 of this module, due to this gorm dependency.
Thanks.

1.4.6 breaks enums

GORM Playground Link

go-gorm/playground#586

Description

This test fails after upgrading gorm.io/driver/postgres from v1.4.5 to v1.4.6

The test adds an enum type that contains a String() method

2023/03/24 14:56:40 testing postgres...
=== RUN   TestGORM

2023/03/24 14:56:40 /home/zchenyu/git8/playground/main_test.go:17 ERROR: invalid input syntax for type integer: "ONE" (SQLSTATE 22P02)
[1.317ms] [rows:0] INSERT INTO "companies" ("name","my_enum") VALUES ('jinzhu',1) RETURNING "id"

2023/03/24 14:56:40 /home/zchenyu/git8/playground/main_test.go:20 record not found
[0.721ms] [rows:0] SELECT * FROM "companies" WHERE "companies"."id" = 0 ORDER BY "companies"."id" LIMIT 1
    main_test.go:21: Failed, got error: record not found

Expected (with v1.4.5):

testing postgres...
2023/03/24 14:55:46 testing postgres...
=== RUN   TestGORM

2023/03/24 14:55:46 /home/zchenyu/git8/playground/main_test.go:17
[5.801ms] [rows:1] INSERT INTO "companies" ("name","my_enum") VALUES ('jinzhu',1) RETURNING "id"

2023/03/24 14:55:46 /home/zchenyu/git8/playground/main_test.go:20
[0.643ms] [rows:1] SELECT * FROM "companies" WHERE "companies"."id" = 1 ORDER BY "companies"."id" LIMIT 1
--- PASS: TestGORM (0.01s)

AutoMigration of XXXserial columns fails

GORM Playground Link

go-gorm/playground#571

Description

If you repeat the migration of a table with a column of type XXXserial the migration attempts to update the column and fails.

2023/02/21 15:00:00 /.../gorm-postgres/migrator.go:633 ERROR: type "bigserial" does not exist (SQLSTATE 42704)
[0.345ms] [rows:0] ALTER TABLE "table_structs" ALTER COLUMN "version" TYPE bigserial
    migrate_test.go:1512: 
        	Error Trace:	/home/manstis/workspaces/git/manstis/forks/gorm-playground/gorm/tests/migrate_test.go:1512
        	Error:      	Auto migrate failed
        	Test:       	TestMigrationPostgresXXXSerial
--- FAIL: TestMigrationPostgresXXXSerial (0.10s)

You'll need to start Postgres

  • docker-compose up
  • Make sure Environment Variable GORM_DIALECT=postgres is set

bigserial type cannot be used in migration (ALTER TABLE)

GORM Playground Link

go-gorm/playground#388

Description

We are migrating from gorm v1 (and PG v10). Our model structs that embed gorm.Model have changed their ID from integer to bigserial. This is OK for a fresh database, because PG supports bigserial syntax at CREATE time. It does not support it in ALTER TABLE commands (because strictly speaking, that would involve altering the type AND creating a SEQUENCE).

Example struct:

type GlobalOption struct {
	gorm.Model
	Key   string `gorm:"unique;not null;index"`
	Value string
}

Example SQL for existing table:

CREATE TABLE public.global_options (
    id integer NOT NULL,
    created_at timestamp with time zone,
    updated_at timestamp with time zone,
    deleted_at timestamp with time zone,
    key text NOT NULL,
    value text
);

During AutoMigrate(), the driver defaults to bigserial here:

postgres/postgres.go

Lines 158 to 159 in 5748fd5

default:
return "bigserial"

Which errors out:

2021/10/11 14:57:23 /Users/atz/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:252 ERROR: type "bigserial" does not exist (SQLSTATE 42704)
[0.876ms] [rows:0] ALTER TABLE "global_options" ALTER COLUMN "id" TYPE bigserial

You can verify that trivially on any version of postgres from v10 to v14:

demo=#CREATE TABLE foobar(id int);
CREATE TABLE
demo=# ALTER TABLE foobar ALTER COLUMN "id" TYPE bigserial;
ERROR:  type "bigserial" does not exist

TL;DR: gorm and driver must distinguish between bigserial for CREATE TABLE and separate logic for ALTER TABLE. It may be that the best approach is not to use bigserial at create time either.

Breaking change by new gorm error translation.

This change:

ecae830

will break errors we previously considered to be returned from gorm API.
e.g. we consider postgres-pgx "unique constraint violation" to be returned from gorm but now it is translated to some gorm error (which doesn't wrap the original pgx error).
Sorry I didn't include playground link because I thought the behaviour change is obvious.

AutoMigrate with table: columns and field: data_type fails if table already exists

GORM Playground Link

I'm working on this part, but it's not entirely possible to reproduce just with the playground by itself, I'm working on it, but please read the context as it's reliant on having tables pre-existing before gorm's db.AutoMigrate is run.

go-gorm/playground#1

Description

Context

TLDR; Tables already exist, migrate them using Gorm with go models

Database tables are set up via Liquibase to act as a source of truth, and while we can create various tables, columns, schemas, views, there's a quirk with Postgres (the information_schema domain, containing the known models) that the gorm Postgres driver will query to determine whether there already exists a Table when performing an AutoMigrate(), and it further uses this to determine what columns may need to be added during the resolution of discovering columns that already exist for the desired Table.

Using any other context where gorm is not setting up database tables (like the case where Liquibase sets up tables for you), and writing semi-automated tests to verify some level of drift compliance between the go models and the models in database, we've come to expect some level of guarantee to use db.AutoMigrate([]interface{}) as a deterministic differentiator to verify what models exist with which columns.

Problem

The problem arises when we have a perfectly acceptable model that Liquibase has already applied, but on attempting to AutoMigrate, gorm/postgres will throw an exception because of an implicit naming collision with two different schemas...
The literal problem is if you define a struct named Column with a field called DataType of any type, the result of the query will return effectively two results including the table that is being queried!
Screen Shot 2021-03-09 at 10 30 02 AM

Example model:

type Column struct {
  gorm.Model
  DataType int // can be whatever type it wants to be
}

func main() {
  // set up gorm db to postgres that already has the models set up in tables
  err := db.AutoMigrate(&Column{})
 // err will not be nil
}

The naive solution

Simply adding this to the select query will resolve the correct expected behavior (it's naive, maybe can be done with an exclude statement instead?)

and table_schema != 'information_schema'

dns Package Introduces Vulnerability

The package github.com/miekg/dns introduces a vulnerabiltiy

Details of vulnerability:

Overview
github.com/miekg/dns is a complete and usable DNS library.

Affected versions of this package are vulnerable to Insecure Randomness. It improperly generates random numbers because math/rand is used. The TXID becomes predictable, leading to response forgeries.

Remediation
Upgrade github.com/miekg/dns to version 1.1.25 or higher.

ERROR: relation "idx_xxx" already exists (SQLSTATE 42P07)

GORM Playground Link

go-gorm/playground#1

Description

2022/12/28 02:48:33 /root/go/src/deeproute.ai/smd/vendor/gorm.io/driver/postgres/migrator.go:271 ERROR: relation "idx_cephfs_subvolumes_subvolume_name" already exists (SQLSTATE 42P07)
[0.650ms] [rows:0] ALTER TABLE "cephfs_subvolumes" ADD CONSTRAINT "idx_cephfs_subvolumes_subvolume_name" UNIQUE("subvolume_name")
I1228 02:48:33.476777       1 log.go:198] ERROR: relation "idx_cephfs_subvolumes_subvolume_name" already exists (SQLSTATE 42P07)

Problems with composite unique indexes in CockroachDB

Description

When you create a unique index in CockroachDB you can find the columns in the query

SELECT c.column_name, constraint_type FROM information_schema.table_constraints tc JOIN information_schema.constraint_column_usage AS ccu USING (constraint_schema, constraint_name) JOIN information_schema.columns AS c ON c.table_schema = tc.constraint_schema AND tc.table_name = c.table_name AND ccu.column_name = c.column_name WHERE constraint_type IN ('PRIMARY KEY', 'UNIQUE') AND c.table_catalog = ? AND c.table_schema = ? AND c.table_name = ?

If you do the same with Postgres, in this query the columns of unique indexes are not returned.

When you define a composite unique indexes with CockroachDB, Gorm incorrectly creates individual unique indexes for each column, because of the behaviour of CockroachDB described above.

I have come up with the following workaround for the problem, but I don't know if this is the best solution to get around the problem:

is (migrator.go, line 437)

// check primary, unique field
		{
			columnTypeRows, err := m.DB.Raw("SELECT c.column_name, constraint_type FROM information_schema.table_constraints tc JOIN information_schema.constraint_column_usage AS ccu USING (constraint_schema, constraint_name) JOIN information_schema.columns AS c ON c.table_schema = tc.constraint_schema AND tc.table_name = c.table_name AND ccu.column_name = c.column_name WHERE constraint_type IN ('PRIMARY KEY', 'UNIQUE') AND c.table_catalog = ? AND c.table_schema = ? AND c.table_name = ?", currentDatabase, currentSchema, table).Rows()
			if err != nil {
				return err
			}

			for columnTypeRows.Next() {
				var name, columnType string
				columnTypeRows.Scan(&name, &columnType)
				for _, c := range columnTypes {
					mc := c.(*migrator.ColumnType)
					if mc.NameValue.String == name {
						switch columnType {
						case "PRIMARY KEY":
							mc.PrimaryKeyValue = sql.NullBool{Bool: true, Valid: true}
						case "UNIQUE":
							mc.UniqueValue = sql.NullBool{Bool: true, Valid: true}
						}
						break
					}
				}
			}
			columnTypeRows.Close()
		}

my solution:

// check primary, unique field
		{
			columnTypeRows, err := m.DB.Raw("SELECT constraint_name FROM information_schema.table_constraints tc JOIN information_schema.constraint_column_usage AS ccu USING (constraint_schema, constraint_name) JOIN information_schema.columns AS c ON c.table_schema = tc.constraint_schema AND tc.table_name = c.table_name AND ccu.column_name = c.column_name WHERE constraint_type IN ('PRIMARY KEY', 'UNIQUE') AND c.table_catalog = ? AND c.table_schema = ? AND c.table_name = ? AND constraint_type = ?", currentDatabase, currentSchema, table, "UNIQUE").Rows()
			if err != nil {
				return err
			}
			uniqueContraints := map[string]int{}
			for columnTypeRows.Next() {
				var constraintName string
				columnTypeRows.Scan(&constraintName)
				uniqueContraints[constraintName]++
			}
			columnTypeRows.Close()

			columnTypeRows, err = m.DB.Raw("SELECT c.column_name, constraint_name, constraint_type FROM information_schema.table_constraints tc JOIN information_schema.constraint_column_usage AS ccu USING (constraint_schema, constraint_name) JOIN information_schema.columns AS c ON c.table_schema = tc.constraint_schema AND tc.table_name = c.table_name AND ccu.column_name = c.column_name WHERE constraint_type IN ('PRIMARY KEY', 'UNIQUE') AND c.table_catalog = ? AND c.table_schema = ? AND c.table_name = ?", currentDatabase, currentSchema, table).Rows()
			if err != nil {
				return err
			}
			for columnTypeRows.Next() {
				var name, constraintName, columnType string
				columnTypeRows.Scan(&name, &constraintName, &columnType)
				for _, c := range columnTypes {
					mc := c.(*migrator.ColumnType)
					if mc.NameValue.String == name {
						switch columnType {
						case "PRIMARY KEY":
							mc.PrimaryKeyValue = sql.NullBool{Bool: true, Valid: true}
						case "UNIQUE":
							if uniqueContraints[constraintName] == 1 {
								mc.UniqueValue = sql.NullBool{Bool: true, Valid: true}
							}
						}
						break
					}
				}
			}
			columnTypeRows.Close()
		}

I query all elements that have constraint_type = 'UNIQUE' and count the elements per constraint_name. Unique constraints are only set if the constraint name has a count of one. Composite unique indices have a count of 2 or more and are ignored in the lower part of the routine, which solves the problem with the wrong unique indexes.

If you want I can create a pull request for this change.

Missing autoIncrement in postgres table schema output

This is my table schema

CREATE TABLE public.log_usage (
    log_id bigint NOT NULL
);

ALTER TABLE public.log_usage ALTER COLUMN log_id ADD GENERATED BY DEFAULT AS IDENTITY (
    SEQUENCE NAME public.log_usage_log_id_seq
    START WITH 1
    INCREMENT BY 1
    NO MINVALUE
    NO MAXVALUE
    CACHE 1
);

ALTER TABLE ONLY public.log_usage
    ADD CONSTRAINT pk_log_usage PRIMARY KEY (log_id);

And this is the generator output:

type LogUsage struct {
	LogID           int64     `gorm:"column:log_id;type:int8;primaryKey" json:"log_id"`
}

Expected:

type LogUsage struct {
	LogID           int64     `gorm:"column:log_id;type:int8;primaryKey;autoIncrement:true" json:"log_id"`
}

No autoincrement for serial column

GORM Playground Link

go-gorm/playground#439

Description

Upgrading from gorm v1.22.5 columns of type serial does not appear to increment appropriately. According to the logs, with the new version of gorm and the gorm postgres driver, 0 is inserted in the serial column as opposed to the previous version where nothing is inserted and the column default sequence is correctly used.

  • gorm v1.3.0, postgress driver v1.3.0: INSERT INTO "tests" ("created_at","updated_at","deleted_at") VALUES ('2022-02-22 12:51:00.618','2022-02-22 12:51:00.618',NULL) RETURNING "id"

  • gorm v1.22.5, postgress driver v1.2.3: INSERT INTO "tests" ("id","created_at","updated_at","deleted_at") VALUES (0,'2022-02-22 12:52:02.765','2022-02-22 12:52:02.765',NULL)

Auto-migrate fails to perform on the same object more than twice

GORM Playground Link

go-gorm/playground#472

Description

Auto-migration fails on the third change of the same object, this happens for very simple migrations.
The issue started with gorm version 1.23.5, it did not occur for us in version 1.23.4.

The test case is running multiple migration steps on the same object, first creating the object then adding a new field and finally adding a second new field. The final step fails with: ERROR: cached plan must not change result type (SQLSTATE 0A000)

func CreateObject() error {
	type Object struct{}
	return DB.AutoMigrate(&Object{})
}

func AddField1() error {
	type Object struct {
		Field1 string
	}
	return DB.AutoMigrate(&Object{})
}

func AddField2() error {
	type Object struct {
		Field2 string
	}
	return DB.AutoMigrate(&Object{})
}

func TestGORM(t *testing.T) {
	if DB.Dialector.Name() != "postgres" {
		return
	}
	if err := CreateObject(); err != nil {
		t.Errorf("FAILED CREATING OBJECT")
		return
	}
	if err := AddField1(); err != nil {
		t.Errorf("FAILED ADDING FIELD1")
		return
	}
	if err := AddField2(); err != nil {
		t.Errorf("I am sad :(")
		return
	}
}

fail to alter column from smallint to boolean in postgres

GORM Playground Link

go-gorm/playground#574

Description

If we want to change column from smallint to boolean, it will report error like below:

2023/02/28 14:03:35 /home/user/go/pkg/mod/gorm.io/driver/[email protected]/migrator.go:318 ERROR: cannot cast type smallint to boolean (SQLSTATE 42846)
[0.374ms] [rows:0] ALTER TABLE "column_structs" ALTER COLUMN "is_active" TYPE boolean USING "is_active"::boolean

We only need to fix sql become USING xxx::int::boolean for boolean field in func AlterColumn in [email protected]/migrator.go

Issue while trying to create records and remove the RETURNING statement.

Your Question

I'm using go-gorm with a Postgres 11 DB and facing an issue where I need to remove the RETURNING clause entirely when creating records. I just want to insert records and get nothing back. In go-pg I could use Returning("null") but I couldn't find a way to do it with GORM.

I have many relations on the database that doesn't support RETURNING statements, so when I try to insert records I get the error "ERROR: cannot perform INSERT RETURNING on relation X".

I tried creating the db connection with the parameter WithoutReturning: true. But then I get a different error "LastInsertId is not supported by this driver". It still tries to get the last inserted id anyway.

Thank you very much for your time!

The document you expected this should be explained

There should be a mention to the RETURNING statement on the "Create" section of the documentation and instructions on how to get rid of it if my query won't support.

https://gorm.io/docs/create.html

Expected answer

A polite answer on how to create records when my DB is not supporting RETURNING statements would be great! =D

write value Nan error

insert NaN error:

2023-03-29 18:27:55.398 CST [654503] new@new ERROR: column "nan" does not exist at character 30483

insert into products (price) values(NaN)

in database:
Table "public.products"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+--------------------------------------
id | integer | | not null | nextval('products_id_seq'::regclass)
price | numeric | | |

insert into products (price) values(NaN);

ERROR: column "nan" does not exist
LINE 1: insert into products (price) values(NaN);
^

insert into products (price) values('NaN');

INSERT 0 1

breaking changes w/ the 0.2.5 change to pgx?

Your Question

go.sum

gorm.io/datatypes v0.0.0-20200620103158-546ecd30e4ba // indirect
gorm.io/driver/postgres v0.2.5
gorm.io/gorm v0.2.19

I'm not sure if encountering bugs as a result of the v0.2.5 switch to pgx or if there is required migration to support the change.

Given the following struct...

type Harumph struct {
	gorm.Model
	Thing     *Thing `gorm:"type:jsonb" json:",omitempty"`
}

type Thing map[string]interface{}

1)

Thing Valuer no longer works for the jsonb column unless typecast the marshaled []byte to string

func (m *Thing) Value() (driver.Value, error) {
	return json.Marshal(m)
}
ERROR: could not parse JSON: unable to decode JSON: invalid character '\\\\' looking for beginning of value (SQLSTATE 22P02)

But this works

func (m *Thing) Value() (driver.Value, error) {
	b, err := json.Marshal(m)
	return string(b), err
}

2)

When implementing Thing Scanner, Scan is called, and appears to update the value of *Thing, but the value of Harumph.Thing is still nil.

func (m *Thing) Scan(src interface{}) error {
	b, ok := src.([]byte)
	if !ok {
		return errors.New("byte assertion failed")
	}

	var value Thing
	if err := json.Unmarshal(b, &value); err != nil {
		return err
	}

	*m = value

	return nil
}

3)

Scan triggers a Valuer call error when using non-pointer reference

func (m Thing) Value() (driver.Value, error)
value method Thing.Value called using nil *Thing pointer

I don't believe Value should require a pointer reference, but Thing.Value will work if changed to pointer *Thing

func (m *Thing) Value() (driver.Value, error)

Expected answer

Looking to understand whether there are bugs w/ the 0.2.5 pgx switch, or new requirements with the change.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.