guregu / null Goto Github PK
View Code? Open in Web Editor NEWreasonable handling of nullable values
License: BSD 2-Clause "Simplified" License
reasonable handling of nullable values
License: BSD 2-Clause "Simplified" License
Purpose: to allow cases where field it's value does not exist inside an wrapping object to be sent to the server as NULL JSON/JSONB Value (where structure type is json.RawMessage)
If someone could implement it it would be even better.
It seems that all types in null except null.Time has IsZero()
method, is it intended?
Hey,
I have a question. The null.String type does not have a key specified. Thus, govet as well as sonarqube give me the warning:
vendor/gopkg.in/guregu/null.v3.String composite literal uses unkeyed fields
for a struct using this type. I was wondering if it's possible to change that or if there is a fix for this?
go version
)?$ go version go version go1.15.6 darwin/amd64
Yes
https://play.golang.org/p/7kgBsW80Mzf
If we would have not empty decoder.errorContext.FieldStack and we received error of fmt.wrapError type from UnmarshalJSON then later addErrorContext wouldn't be able to set proper value for err.Field and err.Struct because error type won't match with *UnmarshalTypeError.
https://github.com/golang/go/blob/a7e16abb22f1b249d2691b32a5d20206282898f2/src/encoding/json/decode.go#L181-L183
https://github.com/golang/go/blob/a7e16abb22f1b249d2691b32a5d20206282898f2/src/encoding/json/decode.go#L255-L258
Unmarshalling and marshalling types in the null
package isn't symmetric. For example:
type T struct {
Name null.String
}
in := []byte(`{"Name":""}`)
var t T
json.Unmarshal(in, &t)
fmt.Printf("%+v\n", t) // => {Name:{NullString:{String: Valid:false}}}
out, _ := json.Marshal(t)
fmt.Println(string(out)) // => {"Name":null}
Why not have in
unmarshal to {Name:{NullString:{String: Valid:true}}}
? In addition to clearing up the issue shown above it would make the null
package incredibly useful for the common issue of how to determine if a RESTful request intends to set a value to the zero value or if it just wasn't included in the request (a PATCH-like request).
Right now the common recommendation is to use pointers for the fields in your structs as described here, but I'd much prefer using the null
package for this if I could.
Let me know what you think. I'm happy to create a PR for this if you think this idea has legs.
I found this error
/home/myname/go/src/github.com/guregu/null/time.go:15:2: undefined: sql.NullTime
Ubuntu 18.04.4 LTS
go version 1.12.10
When using gorp or other ORMs it would be nice to be able to do the same with the Time type
I use beego with "nuu.v4", create struct
type SysUserBean struct {
UserId int64 orm:"auto" json:"userId"
Username null.String json:"username"
}
when run "bee run", then tips, "field: models.SysUserBean.Username, unsupport field type &{{ %!s(bool=false)}}, may be miss setting tag", can't go on
how to fix it
Hello Greg (hope I got the name right :) )
First of all, great library for marshalling and unmarshalling Golang's SQL null datatypes.
Really saved me a lot of time and effort.
However, I noticed that this library is missing one piece of functionality which is storing null values in database.
I have gone through the library and could not find it anywhere. I had to implement this part in my own codebase.
I would be happy to raise a Pull Request so that this functionality can be merged to master.
Please do let me know if I can go ahead with raising a pull request.
Testing my models on GraphQL, I detected some issues regarding text-marshaling implementation in this library.
null.Time returns "null"
func (t Time) MarshalText() ([]byte, error) {
if !t.Valid {
return []byte("null"), nil
}
return t.Time.MarshalText()
}
Other null.X implementations
// MarshalText implements encoding.TextMarshaler.
// It will encode a blank string when this String is null.
func (s String) MarshalText() ([]byte, error) {
if !s.Valid {
return []byte{}, nil
}
return []byte(s.String), nil
}
// MarshalText implements encoding.TextMarshaler.
// It will encode a blank string if this Int is null.
func (i Int) MarshalText() ([]byte, error) {
if !i.Valid {
return []byte{}, nil
}
return []byte(strconv.FormatInt(i.Int64, 10)), nil
}
My question/curiosity is: why does null.Time text-marshaling return a "null" string instead of an empty string like the others?
example
StringFrom("")
``
value
sql.NullString{Value:"", Valid:true}
which is wrong
I'd like to work towards releasing a new version.
zero
support first.Lines 70 to 71 in 80515d4
interface{}
first. I doubt many people use this anyway.Hi guregu,
My teammates and I really like your library, as it has helped us a lot when going from JSON to go structs to databases (thank you!). One small issue we found is unmarshalling large numbers from JSON. Right now, because there is only one number type in Javascript, integers in JSON that are large enough are decoded in scientific notation. Because of this, these numbers cannot be unmarshalled into integer types, in particular your null.Int and zero.Int types. I was wondering if you had any thoughts on the issue.
If you're interested, I've already written an implementation that passes all of the tests, if you want to take a look.
Any ideas or opinions on how to handle generics in this library?
I played around with creating a generic Null[T]
type, but I wonder if it's even useful at that point, because we are essentially recreating the Optional type you might find in other languages. At what point is it going too far?
Maybe ideally we could make generic Null/Zero types and use the materialized version of it for each pre-existing type.
I'm also curious if we'll see a generic sql.Null[T]
type in the standard library, which might obsolete a good chunk of this library, especially if this change lands: golang/go#5901 (which lets you easily specify custom marshalers for JSON, moving the responsibility from the object itself to the encoding site).
See also: golang/go#48702
How can we check null.String value
where is null or where is empty ?
I see the function IsZero
work not as expected.
could not import gopkg.in/guregu/null.V3 (no required module provides package "gopkg.in/guregu/null.V3")
I tried v4 also with the same error
I tried this with Gin-Gonic
type AddUserRequest struct {
// some other properties
DateOfBirth null.Time `json:"user_birthday" time_format:"2006-01-02"`
}
When I send
{
// some other stuff
"user_birthday": "2000-04-05"
}
I still get the (infamous?) parsing time \"\"2000-04-05\"\" as \"\"2006-01-02T15:04:05Z07:00\"\": cannot parse \"\"\" as \"T\""
error.
Is there a workaround for this issue? I don't want to send a string and parse manually.
Thanks!
Like this library, very clean & nice to use.
Would you be interested in adding support for / accepting a PR with an implementation of null.Duration
?
The docs state the following about null.String.IsZero:
IsZero returns true for null or empty strings, for future omitempty support. (Go 1.4?)
Will return false s if blank but non-null.
Which looks like a contradictory statement. In a test with an empty string, the latter held true: false
is returned if the string is equal to "". Could either the behavior of IsZero() change to return true
if string equals empty string or the docs be clarified?
Thanks.
Unmarshaling JSON into an intermediate interface{}
results in all numbers being decoded to float64s:
Lines 47 to 48 in 7cccb14
var v interface{}
json.Unmarshal(data, &v)
Later on it casts the float64 to an int64, but there's no guarantee that the passed value was actually an integer or that it didn't overflow the int64:
Line 51 in 7cccb14
i.Int64 = int64(x)
So if I have it decode a non-integer such as 123.45
it gets silently cast to 123
when I would have expected an error. If I have a number that doesn't fit in an int64 such as 9223372036854775808
, it overflows silently to a negative number as well. I also tested the max int64 size of 9223372036854775807
and found it was overflowing for unknown reasons. Perhaps the intermediate float64 resulted in some loss of precision?
I think the intermediate unmarshal to interface{}
should be avoided if possible, instead attempting to unmarshal the []byte directly into an int64.
/root/go/pkg/mod/gopkg.in/guregu/[email protected]/zero/time.go:15:2:
undefined: sql.NullTime
Less of an issue, and more of an FYI. I have been keeping tabs on the (un)marshaling/(de)encoding issue with omitempty, and it looks like they have a adjacent proposal that may provide a workable solution:
Didn't see anywhere else to leave a comment about this, and I figured it is relevant to this project in particular (I know its relevant to my usage of this project! Thanks by the way!)
cannot use value.Network (type string) as type "gopkg.in/volatiletech/null.v6".String in assignment
hi
how your versioning works?
can you plz help me understand
v4 v8 and none really? :)
In JSON there are effectively two types of null.
1) When the key is provided, and the value is explicitly stated as null.
2) When the key is not provided, and the value is implicitly null.
This is important when constructing a PATCH endpoint because:
There is a more detailed discussion of this issue here, with example code of how to solve the problem:
https://www.calhoun.io/how-to-determine-if-a-json-key-has-been-set-to-null-or-not-provided/
It would be nice if the null library can support this.
Does this library support golang 1.13.
We just upgraded our application to 1.13 and now are getting:
2020/08/24 00:12:38 Failed to build app: building app with command [go build -o /tmp/staging/usr/local/bin/start bitbucket.org/x1x1x1x1/y1y1y1/z1z1z1], env [PATH=/go/bin:/usr/local/go/bin:/builder/google-cloud-sdk/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=cb5c804c1756 HOME=/builder/home BUILDER_OUTPUT=/builder/outputs DEBIAN_FRONTEND=noninteractive GOROOT=/usr/local/go/ GOPATH=/go GOPATH=/tmp/staging/srv/gopath]: err=exit status 2, out=# github.com/guregu/null
srv/gopath/src/github.com/guregu/null/time.go:15:2: undefined: sql.NullTime
It looks like there were some changes to the SQL libraries in the new versions of go lang. Is there something we need to support the changes in 1.13
// NewInt creates a new Int from a string
func NewIntFromString(i string, valid bool) Int {
if valid {
inter, err := strconv.Atoi(i)
if err != nil {
return Int{
NullInt64: sql.NullInt64{
Int64: 0,
Valid: false,
},
}
}
in := int64(inter)
return Int{
NullInt64: sql.NullInt64{
Int64: in,
Valid: valid,
},
}
}
return Int{
NullInt64: sql.NullInt64{
Int64: 0,
Valid: false,
},
}
}
test:
func TestNewIntFromString(t *testing.T) {
type args struct {
i string
valid bool
}
tests := []struct {
name string
args args
want Int
}{
{
name: "equal zero",
args: args{
i: "0",
valid: false,
},
want: IntFrom(0),
// TODO: Add test cases.
},
{
name: "equal one",
args: args{
i: "1",
valid: true,
},
want: IntFrom(1),
// TODO: Add test cases.
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := NewIntFromString(tt.args.i, tt.args.valid); !reflect.DeepEqual(got, tt.want) {
t.Errorf("NewIntFromString() = %v, want %v", got, tt.want)
}
})
}
}
I use this like:
id:= zero.NewIntFromString(c.GetQuery("id"))
where c is a gin.Context (could do similar with http.GetForm)
Each struct of this library uses sql package (persistence or infrastructure layer), while it can be used for JSON marshaling (presentation layer).
It seems to be against some software architecture patterns based on layered architecture or etc.
Can we make StringFrom verify string argument for "emptiness"?
func StringFrom(s string) String {
return NewString(s, true)
}
to
func StringFrom(s string) String {
if s == "" {
return NewString(s, false)
}
return NewString(s, true)
}
It is useful in:
result := null.StringFrom(ctx.Request.Header.Get("some-header"))
And many other cases.
Or a new method with this behaviour?
There might be a super simple answer to this, but as a newbie at Go, I'm having an issue.
I'm fetching some data from MSSQL and have set up some struct properties as null.String
since the DB can contain null values.
Before json encoding the structs, I need to manipulate them a bit and need to do some simple if statements, like: if row.x == "y" { //do stuff }
but since row.x
has the type null.String
, that comparison fails with:
mismatched types null.String and string
If I try to convert it to string first: var x = string(row.x)
, I get:
cannot convert (type null.String) to type string
What is the best way to handle this?
I'm trying to compare nullable types but I couldn't find an existing easy way to do it.
Are you interested in adding support for / accepting a PR that adds functions like the following for all nulltypes?
func StringEquals(a, b null.String) bool {
return a.Valid == b.Valid && (! a.Valid || a.String == b.String)
}
// StringEquals(null.NewString("foo", false), null.NewString("foo", false)) -> true
// StringEquals(null.NewString("foo", false), null.NewString("bar", false)) -> true
// StringEquals(null.NewString("foo", true), null.NewString("foo", false)) -> false
// StringEquals(null.NewString("foo", true), null.NewString("foo", true)) -> true
// StringEquals(null.NewString("foo", true), null.NewString("bar", true)) -> false
I'm not entirely sure about the workings of the zerotypes (I don't use them). If I got it right, they can just be compared using their string/bool/int/... value so they don't need the extra function, right?
From the documentation I understand that types are either fully treated as zero or null values in Go and JSON.
I am trying to understand if I can use null values in Go but have them encoded to zero in JSON.
type Dto struct {
Start null.Time
}
{
"Start": null
}
gorm.Model(&MyTable).Where("id = 1").Update(&dto)
update my_table set start = null where id = 1
The start field is now ignored when used in this way
can add custom time format support for Time?
add a global variable with name is TimeFormats and type is []string
I realize that the db driver doesn't support uint
as a type and that javascript only have the number type. Basically meaning a rather small performance hit because of the conversion we would need to do, not to mention the complications with overflow.
But i still think in some of the structs we have it would make sense to have a null.Uint
just for cosmetic purposes and to enforce correct types at the point of storage to avoid errors. Would a PR with uint
type(s) be accepted?
type JsonUser struct { ID null.Int
json:"id" db:"id"Email null.String
json:"email" db:"email"Password null.String
json:"-" db:"password"FirstName null.String
json:"first_name" db:"first_name"LastName null.String
json:"last_name" db:"last_name"CreatedAt null.Time
json:"created_at" db:"created_at"UpdatedAt null.Time
json:"updated_at" db:"updated_at"`
}
func UserByID(w http.ResponseWriter, req *http.Request) {
vars := mux.Vars(req)
if userID, err := strconv.Atoi(vars["id"]); err != nil {
fmt.Fprint(w, response.Success("No record found.", http.StatusNotFound, nil).JSON())
return
} else {
jsonUser := &JsonUser{}
err = DB.QueryRow("SELECT id, email, first_name, last_name, created_at, updated_at FROM users
WHERE id = ? LIMIT 1;", userID).Scan(&jsonUser.ID, &jsonUser.Email, &jsonUser.FirstName, &jsonUser.LastName, &jsonUser.CreatedAt, &jsonUser.UpdatedAt)
if err != nil {
fmt.Fprint(w, response.Error(err.Error(), http.StatusInternalServerError).JSON())
return
}
fmt.Fprint(w, response.Success("Request successful!", http.StatusOK, jsonUser).JSON())
}
return
}`
CreatedAt is not null, but UpdatedAt is for the particular record I'm retrieving.
The error that is being thrown is: "sql: Scan error on column index 4: null: cannot scan type []uint8 into null.Time: [50 48 49 56 45 48 56 45 50 50 32 48 48 58 53 54 58 50 50]"
It doesn't seem that this package's Time is not handling []uint8 but I'm not sure how to get this working. Thanks!
Hey guys,
is it on purpose that this module can not support uint64 (bigint unsigned)?
casting uint64 to int64 will cause a loss of the last bit. And therefore I would have wrong results in DB
type Thing struct {
Id string `db:"id"`
number null.Int `db:"name"`
Database has the schema:
CREATE TABLE `thing` (
`id` varchar(25) NOT NULL,
`number` bigint unsigned DEFAULT NULL
)
Implemtation of null.IntFrom
Is it sqlx not supporting it? How can I work around that?! :)
related to: go-sql-driver/mysql#715 ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.