go-playground / mold Goto Github PK
View Code? Open in Web Editor NEW:scissors: Is a general library to help modify or set data within data structures and other objects.
License: MIT License
:scissors: Is a general library to help modify or set data within data structures and other objects.
License: MIT License
Add it to https://github.com/avelino/awesome-go.
Use case:
type A struct {
M map[string]*B `mod:"dive"` // WANTED: `mod:"dive,keys,trim,endkeys,dive"`
}
type B struct {
S string `mod:"trim"`
}
With "dive" on M it will process "trim" on S, but won't process map keys.
With "dive,keys,trim,endkeys" on M it will process map keys but won't process "trim" on S.
And the WANTED string result in an error "invalid dive tag configuration".
It appears in the long example that mod
tag is used when I would guess it's supposed to be mold
?
https://github.com/go-playground/mold/blob/master/_examples/full/main.go#L45
What I miss using this instead of https://github.com/leebenson/conform is the check of embedded structs:
https://github.com/leebenson/conform/blob/master/conform.go#L272.
Can we add this?
Maybe optional (default: true) with a tag: mod:"embedded=false"
.
What do you think?
This is URGENT IMO because I cannot use it having a lot of embedded structs.
These lines: https://github.com/go-playground/mold/blob/master/mold.go#L286-L296
Warnings by Goland (JetBrains):
Accessing field
ct.keys may lead to nil pointer dereference
Accessing field
ct.next may lead to nil pointer dereference
Line 17 in 4214131
Maybe this?
scrubbed := fmt.Sprintf("<<scrubbed::email::sha1::%s>>", hashString(input[:idx]))
Should hash the string before @
?
Struct(ctx context.Context, v interface{}) error
func doesn't work on slice fields.
Here is the example code (inspired by _examples/full/main.go
):
import (
"context"
"fmt"
"log"
"reflect"
"github.com/go-playground/mold/v3"
"github.com/go-playground/mold/v3/modifiers"
)
type Address struct {
Name string `mod:"trim"`
Phone string `mod:"trim"`
}
type User struct {
Name string `mod:"trim"`
Age uint8
Gender string `mod:"trim"`
Email string `mod:"trim"`
Address []Address
}
func main() {
user := User{
Name: "test ",
Age: 50,
Gender: "female",
Email: "[email protected]",
Address: []Address{
{
Name: "test address ",
Phone: " 123 456 ",
},
{
Name: "test address ",
Phone: " 123 456 ",
},
},
}
conform := modifiers.New()
if err := conform.Struct(context.Background(), &user); err != nil {
log.Panic(err)
}
fmt.Printf("Conformed:%+v\n\n", user)
}
Output:
Conformed:{Name:test Age:50 Gender:female Email:[email protected] Address:[{Name:test address Phone: 123 456 } {Name:test address Phone: 123 456 }]}
So, Name
and Phone
fields of the Address struct weren't trimmed.
If I add dive
tag to the Address
field the Struct
func will panic:
type User struct {
...
Address []Address `mod:"dive"`
}
Panic:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x32 pc=0x10c0c19]
goroutine 1 [running]:
github.com/go-playground/mold/v3.(*Transformer).setByField(0xc000021ac0, 0x11cf500, 0xc0000160a0, 0x1176760, 0xc000021b00, 0x199, 0xc00018ae70, 0x0, 0x0, 0x0)
/Users/myuser/go/pkg/mod/github.com/go-playground/mold/[email protected]/mold.go:202 +0x79
github.com/go-playground/mold/v3.(*Transformer).setByField(0xc000021ac0, 0x11cf500, 0xc0000160a0, 0x115af80, 0xc00009adf8, 0x197, 0xc00018ae40, 0x0, 0x0, 0x0)
...
The reason for the panic is that the dive
tag could not be used alone, so you'll have to add some tag after it (endkeys
tag doesn't work as well).
The workaround for that issue is to register a dummy transform function:
func items(_ context.Context, _ *mold.Transformer, _ reflect.Value, _ string) error {
return nil
}
...
conform.Register("items", items)
... and add a mod:"dive,items"
tag to the slice field:
type User struct {
...
Address []Address `mod:"dive,items"`
}
I think that the issue should be addressed either by fixing this in the lib or by adding that to the documentation.
What do you think?
P.S.
I can help with a PR if you need it.
Thanks.
Seems like the Transform is a refactoring of the basic struct traversal logic from validator. Was thinking it would be really useful to have a re-usable struct visitor library like this, to build focussed struct processing packages, like one which sets default values, initializes values from environment variables, or validates.
One thing I see missing from mold, which validator uses, is that the Func doesn't know where it is in the hierarchy, so it isn't possible to include that information in error messages or validation problems.
I'm interested in the idea of building other types of struct processors on top of this core because I use a number of them currently, and they are all slightly different in how they work. For example, whether they allow setting the tag name, what they do with fields with no tags, how many tag names that package uses, how they traverse embedded structs, which types they handle, etc. Would be really nice to have a core traversal logic which encouraged some patterns, like one (configurable) tag per package/processor, consistent handling of untagged fields, etc.
Let's say I have this struct:
type User struct {
Username *string
}
and I need to sanitize it from empty string: I want Username
to be nil
if is an empty string (maybe because previous modifiers made it empty).
Is it correct to do this with this package?
Do you have any example how to setup gin with v9 and modifiers/scrubbers?
Coming from other languages this seems like a nice way to handle it without writing a custom validator/processor for every user input. The only thing is, at least for me as a newish guy to golang, I have no clue how to integrate it with gin. The validator-upgrade.go is somewhat understandable, but I'm totally lost on how and where to hook into. Where does gin and the validator do the stuff and how do I inject mold.Transformer and actually execute it, prior to the validator (binding) run?
Can you give any hints or is there any help page I've missed?
If I try to register on modifiers.New()
it compiles but doesn't work.
var conform = modifiers.New()
func init() {
conform.Register("set", transformMyData)
}
func transformMyData(ctx context.Context, t *mold.Transformer, value reflect.Value, param string) error {
value.SetString("test")
return nil
}
func Struct(myStruct interface{}) error {
if err := conform.Struct(context.Background(), myStruct); err != nil {
return err
}
return nil
}
Am I right?
What do you think about conform's email
?
https://github.com/leebenson/conform/blob/master/conform.go#L31-L51
Add docs for json tags list
Why do we need this line here?
https://github.com/go-playground/mold/blob/master/modifiers/multi.go#L21
fmt.Println(param, v.Type())
I'm coming from a Node.js background. My company is switching a lot of text processing stuff from Node to Go and we're looking for validation and sanitization libraries.
We used to use a library called "schema-inspector"
(https://www.npmjs.com/package/schema-inspector) which broke this into two steps. They have a "sanitize" step where the rules are used to modify the data. Then they have a "validate" step where if the data doesn't fit the rule, it throws an error. We use this for strings with rules like "max length". For example, if you have a rule where the username
property has a max length of 5 and this object:
{
"username": "bobbytables"
}
then after the sanitize step, you'll end up with this object:
{
"username": "bobby"
}
And then after the validate step, this would pass because the data has been transformed to meet the rule during the sanitization step. Usually situations where the data would pass sanitization but fail validation are when you've got broad rules like "max length on a string" in the sanitize step but more focussed rules like a regexp match on the validate step.
I was wondering if as the library authors, you are willing to accept PRs to add these features? I notice the following source code in the library:
// New returns a modifier with defaults registered
func New() *mold.Transformer {
mod := mold.New()
mod.SetTagName("mod")
mod.Register("trim", TrimSpace)
mod.Register("ltrim", TrimLeft)
mod.Register("rtrim", TrimRight)
mod.Register("tprefix", TrimPrefix)
mod.Register("tsuffix", TrimSuffix)
mod.Register("lcase", ToLower)
mod.Register("ucase", ToUpper)
mod.Register("snake", SnakeCase)
return mod
}
This is really simple to understand even as a Go novice. I can see how I would extend this by adding more modules I can register. There could be a module added to handle trimming based on a max length for example. If you're willing to accept PRs, I can direct the work here instead of us rolling our own.
Looking forward to hearing back from you!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.