gorilla / schema Goto Github PK

I'd like to response graceful error messages to user for given type of invalid field. See new version of ConversionError below. I'd like to implement this. Refer to link for more information: ajg/form#5

e.g.

type ConversionError struct {
    Key   string
    Type reflect.Type
    Index int    
    Err   error  
}

Hi, first thank for your great toolkit!
I'm using gorilla/schema in my app and I hope it can help me with some simple validate.
For example, with s struct:

type Foo struct {
  X string `schema:",url"`
  Y string `schema:",email"`
}

And then given a map of Validator

var validators = map[string]func([]string) error {
  "email":func(ss []string) error {/*...*/},
}

We can decode and check validate error easily.
Thanks

P/s: If you don't have time or maybe this request not suit to the pkg, I hope you can show me some tip to start it myself.

Decoder (actually parsePath) doesn't recognise the field names if they are "typed" or if they and/or if they are embedded. See the tests below.

internal/testdata

package testdata
type Aa bool

decoder_test.go

import "./internal/testdata"
s1 := &struct{
     testdata.Aa
}{}
v1 := map[string][]string{
    "Aa": {"true"},
}
decoder := NewDecoder()
_ = decoder.Decode(s1, v1)
if s1.Aa != testdata.Aa(true) {
    t.Errorf("s1: expected %v, got %v", true, s1.Aa)
}

v2 := map[string][]string{
    "F01": {"true"},
}
s2 := &struct{
    F01 testdata.Aa
}{}
_ = decoder.Decode(s2, v2)
if s2.F01 != testdata.Aa(true) {
    t.Errorf("s2: expected %v, got %v", true, s2.F01)
}

If you know your fields (and their types) only at runtime, there is no way to decode values to them. Structs have to be defined at compile time.

A possible solution would be to support maps as target values. For example:

values := map[string][]string{
    "Name":  {"John"},
    "Age": {"55"},
}
person := make(map[string]interface{})
person["Name"] = new(string)
person["Age"] = new(int)
decoder := schema.NewDecoder()
decoder.Decode(person, values)

There could also be support for nested maps, but I guess it would be ok to just support flat maps with fields like "nested.key".

Other ideas?

Registering a converter for a map field, e.g. decoder.RegisterConverter(map[string]string{}, func (input string) reflect.Value {}), currently fails with a schema: invalid path error.

The issue seems to be that the decoder does not check for registered converters when building structInfo.

So "fieldname" in a data map should match Fieldname, or FieldName, without having to define a tag. encoding/json does this: http://golang.org/src/pkg/encoding/json/decode.go#L529

I'm pretty sure I'm not doing anything obviously silly here and it's possibly that this just isn't supported as it's not explicitly called out in the overview.

At a high level I'm trying to read parameters into a struct that contains a slice of structs. Within that inner most struct I have a slice of strings (think filter parameters, tags, etc). The innermost slice simply doesn't respond as a slice type an attempts to address it as such cause errors to be thrown when parsing the field path.

Repro case follows:

package main

import (
    "encoding/json"
    "fmt"

    "github.com/gorilla/schema"
)

type Outer struct {
    Level1 []Inner
}

type Inner struct {
    Field string
    Slice []string
}

var bad = map[string][]string{
    "Level1.0.Field":   {"field"},
    "Level1.0.Slice.0": {"entry0"},
}

var good = map[string][]string{
    "Level1.0.Field": {"field0"},
    "Level1.1.Field": {"field1"},
    "Level1.2.Field": {"field2"},
    "Level1.2.Slice": {"Field2 - slice entry"},
}

func main() {
    fd := schema.NewDecoder()
    vs := good
    fs := Outer{}

    e := fd.Decode(&fs, vs)
    fmt.Println(e)
    b, e := json.MarshalIndent(fs, "", "  ")
    if e != nil {
        fmt.Println(e)
        return
    }
    fmt.Println(string(b))
    fmt.Printf("\n\n\n\n\n\n")

    vs = bad
    fs = Outer{}
    e = fd.Decode(&fs, vs)
    if e != nil {
        switch t := e.(type) {
        case schema.MultiError:
            for k, v := range t {
                fmt.Println(k)
                fmt.Printf("  %#v\n", v)
            }
            return
        default:
            fmt.Printf("%#v\n", e)
            return
        }
    }
}

And output, when run, is:

$ go install test && ./test
<nil>
{
  "Level1": [
    {
      "Field": "field0",
      "Slice": null
    },
    {
      "Field": "field1",
      "Slice": null
    },
    {
      "Field": "field2",
      "Slice": [
        "Field2 - slice entry"
      ]
    }
  ]
}






Level1.0.Slice.0
  &errors.errorString{s:"schema: invalid path \"Level1.0.Slice.0\""}

Is this behavior expected? If so what kind of overhead would be involved in updating schema to support nested slices / is that even something that would be considered being added to the library?

Thanks!

The schema documentation states:

"Note: it is a good idea to set a StructLoader instance as a package global, because it caches meta-data about structs, and a instance can be shared safely:"

I cannot find the term StructLoader in the rest of the document. Is StructLoader an old term or perhaps alternate name or misnomer for a schema.NewDecoder() object?

Hi All - I have a use case where I have little control over the incoming JSON. I'd like to do some sanitization on the object keys (all keys, on all nested objects) before I map it to a struct. Specifically, I'd like ensure that all keys have the same case (In my case, camelCase) Is there some way to scrub the JSON object before decoding it?

Great library, thank you!

I'm having a problem making UnmarshalText be nice and cooperate. Here's simplest possible code recreating the issue:

package main

import (
	"fmt"
	"github.com/gorilla/schema"
)

type Email struct {
	Address string
}

func (e *Email) UnmarshalText(text []byte) (err error) {
	e.Address, err = string(text), nil
	return
}

type Person struct {
	Name   string
	Emails []Email
}

func main() {
	v := map[string][]string{
		"name":   {"John"},
		"emails": {"foo", "bar"},
	}
	var p Person

	err := schema.NewDecoder().Decode(&p, v)
	if err != nil {
		fmt.Println(err)
	}

	fmt.Println(p)
}

Output:

schema: invalid path "emails"
{John []}

Is there anything broken or (as is more likely!) am I doing sth wrong?

Seems like UnmarshalText should take precedence for converting values. This would be useful for defining enums and returning an error if the value isn't one of the valid values

package main

import (
    "errors"
    "fmt"

    "github.com/gorilla/schema"
)

type Foo string

func (f *Foo) UnmarshalText(text []byte) error {
    if string(text) != "foo" {
        return errors.New("not foo")
    }
    *f = Foo(text)
    return nil
}

type Stuff struct {
    Value Foo
}

func main() {
    stuff := new(Stuff)
    values := map[string][]string{
        "Value": {"abc"},
    }
    decoder := schema.NewDecoder()
    err := decoder.Decode(stuff, values)

    fmt.Println(err) // <nil>
    fmt.Println(stuff) // &{abc}
}

I am curious if support is included for something like:

var invoices []domain.Invoice
decoder := schema.NewDecoder()
decoder.Decode(invoices, r.PostForm)

The params would come in like Value.0 + Value.1, each value would represent a different domain.Invoice. The goal of this is to easily create a slice of structs. I saw that there is support for a slice of structs if it is a nested value, but I am needing a slice of structs directly. Is this possible?

Hi, I'm using gorilla/schema in my project, however I have an error that am not really sure how to deal with. I will minimize my case as:
I have a structure like:

type Session struct {
	gorm.Model
	Email string
	Password []byte `schema:"-"`
}

I don't want to assign directly the password as I need to hash it using BCrypt, so that's the why I set that tag. So, from Postman I am sending:

pass: 12345
Email: [email protected]

And I get this error:
schema: invalid path "pass"

I already tried sending as 'Password' but nothing, I want to assign all the others fields but not the password. If I don't sent the password the error is gone (but I need to receive the pass)... So, what can be happening here?

We vendor in our dependencies currently, but personally my gospace is still shared across different packages (as a project is just a package anyways). Because of that i think it would be a good idea to use gopkg.in and tag the packages.

What do you think about that?

I'll leave this IRC conversation as a quick request for feedback:

<pepee> honestly, this is still making me nervous:  http://www.gorillatoolkit.org/pkg/schema
<moraes> pepee, why?
<pepee> moraes, what if you pass a very long slice? or a long field in a form, etc.
<moraes> peppee, that's an interesting issue. probably not out of scope for the package.
<moraes> or for those kinds of attacks you shouldn't even touch request data
<moraes> pepee, so: 1. server check (413 status); 2. gorilla/schema could apply a limit tag
<pepee> moraes, yeah, there should be ways of limiting input, at least for the slices
<moraes> i will open an issue
<pepee> they should at least indicate that in the docs
<pepee> "make sure you sanitize your inputs, your user could be sending long strings, blah blah"
<moraes> yup.

The question is: Should gorilla/schema care about DoS attacks, at all? Or this is out of scope and should be handled in the server level or at most before submitting data to the package?

Until #58 or #64 is resolved, can the documentation be updated to convey UnmarshalText does not yet work for type aliases?

http://www.gorillatoolkit.org/pkg/schema
There's also the possibility to create a custom type that implements the TextUnmarshaler interface, and in this case there's no need to registry a converter, like:

type Person struct {
  Emails []Email
}

type Email struct {
  *mail.Address
}

func (e *Email) UnmarshalText(text []byte) (err error) {
    e.Address, err = mail.ParseAddress(string(text))
    return
}

http://www.gorillatoolkit.org/pkg/schema

Is there any reason not to export convertors? I need to decode a payload which is composed either by structs or /and non-struct types (e.g. map[string]float64). Currently the decoder errors out if the destination is not a struct so I've forked the repo and just exported convertors.

Huge fan of your product. I seem unable to figure out, how to encode a simple slice type type People []Person. Using

package main

import (
        "fmt"

        "github.com/gorilla/schema"
)

type Person struct {
        Name string `schema:"name"`
        Age  int
}

type People []Person

func main() {
        jay := Person{"Jay", 34}
        may := Person{"May", 43}
        peeps := People{jay, may}

        values := make(map[string][]string)
        encoder := schema.NewEncoder()
        err := encoder.Encode(peeps, values)
        if err != nil {
                panic(err)
        }
        fmt.Printf("Encoded people: %v\n", values)
}

I get the error panic: schema: interface must be a struct.
Wrapping People into a struct now called Group also does not work:

package main

import (
        "fmt"

        "github.com/gorilla/schema"
)

type Person struct {
        Name string `schema:"name"`
        Age  int
}

type Group struct {
        People []Person
}

func main() {
        jay := Person{"Jay", 34}
        may := Person{"May", 43}
        peeps := Group{[]Person{jay, may}}

        values := make(map[string][]string)
        encoder := schema.NewEncoder()
        err := encoder.Encode(peeps, values)
        if err != nil {
                panic(err)
        }
        fmt.Printf("Encoded group %v\n", values)
}

Exits with error:
panic: schema: encoder not found for [{Jay 34} {May 43}]

What am I doing wrong here? Ideally I would like to work with the first version of type People []Person.

Hello,

I expected this behaviour which match encoding/json package.

Example: https://play.golang.org/p/o6IiMmNMK9

I wrote below the test to match the behaviour above but it fails.

The unit test:

type E5 struct {
	F01 int     `schema:"f01,omitempty"`
	F02 string  `schema:"f02,omitempty"`
	F03 *string `schema:"f03,omitempty"`
	F04 *int8   `schema:"f04,omitempty"`
	F05 float64 `schema:"f05,omitempty"`
}

func TestEncoderWithOmitempty(t *testing.T) {
	vals := map[string][]string{}

	s := E5{}

	encoder := NewEncoder()
	encoder.Encode(&s, vals)

	valNotExists(t, "f01", vals)
	valNotExists(t, "f02", vals)
	valNotExists(t, "f03", vals)
	valNotExists(t, "f04", vals)
	valNotExists(t, "f05", vals)
}

Here is the trace:

Running tool: /usr/local/bin/go test -timeout 30s -tags  -run ^TestEncoderWithOmitempty$

--- FAIL: TestEncoderWithOmitempty (0.00s)
	/Users/aetcheverry/Projects/Go/src/github.com/gorilla/schema/encoder_test.go:273: Key not ommited. Expected: empty; got: 0.
	/Users/aetcheverry/Projects/Go/src/github.com/gorilla/schema/encoder_test.go:273: Key not ommited. Expected: empty; got: null.
	/Users/aetcheverry/Projects/Go/src/github.com/gorilla/schema/encoder_test.go:273: Key not ommited. Expected: empty; got: null.
	/Users/aetcheverry/Projects/Go/src/github.com/gorilla/schema/encoder_test.go:273: Key not ommited. Expected: empty; got: 0.000000.
FAIL
exit status 1
FAIL	github.com/gorilla/schema	0.005s
Error: Tests failed.

Thanks for your help. 😃

If you have a query of b.0.Foo=1&b.0.Bar=2 and b implements TextUnmarshaller then UnmarshalText won't be called. However, if the query is b.0=Foo%3D1%26Bar%3D2 then it works as expected, but I can't expect the caller to know which structs are custom and which aren't.

Test/example case:

package main

import (
	"net/url"
	"github.com/gorilla/schema"
)

type B struct {
	C string
	d interface{}
}

func (b *B) UnmarshalText(text []byte) error {
	q, err := url.ParseQuery(string(text))
	if err != nil {
		return err
	}
	b.C = q.Get("C")
	if b.C == "foo" {
		// you would probably do something different here, but example
		b.d = struct {
			E string
		}{q.Get("E")}
	}
	return nil
}

type A struct {
	B []B
}

func main() {
	d := schema.NewDecoder()

	q := url.Values{}
	q.Set("B.0.C", "foo")
	q.Set("B.0.E", "bar")
	a := &A{}
	d.Decode(a, q)
	// UnmarshalText is never called
	// a is &{B:[{C:foo d:<nil>}]}

	q = url.Values{}
	q.Set("B.0", "C=foo&E=bar")
	a = &A{}
	d.Decode(a, q)
	// UnmarshalText is called
	// a is &{B:[{C:foo d:{E:bar}}]}
}

Hi,
thanks for creating such great package. It did save my time.

For my understanding, if a type implemented encoding.TextUnmarshal interface, it should be
decode according to how we implement UnmarshalText function, right?
But slice type is a different story, because you miss test the interface in L137 https://github.com/gorilla/schema/blob/master/decoder.go#L137

and the following changes works for me

+        _, ok := v.Interface().(encoding.TextUnmarshaler)
-       if conv == nil && t.Kind() == reflect.Slice {
+       if conv == nil && t.Kind() == reflect.Slice && !ok {
                var items []reflect.Value
                elemT := t.Elem()
                isPtrElem := elemT.Kind() == reflect.Ptr
                if isPtrElem {
                        elemT = elemT.Elem()
                }

let me know my understand is wrong, thanks

As example given on home page, when dealing with http request parameters, we have fields that must be nonempty value:

type Person struct {
    Name  string `schema:"name, required"`  // custom name
    Phone string `schema:"phone"` // custom name
    Admin bool   `schema:"-"`     // this field is never set
}

func MyHandler(w http.ResponseWriter, r *http.Request) {
    err := r.ParseForm()

    if err != nil {
        // Handle error
    }

    person := &Person{}
    decoder := schema.NewDecoder()

    // r.PostForm is a map of our POST form values
    err := decoder.Decode(person, r.PostForm)

    if err != nil {
        // error show up when required field empty
    }
    // Do something with person.Name or person.Phone
}

error show up when calling Decode if any required field empty. I got strong desire on this function.

I want to decode to a struct, which embeds fields of another struct. If I do this, I get an "invalid path" error, because schema can't find the field. That's because the cache's create function iterates over the fields using the Field method reflect.Type, which doesn't include embedded fields (but the embedded struct name). A solution would be to use FieldByName (which finds embedded fields) or iterate over the embedded fields.

Hi,

I d like to know if there s any chance the lib could handle file uploads too.

I d like very much to be able to write something like this,

type UserForm struct {
	fw.Form
	Username string `schema:"Username" conform:"trim" validate:"required,gt=4"`
	Email string `schema:"Email" conform:"trim" validate:"required,email,gt=4"`
        UserImage FileUpload `schema:"UserImage"`
}

So later i can add validations on it,

type UserForm struct {
	fw.Form
	Username string `schema:"Username" conform:"trim" validate:"required,gt=4"`
	Email string `schema:"Email" conform:"trim" validate:"required,email,gt=4"`
        UserImage FileUpload `schema:"UserImage" validate:"type=jpeg|gif,sizelt=1200,maxHeight=xxx,maxWidth=yyy"`
}

What do you think ? Can this be done ?

Hi
This is the error that I get

go get github.com/gorilla/schema
# github.com/gorilla/schema
schema\decoder.go:22: undefined: cache

I am running on Windows 10 with go version go1.8.1 windows/amd64

package something

import (
    "github.com/gorilla/schema"
)

var decoder = schema.NewDecoder().IgnoreUnknownKeys(true) // valid syntax

The syntax above is valid go, and is how the docs suggest you use gorilla (keeping a global instance)

however, doing this won't be valid

package something

import (
    "github.com/gorilla/schema"
)

var decoder = schema.NewDecoder()
decoder.IgnoreUnknownKeys(true)  // syntax-error: non-declaration statement outside function body

Sounds like a simple enough change. I'd be happy to send a PR.
What do you guys think?

Found this as a dependency in slackbot and have no clue what that is

Hi, I'm using gorilla/schema to populate my structure, righ now I'm getting a strange behaviour when I Decode the form into my struct. I am developing a CRUD for the table users and right now I am making the Update part.. So, briefly I get an ID, search in db for that register and finally I decode the form into the existent user in order to override the new fields to update, but now, I am testing with the field name and it's not assigned unless I retrieve that value before using req.FormValue . I have something like:

type User struct {
	gorm.Model
	Name string
       Active bool
}

Then in the code to set the values:

user := models.User{}
decoder := schema.NewDecoder()
err := decoder.Decode(&user, req.PostForm)

But that doesn't work, intead of that I have to retrieve some value from the form and then the decode is made successfully, something like:

user := models.User{}
fmt.Println(req.FormValue("Name"))
decoder := schema.NewDecoder()
err := decoder.Decode(&user, req.PostForm)

And here that works pretty well, I have to consult the value I don't know why.

Decode() method panics if source data contain invalid fields having path parts with names of other fields.

Its better to illustrate this with example. This code will panic on execution:

package main

import "fmt"
import "github.com/gorilla/schema"

func main() {
        type Person struct {
                Person string `schema:"person"`
                Phone  string `schema:"phone"`
        }

        values := map[string][]string{
                "person.phone":  {"John"},
                "phone":  {"999-999-999"},
        }
        person := new(Person)
        decoder := schema.NewDecoder()
        decoder.Decode(person, values)
        fmt.Println(person)
}

The panic occurs only if source data have names of other variables. For example changing values to:

        values := map[string][]string{
                "person.phones":  {"John"},
                "phone":  {"999-999-999"},
        }

is handled correctly (phone part of the path is changed to phones which does not collide with other attribute name).

Values is usually in control of the client of the application. This bug allows client to cause panic on the server by manipulating GET or POST arguments.

When calling decode, if you pass a pointer to an interface variable, you get an error:

schema: interface must be a pointer to struct

Here is code to reproduce:

import (
    "testing"
    "github.com/gorilla/schema"
)

type Cat struct {
    name string
}

type SomethingGeneric struct {
    getSomething func() interface{}
}

func getSomeCat() interface{} {
    return Cat{}
}

var somethingForCats = SomethingGeneric{getSomething: getSomeCat}

var decoder = schema.NewDecoder()

func TestCanDecodeInterfacePointer(t *testing.T) {
    dst := somethingForCats.getSomething()

    values := map[string][]string{
        "name": {"Tommy"},
    }

    err := decoder.Decode(&dst, values)

    if err != nil {
        t.Errorf("Could not decode.  Got error %v", err.Error())
    }

}

To avoid some boilerplate code, I need to be able to have some reusable code that can pass an interface variable pointer to decode.

Situation

Have a user struct, populated via a database query that includes a LastName field already populated
We have a user update HTML form which posts the updates.

Issue

When LastName, or other fields, are set to blank in the HTML input the form posts the following:

map[IsActive:[true] FirstName:[Joey] LastName:[] PreferredFirstName:[Joey] Username:[joeybloggs]]

this is correct as you can see LastName has no value; however after running schema decoder like so:

// user originally grabbed from DB in middleware and set on context ( using gorilla contaxt )
user := context.Get(r, "user").(*User)

if err = schemaDecoder.Decode(user, r.MultipartForm.Value); err != nil {
  log.Println(err.Error())
  http.Error(w, "Invalid Request", http.StatusBadRequest)
  return
}

fmt.Println(user.LastName) // outputs Bloggs, the fields original value

LastName was not set to blank and still retains it's original value.

My specific problem

the user struct is validated after schema decoder runs and in this situation validation should report an error, however it all passes, user gets saved to the database and user is notified that the user saved successfully.

Notes

This is not the same issue as #17 as I don't want to zero out fields not posted.

For instance, if I GET /path?key=banana&key=pear, then use schema to parse the url into a type fruit struct { key string }, I'll get either 'banana' or 'pear' (undefined behavior).

I would prefer to get a type error, as I was expecting a string but got an array of string.

Thoughts?

Question about best practice when dealing with form that:

• Has multiple checkboxes as an answers to one question, eg. Shop Type: [] electronics, [] groceries, [] wines, [] something
• Answer options (checkbox values and ids) are fetched from DB

I thought that the best option would be to go with nested struct:

type shopType struct {
    Id int
    Type string
    Selected bool
}

type shopForm struct {
    Name string
    Types []shopType
}

Then populating shopForm.Types from DB and using it as a context in template. Then in template:

{{ range $idx, $type := .Types }}
    <label class="checkbox-inline"><input type="checkbox" name="Types.{{ $idx }}.Selected" {{ if $type.Selected }}checked{{ end }}> {{ $type.Type }}</label>
{{ end }}

But the problem with this approach is that I can only decode Selected field and I'm missing Id and Type. I could add hidden input fields like this:

<input type="hidden" name="Types.{{ $idx }}.Id" value="{{ $type.Id }}">
<input type="hidden" name="Types.{{ $idx }}.Type" value="{{ $type.Type }}">

and then it works OK, but it's ugly and it can't be done when dealing with select option form.

I'm probably missing something obvious :)

Right now gorilla/schema does not have a way to omit non present values to empty.
For example :

type params struct {
   Username string `schema:"name"`
   Password string `scehma:"password,setempty"`
}

If the input map does not contain the key "password" it should set it to an empty value if the tag is provided.

We are running into an issue with this right now.

I have this snippet here, the problem is that the error returned by the Encode method gives (0 errors), this is what err displays when I print in int stdout. I'm not sure if an error occurred or should I treat this as a no error and move on with the code...

func (t *Ticket) Create() {
	encoder := schema.NewEncoder()

	// setup form values
	form := url.Values{}
	err := encoder.Encode(t, form)
	if err != nil {
		panic(err)
	}
}

I've been using schema for a while, but I'd also like the ability to encode url values with the fields of a struct. I've created a basic encoder here, do you have any interest in adding something like this to schema?

Hi,

I've been trying to do this in really many ways, and I took the time to provide a test case with the scenario, does anyone have any ideas how I could work-around this problem ?

I have a object Tags []string and its part of a bigger collection, but the problematic point is this []string.

When I created a update view for this collection, the View is sending the form encoded as:

tags%5B%5D=Conservationists&tags%5B%5D=Zimbabwe

However schema returns me back schema: invalid path "tags[]"

It works if I change the struct schema to schema:"tags[]", but this is awful.

This is the full test with the reported problem:

package test

import (
    "testing"
    "net/http"
    "log"
    "net/http/httptest"
    "fmt"
    "io/ioutil"
    "github.com/stretchr/testify/assert"
    "github.com/gorilla/schema"
    "encoding/json"
    "strings"
)

type Tag struct {
    Tags []string                    `schema:"tags" json:"tags,omitempty"`
}

func TestHTTPMarshalProblem(t *testing.T) {

    ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        err := r.ParseForm()
        assert.Equal(t, err, nil)
        assert.NotEqual(t, r.Form, nil)
        assert.NotEqual(t, r.PostForm, nil)

        log.Println("PostForm ",r.PostForm)
        decoder := schema.NewDecoder()
        targetObject := new(Tag)

        // r.PostForm is a map of our POST form values
        err = decoder.Decode(targetObject, r.PostForm)
        if err != nil {
            log.Println(err)
            assert.Equal(t, err, nil)
            log.Println("targetObject ", targetObject)
        }
        assert.NotNil(t, targetObject,"err should be something")

        ret, err := json.Marshal(&targetObject)
        if err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }

        if len(ret) == 0 {
            log.Println("404 not found")
            w.WriteHeader(http.StatusNotFound)
            return
        }

        w.Write(ret)

    }))
    defer ts.Close()

    var str = "tags%5B%5D=Conservationists&tags%5B%5D=Zimbabwe&tags%5B%5D=have&tags%5B%5D=accused&tags%5B%5D=American&tags%5B%5D=being&tags%5B%5D=alleged&tags%5B%5D=Cecil&tags%5B%5D=Africa%E2%80%99s&tags%5B%5D=lions"

    res, err := http.Post(ts.URL, "application/x-www-form-urlencoded; charset=UTF-8", strings.NewReader(str))

    if err != nil {
        log.Fatal(err)
    }
    obj, err := ioutil.ReadAll(res.Body)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("result %s", obj)
}

output:

2016/05/18 13:30:32 PostForm  map[tags[]:[Conservationists Zimbabwe have accused American being alleged Cecil Africa’s lions]]
2016/05/18 13:30:32 schema: invalid path "tags[]" 
2016/05/18 13:30:32 targetObject  &{[]}

With this struct annotation, Decode() silently fails to decode the array:

type MyStruct struct {
   MyArray []uint32 `schema:"myArray"`
}

With this struct annotation, Decode() succeeds:

type MyStruct struct {
   MyArray []uint32 `schema:"myArray[]"`
}

Any objections to adding this to the documentation?

Right now this package requires the 'schema' tag to be set on each structure. In my particular workflow it's possible for the web client to submit a structure via JSON or via the forms collection. Ideally schema would be able to use the 'schema' tag or the 'json' tag (preferring the first if they both exist) to decode an incoming request.

I expected the following code to decode the HTML form after reading the []Email example in the docs. Is this a bug or am I misunderstanding?

package main

import (
    "io"
    "log"
    "net/http"

    "github.com/gorilla/schema"
)

type Foo struct {
    Bars []string
}

func main() {
    http.HandleFunc("/", handler)
    log.Fatal(http.ListenAndServe(":3000", nil))
}

func handler(w http.ResponseWriter, req *http.Request) {
    if req.Method == http.MethodPost {
        err := req.ParseForm()
        if err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }
        var foo Foo
        decoder := schema.NewDecoder()
        //decoder.IgnoreUnknownKeys(true)
        err = decoder.Decode(&foo, req.PostForm)
        if err != nil {
            // schema: invalid path "Bars.0" (and 2 other errors)
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }
        log.Println(foo) // prints {[]} with IgnoreUnknownKeys(true)
        http.Redirect(w, req, "/", http.StatusSeeOther)
        return
    }
    io.WriteString(w, `<html>
<body>
<form method="post" action="/">
  <input type="hidden" name="Bars.0" value="a">
  <input type="hidden" name="Bars.1" value="b">
  <input type="hidden" name="Bars.2" value="c">
  <button type="submit">Submit</button>
</form>
</body>
</html>`)
}

Currently if I add a custom converter like decoder.RegisterConverter([]string{}, func (input string) reflect.Value {}) it wont call the custom converter but instead ignore it.

package schema_test

import (
    "fmt"
    "reflect"
    "testing"

    "github.com/gorilla/schema"
)

func TestStrings(t *testing.T) {
    decoder := schema.NewDecoder()
    decoder.RegisterConverter([]string{}, func(input string) reflect.Value {
        fmt.Printf("%+v", input)

        return reflect.ValueOf(input)
    })

    i := struct {
        Multiple []string `schema:"multiple"`
    }{}

    decoder.Decode(&i, map[string][]string{
        "multiple": []string{"one,two,three"},
    })
}

Would have suspected i.Multiple to contain a []string{"one", "two", "three"}

I need a way to explicitly set a field value to empty and not have it be ignored by gorilla/schema when processed. It is important to not simply wipe out all missing fields, but only make empty the fields that were provided and somehow marked as intentionally empty.

Use case: struct that is pre-populated from a database entry that is then passed into schema.Decoder to apply matched fields from a web form.

Possibly if the fields exist in the incoming web form and are empty, they can be wiped out while leaving alone the fields in the struct that weren't referenced in the web form. Such a thing would need testing that I haven't myself done.

I've created a very hacky patch[1] to allow deliberately unsetting a field value when empty, instead of just ignoring it. In an application I'm building, I pull in a record from the database and then use schema to merge changes. When the data coming in from a web form is empty, I intend for the value to be wiped out, not just ignored. But because of how structs work, it's really not easy to tell the difference between absent field versus a deliberately empty value. In this very hacky patch, if it hits a string value of "[[[ERASE]]]" it will set the field value to "". It's not good enough and is bound to have loads of gotchas. Maybe there an alternative way to do this that I've missed; if so, sorry for the wishlist request.

[1] scjalliance@fdf73b8 (don't actually use this commit)

If multiple required fields are missing, only the first missing field is described in the error message. The error message should list all missing fields.

In the decoder, lines 110-112 cause a panic in the handling of slices, due to the items array having already been created according to the length of values. Skipping the handling of a value causes the corresponding index in items to contain invalid/zero Value objects.

Adding insult to injury, this even occurs when parsing a slice of strings (e.g. { "0", "1", "" } => []string => panic!).

Seems like this package doesn't allow struct to contain *struct field. Is there any plan to support this?

Here the code that triggers panic:

Here is the panic message:

--- FAIL: TestSlices (0.00s)
exit status 2
FAIL	example.com/x/vendor/github.com/gorilla/schema	0.010s
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation, code=0x1 addr=0x0 pc=0x111e298]

goroutine 6 [running]:
testing.tRunner.func1(0xc420074750)
	/usr/local/Cellar/go/1.8.1/libexec/src/testing/testing.go:622 +0x29d
panic(,0x1145d40, 0x1216840)
	/usr/local/Cellar/go/1.8.1/libexec/src/runtime/panic.go:,489 +0x2cf
example.com/x/vendor/github.com/gorilla/schema.typeEncoder.func1(0x1131640, 0xc42001ea70, 0x1b6, 0x5, 0x1131640)
	/Users/kurt/gopath/src/example.com/x/vendor/github.com/gorilla/schema/encoder.go:117 +0xb8
example.com/x/vendor/github.com/gorilla/schema.(*Encoder).encode(0xc4200132b0, 0x1135740, 0xc42001ea00, 0x16, 0xc42000eb40, 0xc420013280, 0xc4200132c0)
	/Users/kurt/gopath/src/example.com/x/vendor/github.com/gorilla/schema/encoder.go:63 +0x433
,example.com/x/vendor/github.com/gorilla/schema.(*Encoder).Encode(0xc4200132b0, 0x1135740, 0xc42001ea00, 0xc42000eb40,, 0xc42000eb40, 0xc420022000)
	/Users/kurt/gopath/src/example.com/x/vendor/github.com/gorilla/schema/encoder.go:,29 +0x6b
example.com/x/vendor/github.com/gorilla/schema.TestSlices(,0xc420074750)
	/Users/kurt/gopath/src/example.com/x/vendor/github.com/gorilla/schema/encoder_test.go:161, +0x2ec
testing.tRunner(0xc420074750,, 0x11764e0)
	/usr/local/Cellar/go/1.8.1/libexec/src/testing/testing.go:657, +0x96
created by testing.(*T).Run,
	/usr/local/Cellar/go/1.8.1/libexec/src/testing/testing.go:697 +0x2ca
Error: Tests failed.

6e77d17#diff-e03ef7ae325cb767cea50982da58c5cdR58

type Custom1 string
and
type Custom2 string

are of same Kind(), but can use totally different converter, I think this recent change will led to overwriting of converters registered of totally different types.

I have the following struct

type DeviceProfile struct {
    Title        string `json:"title"`
    Model        string `json:"model"`
    Mfr          string `json:"mfr"`
    Ssl          bool   `json:"ssl"`
}

And a checkbox in my html form:

<input type="checkbox" name="Ssl">

When I decode the form DeviceProfile.Ssl is always false because the HTML value for a checked checkbox is the string "on".

It seems like it would be helpful to accept "on" as a valid boolean true value. Note that if the checkbox is not checked the returned value is empty.

The code below provokes a panic. If I revert the changes done in e6c8221 / #91 the code below works.

I am not sure what exactly how the problem can be solved. I tried to tweak the fix in e6c8221 but I didn't make too much progress.

func TestRegisterEncoderCustomArrayType(t *testing.T) {
	vals := map[string][]string{}
	
	type CustomInt []int
	s1 := &struct {
		SomeInts CustomInt `schema:",omitempty"`
	}{
		CustomInt{1,2,3},
	}
	
	encoder := NewEncoder()
	encoder.RegisterEncoder(CustomInt{}, func(value reflect.Value) string {
		return fmt.Sprint(value.Interface())
	})
	
	encoder.Encode(s1, vals)
	t.Log(vals)
}

panic: runtime error: comparing uncomparable type schema.CustomInt [recovered]
	panic: runtime error: comparing uncomparable type schema.CustomInt

goroutine 5 [running]:
testing.tRunner.func1(0xc4200a60f0)
	/usr/local/Cellar/go/1.9.2/libexec/src/testing/testing.go:711 +0x2d2
panic(0x114d340, 0xc420048720)
	/usr/local/Cellar/go/1.9.2/libexec/src/runtime/panic.go:491 +0x283
github.com/gorilla/schema.(*Encoder).encode(0xc42006bc18, 0x113cc20, 0xc42000a0c0, 0x16, 0xc420074390, 0xc42000a0e0, 0xc420044d28)
	/Users/olve/go/src/github.com/gorilla/schema/encoder.go:70 +0x78b
github.com/gorilla/schema.(*Encoder).Encode(0xc420044c18, 0x113cc20, 0xc42000a0c0, 0xc420074390, 0xc420044c60, 0x56600000002)
	/Users/olve/go/src/github.com/gorilla/schema/encoder.go:29 +0x6b
github.com/gorilla/schema.TestRegisterEncoderCustomArrayType(0xc4200a60f0)
	/Users/olve/go/src/github.com/gorilla/schema/encoder_test.go:352 +0x3a9
testing.tRunner(0xc4200a60f0, 0x117f3a0)
	/usr/local/Cellar/go/1.9.2/libexec/src/testing/testing.go:746 +0xd0
created by testing.(*T).Run
	/usr/local/Cellar/go/1.9.2/libexec/src/testing/testing.go:789 +0x2de

For example, I have a struct:

type Post struct { Images []string }

If I want to put in the struct values like (contains three elements, including empty string):

["a", "", "b"]

After decoding I have a slice with two elements excluding an empty string as a result.

Could you please tell me if it Is a bug or a way it works? Thank you!