go-kivik / kivik Goto Github PK
View Code? Open in Web Editor NEWCommon interface to CouchDB or CouchDB-like databases for Go and GopherJS
License: Other
Common interface to CouchDB or CouchDB-like databases for Go and GopherJS
License: Other
It ought to be possible to use a kivik.Attachment
value properly embedded in a document for Put()
, Get()
,AllDocs()
, etc. This needs to be tested and documented.
I'm creating this issue to inform kivik users of an upcoming change, and to invite comment.
My goal is to split the Kivik package up into sub-packages. This is to reduce the dependency tree, as well as to isolate changes to the sub-packages where they matter.
The new packages will be (roughly):
Possibly concurrently, I would like to move all Kivik packages to the new go-kivik namespace.
My plan, such as it is, is to piecemeal duplicate the existing Kivik components (couchdb, pouchdb, etc), into the new namespace. Changes will only be added to the new packages under the go-kivik
namespace. This will allow existing installations to continue functioning during a transition period, while encouraging new users to use the new packages.
Eventually, likely after all sub-packages have been moved to go-kivik
, I will take the plunge, and transfer the core kivik
repository to the go-kivik
namespace. GitHub will handle redirection, but that likely won't be sufficient, due to the kivik/driver
sub-package.
Of course my goal is to make this transition as painless as possible.
Any comments or suggestions are welcome.
The final rename will involve:
kivik
and sub-packages to the old repo name [#260]github.com/go-kivik/kivik
kivik
and remaining sub-packages to the new repo name, with a 2.x tag.*Once the new driver packages are published, I intend not to add any features or bugfixes to the legacy packages. After some time (probably 30-90 days or so), I'll add a single warning (probably in an init
function) that warns that the driver is obsolete, and point to the new location. Hopefully this will get the attention of anyone continuing to download the old location. After a further wait, I will remove the legacy drivers entirely.
WARNING: DATA RACE
Read at 0x00c420184de0 by goroutine 55:
runtime.mapaccess2_faststr()
/home/travis/.gimme/versions/go/src/runtime/hashmap_fast.go:317 +0x0
net/http.(*Request).write()
/home/travis/.gimme/versions/go/src/net/http/request.go:555 +0x59b
net/http.(*persistConn).writeLoop()
/home/travis/.gimme/versions/go/src/net/http/transport.go:1754 +0x2ee
Previous write at 0x00c420184de0 by goroutine 44:
runtime.mapassign_faststr()
/home/travis/.gimme/versions/go/src/runtime/hashmap_fast.go:600 +0x0
net/textproto.MIMEHeader.Set()
/home/travis/.gimme/versions/go/src/net/textproto/header.go:22 +0x74
net/http.Header.Set()
/home/travis/.gimme/versions/go/src/net/http/header.go:31 +0x60
net/http.(*Request).AddCookie()
/home/travis/.gimme/versions/go/src/net/http/request.go:385 +0x31b
net/http.(*Client).send()
/home/travis/.gimme/versions/go/src/net/http/client.go:170 +0x118
net/http.(*Client).Do()
/home/travis/.gimme/versions/go/src/net/http/client.go:595 +0x458
github.com/flimzy/kivik/driver/couchdb/chttp.(*Client).DoReq()
/home/travis/gopath/src/github.com/flimzy/kivik/driver/couchdb/chttp/chttp.go:182 +0x187
github.com/flimzy/kivik/driver/couchdb/chttp.(*Client).DoJSON()
/home/travis/gopath/src/github.com/flimzy/kivik/driver/couchdb/chttp/chttp.go:133 +0x9a
github.com/flimzy/kivik/driver/couchdb.(*db).Put()
/home/travis/gopath/src/github.com/flimzy/kivik/driver/couchdb/db.go:144 +0x37d
github.com/flimzy/kivik.(*DB).Put()
/home/travis/gopath/src/github.com/flimzy/kivik/db.go:70 +0xaf
github.com/flimzy/kivik/test/db.testPutAttachment.func4.1()
/home/travis/gopath/src/github.com/flimzy/kivik/test/db/putattachment.go:101 +0xeb
github.com/flimzy/kivik/test/kt.Retry()
/home/travis/gopath/src/github.com/flimzy/kivik/test/kt/kt.go:270 +0x62
github.com/flimzy/kivik/test/db.testPutAttachment.func4()
/home/travis/gopath/src/github.com/flimzy/kivik/test/db/putattachment.go:99 +0x250
github.com/flimzy/kivik/test/kt.(*Context).Run.func1()
/home/travis/gopath/src/github.com/flimzy/kivik/test/kt/kt.go:155 +0x23c
testing.tRunner()
/home/travis/.gimme/versions/go/src/testing/testing.go:746 +0x16c
Goroutine 55 (running) created at:
net/http.(*Transport).dialConn()
/home/travis/.gimme/versions/go/src/net/http/transport.go:1169 +0xbc9
net/http.(*Transport).getConn.func4()
/home/travis/.gimme/versions/go/src/net/http/transport.go:925 +0xa2
Goroutine 44 (running) created at:
testing.(*T).Run()
/home/travis/.gimme/versions/go/src/testing/testing.go:788 +0x568
github.com/flimzy/kivik/test/kt.(*Context).Run()
/home/travis/gopath/src/github.com/flimzy/kivik/test/kt/kt.go:152 +0x103
github.com/flimzy/kivik/test/db.testPutAttachment()
/home/travis/gopath/src/github.com/flimzy/kivik/test/db/putattachment.go:92 +0x5e5
github.com/flimzy/kivik/test/db.putAttachment.func1.1.2()
/home/travis/gopath/src/github.com/flimzy/kivik/test/db/putattachment.go:31 +0x95
github.com/flimzy/kivik/test/kt.(*Context).Run.func1()
/home/travis/gopath/src/github.com/flimzy/kivik/test/kt/kt.go:155 +0x23c
testing.tRunner()
/home/travis/.gimme/versions/go/src/testing/testing.go:746 +0x16c
At the moment, the only explicit Context support in the PouchDB bindings is for timeout contexts, which sets the AJAX timeout. (And, of course, just returning early, while leaving the process running in the background, if a context is cancelled--not ideal). But cancelling contexts is probably more common, and I should try to cancel an existing AJAX connection when that happens.
I'm reading some JSON from a CouchDB, and the json is stored as:
{
"id": 3411441,
"type": "User"
},
and if I read it in and then marshal it back with:
type Document struct {
Field *json.RawMessage // or interface{} is giving the same result
}
// read
options := kivik.Options{}
row, err := db.Get(ctx, docid, options)
doc := Document{}
row.ScanDoc(&doc)
// marshal
outputBytes, err := json.Marshal(doc)
It ends up producing:
{
"id": 3411441.0,
"type": "User"
},
(note the .0, meaning it got turned into a float)
Then later if I try to unmarshal the json into a struct that is defined as:
type user struct {
id int
type string
}
it fails with: "Error executing action: json: cannot unmarshal number 3411441.0 into Go value of type int"
According to this Stack Overflow post a recommended approach is to use the UseNumber()
method on json.Decoder
UseNumber causes the Decoder to unmarshal a number into an interface{} as a Number instead of as a float64.
If ServerInfo() is worth keeping, it should return a data structure defined in the kivik package, not in driver.
Explainer
was added later, so to preserve backward compatibility it was made a separate interface. At the 2.0 milestone, this should be re-integrated with the Finder
interface.
This doesn't seem important enough to export.
v1.1.0
go version
) -- Kivik 1.0 aims to support all versions of Go from 1.7v1.9
err = client.Authenticate(context.TODO(), &chttp.CookieAuth{
Username: "user",
Password: "foo",
})
fmt.Println(err) // log: <nil>
session, err := client.Session(context.TODO())
fmt.Println(session, err) // log: <nil> kivik: driver does not support sessions
What did you expect to see?
Session should have "RawResponse" member which contain additional fields with an "Token"
What did you see instead?
Log: kivik: driver does not support sessions.
It seems this feature is not implemented https://github.com/flimzy/kivik/blob/325bae4e7ca4f19e39d6d539aacb21044517624c/session.go#L41 or im doing something wrong.
Using with kivik: How can i read/retrieve the cookie token with kivik after Authenticated and how can i use it for the next authentication?
WARNING: DATA RACE
Write at 0x00c4201279f0 by goroutine 52:
github.com/flimzy/kivik/driver/couchdb/chttp.EncodeBody.func1()
/home/jonhall/go/src/github.com/flimzy/kivik/driver/couchdb/chttp/chttp.go:213 +0x14c
Previous read at 0x00c4201279f0 by goroutine 38:
github.com/flimzy/kivik/driver/couchdb.(*db).SetSecurity()
/home/jonhall/go/src/github.com/flimzy/kivik/driver/couchdb/db.go:241 +0x2f4
github.com/flimzy/kivik.(*DB).SetSecurity()
/home/jonhall/go/src/github.com/flimzy/kivik/db.go:163 +0x1ca
github.com/flimzy/kivik/test/db.testSetSecurity.func1()
/home/jonhall/go/src/github.com/flimzy/kivik/test/db/security.go:109 +0x9d
github.com/flimzy/kivik/test/kt.Retry()
/home/jonhall/go/src/github.com/flimzy/kivik/test/kt/kt.go:286 +0x62
github.com/flimzy/kivik/test/db.testSetSecurity()
/home/jonhall/go/src/github.com/flimzy/kivik/test/db/security.go:110 +0x1c2
github.com/flimzy/kivik/test/db.testSetSecurityTests.func1()
/home/jonhall/go/src/github.com/flimzy/kivik/test/db/security.go:94 +0x31e
github.com/flimzy/kivik/test/kt.(*Context).Run.func1()
/home/jonhall/go/src/github.com/flimzy/kivik/test/kt/kt.go:162 +0x23e
testing.tRunner()
/usr/local/go/src/testing/testing.go:657 +0x107
Goroutine 52 (running) created at:
github.com/flimzy/kivik/driver/couchdb/chttp.EncodeBody()
/home/jonhall/go/src/github.com/flimzy/kivik/driver/couchdb/chttp/chttp.go:217 +0x186
github.com/flimzy/kivik/driver/couchdb.(*db).SetSecurity()
/home/jonhall/go/src/github.com/flimzy/kivik/driver/couchdb/db.go:238 +0x126
github.com/flimzy/kivik.(*DB).SetSecurity()
/home/jonhall/go/src/github.com/flimzy/kivik/db.go:163 +0x1ca
github.com/flimzy/kivik/test/db.testSetSecurity.func1()
/home/jonhall/go/src/github.com/flimzy/kivik/test/db/security.go:109 +0x9d
github.com/flimzy/kivik/test/kt.Retry()
/home/jonhall/go/src/github.com/flimzy/kivik/test/kt/kt.go:286 +0x62
github.com/flimzy/kivik/test/db.testSetSecurity()
/home/jonhall/go/src/github.com/flimzy/kivik/test/db/security.go:110 +0x1c2
github.com/flimzy/kivik/test/db.testSetSecurityTests.func1()
/home/jonhall/go/src/github.com/flimzy/kivik/test/db/security.go:94 +0x31e
github.com/flimzy/kivik/test/kt.(*Context).Run.func1()
/home/jonhall/go/src/github.com/flimzy/kivik/test/kt/kt.go:162 +0x23e
testing.tRunner()
/usr/local/go/src/testing/testing.go:657 +0x107
Goroutine 38 (running) created at:
testing.(*T).Run()
/usr/local/go/src/testing/testing.go:697 +0x543
github.com/flimzy/kivik/test/kt.(*Context).Run()
/home/jonhall/go/src/github.com/flimzy/kivik/test/kt/kt.go:163 +0x105
github.com/flimzy/kivik/test/db.testSetSecurityTests()
/home/jonhall/go/src/github.com/flimzy/kivik/test/db/security.go:95 +0xbe
github.com/flimzy/kivik/test/db.setSecurity.func1.1()
/home/jonhall/go/src/github.com/flimzy/kivik/test/db/security.go:78 +0x53
github.com/flimzy/kivik/test/kt.(*Context).Run.func1()
/home/jonhall/go/src/github.com/flimzy/kivik/test/kt/kt.go:162 +0x23e
testing.tRunner()
/usr/local/go/src/testing/testing.go:657 +0x107
This depends on some way to run replications slowly (see #113).
I believe CouchDB 2.0 now returns an empty timestamp for the /_ensure_full_commit endpoint. This needs to be investigated, to determine if returning a timestamp is either supported, or meaningful.
At the moment, the only explicit Context support in the PouchDB bindings is for timeout contexts, which sets the AJAX timeout. (And, of course, just returning early, while leaving the process running in the background, if a context is cancelled--not ideal). But cancelling contexts is probably more common, and I should try to cancel an existing AJAX connection when that happens.
I'd like to change the CreateDB()
call to return a valid DB object. At present, to do the same requires:
_ = client.CreateDB(context.TODO(), "dbname")
db, _ = client.DB(context.TODO(), "dbname")
The new API would allow:
db, _ = client.CreateDB(context.TODO(), "dbname")
1.) This is not a bug or issue, its just a question for how to add a user to "org.couchdb.user:myuser" ! i found nothing about it in yours test.
2.) What happens e.g. when kivik not supporting to addUser? what to do then? maybe something like this: https://github.com/flimzy/kivik/blob/8bd551c52ecc2918bb2e9eb48969a578e791d1c7/serve/couchserver/session_test.go#L23 but in this case httptest.NewRequest("PUT", "/_users", nil)....
?!?
There is an example for getUser https://github.com/flimzy/kivik/blob/987ab2b4511ca6e42961c48a29e899978d21bce9/authdb/usersdb/usersdb.go#L36 but not for addUser
3.) Why do I ask so many questions? Because im very new with golang (since 9 month) and just 1 month with couchdb.
I hope you'll understand :( please show me a way :)
DBUpdate
(by accident) exposes the fields directly (in addition to accessor methods):
type DBUpdate struct {
DBName string
Seq string
Type string
}
All others use only accessor methods.
A decision needs to be made as to which is best, then be consistent.
Kivik has the following iterators:
Find
, AllDocs
, and Query
)It would be nice to investigate the possibility of consolidating these somehow, into a single, hopefully simpler interface.
I need to devise a way to do a replication slowly, so that I can test replication status reports and GetReplications()
midway through.
And for PouchDB, I need a way to determine the progress at all.
Seems less redundant and cleaner.
PouchDB and supports the _bulk_get endpoint documented here. It would be nice to add support to the kivik server, and emulate it in proxy mode, especially for the benefit of PouchDB clients.
I think I will guarantee not to introduce breaking changes to the kivik
or kivik/driver
packages. Other packages probably not. But this needs to be fully considered, then documented.
Memory driver to do list:
ErrNotFound
, ErrNotImplemented
, ErrUnauthorized
and SessionCookieName
should not be exported. (perhaps they shouldn't even exist)
PouchDB 6.3 now supports _explain
; this should be exposed to Kivik.
As this is a rather large project, and utility can be found (at least by me) prior to a complete feature, I decided to create this issue to track the ongoing progress toward a complete memory driver.
In rough order of priority, I wish to:
I recently messed around with _find and indexes. I accidentally produced a query with an miss matching index.
The error returned by ScanDoc() is Iterator is closed
Err() gives me Unexpected key: warning
I popped the query into my GUI REST client and the warning looks like this:
{
"warning": "no matching index found, create an index to optimize query time",
"docs": [
{
...
For now this is the only warning in mango queries:
http://docs.couchdb.org/en/2.0.0/api/database/find.html#creating-selector-expressions
I guess it would be good to expect possible warnings and let them surface anyhow. Maybe store them in a warning type - so you could retrieve it like Err().
Kind Regards
Fabian
See #172 for an example, and partial fix.
I'd like to come up with a solution to completely eliminate such test code in production builds.
couldn't send a comment on your blog (Wordpress error)
A simple example in the doc for a basic auth example for couchdb would be great. Struggling a bit to get it working (golang newbie).
Should a simple connection string work like (basic auth):
client, err := kivik.New(context.TODO(), "couch", "http://user:password@localhost:5984/")
It's barely a convenience. I can easily implement the same, or use chi's version, and remove a dependency.
Initially, replication will only be possible as a client-level function. This means the CouchDB backend will handle CouchDB replications, and the PouchDB backend will handle PouchDB replications.
In the future (some time after 1.0), I intend to allow cross-backend replication support.
I'd like to export a single Error
interface (or possibly struct) that looks something like:
type Error interface {
Error() string
StatusCode() int
}
This will probably combine functionality from the kivik/errors
package, and help obsolete the exported error variables (see #90).
PouchDB's memdown
support in tests isn't working as intended.
Get()
calls could be simplified if errors were deferred until the call to ScanDoc()
. In particular, this would allow shortening this:
row, err := db.Get(...)
if err != nil {
panic(err)
}
if err = row.ScanDoc(&target); err != nil {
panic(err)
}
to:
if err := db.Get(...).ScanDoc(&target); err != nil {
panic(err)
}
sql
's QueryRow
takes the same approach.
/db/_bulk_docs
accepts optional new_edits
parameter in the request JSON object which if false
, prevents the database from assigning new revision IDs (it's true
by default).
There doesn't seem to be a way to set new_edits
to false
, judging by how the request is being constructed.
How can i put a document without predefined _id?
What i want is that the _id should be generated by couch!
curl -H 'Content-Type: application/json' \
-X POST http://127.0.0.1:5984/demo \
-d '{"company": "Example, Inc."}'
How can i resolve this with kivik?
When i use db.Put and pass empty string as second parameter (DocId) it occurs an error panic: Unauthorized: You are not a server admin.
.
Must i be authorized when i put a document without _id? (with _id no error occurs)
Chi includes a functionally superior middleware. Lets use that instead.
CouchDB 2.1.0 was recently released. Kivik should support 2.1.0 fully. See What's New in 2.1.0
This is waiting on an official Docker image (see apache/couchdb-docker#23).
The following CouchDB 2.1.0 changes may require changes in Kivik:
Please add comments to this issue regarding any other relevant changes that should be made or considered for supporting CouchDB 2.1.0.
Should kivik use the kivik.Security
type for setting and reading security, or should it use interface{}
(which may be an instance of kivik.Security
) for greater flexibility?
How to manage & use views?
Found via Travis: https://travis-ci.org/flimzy/kivik/jobs/225934195
WARNING: DATA RACE
Read at 0x00c4203c11b0 by goroutine 57:
strings.(*Reader).Read()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/strings/reader.go:38 +0x62
io/ioutil.(*nopCloser).Read()
<autogenerated>:4 +0x86
github.com/flimzy/kivik.(*Attachment).Read()
<autogenerated>:49 +0x86
io.ReadAtLeast()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/io/io.go:307 +0xb1
io.ReadFull()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/io/io.go:325 +0x72
net/http.newTransferWriter()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/net/http/transfer.go:73 +0x980
net/http.(*Request).write()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/net/http/request.go:517 +0x648
net/http.(*persistConn).writeLoop()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/net/http/transport.go:1649 +0x299
Previous write at 0x00c4203c11b0 by goroutine 53:
strings.(*Reader).Read()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/strings/reader.go:43 +0x1a6
io/ioutil.(*nopCloser).Read()
<autogenerated>:4 +0x86
github.com/flimzy/kivik.(*Attachment).Read()
<autogenerated>:49 +0x86
io.ReadAtLeast()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/io/io.go:307 +0xb1
io.ReadFull()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/io/io.go:325 +0x72
net/http.newTransferWriter()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/net/http/transfer.go:73 +0x980
net/http.(*Request).write()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/net/http/request.go:517 +0x648
net/http.(*persistConn).writeLoop()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/net/http/transport.go:1649 +0x299
Goroutine 57 (running) created at:
net/http.(*Transport).dialConn()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/net/http/transport.go:1063 +0xa49
net/http.(*Transport).getConn.func4()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/net/http/transport.go:885 +0xa2
Goroutine 53 (finished) created at:
net/http.(*Transport).dialConn()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/net/http/transport.go:1063 +0xa49
net/http.(*Transport).getConn.func4()
/home/travis/.gimme/versions/go1.7.5.linux.amd64/src/net/http/transport.go:885 +0xa2
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.