meilisearch / meilisearch-go Goto Github PK
View Code? Open in Web Editor NEWGolang wrapper for the Meilisearch API
Home Page: https://www.meilisearch.com
License: MIT License
Golang wrapper for the Meilisearch API
Home Page: https://www.meilisearch.com
License: MIT License
As for now, go.mod
defines go version to 1.12
. But the CI can test if this builds and runs on different versions in a very simple way, by using an Strategy Matrix in GitHub Actions:
jobs:
build:
runs-on: ubuntu-16.04
strategy:
matrix:
go: [ '1.15', '1.12' ]
name: Go ${{ matrix.go }} tests
steps:
- uses: actions/checkout@v2
- name: Setup go
uses: actions/setup-go@v1
with:
go-version: ${{ matrix.go }}
- run: go test -v ./...
This should be tested in https://github.com/meilisearch/meilisearch-go/blob/master/.github/workflows/go.yml
$ sh run_tests.sh
testmeili
758e874a9ab7fe96ae6c034c7ef6e2032693c4c048a7b6756aa886e84a07271e
=== RUN TestClientDocuments_Get
--- PASS: TestClientDocuments_Get (0.08s)
=== RUN TestClientDocuments_Delete
--- PASS: TestClientDocuments_Delete (0.15s)
=== RUN TestClientDocuments_Deletes
--- PASS: TestClientDocuments_Deletes (0.15s)
=== RUN TestClientDocuments_List
TestClientDocuments_List: client_documents_test.go:171: expected to return the document[1]
--- FAIL: TestClientDocuments_List (0.08s)
=== RUN TestClientDocuments_AddOrReplace
TestClientDocuments_AddOrReplace: client_documents_test.go:215: number of doc should be at least 1
--- FAIL: TestClientDocuments_AddOrReplace (0.08s)
=== RUN TestClientDocuments_AddOrUpdate
TestClientDocuments_AddOrUpdate: client_documents_test.go:259: number of doc should be at least 1
--- FAIL: TestClientDocuments_AddOrUpdate (0.09s)
=== RUN TestClientDocuments_DeleteAllDocuments
--- PASS: TestClientDocuments_DeleteAllDocuments (0.09s)
=== RUN TestClientHealth_Get
--- PASS: TestClientHealth_Get (0.00s)
=== RUN TestClientHealth_Set
--- PASS: TestClientHealth_Set (0.01s)
=== RUN TestClientIndexes_Create
--- PASS: TestClientIndexes_Create (0.02s)
=== RUN TestClientIndexes_Get
--- PASS: TestClientIndexes_Get (0.02s)
=== RUN TestClientIndexes_Delete
--- PASS: TestClientIndexes_Delete (0.03s)
=== RUN TestClientIndexes_List
--- PASS: TestClientIndexes_List (0.07s)
=== RUN TestClientIndexes_UpdateName
--- PASS: TestClientIndexes_UpdateName (0.02s)
=== RUN TestClientIndexes_UpdatePrimaryKey
--- PASS: TestClientIndexes_UpdatePrimaryKey (0.02s)
=== RUN TestClientKeys_Get
--- PASS: TestClientKeys_Get (0.00s)
=== RUN TestClientSearch_Search
--- PASS: TestClientSearch_Search (0.33s)
=== RUN TestClientSettings_GetAll
--- PASS: TestClientSettings_GetAll (0.02s)
=== RUN TestClientSettings_UpdateAll
--- PASS: TestClientSettings_UpdateAll (0.08s)
=== RUN TestClientSettings_ResetAll
--- PASS: TestClientSettings_ResetAll (0.08s)
=== RUN TestClientSettings_GetRankingRules
--- PASS: TestClientSettings_GetRankingRules (0.02s)
=== RUN TestClientSettings_UpdateRankingRules
--- PASS: TestClientSettings_UpdateRankingRules (0.09s)
=== RUN TestClientSettings_ResetRankingRules
--- PASS: TestClientSettings_ResetRankingRules (0.08s)
=== RUN TestClientSettings_GetDistinctAttribute
--- PASS: TestClientSettings_GetDistinctAttribute (0.02s)
=== RUN TestClientSettings_UpdateDistinctAttribute
--- PASS: TestClientSettings_UpdateDistinctAttribute (0.08s)
=== RUN TestClientSettings_ResetDistinctAttribute
--- PASS: TestClientSettings_ResetDistinctAttribute (0.08s)
=== RUN TestClientSettings_GetSearchableAttributes
--- PASS: TestClientSettings_GetSearchableAttributes (0.02s)
=== RUN TestClientSettings_UpdateSearchableAttributes
--- PASS: TestClientSettings_UpdateSearchableAttributes (0.08s)
=== RUN TestClientSettings_ResetSearchableAttributes
--- PASS: TestClientSettings_ResetSearchableAttributes (0.08s)
=== RUN TestClientSettings_GetDisplayedAttributes
--- PASS: TestClientSettings_GetDisplayedAttributes (0.02s)
=== RUN TestClientSettings_UpdateDisplayedAttributes
--- PASS: TestClientSettings_UpdateDisplayedAttributes (0.09s)
=== RUN TestClientSettings_ResetDisplayedAttributes
--- PASS: TestClientSettings_ResetDisplayedAttributes (0.09s)
=== RUN TestClientSettings_GetStopWords
--- PASS: TestClientSettings_GetStopWords (0.02s)
=== RUN TestClientSettings_UpdateStopWords
--- PASS: TestClientSettings_UpdateStopWords (0.08s)
=== RUN TestClientSettings_ResetStopWords
--- PASS: TestClientSettings_ResetStopWords (0.08s)
=== RUN TestClientSettings_GetSynonyms
--- PASS: TestClientSettings_GetSynonyms (0.02s)
=== RUN TestClientSettings_UpdateSynonyms
--- PASS: TestClientSettings_UpdateSynonyms (0.08s)
=== RUN TestClientSettings_ResetSynonyms
--- PASS: TestClientSettings_ResetSynonyms (0.08s)
=== RUN TestClientSettings_GetAcceptNewFields
--- PASS: TestClientSettings_GetAcceptNewFields (0.02s)
=== RUN TestClientSettings_UpdateAcceptNewFields
--- PASS: TestClientSettings_UpdateAcceptNewFields (0.08s)
=== RUN TestClientSettings_GetAttributesForFaceting
--- PASS: TestClientSettings_GetAttributesForFaceting (0.02s)
=== RUN TestClientSettings_UpdateAttributesForFaceting
--- PASS: TestClientSettings_UpdateAttributesForFaceting (0.08s)
=== RUN TestClientSettings_ResetAttributesForFaceting
--- PASS: TestClientSettings_ResetAttributesForFaceting (0.14s)
=== RUN TestClientStats_Get
--- PASS: TestClientStats_Get (0.07s)
=== RUN TestClientUpdates_List
--- PASS: TestClientUpdates_List (0.02s)
=== RUN TestClientVersion_Get
--- PASS: TestClientVersion_Get (0.00s)
=== RUN Test_deleteAllIndexes
--- PASS: Test_deleteAllIndexes (0.24s)
FAIL
ok github.com/meilisearch/meilisearch-go (cached)
0
(= SUCCEED
) when the tests fail? This should be fixed as well because it cannot currently be used like that in a CIAs I think http/client is a rather slow library. I would like to use something faster as the httpclient, for example the fasthttpclient. It also probably makes sense to add methods for sending requests with a raw body ([] byte). I can take it on myself
Hello, I can't import this module. It looks like the module name was change from meilisearch/meilisearch to senyast4745/meilisearch
go: github.com/meilisearch/meilisearch-go: github.com/meilisearch/[email protected]: parsing go.mod: module declares its path as: github.com/senyast4745/meilisearch-go but was required as: github.com/meilisearch/meilisearch-go
Thanks
The search returns no result if I don't pass the Limit
parameter. It does not fail. It just returns empty Hits
.
_, err := client.Search("indexUID").Search(meilisearch.SearchRequest{
Query: "harry pottre",
})
=> no result
_, err := client.Search("indexUID").Search(meilisearch.SearchRequest{
Query: "harry pottre",
Limit: 10,
})
=> result
We should be able to pass only the Query
parameter
I understand that 2 parameters at least are needed because
searchRes, err := client.Search("indexUID").Search(meilisearch.SearchRequest{
Query: "harry pottre",
AttributesToHighlight: "*",
})
returns results as well.
My bad, it does not even run at all, I open an issue for that: #7
v0.9.0
of MeiliSearch.
Related to this general issue
Since the MeiliSearch HTTP server changed from tide to atix they way it handles parameters that are empty has changed. As Go initializes every variable to a Default Zero value, when a parameter is not set to a value by the user, the Go SDK will send the defult zero value to the server. This leads to problems like #37
For the client List
method, the 3 parameters (limit
, offset
and attributesToRetrieve
) are then initialized to their default zero values. If the user does not specify, for example some attributesToRetrieve
, Go SDK is making a request with an empty list of attibutesToRetrieve
. This parameters need to be added to the request only if the user specifies a value for them.
Test on master are failing because of this, as we can see for example here
GitHub actions arre nonetheless accepting those tests and validatign the check, which is a different issue and should be handled in #51
When I try the example in the readme with the AttributesToHighlight
parameter:
resp, err := client.Search(indexUID).Search(meilisearch.SearchRequest{
Query: "harry pottre",
AttributesToHighlight: "*",
})
it does not run, with this error:
# command-line-arguments
./test.go:47:3: cannot use "*" (type string) as type []string in field value
Let's be allies and make this change that means a lot.
Here is a blog post that explain a little more why it's important, and how to easily do it. It will be a bit more complicated with automation, but we still should do it!
I see a weird JSON error when adding a document to Meilisearch server. This may not be a bug, and is probably caused by wrong usage, but I couldn't find any useful troubleshooting documents for this.
2020/09/13 05:58:39 unaccepted status code found: 400 expected: [202], message from api: 'Invalid JSON: invalid type: map, expected a sequence at line 1 column 0', request: {"content":"testing 1234","createdAt":"2020-09-13T05:58:39.362533+09:00","id":"btejcvrpc98mn373um10","image":"https://google.com","link":"","tags":["beans","bananas","tag1"],"title":"OOOH lala","type":"POST","updatedAt":"2020-09-13T05:58:39.362888+09:00","urlSlug":"test-slug"} (path "PUT /indexes/articles/documents" with method "Documents.AddOrUpdate")
# maybe the problem has been caused by wrong conversion method, tried using third-party conversion module and it shows the similar sort of error
# github.com/mitchellh/mapstructure
2020/09/13 06:28:32 unaccepted status code found: 400 expected: [202], message from api: 'Invalid JSON: invalid type: map, expected a sequence at line 1 column 0', request: {"content":"testing 1234","createdAt":{},"deletedAt":{"Time":{},"Valid":false},"id":"btejr03pc98g0r8l6l80","image":"https://google.com","tags":["beans","bananas","tag1"],"title":"OOOH lala","type":"POST","updatedAt":{},"urlSlug":"test-slug"} (path "PUT /indexes/articles/documents" with method "Documents.AddOrUpdate")
no matter how I construct map[string]interface{}
I keep seeing Invalid JSON
error message.
import (
"gorm.io/gorm"
"log"
"github.com/meilisearch/meilisearch-go"
"github.com/novalagung/gubrak/v2"
)
type Article struct {
ID string `gorm:"primaryKey"`
CreatedAt time.Time
UpdatedAt time.Time
DeletedAt gorm.DeletedAt `gorm:"index"`
Type string
Title string
Content string
Image string
Link string
Tags []Tag `gorm:"many2many:article_tags;"`
URLSlug string
}
type Tag struct {
Value string;
ID string;
}
func TagsToStringArray(tags []Tag) []string {
return gubrak.From(tags).Map(func(t Tag) string {
return t.Value
}).Result().([]string)
}
func ArticleToSearch(article *Article) map[string]interface{} {
tagValues := TagsToStringArray(article.Tags)
return map[string]interface{}{
"id": article.ID,
"content": article.Content,
"createdAt": article.CreatedAt,
"updatedAt": article.UpdatedAt,
"title": article.Title,
"type": article.Type,
"image": article.Image,
"link": article.Link,
"urlSlug": article.URLSlug,
"tags": tagValues, // at first I assumed this might be the problem, but it persists even without this item
}
}
func AddToMeili() {
article := Article{}
getArticleFromDB(&article)
meiliDoc := scheme.ArticleToSearch(article)
log.Printf("meili doc %v", meiliDoc)
_, err := meiliClient.Documents("articles").AddOrUpdate(meiliDoc)
log.Printf("%v", err) // <-- the error is detected from here
}
Golint is a linter for Go source code - https://goreportcard.com/report/github.com/meilisearch/meilisearch-go#golint
meilisearch-go/apis.go
Line 10: warning: comment on exported type ApiIndexes should be of the form "ApiIndexes ..." (with optional leading article) (golint)
Line 78: warning: comment on exported type ApiSearch should be of the form "ApiSearch ..." (with optional leading article) (golint)
Line 89: warning: exported type ApiSynonyms should have comment or be unexported (golint)
Line 101: warning: comment on exported type ApiStopWords should be of the form "ApiStopWords ..." (with optional leading article) (golint)
Line 124: warning: comment on exported type ApiUpdates should be of the form "ApiUpdates ..." (with optional leading article) (golint)
Line 142: warning: comment on exported type ApiKeys should be of the form "ApiKeys ..." (with optional leading article) (golint)
Line 163: warning: exported type ApiSettings should have comment or be unexported (golint)
Line 201: warning: exported type ApiSystemInformation should have comment or be unexported (golint)
meilisearch-go/error.go
Line 11: warning: exported type ErrCode should have comment or be unexported (golint)
Line 14: warning: exported const ErrCodeUnknown should have comment (or a comment on this block) or be unexported (golint)
Line 118: warning: exported method Error.WithMessage should have comment or be unexported (golint)
Line 128: warning: exported method Error.WithErrCode should have comment or be unexported (golint)
Line 138: warning: exported method Error.ErrorBody should have comment or be unexported (golint)
meilisearch-go/types.go
Line 5: warning: exported type Unknown should have comment or be unexported (golint)
Line 11: warning: exported type SchemaAttributes should have comment or be unexported (golint)
Line 14: warning: exported const SchemaAttributesDisplayed should have comment (or a comment on this block) or be unexported (golint)
Line 20: warning: exported type Attributes should have comment or be unexported (golint)
Line 22: warning: exported type RawAttribute should have comment or be unexported (golint)
Line 27: warning: exported type RawSchema should have comment or be unexported (golint)
Line 32: warning: exported type Schema should have comment or be unexported (golint)
Line 34: warning: exported type Index should have comment or be unexported (golint)
Line 41: warning: exported type Settings should have comment or be unexported (golint)
Line 47: warning: exported type Synonym should have comment or be unexported (golint)
Line 52: warning: exported type ACL should have comment or be unexported (golint)
Line 55: warning: exported const AclIndexesRead should have comment (or a comment on this block) or be unexported (golint)
Line 65: warning: exported type APIKey should have comment or be unexported (golint)
Line 76: warning: exported type Version should have comment or be unexported (golint)
Line 82: warning: exported type Stats should have comment or be unexported (golint)
Line 88: warning: exported type SystemInformation should have comment or be unexported (golint)
Line 104: warning: exported type SystemInformationPretty should have comment or be unexported (golint)
Line 120: warning: exported type UpdateStatus should have comment or be unexported (golint)
Line 123: warning: exported const UpdateStatusUnknown should have comment (or a comment on this block) or be unexported (golint)
Line 129: warning: exported type Update should have comment or be unexported (golint)
Line 138: warning: exported type AsyncUpdateId should have comment or be unexported (golint)
Line 146: warning: exported type CreateIndexRequest should have comment or be unexported (golint)
Line 152: warning: exported type CreateIndexResponse should have comment or be unexported (golint)
Line 161: warning: exported type SearchRequest should have comment or be unexported (golint)
Line 175: warning: exported type SearchResponse should have comment or be unexported (golint)
Line 183: warning: exported type ListDocumentsRequest should have comment or be unexported (golint)
Line 189: warning: exported type ListSynonymsResponse should have comment or be unexported (golint)
Line 191: warning: exported type BatchCreateSynonymsRequest should have comment or be unexported (golint)
Line 193: warning: exported type CreateApiKeyRequest should have comment or be unexported (golint)
Line 200: warning: exported type UpdateApiKeyRequest should have comment or be unexported (golint)
Currently, the tests run with a MeiliSearch instance that does not provide any master key.
Can we launch a MeiliSearch instance with the option --master-key=masterKey
to test this package? And for course, adapt the tests.
Hello,
I'm running into an issue with adding FacetsDistribution
to my SearchRequest
object. It appears a change made recently is generating an invalid value.
I'm running go 1.13.6
on linux, pulling from master. Though it seems the following change has been implemented for all of 0.11.x
.
meilisearch-go/client_search.go
Line 56 in e80b241
The problem seems to be %q
is escaping the prequoted list, resulting in an empty FacetsDistribution
in the response.
Example usage:
facets := []string{"city", "state"}
results, _ :=client.Search(idx).Search(meilisearch.SearchRequest{
Query: "bob",
FacetsDistribution: facets,
})
// results.FacetsDistribution is empty
The problem can be replicated with
facets := []string{"city", "state"}
fmt.Printf("[%q]", strings.Join(facets, "\",\""))
// Result: ["city\",\"state"]
https://play.golang.org/p/Gh2USjlJvsG
Thanks!
Not printing the the requested json makes it harder to debug (I'm using go client btw)
already sent this:
"[{\"id\":20,\"author\":\"Prof. Laury Herzog\",\"title\":\"Aut perferendis accusantium sit voluptatem consequatur.\",\"body\":\"Consequatur accusantium voluptatem sit perferendis aut. Sit perferendis accusantium consequatur aut voluptatem. Perferendis accusantium aut sit voluptatem consequatur. Accusantium sit consequatur aut perferendis voluptatem. Sit accusantium aut perferendis consequatur voluptatem. Perferendis consequatur accusantium sit aut voluptatem. Perferendis sit voluptatem consequatur accusantium aut. Perferendis voluptatem sit aut consequatur accusantium. Voluptatem perferendis sit aut accusantium consequatur.\",\"created\":1606484683456729702}]"
but still got
{"status":"failed","updateId":19,"type":{"name":"DocumentsAddition","number":1},"error":"document id is missing","errorType":"invalid_request_error","errorCode":"missing_document_id","errorLink":"https://docs.meilisearch.com/errors#missing_document_id","duration":0.000806693,"enqueuedAt":"2020-11-27T13:44:43.459032791Z","processedAt":"2020-11-27T13:44:43.462177939Z"}
apparently I was wrongly set the primary key in the past (because I was calling .Documents(space).AddOrReplaceWithPrimaryKey(row, row.Id)
) -- because I thought the 3rd parameter is ID of that specific document instead of the primary key column name.
curl -X GET 'http://localhost:7700/indexes/articles'
{"name":"articles","uid":"articles","createdAt":"2020-11-27T11:47:01.834279018Z","updatedAt":"2020-11-27T13:44:38.401930845Z","primaryKey":"5"}
so I have to drop it first..
According to meilisearch/integration-guides#37, and after this decision, search method should always use the POST
search route. The GET
search route will be considered deprecated in this SDK.
It seems like the API for per-index stats information is currently not available with this client library, is that right? client.Stats().Get()
always returns an empty stats data.
If I have well understood, we are stuck in an infinite loop, if for any reason, the status of the update is stuck to enqueued
.
Lines 241 to 256 in f9427a2
We should define a default value of timeout.
This value could be change when passing an optional timeOut
parameter in this function.
Plus, the name of the updateID
parameter is not really accurate: the function expects an update "object" and not a updateID.
Currently, if I have already a MeiliSearch isntance running locally, and if I run the tests ./run_tests.sh
, there are several indexes still running in my MeiliSearch instance.
We should provide tests that clean all the indexes created for tests purpose.
Issue fully detailed here.
Dear all,
I frequently got this error while adding document, any idea how to get detailed error? Is it caused by MeiliSearch server? Or the Go SDK?
We don't get the Total Size of the Search Response, but we do get it as nbHits
in the web-interface.
To fix that, could we add :
TotalHits int64 json:"nbHits"
to the struct SearchResponse
in types.go?
Which software license is the project maintained under?
Hello,
The 'Add documents' code example in the README.md file contains a typo, which results in a compile error. The code:
var client = meilisearch.NewClient(meilisearch.Config{
Host: "http://127.0.0.1:7700",
APIKey: "masterKey"
})
Is missing a comma at the end of "masterKey", example:
var client = meilisearch.NewClient(meilisearch.Config{
Host: "http://127.0.0.1:7700",
APIKey: "masterKey", // <- comma needed here :)
})
Without the comma you get a compiler error:
syntax error: unexpected newline, expecting comma or }
Great project!
Thanks.
Currently, we do not have a way to access the _formatted
key in a search response if we ask for an AttributesToHighlight
.
Lines 157 to 163 in f9427a2
I am trying to migrate 40M Postgres records (user comments) to Melisearch. Each time, I am querying 10K records from Postgres and writing to Meilisearch. The migration script has been completed. But it has been more than 3 days, only 4.2 Million records only migrated. The count is increasing very slowly.
I also see the following error messages in the error log:
[2021-02-18T12:48:44Z INFO ureq::unit] sending request POST https://api.amplitude.com/httpapi
[2021-02-18T13:14:58Z ERROR meilisearch_core::database] commit nested transaction failed: Input/output error (os error 5)
[2021-02-18T13:43:50Z ERROR meilisearch_core::database] commit nested transaction failed: Input/output error (os error 5)
[2021-02-18T13:48:45Z INFO ureq::unit] sending request POST https://api.amplitude.com/httpapi
[2021-02-18T14:13:03Z ERROR meilisearch_core::database] commit nested transaction failed: Input/output error (os error 5)
[2021-02-18T14:42:05Z ERROR meilisearch_core::database] commit nested transaction failed: Input/output error (os error 5)
[2021-02-18T14:48:45Z INFO ureq::unit] sending request POST https://api.amplitude.com/httpapi
[2021-02-18T15:12:06Z ERROR meilisearch_core::database] commit nested transaction failed: Input/output error (os error 5)
[2021-02-18T15:41:40Z ERROR meilisearch_core::database] commit nested transaction failed: Input/output error (os error 5)
[2021-02-18T15:48:46Z INFO ureq::unit] sending request POST https://api.amplitude.com/httpapi
Also, I tried to get updates using the following curl
curl 'http://localhost:7700/indexes/comments/updates' | jq
It is running forever, not getting any result from the server.
This code is unsafe for the developer who is using the library.
Query parameters should be built with the URL type. (https://stackoverflow.com/a/30657518/5894301)
Why is it unsafe ?
Imagine the developer who is using the APIDocuments.List set AttributesToRetrieve to an input from an user of his website. A malicious user can give an input like that: ?limit=300000.
What could be cool ?
With the current internalRequest type you can't set query parameters. I think we could add a QueryParams field, build the URL and add thoses query parameters in the Client.sendRequest method. That will avoid to have a special case in this method too (which is not pretty nice).
Every language has it's own specifics, its own rules and good practices. Go is clearly not an exception. But Maintaining several SDKs requires a lot of conciliation between those specific characteristics from one side, and the bigger project (MeiliSearch itself) integrations flow, logic and structure.
This is why I consider it really important to try to find as much coherence as possible between the different SDKs ans integrations. As for now, I feel that the Go SDK is very often considered as the exception, a 'separate case', in every discussion. But this makes maintainability not really cost efficient. Also, as it evolves, it will just keep growing further and further in the future from it's SDK brothers and sisters, and I don't want it to end up alone :)
I spent some time looking to how the other SDKs are built, and why is it that makes it so specific, because I would like to try to make some effort to bring this SDK closer to the other SDKs documentation, logic and structure, to make it easier to evolve, to be used and documented. The work that has been done on it until now is very good (special thanks to @alexisvisco) and I think we can totally keep building over it, and this is why I would like to propose a little refactor for the whole project, keeping as much of this progress as we can, but without fear of exploring new paths.
In my opinion, there is one big thing that can totally give it a new workflow and look, and is not tied to any specificity of Go, but would make it more coherent with the rest of the MeiliSearch project and easier to maintain and live with the other SDKs: the project structure.
Most MeiliSearch SDKs are using an POO paradigm until now. Go doesn't really exactly implement POO principles, but structures, interfaces and methods tied to them offer us more or less the same functionality, inheritance aside. This is something that is pretty much exploited everywhere in this SDK already. Following that tought, I would like to suggest, first of all, a new Package structure:
A Client package, which is responsible for:
An Index package, responsible for:
A Custom Error package widely used in the project following previous error handling discussions
Tests, tests everywhere!
I am picturing a repository having one folder at it's root (and all the config files that you normally find in a go repo) called meilisearch
containing the whole sources of the meilisearch
package. Inside this folder, something that looks like this (more or less)
This would be a very fast scketch of how a Possible Getting started would look like
package main
import (
"fmt"
"github.com/eskombro/meilisearch-go/meilisearch/client"
"github.com/eskombro/meilisearch-go/meilisearch/index"
)
func main() {
meiliHost := "http://localhost:7700"
masterKey := "masterKey"
indexUid := "indexUID"
// Create an index WITH Master Key
msClient = client.Create(client.Client{Host: meiliHost, MasterKey: masterKey})
msIndex = index.Create(msClient, index.IndexConfig{Uid: indexUid})
documents := []map[string]interface{}{
{"book_id": 123, "title": "Pride and Prejudice"},
{"book_id": 456, "title": "Le Petit Prince"},
{"book_id": 1, "title": "Alice In Wonderland"},
{"book_id": 1344, "title": "The Hobbit"},
{"book_id": 4, "title": "Harry Potter and the Half-Blood Prince"},
{"book_id": 42, "title": "The Hitchhiker's Guide to the Galaxy"},
}
update := msIndex.addDocuments(documents)
msIndex.Delete()
}
Methods can then be clearly separated, and use the same logic we use in other SDKs. This structure and example of a getting started is for a general idea. It is not only keeping the same logic and structure than the other SDKs, but also closer to what they do at Algolia (which I bring into the discussion because it seems to offer a very good experience for the developer):
I would really like to have your opinion in this subject, my only goal with this is to start a general discussion of where the SDK should be heading to, about what it's goal should be, and start having a roadmap for it, that will then transform into issues, and then peoples work and contributions for a great MeiliSearch Go Experience! ๐
I would really like to have your opinion on this, @curquiza, @tpayet , @bidoubiwa, @qdequele, @alexisvisco, @derekahn => If you guys have some time to read and give some insigth, I would be very grateful!
Create a basic error handler, with these 3 custom errors:
MeiliSearchApiError
MeiliSearchCommunicationError
MeiliSearchTimeoutError
MeiliSearch API has an error handler, and returns this kind of JSON when an error is found:
{
"message": "Index movies not found",
"errorCode": "index_not_found",
"errorType": "invalid_request_error",
"errorLink": "https://docs.meilisearch.com/errors#index_not_found"
}
The complete list of errorCode is available in the documentation
It could be great to let the user access message
, errorCode
, errorType
and errorLink
to a better understanding an error.
As stated in meilisearch/meilisearch#810 and meilisearch/integration-guides#36 the sys-info routes will be considered as deprecated in MeiliSearch v0.13.0. These routes should be removed from this SDK accordingly.
As MeiliSearch grows so does its SDK's. More and more SDK's rises from the ground and they all deserve to be as well documented as the core engine itself.
The most common way for a user to understand MeiliSearch is to go to its official documentation. As of yesterday all examples in the documentation were made with cURL
. Unfortunately, most of our users do not communicate with MeiliSearch directly with cURL
. Which forces them to search for the specific references somewhere else (in the readme's, in the sdk's code itself,..). This makes for unnecessary friction.
We want our documentation to include all SDK's
As a first step we want all examples to be made using the most possible SDK's. As did Stripe and Algolia.
examples with curl, javascript and soon enough this SDK too!
To achieve this it is expected from this SDK to create a sample file containing all the code samples needed by the documentation.
These are the steps to follow:
The sample file is a yaml file added at the root of each MeiliSearch SDK.
Sample files are created based on the sample-template.yaml
file.
sample-template
file:
get_one_index_1: |-
list_all_indexes_1: |-
create_an_index_1: |-
...
This template is accessible publicly here or in the public directory : .vuepress/public/sample-template.yaml
of the documentation.
The name of the file should be .code-samples.meilisearch.yaml
After creating the sample file with the content of the sample template you should have a file containing all the sampleId
's.
get_one_index_1: |-
list_all_indexes_1: |-
create_an_index_1: |-
...
By the name of the different sampleId
you should know where that code sample will be added to the documentation.
For example, create_an_index_1
is the first example in the API References of the index creation.
Using the already existing cURL
example in the documentation you should see what is expected from that sample. It is very important that you look at the already existing samples in the documentation as it gives you the parameters to use in your sample to match with the rest of the documentation.
Good sample based on documentation:
create_an_index_1: |-
client.createIndex({ uid: 'movies' })
Bad sample that does not match with the response.
create_an_index_1: |-
client.createIndex({ uid: 'otherName' })
Each sample is expected to be written in the respective SDK language.
Javascript example:
get_one_index_1: |-
client.getIndex('movies').show()
list_all_indexes_1: |-
client.listIndexes()
create_an_index_1: |-
client.createIndex({ uid: 'movies' })
...
The complete cURL
sample file is available at the root of the documentation repository.
Every other SDK sample file should be available at the root of their respective repository.
There is one exception to the formatting rule.
When the sampleId
finishes with _md
it means it is expected to be written in markdown format.
JavaScript sample id with _md extension:
Once your sample file is filled with the code samples, you will need to create a pull request on the documentation repository.
Open the following file in your IDE:
.vuepress/code-samples/sdks.json
And add your sample file to the list:
[
...
{
"language": "sdk-language",
"label": "sdk-label",
"url": "url to yaml file"
}
]
The language
key expect a supported language for the code highlight.
The label
key is the name of the tab. While label
and language
could have been the same, it created some conflict (i.e: bash
and cURL
).
The url
is the raw link to access your sample file. It should look like this:
https://raw.githubusercontent.com/[PROJECT]/[REPO]/.code-samples.meilisearch.yaml
Since v0.15.0, MeiliSearch exposes a new functionality: dumps
This feature should be implemented in this SDK following this specificacions
In the development workflow part there is two points about golint:
# Install golint
$ go get -u golang.org/x/lint/golint
# Use golint
$ golint
2 issues:
go.sum
and the go.mod
files. Is it expected?golint
tool is still unavailable for me.When I run go get -u golang.org/x/lint/golint
I get:
go: found golang.org/x/lint/golint in golang.org/x/lint v0.0.0-20200302205851-738671d3881b
go: golang.org/x/tools upgrade => v0.0.0-20200325010219-a49f79bcc224
And the command returns 0 (success).
#88 introduced EasyJson to this repo. It autogenerates a file types_easyjson.go
by using the command
easyjson -all types.go
This should be documented in the CONTRIBUTING.md
More info at https://github.com/mailru/easyjson#usage
Translating tests to testify and testcontainers.
Testify will allow you to write unit tests and is also very convenient if you need to set up an environment. This is a very good JUnit counterpart for Golang. Testcontainers will allow us to raise the container with Meilisearch in runtime and test environment, which is good news in my opinion. Together with this task, it will probably be possible to close issue #16 .
Issue fully detailed here.
In the continuity of this discussion. The Meilisearch Teams has decided to try to bring documentation, logic, and structure of this SDK closer to the others to be more developer-friendly and to have a usage that seems to us more pleasant and more accessible.
package main
import (
"fmt"
"github.com/meilisearch/meilisearch-go"
"github.com/meilisearch/meilisearch-go/client"
)
func main() {
client = meilisearch.NewClient(client.ClientConfig{
Host: "http://127.0.0.1:7700",
MasterKey: "masterKey",
})
// An index is where the documents are stored.
index := client.Index("indexUID")
// If the index 'books' does not exist, MeiliSearch creates it when you first add the documents.
documents := []map[string]interface{}{
{"book_id": 123, "title": "Pride and Prejudice"},
{"book_id": 456, "title": "Le Petit Prince"},
{"book_id": 1, "title": "Alice In Wonderland"},
{"book_id": 1344, "title": "The Hobbit"},
{"book_id": 4, "title": "Harry Potter and the Half-Blood Prince"},
{"book_id": 42, "title": "The Hitchhiker's Guide to the Galaxy"},
}
_, err := index.addDocuments(documents)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
index.Delete()
}
New package structure inspired by what stripe does and following @eskombro guidance:
A Client package, which is responsible for:
An Index package, responsible for:
A Custom Error package widely used in the project following previous error handling discussions
All methods must be well tested a lot
When i update products with the following code;
_, err := conn.Client.Documents(Index+sort).AddOrUpdate(jsonParsed.Data()) // => { "updateId": 0 }
if err != nil {
fmt.Println(err)
os.Exit(1)
}
same times I get the following error? could it be load?
unable to execute request (path "PUT /indexes/productPT/documents" with method "Documents.AddOrUpdate"): Put "https://meili.nobrinde.online/indexes/productPT/documents": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
on the page of Quick start guide, in the create your index section, under the Go tab
Quick Start Guide | Create your index
in the third square box, the example given contains
var client = NewClient(meilisearch/Config{
Host: "http://127.0.0.1:7700",
})
it should be a .
and not /
var client = NewClient(meilisearch.Config{
Host: "http://127.0.0.1:7700",
})
Hi, I'm having an issue with UpdateSearchableAttributes
not saving after a document insert. To reproduce:
go mod init
and go get github.com/meilisearch/meilisearch-go
go run main.go
, you should see [Name]
printed. That is the searchable attribute.createIndex()
in main) and run again[Name Year]
printed, meaning the settings created in createIndex
were not persisted.package main
import (
"fmt"
"time"
"github.com/meilisearch/meilisearch-go"
)
type Movie struct {
Name string
Year int
ID int
}
var uid = "movies4"
var client = meilisearch.NewClient(meilisearch.Config{
Host: "http://127.0.0.1:7700",
})
func createIndex() {
// Create index
client.Indexes().Create(meilisearch.CreateIndexRequest{
UID: uid,
PrimaryKey: "ID",
})
// Update searchable attirbutes
_, err := client.Settings(uid).UpdateSearchableAttributes([]string{"Name"})
if err != nil {
fmt.Println(err)
}
time.Sleep(time.Second)
}
func insert(id int) {
// Insert document
movies := []Movie{Movie{Name: "The Shining", Year: 1980, ID: id}}
_, err := client.Documents(uid).AddOrUpdate(&movies)
if err != nil {
fmt.Println(err)
}
// Get searchable attributes
a, err := client.Settings(uid).GetSearchableAttributes()
if err != nil {
fmt.Println(err)
}
fmt.Println(*a)
}
func main() {
createIndex()
insert(1)
}
I am trying to add around 25K documents to the index and getting the following error when doing so.
unable to execute request (path "PUT /indexes/players/documents" with method "Documents.AddOrUpdate"): Put "http://localhost:7700/indexes/players/documents": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
When I use the functions, DefaultWaitForPendingUpdate or , I get the following error:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x12e23d4]
goroutine 1 [running]:
github.com/meilisearch/meilisearch-go.Client.WaitForPendingUpdate(0x15f2cc8, 0x15, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3b9aca00, ...)
/Users/deshetti/go/pkg/mod/github.com/meilisearch/[email protected]/client.go:257 +0x104
github.com/meilisearch/meilisearch-go.Client.DefaultWaitForPendingUpdate(0x15f2cc8, 0x15, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3b9aca00, ...)
/Users/deshetti/go/pkg/mod/github.com/meilisearch/[email protected]/client.go:239 +0x115
main.main()
/Users/deshetti/Projects/playground/player-database/index-players/main.go:149 +0x406
exit status 2
Here is how I am using the above in my code. I am new to both Golang and Meilisearch, excuse me if I am missing something simple.
// Creates an index in Meilisearch with the name players
createIndex("players")
// Index the players
var client = meilisearch.NewClient(meilisearch.Config{
Host: meiliURI,
})
updateIDRes, err := client.
Documents("players").
AddOrUpdate(articles)
client.WaitForPendingUpdate(context.TODO(), 60*time.Second, "players", updateIDRes)
// client.DefaultWaitForPendingUpdate("players", updateIDRes)
if err != nil {
log.Fatal(err)
}
Any help is greatly appreciated. Thanks.
An update has been made on the code-samples list of samples.
The following samples are missing from this repository .code-samples.meilisearch.yaml
settings_guide_synonyms_1
add_movies_json_1
To know what we expect from these specific samples, please visit the cURL
sample file.
Exemple for settings_guide_synonyms_1
:
$ curl \
-X POST 'http://localhost:7700/indexes/tops/settings' \
--data '{
"synonyms": {
"sweater": ["jumper"],
"jumper": ["sweater"]
}
}'
I'll create a GitHub action for next years.
As in meilisearch/meilisearch#771 search query is now optional. If no query is sent, placeholder search is triggered, letting the user get some default results provided by MeiliSearch.
Following #120 (comment)
Even if this method is used in our tests, it's not concretely tested ๐
We should add tests according to this issue.
Create issue, link a PR and automate labeling
Following meilisearch/meilisearch#824
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.