mijia / modelq Goto Github PK
View Code? Open in Web Editor NEWModelQ is a code generator for creating Golang codes/models to access RDBMS database/tables
License: MIT License
ModelQ is a code generator for creating Golang codes/models to access RDBMS database/tables
License: MIT License
Code generated from a table with no primary key does not compile:
db/schema/subnet.go:10: imported and not used: "fmt"
db/schema/subnet.go:31: cannot use 0 (type int) as type Subnet in return argument
The first error is that the fmt package is only used in a section of the template included if the table has a primary key.
The second error is that 0 is being returned as a dummy value instead of nil. Solution here is probably to not generate the Get method at all if it isn't going to work with no primary key.
Prepare statements are leaking whenever an UPDATE is made. The statements are getting closed for queries (q _Query) query(...)
but not for any updates made (q _Query) exec(...)
see modelq/gmq/package.go
. The prepares get deallocated whenever the connection disappears, but in the case that the connection persists and many updates are being done via transactions, we will eventually hit the maximum number of prepared statements (which by default is 13682) and receive error 1461. This shouldn't happen. A workaround would be to increase the limit, but the preferred way would be to close the statement when it's finished.
https://dev.mysql.com/doc/refman/5.7/en/server-status-variables.html#statvar_Prepared_stmt_count
On line 131 in gmq/query.go , error checks aren't being done after iterating over the rows' result set. As a result, this can lead to silent failures if rows.Next()
returns halfway through due to an intermittent error. The caller should be expected to distinguish between reaching the end of the set or an error occurring when preparing that next item. See https://golang.org/pkg/database/sql/#Rows.Next
when tables are dropped modelq doesn't clean up the generated golang files. are there any plans to address this?
Hi guys,
Thanks for the interesting project. Unfortunately, we have PostgreSQL version < 8.2, so we can't use position_in_unique_constraint.
modelq crashes with next error message:
2015/04/23 12:16:15 [Postgres Driver] Start to load tables schema from database, name, tables=tables
2015/04/23 12:16:16 Cannot load table schemas from database.
2015/04/23 12:16:16 pq: column keycolumnusage.position_in_unique_constraint does not exist
I found, that these are generated in https://github.com/mijia/modelq/blob/master/drivers/postgres/key_column_usage.go#L24 What do you think is the best way to adapt this to my case?
Regards,
Sergii
Do you have ideas on how you want to do aggregate functions like MAX()
? You mention them in the readme, but I don't see anything hinting what your ideas were on this. I'd be happy to take a whack at it regardless.
And ideas on how to do it with #5 in mind would be good.
Hello Mijia:
This code generator looks amazing at it's early stage, so please keep working and i hope it could be something powerful like the Scala Slick Code generator and Framework. I'm in my baby steps for GOLANG, so i'm learning a lot from your clever code. I would like to know if you can release the code with an MIT license and add it to the project, i'll keep watching :).
Rayner
To quote the README:
The generated models rely on the modelq/gmq package, I am not sure if this would be OK, or could this be changable and plugable, no idea so far.
I think different users have different preferences. What I think woul be nice, is if it was easier to support generating sqlx code (with a custom template is OK). sqlx uses a "db" tag rather than a "json" tag by default.
With that in mind, it would be great if the default struct template stated something like:
var modelStruct string = `type {{.Name}} struct {
{{range .Fields}}{{.Name}} {{.Type}} `json:"{{.FieldMeta}}"`{{if .Comment}} // {{.Comment}}{{end}}
{{end}}
}
instead of
var modelStruct string = `type {{.Name}} struct {
{{range .Fields}}{{.Name}} {{.Type}} {{.JsonMeta}}{{if .Comment}} // {{.Comment}}{{end}}
{{end}}
}
That way it would be easier/less hacky to make a "db" tag for sqlx in place of your default JSON tag;-)
Well... We could do that easily if we had access to functionality like the "toType" function, the one that takes (columns []gmq.Column, rb []sql.RawBytes)
. It just does the hard work for you.
There has to be some way to do outlandish queries onto a type, having modelq trust you that you're returning the right columns to it. Even if it's just a blaobjs.Query("query here")
that returns a _BlaQuery
so you can start outputting results. It doesn't have to support any more Filters or anything.
Firstly, modelq looks fantastic, exactly the right approach to reducing the leg work of working with an RDBMS in Go, thanks a lot. Were you inspired by this by any chance?
We're working with PostgreSQL. Any interest in supporting it? Any idea how much work would be involved in adding it if we were to try it ourselves?
It often is such that you can create struts in go that are a merge on one of more tables. These are sometimes called datasets, because they are a set of tables.
Then to fill the dataset, you can call the crud for each table manually doing the joins.
I did this before in c# and it worked OK.
I think we had a dB with almost 1,000 tables, and so this simple technique removed lots of boring coding.
It also properly separates for middler tier / client types from you database types.
The only real work is to write data transformation code that maps your database types to and from you datset types.
For many use cases of just managing simple things in the dB you don't need to use dataset types still. For example a user table. Then just the crud is fine.
The dataset types can be generated using protocol buffers or flat buffers. I prefer protocol buffers because it also generates the service calls also.
Then you have code generated a hell of alot of the boring code.
Just an idea !!
codegen.go only checks for
strings.ToUpper(f.DefaultValue) == "CURRENT_TIMESTAMP"
or
strings.ToUpper(f.DefaultValue) == "NOW()"
when deciding whether a time column has some default value. See https://github.com/mijia/modelq/blob/master/main.go#L28
However, MySql also allows default values like CURRENT_TIMESTAMP(X)
and NOW(X)
where X is a number. See http://dev.mysql.com/doc/refman/5.7/en/timestamp-initialization.html
I think code generator should check for matches against ^NOW(?:\\(\\d\\))?$"
and "^CURRENT_TIMESTAMP(?:\\(\\d\\))?$"
problematic line in question: https://github.com/mijia/modelq/blob/master/main.go#L28
In case when table has multi-column primary key (also if table hase only one unique key and it's multi-column mysql shows it as primary) only one of fields of this index is marked as primary and this breaks generated functions like updating record in database.
If I have a date field - the gmq model created translates this into a string field containing (for example)
2007-09-10T00:00:00Z
I see no obvious reason why this could not be created as a golang datetime?
Any chance of this being the standard behaviour - or maybe optional as a switch?
Our API supports camelCase as a standard, so instead of json:first_name
we support json:firstName
. So it would be nice to use this framework, but it doesn't currently support our needs.
I believe it should be a great feature if we can have struct tags generated for sql column names. Ex:
objs.Select("Id", "Age").Where(objs.FilterAge(">=", 10)).
OrderBy("Age", "+Id").Page(9, 5)
Should be:
objs.Select(User.Id, User.Age).Where(objs.FilterAge(">=", 10)).
OrderBy(User.Age, User.Id).Page(9, 5)
Or
objs.Select( _.Id, _.Age).Where(objs.FilterAge(">=", 10)).
OrderBy( _.Age, _.Id).Page(9, 5)**
This is very important when you're designing big projects and you're refactoring the database models. The compiler we'll help to identify the errors after code generation from the refactored database.
Hi, first of all, nice job. I had the exact same idea for this :) Its great to be able to scaffold repositories for data manipulation, since its quite repetative task and prone to errors.
Just a tought - would it be possible to do something like adding a switch to CLI to use custom templates? I think you can use {{define XXX}} to declare fragments, so parsing one file with all the fragments needed and using them instead of deafults for code generation shouldn't be a overkill.
It would add a great flexibility to the lib. I can help of course ;)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.