The documentation states that we should use the artifact postgresql from the group postgresql, when the last release was in 2011 and newer releases are in the group org.postgresql.

It might be worth creating a quill-mongo and have two other modules depending on it: one async and another sync.

These are just pointers meant to help the development process, feel free to be creative and suggest better solutions!

100% test coverage is not enough. Let's give sbt-pit a try.

Sqls like select * from users where name like '%a%' are normal in real world applications. How to write a quote to express this ?

The dialect creation itself is simple, the hard part is have integration tests in place for it.

Currently, queries accept a List[Any] for parameters in their prepared statements. I think it would be a huge improvement to use an HList or something similar, to retain type safety at the time that those are filled in.

it'd be nice to have support to implicit class or def as example:

implicit class InfixDateTime(date: DateTime) {
  def >(date: DateTime): Boolean = ???
}
case class User(id: Long, createdAt: DateTime)
val now = quote(infix"now()".as[DateTime])
val q1 = quote {
  query[User].filter(_.createdAt == unquote(now))
}
mirrorSource.run(q1).sql
// Information:(22, 18) SELECT x$1.id, x$1.createdAt FROM User x$1 WHERE x$1.createdAt = now()
val q2 = quote {
  query[User].filter(_.createdAt > unquote(now))
}
mirrorSource.run(q2).sql 
// Error:(19, 22) Tree 'A$A81.this.InfixDateTime(x$2.createdAt).>(io.getquill.`package`.unquote[org.joda.time.DateTime](A$A81.this.now))' can't be parsed to 'Ast'
// lazy val q2 = quote {
//                    ^

It would be abolutelly awesome to support finagle-postgres in Quill. I'm might volunteer and give it a try, althought I can't promise any progress in the nearest feature, so I decided to open an issue so anyone else can pick it up.

Hello, currently it's not good that you need to have a application.properties file, it would be better to somehow configure the Configuration.

Create a quill subject for the databen.ch benchmark.

Could specialize quill-sql to generate CQL.

What driver should we use?

https://github.com/websudos/phantom
http://docs.datastax.com/en/developer/java-driver/2.0/java-driver/whatsNew2.html

OrientDB have graph and document api too, support it will be a great

Similar to macro quasiquotes

  implicit class In[T](column: T) {
    def in(values: T*) = quote {
      infix"$column IN (..$values)".as[Boolean]
    }
  }

Trying to chain several leftJoins like so:

case class Person(id: Int, name: String, age: Int)
case class Account(id: Int, personId: Int)
case class Email(id: Int, accountId: Int, email: String)

 val q = quote {
  query[Person].leftJoin(query[Account]).on((p, a) => a.personId == p.id)
    .leftJoin(query[Email]).on { case ((p, aOpt), e) => aOpt.exists(_.id == e.accountId)}
}

results in the following compilation error:

[error] /home/marius/tmp/quill/src/main/scala/test.scala:61: Tree '(_1: Person, _2: Option[Account])(Person, Option[Account])((p @ _), (aOpt @ _))' can't be parsed to 'Ident'
[error]     val q = quote {

Is it possible to chain multiple left/right/outer joins?

The database connection pool is not being closed after SQL probing. This behavior becomes problematic for users running long-lived sbt sessions with ~testOnly.

The sources are loaded at compile time by the ResolveSourceMacro and used by SqlSourceMacro.prepare.

Some alternatives:

1. Close the source for each query probing (probably would affect compilation time)
2. Change the sources cache to have a TTL and close the source on eviction
3. Find a way to receive a "compilation finished" message from the compiler and close the source (not sure it's possible).

Any of these solutions would require to expose a new method to close the source, that could also be useful for the library's users.

See http://martinfowler.com/eaaCatalog/optimisticOfflineLock.html.

Design yet to be defined.

The QueryGenerator is used by NormalizeSpec and SqlQuerySpec. It doesn't generate queries for all supported features.

• Verify if it's worth changing these tests to use scalacheck
• Generate queries using the missing features (check the AST hierarchy)
• If bugs are detected, open new issues form them and mark the tests as pending. Try to group similar bugs in only one issue.

Quill should provide built-in support for master-slave setups.

Provide a simple and clean API for CRUD operations.

For inspiration:
https://spring.io/guides/gs/accessing-data-jpa/
https://github.com/strongtyped/active-slick

Quotations could be translated to optimized execution on top of in-memory collections. Example: https://github.com/johnynek/inliner#collections.

http://thinkaurelius.github.io/titan/

See https://github.com/getquill/quill#sql-specific-operations

• for update
• returning id
• ...

Quill requires explicit definition of all columns:

val a1 = quote {
  (name: String, age: Int) =>
    query[Person].insert(_.name -> name, _.age -> age)
}
val a2 = quote {
  (name: String, age: Int) =>
    query[Person].update(_.name -> name, _.age -> age)
}

the execution uses tuples, not the case class instances:

db.run(a1).using(List(("John", 33), ("Marie", 44)))
db.run(a2).using(List(("John", 33), ("Marie", 44)))

Using case class instances directly should be supported:

val a1 = quote {
  (p: Person) =>
    query[Person].insert(p)
}
db.run(a2).using(List(Person("John", 33), Person("Marie", 44)))

val a2 = quote {
  (p: Person) =>
    query[Person].update(p)
}
db.run(a2).using(List(Person("John", 33), Person("Marie", 44)))

Tasks

• There's a similar mechanism in place for queries. If a case class is selected, the query is flattened (.map(c => (c.attribute1, c.attribute2, ...)) and the final result is decoded to the case class instance. Take some time to read the query mechanism, starting by QueryMacro.
• The ActionMacro is the macro implementation for running actions (insert/update/delete). The flattening of inputs must happen in this macro, similarly to QueryMacro. Read the macro implementation and compare it to QueryMacro.
• Only inserts and updates need to be flattened. The ActionMacro needs to be able to identify the type of the action type in order to avoid flattening deletes. Change the Insertable, Updatable and Deletable to return sub types of Action.
• Identify the actions that need to have their assignments flattened in ActionMacro and implement a mechanism similar to QueryMacros's flattenSelect/selectResultExtractor.
• Ensure that the code is 100% covered by unit tests
• Change PeopleSpec, DepartmentsSpec and all their subclasses to use the new insert/update mechanism.

Depends on #21.

The IO monad could provide an integration with http://getclump.io to optimize the execution plan.

Similar to Slick's DBIO. It should be possible to use it with any of the Source implementations.

The config fails to load because Class.getSimpleName fails for objects defined in the REPL.

Possible solutions:

1. Inject the config name in ResolveSourceMacro, avoiding the runtime reflection (getSimpleClass).
2. Use getName instead of getSimpleName and parse the result.

The first solution is the best one, but it is more complex. The second solution is also acceptable.

Welcome to Scala version 2.11.7 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_60).
Type in expressions to have them evaluated.
Type :help for more information.

scala> import io.getquill.source.jdbc._
import io.getquill.source.jdbc._

scala> 
import io.getquill.naming.Literal
import io.getquill.source.sql.idiom.MySQLDialect

object testDB extends JdbcSource[MySQLDialect, Literal]

scala> import io.getquill.naming.Literal

scala> import io.getquill.source.sql.idiom.MySQLDialect

scala> 
scala> defined object testDB

scala> import io.getquill._
import io.getquill._

scala> case class Person(name: String)
defined class Person

scala> testDB.run(query[Person])
<console>:22: SELECT x.name FROM Person x
       testDB.run(query[Person])
                 ^
17:54:45.225 [main] INFO  com.zaxxer.hikari.HikariDataSource - HikariCP pool HikariPool-0 is starting.
java.lang.InternalError: Malformed class name
  at java.lang.Class.getSimpleName(Class.java:1330)
  at io.getquill.source.Source$$anonfun$1.apply(Source.scala:27)
  at io.getquill.source.Source$$anonfun$1.apply(Source.scala:27)
  at scala.Option.getOrElse(Option.scala:121)
  at io.getquill.source.Source.<init>(Source.scala:26)
  at io.getquill.source.sql.SqlSource.<init>(SqlSource.scala:11)
  at io.getquill.source.jdbc.JdbcSource.<init>(JdbcSource.scala:16)
  ... 35 elided

scala> 

Quill's core should cross-compile to ScalaJS, enabling usage of new submodules like quill-graphql in ScalaJS.

• Check if typesage-config and scala-logging provide ScalaJS artifacts. If not, move the dependencies and specific code to quill-sql.
• Check what version of scalatest provides a ScalaJS artifact. There's a blog post mentioning ScalaJS support.
• Configure the cross compilation.

Hi, just a small thing: I think the license referenced in build.sbt is not correct, it references LGPL and not ASL.

Best regards,
Jan

Quill currently doesn't provide a way to define the isolation level of sql database transactions.

Quill allows the user to read non-optional fields from nullable database fields. For instance, some fallbacks are in place to support this scenario.

Quill should emit compilation warnings during the sql probing if a nullable field is being read to a non-optional value.

Prototype

case class Person(id: Int, name: String, contactId: Int)
case class Contact(id: Int, phone: String)

val personWithContact = 
   nest[Person](_.contactId)(id => query[Contact].filter(_.id == id))

Expands to:

val nested = quote {
  for {
    p <- Person
    c <- Contact if(c.id == p.contactId)
  } yield {
    (p.id, p.name, c)
  }
}

Usage:

val q = quote {
  personWithContact.filter {
    case (personId, personName, contact) =>
      contact.phone != ""
  }
}

Example of failure:

https://travis-ci.org/getquill/quill#L3326

• Change all error messages (search for fail) to have a comprehensive and actionable text
• Add a link to a documentation page with a detailed explanation of the error.
• If the error is a bug, provide a link to submit a github issue. Ex.: https://github.com/getquill/quill/issues/new?title=test&body=body

Quill requires multiple queries to use the upsert (insert or update) pattern:

case class Person(name: String)

val count = quote {
  (name: String) => query[Person].filter(_.name == name).size
}
val insert = quote {
  (name: String) => query[Person].insert(_.name -> name)
}
val update = quote {
  (name: String) => query[Person].update(_.name -> name)
}

db.transaction {
  if(db.run(count) == 0)
    db.run(insert)
  else
    db.run(update)
}

It should support a single upsert statement:

val a = quote {
  (name: String) => query[Person].upsert(_.name -> name)
}
db.run(a)

• Verify if the supported databases (mysql and postgres for now) have support for upserts using a single and atomic sql statement. If not, we might need to redesign the change.
• Add a new trait similar to Insertable/Updatable
• Create a new action AST for it.
• Implement a parser for it.
• Change the actionUnliftable to support it. The Unliftable instances are responsible for reading the quoted AST when expanding them within another quotation.
• Change the actionLiftable to support it. Liftable instances are responsible for lifting the query AST to the quoted instance.
• Fix the compilation warnings of pattern matching not considering the new AST type. Use the implementation for inserts as reference.
• Implement the integration between SqlIdiom and the specific dialect.

• Implement the new dialect using an existing one as example
• Implement the integration tests PeopleSpec, DepartmentsSpec and EncodingSpec for it

The dialect creation itself is simple, the hard part is have integration tests in place for it.

Depends on #6.

The recommended way of using FOR UPDATE is using infix. It should be a feature supported by quill-sql.

• Implement an implicit class in quill-sql using infix. It'll be similar to the readme example, but using the implicit mechanism described by #6.
• Ensure that the code is 100% covered by unit tests.
• No integration tests are required.
• Document the feature.

For inspiration: https://github.com/VirtusLab/unicorn

the idea is to auto map the select in the case class

case class User(id:Long, name:String, password:String)
case class BasicUser(id:Long, name:String)
query[User].mapTo[BasicUser]

Another use case

case class User(id:Long, name:String, age:Option[Long], createdAt:DateTime)
object User {
  case class Create(name:String, age:Option[Long])
  case class Update(name:Option[String], age:Option[Long])
}

val insertQuery = query[User].insert[User.Create].returning(_.id)
val updateQuery = query[User].filter(_.id = 1).update[User.Update]
// or
val insertQuery = query[User].mapTo[User.Update].insert.returning(_.id)
val updateQuery = query[User].filter(_.id = 1).mapTo[User.Update].update

db.run(insertQuery)(User.Create("name", 18))
db.run(updateQuery(User.Update(Some("name"), Some(18))
// or
db.run(updateQuery(User.Update(None, None) // in this case the field name will be ignore and the field age will update to `Null`

Support for nested isolated transactions using savepoints.

Implement a simplified version of the TPC-H benchmark.

Subjects for the first version:

• quill
• slick

Depends on #6.

The recommended way of returning IDs from inserts is using infix. It should be a feature supported by quill-sql.

• Implement an implicit class in quill-sql using infix. It'll be similar to the readme example, but using the implicit mechanism described by #6.
• Ensure that the code is 100% covered by unit tests.
• Implement one integration test for each source type to check that they return the ID information.
• Document the feature.

Decide if Quill should have a migration module. If not, document the recommended library for migrations.

Some people consider query[T] boilerplate code. I don't agree that it is boilerplate, but it'd be nice to provide to these users a convenience implicit from case classes companion objects to query[T]. It should require explicit import, not being available under io.getquill directly.

Quill has support for transactions but it is not documented yet. I didn't document it because the current API needs a review, each Source implementation has a complete different transaction implementation. This issue is meant to review the current implementation, make the changes if necessary, and document the feature.

• Review AsyncSource. It uses a custom execution context that needs to be in the scope of the future composition. The source checks if the execution context is transactional and uses the transaction's connection.
• Review JdbcSource. It uses a DynamicValue, that is a high-level Scala API using ThreadLocal, to store the current connection and use it if defined.
• Review FinagleMysqlSource. It uses a Local, that is a "kind" of thread local for future composition, to store the current transactional client and use it if available.
• Judge if the APIs should be changed to have similar implementations. For instance, the transaction methods could return a new instance of the source that is transactional and the user is responsible for passing around this transactional source. It'd be an approach similar to finagle-mysql
• Share your thoughts here.
• Make the change if we judge necessary.
• Document the feature.

IntelliJ doesn't support whitebox macros. To workaround this limitation, all macros need to either be annotated with return types or converted to blackbox.

• quotation - generates type refinements with type-level information about the AST. Check if IntelliJ doesn't fail to compose quotations with the current API (whitebox is required here). It shouldn't fail, since the macro is definition is already annotated with Quoted[T] and no relevant methods are added.
• execution - executes queries and actions. It is by extended by concrete implementations and used by sources. This macro needs to be changed to blackbox, returning specific types for each type of source and actions/queries.
• bind variables - allows the user to define runtime bind values. This macro needs to be converted to blackbox.

• Normalization
• Query generation
• Result extraction

Use http://openjdk.java.net/projects/code-tools/jmh/

schema

case class User(id: Int, name: String, username: String, age: Int, friends: List[User])

target query

{
  user(id: 1) {
    name
    age
    friends {
      name
    }
  }
}

Option 1 - Translate collection-like queries to GraphQL

val q = quote {
  (id: Int) => query[User].filter(_.id == id).map { user => 
    (user.name, user.age, user.friends.map(_.name))
  }
}

Option 2 - Create a new DSL similar to the target lang

val q = quote {
  (id: Int) =>
    query[User](_.id -> id)(
      _.name,
      _.age,
      _.friends(
        _.name
      )
    )
}

I'm leaning toward the first option.