Comments (7)
I did more investigation and thinking on how to solve this.
-
CockroachDB apparently uses oids in the upper half of the unsigned range. Its also an awesome database that already works with Skunk, and I'd really like to support it.
-
The surface area within Skunk of
oid: Int
is huge. I tried changing tooid: Long
but the compile errors just kept coming in waves.
The saving grace is, we have enough bits in an Int to store an unsigned Int without losing info. We'll just have to accept the caveat, that for the rare case of oids > Int.MaxInt, they will be represented in Scala as a negative Int:
scala> (Int.MaxValue.toLong + 1).toInt
val res2: Int = -2147483648
With this encoding, the fix can be contained entire within the 1-line oid codec.
If maintainers are on board with this solution (which will be backwards compatible with all Postgres oids seen to date, that are less than Int.MaxInt), I'll send a PR?
from skunk.
@benhutchison This seems reasonable to me. Happy to review a PR.
from skunk.
Actually Flyway's performance when a database doesn't require any migration was what drove me nuts and led me to stumble on Dumbo. I have a CLI tool that extracts reports from a database, but includes a shared kernel that also checks migrations.
With Dumbo takes 10868ms to extract a report , with Flyway 19887ms
from skunk.
I'm also hitting this issue.
Postgres OID docs specify "The oid type is currently implemented as an unsigned four-byte integer. "
The relevant Skunk code is here. Oid's sadly isn't representable by an Int
, unless we added an encoding that oid
s above max int were stored as negative ints?
val oid: Codec[Int] = Codec.simple(_.toString, _.toInt.asRight, Type.oid)
Actually, I hit this when trying out @rolang's Dumbo, a (hopefully faster) alternative to Flyway.
AFAICT, because it makes use of Typer.Strategy.SearchPath
which requires querying database metadata, and so hits the oid codec problem.
from skunk.
Actually, I hit this when trying out @rolang's Dumbo, a (hopefully faster) alternative to Flyway.
Offtopic, but since it was mentioned, from some of my testing locally Flyway was always way slower with CockroachDB
. One of the results as example that I was running:
// flyway: 75 migrations executed in 58265ms
// dumbo: 75 migrations executed in 22780ms
Same migration scripts with the same dockerized single node instance and empty database with latest Flyway as of now 10.12.0. But don't take it as proper benchmarking 😅.
from skunk.
Related Issues (20)
- Type inference for type argument in `to[A]` no longer working HOT 1
- Deadlock while using session pool HOT 1
- Unhandled backend message HOT 1
- e HOT 1
- Feature request: logical replication HOT 2
- Aggregate function on empty table produces exception when executing with option HOT 2
- Cannot configure Recycler on connection pool
- Ambiguous overload for 0.6.0 Twiddle lists HOT 3
- Skunk sends all arguments to tracing/logging HOT 6
- Fragment's stripMargin can change the query
- Exceptions in transactions are printed to stderr HOT 3
- Skunk can't find `enum` type in schema HOT 3
- Support query pipelineing in a single session HOT 1
- Relax reliance on `SocketGroup` HOT 1
- Feature reqest: migrations. HOT 1
- Extending the sql interpolator?
- Session pool examples [Question]
- decode joined rows
- Fragment interpolation breaks with nested fragments HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from skunk.