Git Product home page Git Product logo

postgres-nio's Introduction

PostgresNIO

Documentation MIT License Continuous Integration Swift 5.7+ SSWG Incubation Level: Graduated

๐Ÿ˜ Non-blocking, event-driven Swift client for PostgreSQL built on SwiftNIO.

Features:

  • A PostgresConnection which allows you to connect to, authorize with, query, and retrieve results from a PostgreSQL server
  • A PostgresClient which pools and manages connections
  • An async/await interface that supports backpressure
  • Automatic conversions between Swift primitive types and the Postgres wire format
  • Integrated with the Swift server ecosystem, including use of SwiftLog and ServiceLifecycle.
  • Designed to run efficiently on all supported platforms (tested extensively on Linux and Darwin systems)
  • Support for Network.framework when available (e.g. on Apple platforms)
  • Supports running on Unix Domain Sockets

API Docs

Check out the PostgresNIO API docs for a detailed look at all of the classes, structs, protocols, and more.

Getting started

Interested in an example? We prepared a simple Birthday example in the Snippets folder.

Adding the dependency

Add PostgresNIO as dependency to your Package.swift:

  dependencies: [
    .package(url: "https://github.com/vapor/postgres-nio.git", from: "1.21.0"),
    ...
  ]

Add PostgresNIO to the target you want to use it in:

  targets: [
    .target(name: "MyFancyTarget", dependencies: [
      .product(name: "PostgresNIO", package: "postgres-nio"),
    ])
  ]

Creating a client

To create a PostgresClient, which pools connections for you, first create a configuration object:

import PostgresNIO

let config = PostgresClient.Configuration(
  host: "localhost",
  port: 5432,
  username: "my_username",
  password: "my_password",
  database: "my_database",
  tls: .disable
)

Next you can create you client with it:

let client = PostgresClient(configuration: config)

Once you have create your client, you must run() it:

await withTaskGroup(of: Void.self) { taskGroup in
    taskGroup.addTask {
        await client.run() // !important
    }

    // You can use the client while the `client.run()` method is not cancelled.

    // To shutdown the client, cancel its run method, by cancelling the taskGroup.
    taskGroup.cancelAll()
}

Querying

Once a client is running, queries can be sent to the server. This is straightforward:

let rows = try await client.query("SELECT id, username, birthday FROM users")

The query will return a PostgresRowSequence, which is an AsyncSequence of PostgresRows. The rows can be iterated one-by-one:

for try await row in rows {
  // do something with the row
}

Decoding from PostgresRow

However, in most cases it is much easier to request a row's fields as a set of Swift types:

for try await (id, username, birthday) in rows.decode((Int, String, Date).self) {
  // do something with the datatypes.
}

A type must implement the PostgresDecodable protocol in order to be decoded from a row. PostgresNIO provides default implementations for most of Swift's builtin types, as well as some types provided by Foundation:

  • Bool
  • Bytes, Data, ByteBuffer
  • Date
  • UInt8, Int16, Int32, Int64, Int
  • Float, Double
  • String
  • UUID

Querying with parameters

Sending parameterized queries to the database is also supported (in the coolest way possible):

let id = 1
let username = "fancyuser"
let birthday = Date()
try await client.query("""
  INSERT INTO users (id, username, birthday) VALUES (\(id), \(username), \(birthday))
  """, 
  logger: logger
)

While this looks at first glance like a classic case of SQL injection ๐Ÿ˜ฑ, PostgresNIO's API ensures that this usage is safe. The first parameter of the query(_:logger:) method is not a plain String, but a PostgresQuery, which implements Swift's ExpressibleByStringInterpolation protocol. PostgresNIO uses the literal parts of the provided string as the SQL query and replaces each interpolated value with a parameter binding. Only values which implement the PostgresEncodable protocol may be interpolated in this way. As with PostgresDecodable, PostgresNIO provides default implementations for most common types.

Some queries do not receive any rows from the server (most often INSERT, UPDATE, and DELETE queries with no RETURNING clause, not to mention most DDL queries). To support this, the query(_:logger:) method is marked @discardableResult, so that the compiler does not issue a warning if the return value is not used.

Security

Please see SECURITY.md for details on the security process.

postgres-nio's People

Contributors

0xtim avatar bennydebock avatar calebkleveter avatar dnadoba avatar fabianfett avatar flix477 avatar grennis avatar gwynne avatar jaapwijnen avatar jareyesda avatar jccampagne avatar jerry-carter avatar jiahan-wu avatar jordanebelanger avatar jtouzy avatar lovetodream avatar madsodgaard avatar mahdibm avatar mariosangiorgio avatar marius-se avatar maxdesiatov avatar mremond avatar mrmage avatar needleinajaystack avatar nicholas-otto avatar rausnitz avatar sidepelican avatar tanner0101 avatar tbartelmess avatar tkrajacic avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

postgres-nio's Issues

remove row column tableOID parameter

Currently PostgresRow let's you fetch a column for a given tableOID (UInt32). This is not really useful in practice and is holding us back from some optimizations that could improve decoding performance. I think we should consider removing this.

Unsigned integer read as signed

Sometimes my vapor + fluent application crashes with a NIO-ELT-0-#8 (10): Fatal error: Negative value is not representable after a db update. After this happens the app will crash every time I try to load that record until I manually remove it from the postgres db.

The value is defined with the schema: .field("ram_bytes", .uint32) and when the crash happens it looks like it tries to decode the value as a signed 32 bit integer instead of unsigned. Looking at the db record the value has somehow ended up being a negative value (-1872634973). I haven't been able to reliably reproduce this yet but I've seen it happen several times now.

Stack trace

* thread #10, name = 'NIO-ELT-0-#8', stop reason = Fatal error: Negative value is not representable
    frame #0: 0x00007fff72eb8380 libswiftCore.dylib`_swift_runtime_on_report
    frame #1: 0x00007fff72f32243 libswiftCore.dylib`_swift_stdlib_reportFatalErrorInFile + 211
    frame #2: 0x00007fff72bf91dd libswiftCore.dylib`closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._assertionFailure(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 525
    frame #3: 0x00007fff72bf8cf7 libswiftCore.dylib`closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._assertionFailure(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 87
    frame #4: 0x00007fff72bf89c3 libswiftCore.dylib`closure #1 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._assertionFailure(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 99
    frame #5: 0x00007fff72bf85d5 libswiftCore.dylib`Swift._assertionFailure(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 533
    frame #6: 0x00007fff72de048c libswiftCore.dylib`protocol witness for Swift.BinaryInteger.init<A where A1: Swift.BinaryInteger>(A1) -> A in conformance Swift.UInt32 : Swift.BinaryInteger in Swift + 940
    frame #7: 0x00007fff72e8ec48 libswiftCore.dylib`dispatch thunk of Swift.BinaryFloatingPoint.init<A where A1: Swift.BinaryFloatingPoint>(A1) -> A + 8
    frame #8: 0x00007fff72ead449 libswiftCore.dylib`dispatch thunk of Swift.BinaryInteger.init<A where A1: Swift.BinaryInteger>(A1) -> A + 9
    frame #9: 0x0000000100b4f0f9 Run`PostgresData.fwi<I>(type=UInt32, self=PostgresNIO.PostgresData @ 0x0000700000797520) at PostgresData+Int.swift:137:24
    frame #10: 0x0000000100b504d2 Run`FixedWidthInteger.init(postgresData=PostgresNIO.PostgresData @ 0x0000700000797520) at PostgresData+Int.swift:183:38
    frame #11: 0x0000000100b50d9e Run`protocol witness for PostgresDataConvertible.init(postgresData:) in conformance UInt32 at <compiler-generated>:0
    frame #12: 0x0000000100b246bc Run`PostgresDataDecoder.decode<T>(type=UInt32, data=PostgresNIO.PostgresData @ 0x0000700000797520, self=0x0000000102cbaf90) at PostgresDataDecoder.swift:14:43
    frame #13: 0x0000000100b322e7 Run`_PostgreSQLRow.decode<D>(column="products_ram_bytes", type=UInt32, self=PostgresKit._PostgreSQLRow @ 0x00007000007974a0) at PostgresRow+SQL.swift:34:33
    frame #14: 0x0000000100b324bf Run`protocol witness for SQLRow.decode<A>(column:as:) in conformance _PostgreSQLRow at <compiler-generated>:0
    frame #15: 0x0000000100a896d0 Run`_PostgresDatabaseOutput.decode<T>(key=prefix, type=UInt32, self=FluentPostgresDriver._PostgresDatabaseOutput @ 0x00007000007976a0) at PostgresRow+Database.swift:43:14
    frame #16: 0x0000000100a89832 Run`protocol witness for DatabaseOutput.decode<A>(_:as:) in conformance _PostgresDatabaseOutput at <compiler-generated>:0
    frame #17: 0x0000000100a89e95 Run`_SchemaDatabaseOutput.decode<T>(key=string, type=UInt32, self=FluentPostgresDriver._SchemaDatabaseOutput @ 0x0000000102e0dfc0) at PostgresRow+Database.swift:83:25
    frame #18: 0x0000000100a89fad Run`protocol witness for DatabaseOutput.decode<A>(_:as:) in conformance _SchemaDatabaseOutput at <compiler-generated>:0
    frame #19: 0x0000000100065286 Run`FieldProperty.output(output=FluentPostgresDriver._SchemaDatabaseOutput @ 0x0000000102e0dfc0, self=0x0000000102e1a4a0) at Field.swift:100:47
    frame #20: 0x00000001000654fd Run`protocol witness for AnyDatabaseProperty.output(from:) in conformance FieldProperty<A, B> at <compiler-generated>:0
    frame #21: 0x000000010004801e Run`closure #2 in Fields.output(field=0x0000000102e1a4a0, output=FluentPostgresDriver._SchemaDatabaseOutput @ 0x0000000102e0dfc0) at Fields.swift:65:23
    frame #22: 0x0000000100048070 Run`partial apply for closure #2 in Fields.output(from:) at <compiler-generated>:0
    frame #23: 0x0000000100047c54 Run`thunk for @callee_guaranteed (@guaranteed AnyDatabaseProperty) -> (@error @owned Error) at <compiler-generated>:0
    frame #24: 0x0000000100048094 Run`thunk for @callee_guaranteed (@guaranteed AnyDatabaseProperty) -> (@error @owned Error)partial apply at <compiler-generated>:0
    frame #25: 0x00007fff72ce664d libswiftCore.dylib`(extension in Swift):Swift.Sequence.forEach((A.Element) throws -> ()) throws -> () + 381
    frame #26: 0x0000000100047e41 Run`Fields.output(output=FluentPostgresDriver._SchemaDatabaseOutput @ 0x0000000102e0dfc0, self=0x0000000102e0f880) at Fields.swift:64:11
    frame #27: 0x0000000100e22041 Run`protocol witness for Fields.output(from:) in conformance Product at <compiler-generated>:0
    frame #28: 0x00000001000a2493 Run`closure #1 in closure #1 in QueryBuilder.all(output=FluentPostgresDriver._PostgresDatabaseOutput @ 0x0000700000798170, all=0 values) at QueryBuilder.swift:210:27
    frame #29: 0x00000001000a4f23 Run`partial apply for closure #1 in closure #1 in QueryBuilder.all(_:) at <compiler-generated>:0
    frame #30: 0x00000001000a2060 Run`closure #1 in QueryBuilder.all(output=FluentPostgresDriver._PostgresDatabaseOutput @ 0x0000700000798170, onOutput=0x00000001000a2ed0 Run`partial apply forwarder for closure #1 (Swift.Result<A, Swift.Error>) -> () in FluentKit.QueryBuilder.all() -> NIO.EventLoopFuture<Swift.Array<A>> at <compiler-generated>, all=0 values) at QueryBuilder.swift:208:23
    frame #31: 0x00000001000a4456 Run`closure #1 in QueryBuilder.run(output=FluentPostgresDriver._PostgresDatabaseOutput @ 0x0000700000798170, self=0x0000000107814990, onOutput=0x00000001000a21d0 Run`partial apply forwarder for closure #1 (FluentKit.DatabaseOutput) -> () in FluentKit.QueryBuilder.all((Swift.Result<A, Swift.Error>) -> ()) -> NIO.EventLoopFuture<()> at <compiler-generated>) at QueryBuilder.swift:285:13
    frame #32: 0x0000000100a7de24 Run`closure #2 in _FluentPostgresDatabase.execute($0=PostgresNIO.PostgresRow @ 0x00007000007981b0, onOutput=0x00000001000a44b0 Run`partial apply forwarder for closure #1 (FluentKit.DatabaseOutput) -> () in FluentKit.QueryBuilder.run((FluentKit.DatabaseOutput) -> ()) -> NIO.EventLoopFuture<()> at <compiler-generated>, self=FluentPostgresDriver._FluentPostgresDatabase @ 0x0000000107815890) at FluentPostgresDatabase.swift:29:17
    frame #33: 0x0000000100b84498 Run`PostgresParameterizedQuery.respond(message=PostgresNIO.PostgresMessage @ 0x0000700000798cf0, self=0x0000000107815950) at PostgresDatabase+Query.swift:139:17
    frame #34: 0x0000000100b8800a Run`protocol witness for PostgresRequest.respond(to:) in conformance PostgresParameterizedQuery at <compiler-generated>:0
    frame #35: 0x0000000100b3912b Run`PostgresRequestHandler._channelRead(context=0x0000000102d0c6b0, data=NIO.NIOAny @ 0x0000700000799bd0, self=0x0000000102d0a810) at PostgresConnection+Database.swift:64:49
    frame #36: 0x0000000100b39b65 Run`PostgresRequestHandler.channelRead(context=0x0000000102d0c6b0, data=NIO.NIOAny @ 0x0000700000799bd0, self=0x0000000102d0a810) at PostgresConnection+Database.swift:81:22
    frame #37: 0x0000000100b3b131 Run`protocol witness for _ChannelInboundHandler.channelRead(context:data:) in conformance PostgresRequestHandler at <compiler-generated>:0
    frame #38: 0x00000001001369cd Run`ChannelHandlerContext.invokeChannelRead(data=NIO.NIOAny @ 0x0000700000799bd0, self=0x0000000102d0c6b0) at ChannelPipeline.swift:1339:28
    frame #39: 0x0000000100136a61 Run`ChannelHandlerContext.invokeChannelRead(data=NIO.NIOAny @ 0x0000700000799bd0, self=0x0000000102d0c430) at ChannelPipeline.swift:1341:24
    frame #40: 0x000000010013a5d6 Run`ChannelHandlerContext.fireChannelRead(data=NIO.NIOAny @ 0x0000700000799bd0, self=0x0000000102d0fd40) at ChannelPipeline.swift:1152:20
    frame #41: 0x0000000100b7df92 Run`PostgresMessageDecoder.decode(context=0x0000000102d0fd40, buffer=NIO.ByteBuffer @ 0x000070000079a270, self=0x0000000102d0d340) at PostgresMessageDecoder.swift:56:17
    frame #42: 0x0000000100b7e723 Run`protocol witness for ByteToMessageDecoder.decode(context:buffer:) in conformance PostgresMessageDecoder at <compiler-generated>:0
    frame #43: 0x00000001001567fa Run`closure #1 in ByteToMessageHandler.decodeLoop(decoder=0x0000000102d0d340, buffer=NIO.ByteBuffer @ 0x000070000079a270, decodeMode=normal, self=0x0000000102d0b8e0, context=0x0000000102d0fd40, allowEmptyBuffer=false) at Codec.swift:567:49
    frame #44: 0x000000010015e27d Run`partial apply for closure #1 in ByteToMessageHandler.decodeLoop(context:decodeMode:) at <compiler-generated>:0
    frame #45: 0x0000000100155142 Run`ByteToMessageHandler.withNextBuffer(allowEmptyBuffer=false, body=0x000000010015e260 Run`partial apply forwarder for closure #1 (inout A, inout NIO.ByteBuffer) throws -> NIO.DecodingState in NIO.ByteToMessageHandler.(decodeLoop in _F2A740607FEBA425AF6F8C49A24C0AD7)(context: NIO.ChannelHandlerContext, decodeMode: NIO.ByteToMessageHandler<A>.(DecodeMode in _F2A740607FEBA425AF6F8C49A24C0AD7)) throws -> NIO.(B2MDBuffer in _F2A740607FEBA425AF6F8C49A24C0AD7).BufferProcessingResult at <compiler-generated>, self=0x0000000102d0b8e0) at Codec.swift:529:36
    frame #46: 0x0000000100155e9c Run`ByteToMessageHandler.decodeLoop(context=0x0000000102d0fd40, decodeMode=normal, self=0x0000000102d0b8e0) at Codec.swift:563:35
    frame #47: 0x0000000100157c4b Run`ByteToMessageHandler.channelRead(context=0x0000000102d0fd40, data=NIO.NIOAny @ 0x000070000079ae28, self=0x0000000102d0b8e0) at Codec.swift:636:29
    frame #48: 0x00000001001584e9 Run`protocol witness for _ChannelInboundHandler.channelRead(context:data:) in conformance ByteToMessageHandler<A> at <compiler-generated>:0
    frame #49: 0x00000001001369cd Run`ChannelHandlerContext.invokeChannelRead(data=NIO.NIOAny @ 0x000070000079ae28, self=0x0000000102d0fd40) at ChannelPipeline.swift:1339:28
    frame #50: 0x000000010013241d Run`ChannelPipeline.fireChannelRead0(data=NIO.NIOAny @ 0x000070000079ae28, self=0x0000000107606b80) at ChannelPipeline.swift:829:29
    frame #51: 0x00000001000e7eac Run`BaseStreamSocketChannel.readFromSocket(self=0x0000000107606570) at BaseStreamSocketChannel.swift:107:35
    frame #52: 0x00000001000df25c Run`BaseSocketChannel.readable0(self=0x0000000107606570) at BaseSocketChannel.swift:1060:35
    frame #53: 0x00000001000e0e52 Run`BaseSocketChannel.readable(self=0x0000000107606570) at BaseSocketChannel.swift:1044:14
    frame #54: 0x00000001000e22d9 Run`protocol witness for SelectableChannel.readable() in conformance BaseSocketChannel<A> at <compiler-generated>:0
    frame #55: 0x00000001001f7ffa Run`SelectableEventLoop.handleEvent<C>(ev=(rawValue = 4), channel=0x0000000107606570, self=0x0000000102d053a0) at SelectableEventLoop.swift:323:25
    frame #56: 0x00000001001f9b78 Run`closure #1 in closure #2 in SelectableEventLoop.run(ev=NIO.SelectorEvent<NIO.NIORegistration> @ 0x000070000079b7d8, self=0x0000000102d053a0) at SelectableEventLoop.swift:400:30
    frame #57: 0x00000001001f9f09 Run`thunk for @callee_guaranteed (@guaranteed SelectorEvent<NIORegistration>) -> (@error @owned Error) at <compiler-generated>:0
    frame #58: 0x00000001001fe544 Run`partial apply for thunk for @callee_guaranteed (@guaranteed SelectorEvent<NIORegistration>) -> (@error @owned Error) at <compiler-generated>:0
    frame #59: 0x000000010020eb44 Run`Selector.whenReady(strategy=blockUntilTimeout, body=0x00000001001fe530 Run`partial apply forwarder for reabstraction thunk helper from @callee_guaranteed (@guaranteed NIO.SelectorEvent<NIO.NIORegistration>) -> (@error @owned Swift.Error) to @escaping @callee_guaranteed (@in_guaranteed NIO.SelectorEvent<NIO.NIORegistration>) -> (@error @owned Swift.Error) at <compiler-generated>, self=0x0000000102d052f0) at Selector.swift:561:17
  * frame #60: 0x00000001001f9899 Run`closure #2 in SelectableEventLoop.run(self=0x0000000102d053a0, nextReadyTask=0x0000000107509e40) at SelectableEventLoop.swift:395:36
    frame #61: 0x00000001001fd494 Run`partial apply for closure #2 in SelectableEventLoop.run() at <compiler-generated>:0
    frame #62: 0x00000001000cf1ef Run`thunk for @callee_guaranteed () -> (@error @owned Error) at <compiler-generated>:0
    frame #63: 0x00000001001fd4b4 Run`thunk for @callee_guaranteed () -> (@error @owned Error)partial apply at <compiler-generated>:0
    frame #64: 0x00000001001f4292 Run`closure #1 in withAutoReleasePool<T>(execute=0x00000001001fd4a0 Run`reabstraction thunk helper from @callee_guaranteed () -> (@error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out (), @error @owned Swift.Error)partial apply forwarder with unmangled suffix ".8" at <compiler-generated>) at SelectableEventLoop.swift:23:13
    frame #65: 0x00000001001f42ef Run`partial apply for closure #1 in withAutoReleasePool<A>(_:) at <compiler-generated>:0
    frame #66: 0x00007fff73250d8e libswiftObjectiveC.dylib`ObjectiveC.autoreleasepool<A>(invoking: () throws -> A) throws -> A + 46
    frame #67: 0x00000001001f4239 Run`withAutoReleasePool<T>(execute=0x00000001001fd4a0 Run`reabstraction thunk helper from @callee_guaranteed () -> (@error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out (), @error @owned Swift.Error)partial apply forwarder with unmangled suffix ".8" at <compiler-generated>) at SelectableEventLoop.swift:22:16
    frame #68: 0x00000001001f8550 Run`SelectableEventLoop.run(self=0x0000000102d053a0) at SelectableEventLoop.swift:394:17
    frame #69: 0x0000000100177f43 Run`static MultiThreadedEventLoopGroup.runTheLoop(thread=0x0000000102d05240, canEventLoopBeShutdownIndividually=false, selectorFactory=0x000000010017f840 Run`partial apply forwarder for NIO.Selector.__allocating_init() throws -> NIO.Selector<A> at <compiler-generated>, initializer=0x000000010017f640 Run`partial apply forwarder for reabstraction thunk helper from @escaping @callee_guaranteed (@in_guaranteed NIO.NIOThread) -> (@out ()) to @escaping @callee_guaranteed (@guaranteed NIO.NIOThread) -> () at <compiler-generated>, callback=0x000000010017f7c0 Run`partial apply forwarder for closure #1 (NIO.SelectableEventLoop) -> () in closure #1 (NIO.NIOThread) -> () in static NIO.MultiThreadedEventLoopGroup.(setupThreadAndEventLoop in _D5D78C61B22284700B9BD1ACFBC25157)(name: Swift.String, selectorFactory: () throws -> NIO.Selector<NIO.NIORegistration>, initializer: (NIO.NIOThread) -> ()) -> NIO.SelectableEventLoop at <compiler-generated>, self=NIO.MultiThreadedEventLoopGroup) at EventLoop.swift:824:22
    frame #70: 0x0000000100178743 Run`closure #1 in static MultiThreadedEventLoopGroup.setupThreadAndEventLoop(t=0x0000000102d05240, selectorFactory=0x000000010017f840 Run`partial apply forwarder for NIO.Selector.__allocating_init() throws -> NIO.Selector<A> at <compiler-generated>, initializer=0x000000010017f640 Run`partial apply forwarder for reabstraction thunk helper from @escaping @callee_guaranteed (@in_guaranteed NIO.NIOThread) -> (@out ()) to @escaping @callee_guaranteed (@guaranteed NIO.NIOThread) -> () at <compiler-generated>, lock=(mutex = 0x0000000102cba430), _loop=0x0000000102d053a0, loopUpAndRunningGroup=0x0000000102cba470) at EventLoop.swift:844:41
    frame #71: 0x000000010017f733 Run`partial apply for closure #1 in static MultiThreadedEventLoopGroup.setupThreadAndEventLoop(name:selectorFactory:initializer:) at <compiler-generated>:0
    frame #72: 0x0000000100178d3f Run`thunk for @escaping @callee_guaranteed (@guaranteed NIOThread) -> () at <compiler-generated>:0
    frame #73: 0x0000000100244351 Run`partial apply for thunk for @escaping @callee_guaranteed (@guaranteed NIOThread) -> () at <compiler-generated>:0
    frame #74: 0x000000010024b1c3 Run`closure #1 in static ThreadOpsPosix.run($0=(_rawValue = 0x0000000102cba3c0 -> 0x00007fff99c31f58 libswiftCore.dylib`InitialAllocationPool + 23240)) at ThreadPosix.swift:105:13
    frame #75: 0x000000010024b2f9 Run`@objc closure #1 in static ThreadOpsPosix.run(handle:args:detachThread:) at <compiler-generated>:0
    frame #76: 0x0000000102a05c65 libsystem_pthread.dylib`_pthread_start + 148
    frame #77: 0x0000000102a014af libsystem_pthread.dylib`thread_start + 15

Allow introspection of Rows by making PostgresRow.lookupTable readable

Is your feature request related to a problem? Please describe.
For a variety of complicated tasks, I want to use hand-crafted queries (instead of relying on Fluent's ORM). I run these queries and would like to convert them into an array to return them to my client.

To do this, however, I seem to need to know in advance how the row fields are named, since I can only access a row field by name. This is frustrating since I would like to use different row field names in different queries and still use the same code to convert them into a dictionary.

Describe the solution you'd like
I almost got my solution from PostgresRow.description, which loops through self.lookupTable.rowDescription.fields to get the field names.

I tried to write an extension to PostgresRow that would give me the field names, but 'lookupTable' is inaccessible due to 'internal' protection level. So my request is to make the lookupTable property readable.

Describe alternatives you've considered
The alternative I'm going with for now is to build a query and staple to it the list of expected field names and types. This seems a bit unwieldy, especially since the data is right there ๐Ÿ˜…

Thanks and stay awesome <3

assertion failure because internal typename for CHAR(n) is BPCHAR

I hit an assertion failure in PostgresData+Int.swift:

switch self.formatCode {
        case .binary:
            switch self.type {
            case .char, .bpchar:
                assert(value.readableBytes == 1) // <- Here

debugger output at that point:

(lldb) po self.type
โ–ฟ BPCHAR
  - rawValue : 1042

(lldb) po postgresData.debugDescription
"Optional(0x0020 (BPCHAR))"

(lldb) po self.debugDescription
"0x0020 (BPCHAR)"

and value.readableBytes is 2.

Most likely has to do with CHAR(1) columns I have in my tables although those tables are empty when I hit the assertion.

getUUID allows the user to read unintialised memory

func getUUID(at index: Int) -> UUID? {
precondition(index >= 0, "index must not be negative")
return self.withVeryUnsafeBytes { ptr in
guard index <= ptr.count - MemoryLayout<uuid_t>.size else {
return nil
}
var value: uuid_t = (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
withUnsafeMutableBytes(of: &value) { valuePtr in
valuePtr.copyMemory(
from: UnsafeRawBufferPointer(
start: ptr.baseAddress!.advanced(by: index),
count: MemoryLayout<UUID>.size
)
)
}
return UUID(uuid: value)
}
}
}

The code above allows the user to read uninitialised memory which is a security vulnerability.

A better way to write this function is

    func getUUID(index: Int) -> UUID? {
        var uuid: uuid_t = (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
        return self.viewBytes(at: index, length: MemoryLayout.size(ofValue: uuid)).map { bufferBytes in
            withUnsafeMutableBytes(of: &uuid) { target in
                precondition(target.count <= bufferBytes.count)
                target.copyBytes(from: bufferBytes)
            }
            return UUID(uuid: uuid)
        }
    }

Support custom json decoder and encoder when dealing with JSON/JSONB columns

At the moment, every-time a Postgres json value is decoded, a new Foundation.JSONDecoder is created on the spot for the decoding. The same thing is done for encoding json columns.

For performance (notably on Linux) and floating point encoding accuracy reasons, we should be able to use custom, i.e non-Foundation, json encoders and decoders when dealing with json and jsonb columns.

Add PostgresType

Add enum representing postgres type names. This is different than what PostgresDataType represents as that does not include precisions / lengths, it only identifies the wire protocol format.

List of supported postgres types: https://www.postgresql.org/docs/9.5/datatype.html

Note, this type should also be extendable to support custom types created through CREATE TYPE ....

please don't use anything *unsafe*

Code that contains the word *unsafe* is just as unsafe as C(++) code. In Swift there shouldn't be a necessity to use unsafe code unless we're interfacing with C(++).

Please file a NIO bug if there's anything that's not C(++) interop that you can't express without *unsafe*. See #46 , #45 , #44 for what happens with *unsafe* APIs.

Insufficient Data Error when Inserting UInt8

When trying to insert a new row into a table that is represented by a model with a UInt8 field, NIOPostgres throws an insufficient data left in message error.

I've created an MVP to display this behavior: https://github.com/calebkleveter/PostgresNIO-InsufficentData

  1. Clone the repo
  2. Configure a local PostgreSQL database. You can either customize the app configuration using environment variables or use the following values for the database:
    • Host: localhost
    • Port: 5432
    • User: Your current username
    • Password: password
    • Database Name: postgres-test
  3. Boot the application
  4. Run the POST /insert route.

You'll start off by getting an error that there was invalid JSON input:


[ ERROR ] Error(fields: [PostgresNIO.PostgresMessage.Error.Field.file: "json.c", PostgresNIO.PostgresMessage.Error.Field.routine: "report_invalid_token", PostgresNIO.PostgresMessage.Error.Field.detail: "Token \"\u{01}\" is invalid.", PostgresNIO.PostgresMessage.Error.Field.locationContext: "JSON data, line 1: \u{01}", PostgresNIO.PostgresMessage.Error.Field.severity: "ERROR", PostgresNIO.PostgresMessage.Error.Field.localizedSeverity: "ERROR", PostgresNIO.PostgresMessage.Error.Field.sqlState: "22P02", PostgresNIO.PostgresMessage.Error.Field.message: "invalid input syntax for type json", PostgresNIO.PostgresMessage.Error.Field.line: "1248"]) (PostgresNIO/Connection/PostgresRequest.swift:56)
[ ERROR ] NIOPostgres error: server error: invalid input syntax for type json (report_invalid_token) ["uuid": 548F078E-0996-4F1A-90D2-D656FAECA306] (Vapor/Middleware/ErrorMiddleware.swift:21)

This is because Fluent doesn't properly select the column type for a UInt8 field. You can fix this issue by manually setting the data type of the User.value field to .uint8:

struct User: Model {
    static var shared: User { User() }

    var id: Field<UUID?> = "id"
    var value: Field<UInt8> = Field("value", dataType: .uint8)
}

Once that is fixed, hit the POST /insert route again. This time you will get the insufficient data error that I mentioned before:

[ ERROR ] Error(fields: [PostgresNIO.PostgresMessage.Error.Field.line: "535", PostgresNIO.PostgresMessage.Error.Field.file: "pqformat.c", PostgresNIO.PostgresMessage.Error.Field.localizedSeverity: "ERROR", PostgresNIO.PostgresMessage.Error.Field.sqlState: "08P01", PostgresNIO.PostgresMessage.Error.Field.routine: "pq_copymsgbytes", PostgresNIO.PostgresMessage.Error.Field.message: "insufficient data left in message", PostgresNIO.PostgresMessage.Error.Field.severity: "ERROR"]) (PostgresNIO/Connection/PostgresRequest.swift:56)
[ ERROR ] NIOPostgres error: server error: insufficient data left in message (pq_copymsgbytes) ["uuid": 0E8389EE-7FD8-4A7E-95EB-24F70D6C875C] (Vapor/Middleware/ErrorMiddleware.swift:21)

Error in PostgresData.init(int)

In the PostgresData+Int.swift file I get the following errors in the initialisers (Lines 4, 8, 12, 16, and 20):

Cannot convert value of type 'Int' to expected argument type 'ByteBuffer'
Extraneous argument label 'integer:' in call

I created a vapor project with vapor new --fluent and Postgres

From my Package.swift

.package(url: "https://github.com/vapor/vapor.git", from: "4.0.0"),
.package(url: "https://github.com/vapor/fluent.git", from: "4.0.0"),
.package(url: "https://github.com/vapor/fluent-postgres-driver.git", from: "2.0.0")

Resolved Versions:

  • postgres-nio: 1.3.1
  • swift-nio: 2.14.0

Multi-dimensional arrays

I have been trying to parse a GeoJSON, which has a multidimensional array coordinates, example:

{
    "features": [
        {
            "type": "Feature",
            "geometry": {
                "type": "Polygon",
                "coordinates": [
                    [
                        [
                            14.3894460970001,
                            49.951972148
                        ],
                        [
                            14.3891214220001,
                            49.9524958600001
                        ]
                    ]
                ]
            }
        }
    ]
}

I'm having problem of saving multidimensional coordinates of type [[[Double]]] as a column into my database (which is described in my PosgtreSQLModel). In the DDL of created database, it has type jsonb[], and when I try to save it, I get PostgreSQLError.server.error.array_recv: wrong element type. It would be really great if the support for multidimensional arrays would be available.

error when saving boolean value

[ ERROR ] NIOPostgres error: server error: insufficient data left in message (pq_copymsgbytes)
[ ERROR ] Error(fields: [PostgresNIO.PostgresMessage.Error.Field.line: "535", PostgresNIO.PostgresMessage.Error.Field.severity: "ERROR", PostgresNIO.PostgresMessage.Error.Field.sqlState: "08P01", PostgresNIO.PostgresMessage.Error.Field.file: "pqformat.c", PostgresNIO.PostgresMessage.Error.Field.localizedSeverity: "ERROR", PostgresNIO.PostgresMessage.Error.Field.routine: "pq_copymsgbytes", PostgresNIO.PostgresMessage.Error.Field.message: "insufficient data left in message"])```

SimpleQuery request to 'SET TIME ZONE' generates an exception

try conn.simpleQuery("SET TIME ZONE INTERVAL '+5:45' HOUR TO MINUTE").wait()

This line generates a message with identifier 'S' and produces an exception in PostgresSimpleQuery::respond(to:). Specifically,

throw PostgresError.protocol("Unexpected message during simple query: \(message)") 

Update succeeds when no items are updated

Currently an update call can complete successfully but not update any rows in the database. While the UPDATE command executed was technically successful, there is no indication given as to how many rows were updated. This makes it difficult to tell the client wether or not the item was actually updated or not. Vapor will return a 200 status and the model that was supposedly updated (but actually wasn't).

PostgreSQL UPDATE

An UPDATE in PostgreSQL will update any rows that pass the condition. After the command has completed PostgreSQL will return the count of updated items. From the PostgreSQL docs (link):

On successful completion, an UPDATE command returns a command tag of the form

UPDATE count

If you try to perform an update where none of the rows in the table meet the condition then PostgreSQL will return UPDATE 0. According to the PostgreSQL docs, it is not an error if no rows are updated.

Proposal

I propose that postgresql should throw if the update returns with the message UPDATE 0.

Throwing seems like the best mechanism to tell the caller that no rows were updated. I can't think of a situation where it is desirable for an update call to succeed if no rows are updated. Maybe someone else can think of a situation where this would be acceptable.

Initially this would have the effect of Vapor returning a 500 error. I would be ok with that, but think it would be ideal to end up with a 404. I think that work would need to be done in Fluent to translate the "No rows updated" error into a "no row with ID" error (since the only condition Fluent adds on an update is the ID).

Sample Code

To provide a starting point, here is some example code.

postgresql/PostgreSQLConnection+Query.swift line 85

case .close(let response):
    if case let PostgreSQLMessage.CloseResponse.command(commandMessage) = response, commandMessage == "UPDATE 0" {
        // didn't update any rows
        throw PostgreSQLError(identifier: "query", reason: "No rows updated.")
    }
    break

Cannot convert Postgres' varchar[] to Swift's [String]

Given a preexisting postgresql schema with character varying[] column, converting to a [String] field in my Fluent model fails. It works with a Postgres text[] column, but with character varying[] it complains :

field: temp type: Array error: typeMismatch(Swift.Array<Swift.String>, Swift.DecodingError.Context(codingPath: [], debugDescription: "Could not convert to Array: 0x000000010000000000000413000000020000000100000001610000000162 (UNKNOWN 1015)", underlyingError: nil))

Automatic [UInt8] to bytea conversion

Currently, [UInt8] gets the default array PostgresData conformance which results in byte[]. Ideally, this should create bytea which is Postgres' preferred storage format for an array of bytes.

Warning given when saving optional containing nil field.

Trying to save a model containing field with a nil value gives Warning:

[ INFO ] GET /health ["request-id": 4F2F36E9-26A1-4F39-9F53-B0AA8757EA81] (Vapor/HTTP/ApplicationResponder.swift:44)
[ INFO ] create test fields=[id, team, name] input=[[F1AEADD2-DFE8-47C1-BAFD-8E208E983CAC, nil, "NameWorks"]] ["request-id": 4F2F36E9-26A1-4F39-9F53-B0AA8757EA81] (FluentKit/Query/QueryBuilder.swift:702)
[ DEBUG ] No available connections on this event loop, creating a new one ["request-id": 4F2F36E9-26A1-4F39-9F53-B0AA8757EA81] (AsyncKit/ConnectionPool/EventLoopConnectionPool.swift:193)
[ DEBUG ] Logging into Postgres db xyz as xyz (PostgresNIO/Connection/PostgresConnection+Authenticate.swift:42)
[ DEBUG ] INSERT INTO "test" ("id", "team", "name") VALUES ($1, $2, $3) RETURNING id as "fluentID" [F1AEADD2-DFE8-47C1-BAFD-8E208E983CAC, <null>, "SomeName"] ["request-id": 4F2F36E9-26A1-4F39-9F53-B0AA8757EA81] (PostgresNIO/Connection/PostgresClient+Query.swift:39)
[ WARNING ] bind $2 type (UNKNOWN 0) does not match expected parameter type (TEXT) ["request-id": 4F2F36E9-26A1-4F39-9F53-B0AA8757EA81] (PostgresNIO/Connection/PostgresClient+Query.swift:74)
final class SaveError: Model, Content {
    static let schema = "test"
    
    @ID(key: "id")
    var id: UUID?

    @Field(key: "name")
    var name: String
    
    @Field(key: "team")
    var team: String?

    init() {}

    init(id: UUID? = nil, name: String, team: String? = nil) {
        self.id = id
        self.name = name
        self.team = team
    }
}


struct CreateSaveError: Migration {
    func prepare(on database: Database) -> EventLoopFuture<Void> {
        return database.schema("test")
            .field("id", .uuid, .identifier(auto: false))
            .field("name", .string, .required)
            .field("team", .string)
            .create()
    }

    func revert(on database: Database) -> EventLoopFuture<Void> {
        return database.schema("test").delete()
    }
}

As @calebkleveter suggested:
line: https://github.com/vapor/postgres-nio/blob/master/Sources/PostgresNIO/Data/PostgresData%2BOptional.swift#L15
has to be:

return PostgresData(type: Wrapped.postgresDataType)

Refactor PostgresData to Swift type conversion

PostgresData currently offers "fuzzy" conversion to Swift types via its initializer and properties. For example:

let data: PostgresData = ...
print(data.string) // String?

The string property checks to see if the data is any of the Postgres data types that are convertible to a string. This includes most everything like .text, .uuid, .float8, .numeric, etc.

This fuzzy conversion gives a better developer experience but makes building type or performance sensitive code less obvious. For example, if you want to require a BIGINT value, you may use PostgresData.int64. However, BIGINT is only one of several data types that may return successfully for this property. There may be a hidden conversion between types.

To improve this, I suggest separating the PostgresData conversion API into precise and fuzzy counterparts. The precise API would replace the fuzzy API as properties on PostgresData. However, these properties would be renamed to better align with the PostgresDataType they relate to.

let data: PostgresData = ...
print(data.text) // String?
print(data.varchar) // String?
print(data.int8) // Int (or Int64 on 32-bit)
print(data.int4) // Int32 (or Int on 32-bit)
print(data.float4) // Float?
print(data.float8) // Double?

The fuzzy API would be implemented as initializers on the Swift type in question. This is similar to how ByteBuffer's new APIs work.

let data: PostgresData = ...
let string = String(postgresData: data) // String?
let int = Int(postgresData: data) // Int?
let double = Double(postgresData: data) // Double?

lazy data serialization

Currently PostgresData values are serialized to the expected data format upon initialization. For example:

import Foundation
import PostgresNIO

let date = PostgresData(date: Date())

Upon initialization, date contains TIMESTAMPTZ formatted bytes ready for querying. The Date supplied is discarded.

While this approach is simple, it has a draw back. Take the following query:

SELECT $1::JSON as "date"

If we attempted to pass the date data from above to this query, we would get a warning followed by an error:

2019-10-18T15:12:42-0400 warning: bind $1 type (TIMESTAMPTZ) does not match expected parameter type (JSON)
2019-10-18T15:12:42-0400 error: invalid byte sequence for encoding "UTF8": 0x00 (report_invalid_encoding)

It makes sense that TIMESTAMPTZ should not be convertible to JSON, however, the original Date object we had earlier does know how to convert to JSON. It can do so via Codable and the JSONEncoder.

If PostgresData could support lazy serialization of Swift types, then we could support passing a Date to this query dynamically.

LICENSE file missing

README.md has a "License: MIT" badge, but links to a non-existent LICENSE file, and there are no other mentions of the license. To add greater clarity for users, an explicit LICENSE file should be added.

Query metadata is not passed through for update/delete queries.

When I perform update/delete queries in psql console, it returns me a number of affected rows:

test=> CREATE TABLE test_table(pk int PRIMARY KEY);
CREATE TABLE
test=> INSERT INTO test_table VALUES(1);
INSERT 0 1
test=> DELETE FROM test_table;
DELETE 1
test=> DELETE FROM test_table;
DELETE 0

PostgressConnection's api is responding with a list of database rows query(_ string: String, _ binds: [PostgresData] = []) -> EventLoopFuture<[PostgresRow]> and don't have this information.

Document how to close connections when returning Futures

I've been using this package and I keep running into a confusing situation. All your examples use the async API in a sync way, by calling wait. Eg,

let db = getConnection()
defer { try! db.close.wait() }
let results = db.query(...).wait()
... etc ...

But what if I want to keep things asynchronous and write functions that return Futures? If you do that you can't close the connection when you leave the scope, like the example code above. I've tried things like:

func getStuff() -> Fut<Stuff> {
    let db = getConnection()
    let fut = db.query("select * from stuff").map { res in 
        makeStuffFromResult(res)
    }
    fut.whenComplete { let _ = db.close() }
    return fut
}

But it looks kind of ugly and I'm not sure it's correct. I think it would help if the documentation contained an explanation of how to properly do this.

Ability to use custom collections instead of Array

There are some cases where it is semantically more appropriate to use collections other than Array (e. g. Set) as column data types in code. There used to be the PostgreSQLArrayCustomConvertible protocol for that purpose, which I was using extensively.

However, in the 1.0.0 release that protocol is gone and the only way for me to use Set is to reimplement the PostgreSQLDataEncoder.encode(_:) method (since it is internal).

It would be great if the code that encodes arrays
https://github.com/vapor/postgresql/blob/3fcc6263f94e836dd9b3722a37bfc61308c0b5f0/Sources/PostgreSQL/Codable/PostgreSQLValueEncoder.swift#L30-L62
was factored out somehow so the clients could use it to encode their desired collection types.

I also noticed that I cannot use array of PostgreSQLEnums, I get these errors when trying to save a model that contains a column of type [MyEnum]:

[WARNING] [PostgreSQL] Could not determine PostgreSQL array data type: PostgreSQLDataType(primitive: PostgreSQL.PostgreSQLDataType.Primitive.custom("MYENUM"), isArray: false)
[ ERROR ] PostgreSQLError.server.error.array_recv: wrong element type (<...>/.build/checkouts/vapor.git--848376009690173418/Sources/Vapor/Logging/Logger+LogError.swift:17)

(Although I'm glad that 1.0.0 is released, it already solved so many of my problems. Thank you for your work!)

Using bool results in a crash

If I use a Bool column, the app crashes saying something about an invalid character 0x00

public let disabled = Field<Bool>("disabled")

Text format support for arrays

create a table:

CREATE TABLE media
(
    id bigserial NOT NULL PRIMARY KEY,
    tags text[]
);

now do:

let rows = try connection.simpleQuery("SELECT * FROM media WHERE id = '1'").wait()

then loop through the fields and callon the tags column:
column.array(of: String.self)

The system dies here: Array b field did not equal zero.
version: Vapor Postgres-NIO: 1.0.0-rc.1.2
file: PostgresNIO>Data>PostgresData+Array.swift
line:78:

assert(b == 0, "Array b field did not equal zero")

The actual issue:
Please implement support for arrays in simpleQuery.

Low prio requests:

  1. please rename 'b' to a more useful name
  2. please throw an error that is more clear than the b == 0 assertion.

Problem binding to an integer column

I can't seem to bind to an integer numeric type column:

let id = PostgresData(int: 5)
conn.query("SELECT id, first_name, last_name FROM normal.person WHERE id=$1", [id]).wait()
NIOPostgres error: server error: incorrect binary data format in bind parameter 1 (exec_bind_message)

I've tried a few other integer types (int16, uint16,...) but nothing works.

Swift 5.0-release, PostgreSQL 10.7, Ubuntu 18.04

Adding support for PostgreSQL v12?

Any plans supporting latest version of PostgreSQL? Currently only PostgreSQL v11 is supported. Latest v12 is supported by AWS for couple of months already/

Attempt at NUMERIC conversion leads to crash

I see NUMERIC isn't supported yet, but it's hard to tell when PostgreSQL will throw one at you without doing some digging. Trying take an average:

select avg(length(content)) as average_length from message

leads to:

Fatal error: Cannot decode Double from 0x0005000000000010006d245e0d780ac6183f (NUMERIC): file /Users/fwgreen/Documents/SwiftProjects/MessageManager/.build/checkouts/nio-postgres/Sources/NIOPostgres/Data/PostgresData+Double.swift, line 19
Illegal instruction: 4

This workaround avoids the problem:

select cast(avg(length(content)) as double precision) as average_length from message

[1] PostgreSQLMessage.ParseRequest.serialize crashes when trying to use more than 2^15-1 parameters

This calls through to func Utilities.write<T>(array: [T], closure: (inout ByteBuffer, T) -> ()), which calls write(integer: numericCast(array.count), as: Int16.self). And numericCast traps in Release builds for any values outside the representable range.

An example where this can happen is in Fluent queries with \.id ~~ <array with more than 2^15 elements>.

Suggested solution: throw an error when trying to build a query with this many elements.

Assertion failure after restarting database.

Reproduction:

  1. Start postgres (I'm using docker for this)
  2. Run serve and make a request to an endpoint that performs a query
  3. Stop postgres
  4. Perform the request again, and observe the logged error:
    [ ERROR ] connection reset (error set): Connection refused (errno: 61)
  5. Start postgres again
  6. Perform the request again, this time the process exists with: Assertion failed: PostgresConnection deinitialized before being closed.: file /Users/ian/code/streambuddy/.build/checkouts/postgres-nio/Sources/PostgresNIO/Connection/PostgresConnection.swift, line 42

Expected outcome:
serve gracefully handles the restart and makes a new connection to postgres.

Actual outcome:
serve exits.

wrong way to read utf8 bytes

func bytes(_ string: String) -> [UInt8] {
return string.withCString { ptr in
return UnsafeBufferPointer(start: ptr, count: string.count).withMemoryRebound(to: UInt8.self) { buffer in
return [UInt8](buffer)
}
}
}

The code above is very unsafe and incorrect. It reads a somewhat arbitrary number of bytes from an unsafe pointer. The problem is that string.count is not the number of bytes in the pointer that withCString returns. The correct and faster version of this code is:

    func bytes(_ string: String) -> [UInt8] {
        return Array(string.utf8)
    }

And more importantly: Please don't use anything that uses the word unsafe in code that doesn't interface with C.

Connection metadata

Currently, the FluentPostgreSQL integration needs to check PSQL version during boot time. This happens here.

This check requires a known, static database identifier to create a new connection. This makes supporting multiple Postgres databases with possibly different versions difficult / impossible.

Ideally this package would provide a better way to check active Postgres version. I believe this is sent during startup, so it could be stored and accessed on individual connections without the need for any extra queries.

cutesy name

PostgresNIO is a bit overly generic and could conflict with new modules in the future. Given Swift and SwiftPM's poor support for dealing with conflicting module name, we might want to consider giving it a less generic, aka "cutesy", name. Any ideas?

UUID decoding fails

UUID is failing during decode. In code similar to this:

struct MyObject: Content {
   let id: UUID
}
database.simpleQuery("SELECT id from MyTable").all(decoding: MyObject.self)

In MyTable id is declared this way:

id uuid NOT NULL

If id is String decode is successful

struct MyObject: Content {
   let id: String
}

It is failing in PostgresData.uuid. Not familiar with formatCode. But in case if it is text why we can't use string property of PostgresData to convert it to UUID?
Screenshot 2020-10-11 at 00 29 42

Wrong NUMERIC type parsing

  • 10000 parsed as 1
  • 0.00001 parsed as 0.1000

DB: PostgreSQL 11

Tests:

    func testNumericParsing() throws {
        let conn = try PostgresConnection.test(on: eventLoop).wait()
        defer { try! conn.close().wait() }
        let rows = try conn.query("""
        select
            '1234.5678'::numeric as a,
            '-123.456'::numeric as b,
            '123456.789123'::numeric as c,
            '3.14159265358979'::numeric as d,
            '10000'::numeric as e,
            '0.00001'::numeric as f
        """).wait()
        XCTAssertEqual(rows[0].column("a")?.string, "1234.5678")
        XCTAssertEqual(rows[0].column("b")?.string, "-123.456")
        XCTAssertEqual(rows[0].column("c")?.string, "123456.789123")
        XCTAssertEqual(rows[0].column("d")?.string, "3.14159265358979")
        XCTAssertEqual(rows[0].column("e")?.string, "10000")
        XCTAssertEqual(rows[0].column("f")?.string, "0.00001")
    }
swift test --filter testNumericParsing
...
PostgresNIOTests.testNumericParsing : XCTAssertEqual failed: ("Optional("1")") is not equal to ("Optional("10000")") -
PostgresNIOTests.testNumericParsing : XCTAssertEqual failed: ("Optional("0.1000")") is not equal to ("Optional("0.00001")") -

Consider depending on Swift Metrics 2.x (and 1.x)

Swift Metrics 2.0 just got released. It is almost completely compatible with Swift Metrics 1.x unless you exhaustively switch on the TimeUnit enum. I would highly recommend depending on swift-metrics like the following:

// swift-metrics 1.x and 2.x are almost API compatible, so most clients should use
.package(url: "https://github.com/apple/swift-metrics.git", "1.0.0" ..< "3.0.0"),

alternatively, from: "2.0.0" will also work but may in the interim cause compatibility issues with packages that specify from: "1.0.0".

If at all possible, this should be done before tagging the next release here. If using "1.0.0" ..< "3.0.0" this isn't even a SemVer major change because 1.x versions are still accepted.

Should a terminate message be sent to the server on connection close()?

Switching from libpq to postgres-nio, some of my unit tests failed because connections were still alive on the server after closing them on the client.
The connections only disappeared after closing the application.
This can be seen in pg_stat_activity
Sending a terminate message to the server in close() fixed the problem

public func close() -> EventLoopFuture<Void> {
    guard !self.didClose else {
        return self.eventLoop.makeSucceededFuture(())
    }    
    let _ = self.requestTerminate(logger: logger)
    ...
extension PostgresMessage {

    public struct Terminate: PostgresMessageType {
        
        public static var identifier: PostgresMessage.Identifier {
            return .terminate
        }
        
        public var description: String {
            return "Terminate"
        }
        
        public func serialize(into buffer: inout ByteBuffer) {
        }
    }
}

Is this a bug?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.