lightningdevkit / ldk-swift Goto Github PK

Currently we implement RegtestBlockchainManager, but it will be nice to define a common interface that chain observers need.

The "Result_NoneChannelMonitorUpdateErrZ" type does not exist for the version of ldk-swift that is currently in use for swift developers

Getting a crash on this line in react-native-ldk. Seems to only happen when accidentally cooperatively closing a channel with a disconnected peer instead of force closing. Reproducible almost every time so let me know if there are any other details you need.

Adding a check in react-native-ldk to patch this so not too urgent.

example(14018,0x70000eebe000) malloc: *** error for object 0x60000012b300: pointer being freed was not allocated
example(14018,0x70000eebe000) malloc: *** set a breakpoint in malloc_error_break to debug
2022-10-05 12:19:07.023119+0200 example[14018:1418996] [javascript] [Error: Cannot begin shutdown while peer is disconnected or we're waiting on a monitor update, maybe force-close instead?]
example(14018,0x70000eebe000) malloc: *** error for object 0x60000012b300: pointer being freed was not allocated
CoreSimulator 857.7 - Device: iPhone 14 (C8BBB7A1-CC93-456D-A600-9CD38012F9FB) - Runtime: iOS 16.0 (20A360) - DeviceType: iPhone 14

Compare: lightningdevkit/ldk-garbagecollected#35

Hello,

Demo Project is not working. So I created new project and added LDK Framework using SPM. Then I added project files for Polar connection. However I am getting compilation errors. Any suggestions @danielnordh ?

I'm looking to start using LDKin a new SwiftUI app and would really appreciate a brief installation and setup guide.

I found some documentation linked from the LDK website to SwiftLightning, but not sure if that is fully applicable here, and it doesn't mention the installation.

The rn-ldk repo is a good example of a basic guide (so I can probably grok the overview from there, but a dedicated one for idk-swift would save me and others time).

I believe cargo-lipo is now deprecated, and its maintainer strongly recommends using Apple's system lipo instead: https://github.com/TimNN/cargo-lipo

On top of that, it will not be supporting Mac Catalyst, and makes it impossible for us to build libraries for Mac Catalyst targets that support x86_64 and arm64 architectures.

Example: ProbabilisticScorer::init(params: ProbabilisticScoringParameters, networkGraph: NetworkGraph, logger: Logger)

Apple is fun. See here: bitcoindevkit/bdk-ffi#466

(cc @arik-so)

My first attempt at persisting the channel manager and monitors was to try to convert it to Data:

let data = try! JSONEncoder().encode(self.channelManager)swift

But since ChannelManager does not conform to Codable, that doesn't work.

Any chance we could make that class (and ChannelMonitor) Codable?

The linker settings in the CI SPM package are always set: https://github.com/lightningdevkit/ldk-swift/blob/main/ci/LDKSwift/Package.swift#L48

However, on some environments, there is no available address sanitizer library. If there's a way to make linker flags be added conditionally, it would be extremely helpful.

Describe the bug

User cannot add ldk-swift to Xcode project.

This is happening for ldk-swift 0.0.117 and main, but not 0.0.116.

To Reproduce

Steps to reproduce the behavior in Xcode:

1. File -> Add Swift Package Dependencies
2. Enter Package URL https://github.com/lightningdevkit/ldk-swift.git
3. Dependency Rule -> Exact Version -> 0.0.117 or Dependency Rule -> Branch -> main
4. Add Package
5. See error "Invalid Manifest"
6. Add Anyway
7. See more specific errors in Navigator/IssueNavigator

Expected behavior
User can add ldk-swift to Xcode project when using version 0.0.117 or main.

Screenshots

main error

0.0.116 success

0.0.117 error

Desktop (please complete the following information):

I did all of my testing (main, 0.0.117, 0.0.116) with two separate OS and Xcode for additional sanity checking.

macOS 15

• Xcode 15.0 with iOS 17.0 minimum support

macOS 13

• Xcode 14.3.1 with iOS 16.4 minimum support

Additional context
On 10/12/23 a user ran into a similar problem and posted about it in the LDK Discord. I followed up with him in the channel that I had run into this sort of error super recently and gave him the short term fix I used to get things working.

As with Java, CMC doesn't offer the option to initialize a BackgroundProcessor if a user wishes to use RGS. This issue is already known in Java: lightningdevkit/ldk-garbagecollected#122

Would be great to have the repo easily available in the standard ways for swift developers.
Top of my list would be Swift package manager, Cocoa Pods and Carthage less important for me personally but others might like that.

If I figure out how this repo is linked to the LDK source I might try and submit a PR for this, but at the moment I haven't wrapped my head around the code to see how easy or hard this would be.

While attempting to set up LDK with the setup instructions in Xcode 13.2.1, I run into a bunch of errors and warnings.
Some easy to understand and fix, but some less so:

Errors:
override func log(record: String?) - does not override any function in super
override func persist_new_channel(id: OutPoint, data: ChannelMonitor) -> Result_NoneChannelMonitorUpdateErrZ - does not override any function in super
override func update_persisted_channel(id: OutPoint, update: ChannelMonitorUpdate, data: ChannelMonitor) -> Result_NoneChannelMonitorUpdateErrZ - does not override any function in super
let txid = outpoint.get_txid() - needs ? optional flag
let outputIndex = outpoint.get_index() - needs ? optional flag
let monitorBytes: [UInt8] = data.write(obj: data) - Argument passed to call that takes no arguments

chainMonitor = ChainMonitor.init(chain_source: filter, broadcaster: broadcaster, logger: logger, feeest: feeEstimator, persister: persister) - Cannot convert value of type 'MyFilter' to expected argument type 'Option_FilterZ'

let router = NetGraphMsgHandler(chain_access: nil, logger: logger, network_graph: networkGraph) - 'nil' is not compatible with expected argument type 'Option_AccessZ', Argument 'network_graph' must precede argument 'chain_access'

let peerManager = PeerManager(message_handler: messageHandler, our_node_secret: nodeSecret, ephemeral_random_data: peerManagerSeed, logger: logger) - Missing argument for parameter 'custom_message_handler' in call

Warnings:
print("record: \(record)") - need to account for optional
keyData.withUnsafeMutableBytes {SecRandomCopyBytes(kSecRandomDefault, 32, $0.baseAddress!)} - Result of call to 'withUnsafeMutableBytes' is unused
let chainParameters = ChainParameters(network_arg: LDKNetwork_Bitcoin, best_block_arg: bestBlock) - 'init(network_arg:best_block_arg:)' is deprecated: This method passes non-cloneable objects by owned value. Here be dragons.
keyData2.withUnsafeMutableBytes {SecRandomCopyBytes(kSecRandomDefault, 32, $0.baseAddress!)} - Result of call to 'withUnsafeMutableBytes' is unused

I found myself writing a wrapper 'LDKManager' class for all this boilerplate and it would be nice if something like that was included in the library. Requiring only a LDKManager.init() call with any relevant variables to get started.

Purpose

The current steps required to build an xcframework for use is less than ideal, for there are many manual steps involved that also means things going wrong.

The end goal is to get building ldk-swift into just a three-step process:

1. Install rust targets and necessary cargo installs
2. Init rust-lightning and ldk-c-bindings as submodules (I know @arik-so opposes to this, another approach is to have users manually git clone, but that will mean we will be very strict about where they clone to for succeeding automations to work)
3. Build the framework.

Proposed Steps

1. Scrap the current LDKFramework projects and start a new one from ground-up. We should only need one project and one target.
2. Configure the target to only build for architectures you need by reading off $PLATFORM, $ARCHS, $CONFIGURATION, envvars provided by Xcode. An example of something similar can be seen here. The goal will also be to enable "Cleaning" to actually clean intermediary files. This essentially replaces ./compile_dependency_binaries.sh.
3. Call./genbindings.sh and swift generation scripts from Build Phases in Xcode
4. Put Swift files in a package to remove the need to manually drag-and-drop the LDK folder to the project (since Swift packages capture all files within a given directory; so if we add/remove files, it will not break references)
5. Figure out how to export package symbols and move to framework for export to xcframework

Intended Outcomes

1. Reduce friction for developers to start contributing to ldk-swift
2. Reduce build times by not necessitating building a fat framework for all possible architectures we want to support
3. Open up the opportunity to improve CI to ensure builds work for iOS Simulator, macOS, and iOS, and also to run tests for greater developer confidence.
4. The README should be written against CI steps. Project should treat releases as: if it doesn't build in CI, it won't build for anyone else who clones this project according to the README!

Developer Story

Pain Points

• too many steps for developers to need to care about when wanting to build their own xcframework
• hard to get release updates without relying on git submodules and custom scripts to manage cloning the three different code bases (rust-lightning, ldk-swift, and ldk-c-bindings)
• xcframework releases has missing architectural targets (Darwin OS X, simulator arm64)
  • partially due to limitations of cargo lipo that has since been deprecated
• build tools upstream causes complexity downstream for building CI tooling, or building for use

Goals

• simplify framework builds and unit tests
  • achieved through a single xcworkspace that CI can just call to build schemes of
• deprecate use of cargo lipo -- it does not currently support same architecture (arm64) for different plaforms (iOS Sim/iOS/Mac Catalyst)
• expose a single script that a package manager like Cocoapods can call via prepare_command to build an xcframework that gets exposed using the vendored_frameworks property
  • enables further improvements through CI to build xcframework for release to be served up with SwiftPM via binaryTarget

Open Questions

• how to manage versioning of dependent libraries (specifically, rust-lightning and ldk-c-bindings?)
  • i like git submodules via tags

--

s/o to @conradev for helping with this!

ci/build_libldk.sh
ci/build_frameworks.sh
xcode/LDK/build_libldk.sh
xcode/build_xcode_frameworks.sh

Plus whatever the path to the script containing commonalities is gonna be.

No other reason than internal maintenance convenience, i. e. the ability to cancel action runs with a single button press instead of four.

Summarize the flow of how the bindings compilation process works, perhaps adding a section to the Readme.

Currently, arrays passed into native context aren't freed properly.

First of all thanks for the awesome library!

I've added universal xcframework from release page into my iOS/MacOS app but Xcode fails to build macOS target with the message:

While building for macOS, no library for this platform was found in '/User/lightningDevKit/LDK-Universal-Test/Frameworks/LDKFramework.xcframework'

If you inspect LDKFramework.xcframework there are only iOS versions

• ios-arm64
• ios-x86_64-maccatalyst
• ios-x86_64-simulator

Missed versions are (?)

• ios-arm64-simulator
• ios-amr64-maccatalist
• macos-arm64
• macos-x86_64

LDKFramework-only-macOS.xcframework has only iOS version as well

• ios-x86_64-maccatalyst

It seems like static lib wasn't build for x86_64-apple-darwin and arm-apple-darwin targets

Particularly to avoid the liability arising from lightningdevkit/ldk-c-bindings#55

The unit tests in https://github.com/lightningdevkit/ldk-swift/tree/main/ci/LDKSwift/Tests/LDKSwiftTests and https://github.com/lightningdevkit/ldk-swift/tree/main/xcode/DirectBindingsApp/DirectBindingsAppTests are virtually the same. There are some very minor changes in the first couple lines of the files, but the remainder is pretty much identical.

If there is a way to somehow link the CI tests into the Xcode tests and automatically sub the file beginnings, it would be really helpful here.

When attempting

let genesisBlock = BestBlock_from_genesis(Network.testnet)

I get

Cannot convert value of type 'Bindings.Network' to expected argument type 'LDKNetwork'

any thoughts on this @arik-so?

We should update our example to use KVStore::read for deserialization, possibly FilesystemStore directly so users are not mislead into thinking they have to implement persistence from scratch.

We may want to introduce running swiftformat/https://github.com/apple/swift-format as a final step of code generation, as well as enforcing it for all (generated and non-generated) files in CI.

Currently, ci/LDKSwift contains an LDKHeaders directory with header files. That directory is referenced by the Package.swift explicitly. It would be nice to rely on environment-variable-defined paths the way the Xcode projects do.

The parameter names are not very helpful here.
I presume they should reflect the expected types as on other platforms.

let scoreResult = ProbabilisticScorer.read(ser: <#T##[UInt8]#>, argA: <#T##Bindings.ProbabilisticScoringParameters#>, argB: <#T##Bindings.NetworkGraph#>, argC: <#T##Bindings.Logger#>)

ser - ?
argA - ProbabilisticScoringParameters
argB - NetworkGraph
argC - Logger