I'm working on getting ready for Xcode 15 and Swift 5.9 and seeing some warnings coming from IndexStore.swift.

Looks like these were added with this swift evolution proposal, in which they provide some #workarounds-for-common-cases

Here's the build warnings we're seeing:

INFO: From Compiling Swift module @com_github_lyft_swift_index_store//:IndexStore:
external/com_github_lyft_swift_index_store/Sources/IndexStore/IndexStore.swift:49:67: warning: forming 'UnsafeMutableRawPointer' to a variable of type '(String) -> ()'; this is likely incorrect because '(String) -> ()' may contain an object reference.
        indexstore_store_units_apply_f(self.store, /*unsorted*/0, &context) { context, unitName in
                                                                  ^
external/com_github_lyft_swift_index_store/Sources/IndexStore/IndexStore.swift:173:66: warning: forming 'UnsafeMutableRawPointer' to a variable of type '(UnitReader, (UnitDependency) -> ())'; this is likely incorrect because '(UnitReader, (UnitDependency) -> ())' may contain an object reference.
        indexstore_unit_reader_dependencies_apply_f(self.reader, &context) { context, unitDependency in
                                                                 ^
external/com_github_lyft_swift_index_store/Sources/IndexStore/IndexStore.swift:229:80: warning: forming 'UnsafeMutableRawPointer' to a variable of type '(RecordReader, (Symbol) -> ())'; this is likely incorrect because '(RecordReader, (Symbol) -> ())' may contain an object reference.
        indexstore_record_reader_symbols_apply_f(self.reader, /*nocache*/true, &context) { context, symbol in
                                                                               ^
external/com_github_lyft_swift_index_store/Sources/IndexStore/IndexStore.swift:242:67: warning: forming 'UnsafeMutableRawPointer' to a variable of type '(RecordReader, (SymbolOccurrence) -> ())'; this is likely incorrect because '(RecordReader, (SymbolOccurrence) -> ())' may contain an object reference.
        indexstore_record_reader_occurrences_apply_f(self.reader, &context) { context, occurrence in
                                                                  ^
external/com_github_lyft_swift_index_store/Sources/IndexStore/IndexStore.swift:278:66: warning: forming 'UnsafeMutableRawPointer' to a variable of type '(SymbolOccurrence, (Symbol, SymbolRoles) -> ())'; this is likely incorrect because '(SymbolOccurrence, (Symbol, SymbolRoles) -> ())' may contain an object reference.
        indexstore_occurrence_relations_apply_f(self.occurrence, &context) { context, relation in
                                                                 ^

This issue is just a place to link upstream bugs in Swift that might affect the output of this tool. Here are the ones I currently know about:

• swiftlang/swift#64594
• swiftlang/swift#54532
• swiftlang/swift#56255
• swiftlang/swift#56351
• swiftlang/swift#56356

(I haven't recently verified the older issues there, so it's possible those are fixed, if so it would be nice to verify and close them upstream)

#14 improved demangling by trying to handle extension USRs (which have an s:e: prefix). More work is required though. In some cases, the USR for an extension can be of the form s:e:s:<type USR>s:<protocol USR>, whereby multiple USRs are joined with the same s:e: prefix. I've noticed this in one of my own index stores, and confirmed this behavior here: https://github.com/apple/swift/blob/3e3a98f80191821a63e76a7d7cc4b1371ff3e493/lib/AST/USRGeneration.cpp#L343

Not sure why this is done instead of writing a USR which leverages the Extension or ExtensionDescriptor node to create a taller tree around the extension, but regardless of how the USR is generated, I don't think libswiftDemangler handles this construction.

Seems like the path forward is to check for the s:e: prefix, then scan for each s:, and demangle each. But this would also require updating func demangle(symbol: String) to return [DemangledNode]?, or inventing a new kind of DemangledNode that can be the parent of each extension USR. Could also just use Extension and let it ride.

Right now only your own imports are correctly handled. System imports bring a new set of issues. Here are a few I hit while working on this:

• You have to handle @objc (and other attributes) requiring a Foundation import
• extensions on TimeInterval don't find definition because definition is NSTimeInterval. I think the tool would need to understand apinotes to handle this case.
• Some things come from places you might not expect, and that can be ok. SwiftLint provides a way to map imports. When we were using that we used this config:

unused_import:
  require_explicit_imports: true
  always_keep_imports:
    - AVFoundation # Never translate to AVFAudio, which is only available in 14.5+
  allowed_transitive_imports:
    - module: AVFoundation
      allowed_transitive_imports:
        - AVFAudio
    - module: Foundation
      allowed_transitive_imports:
        - CoreFoundation
        - Darwin
        - ObjectiveC
    - module: CoreMedia
      allowed_transitive_imports:
        - _SwiftCoreMediaOverlayShims
    - module: GoogleMaps
      allowed_transitive_imports:
        - GoogleMapsBase
    - module: GoogleNavigation
      allowed_transitive_imports:
        - GoogleMaps
    - module: UIKit
      allowed_transitive_imports:
        - CoreFoundation
        - CoreGraphics
        - CoreText
        - Darwin
        - ObjectiveC
        - QuartzCore

But this can depend on how strict you want to be. Besides AVFoundation / AVFAudio which you can only fix if you have a high deployment target.

• Some things you might want to allow shadowing of, like GoogleMapsBase vs GoogleMaps in the example config above, where that framework is distributed as 3 separate frameworks to avoid GitHub's file size limit, but realistically you likely want users to only ever import GoogleMaps, so I think this being user configurable is required.
• There are some other cases I haven't debugged enough that cause some issues as well, but this is a sense of the work needed

There was a pitch recently on the swift forum about introduction new visibility levels on imports (ex @_implementationOnly imports).
internal and lower levels of imports are useful to reduce number of modules to be rebuild in incremental builds.
But there is no way suggested in the proposal to automate or assist with import visibility lowering.
However it was suggested that the swift index store could be used to get the information about dependencies between entities in different modules. So with the index store it's possible to build a report of connectivity level between modules, which in turn will be a valuable tool to plan work for specifying import visibilities.

I think this is our last WORKSPACE dependency, would be great to have in the registry.

With this integration:

dependencies: [
  .package(url: "https://github.com/lyft/swift-index-store.git", from: "1.2.0"),
],

The following error is produced with Swift Package Manager:

the target 'IndexStore' in product 'IndexStore' contains unsafe build flags

It's possible to work around this by integrating swift-index-store with a branch rather than a version:

dependencies: [
  .package(url: "https://github.com/lyft/swift-index-store.git", branch: "main"),
],

But then SwiftPM isn't happy if this is done in a transitive dependency.

warning from Xcode 15.0 beta-8 on macOS 13.5.1

INFO: From Compiling Swift module @com_github_lyft_swift_index_store//:IndexStore:
external/com_github_lyft_swift_index_store/Sources/IndexStore/IndexStore.swift:49:67: warning: forming 'UnsafeMutableRawPointer' to a variable of type '(String) -> ()'; this is likely incorrect because '(String) -> ()' may contain an object reference.
indexstore_store_units_apply_f(self.store, /unsorted/0, &context) { context, unitName in
^
external/com_github_lyft_swift_index_store/Sources/IndexStore/IndexStore.swift:176:66: warning: forming 'UnsafeMutableRawPointer' to a variable of type '(UnitReader, (UnitDependency) -> ())'; this is likely incorrect because '(UnitReader, (UnitDependency) -> ())' may contain an object reference.
indexstore_unit_reader_dependencies_apply_f(self.reader, &context) { context, unitDependency in
^
external/com_github_lyft_swift_index_store/Sources/IndexStore/IndexStore.swift:235:80: warning: forming 'UnsafeMutableRawPointer' to a variable of type '(RecordReader, (Symbol) -> ())'; this is likely incorrect because '(RecordReader, (Symbol) -> ())' may contain an object reference.
indexstore_record_reader_symbols_apply_f(self.reader, /nocache/true, &context) { context, symbol in
^
external/com_github_lyft_swift_index_store/Sources/IndexStore/IndexStore.swift:248:67: warning: forming 'UnsafeMutableRawPointer' to a variable of type '(RecordReader, (SymbolOccurrence) -> ())'; this is likely incorrect because '(RecordReader, (SymbolOccurrence) -> ())' may contain an object reference.
indexstore_record_reader_occurrences_apply_f(self.reader, &context) { context, occurrence in
^
external/com_github_lyft_swift_index_store/Sources/IndexStore/IndexStore.swift:284:66: warning: forming 'UnsafeMutableRawPointer' to a variable of type '(SymbolOccurrence, (Symbol, SymbolRoles) -> ())'; this is likely incorrect because '(SymbolOccurrence, (Symbol, SymbolRoles) -> ())' may contain an object reference.
indexstore_occurrence_relations_apply_f(self.occurrence, &context) { context, relation in
^

Looks like something changed in Xcode 16 beta 1 (16A5171c) causing some C++ demangle functions to be conflicting with the macOS SDK:

$ xcodebuild -version
Xcode 16.0
Build version 16A5171c
$ swift --version
swift-driver version: 1.109.2 Apple Swift version 6.0 (swiftlang-6.0.0.3.300 clang-1600.0.20.10)
Target: arm64-apple-macosx14.0
$ swift test       
Building for debugging...
error: link command failed with exit code 1 (use -v to see invocation)
ld: warning: duplicate -rpath '/Applications/Xcode-16.0.0-Beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib' ignored
Undefined symbols for architecture arm64:
  "swift::Demangle::Node::getNumChildren() const", referenced from:
      swift::Demangle::Node::hasChildren() const in CSwiftDemangle.cpp.o
      _node_getNumChildren in CSwiftDemangle.cpp.o
      swift::Demangle::Node::getChild(unsigned long) const in CSwiftDemangle.cpp.o
  "swift::Demangle::Node::begin() const", referenced from:
      swift::Demangle::Node::getChild(unsigned long) const in CSwiftDemangle.cpp.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[9/10] Linking IndexStorePackageTests
error: fatalError

This impacts both SwiftPM (swift test) and Bazel (bazel build //...).

Maybe we need to update swiftDemangleLinkerSettings in the package manifest?

A superficial look at unnecessary-testable makes it seem somewhat trivial to also potentially write an unnecessary-import tool that exposes unused imports similar to SwiftLint's UnusedImport rule. A few fixes like removing the isPublic check seemed to work on a sample project, but curious if there be dragons. Would be cool to add it into the repo