[PKG] SDL2 about prefab HOT 29 OPEN

SDL currently uses -fvisibility=hidden

-fvisibility=hidden is not sufficient because it only affects the code that you compile. Symbols provided by libc++_static.a will still be public. A version script really is the only way to do this safely. If you can't use a version script, libc++_shared.so is your best choice.

new and delete usages are part of this, FWIW. C++ allows replacing those and those behaviors do not work well if more than one library in the app defines public definitions of those (there are other quirks that happen if there are multiple private definitions, but they only appear if your API is bad), so those need to be hidden in libsdl.

I plan to provide both in one .aar package, modeled after the CMake targets provided by the the SDL2 package on desktop.
There we provide 4 targets: SDL2::SDL2 for the shared library, SDL2::SDL2-static for the static library, SDL2::SDL2main for a static library providing an entry point (=no-op library on Android) and SDL2::SDL2test (static library providing test routines).

👍

Good to know, otherwise the build matrix would be increased for no gain. Can't it be "none" for a static library, like I will do for the shared library?

No, because the library will have references to libc++ that must be resolved during the user's build. The stl field is less about what you use and more about what you require from consumers. libsdl.a will have unresolved references to libc++ functions that must be provided during the user's build, so if you claim that isn't required with none, and the user also chooses none (maybe they're a C library), the build will fail.

from prefab.

Hello, I'm currently looking into creating a sdl2 aar.
I'm using oboe's prefab_build.sh script as reference.

My goal is to add this build script to the SDL2 repo and, let the SDL2 maintainers push new SDL2 releases to some maven repo.

What I've got so far is this zip: sdl2-2.23.1.zip

I've got a few problems + questions:

• Using "Android studio Chipmuk | 2021.2.1 Patch 1", the generated SDL2Config.cmake look below.
  No SDL2::SDL2 is generated (which should be a shared imported target)

  SDL2Config.cmake generated by prefab

  if(NOT TARGET SDL2::SDL2-static)
  add_library(SDL2::SDL2-static STATIC IMPORTED)
  set_target_properties(SDL2::SDL2-static PROPERTIES
      IMPORTED_LOCATION "/home/maarten/.gradle/caches/transforms-3/74e293f9722769f6db357f1a12ffa871/transformed/sdl2-2.23.1/prefab/modules/SDL2-static/libs/android.x86_64/libSDL2_static.a"
      INTERFACE_INCLUDE_DIRECTORIES "/home/maarten/.gradle/caches/transforms-3/74e293f9722769f6db357f1a12ffa871/transformed/sdl2-2.23.1/prefab/modules/SDL2-static/include"
      INTERFACE_LINK_LIBRARIES "-ldl;-lGLESv1_CM;-lGLESv2;-llog;-landroid;-lOpenSLES;SDL2::cpufeatures"
  )
  endif()
  
  if(NOT TARGET SDL2::SDL2main)
  add_library(SDL2::SDL2main STATIC IMPORTED)
  set_target_properties(SDL2::SDL2main PROPERTIES
      IMPORTED_LOCATION "/home/maarten/.gradle/caches/transforms-3/74e293f9722769f6db357f1a12ffa871/transformed/sdl2-2.23.1/prefab/modules/SDL2main/libs/android.x86_64/libSDL2-static.a"
      INTERFACE_LINK_LIBRARIES ""
  )
  endif()
  
  if(NOT TARGET SDL2::SDL2test)
  add_library(SDL2::SDL2test STATIC IMPORTED)
  set_target_properties(SDL2::SDL2test PROPERTIES
      IMPORTED_LOCATION "/home/maarten/.gradle/caches/transforms-3/74e293f9722769f6db357f1a12ffa871/transformed/sdl2-2.23.1/prefab/modules/SDL2test/libs/android.x86_64/libSDL2test.a"
      INTERFACE_LINK_LIBRARIES ""
  )
  endif()
  
  if(NOT TARGET SDL2::cpufeatures)
  add_library(SDL2::cpufeatures STATIC IMPORTED)
  set_target_properties(SDL2::cpufeatures PROPERTIES
      IMPORTED_LOCATION "/home/maarten/.gradle/caches/transforms-3/74e293f9722769f6db357f1a12ffa871/transformed/sdl2-2.23.1/prefab/modules/cpufeatures/libs/android.x86_64/libcpufeatures.a"
      INTERFACE_LINK_LIBRARIES ""
  )
  endif()

• When using this in an Android project, source code is missing for the java sources.
  How can I include java sources such that people can peek into the SDL2's java support code?
  Right now, Android Studio decompiles the class files.

from prefab.

Hello, I'm currently looking into creating a sdl2 aar.
I'm using oboe's prefab_build.sh script as reference.

My goal is to add this build script to the SDL2 repo and, let the SDL2 maintainers push new SDL2 releases to some maven repo.

🎉

No SDL2::SDL2 is generated (which should be a shared imported target)

The abi.json for the SDL2::SDL2 module (the arm64-v8a one) is the following:

cat prefab/modules/SDL2/libs/android.arm64-v8a/abi.json
{
  "abi": "arm64-v8a",
  "api": 19,
  "ndk": 21,
  "stl": "c++_shared",
  "static": false
}

Most likely the consuming build is not compatible with that. Prefab prints messages that explains why certain modules are omitted from the generated build scripts, but Android Studio will not show you those messages. My guess though is that your externalNativeBuild is using c++_static (the default), which cannot be combined with a shared library that uses c++_shared.

If that's the case, this is all working correctly: prefab is preventing you from using a module that would introduce ODR issues in your app.

When using this in an Android project, source code is missing for the java sources.
How can I include java sources such that people can peek into the SDL2's java support code?
Right now, Android Studio decompiles the class files.

I don't believe AAR supports source distributions based on https://developer.android.com/studio/projects/android-library#aar-contents. You'd have to file an AGP bug to ask though; I have no idea how the Java side of this works.

from prefab.

Most likely the consuming build is not compatible with that. Prefab prints messages that explains why certain modules are omitted from the generated build scripts, but Android Studio will not show you those messages. My guess though is that your externalNativeBuild is using c++_static (the default), which cannot be combined with a shared library that uses c++_shared.

Yup, that was the case. The error message was burried deep in the messages emitted by gradle.

But I am wondering, SDL2 has a C-only interface (it used c++ for a single android-specific source: hid.cpp.
When I would build libSDL2.so with c++_static, could I get away with not adding the "stl" key?
Or am I seeing things wrong and does the stl also encompass C things (such as the stdio functions)?

I don't believe AAR supports source distributions based on https://developer.android.com/studio/projects/android-library#aar-contents. You'd have to file an AGP bug to ask though; I have no idea how the Java side of this works.

Thanks, I will open an issue for this.

from prefab.

But I am wondering, SDL2 has a C-only interface (it used c++ for a single android-specific source: hid.cpp.
When I would build libSDL2.so with c++_static, could I get away with not adding the "stl" key?
Or am I seeing things wrong and does the stl also encompass C things (such as the stdio functions)?

It depends. https://developer.android.com/ndk/guides/middleware-vendors is worth a read.

If you've cautiously hidden all the non-public parts of the library (the only reliable way to do this is with a version script as that page directs, with everything hidden by default and the public interface explicitly exposed), yes, you can set "stl": "none" for the shared library.

If you're also distributing a static library (it's certainly better for users to have both, but that's still on the backlog for even our official ndkports packages), that one must identify the dependency, since the consumer will need to link something. For that case the distinction between c++_shared and c++_static is actually meaningless, since the library hasn't been linked yet. The two are synonymous for static libraries, so just pick one :)

from prefab.

Thanks for the answer!

It depends. https://developer.android.com/ndk/guides/middleware-vendors is worth a read.

If you've cautiously hidden all the non-public parts of the library (the only reliable way to do this is with a version script as that page directs, with everything hidden by default and the public interface explicitly exposed), yes, you can set "stl": "none" for the shared library.

SDL currently uses -fvisibility=hidden + selective __attribute__ ((visibility("default"))).
There was discussion about introducing a version script for SDL. bit it's something that will maybe be used in SDL3.
From your answer and the docs, I conclude that because hid.cpp does not use any STL feature (only operator new and operator delete are used), "stl" for the shared library can be "none".

If you're also distributing a static library (it's certainly better for users to have both, but that's still on the backlog for even our official ndkports packages), that one must identify the dependency, since the consumer will need to link something. For that case the distinction between c++_shared and c++_static is actually meaningless, since the library hasn't been linked yet.

I plan to provide both in one .aar package, modeled after the CMake targets provided by the the SDL2 package on desktop.
There we provide 4 targets: SDL2::SDL2 for the shared library, SDL2::SDL2-static for the static library, SDL2::SDL2main for a static library providing an entry point (=no-op library on Android) and SDL2::SDL2test (static library providing test routines).

The two are synonymous for static libraries, so just pick one :)

Good to know, otherwise the build matrix would be increased for no gain. Can't it be "none" for a static library, like I will do for the shared library?

from prefab.

Thanks for the great help!

I went with your advice and now use c++_shared throughout everything.
By adding the compiled classes in classes.jar and the sources in classes-sources.jar, Android Studio is able to show the sources when clicking on an SDL class in the gui (this was one of my questions in a previous message).

Before I bring this up to SDL's project leadership, I would like to know more about how publishing works.

I suppose we need to publish a (signed) zip containing a .pom and .aar file to some maven repository.
But as a non-Java developer I don't know what maven repo to choose or how this process works.
Google supplies some libraries on its own maven repo.
Are we supposed to add SDL through this process?

sdl2-2.25.0.zip

from prefab.

The plan sounds right to me. I'm only familiar with Google's (which only googlers can publish to, so it won't work for you), so I'm not 100% sure, but yes, I think maven central is what you want. That's the repository that's available by default in new gradle projects, anyway. It's pretty trivial to use other repositories in gradle, so you could even use GitHub's hosting for example, but I don't know of any reasons to prefer that over Maven Central.

from prefab.

Hello! We're getting there.
I got a (first draft) for a build script merged and the project lead is okay with publishing Android packages.

I've got a few questions left. (I'm asking them here because I don't know where else):

• When I do a mvn gpg:sign-and-deploy-file to maven central (which fails because I don't have an account), the error message contains: Could not transfer artifact org.libsdl.android:SDL2:aar:2.25.0 from/to sdl_ossrh.
  My question is whether users will need to add the :aar to their dependencies?
  I assumed org.libsdl.android:SDL2:2.25.0 (without aar) would become the name of the SDL package.
• I also added classes-javadoc.jar because the sonatype has a hard requirement for javadoc jars. But it looks like Android Studio does not know how to handle this. Are aar archives supposed to contain javadocs, or should I just remove it? Or should I open a ticket in Android's issue tracker?

Current state: SDL2-2.25.0.zip.

from prefab.

I saw :) Glad to see they were enthusiastic about it.

Unfortunately I don't know the answer to either question. The publishing system I have to use is entirely different.

from prefab.

Hey @DanAlbert ,

I think providing SDL ourselves is a hornets nest.
Providing SDL2 is easy, but the satellite libraries (SDL2_image/SDL2_ttf/SDL2_mixer) have 3rd party dependencies.
So perhaps it's best to leave packaging SDL2 to others.

Is ndkports open to accepting extra packages? Or does it package only the "bare essential"?

from prefab.

Providing SDL2 is easy, but the satellite libraries (SDL2_image/SDL2_ttf/SDL2_mixer) have 3rd party dependencies.

Is making packages for those not viable?

Is ndkports open to accepting extra packages? Or does it package only the "bare essential"?

We'd love to, but we haven't had the bandwidth to maintain the packages we have to my standards, so not yet.

from prefab.

Is making packages for those not viable?

Well yes it is possible. We do this already in a way: SDL_mixer/SDL_ttf/SDL_image have a possibility to vendor 3rd party libraries instead of using system libraries.
It just feels like a lot of work for a single project, which can be useful for many projects.

In the past, I have contributed to ports for conan (conan center) & vcpk, and had a good experience there.

We'd love to, but we haven't had the bandwidth to maintain the packages we have to my standards, so not yet.

Indeed, any such initiative requires commitment which sucks away time and resources from other projects.

from prefab.

Vendoring those third-party libraries works up to a point. It causes conflicts whenever a library appears in the app multiple times. I'm not sure I understand what conan or vcpkg do that makes that not an issue. aiui the only difference is that those dependencies have already been published by someone else.

If vcpkg already has these dependencies solved, there's your answer. vcpkg can product AARs containing prefab packages. If those get published your SDL package can just depend on those.

from prefab.

The idea is still on the table.
SDL_image (and others) libraries already contain support to vendor external libraries, so it ended up not being so hard to re-use that.
(I was worried a bit that modelling dependencies would become hard)

from prefab.

This package had been published to my GitHub-Hosted Maven Repository: https://github.com/leleliu008/ndk-pkg-prefab-aar-maven-repo

You’re welcome to give it a try.

It was built with android-21 API level.

It provides arm64-v8a armeabi-v7a x86_64 x86 four abis.

It includes both static and shared libraries.

If it doesn't meet your needs, you could try to mannully build it with ndk-pkg, you could use it via GitHub Actions https://github.com/leleliu008/ndk-pkg-package-manually-build

from prefab.

@leleliu008 Without the SDL java classes, a SDL jni-only package is useless.

from prefab.

Uh, I usually do not use java or kotlin write codes. This is for people who use native language only. If you want jni wrapper, I'm sorry, my tool do not do that.

from prefab.

you want a aar contains both jni and prefab ? I think that need a custom processing, my tool is Unified processing for all packages, if do a custom processing, there will be a mount of work to do.

from prefab.

The CMake script of SDL3 does this.
It is very simple: just compile the java sources with the classpath of android.jar added.

Our cmake script is also capable to build a apk without gradle.
Albeit for a single arch (but that is enough for testing purposes).

from prefab.

wait, if I understand correctly, prefab is nothing to do with jni, it is only for native build, jni is not involved in native build, isn't it?

from prefab.

Prefab is just a way to describe a binary package.
Jni is a way to interface between java and native code, so it's only needed when building the SDL binary.
SDL provides java sources, which can be found in the android-project subdirectory.

from prefab.

I think your needs should ask for the SDL2 project, not ask for this project. because this project only provides a protocal for native building on Android.

from prefab.

Mybe you guys could write a tool specially designed for managing jni wrappers.

from prefab.

I'm just saying your maven sdl2 packages are incomplete, and not 100% useful for android developers: you need to provide jars alongside the aar.
libsdl-org will probably (no promises) do this starting with SDL3 (assuming we can easily automatize the release process).

from prefab.

IMO, jni and prefab should published in different AARs, because they have different purposes. pack all in one would increase the aar file size, the common case is developers only need one of them, if developers really need both of them, that's no problem.

from prefab.

Maybe someone could write a tool called jni-pkg that is used to manage jni wrappers. The first package would be SDL2.

from prefab.

IMO, jni and prefab should published in different AARs, because they have different purposes

I'm not familiar with the details of SDL, but that is the deciding factor here. junit-gtest is an example of an AAR where they should be bundled: the Java and native components are coupled and must be used together.

If there's both a JNI interface to SDL and a native interface to SDL, and an app will use one or the other but not both, yes, separate AARs are the way to go. Prefab is overkill for shipping implementation details; the only thing you need for that is the libs directory in the AAR.

from prefab.

If there's both a JNI interface to SDL and a native interface to SDL, and an app will use one or the other but not both, yes, separate AARs are the way to go. Prefab is overkill for shipping implementation details; the only thing you need for that is the libs directory in the AAR.

Yes, that's what I mean.

A traditional AAR file for jni is classes.jar + libs/<ABI>/lib*.so, now we could use another way to archive that, like what junit-gtest does, classes.jar + prefab/modules/<ABI>/lib*.a

I don't known how common this case is, if it is commonly used, I will take it into account.

from prefab.