[FR] metadata improvements about prefab HOT 15 OPEN

The first we can do (and I think we can do it without a schema rev).

The second we can't without really breaking the encapsulation here. The AAR is an implementation detail of how AGP uses Prefab. Prefab knows (and should know) nothing about them. There are other ways we can solve that issue though, so not a problem (I mentioned them on the AGP bug).

from prefab.

The AAR is an implementation detail of how AGP uses Prefab

You mean, Prefab does not have access to the AAR contents outside the prefab tree?

from prefab.

A quick test shows that you should have access to ../jar/x86/libgperf.so (if your pwd looks at prefab.json, and you or AGP put the file there).

from prefab.

Correct. The input to prefab is the prefab directory itself (the one containing the prefab.json file). We can't rely on anything outside that directory having any particular layout.

It's not all that relevant right now, but my hope is that we can extend this to other platforms. AARs are an incredibly convenient distribution method for Android, but that's about their limit. Keeping prefab packing-agnostic keeps it portable.

from prefab.

But that extension will be Android-specific, as it is inside abi.json file.

from prefab.

That's a good point. I still don't like the idea of relying on the contents outside the directory though. The current implementation of AGP would allow it, but it's very much an implementation detail.

(Probably) less importantly, I imagine not all Android users of this will necessarily want an AAR. There are certainly other options.

from prefab.

I imagine not all Android users of this will necessarily want an AAR

What do you mean by that? People who will use prefab with a build tool other than AGP?

from prefab.

I am afraid your approach is too puristic. Yes, in the ideal world nobody should care if a library distribution packages is twice as large as it could be. This is still much lower than the real capacity of the internet broadband or of the storage services, which are often available free of charge.

But same applies to AAR compression, doesn't it? Why don't we distribute libraries as uncompressed tarballs? Why don't we keep separate copies for all possible triplets in NDK distribution, but rather reuse same files, with all the headache of conditional statements?

Same applies to prefab AAR, IMO. If an AAR is 'dual purpose', i.e. both distributes the native shared libs to be included in an app, and supplies these libs for the linker to build some custom native code around them, there should be a single copy of each file in the archive.

from prefab.

I imagine not all Android users of this will necessarily want an AAR

What do you mean by that? People who will use prefab with a build tool other than AGP?

Yeah. AGP is a bad match for building things that aren't either CMake or ndk-build, which still includes a lot of third-party code. A trivial sysroot backend for prefab is something I intend to write, and is something we'll use in ndkports. An AAR is useless here.

from prefab.

#106 makes module.json optional.

from prefab.

As for the other part here, I think it might make sense to invert what you recommended. Rather than making it possible for prefab to look outside of its package, we could probably teach AGP to look for jnilibs from prefab. I don't exactly know what that would look like off the top of my head, but it would remove the duplication and keeps AGP and prefab from needing to know much about each others' implementation details.

from prefab.

A trivial sysroot backend for prefab is something I intend to write, and is something we'll use in ndkports. An AAR is useless here.

AAR in this context is but a zip filesystem. It doesn't make a difference which of the two copies of a binary is a 'real' one, and which is a link. I'd say that the way AGP expects to see the binaries (all in a single directory like jni/x86) is easier to handle than putting each one of them in their own directory. If the 'link' is not handled by zip, but rather a part of the abi.json, as I proposed above, the advantage is that generation of the whole prefab package can be done (in Android Studio) by simply adding some text files (easily backed by git) to an 'com.android.library' project. No extra scripts required to create a double-purposed AAR, that can be used either as an Android library dependency, or as a prefab dependency.

… or both, as I do in https://github.com/alexcohn/ndk-samples/tree/prefab/unified-aar/prefab/hello-libs. There is still a caveat with Android Studio 4.0: the gradle dependency must be specified as implementation files('../gen-libs/build/outputs/aar/gen-libs-debug.aar'). Unfortunately,
implementation project(':gen-libs') does not trigger prefab.

from prefab.

On the second thought, if Prefab could get involved for 'regular' library dependency, then it would generate the jniLibs folder for the dependent module, built according to all the metadata rules of prefab (e.g. matching the NDK version and STL variation).

In the long run, this would be more powerful than relying on the current jni/ directory of an AAR.

from prefab.

the advantage is that generation of the whole prefab package can be done (in Android Studio) by simply adding some text files (easily backed by git) to an 'com.android.library' project. No extra scripts required to create a double-purposed AAR, that can be used either as an Android library dependency, or as a prefab dependency.

There's no need to do any of that when Studio can build these for you anyway: https://developer.android.com/studio/preview/features#4.1-prefab-publish

On the second thought, if Prefab could get involved for 'regular' library dependency, then it would generate the jniLibs folder for the dependent module, built according to all the metadata rules of prefab (e.g. matching the NDK version and STL variation).

In the long run, this would be more powerful than relying on the current jni/ directory of an AAR.

I don't think I understand what you're describing here. It sounds like you're saying prefab should be responsible for installing all native libraries to APKs/AARs, even when the user isn't using the NDK themselves. For example, if I depend on com.foo:whatever because I use its Java API in my app, prefab should be the thing that deals with installing the JNI implementation of com.foo:whatever into my APK.

I don't think that's what you're suggesting, but maybe my misunderstanding makes it easier for you to clarify :)

from prefab.

I am rather skeptical about prefab-pubish. If a library is built within Gradle, it's probably easier to distribute it as source code. Prefab is important for tricky setups, like ffmpeg.

As for the second thought, the idea is exactly that: the com.foo.whatever.aar does not need the jni/ folder for all ABI's. Instead, Android Studio will pick the native libraries from the prefab/ folder.

from prefab.