ELF resources about type3-runtime HOT 14 OPEN

Just popping in to reaffirm this is not a good fit for FatELF, as others pointed out.

My suspicion is that the technically correct (but not easiest) solution is to define a new ELF section. But the path of least resistance here is Haiku's approach.

--ryan.

from type3-runtime.

Currently we are using ELF sections to store various metadata, but this is not elegant. (While I am writing this I am asking myself: Why is this not elegant? Why do we need something else?)

Well, if you store metadata outside the ELF part, you gain many advantages. For instance, the entire runtime could be signed by us and this could be checked easily (i.e., without having to skip sections and other things). When adding metadata, one does not have to fiddle with these weird sections, there's no dependencies on external tools, the complexity is very low compared to what's done now. This has been discussed in other issues already

The "AppImage header" is just about AppImage-specific metadata like update information or signatures. We can investigate alternative specs as well, if they're as portable as our approach. IMO we should however just try to get a type 3 done and could later improve on this in a fourth type.

Haiku and FatELF are bad examples by the way (not sure about ELFicon): both will not run on a regular Linux system, they need kernel-side support. That's a kind of dependency we don't want at all. ELFIcon looks somewhat dead anyway.

from type3-runtime.

signed by us and this could be checked easily (i.e., without having to skip sections and other things)

Imho whenever you checksum/sign a file and then want to store the checksum/signature inside the file you have checksummed/signed, you will need to exclude the place where you store the checksum/signature from the checksumming/signature checking mechanism. But you should not skip much more than that, or else the whole effort is for nothing. No matter which format you use. Am I wong? (How is this done for deb and rpm files?) I'd be interested how ELFIcon, Haiku resources, and FatELF deal with signatures and/or checksums.

from type3-runtime.

The "AppImage header" is just about AppImage-specific metadata like update information or signatures

That's my point, we are trying to find a way to store resources, there is noting AppImage specific about this. E.g., an icon stored there has nothing AppImage specific about it. By inventing something that is too closely tied to AppImage, we would guarantee that e.g., RHEL will never automatically show icons stored at that location, because they will never do work that is "only" about AppImages. This is why I am looking for a more generic solution.

from type3-runtime.

need kernel-side support

Well, for FatELF the objective is to run ELFs stored inside those "fat" files, and I can see how the low-level OS (kernel or at least linker/loader) needs to support this. For non-executable resources like icons and update information I don't see why you'd need special kernel support. Not sure about why Haiku resources would need kernel support.

from type3-runtime.

That's my point, we are trying to find a way to store resources, there is noting AppImage specific about this.

I'm talking about metadata, not resources. They're different things. That's what I want to point out.

For non-executable resources like icons and update information I don't see why you'd need special kernel support.

If they're real specifications which modify the ELF format and don't hack that in there (like AppImage does right now), they will need some special system support. Even a modified linker is a no-go, as it breaks portability.

from type3-runtime.

I'm talking about metadata, not resources. They're different things. That's what I want to point out.

To me, metadata is a special kind of resources.

E.g., checksums, signatures, icons, translated strings,... basically everything (and more) that today is referenced in an AppStream metainfo file and in a desktop file.

from type3-runtime.

Haiku ELF resources are simply appended at the end of the file. Since ELF section header specifies a size, that extra data is simply ignored by Haiku's elf loader.

This is a bit annoying to manage, for example using strip or any other binutils based tool (objcopy, ...) will remove the resources.

We will probably change this in Haiku at some point to use a dedicated ELF section instead. That seems more elegant to me.

from type3-runtime.

Thanks @pulkomandy for chiming in here. So do you think the Haiku resource format could work on Linux?

Haiku ELF resources are simply appended at the end of the file

That sounds a lot like what @TheAssassin is proposing here:

The new AppImage file format might therefore look like this:

+------------------+
| AppImage runtime |
|   (ELF binary)   |
+------------------+
| AppImage header  |  <-- new
+------------------+
|                  |
|     payload      |
| (squashfs image) |
|                  |
.                  .
.                  .
.                  .
+------------------+

from type3-runtime.

Well, nobody'll run strip on an AppImage. This approach is used by many projects anyway. Binary packers, self-extracting archives, ...

from type3-runtime.

Indeed I had been thinking of putting the payload filesystem image into an ELF section of its own, which would make it a "really standard" ELF. Why not? As long as there is (or we can make) a tool to add the ELF section for an existing ELF. It would feel a lot "cleaner" to me. How do you normally say, what we now have is "a hack"?

Only reason I didn't do it is that I don't understand enough about ELF to have a good sense of pros and cons.

from type3-runtime.

Mentioned here, I think using a ZIP appended to the end of an ELF would be a good way to store resources inside an executable.

It can be made very generic as to not be specific to AppImage, portable and incredibly easy to implement in any language as the file could simply be opened as a zip. Icons and other resource names would have to be standardized. An example could be the zip root with a directory "icons", under it being "16.png", "48.png", "64.png" etc. Keeping all file names static eg: "desktop_entry" instead of "[APP NAME].desktop" would also simplify extraction of resources

from type3-runtime.

I haven't read this whole discussion to see if someone mentioned this, but please don't append a zip to an ELF as a standard, because then you remove the ability for anyone else to append a zip to an ELF (self-extracting archives, games built into a single binary, etc).

ELF files can have arbitrary records that existing ELF loaders can ignore, just add a new one...?

from type3-runtime.

It wouldn't remove the ability as either they could share the same zip file (store resources under a subdirectory instead of the root), or simply not use it as it would be completely optional eye candy

I think the odds of any solution here being used outside of AppImage is incredibly unlikely as there are already existing solutions that aren't being standardized. The zip route is interesting to me as it doesn't just stop at ELF files, it could potentially be a way to add resources to bytecode, scripts, etc.

from type3-runtime.