ximion / appstream-generator Goto Github PK

View Code? Open in Web Editor NEW

42.0 8.0 28.0 2.5 MB

A fast AppStream metadata generator

License: GNU Lesser General Public License v3.0

D 90.54% HTML 4.32% CSS 1.83% Shell 1.44% Meson 1.42% Python 0.46%

metadata appstream appstream-generator

appstream-generator's People

Contributors

Stargazers

Watchers

appstream-generator's Issues

meta-packages and missing-desktop-file

Hi,

From https://wiki.debian.org/AppStream/Guidelines

AppStream metadata is to be placed in the /usr/share/metainfo/ directory and must be placed in the package which should be installed in order to get the software described by the respective metadata. This means that you might need to move the *.metainfo.xml or *.appdata.xml to the right (meta)package.

In case your AppStream metadata is describing a desktop-application (you can tell by the XML root-node having a type="desktop-application" or type="desktop" attribute), the .appdata.xml file must be placed in the same package as the .desktop file already is, which (as above) must be the package containing the application itself.

In my case I have a freedink-engine package which is the game itself and the editor, and a freedink-dfarc package that allows launching the games with different options and managing game extensions.

I move the .appdata.xml file in the "freedink" meta-package, so users install these 2 recommended components (freedink-engine and freedink-dfarc), but now it cannot find the .desktop file(s) in the dependent packages.
https://appstream.debian.org/sid/main/issues/freedink.html

It does not make sense to move the .desktop files in the meta-package, since the packages are independent and can be installed separately - in which case they do need their .desktop files, they can't depend on the meta-package for that.

Note: I didn't add a <launchable> tag yet, I didn't know about it, but from the wiki page and the hint it doesn't sound like this would make a difference.

I thought the generator would be able to locate the desktop files in different packages (just like I expect it to find the game translations in another package).
What do you think?

(by the way, is there a way to test the distro processing of my .appdata.xml without uploading and waiting for a few days?)

Chokes on repo components that don't have any packages in them

Eg, xenial-updates/main/binary-amd64/Packages.gz doesn't have any packages in it yet.

The error is:

object.Exception@source/archive.d(84): Unable to open compressed file '.../Packages.gz'

FYI, I'm not actually running this on the real Ubuntu repo. I just gave that as an example. We have a company-specific repo that I'm trying to generate data for. I've had to remove the empty components from asgen-config.json temporarily.

Requires newer libarchive but doesn't show that prior to building

$ dub build
Performing "debug" build using gdc for x86_64.
mustache-d 0.1.1: building configuration "library"...
appstream-generator 0.4.0+commit.27.g8623924: building configuration "application"...
/tmp/ccIWdw3U.o: In function `_D2ag7archive15compressAndSaveFAhxAyaE2ag7archive11ArchiveTypeZv':
/home/me/src/git/appstream-generator/source/archive.d:424: undefined reference to `archive_write_set_format_raw'
collect2: error: ld returned 1 exit status
gdc failed with exit code 1.

Ideally it should build. Non-ideally it should at least do a version detection on libarchive and fail properly instead of randomly attempting to link stuff that might or might not be there.

cockpit (type web-application) not considered for Debian

I went through the AppStream hints of Debian's cockpit package recently, which shows some complaints:

Error: web-app-without-icon: The component is a GUI web application, but it either has no icon set in its metainfo file, or we could not find a matching icon for this application.
Warning: asv-metainfo-invalid-icon-type cockpit.appdata.xml:111 - local
Metainfo files may only contain icons of type stock or remote, the set type is not allowed.

Cockpit's AppStream data specifies type="web-application" indeed. It's a bit weird as it gets delivered/installed as a proper distribution package, but you use it through a web browser. I. e. it's a "local web application".

It also specifies

 <icon type="local" width="128" height="128">/usr/share/pixmaps/cockpit.png</icon>

which is shipped by the "cockpit" package (rpm/deb). This seems correct to me, as there is no sensible other way to deliver the icon, except for putting it onto some web server (which is a privacy issue). It's not part of the usual stock icons, so I can't use that.

This doesn't seem to be a problem in Fedora. GNOME Software shows Cockpit just fine (including icon). There it gets processed into /usr/share/app-info/xmls/fedora.xml.gz as a locally cached icon:

<icon type="cached" height="128" width="128">org.cockpit_project.cockpit.png</icon>
<icon type="cached" height="64" width="64">org.cockpit_project.cockpit.png</icon>

and shipped in /usr/share/app-info/icons/fedora/64x64/cockpit.png (same for 128).

Debian has a different approach, it doesn't ship the AppStream metadata in a package (like Fedora's appstream-data), but as part of the package manager indexes: https://appstream.debian.org/data/bullseye/main/ . I don't see cockpit at all in Components-amd64.yml.xz and the icons-*.tar.gz is also missing cockpit.png. So supposedly the generation has skipped cockpit because of that error?

How can this be fixed?

Thank you!

Doesn't build with gir-to-d 0.14

Build fails at:

[78/449] Compiling D object 'girbindings@sta/.._build_girepo_gio_DBusUtilities.d.o'.
FAILED: girbindings@sta/.._build_girepo_gio_DBusUtilities.d.o 
dmd  -Igirbindings@sta -I. -I../appstream-generator-0.7.0 -Igirepo -I../appstream-generator-0.7.0/girepo -color=on -wi -g  -of'girbindings@sta/.._build_girepo_gio_DBusUtilities.d.o' -c ../build/girepo/gio/DBusUtilities.d
../build/girepo/gio/DBusUtilities.d(297): Error: template instance `sliceNew!GValue` template `sliceNew` is not defined

Works with 0.13

Segfault when generating Arch metadata

Getting a new segfault with 0.6.5

Core was generated by `appstream-generator process arch'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f268dfed1a5 in g_string_new () from /usr/lib/libglib-2.0.so.0
[Current thread is 1 (Thread 0x7f0673206700 (LWP 6903))]
(gdb) 
(gdb) bt
#0  0x00007f268dfed1a5 in g_string_new () at /usr/lib/libglib-2.0.so.0
#1  0x00007f268d762628 in  () at /usr/lib/libappstream.so.4
#2  0x00007f268a47ee96 in  () at /usr/lib/libxml2.so.2
#3  0x00007f268a480635 in __xmlRaiseError () at /usr/lib/libxml2.so.2
#4  0x00007f268a484064 in  () at /usr/lib/libxml2.so.2
#5  0x00007f268a48aabe in xmlParseEntityRef () at /usr/lib/libxml2.so.2
#6  0x00007f268a497542 in xmlParseReference () at /usr/lib/libxml2.so.2
#7  0x00007f268a498060 in xmlParseContent () at /usr/lib/libxml2.so.2
#8  0x00007f268a498a33 in xmlParseElement () at /usr/lib/libxml2.so.2
#9  0x00007f268a49804f in xmlParseContent () at /usr/lib/libxml2.so.2
#10 0x00007f268a498a33 in xmlParseElement () at /usr/lib/libxml2.so.2
#11 0x00007f268a49913b in xmlParseDocument () at /usr/lib/libxml2.so.2
#12 0x00007f268a499597 in  () at /usr/lib/libxml2.so.2
#13 0x00007f268d762ddf in  () at /usr/lib/libappstream.so.4
#14 0x00007f268d76fc61 in  () at /usr/lib/libappstream.so.4
#15 0x00007f268d76b5a8 in as_metadata_components_to_collection () at /usr/lib/libappstream.so.4
#16 0x000000000093d8cd in appstream.Metadata.Metadata.componentsToCollection(appstream.c.types.AsFormatKind) (this=0x7f268e3588c0, format=<incomplete type>) at /build/appstream-generator/src/build/girepo/appstream/Metadata.d:168
#17 0x00000000006e9fba in asgen.datastore.DataStore.addGeneratorResult(asgen.config.DataType, asgen.result.GeneratorResult) (this=0x7f268e355800, gres=0x7f063fb32c00, dtype=<incomplete type>) at ../appstream-generator-0.6.5/src/asgen/datastore.d:343
#18 0x0000000000709d2f in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (this=0x7ffe0bbe9180, pkg=@0x7f06758377a0: 0x7f067d95b000) at ../appstream-generator-0.6.5/src/asgen/engine.d:139
#19 0x0000000000717f41 in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() (this=0x7ffe0bbe90b0) at /usr/include/dlang/dmd/std/parallelism.d-mixin-3823:3869
#20 0x00000000009b0cb4 in std.parallelism.run!(void() delegate).run(void() delegate) ()
#21 0x00000000009b0814 in std.parallelism.Task!(std.parallelism.run, void() delegate).Task.impl(void*) ()
#22 0x00000000009fc77b in std.parallelism.AbstractTask.job() ()
#23 0x00000000009af55c in std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*) ()
#24 0x00000000009af6b2 in std.parallelism.TaskPool.executeWorkLoop() ()
#25 0x00000000009af65a in std.parallelism.TaskPool.startWorkLoop() ()
---Type <return> to continue, or q <return> to quit---
#26 0x0000000000a072ae in core.thread.Thread.run() ()
#27 0x0000000000a070d1 in thread_entryPoint ()
#28 0x00007f268c18e297 in start_thread () at /usr/lib/libpthread.so.0
#29 0x00007f268b59a1ef in clone () at /usr/lib/libc.so.6
(gdb) thread apply all bt

Thread 8 (Thread 0x7f0672204700 (LWP 6906)):
#0  0x00007f268b58bfbd in read () at /usr/lib/libc.so.6
#1  0x00007f268b522598 in __GI__IO_file_underflow () at /usr/lib/libc.so.6
#2  0x00007f268b521608 in __GI__IO_file_xsgetn () at /usr/lib/libc.so.6
#3  0x00007f268b515eb1 in fread () at /usr/lib/libc.so.6
#4  0x00000000006ae9a6 in std.stdio.trustedFread!(ubyte).trustedFread(shared(core.stdc.stdio._IO_FILE)*, ubyte[]) (obj=..., f=0x7f06543cf530)
    at /usr/include/dlang/dmd/std/stdio.d:4055
#5  0x00000000006ae8e1 in std.stdio.File.rawRead!(ubyte).rawRead(ubyte[]) (this=..., buffer=...) at /usr/include/dlang/dmd/std/stdio.d:919
#6  0x00000000009b55aa in std.stdio.File.ByChunk.prime() ()
#7  0x00000000009b56dc in std.stdio.File.ByChunk.this(std.stdio.File, ubyte[]) ()
#8  0x00000000009b5616 in std.stdio.File.ByChunk.this(std.stdio.File, ulong) ()
#9  0x00000000009b58fe in std.stdio.File.byChunk(ulong) ()
#10 0x00000000009b3955 in std.process.executeImpl!(std.process.pipeProcess, const(char[])[]).executeImpl(const(char[])[], const(immutable(char)[][immutable(char)[]]), std.process.Config, ulong, const(char[])) ()
#11 0x00000000009b2fc1 in std.process.execute(const(char[][]), const(immutable(char)[][immutable(char)[]]), std.process.Config, ulong, const(char[])) ()
#12 0x00000000007c8144 in asgen.image.optimizePNG(immutable(char)[]) (fname=...) at /usr/include/dlang/dmd/std/process.d:2193
#13 0x00000000007c8961 in asgen.image.Image.savePng(immutable(char)[]) (this=0x7f0646d20740, fname=...)
    at ../appstream-generator-0.6.5/src/asgen/image.d:225
#14 0x000000000077b324 in asgen.handlers.iconhandler.IconHandler.storeIcon(appstream.Component.Component, asgen.result.GeneratorResult, immutable(char)[], asgen.backends.interfaces.Package, immutable(char)[], asgen.utils.ImageSize) (this=0x7ffe0bbe9340, size=..., iconPath=..., sourcePkg=0x7f067fc78b00, cptExportPath=..., gres=0x7f0646b14e00, cpt=0x7f0646b6d080) at ../appstream-generator-0.6.5/src/asgen/handlers/iconhandler.d:507
#15 0x000000000077bd3b in asgen.handlers.iconhandler.IconHandler.process(asgen.result.GeneratorResult, appstream.Component.Component).findAndStoreXdgIcon(asgen.backends.interfaces.Package) (this=0x7f0672202608, epkg=0x7f067fc78b00) at ../appstream-generator-0.6.5/src/asgen/handlers/iconhandler.d:591
#16 0x000000000077b845 in asgen.handlers.iconhandler.IconHandler.process(asgen.result.GeneratorResult, appstream.Component.Component) (this=0x7ffe0bbe9340, cpt=0x7f0646b6d080, gres=0x7f0646b14e00) at ../appstream-generator-0.6.5/src/asgen/handlers/iconhandler.d:633
#17 0x000000000072d972 in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=0x7ffe0bbe90e8, pkg=0x7f067fc78b00)
    at ../appstream-generator-0.6.5/src/asgen/extractor.d:267
#18 0x0000000000709cf6 in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (this=0x7ffe0bbe9180, pkg=@0x7f0675838e00: 0x7f067fc78b00) at ../appstream-generator-0.6.5/src/asgen/engine.d:136
---Type <return> to continue, or q <return> to quit---
#19 0x0000000000717f41 in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() (this=0x7ffe0bbe90b0) at /usr/include/dlang/dmd/std/parallelism.d-mixin-3823:3869
#20 0x00000000009b0cb4 in std.parallelism.run!(void() delegate).run(void() delegate) ()
#21 0x00000000009b0814 in std.parallelism.Task!(std.parallelism.run, void() delegate).Task.impl(void*) ()
#22 0x00000000009fc77b in std.parallelism.AbstractTask.job() ()
#23 0x00000000009af55c in std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*) ()
#24 0x00000000009af6b2 in std.parallelism.TaskPool.executeWorkLoop() ()
#25 0x00000000009af65a in std.parallelism.TaskPool.startWorkLoop() ()
#26 0x0000000000a072ae in core.thread.Thread.run() ()
#27 0x0000000000a070d1 in thread_entryPoint ()
#28 0x00007f268c18e297 in start_thread () at /usr/lib/libpthread.so.0
#29 0x00007f268b59a1ef in clone () at /usr/lib/libc.so.6

Thread 7 (Thread 0x7f0672a05700 (LWP 6905)):
#0  0x00007f268b58bfbd in read () at /usr/lib/libc.so.6
#1  0x00007f268b522598 in __GI__IO_file_underflow () at /usr/lib/libc.so.6
#2  0x00007f268b521608 in __GI__IO_file_xsgetn () at /usr/lib/libc.so.6
#3  0x00007f268b515eb1 in fread () at /usr/lib/libc.so.6
#4  0x00000000006ae9a6 in std.stdio.trustedFread!(ubyte).trustedFread(shared(core.stdc.stdio._IO_FILE)*, ubyte[]) (obj=..., f=0x7f0660044da0)
    at /usr/include/dlang/dmd/std/stdio.d:4055
#5  0x00000000006ae8e1 in std.stdio.File.rawRead!(ubyte).rawRead(ubyte[]) (this=..., buffer=...) at /usr/include/dlang/dmd/std/stdio.d:919
#6  0x00000000009b55aa in std.stdio.File.ByChunk.prime() ()
#7  0x00000000009b56dc in std.stdio.File.ByChunk.this(std.stdio.File, ubyte[]) ()
#8  0x00000000009b5616 in std.stdio.File.ByChunk.this(std.stdio.File, ulong) ()
#9  0x00000000009b58fe in std.stdio.File.byChunk(ulong) ()
#10 0x00000000009b3955 in std.process.executeImpl!(std.process.pipeProcess, const(char[])[]).executeImpl(const(char[])[], const(immutable(char)[][immutable(char)[]]), std.process.Config, ulong, const(char[])) ()
#11 0x00000000009b2fc1 in std.process.execute(const(char[][]), const(immutable(char)[][immutable(char)[]]), std.process.Config, ulong, const(char[])) ()
#12 0x00000000007c8144 in asgen.image.optimizePNG(immutable(char)[]) (fname=...) at /usr/include/dlang/dmd/std/process.d:2193
#13 0x00000000007c8961 in asgen.image.Image.savePng(immutable(char)[]) (this=0x7f06467eee60, fname=...)
---Type <return> to continue, or q <return> to quit---
    at ../appstream-generator-0.6.5/src/asgen/image.d:225
#14 0x000000000079910a in asgen.handlers.screenshothandler.processScreenshot(asgen.result.GeneratorResult, appstream.Component.Component, appstream.Screenshot.Screenshot, immutable(char)[], uint) (scrNo=1, mediaExportDir=..., scr=0x7f063ffd2b80, cpt=0x7f063fb20f40, gres=0x7f063f9f2980)
    at ../appstream-generator-0.6.5/src/asgen/handlers/screenshothandler.d:123
#15 0x00000000007989f1 in asgen.handlers.screenshothandler.processScreenshots(asgen.result.GeneratorResult, appstream.Component.Component, immutable(char)[]) (mediaExportDir=..., cpt=0x7f063fb20f40, gres=0x7f063f9f2980) at ../appstream-generator-0.6.5/src/asgen/handlers/screenshothandler.d:52
#16 0x000000000072d9ec in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=0x7ffe0bbe90e8, pkg=0x7f067f0f5100)
    at ../appstream-generator-0.6.5/src/asgen/extractor.d:274
#17 0x0000000000709cf6 in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (this=0x7ffe0bbe9180, pkg=@0x7f0675837f08: 0x7f067f0f5100) at ../appstream-generator-0.6.5/src/asgen/engine.d:136
#18 0x0000000000717f41 in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() (this=0x7ffe0bbe90b0) at /usr/include/dlang/dmd/std/parallelism.d-mixin-3823:3869
#19 0x00000000009b0cb4 in std.parallelism.run!(void() delegate).run(void() delegate) ()
#20 0x00000000009b0814 in std.parallelism.Task!(std.parallelism.run, void() delegate).Task.impl(void*) ()
#21 0x00000000009fc77b in std.parallelism.AbstractTask.job() ()
#22 0x00000000009af55c in std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*) ()
#23 0x00000000009af6b2 in std.parallelism.TaskPool.executeWorkLoop() ()
#24 0x00000000009af65a in std.parallelism.TaskPool.startWorkLoop() ()
#25 0x0000000000a072ae in core.thread.Thread.run() ()
#26 0x0000000000a070d1 in thread_entryPoint ()
#27 0x00007f268c18e297 in start_thread () at /usr/lib/libpthread.so.0
#28 0x00007f268b59a1ef in clone () at /usr/lib/libc.so.6

Thread 6 (Thread 0x7f268e455800 (LWP 6901)):
#0  0x00007f268b59024d in poll () at /usr/lib/libc.so.6
#1  0x00007f067041e599 in  () at /usr/lib/libcurl.so
#2  0x00007f0670418ff2 in curl_multi_wait () at /usr/lib/libcurl.so
#3  0x00007f0670411750 in curl_easy_perform () at /usr/lib/libcurl.so
#4  0x00000000009c9680 in std.net.curl.Curl.perform(std.typecons.Flag!("throwOnError").Flag) ()
#5  0x00000000009c6994 in std.net.curl.HTTP.perform(std.typecons.Flag!("throwOnError").Flag) ()
---Type <return> to continue, or q <return> to quit---
#6  0x000000000079a7e1 in std.net.curl._basicHTTP!(ubyte)._basicHTTP(const(char)[], const(void)[], std.net.curl.HTTP) (client=..., sendData=..., url=...)
    at /usr/include/dlang/dmd/std/net/curl.d:1049
#7  0x000000000079a5fc in std.net.curl.get!(std.net.curl.HTTP, ubyte).get(const(char)[], std.net.curl.HTTP) (conn=..., url=...)
    at /usr/include/dlang/dmd/std/net/curl.d:540
#8  0x0000000000798d0b in asgen.handlers.screenshothandler.processScreenshot(asgen.result.GeneratorResult, appstream.Component.Component, appstream.Screenshot.Screenshot, immutable(char)[], uint) (scrNo=1, mediaExportDir=..., scr=0x7f0646846780, cpt=0x7f0645ecba40, gres=0x7f0645c42500)
    at ../appstream-generator-0.6.5/src/asgen/handlers/screenshothandler.d:98
#9  0x00000000007989f1 in asgen.handlers.screenshothandler.processScreenshots(asgen.result.GeneratorResult, appstream.Component.Component, immutable(char)[]) (mediaExportDir=..., cpt=0x7f0645ecba40, gres=0x7f0645c42500) at ../appstream-generator-0.6.5/src/asgen/handlers/screenshothandler.d:52
#10 0x000000000072d9ec in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=0x7ffe0bbe90e8, pkg=0x7f06768aab00)
    at ../appstream-generator-0.6.5/src/asgen/extractor.d:274
#11 0x0000000000709cf6 in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (this=0x7ffe0bbe9180, pkg=@0x7f0675838900: 0x7f06768aab00) at ../appstream-generator-0.6.5/src/asgen/engine.d:136
#12 0x0000000000717f41 in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() (this=0x7ffe0bbe90b0) at /usr/include/dlang/dmd/std/parallelism.d-mixin-3823:3869
#13 0x00000000009b0cb4 in std.parallelism.run!(void() delegate).run(void() delegate) ()
#14 0x00000000009b0814 in std.parallelism.Task!(std.parallelism.run, void() delegate).Task.impl(void*) ()
#15 0x00000000009fc77b in std.parallelism.AbstractTask.job() ()
#16 0x00000000009b04ed in std.parallelism.submitAndExecute(std.parallelism.TaskPool, scope void() delegate) ()
#17 0x0000000000717e41 in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate) (this=..., dg=...) at /usr/include/dlang/dmd/std/parallelism.d-mixin-3823:3875
#18 0x0000000000709c3b in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler) (this=0x7f268e358840, iconh=0x7ffe0bbe9340, pkgs=...) at ../appstream-generator-0.6.5/src/asgen/engine.d:131
#19 0x000000000070c39d in asgen.engine.Engine.run(immutable(char)[]) (this=0x7f268e358840, suite_name=...)
    at ../appstream-generator-0.6.5/src/asgen/engine.d:517
#20 0x0000000000671eed in D main (args=...) at ../appstream-generator-0.6.5/src/app.d:119

Thread 5 (Thread 0x7f0671202700 (LWP 6908)):
#0  0x00007f2689509cc9 in  () at /usr/lib/liblzma.so.5
#1  0x00007f268950b622 in  () at /usr/lib/liblzma.so.5
---Type <return> to continue, or q <return> to quit---
#2  0x00007f2689502813 in  () at /usr/lib/liblzma.so.5
#3  0x00007f26894fd23b in  () at /usr/lib/liblzma.so.5
#4  0x00007f26894fed00 in  () at /usr/lib/liblzma.so.5
#5  0x00007f26894f67f1 in lzma_code () at /usr/lib/liblzma.so.5
#6  0x00007f268d2b6be7 in  () at /usr/lib/libarchive.so.13
#7  0x00007f268d2ab918 in __archive_read_filter_consume () at /usr/lib/libarchive.so.13
#8  0x00007f268d2d3372 in  () at /usr/lib/libarchive.so.13
#9  0x00007f268d2aaead in archive_read_data_skip () at /usr/lib/libarchive.so.13
#10 0x00000000006825c6 in asgen.zarchive.ArchiveDecompressor.readData(immutable(char)[]) (this=..., fname=...)
    at ../appstream-generator-0.6.5/src/asgen/zarchive.d:308
#11 0x0000000000883b33 in asgen.backends.archlinux.alpkg.ArchPackage.getFileData(immutable(char)[]) (this=0x7f268e3bf600, fname=...)
    at ../appstream-generator-0.6.5/src/asgen/backends/archlinux/alpkg.d:82
#12 0x000000000077abee in asgen.handlers.iconhandler.IconHandler.storeIcon(appstream.Component.Component, asgen.result.GeneratorResult, immutable(char)[], asgen.backends.interfaces.Package, immutable(char)[], asgen.utils.ImageSize) (this=0x7ffe0bbe9340, size=..., iconPath=..., sourcePkg=0x7f268e3bf600, cptExportPath=..., gres=0x7f063db12d00, cpt=0x7f063dfad380) at ../appstream-generator-0.6.5/src/asgen/handlers/iconhandler.d:462
#13 0x000000000077bd3b in asgen.handlers.iconhandler.IconHandler.process(asgen.result.GeneratorResult, appstream.Component.Component).findAndStoreXdgIcon(asgen.backends.interfaces.Package) (this=0x7f0671200608, epkg=0x0) at ../appstream-generator-0.6.5/src/asgen/handlers/iconhandler.d:591
#14 0x000000000077b870 in asgen.handlers.iconhandler.IconHandler.process(asgen.result.GeneratorResult, appstream.Component.Component) (this=0x7ffe0bbe9340, cpt=0x7f063dfad380, gres=0x7f063db12d00) at ../appstream-generator-0.6.5/src/asgen/handlers/iconhandler.d:636
#15 0x000000000072d972 in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=0x7ffe0bbe90e8, pkg=0x7f067dfc7600)
    at ../appstream-generator-0.6.5/src/asgen/extractor.d:267
#16 0x0000000000709cf6 in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (this=0x7ffe0bbe9180, pkg=@0x7f0675837dd0: 0x7f067dfc7600) at ../appstream-generator-0.6.5/src/asgen/engine.d:136
#17 0x0000000000717f41 in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() (this=0x7ffe0bbe90b0) at /usr/include/dlang/dmd/std/parallelism.d-mixin-3823:3869
#18 0x00000000009b0cb4 in std.parallelism.run!(void() delegate).run(void() delegate) ()
#19 0x00000000009b0814 in std.parallelism.Task!(std.parallelism.run, void() delegate).Task.impl(void*) ()
#20 0x00000000009fc77b in std.parallelism.AbstractTask.job() ()
#21 0x00000000009af55c in std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*) ()
#22 0x00000000009af6b2 in std.parallelism.TaskPool.executeWorkLoop() ()
---Type <return> to continue, or q <return> to quit---
#23 0x00000000009af65a in std.parallelism.TaskPool.startWorkLoop() ()
#24 0x0000000000a072ae in core.thread.Thread.run() ()
#25 0x0000000000a070d1 in thread_entryPoint ()
#26 0x00007f268c18e297 in start_thread () at /usr/lib/libpthread.so.0
#27 0x00007f268b59a1ef in clone () at /usr/lib/libc.so.6

Thread 4 (Thread 0x7f0673a07700 (LWP 6902)):
#0  0x00007f2689507d30 in  () at /usr/lib/liblzma.so.5
#1  0x00007f268950b622 in  () at /usr/lib/liblzma.so.5
#2  0x00007f2689502813 in  () at /usr/lib/liblzma.so.5
#3  0x00007f26894fd23b in  () at /usr/lib/liblzma.so.5
#4  0x00007f26894fed00 in  () at /usr/lib/liblzma.so.5
#5  0x00007f26894f67f1 in lzma_code () at /usr/lib/liblzma.so.5
#6  0x00007f268d2b6be7 in  () at /usr/lib/libarchive.so.13
#7  0x00007f268d2ab918 in __archive_read_filter_consume () at /usr/lib/libarchive.so.13
#8  0x00007f268d2d3372 in  () at /usr/lib/libarchive.so.13
#9  0x00007f268d2aaead in archive_read_data_skip () at /usr/lib/libarchive.so.13
#10 0x00000000006825c6 in asgen.zarchive.ArchiveDecompressor.readData(immutable(char)[]) (this=..., fname=...)
    at ../appstream-generator-0.6.5/src/asgen/zarchive.d:308
#11 0x0000000000883b33 in asgen.backends.archlinux.alpkg.ArchPackage.getFileData(immutable(char)[]) (this=0x7f06763c8300, fname=...)
    at ../appstream-generator-0.6.5/src/asgen/backends/archlinux/alpkg.d:82
#12 0x000000000077abee in asgen.handlers.iconhandler.IconHandler.storeIcon(appstream.Component.Component, asgen.result.GeneratorResult, immutable(char)[], asgen.backends.interfaces.Package, immutable(char)[], asgen.utils.ImageSize) (this=0x7ffe0bbe9340, size=..., iconPath=..., sourcePkg=0x7f06763c8300, cptExportPath=..., gres=0x7f063d360d00, cpt=0x7f063e581e80) at ../appstream-generator-0.6.5/src/asgen/handlers/iconhandler.d:462
#13 0x000000000077bd3b in asgen.handlers.iconhandler.IconHandler.process(asgen.result.GeneratorResult, appstream.Component.Component).findAndStoreXdgIcon(asgen.backends.interfaces.Package) (this=0x7f0673a05608, epkg=0x0) at ../appstream-generator-0.6.5/src/asgen/handlers/iconhandler.d:591
#14 0x000000000077b870 in asgen.handlers.iconhandler.IconHandler.process(asgen.result.GeneratorResult, appstream.Component.Component) (this=0x7ffe0bbe9340, cpt=0x7f063e581e80, gres=0x7f063d360d00) at ../appstream-generator-0.6.5/src/asgen/handlers/iconhandler.d:636
#15 0x000000000072d972 in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=0x7ffe0bbe90e8, pkg=0x7f06763c8300)
    at ../appstream-generator-0.6.5/src/asgen/extractor.d:267
---Type <return> to continue, or q <return> to quit---
#16 0x0000000000709cf6 in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (this=0x7ffe0bbe9180, pkg=@0x7f06758384c0: 0x7f06763c8300) at ../appstream-generator-0.6.5/src/asgen/engine.d:136
#17 0x0000000000717f41 in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() (this=0x7ffe0bbe90b0) at /usr/include/dlang/dmd/std/parallelism.d-mixin-3823:3869
#18 0x00000000009b0cb4 in std.parallelism.run!(void() delegate).run(void() delegate) ()
#19 0x00000000009b0814 in std.parallelism.Task!(std.parallelism.run, void() delegate).Task.impl(void*) ()
#20 0x00000000009fc77b in std.parallelism.AbstractTask.job() ()
#21 0x00000000009af55c in std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*) ()
#22 0x00000000009af6b2 in std.parallelism.TaskPool.executeWorkLoop() ()
#23 0x00000000009af65a in std.parallelism.TaskPool.startWorkLoop() ()
#24 0x0000000000a072ae in core.thread.Thread.run() ()
#25 0x0000000000a070d1 in thread_entryPoint ()
#26 0x00007f268c18e297 in start_thread () at /usr/lib/libpthread.so.0
#27 0x00007f268b59a1ef in clone () at /usr/lib/libc.so.6

Thread 3 (Thread 0x7f0671a03700 (LWP 6907)):
#0  0x00007f2689507871 in  () at /usr/lib/liblzma.so.5
#1  0x00007f268950b622 in  () at /usr/lib/liblzma.so.5
#2  0x00007f2689502813 in  () at /usr/lib/liblzma.so.5
#3  0x00007f26894fd23b in  () at /usr/lib/liblzma.so.5
#4  0x00007f26894fed00 in  () at /usr/lib/liblzma.so.5
#5  0x00007f26894f67f1 in lzma_code () at /usr/lib/liblzma.so.5
#6  0x00007f268d2b6be7 in  () at /usr/lib/libarchive.so.13
#7  0x00007f268d2ab918 in __archive_read_filter_consume () at /usr/lib/libarchive.so.13
#8  0x00007f268d2d3372 in  () at /usr/lib/libarchive.so.13
#9  0x00007f268d2aaead in archive_read_data_skip () at /usr/lib/libarchive.so.13
#10 0x00000000006825c6 in asgen.zarchive.ArchiveDecompressor.readData(immutable(char)[]) (this=..., fname=...)
    at ../appstream-generator-0.6.5/src/asgen/zarchive.d:308
#11 0x0000000000883b33 in asgen.backends.archlinux.alpkg.ArchPackage.getFileData(immutable(char)[]) (this=0x7f067efa0000, fname=...)
    at ../appstream-generator-0.6.5/src/asgen/backends/archlinux/alpkg.d:82
---Type <return> to continue, or q <return> to quit---
#12 0x000000000072d375 in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=0x7ffe0bbe90e8, pkg=0x7f067efa0000)
    at ../appstream-generator-0.6.5/src/asgen/extractor.d:198
#13 0x0000000000709cf6 in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (this=0x7ffe0bbe9180, pkg=@0x7f0675838318: 0x7f067efa0000) at ../appstream-generator-0.6.5/src/asgen/engine.d:136
#14 0x0000000000717f41 in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() (this=0x7ffe0bbe90b0) at /usr/include/dlang/dmd/std/parallelism.d-mixin-3823:3869
#15 0x00000000009b0cb4 in std.parallelism.run!(void() delegate).run(void() delegate) ()
#16 0x00000000009b0814 in std.parallelism.Task!(std.parallelism.run, void() delegate).Task.impl(void*) ()
#17 0x00000000009fc77b in std.parallelism.AbstractTask.job() ()
#18 0x00000000009af55c in std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*) ()
#19 0x00000000009af6b2 in std.parallelism.TaskPool.executeWorkLoop() ()
#20 0x00000000009af65a in std.parallelism.TaskPool.startWorkLoop() ()
#21 0x0000000000a072ae in core.thread.Thread.run() ()
#22 0x0000000000a070d1 in thread_entryPoint ()
#23 0x00007f268c18e297 in start_thread () at /usr/lib/libpthread.so.0
#24 0x00007f268b59a1ef in clone () at /usr/lib/libc.so.6

Thread 2 (Thread 0x7f066bfff700 (LWP 6904)):
#0  0x00007f2689507442 in  () at /usr/lib/liblzma.so.5
#1  0x00007f268950b622 in  () at /usr/lib/liblzma.so.5
#2  0x00007f2689502813 in  () at /usr/lib/liblzma.so.5
#3  0x00007f26894fd23b in  () at /usr/lib/liblzma.so.5
#4  0x00007f26894fed00 in  () at /usr/lib/liblzma.so.5
#5  0x00007f26894f67f1 in lzma_code () at /usr/lib/liblzma.so.5
#6  0x00007f268d2b6be7 in  () at /usr/lib/libarchive.so.13
#7  0x00007f268d2ab918 in __archive_read_filter_consume () at /usr/lib/libarchive.so.13
#8  0x00007f268d2d3372 in  () at /usr/lib/libarchive.so.13
#9  0x00007f268d2aaead in archive_read_data_skip () at /usr/lib/libarchive.so.13
#10 0x00000000006825c6 in asgen.zarchive.ArchiveDecompressor.readData(immutable(char)[]) (this=..., fname=...)
    at ../appstream-generator-0.6.5/src/asgen/zarchive.d:308
---Type <return> to continue, or q <return> to quit---
#11 0x0000000000883b33 in asgen.backends.archlinux.alpkg.ArchPackage.getFileData(immutable(char)[]) (this=0x7f067e49bd00, fname=...)
    at ../appstream-generator-0.6.5/src/asgen/backends/archlinux/alpkg.d:82
#12 0x000000000072d375 in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=0x7ffe0bbe90e8, pkg=0x7f067e49bd00)
    at ../appstream-generator-0.6.5/src/asgen/extractor.d:198
#13 0x0000000000709cf6 in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (this=0x7ffe0bbe9180, pkg=@0x7f0675838be8: 0x7f067e49bd00) at ../appstream-generator-0.6.5/src/asgen/engine.d:136
#14 0x0000000000717f41 in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() (this=0x7ffe0bbe90b0) at /usr/include/dlang/dmd/std/parallelism.d-mixin-3823:3869
#15 0x00000000009b0cb4 in std.parallelism.run!(void() delegate).run(void() delegate) ()
#16 0x00000000009b0814 in std.parallelism.Task!(std.parallelism.run, void() delegate).Task.impl(void*) ()
#17 0x00000000009fc77b in std.parallelism.AbstractTask.job() ()
#18 0x00000000009af55c in std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*) ()
#19 0x00000000009af6b2 in std.parallelism.TaskPool.executeWorkLoop() ()
#20 0x00000000009af65a in std.parallelism.TaskPool.startWorkLoop() ()
#21 0x0000000000a072ae in core.thread.Thread.run() ()
#22 0x0000000000a070d1 in thread_entryPoint ()
#23 0x00007f268c18e297 in start_thread () at /usr/lib/libpthread.so.0
#24 0x00007f268b59a1ef in clone () at /usr/lib/libc.so.6

Thread 1 (Thread 0x7f0673206700 (LWP 6903)):
#0  0x00007f268dfed1a5 in g_string_new () at /usr/lib/libglib-2.0.so.0
#1  0x00007f268d762628 in  () at /usr/lib/libappstream.so.4
#2  0x00007f268a47ee96 in  () at /usr/lib/libxml2.so.2
#3  0x00007f268a480635 in __xmlRaiseError () at /usr/lib/libxml2.so.2
#4  0x00007f268a484064 in  () at /usr/lib/libxml2.so.2
#5  0x00007f268a48aabe in xmlParseEntityRef () at /usr/lib/libxml2.so.2
#6  0x00007f268a497542 in xmlParseReference () at /usr/lib/libxml2.so.2
#7  0x00007f268a498060 in xmlParseContent () at /usr/lib/libxml2.so.2
#8  0x00007f268a498a33 in xmlParseElement () at /usr/lib/libxml2.so.2
#9  0x00007f268a49804f in xmlParseContent () at /usr/lib/libxml2.so.2
---Type <return> to continue, or q <return> to quit---
#10 0x00007f268a498a33 in xmlParseElement () at /usr/lib/libxml2.so.2
#11 0x00007f268a49913b in xmlParseDocument () at /usr/lib/libxml2.so.2
#12 0x00007f268a499597 in  () at /usr/lib/libxml2.so.2
#13 0x00007f268d762ddf in  () at /usr/lib/libappstream.so.4
#14 0x00007f268d76fc61 in  () at /usr/lib/libappstream.so.4
#15 0x00007f268d76b5a8 in as_metadata_components_to_collection () at /usr/lib/libappstream.so.4
#16 0x000000000093d8cd in appstream.Metadata.Metadata.componentsToCollection(appstream.c.types.AsFormatKind) (this=0x7f268e3588c0, format=<incomplete type>) at /build/appstream-generator/src/build/girepo/appstream/Metadata.d:168
#17 0x00000000006e9fba in asgen.datastore.DataStore.addGeneratorResult(asgen.config.DataType, asgen.result.GeneratorResult) (this=0x7f268e355800, gres=0x7f063fb32c00, dtype=<incomplete type>) at ../appstream-generator-0.6.5/src/asgen/datastore.d:343
#18 0x0000000000709d2f in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (this=0x7ffe0bbe9180, pkg=@0x7f06758377a0: 0x7f067d95b000) at ../appstream-generator-0.6.5/src/asgen/engine.d:139
#19 0x0000000000717f41 in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() (this=0x7ffe0bbe90b0) at /usr/include/dlang/dmd/std/parallelism.d-mixin-3823:3869
#20 0x00000000009b0cb4 in std.parallelism.run!(void() delegate).run(void() delegate) ()
#21 0x00000000009b0814 in std.parallelism.Task!(std.parallelism.run, void() delegate).Task.impl(void*) ()
#22 0x00000000009fc77b in std.parallelism.AbstractTask.job() ()
#23 0x00000000009af55c in std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*) ()
#24 0x00000000009af6b2 in std.parallelism.TaskPool.executeWorkLoop() ()
#25 0x00000000009af65a in std.parallelism.TaskPool.startWorkLoop() ()
#26 0x0000000000a072ae in core.thread.Thread.run() ()
#27 0x0000000000a070d1 in thread_entryPoint ()
#28 0x00007f268c18e297 in start_thread () at /usr/lib/libpthread.so.0
#29 0x00007f268b59a1ef in clone () at /usr/lib/libc.so.6

Updates all suites if any interesting package exists

Because there's no suite tracking, seedContentsData causes the output for all suites to be regenerated if there are any interesting packages at all, even if those haven't changed.

This is a problem because then every client will end up redownloading the output Components and icons files with every archive update even if nothing has changed. I'm a bit worried about deploying asgen with this unfixed.

Can we think of a way to avoid this happening? Like

If a suite hasn't changed since we last ran, don't update it. (How do we know? Backend specific - for Debian, store a checksum of Packages?)
Or, compare the output to the previous output by parsing it back in and then check if we have the same packages or if any of those have been reprocessed this time (for forget).
Or, do suite tracking and then we can only regenerate suites if any packages in them have changed.

Any thoughts/ideas?

Retry downloading screenshot on "temporary server issue"

Hi,

I see that one screenshot is missing at:
https://appstream.debian.org/sid/main/issues/meritous.html

screenshot-download-error
Error while downloading screenshot from 'http://asceai.net/meritous/scr/screen4_full.png': Failure when receiving data from the peer on handle 7F2E054A02D0
This might be a temporary server issue.

It's been that way for weeks now, so I believe that while this is reported as a transient error, its effect are permanent.

Intuitively I would expect appstream-generator so mark the screenshot for retry for the next cron :)

Add modalias support

In order to migrate the driver finding in Ubuntu to PackageKit, we think it makes sense to use AppStream to do that. I think what we have to do for that is

Document the modalias list for provides for the appstream yaml format
extract modalias list from the packages Modaliases field, which looks like this:
Modaliases: nvidia(pci:v000010DEd000005E7sv*sd00000595bc03sc*i*, ...)
Make software-properties query appstream for modaliases.

Does that sound about right?

Build fails with LDC

appstream-generator 0.6.0 fails to build with ldc 1.0

FAILED: asgen_test@exe/src_asgen_engine.d.o
ldc2 '-Iasgen_test@exe' '-enable-color' '-I../appstream-generator-0.6.0' '-I.' '-Isrc/' '-I../appstream-generator-0.6.0/src/' '-Isrc/girepo/' '-I../appstream-generator-0.6.0/src/girepo/' '-I/usr/include/d/mustache-d/src' '-w' '-enable-inlining=false' '-unittest' '-g' '-O0' '-I/usr/include/glib-2.0' '-I/usr/lib/glib-2.0/include' '-I/usr/include/AppStream' '-I/usr/include/cairo' '-I/usr/include/pixman-1' '-I/usr/include/freetype2' '-I/usr/include/libpng16' '-I/usr/include/harfbuzz' '-I/usr/include/libdrm' '-I/usr/include/gdk-pixbuf-2.0' '-I/usr/include/librsvg-2.0' '-I/usr/include/pango-1.0' -of 'asgen_test@exe/src_asgen_engine.d.o' -c ../appstream-generator-0.6.0/src/asgen/engine.d
../appstream-generator-0.6.0/src/asgen/engine.d(560): Error: no property 'clear' for type 'string[string]'
../appstream-generator-0.6.0/src/asgen/engine.d(561): Error: no property 'clear' for type 'ulong[string]'

Build fails with DMD 2.076

../appstream-generator/src/asgen/backends/archlinux/alpkgindex.d(82): Error: foreach: cannot make entry ref

changing 'foreach (ref entry' to 'foreach (entry' fixes the problem, but I don't know if it has any side effects.

FTR, phobos commit that breaks it is dlang/phobos@5983bcc8

Can not add component without ID to results set

object.Exception@../src/asgen/result.d(130): Can not add component from 'gthumb/3:3.5.1-3ubuntu1/armhf' without ID to results set: [desktop-application::unknown/os/package/(null)]> name: gThumb | summary: Image viewer | package: <none>

I got called by our automated monitoring the middle of the night about this! :)

Not sure what it's trying to tell me?

Font handler should pick up embedded URLs

Ideally, a human being would make sure that a font package has the right homepage URL in its metainfo.xml file.

Failing that, however, the font handler can look for the "Vendor URL" entry in the fon't name table. That's the nameID 11 field.

It certainly isn't guaranteed to be specific to one font, but it is better than no URL.

( detail at https://docs.microsoft.com/en-gb/typography/opentype/spec/name#name-ids )

"Addon" archive support

For things like PPAs, (eventually) bikesheds, ISVs and so on - we should support archives which are "on top" of suites we already know about.

I think the problem here is mainly about stock icons?

The workflow should make it easy for both the central instance and random external instances to operate in this mode - the former for PPAs and the latter for ISVs. Maybe this is the same though - acquire the preprocessed data either by downloading it or having it on the filesystem, generate a configuration, and run.

Uses the oldest .deb in archive instead of the newest

In KDE neon we are running appstream-generator over our archive which includes many old builds. Appstream-generator seems to take the oldest build in the archive and ignore newer ones. This means fixes to the appstream files don't get used.
http://build.neon.kde.org/view/mgmt/job/mgmt_appstream-generator/97/console
18:59:05 2016-11-02 18:58:51 - INFO: Processed minuet/16.04.3-0neon+16.04+build8/amd64, components: 0, hints: 1
But minuet is on 16.08.2-0neon+16.04+build17 which includes many fixes in the appstream file.

Doesn't export uncompressed files but apt requires them

I had trouble figuring out why apt-get update wasn't fetching any of the dep11 files from my repo, even though the files were mentioned in the Release file. Eventually I discovered that it looks for the uncompressed Components and icons files, and then downloads the compressed ones. (It actually downloads .xz, then recompresses as .gz.) So currently I'm having to generate the uncompressed files in a separate step after running asgen.

This is more of a bug in apt than in asgen but since apt is out there like this in Xenial right now, it's probably a good idea to match what it expects.

appstream.debian.org claims that scdaemon metainfo file is GPL-licensed, but it is CC0-1.0

https://appstream.debian.org/sid/main/issues/scdaemon.html says:

metainfo-license-invalid

But, debian/copyright for the source package says:

Files: debian/org.gnupg.scdaemon.metainfo.xml
Copyright: 2017 Daniel Kahn Gillmor <[email protected]>
Comment: This file is licensed permissively for the sake of AppStream
License: CC0-1.0

perhaps this is a mis-parsing of the debian/copyright file? or something else?

Can't provide updates to frozen suites

For Ubuntu we would like to update from the old appstream-dep11 to asgen in Xenial and Yakkety, which released with data from -dep11.

I thought we could deliver the updates via the -updates suite, as another Components and icons file: Components-fixup or so. The component would have a dataPriority higher than the release but lower than proposed and `updates.

Then the config would have something like

"Suites":
  {
    "yakkety":
      {
        "redirect-to": "yakkety-updates",
        "useIconTheme": "Humanity",
        "sections": ["main"],
        "architectures": ["amd64"]
      }
  }

redirect-to means that the output is placed in yakkety-updates/Component-fixup.tar.xz.

Questions

How does this interact with immutable? I think redirect-to probably sort of implies immutable. But what if it is added later on after the suite has already been immutable (and the data has been dropped)?
What about dropping data - how and when? We need to keep the data for the release so that people who don't have updates aren't broken.

not correctly handling http status

Downloading from debian repos without Packages.xz file fails silently because the status line isn't checked (i.e. the current code ignores 404 or any other fatal code entirely)

debpkgindex.d calls downloadIfNecessary for remote roots. downloadIfNecessary iters xz,gz,bz2 to find a suitable Package.* file, it calls utils.d's downloadFile which in turn simply wraps download which is where the trouble is.

download calls HTTP.perform from std.net.curl, perform however seems to not care about the status code. From the documentation, and a casual look at curl.d, I'd say that when using perform one needs to set callback handlers for everything manually. e.g. https://github.com/dlang/phobos/blob/master/std/net/curl.d#L1075 is where .get and friends get their exception thrown, perform has no such convenience magic, and the docs for .get explicitly state that a statusexception may be raised.

Some printfdebugging seems to support that theory. Setting a status line callback in download

            downloader.onReceiveStatusLine = (HTTP.StatusLine status) { logDebug ("status %s", status); };

it definitely gets code 404 but ignores it.

2018-05-15 08:29:44 - DEBUG: Downloading https://origin.archive.neon.kde.org/dev/unstable/dists/xenial/main/binary-amd64/Packages.xz
2018-05-15 08:29:44 - DEBUG: status 404 Not Found (1.1)
2018-05-15 08:29:44 - DEBUG: Downloaded https://origin.archive.neon.kde.org/dev/unstable/dists/xenial/main/binary-amd64/Packages.xz

To that end I think all that's needed to fix this is to enforce the status code, as seen in curl.d's _basicHTTP.

Config I am using:

{
"ArchiveRoot": "https://origin.archive.neon.kde.org/dev/unstable/",
"MediaBaseUrl": "https://metadata.neon.kde.org/appstream/dev/unstable/media",
"HtmlBaseUrl": "https://metadata.neon.kde.org/appstream/dev/unstable/html",
"Backend": "debian",
"Features":
  {
    "validateMetainfo": true
  },
"Suites":
  {
    "xenial":
      {
        "sections": ["main"],
        "architectures": ["amd64"],
        "useIconTheme": "breeze",
        "dataPriority": 1
      }
  }
}

unit test failure with v0.8.1 on Fedora due to missing ffprobe

When trying to update appstream-generator to v0.8.1, I got this failure when running the tests:

TEST: RpmMDPackageIndex
--- stderr ---
2020-03-19T19:25:08.683 [info] hashmap.d:995:__unittest_L991_C7 Testing hash tables
core.exception.AssertError@../src/asgen/handlers/screenshothandler.d(452): Assertion failure
----------------
??:? _d_assert [0x7f7ca793c4dc]
screenshothandler.d:452 void asgen.handlers.screenshothandler.__unittest_L436_C1() [0x752a32]
??:? int core.runtime.runModuleUnitTests().__foreachbody2(object.ModuleInfo*) [0x7f7ca7944be8]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))).__foreachbody2(ref rt.sections_elf_shared.DSO) [0x7f7ca7971e3a]
??:? int rt.sections_elf_shared.DSO.opApply(scope int delegate(ref rt.sections_elf_shared.DSO)) [0x7f7ca7973128]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))) [0x7f7ca7971dcb]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)) [0x7f7ca795dc0e]
??:? runModuleUnitTests [0x7f7ca7944a94]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x7f7ca7968e8a]
??:? _d_run_main2 [0x7f7ca7968cce]
??:? _d_run_main [0x7f7ca7968b3d]
entrypoint.d:35 main [0x63fbfc]
??:? __libc_start_main [0x7f7ca76b3041]
??:? _start [0x63fa7d]
1/26 unittests FAILED
-------

The full test log file from Meson is here: appstream-generator-0.8.1-meson-testlog.txt

media is always put in `media` directory

I want to override the MediaBaseUrl to something else so that I can have appstream-dep11's media under /media. The problem is that asgen always puts the media in a media directory on the filesystem, so you can't just expose the export directory, you need to transform it to match the MediaBaseUrl by renaming.

Maybe a MediaDirectory configuration option that is used for the output directory and also appended to the MediaBaseUrl in the output, if it is present or something?

I think in reality I'll just keep using media because ImmutableSuites means there's not a conflict between media/main (as-dep11) and media/suite/main (asgen), so this isn't that urgent for me.

i686 build failure

0.5.0 doesn't build on 32 bits with gdc:

appstream-generator ~master: building configuration "application"...

source/utils.d:373:13: error: function std.net.curl.HTTP.Protocol!().onReceive (uint delegate(ubyte[]) callback) is not callable using argument types (void)
             downloader.onReceive = (data) => onReceiveCb (dest, data);
             ^
source/utils.d:379:13: error: function std.net.curl.FTP.Protocol!().onReceive (uint delegate(ubyte[]) callback) is not callable using argument types (void)
             downloader.onReceive = (data) => onReceiveCb (dest, data);
             ^
gdc failed with exit code 1.
make: *** [Makefile:3: all] Error 2

0.6.5 generates truncated metadata for some packages

With 0.6.5 metadata is incorrectly extracted for some packages, giving a broken xml that is not validated by appstream

Fonts don't appear in the appstream db

From ximion/appstream#38

Split processing into separate scan and generate operations

I understand that speed and cost are an issue for generating metadata. How would it be if you only looked at newly-added packages? You already have data about the existing packages cached in the database, so they don't all need to be rescanned. The job that processes incoming packages could tell asgen which packages to consider by passing it the .changes files to look at. That would avoid the expensive loading of indices. (This would be a good option in my infrastructure, where we could call asgen from the cron job that scans the incoming directories for changes.)

It would still be necessary to regenerate the whole of the Components and icons files, but this would be done from cached data in the database and would be considerably less expensive than reading the indices since there are far fewer apps than packages. Alternatively, it could be done by a cron job, but that could be run more frequently since this part of the operation is less costly.

I propose that the current process operation be split into two separate operations, scan and generate. The scan operation would take a list of .changes or .deb files to look at. There would still be a process operation to scan an entire suite, but it would use these two other operations internally.

font handler should prefer embedded language information if available

There are two options for language detection that are not currently exploited by the font handler.

The first is the "design languages" and "supported languages" fields in the OpenType/TrueType meta table. The table entries are dlng and slng, respectively, and are references to private entries in the font's general name table.

A "design language" would take precedence over a "supported language" if the two language-lists aren't identical; the theory being that "design language" might indicate special care taken for a particular language.

Example: Vietnamese often has to stack multiple diacritics and tone marks, so a font designed for Vietnamese might use slightly smaller-x-height lowercase letters and have diacritics ensured to work ... whereas a font designed for English might "support" Vietnamese, but look bad because (without that requirement in mind) its stacks of diacritics get too tall and are often cut off at the top.

( The table is detailed here: https://docs.microsoft.com/en-gb/typography/opentype/spec/meta )

These entries aren't automatically generated, so if they're present in the binary they ought to be given considerable weight.

The other is more of a fallback, but the list of languages in the font's GSUB table ought to be a practical list of the supported languages.

( https://docs.microsoft.com/en-gb/typography/opentype/spec/chapter2#slTbl_sRec )

0.6.4 build fails at D interface generation

Meson encountered an error in file meson.build, line 129, column 4:
Error encountered: Unable to build D intefaces from GIR:
Warning /build/appstream-generator/src/appstream-generator-0.6.4/contrib/girwrap/APILookup.txt(118) : Don't use inputRoot, it has been removed as it was never implemented.
Warning /build/appstream-generator/src/appstream-generator-0.6.4/contrib/girwrap/APILookup.txt(120) : Don't use bindDir, it is no longer used since the c definitions have moved.
Warning /build/appstream-generator/src/appstream-generator-0.6.4/contrib/girwrap/APILookupAppStream.txt(23): Don't use absolute paths for specifying gir files.
Warning /build/appstream-generator/src/appstream-generator-0.6.4/contrib/girwrap/APILookupGLib.txt(23): Don't use absolute paths for specifying gir files.
Warning /build/appstream-generator/src/appstream-generator-0.6.4/contrib/girwrap/APILookupGLib.txt(24): Don't use absolute paths for specifying gir files.
Warning /build/appstream-generator/src/appstream-generator-0.6.4/contrib/girwrap/APILookupGio.txt(23): Don't use absolute paths for specifying gir files.
core.exception.AssertError@/usr/include/dlang/ldc/std/regex/package.d(563): wrong match: 17..0
----------------
??:? _d_assert_msg [0x121a3021]
??:? [0x49b822]
??:? [0x49ac91]
??:? [0x498cb9]
??:? [0x49a504]
??:? [0x45ce9a]
??:? _aaApply [0x121c6592]
??:? [0x45c76c]
??:? _D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv [0x121d470e]
??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0x121d4653]
??:? _d_run_main [0x121d457f]
??:? [0x4671f4]
??:? __libc_start_main [0x11d99510]
??:? [0x45c279]

this is with lcd 1.2, but same issue when using dmd (for both girtod ans asgen)

Should loop through all configured suites when one isn't specified

The --help option says:

Usage:
  appstream-generator <subcommand> [OPTION...] - AppStream Generator.

AppStream Metadata Generator

Subcommands:
  process [SUITE]      - Process new metadata for the given distribution suite.
  cleanup              - Cleanup old metadata and media files.
  remove-found [SUITE] - Drop all valid processed metadata and hints.

Help Options:
  -h, --help       Show help options

Application Options:
  --version        Show the program version
  --verbose        Show extra debugging information
  -w|--workspace   Define the workspace location

So [SUITE] is shown as being optional for process and remove-found. However, if SUITE is omitted, process and remove-found say:

Invalid number of parameters: You need to specify a suite name.

Unfindable package after bad packaging

So I had a package in the repository that had no DEBIAN/md5sum lines. When I would run process --force --verbose the package would be found, but it had no interesting files and would be skipped.

I fixed the package, reuploaded it to the repository, verified it had the DEBIAN/md5sum lines, and was included in the repository packages file. But after forget-found --force and cleanup, appstream-generator doesn't seem to find the package. There is no mention of it when running process --force --verbose even though all the other packages are found.

Appstream html
Repository Url
Actual Package link

Using v0.6.5 built from source
Appstream version 0.11.2 also built from source
Ubuntu xenial 16.04

Will post appstream-generator output tonight.

warning on https->http redirect of screenshot downloads?

Screenshots that had https as url currently can get redirected to http without any warnings being raised as it seems curl's get! simply follows https->http.

As this degrades authenticity of the data it would be prudent to at least raise a warning about this and have authors either use http on the url to begin with or ensure the redirect chain does not drop to http. A MITM attack could inject unwanted art into the final data set, making both the distributor and the software look bad.

Move the graph key to the top left

Presumably we won't have the number of appstream files decreasing in a normal case, so the place where this key is least likely to cover important information is the top left instead of the top right

Where it is

Where I would like it

Can't generate metadata from GNUstep apps

Ubuntu's AppStream issues list is generated by appstream-generator 0.6.6:
http://appstream.ubuntu.com/bionic/universe/issues/index.html

36 of the 37 GNUstep apps on the list (search for ".app") have errors, and 32 of those errors are 'gui-app-without-icon'.

Some examples:
http://appstream.ubuntu.com/bionic/universe/issues/gworkspace.app.html
http://appstream.ubuntu.com/bionic/universe/issues/helpviewer.app.html
http://appstream.ubuntu.com/bionic/universe/issues/lynkeos.app.html
http://appstream.ubuntu.com/bionic/universe/issues/mpdcon.app.html
http://appstream.ubuntu.com/bionic/universe/issues/pikopixel.app.html
(I'm the upstream developer of PikoPixel.app.)

TextEdit.app is the only GNUstep app that successfully generates metadata:
http://appstream.ubuntu.com/bionic/universe/metainfo/textedit.app.html

As a bit of background, GNUstep apps generally leave their icons within in the app's own directory instead of moving them to /usr/share/pixmaps/ or /usr/share/icons/:

PikoPixel.desktop's Icon entry:
Icon=/usr/lib/GNUstep/Applications/PikoPixel.app/Resources/GNUstepAppIcon.png

TextEdit.desktop's Icon entry:
Icon=/usr/share/GNUstep/TextEdit.app/accessories-text-editor_128x128x32.png

From a quick scan of the appstream-generator sources, it looks like the generator searches only the package contents for icon filepaths (without checking the .desktop file), and if no filepaths matching "/usr/share/icons/" or "/usr/share/pixmaps/" are found, it returns a 'gui-app-without-icon' error:

appstream-generator/src/asgen/contentsstore.d

Lines 210 to 217 in 9776d8b

 // filter out icon filenames and filenames of icon-related stuff (e.g. theme.index) 

 auto iconInfo = appender!(string[]); 

 foreach (ref c; contents) { 

 if ((c.startsWith ("/usr/share/icons/")) || 

 (c.startsWith ("/usr/share/pixmaps/"))) { 

 iconInfo ~= c; 

 } 

 }

If this is what's causing the issue (though it doesn't explain why TextEdit.app is successfully parsed), my suggestion would be to also check the .desktop file for the icon filepath before returning the error.

Base suite packages are scanned every time if the base suite isn't itself in the config

After I fixed #31, I noticed that every run scans all packages in xenial:

2016-11-15 10:30:26 - INFO: Scanning new packages for xenial-proposed/main [amd64]
2016-11-15 10:30:35 - INFO: Scanned xchat-gnome/1:0.30.0~git20141005.816798-0ubuntu9/amd64, could be interesting.
2016-11-15 10:30:35 - INFO: Scanned gnome-session-common/3.18.1.2-1ubuntu1/amd64, could be interesting.
2016-11-15 10:30:35 - INFO: Scanned transmission-gtk/2.84-3ubuntu3/amd64, could be interesting.
2016-11-15 10:30:35 - INFO: Scanned gnome-system-monitor/3.18.2-1/amd64, could be interesting.

and so on for all packages in xenial.

I think it's because here we would normally skip the package if it's been seen before, but we don't because it's not in dstore. AFAICS that is because only the packages for the actual suite are processed.

Is that the bug? Do we need to process base suite packages? What do we need base suite packages for - just icons? If so, do they need to be in seedContentsData at all?

Cleanup doesn't clean up the export directory

I just dropped powerpc from the zesty config, because it's deleted in the archive. The output files are still there, just not updated any more.

I wonder if asgen could take care of cleaning up the output directory? That is - delete everything in there that isn't either currently being updated or immutable.

Good idea or bad?

Add support for 48px icons

appstream-generator/src/asgen/handlers/iconhandler.d

Line 55 in 91c04ab

 public immutable wantedIconSizes = [ImageSize (64), ImageSize (128), ImageSize (64, 64, 2), ImageSize (128, 128, 2)]; 

AppCenter uses 48px icons in list views. Since this is a standard Gtk.IconSize it would be great to have it supported :)

font: wrong language is used for sample text

I found with Korean fonts, "bg" seems to be chosen as the language for sample text. I think it's because "bg" is alphabetically first in the supported language list.

$ fc-query /usr/share/fonts/truetype/baekmuk/batang.ttf |grep lang
	familylang: "en"(s)
	stylelang: "en"(s)
	fullnamelang: "en"(s)
	lang: bg|fj|ho|ia|ie|io|ko|kum|nr|om|os|ru|sel|so|ss|st|sw|ts|uz|xh|zu|kj|kwm|lg|ms|ng|rn|rw|sn|za(s)
$

Fonts don't seem to have information to finding out their representitive language. So how about just putting language for sample text in metainfo file?

font handler should prefer embedded sample string if available

It's good that the font handler tries to be smart about selecting the sample-text string that will be shown to users.

There is one option it is not currently making use of, however: in OpenType and TrueType fonts, the name table includes a field for sample strings at nameID 19. If this nameID is present in the font, it means that the designer or publisher has selected the sample string that (hopefully) best shows the relevant characteristics of the font.

It could be identical to a standard pangram, but then again it could be different. So, if present in the binary, it is preferable to an arbitrary string choice.

(The table is detailed here: https://docs.microsoft.com/en-gb/typography/opentype/spec/name#name-ids )

Export data using the same directory hierarchy as the repo

The repo expects data to be in a dep11 subdirectory but asgen puts it directly under the directory for the distribution. This stops me from rsyncing the data directly into the dists directory of the repo.

The following change fixes this:

diff --git a/source/engine.d b/source/engine.d
index ff3837c..f9a960f 100644
--- a/source/engine.d
+++ b/source/engine.d
@@ -216,7 +216,7 @@ public:
         mdataEntries ~= getMetadataHead (suite, section);

         // prepare destination
-        auto dataExportDir = buildPath (exportDir, "data", suite.name, section);
+        auto dataExportDir = buildPath (exportDir, "data", suite.name, section, "dep11");
         auto hintsExportDir = buildPath (exportDir, "hints", suite.name, section);

         mkdirRecurse (dataExportDir);

Doesn't build with current dmd or ldc due to stream.d usage

stream.d has been removed from phobos, so asgen can't be built with the latest releases of dmd (2.074) or ldc (1.2.0)

FAILED: appstream-generator@exe/src_asgen_result.d.o
ldc2 '-Iappstream-generator@exe' '-I.' '-I../appstream-generator-0.6.3' '-I../appstream-generator-0.6.3/src/' '-I../appstream-generator-0.6.3/src/girepo/' '-I/usr/include/d/mustache-d' '-I/usr/include/glib-2.0' '-I/usr/lib/glib-2.0/include' '-I/usr/include/AppStream' '-I/usr/include/cairo' '-I/usr/include/pixman-1' '-I/usr/include/freetype2' '-I/usr/include/harfbuzz' '-I/usr/include/libdrm' '-I/usr/include/libpng16' '-I/usr/include/gdk-pixbuf-2.0' '-I/usr/include/librsvg-2.0' '-I/usr/include/pango-1.0' '-enable-color' '-wi' '-g' '-O0' '-enable-cross-module-inlining=false' -of 'appstream-generator@exe/src_asgen_result.d.o' -c ../appstream-generator-0.6.3/src/asgen/result.d
../appstream-generator-0.6.3/src/asgen/result.d(207): Error: module stream is in file 'std/stream.d' which cannot be read

Ignore Hidden=true .desktop files

appstream-generator ignores NoDisplay=true files but it should probably also ignore Hidden=true files which hides it from application menus. If it's not in an app menu it's unlikely to be interesting to software managers.

e.g. https://cgit.kde.org/kaddressbook.git/tree/src/data/kaddressbook-importer.desktop?id=4577f0e7e2ab374b592b39a770ed5d5456821a20

Adding support for Alpine Linux

Hello,

I'm currently looking into adding support for Alpine Linux to appstream-genearator so we can ship appstream data in Alpine Linux 3.12 (and as such postmarketOS) since we've recently gotten support for installing packages via GNOME Software. I figured it'd be good to discuss some things before I go ahead and start implementing something.

I see most other backends parse the repository index manually. I'd rather not parse it manually since we use a custom format for our APKINDEX and I have D bindings for our package manager, apk-tools, anyway. Would it be acceptable to make the apk backend optional and depend on that lib?
We currently don't embed file lists into our repository index, nor do we embed it in the metadata for our packages. This will hopefully be fixed in the next version of our package manager, but right now the only ways to know what files are in a package are to either peek into the package archive or to query https://pkgs.alpinelinux.org .

Segfault when generating metadata on Arch

appstream-generator 0.6.2, compiled against ldc 1.1

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000590fff in asgen.result.GeneratorResult.finalize() (this=0x7f3a0bbbc100) at result.d:252
252     result.d: No existe el fichero o el directorio.
[Current thread is 1 (Thread 0x7f4a3e56d700 (LWP 1220))]
(gdb) thread apply all bt

Thread 8 (Thread 0x7f6a4bbce900 (LWP 1219)):
#0  0x00007f6a48cba48d in poll () at /usr/lib/libc.so.6
#1  0x00007f4a3c32e955 in  () at /usr/lib/libcurl.so
#2  0x00007f4a3c32934c in curl_multi_wait () at /usr/lib/libcurl.so
#3  0x00007f4a3c321c74 in curl_easy_perform () at /usr/lib/libcurl.so
#4  0x00007f6a497d258f in std.net.curl.Curl.perform(std.typecons.Flag!("throwOnError").Flag) ()
    at /usr/lib/libphobos2-ldc.so.71
#5  0x00007f6a497d241b in std.net.curl.HTTP.perform(std.typecons.Flag!("throwOnError").Flag) ()
    at /usr/lib/libphobos2-ldc.so.71
#6  0x000000000056385a in std.net.curl._basicHTTP!(ubyte)._basicHTTP(const(char)[], const(void)[], std.net.curl.HTTP) (url=..., sendData=..., client=...) at screenshothandler.d:1013
#7  0x0000000000562e28 in std.net.curl.get!(std.net.curl.HTTP, ubyte).get(const(char)[], std.net.curl.HTTP) (url=..., conn=...) at screenshothandler.d:536
#8  0x0000000000561be0 in asgen.handlers.screenshothandler.processScreenshot(asgen.result.GeneratorResult, appstream.Component.Component, appstream.Screenshot.Screenshot, immutable(char)[], uint) (gres=0x7f3a0bba5900, cpt=0x7f3a0bc4b980, scr=0x7f3a0bec2c80, mediaExportDir=..., scrNo=1) at screenshothandler.d:98
#9  0x0000000000561722 in asgen.handlers.screenshothandler.processScreenshots(asgen.result.GeneratorResult, appstream.Component.Component, immutable(char)[]) (gres=0x7f3a0bba5900, cpt=0x7f3a0bc4b980, mediaExportDir=...) at screenshothandler.d:53
#10 0x000000000052d777 in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=..., pkg=0x7f4a35698500) at extractor.d:216
#11 0x00000000005188da in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (pkg=<optimized out>) at engine.d:139
#12 0x000000000051db2c in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() () at /usr/include/dlang/ldc/std/parallelism.d-mixin-3785:3829
#13 0x00007f6a4972a32f in std.parallelism.submitAndExecute(std.parallelism.TaskPool, scope void() delegate) ()
    at /usr/lib/libphobos2-ldc.so.71
#14 0x0000000000518827 in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate) (this=..., dg=...) at /usr/include/dlang/ldc/std/parallelism.d:3835
#15 0x00000000005186dc in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler) (this=0x7f6a4bace900, pkgs=..., iconh=0x7f4a0ef4c480) at engine.d:134
#16 0x000000000051c079 in asgen.engine.Engine.run(immutable(char)[]) (this=0x7f6a4bace900, suite_name=...) at engine.d:505
#17 0x00000000004b95e8 in D main (args=...) at app.d:118

Thread 7 (Thread 0x7f4a37fff700 (LWP 1224)):
#0  0x00007f6a48cba48d in poll () at /usr/lib/libc.so.6
#1  0x00007f4a3c32e955 in  () at /usr/lib/libcurl.so
#2  0x00007f4a3c32934c in curl_multi_wait () at /usr/lib/libcurl.so
#3  0x00007f4a3c321c74 in curl_easy_perform () at /usr/lib/libcurl.so
#4  0x00007f6a497d258f in std.net.curl.Curl.perform(std.typecons.Flag!("throwOnError").Flag) ()
---Type <return> to continue, or q <return> to quit---
    at /usr/lib/libphobos2-ldc.so.71
#5  0x00007f6a497d241b in std.net.curl.HTTP.perform(std.typecons.Flag!("throwOnError").Flag) ()
    at /usr/lib/libphobos2-ldc.so.71
#6  0x000000000056385a in std.net.curl._basicHTTP!(ubyte)._basicHTTP(const(char)[], const(void)[], std.net.curl.HTTP) (url=..., sendData=..., client=...) at screenshothandler.d:1013
#7  0x0000000000562e28 in std.net.curl.get!(std.net.curl.HTTP, ubyte).get(const(char)[], std.net.curl.HTTP) (url=..., conn=...) at screenshothandler.d:536
#8  0x0000000000561be0 in asgen.handlers.screenshothandler.processScreenshot(asgen.result.GeneratorResult, appstream.Component.Component, appstream.Screenshot.Screenshot, immutable(char)[], uint) (gres=0x7f3a0bba5a00, cpt=0x7f3a0be42d80, scr=0x7f3a0bffdd80, mediaExportDir=..., scrNo=1) at screenshothandler.d:98
#9  0x0000000000561722 in asgen.handlers.screenshothandler.processScreenshots(asgen.result.GeneratorResult, appstream.Component.Component, immutable(char)[]) (gres=0x7f3a0bba5a00, cpt=0x7f3a0be42d80, mediaExportDir=...) at screenshothandler.d:53
#10 0x000000000052d777 in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=..., pkg=0x7f4a3ed50100) at extractor.d:216
#11 0x00000000005188da in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (pkg=<optimized out>) at engine.d:139
#12 0x000000000051db2c in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() () at /usr/include/dlang/ldc/std/parallelism.d-mixin-3785:3829
#13 0x00007f6a49729159 in std.parallelism.TaskPool.executeWorkLoop() () at /usr/lib/libphobos2-ldc.so.71
#14 0x00007f6a491f8a51 in thread_entryPoint () at /usr/lib/libdruntime-ldc.so.71
#15 0x00007f6a48752454 in start_thread () at /usr/lib/libpthread.so.0
#16 0x00007f6a48cc37df in clone () at /usr/lib/libc.so.6

Thread 6 (Thread 0x7f4a3d56b700 (LWP 1222)):
#0  0x00007f6a48cb64ed in read () at /usr/lib/libc.so.6
#1  0x00007f6a4aa382f0 in  () at /usr/lib/libarchive.so.13
#2  0x00007f6a4aa3236a in __archive_read_filter_ahead () at /usr/lib/libarchive.so.13
#3  0x00007f6a4aa3da41 in  () at /usr/lib/libarchive.so.13
#4  0x00007f6a4aa32758 in __archive_read_filter_consume () at /usr/lib/libarchive.so.13
#5  0x00007f6a4aa5a1f2 in  () at /usr/lib/libarchive.so.13
#6  0x00007f6a4aa31ced in archive_read_data_skip () at /usr/lib/libarchive.so.13
#7  0x00000000004d1285 in asgen.zarchive.ArchiveDecompressor.readData(immutable(char)[]) (this=..., fname=...)
    at zarchive.d:312
#8  0x00000000005a790d in asgen.backends.archlinux.alpkg.ArchPackage.getFileData(immutable(char)[]) (this=0x7f4a34c47900, fname=...) at alpkg.d:81
#9  0x000000000052c8c7 in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=..., pkg=0x7f4a34c47900) at extractor.d:90
#10 0x00000000005188da in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (pkg=<optimized out>) at engine.d:139
#11 0x000000000051db2c in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope---Type <return> to continue, or q <return> to quit---
 int(ref asgen.backends.interfaces.Package) delegate).doIt() () at /usr/include/dlang/ldc/std/parallelism.d-mixin-3785:3829
#12 0x00007f6a49729159 in std.parallelism.TaskPool.executeWorkLoop() () at /usr/lib/libphobos2-ldc.so.71
#13 0x00007f6a491f8a51 in thread_entryPoint () at /usr/lib/libdruntime-ldc.so.71
#14 0x00007f6a48752454 in start_thread () at /usr/lib/libpthread.so.0
#15 0x00007f6a48cc37df in clone () at /usr/lib/libc.so.6

Thread 5 (Thread 0x7f4a3cd6a700 (LWP 1223)):
#0  0x00007f6a48cb64ed in read () at /usr/lib/libc.so.6
#1  0x00007f6a48c4dad4 in __GI__IO_file_xsgetn () at /usr/lib/libc.so.6
#2  0x00007f6a48c42b29 in fread () at /usr/lib/libc.so.6
#3  0x00000000004d3282 in std.stdio.File.rawRead!(ubyte).rawRead(ubyte[]) (this=..., buffer=...)
    at /usr/include/dlang/ldc/std/stdio.d:929
#4  0x00007f6a49743842 in std.stdio.File.ByChunk.this(std.stdio.File, ubyte[]) () at /usr/lib/libphobos2-ldc.so.71
#5  0x00007f6a49743c2d in std.stdio.File.byChunk(ulong) () at /usr/lib/libphobos2-ldc.so.71
#6  0x00007f6a4972ff59 in std.process.executeImpl!(std.process.pipeProcess, const(char[])[]).executeImpl(const(char[])[], const(immutable(char)[][immutable(char)[]]), std.process.Config, ulong, const(char[])) () at /usr/lib/libphobos2-ldc.so.71
#7  0x00007f6a4972fe31 in std.process.execute(const(char[][]), const(immutable(char)[][immutable(char)[]]), std.process.Config, ulong, const(char[])) () at /usr/lib/libphobos2-ldc.so.71
#8  0x000000000056f57b in asgen.image.optimizePNG(immutable(char)[]) (fname=...) at image.d:64
#9  0x0000000000571698 in asgen.image.Canvas.savePng(immutable(char)[]) (this=0x7f3a0c066000, fname=...) at image.d:427
#10 0x0000000000557d2f in asgen.handlers.iconhandler.IconHandler.storeIcon(appstream.Component.Component, asgen.result.GeneratorResult, immutable(char)[], asgen.backends.interfaces.Package, immutable(char)[], asgen.utils.ImageSize) (this=0x7f4a0ef4c480, cpt=0x7f3a0be82540, gres=0x7f3a0bbbc000, cptExportPath=..., sourcePkg=0x7f4a3fe26a00, iconPath=..., size=...)
    at iconhandler.d:483
#11 0x0000000000558c7e in asgen.handlers.iconhandler.IconHandler.process(asgen.result.GeneratorResult, appstream.Component.Component) (this=0x7f4a0ef4c480, gres=0x7f3a0bbbc000, cpt=0x7f3a0be82540) at iconhandler.d:548
#12 0x000000000052d6d8 in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=..., pkg=0x7f4a3fe26a00) at extractor.d:209
#13 0x00000000005188da in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (pkg=<optimized out>) at engine.d:139
#14 0x000000000051db2c in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() () at /usr/include/dlang/ldc/std/parallelism.d-mixin-3785:3829
#15 0x00007f6a49729159 in std.parallelism.TaskPool.executeWorkLoop() () at /usr/lib/libphobos2-ldc.so.71
#16 0x00007f6a491f8a51 in thread_entryPoint () at /usr/lib/libdruntime-ldc.so.71
#17 0x00007f6a48752454 in start_thread () at /usr/lib/libpthread.so.0
#18 0x00007f6a48cc37df in clone () at /usr/lib/libc.so.6

Thread 4 (Thread 0x7f4a377fe700 (LWP 1225)):
#0  0x00007f6a48cb64ed in read () at /usr/lib/libc.so.6
#1  0x00007f6a4aa382f0 in  () at /usr/lib/libarchive.so.13
---Type <return> to continue, or q <return> to quit---
#2  0x00007f6a4aa3236a in __archive_read_filter_ahead () at /usr/lib/libarchive.so.13
#3  0x00007f6a4aa3da41 in  () at /usr/lib/libarchive.so.13
#4  0x00007f6a4aa32758 in __archive_read_filter_consume () at /usr/lib/libarchive.so.13
#5  0x00007f6a4aa5a1f2 in  () at /usr/lib/libarchive.so.13
#6  0x00007f6a4aa31ced in archive_read_data_skip () at /usr/lib/libarchive.so.13
#7  0x00000000004d1285 in asgen.zarchive.ArchiveDecompressor.readData(immutable(char)[]) (this=..., fname=...)
    at zarchive.d:312
#8  0x00000000005a790d in asgen.backends.archlinux.alpkg.ArchPackage.getFileData(immutable(char)[]) (this=0x7f4a2ddc2d00, fname=...) at alpkg.d:81
#9  0x000000000052d00c in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=..., pkg=0x7f4a2ddc2d00) at extractor.d:159
#10 0x00000000005188da in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (pkg=<optimized out>) at engine.d:139
#11 0x000000000051db2c in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() () at /usr/include/dlang/ldc/std/parallelism.d-mixin-3785:3829
#12 0x00007f6a49729159 in std.parallelism.TaskPool.executeWorkLoop() () at /usr/lib/libphobos2-ldc.so.71
#13 0x00007f6a491f8a51 in thread_entryPoint () at /usr/lib/libdruntime-ldc.so.71
#14 0x00007f6a48752454 in start_thread () at /usr/lib/libpthread.so.0
#15 0x00007f6a48cc37df in clone () at /usr/lib/libc.so.6

Thread 3 (Thread 0x7f4a36ffd700 (LWP 1226)):
#0  0x00007f6a48cb64ed in read () at /usr/lib/libc.so.6
#1  0x00007f6a4aa382f0 in  () at /usr/lib/libarchive.so.13
#2  0x00007f6a4aa3236a in __archive_read_filter_ahead () at /usr/lib/libarchive.so.13
#3  0x00007f6a4aa3da41 in  () at /usr/lib/libarchive.so.13
#4  0x00007f6a4aa32758 in __archive_read_filter_consume () at /usr/lib/libarchive.so.13
#5  0x00007f6a4aa5a1f2 in  () at /usr/lib/libarchive.so.13
#6  0x00007f6a4aa31ced in archive_read_data_skip () at /usr/lib/libarchive.so.13
#7  0x00000000004d1285 in asgen.zarchive.ArchiveDecompressor.readData(immutable(char)[]) (this=..., fname=...)
    at zarchive.d:312
#8  0x00000000005a790d in asgen.backends.archlinux.alpkg.ArchPackage.getFileData(immutable(char)[]) (this=0x7f4a3476db00, fname=...) at alpkg.d:81
#9  0x000000000052d00c in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=..., pkg=0x7f4a3476db00) at extractor.d:159
#10 0x00000000005188da in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (pkg=<optimized out>) at engine.d:139
#11 0x000000000051db2c in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() () at /usr/include/dlang/ldc/std/parallelism.d-mixin-3785:3829
#12 0x00007f6a49729159 in std.parallelism.TaskPool.executeWorkLoop() () at /usr/lib/libphobos2-ldc.so.71
#13 0x00007f6a491f8a51 in thread_entryPoint () at /usr/lib/libdruntime-ldc.so.71
---Type <return> to continue, or q <return> to quit---
#14 0x00007f6a48752454 in start_thread () at /usr/lib/libpthread.so.0
#15 0x00007f6a48cc37df in clone () at /usr/lib/libc.so.6

Thread 2 (Thread 0x7f4a3dd6c700 (LWP 1221)):
#0  0x00007f6a48c92ffd in nanosleep () at /usr/lib/libc.so.6
#1  0x00007f6a491f97ed in core.thread.Thread.sleep(core.time.Duration) () at /usr/lib/libdruntime-ldc.so.71
#2  0x00007f6a49201c82 in core.internal.spinlock.SpinLock.lock() shared () at /usr/lib/libdruntime-ldc.so.71
#3  0x00007f6a49212cbe in gc.gc.GC.runLocked!(gc.gc.GC.extendNoSync(void*, ulong, ulong, const(TypeInfo)), gc.gc.extendTime, gc.gc.numExtends, void*, ulong, ulong, const(TypeInfo)).runLocked(ref void*, ref ulong, ref ulong, ref const(TypeInfo)) ()
    at /usr/lib/libdruntime-ldc.so.71
#4  0x00007f6a49212c71 in gc.gc.GC.extend(void*, ulong, ulong, const(TypeInfo)) () at /usr/lib/libdruntime-ldc.so.71
#5  0x00000000004ded4d in std.array.Appender!(char[]).Appender.ensureAddable(ulong) (this=..., nelems=1)
    at /usr/include/dlang/ldc/std/array.d:2773
#6  0x00000000004de71e in std.array.Appender!(char[]).Appender.put!(char).put(char) (this=..., item=47 '/')
    at /usr/include/dlang/ldc/std/array.d:2833
#7  0x00000000004dbfeb in std.array.array!(std.path.asNormalizedPath!(const(char)[]).asNormalizedPath(const(char)[]).Result).array(std.path.asNormalizedPath!(const(char)[]).asNormalizedPath(const(char)[]).Result) (r=...)
    at /usr/include/dlang/ldc/std/array.d:135
#8  0x00000000004d08f0 in std.path.buildNormalizedPath!(char).buildNormalizedPath(const(char[])[]...) (paths=...)
    at /usr/include/dlang/ldc/std/path.d:1597
#9  0x00000000004d06ce in asgen.zarchive.ArchiveDecompressor.pathMatches(immutable(char)[], immutable(char)[]) (this=..., path1=..., path2=...) at zarchive.d:198
#10 0x00000000004d0e42 in asgen.zarchive.ArchiveDecompressor.readData(immutable(char)[]) (this=..., fname=...)
    at zarchive.d:283
#11 0x00000000005a790d in asgen.backends.archlinux.alpkg.ArchPackage.getFileData(immutable(char)[]) (this=0x7f4a34921c00, fname=...) at alpkg.d:81
#12 0x000000000052c8c7 in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=..., pkg=0x7f4a34921c00) at extractor.d:90
#13 0x00000000005188da in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (pkg=<optimized out>) at engine.d:139
#14 0x000000000051db2c in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() () at /usr/include/dlang/ldc/std/parallelism.d-mixin-3785:3829
#15 0x00007f6a49729159 in std.parallelism.TaskPool.executeWorkLoop() () at /usr/lib/libphobos2-ldc.so.71
#16 0x00007f6a491f8a51 in thread_entryPoint () at /usr/lib/libdruntime-ldc.so.71
#17 0x00007f6a48752454 in start_thread () at /usr/lib/libpthread.so.0
#18 0x00007f6a48cc37df in clone () at /usr/lib/libc.so.6

Thread 1 (Thread 0x7f4a3e56d700 (LWP 1220)):
#0  0x0000000000590fff in asgen.result.GeneratorResult.finalize() (this=0x7f3a0bbbc100) at result.d:252
#1  0x000000000052d839 in asgen.extractor.DataExtractor.processPackage(asgen.backends.interfaces.Package) (this=..., pkg=0x7f---Type <return> to continue, or q <return> to quit---
4a3fe95400) at extractor.d:234
#2  0x00000000005188da in asgen.engine.Engine.processPackages(asgen.backends.interfaces.Package[], asgen.handlers.iconhandler.IconHandler).__foreachbody3(ref asgen.backends.interfaces.Package) (pkg=<optimized out>) at engine.d:139
#3  0x000000000051db2c in std.parallelism.ParallelForeach!(asgen.backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref asgen.backends.interfaces.Package) delegate).doIt() () at /usr/include/dlang/ldc/std/parallelism.d-mixin-3785:3829
#4  0x00007f6a49729159 in std.parallelism.TaskPool.executeWorkLoop() () at /usr/lib/libphobos2-ldc.so.71
#5  0x00007f6a491f8a51 in thread_entryPoint () at /usr/lib/libdruntime-ldc.so.71
#6  0x00007f6a48752454 in start_thread () at /usr/lib/libpthread.so.0
#7  0x00007f6a48cc37df in clone () at /usr/lib/libc.so.6

MDB_BAD_RSLOT: Invalid reuse of reader locktable slot

I got this

object.Exception@source/contentsstore.d(64): mdb_txn_begin[-30783]: MDB_BAD_RSLOT: Invalid reuse of reader locktable slot
----------------
0x4c7f48 void contentsstore.ContentsStore.checkError(int, immutable(char)[])
        source/contentsstore.d:64
0x4c8052 bindings.lmdb.MDB_txn_s* contentsstore.ContentsStore.newTransaction(uint)
        source/contentsstore.d:141
0x4c8052 bool contentsstore.ContentsStore.packageExists(immutable(char)[])
        source/contentsstore.d:186
0x4d0cee __foreachbody5
        source/engine.d:184
0x4f67bb doIt
        /usr/lib/gcc/x86_64-linux-gnu/5/include/d/std/parallelism.d:3859
0x5d11cb void std.parallelism.AbstractTask.job()
        ../../../../src/libphobos/src/std/parallelism.d:415
0x5d11cb void std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*)
        ../../../../src/libphobos/src/std/parallelism.d:1083
0x5d11cb void std.parallelism.TaskPool.executeWorkLoop()
        ../../../../src/libphobos/src/std/parallelism.d:1138
0x5d11cb void std.parallelism.TaskPool.startWorkLoop()
        ../../../../src/libphobos/src/std/parallelism.d:1117
0x6197f6 void core.thread.Thread.run()
        ../../../../src/libphobos/libdruntime/core/thread.d:1364
0x6197f6 thread_entryPoint
        ../../../../src/libphobos/libdruntime/core/thread.d:371
0x7ffff5c4a6f9 start_thread
        ???:0
0x7ffff5562b5c clone
        ???:0
0xffffffffffffffff ???
        ???:0

The documentation says "A transaction and its cursors must only be used by a single thread, and a thread may only have a single transaction at a time." - is it possible for this to be violated here?

Build fails with gir-to-d 0.13

[29/464] Compiling D object 'girbindings@sta/.._build_girepo_gio_AppInfo.d.o'.
FAILED: girbindings@sta/.._build_girepo_gio_AppInfo.d.o 
dmd  -Igirbindings@sta -I. -I../appstream-generator-0.6.8 -Igirepo -I../appstream-generator-0.6.8/girepo -color=on -wi -g  -of'girbindings@sta/.._build_girepo_gio_AppInfo.d.o' -c ../build/girepo/gio/AppInfo.d
girepo/gobject/ObjectG.d(183): Error: cannot create instance of interface AppInfoIF
girepo/gio/AppInfoT.d(133): Error: template instance gobject.ObjectG.ObjectG.getDObject!(AppInfoIF, AppInfoIF, GAppInfo*) error instantiating
girepo/gobject/ObjectG.d(183): Error: cannot create instance of interface IconIF
girepo/gio/AppInfoT.d(538): Error: template instance gobject.ObjectG.ObjectG.getDObject!(IconIF, IconIF, GIcon*) error instantiating

.desktop ID divergence between manual data and generated data

For quite a while in neon we've been observing inconsistencies between desktop ids the same entity (e.g. org.kde.kdenlive) would sometimes be available as ID: org.kde.kdenlive and other times as ID: org.kde.kdenlive.desktop and as far as neon is concerned it would always have a superset of both.

The reason appears to be that when asgen autogenerates data from the desktop file it picks ID: org.kde.kdenlive whereas the actual upstream information is declaring it ID: org.kde.kdenlive.desktop client-side these two are not treated as equal resulting in duplicated entries.

Either the client should treat them as the same entity or the autogenerator ought to append .desktop I suppose.

$ appstreamcli search org.kde.kdenlive        
Identifier: org.kde.kdenlive [desktop-application]
Name: Kdenlive
Summary: Nonlinear video editor by KDE
Package: kdenlive
Icon: kdenlive_kdenlive.png
---
Identifier: org.kde.kdenlive.desktop [desktop-application]
Name: Kdenlive
Summary: Nonlinear video editor for KDE
Package: kdenlive
Icon: kdenlive_kdenlive.png

Oxygen font error: font-metainfo-but-no-font

Ubuntu 17.04
appstream-generator 0.6.3

The same version (5.4.3-3) of fonts-oxygen is present in both Debian and Ubuntu, but Ubuntu's appstream generator gives an error:

font-metainfo-but-no-font
A metainfo file with component-type font was found, but we could not find any matching
font file (TrueType or OpenType) in the package.
This can mean that the <provides> - <font> tags contain wrong values that we could not
map to the actual fonts, or that the package simply contained no fonts at all.

http://appstream.ubuntu.com/zesty/universe/issues/fonts-oxygen.html

https://appstream.debian.org/sid/main/metainfo/fonts-oxygen.html
(No issues page on Debian)

fonts-oxygen does contain these fonts:

/usr/share/fonts/truetype/oxygen/Oxygen-Sans-Bold.ttf
/usr/share/fonts/truetype/oxygen/Oxygen-Sans.ttf
/usr/share/fonts/truetype/oxygen/OxygenMono-Regular.ttf

https://sources.debian.net/src/oxygen-fonts/unstable/debian/patches/01_add-metainfo.patch

A different font works

I recently added appstream metainfo for Cantarell in Ubuntu 17.04. I didn't specify any Provides but it shows up in Software (only if you explicitly search for Cantarell; I don't see it listed in any Category or in Installed.)

Support languages tag

I was looking at the kudo code in gnome-software, and the "translated into my languages" thing was reported as a problem in Ubuntu since it was never shown as enabled.

For that to work, we would need to support emitting the languages tag.

Has this been considered before? It would require looking into `.mo' files, which means extracting packages, which is a bit slow. Maybe therefore it should be a feature?

Hang in contentsstore

I checked on the asgen logs this morning, and there is a hung process. Here's the backtrace.

Thread 8 (Thread 0x7f9c0bfff700 (LWP 6864)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x0000000000616409 in core.sync.condition.Condition.wait() (this=0x7f9c1c1c6600) at ../../../../src/libphobos/libdruntime/core/sync/condition.d:159
#2  0x00000000005d06be in std.parallelism.TaskPool.wait() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1357
#3  std.parallelism.TaskPool.pop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1151
#4  std.parallelism.TaskPool.executeWorkLoop() (this=<optimized out>) at ../../../../src/libphobos/src/std/parallelism.d:1127
#5  std.parallelism.TaskPool.startWorkLoop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1117
#6  0x0000000000618dc7 in core.thread.Thread.run() (this=0x7f9c17910700) at ../../../../src/libphobos/libdruntime/core/thread.d:1364
#7  thread_entryPoint (arg=0x7f9c17910700) at ../../../../src/libphobos/libdruntime/core/thread.d:371
#8  0x00007fbc305a76fa in start_thread (arg=0x7f9c0bfff700) at pthread_create.c:333
#9  0x00007fbc2febfb5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 7 (Thread 0x7f9c10ff6700 (LWP 6863)):
#0  __lll_lock_wait_private () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
#1  0x00007fbc2fe3d402 in __libc_calloc (n=140307790299168, elem_size=<optimized out>) at malloc.c:3209
#2  0x00007fbc322b6c36 in mdb_txn_begin () from /usr/lib/x86_64-linux-gnu/liblmdb.so.0
#3  0x00000000004c802f in contentsstore.ContentsStore.newTransaction(uint) (flags=131072, this=0x7fabca4b1580) at source/contentsstore.d:142
#4  contentsstore.ContentsStore.packageExists(immutable(char)[]) (this=0x7fabca4b1580, pkid=...) at source/contentsstore.d:188
#5  0x00000000004d0d2f in engine.Engine.__foreachbody5 (this=0x7ffd11c37100, pkg=@0x7fac1f5531d0: 0x7fac0b9c6c00) at source/engine.d:184
#6  0x00000000004f676c in std.parallelism.ParallelForeach.doIt (this=0x7ffd11c36ee0) at /usr/lib/gcc/x86_64-linux-gnu/5/include/d/std/parallelism.d:3859
#7  0x00000000005d079c in std.parallelism.AbstractTask.job() (this=...) at ../../../../src/libphobos/src/std/parallelism.d:415
#8  std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*) (job=0x7ffd11c35e88, this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1083
#9  std.parallelism.TaskPool.executeWorkLoop() (this=<optimized out>) at ../../../../src/libphobos/src/std/parallelism.d:1138
#10 std.parallelism.TaskPool.startWorkLoop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1117
#11 0x0000000000618dc7 in core.thread.Thread.run() (this=0x7f9c17910600) at ../../../../src/libphobos/libdruntime/core/thread.d:1364
#12 thread_entryPoint (arg=0x7f9c17910600) at ../../../../src/libphobos/libdruntime/core/thread.d:371
#13 0x00007fbc305a76fa in start_thread (arg=0x7f9c10ff6700) at pthread_create.c:333
#14 0x00007fbc2febfb5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 6 (Thread 0x7f9c117f7700 (LWP 6862)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x0000000000616409 in core.sync.condition.Condition.wait() (this=0x7f9c1c1c6600) at ../../../../src/libphobos/libdruntime/core/sync/condition.d:159
#2  0x00000000005d06be in std.parallelism.TaskPool.wait() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1357
#3  std.parallelism.TaskPool.pop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1151
#4  std.parallelism.TaskPool.executeWorkLoop() (this=<optimized out>) at ../../../../src/libphobos/src/std/parallelism.d:1127
#5  std.parallelism.TaskPool.startWorkLoop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1117
#6  0x0000000000618dc7 in core.thread.Thread.run() (this=0x7f9c17910500) at ../../../../src/libphobos/libdruntime/core/thread.d:1364
#7  thread_entryPoint (arg=0x7f9c17910500) at ../../../../src/libphobos/libdruntime/core/thread.d:371
#8  0x00007fbc305a76fa in start_thread (arg=0x7f9c117f7700) at pthread_create.c:333
#9  0x00007fbc2febfb5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 5 (Thread 0x7f9c097f7700 (LWP 6861)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x0000000000616409 in core.sync.condition.Condition.wait() (this=0x7f9c1c1c6600) at ../../../../src/libphobos/libdruntime/core/sync/condition.d:159
#2  0x00000000005d06be in std.parallelism.TaskPool.wait() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1357
#3  std.parallelism.TaskPool.pop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1151
#4  std.parallelism.TaskPool.executeWorkLoop() (this=<optimized out>) at ../../../../src/libphobos/src/std/parallelism.d:1127
#5  std.parallelism.TaskPool.startWorkLoop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1117
#6  0x0000000000618dc7 in core.thread.Thread.run() (this=0x7f9c17910300) at ../../../../src/libphobos/libdruntime/core/thread.d:1364
#7  thread_entryPoint (arg=0x7f9c17910300) at ../../../../src/libphobos/libdruntime/core/thread.d:371
#8  0x00007fbc305a76fa in start_thread (arg=0x7f9c097f7700) at pthread_create.c:333
#9  0x00007fbc2febfb5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 4 (Thread 0x7f9c11ff8700 (LWP 6860)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x0000000000616409 in core.sync.condition.Condition.wait() (this=0x7f9c1c1c6600) at ../../../../src/libphobos/libdruntime/core/sync/condition.d:159
#2  0x00000000005d06be in std.parallelism.TaskPool.wait() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1357
#3  std.parallelism.TaskPool.pop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1151
#4  std.parallelism.TaskPool.executeWorkLoop() (this=<optimized out>) at ../../../../src/libphobos/src/std/parallelism.d:1127
#5  std.parallelism.TaskPool.startWorkLoop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1117
#6  0x0000000000618dc7 in core.thread.Thread.run() (this=0x7f9c17910100) at ../../../../src/libphobos/libdruntime/core/thread.d:1364
#7  thread_entryPoint (arg=0x7f9c17910100) at ../../../../src/libphobos/libdruntime/core/thread.d:371
#8  0x00007fbc305a76fa in start_thread (arg=0x7f9c11ff8700) at pthread_create.c:333
#9  0x00007fbc2febfb5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 3 (Thread 0x7f9c1f5e5700 (LWP 6859)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x0000000000616409 in core.sync.condition.Condition.wait() (this=0x7f9c1c1c6600) at ../../../../src/libphobos/libdruntime/core/sync/condition.d:159
#2  0x00000000005d06be in std.parallelism.TaskPool.wait() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1357
#3  std.parallelism.TaskPool.pop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1151
#4  std.parallelism.TaskPool.executeWorkLoop() (this=<optimized out>) at ../../../../src/libphobos/src/std/parallelism.d:1127
#5  std.parallelism.TaskPool.startWorkLoop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1117
#6  0x0000000000618dc7 in core.thread.Thread.run() (this=0x7f9c17910000) at ../../../../src/libphobos/libdruntime/core/thread.d:1364
#7  thread_entryPoint (arg=0x7f9c17910000) at ../../../../src/libphobos/libdruntime/core/thread.d:371
#8  0x00007fbc305a76fa in start_thread (arg=0x7f9c1f5e5700) at pthread_create.c:333
#9  0x00007fbc2febfb5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 2 (Thread 0x7fac24960700 (LWP 6858)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x0000000000616409 in core.sync.condition.Condition.wait() (this=0x7f9c1c1c6600) at ../../../../src/libphobos/libdruntime/core/sync/condition.d:159
#2  0x00000000005d06be in std.parallelism.TaskPool.wait() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1357
#3  std.parallelism.TaskPool.pop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1151
#4  std.parallelism.TaskPool.executeWorkLoop() (this=<optimized out>) at ../../../../src/libphobos/src/std/parallelism.d:1127
#5  std.parallelism.TaskPool.startWorkLoop() (this=0x7f9c1c1c6400) at ../../../../src/libphobos/src/std/parallelism.d:1117
#6  0x0000000000618dc7 in core.thread.Thread.run() (this=0x7f9c17909f00) at ../../../../src/libphobos/libdruntime/core/thread.d:1364
#7  thread_entryPoint (arg=0x7f9c17909f00) at ../../../../src/libphobos/libdruntime/core/thread.d:371
#8  0x00007fbc305a76fa in start_thread (arg=0x7fac24960700) at pthread_create.c:333
#9  0x00007fbc2febfb5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 1 (Thread 0x7fbc329297c0 (LWP 6798)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1  0x0000000000616409 in core.sync.condition.Condition.wait() (this=0x7f9c1c1c6680) at ../../../../src/libphobos/libdruntime/core/sync/condition.d:159
#2  0x00000000005d204f in std.parallelism.TaskPool.waitUntilCompletion() (this=<optimized out>) at ../../../../src/libphobos/src/std/parallelism.d:1378
#3  std.parallelism.Task!(std.parallelism.run, void() delegate).Task.yieldForce() (this=...) at ../../../../src/libphobos/src/std/parallelism.d:644
#4  std.parallelism.submitAndExecute(std.parallelism.TaskPool, scope void() delegate) (pool=<optimized out>, doIt=...) at ../../../../src/libphobos/src/std/parallelism.d:3465
#5  0x0000000000542a3f in std.parallelism.ParallelForeach!(backends.interfaces.Package[]).ParallelForeach.opApply(scope int(ref backends.interfaces.Package) delegate) (this=..., dg=...) at /usr/lib/gcc/x86_64-linux-gnu/5/include/d/std/parallelism.d:3865
#6  0x00000000004ec8f9 in engine.Engine.seedContentsData(config.Suite, immutable(char)[], immutable(char)[]) (arch=..., section=..., suite=..., this=0x7fbc32831800) at source/engine.d:180
#7  engine.Engine.run(immutable(char)[]) (this=<optimized out>, suite_name=...) at source/engine.d:481
#8  0x00000000004e45f0 in D main (args=...) at source/app.d:118

It looks like all of the threads are waiting on the task in thread 7 to finish, but that is deadlocked.

MDB_KEYEXIST errors

When I run asgen on my full repo (which has some empty sections) I get exceptions as follows:

object.Exception@source/datacache.d(85): mdb_put (stats)[-30799]: MDB_KEYEXIST: Key/data pair already exists
----------------
0x51bb51 void ag.datacache.DataCache.checkError(int, immutable(char)[])
    source/datacache.d:85
0x51eef9 void ag.datacache.DataCache.addStatistics(immutable(char)[])
    source/datacache.d:569
0x5377bd void ag.reportgenerator.ReportGenerator.saveStatistics(immutable(char)[], immutable(char)[], ag.reportgenerator.ReportGenerator.DataSummary)
    source/report-generator.d:634
0x5395d6 void ag.reportgenerator.ReportGenerator.processFor(immutable(char)[], immutable(char)[], ag.backend.intf.Package[])
    source/report-generator.d:722
0x5225c6 void ag.engine.Engine.run(immutable(char)[])
    source/engine.d:381
0x453af8 _Dmain
    source/app.d:112
0x60026e _D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv
    ../../../../src/libphobos/libdruntime/rt/dmain2.d:411
0x60039e void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate())
    ../../../../src/libphobos/libdruntime/rt/dmain2.d:386
0x600778 void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll()
    ../../../../src/libphobos/libdruntime/rt/dmain2.d:411
0x60039e void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate())
    ../../../../src/libphobos/libdruntime/rt/dmain2.d:386
0x600525 _d_run_main
    ../../../../src/libphobos/libdruntime/rt/dmain2.d:419
0x45367a main
    /usr/lib/gcc/x86_64-linux-gnu/5/include/d/__entrypoint.di:62
0x7f5a08a5660f __libc_start_main
    ???:0
0x453588 _start
    ???:0
0xffffffffffffffff ???
    ???:0

This occurs 8 times, and I have 8 suites configured:
{trusty,xenial}{,-{updates,proposed,experimental}}
This may be coincidence, but we do often have the same package appear in multiple suites if it's arch-independent and simply hasn't needed a rebuild recently.

Here's the complete output of the asgen run: asgen-out.txt.zip

If you need access to the entire repository, I can give you anonymous rsync access — PM me for details. It's about 8.5GB.

Segmentation fault in "get_free_list () at arena.c:855"

I think that's inside the libc.

A few of my asgen runs end when the generator segfaults :(.

Thread 355 (Thread 0x7fefcd7fa700 (LWP 12934)):
#0  get_free_list () at arena.c:855
#1  0x00007ffff54df58d in arena_get2 (avoid_arena=0x0, size=552) at arena.c:950
#2  __GI___libc_malloc (bytes=552) at malloc.c:2912
#3  0x00007ffff54c9c8d in __fopen_internal (filename=0x7ffff55eb11b "/etc/resolv.conf", mode=0x7ffff55e7f73 "rce",
    is32=1) at iofopen.c:69
#4  0x00007ffff54c9d2a in _IO_new_fopen (filename=<optimized out>, mode=<optimized out>) at iofopen.c:97
#5  0x00007ffff5582454 in __res_vinit (statp=0x7fefcd7fadb8, preinit=preinit@entry=0) at res_init.c:230
#6  0x00007ffff5582dd7 in __GI___res_ninit (statp=<optimized out>) at res_init.c:143
#7  0x00007ffff55841cd in __GI___res_maybe_init (resp=0x7fefcd7fadb8, preinit=preinit@entry=0) at res_libc.c:125
#8  0x00007ffff5546c08 in gaih_inet (name=name@entry=0x37aee40 "ftpmaster.internal", service=<optimized out>,
    req=req@entry=0x55c1090, pai=pai@entry=0x7fefcd7f8ca8, naddrs=naddrs@entry=0x7fefcd7f8ca4)
    at ../sysdeps/posix/getaddrinfo.c:819
#9  0x00007ffff554a4ae in __GI_getaddrinfo (name=<optimized out>, service=0x7fefcd7f8e40 "80", hints=0x55c1090,
    pai=0x7fefcd7f8df0) at ../sysdeps/posix/getaddrinfo.c:2417
#10 0x00007ffff7ba5887 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4
#11 0x00007ffff7bb06ca in ?? () from /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4
#12 0x00007ffff7bade5b in ?? () from /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4
#13 0x00007ffff5c4a6fa in start_thread (arg=0x7fefcd7fa700) at pthread_create.c:333
#14 0x00007ffff5562b5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
Thread 354 (Thread 0x7fefce7fc700 (LWP 12930)):
#0  __lll_lock_wait_private () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
#1  0x00007ffff54d99c4 in get_free_list () at arena.c:867
#2  0x00007ffff54df58d in arena_get2 (avoid_arena=0x0, size=552) at arena.c:950
#3  __GI___libc_malloc (bytes=552) at malloc.c:2912
#4  0x00007ffff54c9c8d in __fopen_internal (filename=0x7ffff55eb11b "/etc/resolv.conf", mode=0x7ffff55e7f73 "rce",
    is32=1) at iofopen.c:69
#5  0x00007ffff54c9d2a in _IO_new_fopen (filename=<optimized out>, mode=<optimized out>) at iofopen.c:97
#6  0x00007ffff5582454 in __res_vinit (statp=0x7fefce7fcdb8, preinit=preinit@entry=0) at res_init.c:230
#7  0x00007ffff5582dd7 in __GI___res_ninit (statp=<optimized out>) at res_init.c:143
#8  0x00007ffff55841cd in __GI___res_maybe_init (resp=0x7fefce7fcdb8, preinit=preinit@entry=0) at res_libc.c:125
#9  0x00007ffff5546c08 in gaih_inet (name=name@entry=0x37aee80 "ftpmaster.internal", service=<optimized out>,
    req=req@entry=0xa4c300, pai=pai@entry=0x7fefce7faca8, naddrs=naddrs@entry=0x7fefce7faca4)
    at ../sysdeps/posix/getaddrinfo.c:819
#10 0x00007ffff554a4ae in __GI_getaddrinfo (name=<optimized out>, service=0x7fefce7fae40 "80", hints=0xa4c300,
    pai=0x7fefce7fadf0) at ../sysdeps/posix/getaddrinfo.c:2417
#11 0x00007ffff7ba5887 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4
#12 0x00007ffff7bb06ca in ?? () from /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4
#13 0x00007ffff7bade5b in ?? () from /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4
#14 0x00007ffff5c4a6fa in start_thread (arg=0x7fefce7fc700) at pthread_create.c:333
#15 0x00007ffff5562b5d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

No debugging symbols for libcurl atm, sorry. But that's two threads in the same codepath.

I think it's called from this thread

Thread 1 (Thread 0x7ffff7fc87c0 (LWP 12527)):
#0  0x00007ffff5556e8d in poll () at ../sysdeps/unix/syscall-template.S:84
#1  0x00007ffff7ba2564 in ?? () from /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4
#2  0x00007ffff7b966ec in curl_easy_perform () from /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4
#3  0x00000000005c288d in std.net.curl.Curl.perform(std.typecons.Flag!("throwOnError").Flag) (throwOnError=true,
    this=...) at ../../../../src/libphobos/src/std/net/curl.d:3808
#4  std.net.curl.HTTP.perform(std.typecons.Flag!("throwOnError").Flag) (this=...,
    throwOnError=throwOnError@entry=true) at ../../../../src/libphobos/src/std/net/curl.d:2280
#5  0x00000000004e4902 in utils.download(const(immutable(char)[]), ref std.stdio.File, const(uint)) (url=...,
    dest=..., retryCount=retryCount@entry=4) at source/utils.d:371
#6  0x00000000004e4a2f in utils.download(const(immutable(char)[]), ref std.stdio.File, const(uint)) (url=...,
    dest=..., retryCount=retryCount@entry=5) at source/utils.d:386
#7  0x00000000004e4ae3 in utils.getFileContents(const(immutable(char)[]), const(uint)) (path=...,
    retryCount=retryCount@entry=5) at source/utils.d:416
#8  0x00000000004e4ca7 in backends.debian.debpkgindex.DebianPackageIndex.findTranslations(const(immutable(char)[]), const(immutable(char)[])) (this=this@entry=0x7ffff7eced80, suite=..., section=...)
    at source/backends/debian/debpkgindex.d:78
#9  0x00000000004e6a26 in backends.debian.debpkgindex.DebianPackageIndex.loadPackageLongDescs(backends.debian.debpkg.DebPackage[immutable(char)[]], immutable(char)[], immutable(char)[]) (this=this@entry=0x7ffff7eced80, pkgs=...,
    suite=..., section=...) at source/backends/debian/debpkgindex.d:99
#10 0x00000000004e79cd in backends.debian.debpkgindex.DebianPackageIndex.loadPackages(immutable(char)[], immutable(char)[], immutable(char)[]) (this=0x7ffff7eced80, suite=..., section=..., arch=...)
    at source/backends/debian/debpkgindex.d:225
#11 0x00000000004e7b94 in backends.debian.debpkgindex.DebianPackageIndex.packagesFor(immutable(char)[], immutable(char)[], immutable(char)[]) (this=0x7ffff7eced80, suite=..., section=..., arch=...)
    at source/backends/debian/debpkgindex.d:234
#12 0x00000000004e7c89 in backends.ubuntu.ubupkgindex.UbuntuPackageIndex.packagesFor(immutable(char)[], immutable(char)[], immutable(char)[]) (this=0x7ffff7eced80, suite=..., section=..., arch=...)
    at source/backends/ubuntu/ubupkgindex.d:60
#13 0x00000000004ec834 in engine.Engine.seedContentsData(config.Suite, immutable(char)[], immutable(char)[]) (
    arch=..., section=..., suite=..., this=0x7ffff7ed07c0) at source/engine.d:177
#14 engine.Engine.run(immutable(char)[]) (this=<optimized out>, suite_name=...) at source/engine.d:481
#15 0x00000000004e45f0 in D main (args=...) at source/app.d:118

It looks like something is blowing up when trying to resolve a DNS name. curl is supposed to be thread safe, so hopefully it's not that. Any ideas?

	// filter out icon filenames and filenames of icon-related stuff (e.g. theme.index)
	auto iconInfo = appender!(string[]);
	foreach (ref c; contents) {
	if ((c.startsWith ("/usr/share/icons/")) \|\|
	(c.startsWith ("/usr/share/pixmaps/"))) {
	iconInfo ~= c;
	}
	}

ximion / appstream-generator Goto Github PK

appstream-generator's People

Contributors

Stargazers

Watchers

Forkers

appstream-generator's Issues

A different font works

Recommend Projects

Recommend Topics

Recommend Org