Feature: install_* to respect path of files about meson HOT 16 CLOSED

Why was this closed? install_subdir is just a work around. nobase is a feature that should be implemented. I don't want Meson flattening my headers on install. It is currently throwing off my entire library because the headers themselves use the nested paths of my project, but when they install, they are flat and thus the includes don't resolve. This is a huge bug in Meson in my opinion. Following the directory structure should be the default. Please reopen the issue.

My header structure looks like this

.
├── auth
├── client
│   └── objects
│       └── harvest-client.h
├── company
├── estimate
│   └── objects
│       ├── harvest-estimate.h
│       ├── harvest-estimate-item-category.h
│       └── harvest-estimate-line-item.h
├── expense
├── invoice
│   └── objects
│       ├── harvest-invoice.h
│       ├── harvest-invoice-item-category.h
│       └── harvest-invoice-line-item.h
├── project
│   └── objects
│       └── harvest-project.h
├── role
├── shared
│   ├── harvest-api-client.h
│   ├── harvest-common.h
│   ├── harvest-error.h
│   ├── harvest-pageable.h
│   ├── objects
│   │   └── harvest-creator.h
│   ├── requests
│   │   └── harvest-request.h
│   └── responses
│       ├── harvest-links.h
│       └── harvest-paged.h
├── task
│   └── objects
│       ├── harvest-task-assignment.h
│       └── harvest-task.h
├── timesheet
│   ├── bodies
│   ├── objects
│   │   └── harvest-time-entry.h
│   ├── params
│   └── requests
│       └── harvest-late-request.h
└── user
    └── objects
        ├── harvest-user-assignment.h
        └── harvest-user.h

After a meson install, the lib looks like this

.
├── harvest-api-client.h
├── harvest-client.h
├── harvest-creator.h
├── harvest-estimate.h
├── harvest-estimate-item-category.h
├── harvest-estimate-line-item.h
├── harvest-invoice.h
├── harvest-invoice-item-category.h
├── harvest-invoice-line-item.h
├── harvest-late-request.h
├── harvest-links.h
├── harvest-pageable.c
├── harvest-paged.h
├── harvest-project.h
├── harvest-request.h
├── harvest-task-assignment.h
├── harvest-task.h
├── harvest-time-entry.h
├── harvest-user-assignment.h
└── harvest-user.h

This is a game breaking bug. If some third-party wanted to make use of my lib, their project wouldn't even compile.

from meson.

I think there should be two ways of install headers, similar to what described here.
nobase_headers

Currently we have install_subdir as a workaround, but it is not obvious that it should be used to install header if I am not looking at this github issue.

from meson.

The main reason for nobase based on that page is that it cuts down on recursive Make invocations. Meson does not have this issue, you can go to subdirectories all you want without any performance penalties. Unless your header directory is huge, just doing that seems to be the simpler approach.

from meson.

The primary use case for me would be installing a tree of headers in a single invocation, by generating the list of headers from an external script while respecting the paths to the files, so they get installed in the right places.

from meson.

Could you give an example of what you are trying to do? Specifically how the tree that holds your headers looks like and how you would expect to look like once installed.

from meson.

Directory structure:

README
src/
    mylib.cpp
include/
    mylib/
        component.hpp
        component/
            sub-component.hpp
        other-component/
            sub-other-component.hpp

Meson install fragment, invoked using subdir inside the include directory:

install_headers(run_command('find -name *.hpp').stdout().strip().split(), nobase: true)

So that the find command returns:

mylib/component.hpp
mylib/component/sub-component.hpp
mylib/other-component/sub-other-component.hpp

Which are installed in that structure in /usr/include.

from meson.

run_command('find -name *.hpp')

Is this actually working? Meson should run that as a single command (basically "find -name *.hpp" which would lead to a missing command failure. It also quotes every argument so the star does not get expanded. If you want to do globbing you need to do it in a script. (Yes, this is inconvenient but is you want to be reliable over multiple platforms, this is the only way do do it.)

That being said the way to do this currently is to have a script that does the globbing, so something like this at the top level

globber = find_program('myglobber.sh')

. Then you would put something like the following in you Meson files in the include directory.

subdir('foo') # if required
install_headers(run_command(myglobber).stdout().strip(), subdir : 'thissubdir')

Preserving the directory structure seems sensible, though. I'll need to think about what would be the best way to handle this.

Note also that if you just want to copy a large subtree, the simplest approach is to write a custom script to run at install time and just do 'cp -r' there.

from meson.

@jpakkane that was a quick example off the top of my head, I didn't expect it to work. It was just a demonstration of what I would like Meson to do. Respecting the full path to files is a very powerful feature for GNU Automake IMHO, and it'd be great to see Meson implement it also.

In-built globbing would be nice too, I understand that it wouldn't be fast, but then neither is any invocation of an external program, which Meson supports just fine.

from meson.

The FAQ explains quite thoroughly why Meson does not (and will not) provide globbing. tl/dr: it can't be made both reliable and fast.

from meson.

OK, perhaps globbing isn't what I meant. Meson attempts to install files regardless of if they already exist at the destination (sensible, since the files might be different), so having the ability to directly install an entire directory wouldn't slow it down at all, and it is just as reliable as installing individual files. It would also make a nobase option redundant (although it would still be nice to have it!)

from meson.

So you would want something like this:

install_subdir('dirname', install_dir : 'share/something')

to copy dirname and everything under it to ${prefix}/share/something?

from meson.

Yes, that would be a good feature to have. I can imagine there will be some cases where a nobase option for headers in particular would be useful, but most projects arrange their headers in the required hierarchy so an install_subdir() command would be suitable.

from meson.

My workaround:

# Install files from sources array
# Loop needed as meson doesn't preserve the path for some reason
# See: https://github.com/mesonbuild/meson/issues/3371
include_subdir = 'mylib'
foreach file : mylib_sources
    file_path = file.split('/')

    # Extract folder
    folder_path = []
    foreach path : file_path
        if path != file_path[-1]
            folder_path += path
        endif
    endforeach

    folder = join_paths(folder_path)
    install_headers(file, subdir: join_paths(include_subdir, folder))
endforeach

Probably not portable.

from meson.

I don't want Meson flattening my headers on install.

That sounds like a bug. I don't believe that's the intention of the call. Could you please open a new issue about that?

from meson.

@nirbheek I believe #3371 is about this behavior.

from meson.

This ticket was originally about having a nobase-style approach to the install_headers() function. Despite adding install_subdir, this did not actually answer the request, even if it could be used that way (in fact, many, many install_subdir uses have nothing to do with headers and people don't always realize it can be used for this).

It also doesn't work when there are some private headers and some public headers.

Anyway, it is now implemented in #10299 and will be available in the next version of Meson as install_headers('list', 'of', 'headers', preserve_path: true)

from meson.