This is a discussion issue to figure out how to make the combination of Rust macros and module system compatible with IDE requirements.
Let's start with formulating a specific narrow task:
maintain a "tree of modules" data structure for a package
Specifically, this data structure should handle goto definition on super
in super::foo
, on foo
in self::foo::bar
and to find the root module for a given file.
To start, let's pretend that macros in Rust do not exist, and solve the problem in this setting. This is how ModuleMap datastructure works.
It maintains a set of links, where each link is a pair of module declaration (mod foo;
) and a file to which this declaration resolves two. To serve "go to parent module", the links are indexed by destination. To serve "go to child module", the links are indexed by source and name, to serve "find crate root", "go to parent" is applied repeatedly (which is ok, b/c crates are shallow).
The main property which makes this data structure "IDE-ready" is that invalidation is cheep: there's roughly a constant amount of work to update a data structure after a single file is changed, independent of the size of the package. When a file is added/deleted, only links which could point to it are updated (based on file_stem), when a file is changed, only links which originate from this file are invalidated.
Another nice property of the macro-less setting is that module tree is isolated to a single package: changes in upstream and downstream crates do not affect module tree at all.
Now, when macros enter the picture, everything becomes significantly more complicated. First of all, macro interfere with name resolution, so use statements become to matter, and upstream crates become to matter. Second, we really need to expand all macros to handle everything correctly. (consider println!("{}", { #[path="sneaky.rs"] mod foo; 92})
).
This last point is worth emphasizing, because it makes the complexity of reacting to modifications pretty bad for IDEs. Consider, for example, this sequence of events:
- user comments out
extern crate failure;
at the crate root
- user invokes goto definition on some
super
.
To handle the goto request, we need to reexpand all failure macros in the whole crate, and that's O(N)
work for O(1)
modification. What makes this feel wrong is that although any macro expansion could affect module tree, in reality almost none do. That is, for IDE-ready macro expansion the core requirement I think is
Do not expand macros in function bodies unless doing analysis of the function body itself
This model breaks for two reasons, one of which is mod decls with #[path]
attribute, another is impls (a macro-generated impl inside function body is globally visible). I wonder if there's a lanauge-level solution here? Could we require (via a warn lint) to annotate such globaly-visible macro invocations/declarations inside function bodies with #[global_effects]
or something?
If we indeed restrict ourselves to expanding only module-level macros, than I think the original macro-less solution could work reasonably if we model macro expansion as addition of new files?