linebender / xilem Goto Github PK
View Code? Open in Web Editor NEWAn experimental Rust native UI framework
License: Apache License 2.0
An experimental Rust native UI framework
License: Apache License 2.0
Trying to build an example you get this error
error[E0609]: no field `inner` on type `&AppRootInner`
--> /Users/weethet/.cargo/registry/src/index.crates.io-6f17d22bba15001f/masonry-0.1.2/src/app_root.rs:846:14
|
846 | self.inner.borrow().app.hide()
| ^^^^^ unknown field
|
= note: available fields are: `app_handle`, `debug_logger`, `app_delegate`, `command_queue`, `action_queue` ... and 8 others
I suppose this is due to maintainers not having macOS laptop, so it's hard to check, but maybe github actions could bes set up to prevent this any further
Masonry's pointer handling is inherited from Druid, and there are some gray areas that need to be better specified, documented and enforced.
Hot status now roughly means "This widget is the one that will receive clicks from the mouse".
There are two things we need to specify:
(1) means we either get hot state for only the primary pointer, or the last active pointer; or we get multiple hot states, and is_hot()
will take a pointer id as parameter, or we get hot state for every pointer without distinction of which pointer makes which widget hot.
(2) is especially annoying, because it breaks the unidirectionality of the "events -> layout -> paint" flow.
We may also want to rethink how hot status interacts with widgets overlapping one another; eg if we want a browser-like "mouseleave" event that doesn't trigger where the mouse goes from a container widget to one of its children.
See also: https://gist.github.com/PoignardAzur/c60cd72d790c083b7b55269bc91912db#file-hot_state-md
Currently, active is a flag which:
Aaaaand... that's it. Any other information it might represent (eg that it means "the user is in the middle of a mouse press on this widget") is purely down to widget implementation.
Ideally, I would like "active" to be better integrated in the framework:
We might even want to be more opinionated and have the framework guarantee that hot state and active state are correlated (eg when user clicks, active state is given to the hot widget).
See also: https://gist.github.com/PoignardAzur/c60cd72d790c083b7b55269bc91912db#file-active_status-md
See here:
Running on M1 Mac in release mode.
What's the performance bottleneck here? How can this be fixed?
Masonry should have infrastructure to measure various runtime performance characteristics.
Because it's a GUI app, profiling is a bit more complicated than "run program 100x times, measure average performance". What we'd want to do is measure things like average FPS count, 99% latency, startup times, etc.
We need to look up industry best practices for profiling GUI and, if necessary, make up our own. At the very least, we should have tooling built into the pass system (maybe using tracing
?) to measure the targets listed above, and easy ways to dump the results. We might even want a "dump some core indicators to stdout on program exit" feature to be enabled by default in debug mode, to encourage thinking about these indicators.
We should produce some benchmarks; again, we may want to both consider the state of the art (eg js-framework-benchmark) and make our own.
I'm especially interested in cases that stress-test Masonry: lists with millions of items, a widget hierarchy which is thousand of nodes deep, ridiculously high resolutions, etc.
For all those cases, we should consider not only the passive performance (eg how long does it take to render a frame), but also the performance given mouse events, keyboard events, etc.
When porting from Druid to Masonry, I've removed the ability to trigger commands from links.
While I'm not sure how to handle that feature on the long term, on the short-term it should be fairly easy to re-enable. Most of the code is still there, just commented out.
Essentially, we just want Label widgets to send Commands during MouseUp events when the cursor is over a link. Ideally we'd want to handle active status and cursor changes and stuff, but we'll keep it simple for now.
Masonry has pretty good unit testing tooling, but a lot of unit tests are still missing.
Ideally, we're aiming for 100% coverage (with opt-outs) for at least every widget and the WidgetPod code. Each widget should have tests covering:
WidgetMut
.The first step would be to go over the code (potentially using cargo tarpaulin
) and list all the places missing unit tests.
Virtually every module should be private. The only public modules should inline modules that gather public-facing re-exports.
Most items should be exported from the root module, with no other public-facing export. This makes documentation more readable; readers don't need to click on multiple modules to find the item they're looking for.
There should be only three public modules:
widgets
commands
test_widgets
Module files should not be foobar/mod.rs
. Instead, they should _foobar.rs
(thus in the parent folder); the full name is for readability, the leading underscore is so these names appear first in file hierarchies.
We should avoid use super::xxx
imports, except for specific cases like unit tests.
Most imports in the code base should use the canonical top-level import.
Masonry should have no prelude. Examples and documentation should deliberately have a list of imports that cover everything users will need, so users can copy-paste these lists.
In current code, calling declare_widget!(FoobarMut, Foobar)
will generate FoobarMut
as a tuple of a WidgetState
and a Foobar
. It should instead be a struct with these two fields, with names like "state" and "widget", for clearer widget code.
Hi! I wanna start contribute to xilem and wanted to know if there is a roadmap / board for what needs to be done and what would be easy and generally how to work with this code structure and what the goals of the project are. Also: is there a discord server? I read that there's a zulip thing but i never worked with it and it seems more like another issue tool instead of active communication
I was able to compile on my laptop, but both example apps fail to run. I could compile but not successfully run all the wgpu project tests. I was able to get some of the wgpu showcase apps to run e.g. the n-body simulation of galaxies. I was able to successfully compile and run the tests of piet-gpu.
Here's a stacktrace.
Running `target/debug/xilem counter`
MESA-INTEL: warning: Haswell Vulkan support is incomplete
size = 80.09600067138672Wร24.0H
layout size = 80.09600067138672Wร24.0H
render size: 1024.0Wร768.0H
thread 'main' panicked at 'Error in Surface::configure: surface does not support the adapter's queue family', /home/nmrp3/.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-0.14.2/src/backend/direct.rs:274:9
stack backtrace:
0: rust_begin_unwind
at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:584:5
1: core::panicking::panic_fmt
at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/panicking.rs:142:14
2: wgpu::backend::direct::Context::handle_error_fatal
at /home/nmrp3/.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-0.14.2/src/backend/direct.rs:274:9
3: <wgpu::backend::direct::Context as wgpu::Context>::surface_configure
at /home/nmrp3/.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-0.14.2/src/backend/direct.rs:1017:13
4: wgpu::Surface::configure
at /home/nmrp3/.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-0.14.2/src/lib.rs:3715:9
5: piet_wgsl::util::RenderContext::create_surface
at /home/nmrp3/.cargo/git/checkouts/piet-gpu-adec1df56a2c5942/5718222/piet-wgsl/src/util.rs:70:9
6: xilem::app_main::MainState<T,V>::render
at ./src/app_main.rs:214:33
7: <xilem::app_main::MainState<T,V> as glazier::window::WinHandler>::paint
at ./src/app_main.rs:107:9
8: glazier::backend::x11::window::Window::render::{{closure}}
at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/window.rs:568:13
9: glazier::backend::x11::window::Window::with_handler_and_dont_check_the_other_borrows
at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/window.rs:505:31
10: glazier::backend::x11::window::Window::render
at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/window.rs:567:9
11: glazier::backend::x11::window::Window::redraw_now
at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/window.rs:673:9
12: glazier::backend::x11::window::Window::run_idle
at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/window.rs:932:29
13: glazier::backend::x11::application::Application::run_inner
at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/application.rs:700:25
14: glazier::backend::x11::application::Application::run
at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/application.rs:710:25
15: glazier::application::Application::run
at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/application.rs:150:9
16: xilem::app_main::AppLauncher<T,V>::run
at ./src/app_main.rs:90:9
17: xilem::main
at ./src/main.rs:19:5
18: core::ops::function::FnOnce::call_once
at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
and some vital stats:
$ cargo --version
cargo 1.65.0 (4bc8f24d3 2022-10-20)
$ uname -a
Linux persephone 5.4.0-131-generic #147-Ubuntu SMP Fri Oct 14 17:07:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
$ prime-select query
nvidia
$ sudo lshw -class display
[sudo] password for nmrp3:
*-display
description: 3D controller
product: GM108M [GeForce 840M]
vendor: NVIDIA Corporation
physical id: 0
bus info: pci@0000:01:00.0
version: a2
width: 64 bits
clock: 33MHz
capabilities: pm msi pciexpress bus_master cap_list rom
configuration: driver=nvidia latency=0
resources: irq:35 memory:f6000000-f6ffffff memory:e0000000-efffffff memory:f0000000-f1ffffff ioport:e000(size=128) memory:f7000000-f707ffff
*-display
description: VGA compatible controller
product: 4th Gen Core Processor Integrated Graphics Controller
vendor: Intel Corporation
physical id: 2
bus info: pci@0000:00:02.0
version: 06
width: 64 bits
clock: 33MHz
capabilities: msi pm vga_controller bus_master cap_list rom
configuration: driver=i915 latency=0
resources: irq:34 memory:f7400000-f77fffff memory:d0000000-dfffffff ioport:f000(size=64) memory:c0000-dffff
The following tests are failing for me:
widget::align::tests::centered
widget::align::tests::left
widget::align::tests::right
widget::button::tests::simple_button
widget::checkbox::tests::simple_checkbox
widget::flex::tests::flex_col_cross_axis_snapshots
widget::flex::tests::flex_col_main_axis_snapshots
widget::flex::tests::flex_row_cross_axis_snapshots
widget::flex::tests::flex_row_main_axis_snapshots
widget::label::tests::line_break_modes
widget::label::tests::simple_label
widget::label::tests::styled_label
widget::portal::tests::button_list
widget::sized_box::tests::label_box_no_size
widget::sized_box::tests::label_box_with_size
widget::split::tests::columns
widget::split::tests::rows
widget::textbox::tests::simple_textbox
And the .diff.png
files all only show the text, e.g:
(I'm on Debian.)
Some dependencies can probably be removed with minimal effort to reduce build times / sizes:
An important step in building out the widget tree is to get the trait right. Further iteration is possible, but getting it basically right at this time will save hassle later.
The high-level approach is to start with the Druid Widget trait and make some changes.
The data
argument goes away (as does being generic on T
). This is a natural consequence of the Xilem architecture.
The env
argument goes away, I think. This is actually a much bigger question, as we rely on env to retrieve style data, so probably deserves some thought about what will replace that.
The paint
method evolves to generate a piet-gpu scene fragment rather than drawing into a Piet render context.
The trait also needs to update the accessibility tree. In the prototype Druid integration for AccessKit, there an accessibility()
method that generates the node, but that's based on generating the entire tree every time rather than doing incremental update. This needs to be worked out in detail.
As discussed in today's office hours, we will not be making significant changes to layout at this time, so those methods will basically be the same as Druid's existing Flutter-like mechanism.
update
methodRight now the update
method isn't doing much, and if #6 gets adopted, it will do even less. In current Druid, one of its other roles is to respond to env
changes (if theme or styling changes), but that may well go away as well.
A potential role may be to update the accessibility tree, but likely that should be a separate method, as it should be lazy (only called when a screen reader is connected).
So one of the questions which needs to be decided is whether to keep the update
method or remove it. An excellent way to help resolve it is to go through the existing use cases in Druid and see if any really motivate keeping it.
lifecycle
methodWe can also consider combining lifecycle
and event
, as the motivation for having them separate may no longer be compelling. I consider this a lower priority, as I'm sure it's fine either way, but again if we do decide to change this from Druid, it would be less work to do it now than later.
Follow the instructions to make a simple demo trainer, but there is a compilation error. Is there any information you can provide, thanks!
E:\MyProjects\rust-app>cargo build
Updating tuna
index
Compiling masonry v0.1.2
error[E0609]: no field app
on type RefMut<'_, AppRootInner>
--> C:\Users\liaoxuewei.cargo\registry\src\mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd\masonry-0.1.2\src\app_root.rs:217:19
|
217 | inner.app.quit();
| ^^^ unknown field
For more information about this error, try rustc --explain E0609
.
error: could not compile masonry
due to previous error
Masonry introduces a lot of complex concepts compared to other GUI frameworks. Many of these concepts are intended to reduce future complexity, but crate users don't get those benefits if they can't use the API in the first place.
Masonry should have a tutorial integrated directly into the documentation with rustdoc.
The tutorial should have the following chapters:
create_widget
macro, how to do tests.Other parts of the documentation should refer to these tutorials.
Widgets should be able to require paint-only animations, eg animations that don't change their layout.
When dispatching animations, you could then skip offscreen widgets. Animation event should include some kind of timestamp, so that widget that skip animation frames can display the correct contents.
Right now the framework only gives a children_changed()
method when a Widget has changed its data in a way that adds or removes children.
Instead, it should provide three methods:
child_added()
child_stashed(bool)
child_removed()
These methods should update semantic information in the widget tree, eg:
(Obviously, we'll need to add lots of unit tests for all that)
Hi - I read the blog post and am interested in the concept. I was wondering if you had some kind of roadmap? Things look sufficiently in flux that I suspect you are not ready to accept random contributions at this time?
The Julia ecosystem has a number of grid layouts optimised for charts and plots with titles, axes and so on. It has an extremely stateful and side-effect-driven API, but the underlying logic seems sound and results in very nice and flexible placement of elements. Once the widget set and layouts for this toolkit mature a bit, it would be nice to explore these kinds of layouts. They are more generally applicable than plotting. For example, flexibly attaching labels to input widgets while maintaining good alignment.
For example:
https://docs.juliahub.com/AbstractPlotting/6fydZ/0.14.1/makielayout/grids.html
https://docs.juliaplots.org/latest/layouts/
The TestHarness type aims to emulate an actual GUI environment with as much fidelity as it can while staying performant. However, some of its internal abstractions are still a bit leaky, and should be improved:
edit_root_widget
method should perform a downcast internally and give the WidgetMut to the correct widget type.move_timers_forward
method should also move animations forward.The project needs a logo, for all the usual reasons (to stand out, to give a professional vibe, to give a visual association that people will remember, etc).
I'm thinking the logo should probably be very simple, something like a SVG brick wall within a Windows-95-style window, maybe.
EDIT - Something like this:
Hi there! Really interesting project :) I wanted to try out the examples locally, but failed to compile them.
The scroll
example requires hex
and sha2
, and the counter
example can't resolve various imports from xilem
.
I think you can have a lot of power if you can implement only a subset of the DOM drawing and have a typescript/js interpreter in RUST that can leverage the xilem primitives.
Text handling right now is a huge mess inherited from Druid. Even the original authors of that code are a bit hazy on what some parts of the code do.
The text handling code should be refactored completely.
I was experimenting with xilem (see #14 (comment)) and noticed that the memory and CPU consumption of xilem was exceptionnally high:
Now, I'm totally aware that xilem is in the experimental state, but surely it shouldn't ever consume that much, right?
I didn't properly investigate the CPU consumption yet, but as for the memory consumption, I used heaptrack to do some heap profiling. It's the first time I do something like this so I was quite amazed at how quickly I arrived to a conclusion:
There are 2 huge culprits of memory consumption, one taking ~874 MB and the other taking ~894 MB. It turns out, both come from parley::FontCache::new()
.
I looked at the code inside parley, and it seems that it's loading every single font from my system into memory, which is less than ideal...
I'm of course going to forward this issue to parley, but in the mean time, is there any simple workaround for xilem so that it doesn't take 2 gigs of ram just for experimentation?
Here's the .zst
file that you can open with heaptrack_gui
(I think it only works on Linux).
heaptrack.xilem.44968.zip
Portals are widgets that are meant to display a view to a bigger area. They're used for Scroll widgets.
They are still conceptually unclear. We need to figure out what their semantics are and how they interact with other concepts.
Portal
should probably be renamed to ScrollPortal
for clarity.register_as_portal
(the fact that it's only used by a single widget is a bit of a code smell).Scroll
widget needs mutable access to its child Portal
when computing layout, which kinda breaks the core abstraction.Masonry should have infrastructure similar to rustc's to measure the evolution of build times.
While at the moment I'm writing this, short build times are far from a priority, on the long term they will be an essential part of the development experience. Short build times mean short edit-build-get-feedback cycles, something we're definitely aiming for.
Some indicators to consider:
IME events need to be reworked both in Masonry and in Glazier.
The current workflow is extremely complex, in part due to the inherent complexity of IME. However, I suspect it would be possible to make it simpler in some ways. In particular, I think it might be possible to pass glazier::text::Action
directly to widget events.
Also, it should be possible for WinHandler::acquire_input_lock
to return &dyn InputHandler
instead of a Box.
I think a derive
macro for the view trait, especially on enums, would really benefit ergonomics.
Views that return multiple types on a condition, such as an if
block or match
statement, could instead return an enum with View
derived.
For example, for conditional views we could return a custom enum like:
#[derive(View)]
enum UserForm {
Login(LoginView),
SignUp(SignUpView)
}
The proposals here came from me looking into what it would take to integrate Taffy layout into Xilem. But nothing proposed here is really specific to the CSS style layout modes (Flexbox and CSS Grid) that Taffy implements. Nor would they commit Xilem to CSS style layout. Rather, I believe they would enable Taffy layout modes to implemented in Xilem as widgets (which could live in an external crate), in much the same way that the existing Flex
widget is implemented in Druid.
I suspect that we could make a much more streamlined system if support for associating arbitrary data (e.g. "styles") with elements such that a parent widget could access them on a child widget the chidl widget having to add support for them was implemented (ala linebender/druid#2207). But that's a much more significant change, which I think can wait.
I have also written a prototype integration of Taffy with Iced (Iced also uses a similar layout mechanism to Druid and Xilem). And despite having to work around some limitation of Iced's system (like no measure
method, and layout
taking &self
rather than &mut self
), the integration actually ended up being relatively straightforward (you can see the implementation of Iced's layout
method here (calling into Taffy from Iced), and the implementation of Taffy's perform_child_layout
method here (calling back into Iced from Taffy)).
Here I lay out the state of things as they are in Xilem, Druid, and Taffy.
A Size<T>
in Taffy is defined as:
struct Size<T> {
width: T,
height: T,
}
A Size
in Druid/Xilem (kurbo) is a Size<f64>
using the above definition. For the remainder of this post I will translate this to Size<f64>
in the function signatures below for clarity.
A BoxConstraints
in Druid is defined as:
struct BoxConstraints {
min: Size<f64>,
max: Size<f64>,
}
An AvailableSpace
in Taffy is defined as:
enum AvailableSpace {
MinContent,
MaxContent,
Definite(Size<f32>),
}
A SizingMode
in Taffy is defined as:
enum SizingMode {
ContentSize, // Size ignoring explicit styles
InherentSize, // Size including explicit styles
}
/// Compute intrinsic sizes.
/// The returned sizes are (min_size, max_size)
fn measure(&mut self, cx: &mut LayoutCx) -> (Size<f64>, Size<f64>);
/// Compute size given proposed size.
fn layout(&mut self, cx: &mut LayoutCx, proposed_size: Size<f64>) -> Size<f64>;
/// Max intrinsic/preferred dimension is the dimension the widget could take, provided infinite constraint on that axis.
/// Intrinsic is a *could-be* value. It's the value a widget *could* have given infinite constraints. This does not mean the value returned by layout() would be the same.
/// This method **must** return a finite value.
fn compute_max_intrinsic(
&mut self,
ctx: &mut LayoutCtx,
axis: Axis,
bc: &BoxConstraints,
data: &T,
env: &Env,
) -> f64
/// For efficiency, a container should only invoke layout of a child widget
/// once, though there is nothing enforcing this.
fn layout(
&mut self,
ctx: &mut LayoutCtx,
bc: &BoxConstraints,
data: &T,
env: &Env
) -> Size;
fn measure_size(
tree: &mut impl LayoutTree,
node: Node,
known_dimensions: Size<Option<f32>>,
parent_size: Size<Option<f32>>,
available_space: Size<AvailableSpace>,
sizing_mode: SizingMode,
) -> Size<f32>
fn perform_layout(
tree: &mut impl LayoutTree,
node: Node,
known_dimensions: Size<Option<f32>>,
parent_size: Size<Option<f32>>,
available_space: Size<AvailableSpace>,
sizing_mode: SizingMode,
) -> Size<f32>
There are a few difference which look like they might be important, but I suspect that they are actually not:
f64
and Taffy uses f32
. Perhaps @raphlinus can comment on if/why he thinks f64
is needed, but in any case we can trivially convert between the two types (accepting the loss of precision), or if it came to it, it would be simple enough (if verbose) to extend Taffy to work with f64
. I'm going to use f32
everywhere for the remainder of this post, but it could just as easily be f64
.Option<f32>
where Druid/Xilem just use f32
. However, where Taffy uses Option::None
to represent an infinite/unset size (never using f32::INFINITY
) Druid/Xilem use f32::INFINITY
to represent this case. Again, this is a trivial conversion that could easily be handled as part of a widget or similar.data
and env
parameters which provide extra data. I don't quite understand env
, but I think it's some kind of context. We will need something like that in Taffy at some point for allowing styles (particularly things like writing mode / direction) to inherit down the tree. But for now, I think we can ignore this.tree
parameter which provides access to style information, the ability to request that children size themselves, and the ability to store the final computed size and position of nodes. I think this can all be handled by the widget implementation, so again we can ignore this (although this is one place where we might later get nicer DX with tigher integration with Xilem).layout
function that implements a full layout of that node and all children and returns a Size<f32>
(modulo the aforementioned f32/f64 difference). Druid suggests that this should only be called once (but that this isn't actually enforced). Taffy only does call this method once in the usual case, but may need to call it multiple times to support baseline alignment (only if baseline alignment is actually used in the layout).measure
function (this is compute_max_intrinsic
in Druid) which allow child nodes to compute their intrinsic (content) size(s) under the provided constraints and hints and return it to parent nodes. However, they all work slightly differently:
measure
function returns:
Size
)min
and max
sizes (as a tuple)measure
function:
Size
)min
or max
size depending on the available_space
parametercompute_max_intrinsic
function
max
size. Druid has no concept of a min content size.I would suggest that the concept of a "min content size" is important and should definitely be included. I would also suggest that the function should not compute both the min
and max
sizes at once as Xilem currently does, as this could be expensive (e.g. for a text node) and at least for CSS layout it's relatively common that only one of the sizes is required.
Whether both axis are computed together or seperately I don't have too strong an opinion about. Taffy layout modes would probably compute both either way and cache the other one the other one for future queries. Update: I now believe that the single-axis-at-a-time model is superior.
Constaint parameters have a direct relationship with the returned size and must be respected by nodes' measurement/layout functions (and/or the sizes returned will be ignored/clamped if they are not).
Size<Option<f32>>
) - it is often the case that a parent node wants to ask a child node to size itself in one dimension while treating the other dimension as fixed, effectively asking the child a question like "suppose your width is 100px, what would your height be?" (perhaps the width has already been determined in an earlier part of the algorithm). This parameter provides a way to specify those fixed dimensions.BoxConstraints
). Druid's box constraints offer a strict superset of this functionality (setting min
=max
=some finite number
in a given dimension is equivalent to setting the known dimension in that dimension; setting min=0
, max=Infinity
is equivalent to not setting the known dimension in that dimension). The max contraint also seems useful in it's own right. It makes sense to ask a node to size itself within a certain bounding box. Taffy has it's own version of this in the available_space parameter
Hint parameters provide extra information that nodes may use to help choose their size. These are merely hints and may be ignored in some cases. But will likely be very helpful to allow the parent and child node to cooperatively choose a good size.
Size<f32>
): Xilem uses a proposed_size: Size<f32>
parameter, which seems to be used primarily by the v_stack
component which sizes children in order and uses proposed_size
to pass "remaining available space" which is equal to it's own proposed_size
minus "the size of any already sized children" minus "spacing between children". I would suggest that this is replaced by a more general available_space
parameter (see below).parent_size
is a finite definite pixel size. But if the parent size is unknown then this enum carries an additional hint: whether the content based size should be a "min content" or a "max content" size. Taffy doesn't have an hstack/vstack-like layout, but I think this parameter would be a good place to pass the "remaining available space" that Xilem's current v_stack widget calls proposed_size
(in this case available_space
would differ from parent_size
). I think this is useful and should be kept, however I think it is potentially confusing to couple the min/max content sizing hint with this size, so I suggest that we split this into a seperate enum parameter.The following type definitions are used in the propsoal below:
struct Size<T> {
width: T,
height: T,
}
struct BoxConstraints {
min: Size<f32>,
max: Size<f32>,
}
enum RequestedSize {
MinContent,
MaxContent,
FitAvailableSpace,
}
I propose that the Xilem widget trait has the following two methods for layout, replacing the existing layout
and measure
methods:
fn measure(
&mut self,
box_constraints: BoxConstraints,
parent_size: Size<f32>,
available_space: Size<f32>,
requested_size: Size<RequestedSize>,
axis: Axis,
) -> Size<f32>;
fn layout(
&mut self,
box_constraints: BoxConstraints,
parent_size: Size<f32>,
available_space: Size<f32>,
requested_size: Size<RequestedSize>,
) -> Size<f32>;
I believe this would provide a strong framework within which lots of powerful layout paradigms could be implemented. But I'm sure I haven't thought of everything and feedback and discussion is of course enouraged!
I have several issues with how layout works in Druid (and thus currently in Masonry).
I think this is more due to the Flex algorithm than Druid's specific implementation. But either way, it has quite a few footguns:
box_constraints.max()
as their size, even though the Scroll
widget will give infinite max constraints to its children.Flex::row(child1, child2)
and Split::columns().with_flex_child(child1).with_flex_child(child2)
render differently, even though they conceptually do the same thing.Split
widget) will grow enormous, even having a button take half your screen makes no sense visually..center()
to your widget will change not only its position, but also its size. Eg in my_widget.center().expand()
, my_widget doesn't have the same size as in my_widget.expand()
..expand()
to a flex container does nothing, because .expand()
works by passing bigger minimum-size constraints to its child, and Flex calls .loosen()
on all its children.(note that some of these examples don't quite translate to Masonry because Masonry does not use WidgetExt
, but the general idea is the same)
Overall, the Flex algorithm doesn't seem to be "continuous" enough. That is, local changes to the code may create non-local changes to the UI; sometimes changes may have no effects or invisible effects.
Ideally, I'd want the layout algorithm to be discoverable. The default case should give sensible good-enough UI, and it should be possible to implement common patterns by combining layout primitives with little guessing.
The base of the layout algorithm is this: each widget is given size constraints (a minimum and a maximum size) and has to return its size.
Currently, there is no specification as to what happens if the returned size doesn't match these constraints. There's also no specification for what happens if a widget passes invalid constraints to its children (NaN or negative values or something).
Which means that in practice, what happens is either "nothing" or "graphical bugs".
We might want to emit warnings in those cases. Though it's always possible that size constraints aren't respected because the user made the window too small or something similar, in which case warnings would just be visual pollution. But we do want a way to detect it when it happens.
We might also want a mechanism for detecting when creating a layout is impossible. For instance, if we have a box with 16px of padding, and only 10px of free space, there is no way to fit that box's contents in the given space. We could say layout()
returns an Option<Size>
and returns a None
when layout is impossible.
The first step would be to work the concept of "optional layout" into the architecture. Widgets may have optional positions, and widgets with None positions would be marked as un-placed. (Not sure how clear that would be when debugging)
We should rework and/or remove these widgets:
Finally, we need to come up with better geometry primitives to represent UI layout and avoid doing math operations "by hand".
Right now the assert_render_snapshot!
macro and the underlying code are held together with duck-tape.
Some possible improvements:
cargo-insta
style UI tool that shows you the image and asks you to validate one or the other.For example, if a button has a boolean state of pressed
and I have a lot of them, do they have to become part of the global state now?
If so I think some way of combining the data tree with the widget tree could make this more flexible
I suspect this is not intended behavior? All hover state should be discarded on cursor exiting the window.
Vello and Glazier are two successor projects to dependencies Masonry uses, respectively Piet and druid-shell. Both of these dependencies are unlikely to to be maintained in the near future (especially druid-shell), while Vello and Glazier are actively worked on by the Druid community.
As both these projects have APIs very similar to the ones they're succeeding, the switch should be a relatively straightforward affair (famous last words). It should not impact the semantics of Masonry.
Note: If I understood correctly, 2193b87 is where this was implemented in Xilem.
Some things to improve:
FnMut
taking the same parameters as that method.Is it possible to add functionality to send a signal to a list view that the underlying data has changed, like for example that some rows were added, deleted, or changed? In addition to the straightforward use, this would allow trees to be implemented on top of lists: deleting rows can be used when collapsing, and inserting rows when expanding.
The Env class and the Data trait are vestigial by this point in the Masonry codebase.
There is currently no support for updating widgets when the Env is changed, and I have no intention of adding that support (Xilem seems to be moving in the same direction). This means Env is basically just a way to pass a singleton around in Widget code.
We should remove Env and data, and just hardcode the values that we currently get from Env, with future plans to pass them as arguments or something. Frontend frameworks might still use something Env-like to get style data, but that'll be their responsibility.
WebImage was removed for a few reasons:
reqwest::blocking::get(url)
in a background thread, which isn't ideal for a few reasons:ImageBuf::from_data
, which caused some build problems because that method wasn't reliably generated depending on feature flags.I would like to add that type back eventually, with a few caveats:
reqwest::blocking
.I want to make some pretty deep changes to the Widget trait. The ultimate trait would look something like this:
pub trait Widget2: Any {
fn on_event(&mut self, ctx: &mut EventCtx, event: &Event);
fn lifecycle(&mut self, ctx: &mut LifeCycleCtx, event: &LifeCycle);
fn layout(&mut self, ctx: &mut LayoutCtx, bc: &BoxConstraints) -> Size;
fn compute_max_intrinsic(&mut self, axis: Axis, ctx: &mut LayoutCx, bc: &BoxConstraints) -> f64;
fn accessibility(&mut self, cx: &mut AccessCx);
// Maybe some other arguments after the switch to Vello
fn paint(&mut self, ctx: &mut PaintCtx);
fn children(&self) -> SmallVec<[WidgetRef<'_, dyn Widget>; 16]>;
// Got to figure out what argument the callback takes
fn call_on_children(&mut self, callback: impl FnMut(&mut impl Widget));
fn get_debug_text(&self) -> Option<String>;
// --- Auto-generated implementations ---
fn make_trace_span(&self) -> Span;
fn get_child_at_pos(&self, pos: Point) -> Option<WidgetRef<'_, dyn Widget>>;
fn get_child_with_id(&self, id: WidgetId) -> Option<WidgetRef<'_, dyn Widget>>;
fn call_on_child_at_pos(&mut self, pos: Point, callback: impl FnOnce(&mut impl Widget));
fn call_on_child_with_id(&mut self, id: WidgetId, callback: impl FnOnce(&mut impl Widget));
fn type_name(&self) -> &'static str;
fn short_type_name(&self) -> &'static str;
}
Notable changes:
Not sure about this one.
The requirement to call ctx.init()
before any method call for all methods of SomethingCtx types was added to prevent the case where someone would get a widget ref and call Widget::some_method()
on it instead of calling WidgetPod::some_method
, which would be slightly buggy.
I'm not sure it's necessary anymore. Removing WidgetPod::widget_mut
might be enough to remove all instances where that problem existed. And the ergonomic cost of panicking if you don't add ctx.init()
everywhere is pretty huge.
One thing that came up in the review of #26 is that the Widget
trait now depends on Message
, which is conceptually at the Xilem level. That's arguably a layering violation, and may make life more difficult for other possible reactive layers on top of the widget set.
The most flexible thing to do would be to make the Widget
trait generic on the message type, but that would add a lot of type noise. The Message
type is perhaps an acceptable type to bring into the widget layer, as it's basically just a Vec<Id>
and a Box<dyn Any>
.
Part of the reason I'm writing this issue is to get feedback about other potential uses of the widget layer. Is just having widgets produce Message
good enough, or do we really need the type to be more flexible? Accommodating other reactive layers than Xilem feels a bit YAGNI right now, and perhaps we want to do a more careful review of layering if and when we get to that point.
The AppRoot
is the composition root of everything in a Masonry program; everything passes through an AppRoot instance.
AppRoot instances each own an AppRootInner, which it needs to lock to perform operations. However, some AppRoot methods may call WindowHandle methods which themselves may end up indirectly calling AppRoot methods; thus, these methods should be re-entrant.
Which methods need to be re-entrant and why should be documented (which will probably require familiarity with Glazier code).
I'm on Ubuntu 22.10 (wayland) and after cargo build I get an error about missing cairo libraries.
The event model of Masonry still needs a lot of improvement.
(I'm including lifecycle here. Lifecycle events are kept separate from Events, but it's unclear if they're actually different, and the code handling them is mostly the same)
Container widgets are expected to call the event method on each of their children. This often doesn't make sense because an event will be targeted directly to a specific widget, and most of those children don't need to be called. And this creates fragility if containers forget to recurse in some situations.
The WidgetPod has a lot of special-case machinery to handle specific events, which makes it pretty hard to read and understand.
Commands, Notifications and Promises are integrated into events, and have their own problems:
Widgets can also send Actions. Actions are a centralized queue of messages intended for the root application, mostly for a Panoramix-like framework to handle. They are inspired by Facebook's Flux architecture. Each Action represents a semantically meaningful event from the UI (eg "button clicked").
Actions are currently under-designed. Right now, they only store the emitter's id, plus a payload among a small set of types with Box<dyn Any>
as a wildcard. Ideally, I'd want a way to connect the type of an action from the widget that sent it, so that each widget would have an associated action type. This should probably be co-designed in parallel with Panoramix.
Container widgets should implement a recurse()
method that would call an FnMut param on each of their children. They could also have specialized recurse_at_pos
and recurse_at_id
methods for better targeting.
Instead of the workflow being "WidgetPod::on_event
calls Widget::on_event
calls WidgetPod::on_event
calls Widget::on_event
etc...", the workflow would be more like "framework calls Widget::recurse
which calls Widget::recurse
etc... all of which call Widget::on_event
. Widgets would never be expected to call WidgetPod::on_event
.
Some events might still follow a browser-like model where parent widgets can intercept events from their children; if we implemented that, it would have to be explicit intercepting, not just the parent choosing not to call .on_event
on a child.
Lifecycle events might be merged with regular events, unless they prove to have a distinct niche.
Users should be able to send commands from any Widget and from Delegates, to any other Widgets, to windows, and to delegates (?). Delegates should also be able to intercept notifications (?).
Target::Auto
should be removed.
Dialogs, especially platform-created file pickers, are a feature from Druid that was ripped out when I wrote Masonry and haven't added back in yet.
Part of the code is still in the codebase, commented out, taunting me.
Ideally, I would want the Masonry implementation to use the promise feature, but this is a more complicated task than it might appear.
The goal of debug_logger is to serialize the state of the widget tree in real time, to a stream that can be read by an external debugging application which can inspect the stream and display what the widget tree looked like at a given time, and inspect internal values and stuff.
While said debugging application has proven extremely invaluable for me for debugging masonry, I don't expect anybody else to use it, as its UX right now is absolutely abysmal. I do intend to improve it eventually.
In the meantime, debug_logger is a pretty clunky, unwieldy tools. Among other problems:
Now, I think debug_logger is an idea that has some real potential. In my ideal vision, it would be a record-replay framework of sorts, one that would produce a lightweight record of the widget tree and application state over time. It would probably use the tracing crate as the serialization layer, and it would do some fancy caching to only record changed information to avoid unnecessary overhead.
I need to write a full design document for that ideal logger, and then get started on implementing it.
I cloned the repository and attempted to run the default main application as a sanity check for myself, however my screen is presented with an empty button.
The only notable thing I could deduce while poking around the source code, is that the if let Some(fragment)
statement on text.rs:40
always fails, and nothing is ever appended to builder. Could it be a failure in default font discovery?
My system is Linux Mint, running x11 with i3 as my WM and Plasma as my DE.
Looking at the rustdoc documentation my thoughts are the following:
The example on the main page is too long ... either shorten it, or move it into a submodule or just reference the examples in the examples/
directory.
The structs and enums in the top-level namespace are all rather randomly jumbled together ... making it very hard to grasp the API from looking at its documentation. I think it would be better to make modules public rather than pub use
ing all these structs / enums in lib.rs
.
Seeing things like InternalEvent
and InternalLifeCycle
makes me question why these are exposed in the public API in the first place ... in any case they should probably not be in the top-level scope.
In general I think the code could use some more structure ... e.g. the command
and event
modules are currently on the same level, begging the question what the difference between commands and events is. Looking into it Command appears to be part of an Event variant, so I think it would make sense to move the command
module under the event
module (same with the mouse
module which currently only contains the MouseEvent
enum).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.