Failing docs build about lvgl HOT 16 CLOSED

@kisvegabor

See issue #6267

This is why the documentation build system is failing. It is not designed to parse C++ code which is what is trying to be done. In order to correct this it would require a very large amount of work to create a parser that is able to read C++ code. It would need to be done to generate binding code for micropython and also for the documentation build system. It would be a very large undertaking to do this. It is easier to write C wrappers around the C++ code to handle it.

from lvgl.

do we have a CI output somewhere I am able to take a look at to see what is going on.

from lvgl.

@kisvegabor

So lets take a look at this error.

Traceback (most recent call last):
  File "/home/runner/work/lvgl/lvgl/docs/build.py", line 147, in <module>

    doc_builder.run(
  File "/home/runner/work/lvgl/lvgl/docs/doc_builder.py", line 663, in run
    globals()[compound.attrib['kind'].upper()](
  File "/home/runner/work/lvgl/lvgl/docs/doc_builder.py", line 494, in __init__
    cls = globals()[member.attrib['kind'].upper()]
KeyError: 'FRIEND'

It is looking for an XML base element named FRIEND. Now lets think what in C code would be called a friend? Nothing would. What in C++ code would be called a friend? OH.. There it is, a class would be.... So whatever is happening is happening because the documentation being parsed is C++ code and the documentation build system is designed for C code. If there is an inclusion of a CPP header file into a C header file that is going to cause an issue. CPP headers should only be included in C source files. This way it is not seen by doxygen or pycparser. I say doxygen because sphinx is not able to handle C or CPP code. Doxygen is able to do both but there has to be a connection between doxygen and sphinx and that is a sphinx addon. That addon is only capable of handling the output from doxygen for C code not CPP code.

I am sure I could find some way to wiggle around the issue but I need to know the exact thing that is causing it. It's not being caused by lv_rlottie because that is all C code. It has to be something else.

We didn't want to have to maintain 2 separate config files one for general use and one for documentation. so we opted to go the route of having the config file generated at runtime. Well that script is written to turn on everything that has "LV_USE" at the beginning of it. There is no way for the script to know if a feature is written in C or C++.

How we can work around the problem is if C++ header files have the extension ".hpp" instead of ".h". I would be able to tell the documentation build system to skip over any files that end in ".hpp". so we wither need to keep a specific list that would need to be updated of LV_USE_* config options that we do not turn on or the extension of the header files for C++ code would need to be different.

Your choice on what to do there. either works for me.

from lvgl.

The best would be to not include any cpp things publicly. However I have no idea where these cpp things are included. lv_lottie.h is pure C. Do you see the source of this "friend"? Btw, it can be the "friend" cpp keyword, meaning a friend class. Its really used in ThorVG but it's not included in lvgl.h.

from lvgl.

All of the thorVG code in LVGL is written in C++

from lvgl.

if ThorVG is not included anywhere then what does LV_USE_THORVG do??

from lvgl.

if ThorVG is not included anywhere then what does LV_USE_THORVG do??

ThorVG is used only internally. It cannot be (shouldn't be) reached from lvgl.h.

from lvgl.

It's not just the C++ stuff.

551 errors in the rst files
14 errors from anonymous enums
63 undocumented item warnings.

There are a bunch of bad typedefs for enums that don't have the proper doxygen macros for the typedefs so the documentation will link the typedef to the enum properly.

You are only building the documentation when a release occurs and that is a mistake to do. The documentation build should be run for every single PR. There are too many modifications being made to the documentation to not be checking it each PR to make sure things are documented and also that they are documented properly.

It is going to take me a long while to clean up all of the problems. There are changes made to the build system and those changes have not been tested properly to ensure that are not going to cause any problems. The changes that are made to the build system are extensive. I do not know if any of the things that have been changed are causing any issues. In order for me to determine that I need to revert each change one at a time and test to see what is going on and what the change impacts. It looks like there have been 16 changes made to the build system that I did not make and I don't know what those changes did.

The rst files being updated is one thing but when the build system gets modified that can really break things. Changes to the rst files being done correctly is going to require the documentation being built every single PR and not just when a release is made.

Here is an example of incorrect enum to typedef declarations. There is more than one enum like this. Now we have an issue with an anonymous enum and those end up in limbo and cannot be linked to any documentation anywhere. There is also no documentation for the items in the enum either.

enum {
    LV_DRAW_SW_MASK_RES_TRANSP,
    LV_DRAW_SW_MASK_RES_FULL_COVER,
    LV_DRAW_SW_MASK_RES_CHANGED,
    LV_DRAW_SW_MASK_RES_UNKNOWN
};

typedef uint8_t lv_draw_sw_mask_res_t;

it should read

enum _lv_draw_sw_mask_res_t{
    LV_DRAW_SW_MASK_RES_TRANSP,
    LV_DRAW_SW_MASK_RES_FULL_COVER,
    LV_DRAW_SW_MASK_RES_CHANGED,
    LV_DRAW_SW_MASK_RES_UNKNOWN
};

#ifdef DOXYGEN
typedef _lv_draw_sw_mask_res_t lv_draw_sw_mask_res_t;
#else
typedef uint8_t lv_draw_sw_mask_res_t;
#endif

It's gonna take me a while to sort this mess out. It's over a years worth of neglect that I have to fix. It's not gonna happen in hours or days. Gonna take me a few weeks to clean it up at a minimum.

from lvgl.

Myself and @liamHowatt really needs to spend some time to go over the documentation build system and what it does and why. this way more than one person knows how the damned thing works and is able to maintain it. LVGL is a monster sized project and to get documentation properly handled it is not a one person task. He is making a very large change to the API and that change is really going to make a mess of the documentation if things are not done properly. It is easier to address things while the work is being done then after the documentation ends up a disaster. There is going to be a lot to go over with how it works and what it does not something I want to have to type out that's for sure. doing a voice chat with desktop sharing would make things easier to do.

The reason why sphinx is being used is because frankly doxygen's html output sucks big time. sphinx is way more customizable but with customization you get complexity. Sphinx does python and not C or C++ code that's where doxygen steps in because it does handle that and it is very good at doing that. There is no way to attach directly to doxygen so we have to read the xml output from it in order to inject the C/C++ documentation into Sphinx so we end up with an easy to use final product that looks professional.

As you know when using an external library that was written and is maintained by a 3rd party it may not always function properly or how we need it to function in order to get the desired results. Getting things fixed can sometimes take a long time if at all. It only took a year and a 1/2 for an issue with parenthesis mucking up function pointers to get fixed. I was the first to report the problem to them and it got dismissed as we are declaring function pointers incorrectly and after other people also commented about the problem and that it needed to be fixed is when it finally did get fixed, took a year and a half for enough people to say that I was right and doxygen was broken to get that done. It is a headache to have to work around those kind of problems.

We do need to figure out the C++ code. If it is in LVGL then it needs to be documented. plain and simple. There is a grip of C++ code that has been added so we need to figure out how to get that added into the documentation properly. I don't understand why there is an internal and an external for ThorVG. It looks like the code for ThorVG is the same code that is seen in the ThorVG repository. I am not sure why there is what appears to be a copy in LVGL but there is. And once again there is yet another problem that stops LVGL from being cross platform. There are some shell/bash scripts that get run that is in ThorVG for LVGL. use of shell/bash scripts limits LVGL to only being able to be compiled on a nix type OS. Windows is not a nix type OS and it is not able to run those scripts natively, this will cause the build to fail.

One of the things you need to keep at the forefront of your mind is OS market share.
As of April 2024

Windows: 73.5% (up from 68.28% 1 year ago)
macOS: 14.7% (down from 18.96% 1 year ago)
Linux: 3.88% (up from 2.7% 1 year ago)

In 2010 linux was at 0.75% and macOS was at 0.33%. Windows was at 92.4%. It has taken almost 15 years for Linux to climb ~2%.

Things need to be able to compile on Windows as that is what is mostly being used. Shouldn't have to jump through hoops to get it to compile either. I know you guys like Linux but the reality is the world likes Windows.

from lvgl.

We build the docs on each commit (i.e. on each merged PR).

I know that there are errors and warnings in docs but despite of that it worked before merging the Lottie update. Don't misunderstand me, I think it's important to fix them, but they shouldn't be the root of the issue.

Regarding the OS support: Are there any issues with using WSL on Windows?

from lvgl.

Regarding the OS support: Are there any issues with using WSL on Windows?

Yeah getting it to work properly. It also takes up a very large amount of space and gobbles up memory like it's going out of style.

Here is the size of the virtual disk for WSL running Ubuntu 22.04. The ONLY thing I use this for is compiling LVGL and MicroPython, that's all. The amount of space it takes up is massive,

and here is the memory use. 24GB commit. That is after I did a very large amount of tweaking to push most of the use into the page file on the system.

The other issue you have is you cannot compile LVGL to run on Windows. WSL is a virtual machine and doesn't allow compiling native windows applications.

from lvgl.

(i.e. on each merged PR)

so the person that makes a PR has no way of knowing if something they have done when adding to the documentation is correct or not. It only compiles the documentation once the PR is merged??. If that is the case then that would not be ideal to ensure that everything is correct with the documentation. If the documentation is kept error free as things are added to it that would be the best thing.

Having it spit out errors that trigger the CI to fail if documentation has not been added for new things is also the way to go.

from lvgl.

Wow, 80GB is really massive! What would be required to make it work on Windows?

It would be really nice to to compile the docs in the PRs not only after merging.

from lvgl.

yeah 80GB just to be able to compile using WSL is a bit crazy. The memory usage is also insane as well.

To compile under Windows you need to make sure that shell scripts are not used or there is a shell script and a matching batch file for windows. You would have to instruct make or cmake on which one to run depending on the OS. Python pretty much can run on just about anything which is why it is good to use for these kinds of things. It is a single script that would need to be maintained instead of multiple scripts.

mingw has not really been all that active with fixes/improvements. It is on it's way to not being maintained. People are steering away from it because of cmake and it being a true cross platform build system. Ninja has taken care of cmake's lack of proper handling of multicore processors and not using all of the cores. Make is also in the decline as well because it is not cross platform.

Developers don't want to maintain several different build systems. It's a royal pain to do. Windows is the real challenge because of how Visual Studio builds work. It has gotten a lot better than it used to be I will say that much. Trying to detect where MSVC was installed onto the system was a complete nightmare prior to Visual Studio 2017. Now there is a way to get all of the needed information to set up a build using Windows COM interfaces.

The problem with MSVC is the command line arguments not aligning with other compilers like GCC and CLANG. Even cmake doesn't convert the command line parameters between the different compilers. I think it should at least handle the ones that have a direct match between the different compilers but it doesn't even do that.

from lvgl.

The document build system was working on Windows until that shell script was added for ThorVG. If ThorVG is turned off the documentation will compile using MSVC to generate the preprocessor output.

from lvgl.

Just to be sure, which ThorVG shell script do you mean?

from lvgl.