Git Product home page Git Product logo

cesium-native's Introduction

Cesium Native

Cesium Native is a set of C++ libraries for 3D geospatial, including:

  • 3D Tiles runtime streaming
  • lightweight glTF serialization and deserialization, and
  • high-precision 3D geospatial math types and functions, including support for global-scale WGS84 ellipsoids.

License Build Status

Cesium Native powers Cesium's runtime integrations for Cesium for Unreal, Cesium for Unity, Cesium for Omniverse, and Cesium for O3DE. Cesium Native is the foundational layer for any 3D geospatial software, especially those that want to stream 3D Tiles.

Cesium Platform and Ecosystem

A high-level Cesium platform architecture with the runtime integrations powered by Cesium Native and streaming content from Cesium ion.

🗃️Libraries Overview

Library Description
Cesium3DTiles Lightweight 3D Tiles classes.
Cesium3DTilesReader 3D Tiles deserialization, including 3D Tiles extension support.
Cesium3DTilesWriter 3D Tiles serialization, including 3D Tiles extension support.
Cesium3DTilesSelection Runtime streaming, decoding, level of detail selection, culling, cache management, and decoding of 3D Tiles.
CesiumAsync Classes for multi-threaded asynchronous tasks.
CesiumGeometry Common 3D geometry classes; and bounds testing, intersection testing, and spatial indexing algorithms.
CesiumGeospatial 3D geospatial math types and functions for ellipsoids, transforms, projections.
CesiumGltf Lightweight glTF processing and optimization functions.
CesiumGltfReader glTF deserialization / decoding, including glTF extension support (KHR_draco_mesh_compression etc).
CesiumGltfWriter glTF serialization / encoding, including glTF extension support.
CesiumIonClient Functions to access Cesium ion accounts and 3D tilesets using ion's REST API.
CesiumJsonReader Reads JSON from a buffer into statically-typed classes.
CesiumJsonWriter Writes JSON from statically-typed classes into a buffer.
CesiumUtility Utility functions for JSON parsing, URI processing, etc.

📗License

Apache 2.0. Cesium Native is free for both commercial and non-commercial use.

💻Developers

⭐Prerequisites

  • Visual Studio 2019 (or newer), GCC v11.x+, Clang 12+. Other compilers are likely to work but are not regularly tested.
  • CMake 3.15+
  • For best JPEG-decoding performance, you must have nasm installed so that CMake can find it. Everything will work fine without it, just slower.

🚀Getting Started

Clone the repo

Check out the repo with:

git clone [email protected]:CesiumGS/cesium-native.git --recurse-submodules

If you forget the --recurse-submodules, nothing will work because the git submodules will be missing. You should be able to fix it with:

git submodule update --init --recursive

Compile from command line

## Windows compilation using Visual Studio
cmake -B build -S . -G "Visual Studio 15 2017 Win64"
cmake --build build --config Debug
cmake --build build --config Release

## Linux compilation
cmake -B build -S .
cmake --build build

Compile from Visual Studio Code

  1. Install the CMake Tools extension. It should prompt you to generate project files from CMake.
  2. On Windows, choose Visual Studio 2017 Release - amd64 as the kit to build. Or choose an appropriate kit for your platform.
  3. Then press Ctrl-Shift-P and execute the CMake: Build task or press F7.

Compile with any Visual Studio version using CMake generated projects

  1. Open the CMake UI (cmake-gui)
  2. Under "Where is the source code", point to your repo
  3. Specify your output folder in "Where to build the binaries"
  4. Click "Configure".
  5. Under "Specify the generator for this project", choose the VS version on your system
  6. Click Finish, wait for the process to finish
  7. Click Generate

Look for cesium-native.sln in your output folder.

Unit tests can also be run from this solution, under the cesium-native-tests project.

image

Generate Documentation

  • Install Doxygen.
  • Run: cmake --build build --target cesium-native-docs
  • Open build/doc/html/index.html

Regenerate glTF and 3D Tiles classes

Much of the code in CesiumGltf, Cesium3DTiles, CesiumGltfReader, and Cesium3DTilesReader is generated from the standards' JSON Schema specifications. To regenerate the code:

  • Make sure you have a relatively recent version of Node.js installed.
  • Install dependencies by running:
npm install
cd tools/generate-classes
npm install
cd ../..
  • From the repo root directory, run these commands
    • npm run generate-gltf
    • npm run generate-3d-tiles
    • npm run generate-quantized-mesh-terrain
  • On Windows, the line endings of the generated files will be different than those checked into the repo. Just git add them and git will fix the line endings (no need to commit).

cesium-native's People

Contributors

azrogers avatar baothientran avatar csciguy8 avatar eziohelios avatar hjanetzek avatar ianlilleyt avatar insar-dev avatar j-j-m avatar j9liu avatar javagl avatar jherico avatar jiangheng90 avatar joseph-kaile avatar josephbirkner avatar kring avatar krupkad avatar lilleyse avatar lyhkop avatar miatang13 avatar nithinp7 avatar samulus avatar shehzan10 avatar themostdiligent avatar timoore avatar ulvesson avatar v2i-git avatar xuelongmu avatar zackofalltrades avatar zagrizzl avatar zy6p avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cesium-native's Issues

Make #includes match C++ Core Guidelines

There currently are some subtle inconsistencies regarding include directives. Specifically, the pattern of brackets/quotes for <system_includes> and "own_includes" is not applied consistently. The relevant comment that summarizes this is at #56 (comment)

This should be fixed, preferably in one, dedicated cleanup step.

Avoid crashes in case of errors

There are some places in the code where errors are handled by printing an error message, but then still bailing out in a form that causes a crash. One example is

return pAggregatedOverlay->createTileProvider(asyncSystem, pCreditSystem, pPrepareRendererResources, pLogger, pOwner);
, which is a null pointer access if there was an error.

Right now, it should be relatively easy to clean this up, with some full-text searches for SPDLOG_LOGGER_ERROR, return nullptr or return std::nullopt, and could/should be done in one dedicated pass.

Add an option to _not_ view-frustum cull tiles during selection

This sounds a little crazy, but hear me out.

The motivation is, we should be able to render the whole globe at all times. This avoids a bunch of problems:

  • Missing tiles (holes) or very low detail and cracking when spinning the camera around quickly.
  • Flashes of black at the edges of the screen in UE caused by the game thread lagging a frame behind the render thread (I think).
  • Various physics glitches, like stepping backward in a first-person view and falling through the Earth because no tiles are loaded behind the camera. Also a general inability to simulate physics outside the view frustum, at any level of fidelity.

The downside to not view-frustum culling, of course, is that we end up loading a lot more tiles. We can mitigate this somewhat by (significantly?) reducing the SSE for tiles outside the view frustum, rather than culling them entirely.

This might be a really good trade-off for some types of applications, and it'd be relatively easy to implement and see how it goes (e.g. how many more tiles we end up loading).

Race condition of asset requests

There is a race condition when requesting assets and notifying the callbacks about the data that has been loaded.

  1. An asset is requested, using the IAssetAccessor::requestAsset function. The result is an IAssetRequest. A callback is bound to this request, by calling the bind function. An example from BingMapsRasterOverlay:

     this->_pMetadataRequest = externals.pAssetAccessor->requestAsset(metadataUrl);
     this->_pMetadataRequest->bind([this, callback, pOwner, &externals](IAssetRequest* pCompletedRequest) {
    
  2. The IAssetAccessor implementation generates an internal request, and binds an internal callback to responseReceived to this request. In responseReceived, the _callback is called. An example from UnrealAssetAccessor:

     this->_pRequest->OnProcessRequestComplete().BindRaw(this, &UnrealAssetRequest::responseReceived);
     this->_pRequest->ProcessRequest();  
    

    An important point here is: It calls "ProcessRequest" immediately.

It could happen that

  • IAssetAccessor::requestAsset is called. An IAssetRequest is created.
  • The internal request is created, and ProcessRequest is called
  • The request completes quickly
  • The internal callback calls responseReceived (at this point, the _callback is still nullptr!)
  • then, the IAssetRequest is returned
  • *then, the _callback is attached to this IAssetRequest
  • (...but the callback is never called, because the request was already completed, and responseReceived was already called!)

Although this is "unlikely", it all depends on the order of execution. Essentially, whether the request is completed between the call to send out the request, and the call to the bind function.

The situation is far more complex and critical with RasterOverlayTile. There, the pending request is passed to the constructor, and the bind call happens later, when RasterOverlayTile::load is called - and this, in turn, is called at places and times that are hard to pin down...


Very rough ideas of how this could be solved (which of them is feasible and/or the most appropriate has to be sorted out) :


  1. Differentiate between "creating the request" and "starting the request".

The Unreal HTTP request does this (as it can be seen in the second code block above). (Probably to avoid race conditions...). It would mean that after creating the request, it would have to be started explicitly, as in this pseudocode:

    IAssetRequest request = assetAccessor->requestAsset(url);
    request->bind(someCallback);
    request->startRequest();  // Explicitly have to start it

(One could then argue that requestAsset should be renamed to createAssetRequest, to indicate that it does not request the asset, but only create the request, but that's a detail for now).

The question is: Who is going to call startRequest? It is then basically in the responsiblity of the code that also calls bind. One could then even go further, and argue whether these are different operations, or whether it should be a single request->startAndReportTo(callback) function. (Probably not, but why not? Binding without starting does not make sense. Starting without binding rarely makes sense either...)


  1. Pass the callback to the constructor.

That would mean that instead of the dedicated bind call, the callback would be passed to requestAsset as in

IAssetRequest request = assetAccessor->requestAsset(url, someCallback); // Pass in callback here
request->bind(someCallback);// Instead of attaching it here

One has to check whether the callback is always already available at this point. Specifically, for RasterOverlayTile, the creation and the bind call happen at totally different places and times.


  1. Store the state of the internal request

One could track whether the internal request was already completed, and inform callbacks immediately if they are reqistered later. Roughly as in

class UnrealAssetRequest : public IAssetRequest {

    InternalRequest _internalRequest;
    Callback _callback;
    bool _internalRequestCompleted = false;

    UnrealAssetRequest() {
        _internalRequest = create();
        _internalRequest.bind(this::internalCompleted);
        _internalRequest.start(); 
    }

    void internalCompleted() {
        if (_callback != nullptr) _callback.call();

        _internalRequestCompleted = true; // Store info that internal callback was completed
    }

    void bind(Callback callback) {
        this._callback = callback;
        if (_internalRequestCompleted) { // Immediately call callback if internal one was completed
            _callback.call();
        }
    }
}

But this would have to be done very carefully, and probably involve some atomics if it should be done right...


Support composite (cmpt) tile content

#103 adds some support for .cmpt tile content, but it currently only uses the first inner content that it knows how to load.

I can think of two possible ways to support multiple sets of content:

  • Combine them all into a single glTF. This will require copying and renumbering a bunch of glTF objects. We shouldn't do this based on tinygltf if we're switching away from it.
  • Create sub-tiles, one for each inner content object. This will require a small amount of refactoring to allow a tile to acquire loaded content without going through the usual loadContent process.

I can't currently see any strong reason to prefer one of these approaches over the other.

Detect and fix warnings that are found by IntelliSense code analysis

As a follow-up of #27 (comment) :

The current warning level for MSVC is set to a nicely strict /W4 at the moment. But there are still some warnings generated only by IntelliSense in Visual Studio (i.e. not during the compilation, and not by IntelliSense in VS Code).

The goal of this issue is to find out how these warnings can be caught during the compilation process.

This has the caveat that there will be many warnings due to include of Third-Party headers, but it should be possible to avoid most of them via "pragma guards", as it is done in https://github.com/CesiumGS/cesium-native/blob/master/CesiumUtility/include/CesiumUtility/Json.h . If it is easily possible to simply omit the warnings that are created by the draco compilation, that would also be nice.

Clamp points to terrain

Keep a list of points on (or relative to) the terrain surface as the terrain level-of-detail changes. This will likely work like CesiumJS, where points are associated with tiles and are moved between them as tiles refine/unrefine.

Race condition related to TileContext

After #58, background threads used to process loaded tiles mostly don't have direct access to Tile, RasterOverlayTile, or other instances that are owned by the main thread.

The one exception to this that I'm aware of is that TileContentFactory::createContent (which is called from a worker thread) is given a direct reference to the TileContext of the Tile that contains the content. Every content type just ignores this parameter, except for one: ExternalTilesetContent. And the only thing that ExternalTilesetContent does with it is copy it.

A TileContext is mostly immutable after it's created. But unfortunately there's one exception to that, too. When the Cesium ion token expires, and a new one is obtained, the new token gets put into TileContext::requestHeaders.

So, if the Cesium ion token expires and is refreshed, while at the same time the context is copied because an external tileset is loaded, we have a race condition that can lead to undefined behavior. It's pretty unlikely, but in theory it can happen.

A good solution to this isn't obvious. We don't want to copy the TileContext prior to launching the worker thread, because it can sometimes be expensive to copy (e.g. when it has a long list of QuadtreeTileAvailability, as Cesium World Terrain does), and because it's almost never actually needed (it's only needed by external tilesets).

Logging and error handling

Logging

There currently is no logging in cesium-native.

A detailed investigation of the available logging libraries, e.g. starting from the List at cppreference.com, could be difficult. But there are two libraries which I have used until now:

  • Boost.Log: One can assume that it is well engineered, tested and reviewed, and will be maintained in the future. But it's a bit complex to set up.

  • spdlog: Seems to be rather simple, but powerful enough. Optionally to be used header-only, or as a library.

(The decision about the right library may be difficult, because logging is "invasive", insofar that it can be tedious to change it later. Therefore, I'd even advocate for a thin, custom layer around the respective library, maybe even in form of a simple macro or singleton class, but that's another point).


Things to keep in mind:

  • Obvious requirements (thread safety, the option to log into files...)
  • It should be possible to hide any logging that uses iostream under the hood, when cesium-native should be compiled for web assembly
  • It should be possible to pass logging messages to Unreal, preferably depending on the log level.

Unreal uses some very (very, very...) complex macro magic to set up the logging internally. At the surface, however, most of the interaction with the log mechanisms seems to appear through the UE_LOG macro. During a first, quick test, it seemed to be pretty simple to set up a custom sink in spdlog that just forwarded to the UE macro. (details, like translating the spdlog levels into unreal "Verbosity" levels would still have to be implemented).

Boost.Log (and probably other libraries) also has the concept of "Logging Sinks", where it should be possible to forward log output to Unreal, but I have not yet tried that out: https://www.boost.org/doc/libs/1_74_0/libs/log/doc/html/log/extension.html


Error handling

There currently is no elaborate concept for error handling. Many (hopefully most) places where error handling has to be introduced are marked with TODO comments, though.

Error handling should at least include proper exception handling, and logging a sensible error message (to the console, or the UE log accordingly).

The tricky part here are errors in asynchronous operations. Two cases come to my mind (but there may be more) :

  • When a function is passed to ITaskProcessor::startTask. Such a function might cause arbitrary errors, and there is no channel to report them to right now.

  • When a callback is attached via IAssetRequest::bind. Currently, errors like unknown content types are handled by calling the callback with nullptr, but no further details can be reported back.

There are different approaches to handle this

  • One could just leave the responsibility for error handling and reporting to the implementing function. E.g. one could just wrap the function call into a try..catch (as a safety net of last resort), and otherwise expect the implementor of the function to log error details, using the appropriate logger infrastructure.

  • There could be something like a simple error callback. Very roughly speaking, this could just be a function<void(exception&)> that would be called with information about the error. At the receiving end, these errors could be collected to logged as desired.

  • There could be a more generic "result callback" - mainly in order to dedicatedly collect possible warnings as well. Again, very roughly, maybe a function that receives a vector<string> warningMessages and errorMessages (or a vector<exception> for the errors)

Handling of duplicate resources

There might already be decision on how this should be handled - maybe only implicitly, or maybe by ~"doing it the same way as CesiumJS" - but: What is the intended strategy for handling duplicate resources?

When there is a Tileset that refers to the same "content.uri" = "http://example.com/content.b3dm", multiple times, then the requests are going out repeatedly, the responses are handled repeatedly, and all the resource handling functions (prepareInMainThread, free etc) are called repeatedly, The performance implications of this may obviously overshadow any other optimization.

There are, very roughly speaking, different levels on which this could be addressed:

  • It could all be left to some omniscent caching infrastructure. (Handwavingly: ~"What the browser does", or "On the networking level", "With some local proxy"). This would still mean that cesium-native has to do the same work again and again.
  • The request could be sent out only once. This would mean that there is some sort of set<string> urlsForWhichRequestsAreInFlight somewhere. This might require some thought to find a sensible structure. (The fact that there might be multiple, distinct URLs that point to the same resource should not be sooo important. But the URLs [sh/c]ould be normalized at least, so that /foo/.../bar.b3dm and /bar.b3dm are considered to be equal).
  • The response could be processed only once (essentially meaning that the request will be sent out multiple times, but the response is stored in some sort of lookup - there may be things to consider that I don't have on the radar in all detail, like server-side caching or resources that might change over time...).

Use interface instead of std::function for tile content loaders, and pass parameters to loaders in an object

There are currently three functions that can load tile content:

The functions are registered as implementations for loading tile content in the TileContentFactory. They all return a TileContentLoadResult.

The (very!) high-level goal of this is to be able to...

  • throw the arguments into a generic function for loading tile content
  • finding the appropriate implementation that can load the given content
  • return the result of loading this content

without having to handle the different implementations. This sort of abstraction is certainly a good thing. But there are some things that raise questions for me:

(Disclaimer: These should not be considered as direct criticism, and not even as "suggestion to change something here". I'm just mentioning this from the perspective of someone who just started looking at the code and tried to document it)

  • The functions have the same signature. One might consider a dedicated interface for that. Right now, they are static, so this is not directly doable, even though they are already implicitly handled accordingly, by treating all of them as a FactoryFunction.
  • There are 9 parameters for the load function. This raises questions about the consistency and constraints for these parameters. If they were combined in a parameter object (like TileContentLoadInput), one could ensure some consistency at runtime. But constructing such objects for large tilesets may have negative performance implications.
  • Probably not all of the parameters are required for each load function (?). The TileContentLoadResult from these functions consists of several optional elements. Very roughly speaking, and blatantly suggestive: If a single function is called with (FooInput, BarInput) where each parameter may be nullptr, and it returns an { optional<Foo>, optional<Bar> } where the caller has to figure out whether the Foo or the Bar is present, then it may be hard to justify having only a single function in the first place...

In any case: The parameters are currently only documented in one place, with "placeholder comments". The constraints for the parameters, and how the result depends on the input parameters, should be documented more thoroughly.

Non-base raster layers

CesiumJS distinguishes between "base" imagery layers, which are stretched over the whole globe regardless of their actual extent, and "non-base" which may only cover a small part of the globe. Currently cesium-unreal treats every layer as a base layer, which isn't right.

Extending the API documentation

This issue is intended for discussing and collecting tasks for the goal of having a more complete and useful API documentation. For now, this refers to cesium-native only, but some general guidelines should be derived from that, and consistently applied to other projects. This primarily refers to the Doxygen API documentation, or documentation/commenting in a slightly broader sense, but not to "Coding and Formatting conventions" (unless this directly affects the documentation).

Things that we could talk about:

  1. I think that every file should have a copyright header. I found some potential templates for that, but not a definitve one.

  1. I assume that all public and protected elements of the API should be documented.

One could consider to only document things that are explicitly exported via declaration specifier macros like CESIUM3DTILES_API, c.f. #9 ). But I don't have an in-depth knowledge about the implications of that. (In fact, these macros caused some issues for me - more on that below).

I know that some people consider comments like the this

     /**
      * @brief The minimum x-coordinate
      */
     double minimumX;

as "noise" or unnecessary. But I'm usually on the side of religiously documenting everything. (In my private projects, this even refers to private elements...). If we don't do this, we end up arguing about "How complex does a function have to be before it is 'worth' writing a short comment?"...


  1. The style of the documentation should be consistent.

Doxygen offers many degrees of freedom, starting with the \link vs @link syntax, but beyond that, ranging from

/// <summary>
/// Gets the look direction of the camera in Earth-centered, Earth-fixed coordinates.
/// </summary>

to

/**
 * @brief Gets the look direction of the camera in Earth-centered, Earth-fixed coordinates.
 */

It appeared to me that most of the code used the latter (so in fact, I already changed that locally in some parts of the code). In general, I assume that we'll usually use

  • the /**...*/ syntax
  • the @param syntax
  • the @link syntax
  • the `code` (backticks) syntax for markdown-like code blocks (with <code> ... </code> being another option)
  • ...

  1. The comments should be complete and generate "nice" output

This means that every parameter should have a @param and every return value should have a @return.

Additionally, every variable/class/method/function should have a @brief summary. This is helpful when skimming over the "Public Member Functions" and "Public Attributes" lists in the resulting documentation, to quickly get a summary about what the function does.


I have started some basic fixes in the DRAFT PR at #23

Until now, these are only documentation fixes (i.e. no changes to the actual code), with two small exceptions:

  • I have updated the doc/CMakeLists.txt to handle the macros like CESIUM3DTILES_API.

This is done here. These macros had been translated into declspecs, which Doxygen interpreted as some sort of "function name", causing the respective elements to be omitted in the documentation. The solution is based on the example in https://www.doxygen.nl/manual/preprocessing.html , but should be reviewed

  • I have added "Documentation" for the namespaces (just one sentence for now...)

This is done via this commit. I have currently added them in the Library.h header files, and I know that this is almost certainly not the right place!

But the namespaces should be documented somewhere, because otherwise, they do not appear in the doxygen output at all, and typedefs that had been declared in the namespace didn't appear either. (For example, the BoundingVolume was missing then...)

The right place for documenting the namespace could be in a "top-level header", but this also affects the code and usage: Currently, there is no "top-level header". I think that it could be good to have a header like Cesium3DTiles.h, where not only the namespace is documented, but which also includes the rest of the public API, so that users can just do an #include <Cesium3DTiles.h> and are good to go. (I think this is done similarly in boost and other libraries).

Expose 3D Tiles feature attributes

Because tiles are "sent" to the renderer as glTFs, we should consider encoding feature attributes the way they [are / will be] encoded in 3D Tiles Next.

Handle web mercator raster overlays on geometry tiles that extend outside the web mercator bounds

Web Mercator texture coordinates get dodgy north/south of ~+/-85 degrees latitude. CesiumJS deals with this by reprojecting web mercator imagery tiles to geographic when they're attached to terrain tiles that cross this boundary. We could do the same here, or alternatively we could insert extra vertices in the geometry at ~+/-85 degrees latitude so that we never try to texture map across this boundary.

Remove prototype "clipping rectangle" support

We originally implemented this very useful feature that allows rectangles to be cut out of a tileset, so that another tileset can be inlaid, for Project Anywhere. But there are too many caveats for its use. So for the initial release of Cesium for Unreal and cesium-native, we're going to remove this feature. Actually, we'll move it to a branch.

Add support for WebP images

Currently we're using stb (via tinygltf) for image decoding, but it doesn't support every format. Most notably missing is WebP. We should add support for this format for images inside glTFs and as raster overlays.

Review the structure of the Camera

The current Camera class essentially consists of the following properties:

  • position, direction, up vector
  • viewport size;
  • horizontal and vertical FOV
  • an Ellipsoid (?!)
  • (+ some derived properties)

According to a recent comment in another issue, it is supposed to be used as a "value-like" object. This means that, for example, in cesium-unreal, the camera is not really modified. Instead, a new camera instance is created, for each frame, from the current Unreal camera/view settings, and this instance is passed to cesium-native.

As such, the primary purpose of this class is not "to be a virtual camera", in the most abstract sense of how a camera could be modeled. Instead, just summarizes everything that is required for Tileset::updateView.


A related issue in cesium-unreal is CesiumJS-style camera , meaning that there may be a more sophisiticated camera class in the future.

Some initial, basic thoughts and things that one could consider:

  • Iff there should be a structure that just "summarizes stuff for updateView", then this class could/should have a different name - maybe something like ViewConfiguration or so
  • Related to that name: The current Camera class combines two things that are actually unrelated, namely a camera (defined by position, orientation and FOV), and a view (mainly defined by the viewport size).
    • Generally, I have never seen these things combined. The viewport size is also not part of the camera in CesiumJS. It should be possible for a camera to merely exist, without the need for a rendering window. Combining them raises lots of questions - an important one: Who tracks the size of the rendering window (and how), and informs the camera about changes in the window size? A technical limitation of combining them would also be that it would no longer be possible to use the same camera to render into two different-sized viewports.
  • Such a camera should probably have a listener infrastructure (c.f. #74 ), so that users can be informed about changes. For example, as in camera.addListener([](){ triggerRendering(); });, to trigger a rendering whenever the camera is changed.
  • I'm somewhat pedantic sometimes. Imagine someone asked you: "What constitutes a 'virtual camera', and how can this be modeled?". You'd immediately come up with a list of possible properties and representations (Hughes, CGP&P, Chapter 6). But you would certainly not say: "A camera has an ellipsoid". This simply does not belong there, regardless of how tempting the sweet poison of "this is currently very convenient for us" may be.

Access violation near RasterOverlayTile when exiting editor

I just encountered this crash when closing the unreal editor. It happened "randomly", so until now, I cannot describe a dedicated procedure of how to reproduce it, but will try to add more details if it appears again and I recognize a pattern.

For now, this is just to "not ignore" it:

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address 0x00000000000067db

UE4Editor_Cesium_0007!std::_Hash<std::_Umap_traits<CesiumGeometry::QuadtreeTileID,std::unique_ptr<Cesium3DTiles::RasterOverlayTile,std::default_delete<Cesium3DTiles::RasterOverlayTile> >,std::_Uhash_compare<CesiumGeometry::QuadtreeTileID,std::hash<CesiumGeom() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xhash:1654]
UE4Editor_Cesium_0007!std::_Hash<std::_Umap_traits<CesiumGeometry::QuadtreeTileID,std::unique_ptr<Cesium3DTiles::RasterOverlayTile,std::default_delete<Cesium3DTiles::RasterOverlayTile> >,std::_Uhash_compare<CesiumGeometry::QuadtreeTileID,std::hash<CesiumGeom() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xhash:1317]
UE4Editor_Cesium_0007!std::_Hash<std::_Umap_traits<CesiumGeometry::QuadtreeTileID,std::unique_ptr<Cesium3DTiles::RasterOverlayTile,std::default_delete<Cesium3DTiles::RasterOverlayTile> >,std::_Uhash_compare<CesiumGeometry::QuadtreeTileID,std::hash<CesiumGeom() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xhash:1329]
UE4Editor_Cesium_0007!Cesium3DTiles::RasterOverlayTileProvider::removeTile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\RasterOverlayTileProvider.cpp:377]
UE4Editor_Cesium_0007!Cesium3DTiles::RasterOverlayTile::releaseReference() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\RasterOverlayTile.cpp:80]
UE4Editor_Cesium_0007!CesiumUtility::IntrusivePointer<Cesium3DTiles::RasterOverlayTile>::releaseReference() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\CesiumUtility\include\CesiumUtility\IntrusivePointer.h:151]
UE4Editor_Cesium_0007!CesiumUtility::IntrusivePointer<Cesium3DTiles::RasterOverlayTile>::~IntrusivePointer<Cesium3DTiles::RasterOverlayTile>() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\CesiumUtility\include\CesiumUtility\IntrusivePointer.h:46]
UE4Editor_Cesium_0007!Cesium3DTiles::RasterMappedTo3DTile::~RasterMappedTo3DTile()
UE4Editor_Cesium_0007!Cesium3DTiles::RasterMappedTo3DTile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::RasterMappedTo3DTile> >::destroy<Cesium3DTiles::RasterMappedTo3DTile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::RasterMappedTo3DTile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::RasterMappedTo3DTile,std::allocator<Cesium3DTiles::RasterMappedTo3DTile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::RasterMappedTo3DTile,std::allocator<Cesium3DTiles::RasterMappedTo3DTile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::RasterMappedTo3DTile,std::allocator<Cesium3DTiles::RasterMappedTo3DTile> >::~vector<Cesium3DTiles::RasterMappedTo3DTile,std::allocator<Cesium3DTiles::RasterMappedTo3DTile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::_Default_allocator_traits<std::allocator<Cesium3DTiles::Tile> >::destroy<Cesium3DTiles::Tile>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:700]
UE4Editor_Cesium_0007!std::_Destroy_range<std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\xmemory:961]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Destroy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1613]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::_Tidy() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:1695]
UE4Editor_Cesium_0007!std::vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >::~vector<Cesium3DTiles::Tile,std::allocator<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\vector:678]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::~Tile() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tile.cpp:40]
UE4Editor_Cesium_0007!Cesium3DTiles::Tile::`scalar deleting destructor'()
UE4Editor_Cesium_0007!std::default_delete<Cesium3DTiles::Tile>::operator()() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\memory:2402]
UE4Editor_Cesium_0007!std::unique_ptr<Cesium3DTiles::Tile,std::default_delete<Cesium3DTiles::Tile> >::~unique_ptr<Cesium3DTiles::Tile,std::default_delete<Cesium3DTiles::Tile> >() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\memory:2514]
UE4Editor_Cesium_0007!Cesium3DTiles::Tileset::~Tileset() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tileset.cpp:130]
UE4Editor_Cesium_0007!ACesium3DTileset::DestroyTileset() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\Source\Cesium\Private\ACesium3DTileset.cpp:418]
UE4Editor_Cesium_0007!ACesium3DTileset::BeginDestroy() [C:\Develop\CesiumUnreal\Code\cesium-unreal-demo\Plugins\cesium-unreal\Source\Cesium\Private\ACesium3DTileset.cpp:673]
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor
UE4Editor
UE4Editor
UE4Editor
UE4Editor
UE4Editor
UE4Editor
kernel32
ntdll

(In general, the line numbers in these traces do not seem to match the actual line numbers of the code. But maybe someone has a gut feeling of what might be wrong here...)

Remove glTF compressed buffers after decompressing

When a glTF contains a Draco-compressed buffer or a jpg/png compressed image, we decompress/decode them at load time. However, the original compressed data hangs around. We could save memory by freeing these original buffers after we're done loading data from them.

Doing this is mostly trivial, except for (unlikely?) edge cases where a single block of compressed data in a glTF Buffer is used by multiple primitives (draco) or images (jpg/png).

Mark classes final

Most of the types in cesium-native are not suitable for inheritance and should be marked final. The only exceptions I can think of are RasterOverlay, RasterOverlayTileProvider, ITaskProcessor, IPrepareRendererResources, and the IAsset* interfaces.

generate-gltf-classes: Request glTF schema files from github instead of embedding as a submodule

The glTF spec repo isn't tiny, and we only need it when generating glTF classes. And even then we only need the JSON schema files., which are very small compared to the whole repo. So a nice improvement (originally suggested by @javagl in #106) would be to request the schema files from github as needed, i.e. in SchemaCache.js instead of forcing users to download the whole glTF repo as a submodule.

@javagl's suggested code in SchemaCache.js:

var request = require('sync-request');
...
var res = request('GET', new URL(name, this.schemaPath).href);
const result = JSON.parse(res.getBody(), "utf-8");

And in package.json:

"generate": "node index.js --schema https://raw.githubusercontent.com/KhronosGroup/glTF/master/specification/2.0/schema/ --output ../../CesiumGltf --readerOutput ../../CesiumGltfReader --extensions https://raw.githubusercontent.com/KhronosGroup/glTF/master/extensions/2.0/ --config glTF.json"

Support raster overlays with an arbitrary projection

Using PROJ or something, probably.

We will need to wait until the geometry is loaded, compute the bounds of all vertices in the projected coordinate system, and then select the raster overlay tiles with the appropriate LOD to cover all the vertices.

Crash in Tileset when trying to read from a URL

(The core of this issue is probably in cesium-native - it might also be in cesium-unreal. In doubt, the issue can be closed here and reopened there)

When entering an actual URL in the "URL" field of a Cesium3DTileset in the Unreal Editor (like http://example.com), then this causes the following crash:

UE4Editor_Cesium!nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<std::map,std::vector,std::basic_string<char,std::char_traits<char>,std::allocator<char> >,bool,__int64,unsigned __int64,double,std::allocator,nlohmann::adl_serializer> >::parse_error() [C:\Develop\CesiumUnreal\Code\cesium-unreal\extern\cesium-native\extern\tinygltf\json.hpp:4491]
UE4Editor_Cesium!nlohmann::detail::parser<nlohmann::basic_json<std::map,std::vector,std::basic_string<char,std::char_traits<char>,std::allocator<char> >,bool,__int64,unsigned __int64,double,std::allocator,nlohmann::adl_serializer> >::sax_parse_internal<nlohm() [C:\Develop\CesiumUnreal\Code\cesium-unreal\extern\cesium-native\extern\tinygltf\json.hpp:5259]
UE4Editor_Cesium!nlohmann::detail::parser<nlohmann::basic_json<std::map,std::vector,std::basic_string<char,std::char_traits<char>,std::allocator<char> >,bool,__int64,unsigned __int64,double,std::allocator,nlohmann::adl_serializer> >::parse() [C:\Develop\CesiumUnreal\Code\cesium-unreal\extern\cesium-native\extern\tinygltf\json.hpp:5044]
UE4Editor_Cesium!nlohmann::basic_json<std::map,std::vector,std::basic_string<char,std::char_traits<char>,std::allocator<char> >,bool,__int64,unsigned __int64,double,std::allocator,nlohmann::adl_serializer>::parse<gsl::details::span_iterator<unsigned char con() [C:\Develop\CesiumUnreal\Code\cesium-unreal\extern\cesium-native\extern\tinygltf\json.hpp:18573]
UE4Editor_Cesium!<lambda_772a8ec5eae40716aed7d48579fb5ba7>::operator()() [C:\Develop\CesiumUnreal\Code\cesium-unreal\extern\cesium-native\Cesium3DTiles\src\Tileset.cpp:250]
UE4Editor_Cesium!std::invoke<<lambda_772a8ec5eae40716aed7d48579fb5ba7> &>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\type_traits:1597]
UE4Editor_Cesium!std::_Invoker_ret<void,1>::_Call<<lambda_772a8ec5eae40716aed7d48579fb5ba7> &>() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\type_traits:1641]
UE4Editor_Cesium!std::_Func_impl_no_alloc<<lambda_772a8ec5eae40716aed7d48579fb5ba7>,void>::_Do_call() [C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include\functional:904]
UE4Editor_Core
UE4Editor_Core
UE4Editor_Core
UE4Editor_Core
UE4Editor_Core
UE4Editor_Core
kernel32
ntdll

The code at Tileset.cpp:250:

        externals.pTaskProcessor->startTask([data, &externals, this, &context]() {
            using nlohmann::json;
            json tileset = json::parse(data.begin(), data.end());

The documentation at https://nlohmann.github.io/json/api/basic_json/parse/ suggest that this might simply be a JSON parsing error that threw up (or the input data for the parse call was invalid). In any case, this should be handled more graciously, if possible.

Tile content is unloaded and freed multiple times

The Tile::unloadContent function is called multiple times, once from the Tile destructor and once from the Tileset destructor.

Is this intentional? Can the call in the Tileset be omitted?

In any case, this also causes the IPrepareRendererResources::free function to be called multiple times. While one could state that implementors MUST accept and handle that accordingly, it could probably be avoided by adding a check at the beginning of the function, to see whether the tile is actually (still or already) in the Unloaded state, as in

bool Tile::unloadContent() {

    if (this->getState() == Tile::LoadState::Unloaded) {
        return true;
    }
...

Consider exposing fewer of the internal Tile functions to the user

At the time of writing this, the Tile class has >30 functions that are public.

I'm listing them here (omitting possible const versions), as an overview, and from a very short glance (just to have an estimate!), only the first half of this list should actually be public:

    Tileset* getTileset();
    TileContext* getContext();
    Tile* getParent();
    gsl::span<Tile> getChildren();
    const BoundingVolume& getBoundingVolume() const;
    const std::optional<BoundingVolume>& getViewerRequestVolume() const;
    const std::optional<BoundingVolume>& getContentBoundingVolume() const;
    double getGeometricError() const;
    TileRefine getRefine() const;
    const glm::dmat4x4& getTransform() const;
    TileContentLoadResult* getContent();
    void* getRendererResources() const;
    bool isRenderable() const;

    void prepareToDestroy();
    void setContext(TileContext* pContext);
    void setParent(Tile* pParent);
    void createChildTiles(size_t count);
    void createChildTiles(std::vector<Tile>&& children);
    void setBoundingVolume(const BoundingVolume& value);
    void setViewerRequestVolume(const std::optional<BoundingVolume>& value);
    void setGeometricError(double value);
    void setRefine(TileRefine value);
    void setTransform(const glm::dmat4x4& value);
    const TileID& getTileID() const;
    void setTileID(const TileID& id);
    void setContentBoundingVolume(const std::optional<BoundingVolume>& value);
    LoadState getState() const;
    TileSelectionState& getLastSelectionState();
    void setLastSelectionState(const TileSelectionState& newState);
    void loadContent();
    bool unloadContent();
    void update(uint32_t previousFrameNumber, uint32_t currentFrameNumber);

The second part (particularly, basically all set... functions) are only or mainly used for the initialization and setup, and maintaining the tiles structure during the rendering process. Specifically, most of them are only called by the Tileset (and few of them by other internal classes).

One could consider to reduce the size of the "surface" of this class: Users should not call things like tile->setParent(nullptr), and thus, should preferably not be able to do this.

(Some of the function may also only be relevant for certain use-cases. This should be pointed out in the documentation)

It might be possible to solve this by simply letting Tile and Tileset be friend classes, but more elaborate (or "clean") solutions may exist. (Similar questions may arise for other classes that are publicly exposed to the user, but to a much lesser extent).

Investigate the use of cesium-native in a WebAssembly build

One of the potential future uses of cesium-native - though largely unexplored so far - is to compile parts of it to WebAssembly and incorporate it into CesiumJS and potentially other in-browser rendering engines.

Meanwhile, we haven't been bashful about using C++17, STL, and third-party libraries where they are helpful, all of which could potentially bloat a WebAssembly build beyond the point where it is practical to use it in this way. On top of that, some features, such as C++ exceptions, may be problematic in WebAssembly builds.

There is a tradeoff here, between moving quickly now by using the best tools at hand and not worrying excessively about the future impact on a WebAssembly build, and possibly needing to make major changes to cesium-native in the future if and when we do need to build it for WebAssembly.

For now, I don't think we'll do anything drastic, but will:

  • Avoid (but not outright forbid) the use of exceptions.
  • Select third-party libraries carefully. Don't use Boost.
  • Deal with most of the consequences later.
  • Perhaps it'd be worth spending a little time doing a WebAssembly build just to see where we stand?

CC @jtorresfabra @pjcozzi

Remove third-party glTF loader from the cesium-native public interface

cesium-native uses glTF as its internal model representation. When it wants to say to a game engine "here's a tile, please render it", that model is represented as a glTF. Currently we do this by explicitly passing a tinygltf::Model to the renderer, which works well enough. But given #31, we shouldn't bake tinygltf into our interface, because it will be very hard to remove later.

The best solution here - even though it's a hassle - is probably to wrap the third-party loader's types in our own types, being careful not to add a bunch of overhead in the process. At that point, it may be a small step to simply skip the third-party loader altogether, and implement our own glTF loader based on a fast JSON parser.

I was initially very resistant to this idea of wrapping or replacing the glTF loader, but I'm coming around. Because:

  • glTF is so central to 3D Tiles and Cesium in general, and likely to be moreso over time (e.g. 3D Tiles Next). Cesium is heavily invested in glTF.
  • CPU and memory efficiency really do matter here, given the large number of glTFs loaded and rendered by 3D Tiles.
  • We're not particularly thrilled with any of the existing glTF loaders out there.
  • As mentioned above, switching our glTF loader later will be a (painful) breaking change in cesium-native.

Extras serialization fails if array encountered

#include "catch2/catch.hpp"
#include <CesiumGltf/Reader.h>
#include <iostream>

using namespace CesiumGltf;
TEST_CASE("Nested extras serializes properly") {
  const std::string s = R"(
    {
        "asset" : {
            "version" : "1.1"
        },
        "extras": { 
            "A": "Hello World",
            "B": 1234567,
            "C": {
                "C1": {},
                "C2": [1,2,3,4,5]
            }
        }
    }
)";

  ModelReaderResult result = CesiumGltf::readModel(
      gsl::span(reinterpret_cast<const std::byte*>(s.c_str()), s.size()));

  for (const auto& err : result.errors) {
    std::cout << err << std::endl;
  }

  REQUIRE(result.errors.empty());
}

This is valid Json but I get
glTF JSON parsing error at byte offset 224: Parsing was terminated. when I run this. We need this to work for our work so if anyone has a few minutes to look into it that would be 👌

Performance testing and optimization

This is a time-boxed (10 days) task to examine various aspects of the cesium-native and Cesium for Unreal performance to measure where it spends its time/memory and look for ways to improve it. Including:

  • Memory usage
  • CPU usage
  • GPU usage
  • Number of tiles requests

We should look for ways to instrument the tile load pipeline in order to collect statistics, such as:

  • how long tiles take to download a tile
  • how long it takes to parse a glTF
  • how long it takes to decode Draco buffers and images
  • how long it takes to upsample tiles for raster overlays
  • total time from "tile first identified as needed" to "tile ready to render"

Unreal Engine has a nice system for collecting this kind of information without a major impact on performance, but we can't easily use that from cesium-native. What can we do instead?

Review the CMake- and build infrastructure

First a disclaimer: I'm not a CMake expert, and not an expert for the setup, structuring, building, installation, dependency management, and deployment of C++ projects. But there are some things that caused me to raise an eyebrow when getting started. A such, this is more of a brain dump - and although I try to structure this sensibly, some of these points are strongly related to each other. Responses to some of these points may vary between "That's just the way it is, get over it" and "Yes, that will be changed, and the progress for doing so will be tracked in issue XYZ", but I want to mention them, at least.


(Note: It might be that the "root cause" of the first points is mentioned in the last point, namely, that there are "No installation options", but I'm not entirely sure about that)


  1. The use of nested submodules.

Right now, cesium-unreal-demo contains cesium-unreal as a submodule. And cesium-unreal contains cesium-native as a submodule. And cesium-native contains all external libraries as a submodule. I see the advantages of submodules, particularly for third-party libraries, and to some extent even for ~"dependency/version management" during the development of in-house libraries. But it has some drawbacks. (FWIW, I have created a .BAT file locally, to do a cd cesium-unreal-demo/Plugins/cesium-unreal/extern/cesium-native...). One technical limitation is that it is essentially not possible to have a directory structure like

  • \cesium-native
  • \cesium-unreal (using cesium-native)
  • \cesium-other (also using cesium-native)

When having a branch featureX in cesium-native, and a matching branch featureX in cesium-unreal, keeping the checked-out versions and submodule versions in sync is a bit clumsy.


  1. The project/build is huge (and slow)

Right now, when checking out and building cesium-native, the resulting Visual Studio solution contains more than 40 (forty) projects. The build takes a considerable amount of time (at least, much longer than the "core" cesium-native build would take, which consists only of 3DTiles, Geospatial, Geometry, Utility, and Async). Every change in the CMake file will trigger a full reload/rebuild of all libraries.

The thing that makes this look odd for me is: It's building all dependencies, each time. For example, the "draco" libraries. They are checked out in one version. We do not modify anything there. They will hardly ever change. They should be built once, then installed, and the result should simply be used by cesium-native.


  1. The project structure in Visual Studio

One result of 2. is that it's becoming increasingly hard to use Visual Studio sensibly. For example, searching for a certain function name takes roughly 30 seconds (because it's searching in the whole solution, and not only in the cesium-native core libraries). The list of "Header Files" in the solution explorer for Cesium3DTiles contains roughly 150 files, and many of them are not header files of Cesium3DTiles.

(FWIW: I usually have an Explorer and TextPad open. The Explorer shows the actual header files. TextPad does a full-text search in ~1 second. But this should not be necessary).


  1. Dependency management in CMake

CMake is pure anarchy. There are always 50 ways to achieve the same thing, and usually, 49 of them are wrong in one way or the other. The fact that there are no "best practices" is annoying, and the fact that "established practices" seem to have changed significantly between CMake 2.x and 3.x makes it even harder. I've used CMake 2.x a while ago, and now started reading, for example, https://pabloariasal.github.io/2018/02/19/its-time-to-do-cmake-right/ , as an update. But it's not trivial to sensibly apply this in practice. However, if someone wants to use, for example, Cesium3DTiles, then he should be able to do this by adding

find_package(Cesium3DTiles 1.0 REQUIRED)
target_link_libraries(example Cesium3DTiles::Cesium3DTiles)

to the CMakeLists.txt and be done. There should not be the need to add cesium-native as a Git submodule. There should not be a need to do some add_subdirectory. It should not be necessary to manually set any Cesium3DTiles_INCLUDE_DIR variable.


  1. Compilation settings in CMake

The way how the third-party libraries are currently integrated via CMake makes it hard to apply the high warning levels that should be standard for the Cesium libraries. I'm not familiar with the way how things like that are done in CMake 3.x nowadays, but the classification of dependencies, headers, or target_compile_features as SYSTEM, PUBLIC, PRIVATE seem to offer some options to potentially solve that more cleanly.


  1. No installation options

This may be the root cause of many of the aforementioned points. The workflow/structure that I knew until now was that there was a library (in-house or third-party). This had some CMake files that could be used to create the Visual Studio project. The project then had an INSTALL target that could be executed, causing the (compiled) dependencies and headers to be put into some thirdParty/Debug/xyz directory, with include and xyz.lib being there, ready to be picked up by find_package. The actual VS solution that was actively developed then consisted of a single project, with all the dependencies ready and in place. I have never set up such an "INSTALL" infrastructure on my own, though (only "copied+pasted" the specific parts that have been required for new projects), and all this was with CMake 2.x, and the way how this is set up may have changed considerably since then.

Consider adding Listener mechanisms

This has low priority*, but we might want to talk about this at some point:

There are, very broadly speaking, some elements and structures that may undergo modifications, in a way that makes it necessary (or at least beneficial) to have the option to be informed about these changes.

One very specific and concrete example here is the Camera. It has several functions like updatePositionAndOrientation and updateViewParameters that modify the internal state. Right now, it is not possible to detect such modifications.

I tried to have a look at how the Camera is handled in cesium-unreal. There is quite some code involved that I still have to wrap my head around, but from an attempt to "zoom" into the place where the actual camera is used, it appears to be that in ACesium3DTileset::Tick, a new Cesium camera is created from the state of the Unreal Camera and the unreal Viewport. This camera is then passed to the Tileset::updateView function. With this pattern, it is not necessary to detect changes, but it is very specific for Unreal (and may even depend on bCanEverTick being set, deciding whether Tick is called in each frame...).

From an Unreal-agnostic, high-level viewpoint, one could consider to have something like

camera.addListener([]() { renderBecauseCameraChanged(); });

More generally, being able to be informed about state changes can be beneficial for other classes as well. This may refer to direct, functional use-cases (like the camera changes that may trigger a rendering), but also to things like logging, roughly as in

tileset.addListener([] (Info &info) { log("{} tiles loaded, take one down and pass it around", info.numTiles) });

Whether or not (and how) such structures should be added is open for debate. The Camera was the class that essentially triggered this thought. It might be the only class where this could make sense at the moment.


* It has low priority, because such a listener mechanism can always be added in a later release. One exception is when there already is a callback mechanism, as in something.bind(callback), where one would have to essentially consider allowing "multiple callbacks", as in something.addListener(callback); something.addListener(anotherOneThatShouldAlsoBeCalled); .

Discussion about the traversal process

The current traversal procedure of tiles is - virtually by definition - the most simple traversal process that will ever exist.

In the future, there will be ...

  • additional culling criteria.
  • additional visibility checks.
  • additional flags that can be switched on or off.
  • additional tile types and providers.
  • refinements for the refinements.
  • extensions for the prioriztations of tile loading.
  • additional information (statistics, performance measures) that have to be collected during the traversal.
  • additional operations that are supposed to be done during the traversal.
  • additional strategies for the order in which the children of a tile will be visited.

Each of these could offer great options for new features and improvements. But it also means that the complexity of the traversal can only increase (and I'd say it's bound to increase dramatically). If there is no clear separation of concerns, no abstraction, and no clear, high-level outline for the process itself and the state transitions, the state space will face a combinatorial explosion.

But even though the current state is the most simple state that will ever exist, it's already ... complicated.


An aside: It may sound like an inappropriate and/or bold claim, but I'm pretty sure that we're already at a point where nobody knows exactly what happens during the traversal. One example for what caused my concerns here is that the traversal starts with this method:

Tileset::TraversalDetails Tileset::_visitTileIfNeeded(
    const FrameState& frameState,
    uint32_t depth,
    bool ancestorMeetsSse,
    Tile& tile,
    ViewUpdateResult& result

I tried to figure out the meaning of the ancestorMeetsSse flag. And I think this is essentially not possible. (Spoiler alert: It certainly does not say whether an ancestor meets the SSE). And when it's impossible to clearly state the conditions under which this flag, which is passed into the top-level-traversal call (!) and passed along (and changed occasionally...) is true or false, then that's not good: Any unit-test that is supposed to do something that is even indirectly affected by this flag could just capture the current state, but not say whether the current state is "right" or "wrong".


From an idealistic perspective, I wondered whether it could be possible to move towards a state where the traversal process looks more like this pseudocode...:

ViewUpdateResult updateView(ViewState viewState) {
    TraversalResult traversalResult = traverse(root, TraversalState(viewState, frameNumber));

    addTilesToLoadQueues(traversalResult.tilesToLoad);
    passTilesToRenderer(traversalResult.tilesToRender);
    removeTilesFromRenderer(traversalResult.tilesToRemove);
    
    return ViewUpdateResult(traversalResult);
}
TraversalResult traverse(Tile tile, TraversalState traversalState) {
    TraversalResult traversalResult(traversalState);
    
    if (shouldNotBeVisitedForWhateverReason(tile)) {
        traversalResult.visited = 0;
        return traversalResult;
    }
    if (!isVisible(tile, traversalState)) {
        traversalResult.culled = 1;
        butMaybeLoadSiblingsOf(tile, traversalResult);
        return traversalResult;
    }
    if (shouldBeLoaded(tile, traversalState)) {
        load(tile, traversalResult);
    }
    if (shouldBeRendered(tile, traversalState)) {
        render(tile, traversalResult);
    }
    if (shouldBeRefined(tile, traversalState)) {
        refine(tile, traversalResult);
    }
    if (shouldBeCoarsened(tile, traversalState)) {
        removeChildren(tile, traversalResult);
    }
    for (Tile child : tile.children) {
        TraversalState childTraversalState = traversalState.withDepthPlusOne();
        TraversalResult childTraversalResult = traverse(child, childTraversalState);
        traversalResult.combineWith(childTraversalResult);
    }
    return traversalResult;
}

Yes, I know that it cannot be that simple. But some rough (!) ideas reflected by this pseudocode being

  • There is a top-level updateView function. It receives the FrameState that is constant throughout the traveral. It returns the ViewUpdateResult that summarizes the result of the whole traversal, and contains exactly the information that is supposed to be available for clients (that's roughly as in the current implementation)
  • There is an internal traverse method (which might even be encapuslated in some TilesetVisitor, FWIW). It receieves a TraversalState that is constant for that particular branch. It returns the TraversalResult which summarizes the information that was accumulated while traversing that particular branch.
  • The TraversalResult of consists of the information for the given tile, combined with the traversal results for all its children
  • The conditions for when a tile shouldBeLoaded, shouldBeRendered, shouldBeRefined ... are clearly stated (in the documentation of the respective methods). More broadly speaking, the goal would be to assign clear, concise meanings to terms like "visiting", "loading", "rendering", "refining"
  • It should be clear where state transitions take place.

The state transitions (reflected with the LoadState and TileSelectionState::Result) already are complex, and every attempt to simplify this will require diligence, a deep understanding of the process, and the goals. But for example, stating the criterion for "rendering a tile" with a method like

bool shouldBeRendered(Tile tile, TraversalState traversalState) {
    if (traversalState.parentWasRendered) {
        return false;
    }
    if (!tile.isLoaded()) {
        return false;
    }
    if (isAnyLoading(tile.children)) {
        return false;
    }
    return true;
}

and establishing a clear connection between the decision of whether a tile should be rendered and the state change, as in

if (!tile.rendered) {
    if (shouldBeRendered(tile)) {
        render(tile);
        tile.rendered = true;
    }
}

could allow to document the meaning of "rendering", and the unit tests could be straightforward, as in

assertFalse(shouldBeRendered(unloadedTile, traversalState));
assertFalse(shouldBeRendered(tileWithLoadingChildren, traversalState));
assertFalse(shouldBeRendered(tile, traversalStateWithRenderedParent));
assertTrue (shouldBeRendered(tile, traversalState));

or

assertFalse(tile.rendered);
render(tile);
assertTrue (tile.rendered);

In contrast to that, and referring to the (maybe overly specific) example above: The ancestorMeetsSse flag will (among other things) affect whether a tile will be "rendered". And whether it is true depends (among other things) on whether a child of the tile is "not renderable". And a child tile is "renderable" when...

this->getState() >= LoadState::ContentLoaded &&
(!this->_pContent || this->_pContent->model.has_value()) &&
!std::any_of(this->_rasterTiles.begin(), this->_rasterTiles.end(), [](const RasterMappedTo3DTile& rasterTile) {
    return rasterTile.getLoadingTile() && rasterTile.getLoadingTile()->getState() == RasterOverlayTile::LoadState::Loading;
});

I cannot imagine how to write a sensible test, stating the conditions under which a certain tile will be "rendered", and achieving a sensible coverage of the state space that determines whether these conditions are actually met or not.


I know that there is nothing harder than trying to make complex things simple. But I think it is crucial to at least try to work towards such a simpler, clearer state. I also know that what I wrote here may appear to be naive or overly idealistic, and it does not allow deriving immediate actions. But maybe others can share their thoughts or ideas about the possible paths forward.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.