https://github.com/uber-web/loaders.gl/blob/92c25fad40ebe9b8c83fde9f3889e0078b9388d6/modules/gltf/src/gltf/gltf-parser.js#L307

getImageSize tests only work under Node.

https://github.com/uber-web/loaders.gl/blob/ecbca8a9c76aa3482d02198342f825c9e406b819/modules/gltf/src/gltf/gltf-parser.js#L311

I can't make it work with rollup? any example? thanks

Reported in aurora-opensource/xviz#384

The text-decoder polyfill is no longer maintained and this generates warnings during install which is unsettling for users.

Suggested action:

• Move the text-decoder polyfill into loaders.gl and cut the dependency.

@ibgreen @tsherif The deck.gl SimpleMeshLayer requires normals in the mesh attributes. It is not technically required to construct a luma Geometry; however our shading module will not work without it.

The majority of mesh files that I encounter do not contain normals. This can be seen in 2/3 of our test files, and online repositories here, and here.

Normals can easily be calculated from a triangle. We could theoretically back fill this missing attribute by averaging the normals at each vertex. We may need to investigate how this can be done with memory efficiency in the streaming case.

We can add a utility for this to @loaders.gl/core and share it among draco/ply/obj.
Having this option will make our mesh stack much more usable.

See RFC.

While debugging draco compression being added to XVIZ it seems the interface won't work just with TypeArrays as the current packBinaryJson() is setup to do. There is more context that is necessary to properly pack than is currently available.

The flow in xviz today:

1. XVIZBuilder adds typed arrays to object
2. Writing a frame will call gltfBuilder.addApplicationData() which will call packBinaryJson()
3. packBinaryJson() walks the object recursively and will call addImage() or addBuffer()

Problems:

1. We need to call addCompressedPointCloud() not addBuffer() so some context is required on an object to know if it is a point cloud or other. GLBBulider nor GLTFBuilder cannot know this generically

2. We need to gather attributes (well, we can gather attributes (color & points)) and the current object iteration operates on a single field at a time

3. When I hacked the above colors appears to expect 3 values per color, not 4 :(
   3a. In the hack, the the JSONPointer replacement needs to be understood. Could be solved with the right interface but it's an question to answer.

4. Everything done here needs to be equally handled in the parser

(deckgl) alleon_g$ npm i @loaders.gl/gltf --save
npm ERR! path /Users/alleon_g/0x-TechWork/jsbsim-react/jsbsim-view/node_modules/@loaders.gl/gltf/bin/glbdump.js
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall chmod
npm ERR! enoent ENOENT: no such file or directory, chmod '/Users/alleon_g/0x-TechWork/jsbsim-react/jsbsim-view/node_modules/@loaders.gl/gltf/bin/glbdump.js'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR! /Users/alleon_g/.npm/_logs/2019-02-28T05_30_59_335Z-debug.log

The link on loaders.gl site points to https://loaders.gl/docs/get-started, and gives a GitHub pages error.

3D Tiles

• Cesium Commercial Tiling Service
• github: 3d-tiles-tools - Processes tile files but does not generate tiles

Potree

• [Generating Potree] (https://wiki.cyverse.org/wiki/display/~tyson_swetnam/Generating+Potree+point+clouds)
• github: Potree Converter
• github: generator-potree

EPT Tools

• EPT Tools project (which also demonstrates conversion from EPT->3D-Tiles) in Javascript, and PDAL's EPT reader in C++.

PDAL

• I3S point cloud is implemented in PDAL and the LEPCC encoder it uses is known to compile to JavaScript, but the PDAL project hasn't done any of that so far.

• I wanted to support Progress callbacks like XMLHttpRequest does, but fetch API doesn't AFAIK. Maybe the new browser streams could work here?

When .svg files contain, for example chinese characters, parse-image.js throws error: Uncaught (in promise) DOMException: Failed to execute 'btoa' on 'Window': The string to be encoded contains characters outside of the Latin1 range.

Using btoa(unescape(encodeURLComponent(xml))) instead of btoa(xml) could solve this problem to make parse-image.js more robust.

TypeError: "2 is not a valid argument count for any overload of Window.createImageBitmap."

See https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/createImageBitmap#Browser_compatibility

You already have one example for glTF loader: https://loaders.gl/examples/gltf

But it is hard to understand for entry level. I believe a lot of code here because of select boxes.

Examples for feature - only are better for documentation: https://threejs.org/docs/index.html#examples/loaders/GLTFLoader

If you expect that your library will use only experienced developers in webgl, you can close this issue unresolved.

Also see #7 GLTF Punchlist

v1.1

• Add conformance tests for loader object API (inverse test with dummy loader)
• loaders should not import core parser/encoder drivers. Split new utils method out of core, e.g. separate the minimal parser/encoder drivers from loader utility methods.

Also see #7 GLTF Punchlist

v1.0

• Finalize design decisions on loader object API
• Finalize design decisions on mesh formats, add conformance tests.
• Refine module structure, particularly core and images
  • Collect concerns about using core
  • Reduce size of core
  • Define clearly what goes into core and images
• Define mechanism to register loaders so that they are available during parse
  • Allows apps to cherry pick what loaders to include, and other frameworks to leverage them.
• Clean up core image API (@ibgreen)
• Fix browser image handling in core and images (@ibgreen)

v0.9

• ocular-gatsby website (@jckr)
• Update deck.gl examples to use loaders.gl (@gnavvy)

v0.8+

• Pass on docs
• Adopt ocular-dev-tools

v0.7

• Initial streaming support
• remove IO code in deck.gl/test-utils (@ibgreen)
• Copy luma.gl IO libraries into loaders.gl and consolidate (@ibgreen)
• Move @deck.gl/test-utils IO code into loaders.gl (@ibgreen)
• Add conformance tests for mesh category

v0.6

• New module @loaders.gl/images - Reconstruct luma.gl's original Node-based image loaders as new submodule. (@ibgreen)
• Log handling
• All loading through loadFile/parseFile instead of using loaderObject methods
• Remove IO code in luma.gl (@ibgreen)
• Basic Threaded loading support

v0.5

• Fixes for luma.gl glTF loading
• Fix LAS loader (and make it synchronous) #20

v0.3.x Add point cloud loaders

• One npm module per format
• Test cases for LAS/PLY/PCD/OBJ
• Fix OBJ mesh loader

OSS tasks

• Break out from luma.gl
• Approved
• npm license crawler for dependencies
• Flip public switch
• Publish docs on gh pages

As of uber-web/ocular#197, published modules properly include their full license text. deck.gl has published v7.0.10 in response, can this repo do the same? Thanks!

v1.2

MISSING FUNCTIONALITY

• parseFile - Binary/Text parsing selection logic is fragile/likely incorrect in a few cases.
  • select based on input (string vs ArrayBuffer)
• loader.test... - no way to test binary files. Most appropriate in many cases.
• decodeURI - Uses Buffers instead of ArrayBuffers (loaders.gl is trying to standardize on the latter). Apart from the code base dissonance, this might make bundler include some Buffer polyfill?

General Perf

• Add benchmarks for all Parsers...
• loader.testText - requires converting entire file instead of just header

PLYLoader

• PLYLoader (binary) converts entire buffer to text just to parse header...

addBufferView in gltf-scenegraph.js is handling the bufferOffset incorrectly which causes addImage to be broken.

• this.byteLength is not initialized
• this.byteLength += padTo4Bytes(byteLength); has to be done after glTFBufferView is constructed

This looks like an easy fix but I currently have no working dev setup so I would be very glad if this could be fixed.

Get started

• Examples
  • dated API, refresh, delete?

Developer Guide.

• Big ints

  • better explanation needed #214
  • are they just 64 bit ints, or even bigger, variable length ints? Some more examples...

• data frame operations

  • Q: Does length queries get updated when filters are set, or just affects iteration?

• data types

  • needs a bit more text, add more examples

• data sources and sinks

  • A bit more text/examples

• Memory management - a bit more text.

  • [] Q: Assume memory chunks often get reallocated/joined on ingestion from streams if batches span multiple arraybuffer chunks.

• Tables

  • Double check examples...

• Typescript

  • Anything you want to add? Any ingenious constructions worth pointing out?

API Reference

TBA

Strings here modules/gltf/src/gltf/gltf-builder.js must be known and documented to access this data.

The issue is present in @loaders.gl/gltf at 1.0.3 as used in @xviz/builder at v1.0.0-beta.10, but appears fixed in by #179. @twojtasz and @ibgreen not sure if belongs in #7 at all, but wanted to file this separately so I can note the work around I have to do in another library.

Specifically the older code wrote the un-padded length out to the GLB file while in the new code it is fixed.

v2.x

• Move ImageLoader to a separate @loaders.gl/image module.

v2.1

Docs

• Document what image formats are supported for the ImageLoader
• Document what image formats are supported for the functions like the image type detection util (that extracts MIME type and size) #276
• Write Image category docs

Tests/Examples

• Add example with one test file for each of the supported formats, to show that they loads in Chrome etc. See https://toji.github.io/texture-tester/

Code

• ImageLoader worker support with throughput benchmarks
• Data textures: options.image.data #597
• Add test cases to BasisLoader #596
• Fix ImageBitmap support
• Support path prefix #541

v2.0

• Making image loaders into pure async data parsers (i.e remove loadAndParse from loaders.gl).

https://github.com/uber-web/loaders.gl/blob/ecbca8a9c76aa3482d02198342f825c9e406b819/modules/gltf/src/gltf/gltf-parser.js#L222

v2.x

worker support

• Set up yarn start-local or yarn start-local-worker to use local workers.
• Add documentation about how to copy worker scripts to app to avoid relying on CDN (partly done)
• Add debug option to debug workers - add logging and/or break in debugger
• Finalize worker override system/security audit (options vs dedicated API etc)
• New API initializeWorkers([DracoWorkerLoader, ...]) (let app preload worker scripts before parsing is needed for reduced "time to first pixel")
• Better control of max concurrency (global options, per-loader options, auto detect number of cores, ...)

stats

• time worker startup times
• time parsers
• Export stats object from loaders.gl
• Total time in parsers
• Total time in parsers on main thread
• Time to first data (SSL initialization etc)

embedded binaries

• Flag to indicate whether binary loader supports offsets or whether arraybuffer copy is needed before invocation?
• Implement arraybuffer copy before call if needed (auto detect alignment issues?)
• DracoLoader: Binary test function handle typed arrays with offsets (embedded binaries): byteOffset, byteLength
• Ability to parse embedded binaries on workers (byteOffset, byteLength) (copy and transfer?)
• Binary test function handle typed arrays with offsets (embedded binaries): byteOffset, byteLength

v2.0

• Build ES6/ES2018 debug workers for all loaders
• Build ES6/ES2018 debug workers - Draco only
• Conditional workers (Image only when imagebitmap output, CSV, JSON only when binary output etc) @ibgreen
• Update worker override system (loader-specific options) @ibgreen
• Conditional worker support: loader.useWorker(options) @ibgreen
• Support calling "worker parsers" from other worker parsers @Pessimistress
• Support calling "async parsers" from workers @Pessimistress

loader objects

• Remove loadAndParse

loader registry and selection

• registry is currently a map with each extension, which causes same loader to be tests many times...
• More rigorous selection, documented semantics, public API

I'm seeing the following error on this sample model:

gltf-post-processor-old.js:175 glTF file error: Could not find samplers[undefined]

The texture.sampler property is not required, and can use a default value if unspecified:

willow2.glb.zip

2.1

• Fix filedrop
• Simplify Metrics and metrics panel

1.3

• pointcloud example: separate out mesh layers
• ionAccessToken can not be passed from webpack config
• minimum loader examples
• 3d tiles example: correctly display instanced 3d tile examples
• gltf example: Breakdown example into main and control-panel files

1.2

• stats not update correctly
• auto extract viewport from tileset file
• load instanced 3d tile examples

This issue tracks remaining work for the table category of loaders

loaders.gl v2.1

@loaders.gl/core

• Finalize streaming (parseInBatches/loadInBatches) architecture, implementation and docs. (@ibgreen)

@loaders.gl/tables (New Table category module)

• Rename @loaders.gl/experimental module to @loaders.gl/tables (@ibgreen)
• Document TableBatch, schema etc in @loaders.gl/tables (@ibgreen)

General Options

• Add `table.rowType: 'object'/'array'? whether table loaders returns arrays or objects @ibgreen

ArrowLoader

• Define metadata mappings

CSVLoader

• Add `csv.rowType: 'object'/'array'? whether table loaders returns arrays or objects (For CSV with header the async parser currently returns arrays...) see table category @ibgreen
• Keep Papaparse? We have some indication that there is a faster CSV parser available (at least for the "external" CSV to "internal" Arrow case), so maybe we shouldn't invest too much in this implementation. @ibgreen

JSONTableLoader

• Incremental/streaming JSONLoader (based on clarinet or jsonstream?) (@ibgreen)
• Minimal implementation of JSONPath
• WorkerLoader setup
• Keep separate JSONLoader / JSONTableLoader or just use options

XMLTableLoader P2

• Incremental/streaming XMLLoader (based on sax-js?) (@ibgreen)

Not yet deployed

• deck.gl based examples don't resize properly P1
• project dropdown doesn't work properly P1

Deployed

• Get started link is wrong #261 P1
• Layout is broken if accessing doc page directly from URL (e.g. https://docs/contributing) P0
• Topbar Github link is broken P1
• loaders.gl dropdown doesn't work P2

The LASLoader uses a webworker internally.

Currently the worker setup doesn't work, so the LAS parser is not functional.

The target design for loaders.gl is that parsers are synchronous and can be run either on main thread or on worker, so ideally we should refactor the LASLoader accordingly as we implement generic worker support.

I met this problem, it caused by sometimes the process is not undefined but the process.version is undefined, in @loaders.gl/core/dist/esm/utils/globals.js:45, process.version.match will cause this problem.
Cannot read property 'match' of undefined
TypeError: Cannot read property 'match' of undefined at Module. (http://localhost:8000/vendors.async.js:19546:65) at Module../node_modules/@[email protected]@@loaders.gl/core/dist/esm/utils/globals.js (http://localhost:8000/vendors.async.js:19549:30) at webpack_require (http://localhost:8000/umi.js:775:30) at fn (http://localhost:8000/umi.js:130:20) at Module../node_modules/@[email protected]@@loaders.gl/core/dist/esm/javascript-utils/stream-utils.js (http://localhost:8000/vendors.async.js:17622:72) at webpack_require (http://localhost:8000/umi.js:775:30) at fn (http://localhost:8000/umi.js:130:20) at Module../node_modules/@[email protected]@@loaders.gl/core/dist/esm/lib/loader-utils/get-data.js (http://localhost:8000/vendors.async.js:18590:88) at webpack_require (http://localhost:8000/umi.js:775:30) at fn (http://localhost:8000/umi.js:130:20) at Module../node_modules/@[email protected]@@loaders.gl/core/dist/esm/lib/parse-with-loader.js (http://localhost:8000/vendors.async.js:18906:80) at webpack_require (http://localhost:8000/umi.js:775:30) at fn (http://localhost:8000/umi.js:130:20) at Module../node_modules/@[email protected]@@loaders.gl/core/dist/esm/lib/parse.js (http://localhost:8000/vendors.async.js:19111:76) at webpack_require (http://localhost:8000/umi.js:775:30) at fn (http://localhost:8000/umi.js:130:20) at Module../node_modules/@[email protected]@@loaders.gl/core/dist/esm/index.js (http://localhost:8000/vendors.async.js:16703:68) at webpack_require (http://localhost:8000/umi.js:775:30) at fn (http://localhost:8000/umi.js:130:20) at Module../node_modules/@[email protected]@@deck.gl/core/dist/esm/lib/init.js (http://localhost:8000/vendors.async.js:7703:74) at webpack_require (http://localhost:8000/umi.js:775:30) at fn (http://localhost:8000/umi.js:130:20) at Module../node_modules/@[email protected]@@deck.gl/core/dist/esm/index.js (http://localhost:8000/vendors.async.js:3622:67) at webpack_require (http://localhost:8000/umi.js:775:30) at fn (http://localhost:8000/umi.js:130:20) at Module../node_modules/@[email protected]@@deck.gl/react/dist/esm/deckgl.js (http://localhost:8000/vendors.async.js:16227:71) at webpack_require (http://localhost:8000/umi.js:775:30) at fn (http://localhost:8000/umi.js:130:20) at Module../node_modules/@[email protected]@@deck.gl/react/dist/esm/index.js (http://localhost:8000/vendors.async.js:16410:65) at webpack_require (http://localhost:8000/umi.js:775:30) at fn (http://localhost:8000/umi.js:130:20) at Module../src/pages/entry/components/Map/MapGL.jsx (http://localhost:8000/p__testMap__index.async.js:19:72) at webpack_require (http://localhost:8000/umi.js:775:30) at fn (http://localhost:8000/umi.js:130:20) at Module../src/pages/testMap/index.js (http://localhost:8000/p__testMap__index.async.js:97:90) at webpack_require (http://localhost:8000/umi.js:775:30) at fn (http://localhost:8000/umi.js:130:20)

Related

• #257
• #424

Parsing code

• Implement Binary Attribute parser
• Implement HRC parser #349
• Add test cases to loaders.gl folder.

Conversion

• Convert HRC files and top-level json to standardized "3D tile category" data
• Define and set standardized "metadata" fields for tileset
• Calculate Bounding Boxes for each hierarchy node.

Tile traversal code.

• Update traversal algorithm to handle potree characteristics
• Update traversal to handle non-geospatial coordinates

Example

• Update example to handle non-geospatial clouds (e.g. scans of statues etc)
• Use COORDINATE_SYSTEM.IDENTITY, different controller, disable base map etc.

Notes on the potree data format:

• Only point clouds, using a basic octree dividing a top-level axis oriented bounding box (no kd-tree and advanced bounding regions like in 3D tiles).
• Because of the simple octree division system, the "tile files" have a simple predictable naming scheme r0703... where each digit hierarchically determines a sub-octant (0-7)
• The "tile files" are either LAS or LAZ encoded (which we already support) or just contain concatenated binary buffers for positions, colors etc.
• The points in the tiles are always additive (parent tile is rendered in addition to child tile).
• There is a minimum distance between points in the top-level tiles, which halves with each octree division.
• Just like 3D tiles, the "magic" of potree lies in the traversal and rendering code (selecting which tiles to load and render etc). We should be able to render potree data sets with a small additional effort.

Tooling Overview

3D Tiles

• Cesium Commercial Tiling Service
• github: 3d-tiles-tools - Processes tile files but does not generate tiles

Potree

• [Generating Potree] (https://wiki.cyverse.org/wiki/display/~tyson_swetnam/Generating+Potree+point+clouds)
• github: Potree Converter
• github: generator-potree

EPT Tools

• EPT Tools project (which also demonstrates conversion from EPT->3D-Tiles) in Javascript, and PDAL's EPT reader in C++.

PDAL

• I3S point cloud is implemented in PDAL and the LEPCC encoder it uses is known to compile to JavaScript, but the PDAL project hasn't done any of that so far.

https://github.com/uber-web/loaders.gl/blob/ecbca8a9c76aa3482d02198342f825c9e406b819/modules/gltf/src/glb/unpack-glb-buffers.js#L65

Open tasks moved to #1245

loaders.gl v2.0 (for deck.gl 8.0)

glTF-based Tile parsing

• @loaders.gl/gltf: Tile3DLoader and GLTFLoader to wait for all sub loaders P0 (@ibgreen)
• @loaders.gl/core: Improve support for sub-loaders P0 @ibgreen
• @loaders.gl/core: Move loader options into nested sub objects (to simplify sending options to sub-loaders) @ibgreen P0

loaders.gl v1.3 (for deck.gl 7.3)

Tile3DLayer

• Tests for Tile3DLayer P0 (@xintongxia)
• Audit Tile3DLayer P0 (@xintongxia)
• Move: Tile3DLayer to deck.gl P0 (@jianhuang01)
• Document Tile3DLayer P0 (@xintongxia)
• Tile3DLayer cleanup: Ion Asset Handling P0 (@xintongxia)
• Tile3DLayer cleanup: General P0 (@xintongxia)

3d-tiles example

• Clean up 3d-tiles example: Ion Asset Handling P0 (@ibgreen)
• Add ion logo when loading from Cesium ion server https://cesium.com/press/ P0 (@ibgreen)
• Fix the stats widget below tile selector (use stats object from 3d-tiles)
• Create reference sandbox to test tilesets in Cesium (@xintongxia)

3d-tiles module documentation

• Improve documentation of generic "3d-tiles loader category" P0 (@xintongxia / @ibgreen)
• Improve documentation of Tileset3D class P0 (@ibgreen)

glTF-based Tile parsing

• Fix WebGL issues (texture!?!) with glTF tilesets (@jianhuang01)
• Fix transform issues with gltf tilesets (@OmarShehata)
• Load photogrammetry-based tilesets (e.g vricon) (@xintongxia)
• Tile3DLayer: Map instanced model tile data to deck.gl ScenegraphLayer attributes #246 (@xintongxia)
• Tile3DLayer: Support batched gltf tiles #246 (@xintongxia)

Point Cloud Parsing

• Support quantized/scaled positions on GPU (@xintongxia / @ibgreen)
• Support quantized/scaled positions on CPU (@xintongxia / @ibgreen)
• Support oct-encoded normals on CPU (@xintongxia)
• Port attribute compression lib (oct encoding) #321 (@ibgreen)
• Support different color arrays without copying (@ibgreen/@xintongxia)

3d-tiles module code

• Add request synchronizer to avoid loading tiles no longer in view (@ibgreen)
• Fix/confirm cache unloading P0 (@loshjawrence)
• Perf: Solve excessive worker creation overhead (worker pool?) P0
• DracoWorkerLoader is not working (@xintongxia)
• Handle nested tilesets (@loshjawrence / @ibgreen)
• Calculate memory use per tile during parsing to enable cache unloads, see #328 (@ibgreen)
• Use probe.gl stats object to collect stats on loading and memory use (@ibgreen)

Tileset3D refactor

• ensure all loading/unloading and associated state keeping/stats tracking goes through a single set of clearly defined methods on Tileset3D.
• refactor Tileset3DTraversal so that it does not access Tileset directly (@ibgreen)
• refactor Tileset3D into public/private methods (@ibgreen)

deck.gl issues

• Bug: ScenegraphLayer opacity prop is broken (@jianhuang01)
• Bug: ScenegraphLayer modelMatrix is broken P0 (@jianhuang01)
• Bug: ScenegraphLayer PBR color (not texture) is broken (@jianhuang01)
• Bug: Unnecessary attribute copy in deck.gl (@jianhuang01)

loaders.gl v1.2 (for deck.gl 7.2?)

Traversal

• Integrate tileset traversal with math.gl/culling (@loshjawrence)
• Implement tile cache (@loshjawrence)
• Implement basic tileset traversal (@loshjawrence)

glTF-based Tile parsing
(@xintongxia)

• Demo: Fix ScenegraphLayer rendering for sync data? (@georgios-uber)
• Demo: Map batched model to deck.gl ScenegraphLayer? (@ibgreen)
• Connect 3DTile parsers to glTF parser (@ibgreen)
• Enable parsing of embedded GLBs at arbitrary byte offsets (@ibgreen) #163

loaders.gl v1.1 (for deck.gl 7.1)

Use the transforms in the tileset and the tile to position the tiles correctly on the base map.

• Add a base map to DEMO
• Show 3D tiles correctly positioned on top of base map (@xintongxia)
• Create a CompositeLayer that takes a Tileset3D and renders the various subtiles (@xintongxia)
• Load all tiles in tileset (@ibgreen/@xintongxia)
• Implement Ellipsoid class to enable conversion of WSG84 "cartesians" to cartographic/lnglatz. (@ibgreen)
• Implement Tile3DHeader.loadContent() (@ibgreen) #224
• Create Tile3DHeaders for all tiles in tileset (@ibgreen)
• Parse tilesets and create Tileset3D class, unit tests (@ibgreen)

Math primitives

• Geospatial: Refined Port Ellipsoid etc classes. #223
• Culling: Refined Port OrientedBoundingBox, BoundingSphere, PerspectiveFrustum, CullingVolume classes.
• Geospatial: Raw Port Ellipsoid etc classes. (@ibgreen)
• Culling: Raw Port OrientedBoundingBox, BoundingSphere, PerspectiveFrustum, CullingVolume classes. (@ibgreen)

Point Clouds

• Show point cloud correctly positioned on top of base map (@xintongxia)
• Demo: Map point cloud data to deck.gl PointCloudLayer attributes (@xintongxia)
• Implement Draco support for point cloud tiles (@xintongxia)
• Implement support for various point cloud formats (@xintongxia)
• Implement batch table parsing (@ibgreen)

Initial Tile Parsing

• Add unit test data from Cesium repo (@ibgreen)
• Implement composite tile parser (@ibgreen)
• Implement point cloud tile parser (@ibgreen)
• Implement instanced model tile parser (@ibgreen)
• Implement batched model tile parser (@ibgreen)

See #91

How to reproduce?

Go to examples/las, replace the LASLoader with LASWorkerLoader. And do yarn start-local.

Line 8 - 13 should like the following,

import {LASWorkerLoader} from '@loaders.gl/las';
// import {PLYWorkerLoader} from '@loaders.gl/ply';
import {load, registerLoaders} from '@loaders.gl/core';

// Additional format support can be added here, see
registerLoaders(LASWorkerLoader);

There seems to be an extra fetch related class to handle abort now, not sure if it is portable between browsers yet.

See #92

Talked to Sean, here's the documentation for making requests to ion
https://cesium.com/docs/tutorials/rest-api/
https://cesium.com/docs/rest-api/#section/Overview
https://cesium.com/docs/rest-api/#operation/getAssetEndpoint

Example code to retrieve a list of assets available with the supplied accessToken

const request = require('request-promise');
const accessToken = '<your_access_token>'; // Replace this with your token from above.
const response = await request({
url: 'https://api.cesium.com/v1/assets',
headers: { Authorization: Bearer ${accessToken} },
json: true
});

This returns a json where where the items member contains an array of asset metadata. The id is used to request information about how to access it.
{"items":[{"id":1,"type":"3DTILES","name":"AGI HQ","description":"AGI Headquarters","bytes":0,"attributition":"AGI","dateAdded":"2019-05-29T18:56:13.088Z","status":"COMPLETE","percentComplete":100}, ...]}

Example code to get information about the tileset related to an asset id:

// Issue a GET request for the metadata
const assetMetadata = await request({
url: https://api.cesium.com/v1/assets/${assetId}/endpoint,
headers: { Authorization: Bearer ${accessToken} },
json: true
});

As an example, this could return something like:

{
"type": "3DTILES",
"url": "https://assets.cesium.com/23912/tileset.json",
"accessToken": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJqdGkiOiIyMjZmNzU0Yy02ZmVkLTQ4ODktOTUyMC0zMDRlMmNjODdiMzEiLCJpZCI6NDQsImFzc2V0cyI6eyIxIjp7InR5cGUiOiJURVJSQUlOIiwiZXh0ZW5zaW9ucyI6W3RydWUsdHJ1ZSx0cnVlXSwicHVsbEFwYXJ0VGVycmFpbiI6ZmFsc2V9fSwic3JjIjoiOGE2NjVmYmMtZDQzMy00ZmMxLWE3NDgtNjRhYWE3MjFiOTgzIiwiaWF0IjoxNTU0MTI4NTUzLCJleHAiOjE1NTQxMzIxNTN9.CWjqdF1LORmCd7nVbHupXyIjCPOfuSOpXwQoaQU_a94",
"attributions": [
{
"html": "<a href="https://cesium.com" target="_blank"><img alt="Cesium ion" src="http://assets.cesium.com/ion-credit.png">",
"collapsible": false
}
]
}

Looking at https://cesium.com/docs/rest-api/#operation/getAssetEndpoint
the url is for the tileset.json and the accessToken is as specified:

The token to be included with each tile request, using Bearer format. This token is different than the access token used throughout the rest of the REST API. It provides access to the asset for approximately one hour. To continue using the asset after the allotted time, simple re-request the endpoint for a new token.

There are indications that we might be including too many tiles during traversal (which obviously will negatively affect loading and rendering performance). Based on suggestions from @loshjawrence:

General debugging tools for culling

• Break out a second debug camera out from the render camera
• Draw debug outline meshes for tile bounding boxes and view frustum

Less costly approaches:

• For culling maybe edit the camera transform so its pushed back from where it should be, then the culling planes would occupy a smaller space on the screen and you can hopefully see the tiles getting cullled when they move to the edge of the screen (before they actually go off)
• Maybe edit the tile color to be the current closests distance to any cullling plane
  -[ ] Color the tile based on its bounding radius/diagonal relative to the root's bounding radius/diagonal. hopefully gets darker as you go down the tree
  unfortunately the problem is probably deeper than what any of this would reveal.

Possible investigations tracks:

• For region-based bounding boxes, ensure we transform into optimal BoundingSpheres or OBBs. (@xintongxia)
• Add Tileset3d.stats indicators to indicate if any tiles in tileset use regions and other features that may be suspected to cause issues (@xintongxia)

This seems to have regressed when loaders.gl was broken out from luma.gl:

• Fix glbdump in modules/core/.bin
• ensure it works in user modules when loaders.gl is installed.
• Ideally, it should be possible to run glbdump (npx glbdump) while working inside loaders.gl, not just modules/core

As far as I can tell, worker threads are started when importing a module, even if the code never references the WorkerLoader object.

This happens in glTF loader, where the Draco worker starts even when not used.

We may need to refine our bundling strategy.

Also the Draco worker appears to keep receiving empty messages in an "infinite loop", making debugging very sluggish.

@Pessimistress

v2.x

glTF standards compliant loader: missing features

• morph/skin/joint animations (@georgios-uber)

v2.1

Loader Features:

• Support basis subloader (@georgios-uber)

Other

• Support API simplifications in luma.gl
• Fix async issue in deck.gl

Tests:

• Add glTF/GLB test files with base64, linked files, Draco... (@georgios-uber)
• Unit tests (@georgios-uber)
• Render tests (@georgios-uber)

Extensions:

• Test Uber point cloud extension
• Implement fallback semantics for KHR draco extension per spec (@georgios-uber)
• Make sure pre-decoded Draco glTF is indistinguishable from non-Draco file (currently attribute descriptors are slightly different).

v2.0

• Support draco subloader (@ibgree)
• Use nested options.gltf (@igbreen)
• Remove v1 glTF parser (@ibgreen)
• Get v2 glTF parser to feature parity with v1 parser (@ibgreen)

Extensions:

• Support material_unlit extension (@ibgreen)

v1.2

• Refactor: Remove GLTF Accessor/Image knowledge from GLB classes (Separation of concerns).
• Add manual draco mesh decode option for XVIZ.

v1.0

• change from nopack -> packTypedArray @ibgreen
• Documentation: Cleanup pass on GLTF docs @ibreen
• Documentation: GLTF extension support (#65) @ibreen
• API Audit: GLBBuilder,GLTFBuilder - nopack option renamed to packTypedArrays (#94) @ibreen
• Refactor: Break out non-standard postprocessing from GLTFParser (Separation of concerns). @ibreen
• Feature: GLTFLoader & GLTFParser.parse() supports async loading of linked resource files and base64 encoded buffers. @georgios-uber
• Feature: GLTFParser.parse() supports the following inputs: ArrayBuffer containing GLB, ArrayBuffer containing UTF8 encoded JSON, string containing JSON as well as decoded JSON. @ibreen

v0.5.x XVIZ integration

• Draco point cloud support (interface options/docs on how to use / expose)
• Verify functional support parity with existing code or changes in XVIZ as required
• Verify image support in XVIZ
• removal of XVIZ magic header

v0.4.x Flesh out GLTF support

• move JSON pack/unpack to same layer in stack (GLTFBuilder/GLTFParser)
• Fluent GLBBuilder/GLTFBuilder APIs
• Fluent GLTFBuilder API for adding extensions
• Remove custom jsonKey, in favor of extensions/extras