Git Product home page Git Product logo

cjio's Introduction

cjio, or CityJSON/io

License: MIT image1

Python CLI to process and manipulate CityJSON files. The different operators can be chained to perform several processing operations in one step, the CityJSON model goes through them and different versions of the CityJSON model can be saved as files along the pipeline.

Documentation

cjio.readthedocs.io

Installation

It uses Python 3.7+ only.

To install the latest release:

pip install cjio

Note

The commands export, triangulate, reproject, and validate require extra packages that are not install by default. You can install these packages by specifying the commands for pip.

pip install 'cjio[export,reproject,validate]'

To install the development branch, and still develop with it:

git checkout develop
virtualenv venv
. venv/bin/activate
pip install --editable '.[develop]'

Note for Windows users

If your installation fails based on a pyproj or pyrsistent error there is a small hack to get around it. Based on the python version you have installed you can download a wheel (binary of a python package) of the problem package/s. A good website to use is here. You then run:

pip install [name of wheel file]

You can then continue with:

pip install cjio

Supported CityJSON versions

Currently it supports CityJSON v2.0, but v1.1 and v1.0 files can be upgraded automatically with the operator upgrade`.

The operators (cjio --version) expect that your file is using the latest version CityJSON schema. If your file uses an earlier version, you can upgrade it with the upgrade operator: cjio old.json upgrade save newfile.city.json

Usage of the CLI

After installation, you have a small program called cjio, to see its possibilities:

cjio --help

Commands:
  attribute_remove  Remove an attribute.
  attribute_rename  Rename an attribute.
  crs_assign        Assign a (new) CRS (an EPSG).
  crs_reproject     Reproject to a new EPSG.
  crs_translate     Translate the coordinates.
  export            Export to another format.
  info              Output information about the dataset.
  lod_filter        Filter only one LoD for a dataset.
  materials_remove  Remove all materials.
  merge             Merge the current CityJSON with other ones.
  metadata_create   Add the +metadata-extended properties.
  metadata_get      Shows the metadata and +metadata-extended of this...
  metadata_remove   Remove the +metadata-extended properties.
  metadata_update   Update the +metadata-extended.
  print             print the (pretty formatted) JSON to the console.
  save              Save to a CityJSON file.
  subset            Create a subset, City Objects can be selected by: (1)...
  textures_locate   Output the location of the texture files.
  textures_remove   Remove all textures.
  textures_update   Update the location of the texture files.
  triangulate       Triangulate every surface.
  upgrade           Upgrade the CityJSON to the latest version.
  validate          Validate the CityJSON: (1) against its schemas (2)...
  vertices_clean    Remove duplicate vertices + orphan vertices

Or see the command-specific help by calling --help after a command:

Usage: cjio INPUT subset [OPTIONS]

Create a subset, City Objects can be selected by: (1) IDs of City Objects;
(2) bbox; (3) City Object type(s); (4) randomly.

These can be combined, except random which overwrites others.

Option '--exclude' excludes the selected objects, or "reverse" the
selection.

Usage examples:

  cjio myfile.city.json subset --bbox 104607 490148 104703 490257 save out.city.json
  cjio myfile.city.json subset --radius 500.0 610.0 50.0 --exclude save out.city.json
  cjio myfile.city.json subset --id house12 save out.city.json
  cjio myfile.city.json subset --random 5 save out.city.json
  cjio myfile.city.json subset --cotype LandUse --cotype Building save out.city.json

Options:
  --id TEXT          The ID of the City Objects; can be used multiple times.
  --bbox FLOAT...    2D bbox: minx miny maxx maxy.
  --radius FLOAT...  x y radius
  --random INTEGER   Number of random City Objects to select.
  --cotype TEXT      The City Object types; can be used multiple times.
  --exclude          Excludes the selection, thus delete the selected
                     object(s).
  --help             Show this message and exit.

Pipelines of operators

The input 3D city model opened is passed through all the operators, and it gets modified by some operators. Operators like info and validate output information in the console and just pass the 3D city model to the next operator.

cjio example.city.json subset --id house12 remove_materials save out.city.json
cjio example.city.json remove_textures info
cjio example.city.json upgrade validate save new.city.json
cjio myfile.city.json merge '/home/elvis/temp/*.city.json' save all_merged.city.json

stdin and stdout

Starting from v0.8, cjio allows to read/write from stdin/stdout (standard input/output streams).

For reading, it accepts at this moment only CityJSONL (text sequences with CityJSONFeatures). Instead of putting the file name, stdin must be used.

For writing, both CityJSON files and CityJSONL files can be piped to stdout. Instead of putting the file name, stdout must be used. Also, the different operators of cjio output messages/information, and those will get in the stdout stream, to avoid this add the flag --suppress_msg when reading the file, as shown below.

cat mystream.city.jsonl | cjio --suppress_msg stdin remove_materials save stdout
cjio --suppress_msg myfile.city.json remove_materials export jsonl stdout | less
cat myfile.city.json | cjio --suppress_msg stdin crs_reproject 7415 export jsonl mystream.txt

Generating Binary glTF

Convert the CityJSON example.city.json to a glb file /home/elvis/gltfs/example.glb

cjio example.json export glb /home/elvis/gltfs

Convert the CityJSON example.city.json to a glb file /home/elvis/test.glb

cjio example.city.json export glb /home/elvis/test.glb

Usage of the API

cjio.readthedocs.io/en/stable/tutorials.html

Docker

If docker is the tool of your choice, please read the following hints.

To run cjio via docker simply call:

docker run --rm  -v <local path where your files are>:/data tudelft3d/cjio:latest cjio --help

To give a simple example for the following lets assume you want to create a geojson which represents the bounding boxes of the files in your directory. Lets call this script gridder.py. It would look like this:

from cjio import cityjson
import glob
import ntpath
import json
import os
from shapely.geometry import box, mapping

def path_leaf(path):
    head, tail = ntpath.split(path)
    return tail or ntpath.basename(head)

files = glob.glob('./*.json')

geo_json_dict = {
    "type": "FeatureCollection",
    "features": []
}

for f in files:
    cj_file = open(f, 'r')
    cm = cityjson.reader(file=cj_file)
    theinfo = json.loads(cm.get_info())
    las_polygon = box(theinfo['bbox'][0], theinfo['bbox'][1], theinfo['bbox'][3], theinfo['bbox'][4])
    feature = {
        'properties': {
            'name': path_leaf(f)
        },
        'geometry': mapping(las_polygon)
    }
    geo_json_dict["features"].append(feature)
    geo_json_dict["crs"] = {
        "type": "name",
        "properties": {
            "name": "EPSG:{}".format(theinfo['epsg'])
        }
    }
geo_json_file = open(os.path.join('./', 'grid.json'), 'w+')
geo_json_file.write(json.dumps(geo_json_dict, indent=2))
geo_json_file.close()

This script will produce for all files with postfix ".json" in the directory a bbox polygon using cjio and save the complete geojson result in grid.json in place.

If you have a python script like this, simply put it inside your local data and call docker like this:

docker run --rm  -v <local path where your files are>:/data tudelft3d/cjio:latest python gridder.py

This will execute your script in the context of the python environment inside the docker image.

Example CityJSON datasets

There are a few example files on the CityJSON webpage.

Alternatively, any CityGML file can be automatically converted to CityJSON with the open-source project citygml-tools (based on citygml4j).

Acknowledgements

The glTF exporter is adapted from Kavisha's CityJSON2glTF.

cjio's People

Contributors

athelena avatar balazsdukai avatar dependabot[bot] avatar duswie avatar fiodccobw avatar fiodccobwang avatar gperonato avatar hugoledoux avatar ipadjen avatar jliempt avatar kenohori avatar kkimmy avatar liberostelios avatar lordi avatar mwussow avatar vvmruder avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cjio's Issues

validate fails on MultiSolid

Validating MultiSolids msolid_buildings.zip fails with the vague 'is not valid under any of the given schemas' error. val3dity does validate the same file (although the geometry is invalid).

The same file as CompositeSolid (replaced MultiSolid to CompositeSolid) validates with cjio.
csolid_buildings.zip

The MultiSolids were made by hand from MultiSurfaces of the attached file in #36

Getting semantic surface boundaries are confusing

Expected this to work:

ground = geom.get_surfaces('groundsurface')
ground
Out[20]: {1: {'surface_idx': [[0, 4]], 'type': 'GroundSurface'}}
geom.get_surface_boundaries(ground)
Traceback (most recent call last):
  File "/home/balazs/.local/share/virtualenvs/cjio-xu4bWm9P/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3296, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-21-c8e7cbab9dea>", line 1, in <module>
    geom.get_surface_boundaries(ground)
  File "/home/balazs/Development/cjio/cjio/models.py", line 351, in get_surface_boundaries
    raise TypeError("surface must be a single surface")
TypeError: surface must be a single surface

But instead this is the required workflow:

ground_boundaries = []
for g in ground.values():
    ground_boundaries.append(geom.get_surface_boundaries(g))

CJIO fails on bigger Dutch OpenData files

In The Netherlands PDOK makes cityjson files availabe of NL:

https://brt.kadaster.nl/basisvoorziening-3d/

For example the part with my home town haarlem is this one:

https://download.pdok.nl/kadaster/basisvoorziening-3d/v1_0/2018/volledig/25az1.volledig.zip

The ZIP is 553Mb, unzipped it is 2.7Gb(!)

Trying to cut out a small peace (to load in in QGIS with the cityjson plugin):

cjio 25az1.json subset --bbox 104607 490148 104703 490257 save myarea.json

My (rather beefy Linux) laptop (16Gb, 8 threads) kills the process after a lot of swapping...

So my question:

  • is CJIO able to handle such files
  • OR should pdok not create such giant files?
  • ...

Any hint on how to handle this opendata is appreciated :-)

AttributeError when exporting to .obj

Using cjio version 0.5.5.
When I try to export into .obj file:
cjio tudelft_campus.json export tudelft_campus.obj
I get this error message:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/bin/cjio", line 11, in
load_entry_point('cjio==0.5.5', 'console_scripts', 'cjio')()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/core.py", line 764, in call
return self.main(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/core.py", line 1164, in invoke
return _process_result(rv)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/core.py", line 1102, in _process_result
**ctx.params)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/cjio/cjio.py", line 92, in process_pipeline
cm = processor(cm)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/cjio/cjio.py", line 254, in processor
if format.lower() == '3dtiles' or not isinstance(cm, list):
AttributeError: 'NoneType' object has no attribute 'lower'

Am I doing something wrong?
tudelft_campus_subsetEWI_Building.json.zip

Thank you!

Python module 'mapbox_earcut' missing

I've installed via 'pip3 cjio'

cjio test.json export test.obj
Parsing test.json
OBJ export skipped: Python module 'mapbox_earcut' missing (to triangulate faces)
Install it: https://github.com/skogler/mapbox_earcut_python

Might be related to this error as well.

cjio test.json remove_textures export test.obj
Parsing test.json
Remove all textures
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/bin/cjio", line 8, in
sys.exit(cli())
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 764, in call
return self.main(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 1164, in invoke
return _process_result(rv)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 1102, in _process_result
**ctx.params)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cjio/cjio.py", line 96, in process_pipeline
cm = processor(cm)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cjio/cjio.py", line 406, in processor
cm.remove_textures()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cjio/cityjson.py", line 839, in remove_textures
if self.j["appearance"] is None or len(self.j["appearance"]) == 0:
KeyError: 'appearance'

Do not add empty semantics to the Geometry when there are no semantics at all

When creating a Geometry instance through the API with geom = Geometry(lod=1), and empty semantics is automatically added to the geometry.
Eg. "semantics": {"surfaces": [], "values": [[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]]}

This is unnecessary in this case and increases the size and complexity of the file. Do not do it.

convert to obj

Hello guys,

thanks very much for the cjio tool!

I have pip3-installed it and would like to convert some cityjson.
However the cjio cli tool seems to not be callable from shell. But I can import cjio as a module in my python script. Is there some docs how to use it for the conversion?

[I'm using ubuntu 18.04 ]

Thanks very much in advance!

BOM issues

Some software quietly adds a BOM to a file. While this is illegal in the JSON specs, the truth is it still happens a lot with JSON files. The consequence here is it causes cjio to crash. I got the error:

Error: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

Perhaps, it's better to flag it as an issue but not to crash? That's what the R library jsonlite does for example, it ignores the BOM and warns but continues.

CityJSON.get_info() only gets info from .j

When creating an empty CityJSON and adding one CityObject to it, print(CityJSON) still returns and empty CityJSON, because it doesn't read the contents of the .cityobjects.

{
  "cityjson_version": "1.0",
  "epsg": null,
  "bbox": [
    9000000000.0,
    9000000000.0,
    9000000000.0,
    -9000000000.0,
    -9000000000.0,
    -9000000000.0
  ],
  "transform/compressed": false,
  "cityobjects_total": 0,
  "cityobjects_present": [],
  "materials": false,
  "textures": false
}

How to install

Hi, I am a Windows 10 user but I don't know how to install cjio. Can someone help me with this?
When I open commandprompt and type: "pip install cjio" it says: pip' is not recognized as an internal or external command,
operable program or batch file.

export to obj compressed

I am trying to convert some CityJSON files to obj, but the output I get looks like a compressed obj file:

test.txt
111

Should the file be automatically compressed in the conversion? When I put decompress before the export function, the output looks good
222

Revert conditional Numpy import

In geom_helper numpy was set to be conditional, because of PyInstaller issues on Windows. Numpy is "only" needed for triangulation (eg. OBJ output), so I think not big deal for now, but it needs to be put back.

PyInstaller issues when running the compiled executable:

Traceback (most recent call last):

  File "site-packages\numpy\core\__init__.py", line 24, in <module>

  File "c:\python38\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 621, in exec_module

    exec(bytecode, module.__dict__)

  File "site-packages\numpy\core\multiarray.py", line 14, in <module>

  File "c:\python38\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 621, in exec_module

    exec(bytecode, module.__dict__)

  File "site-packages\numpy\core\overrides.py", line 7, in <module>

ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.

Output of merge

When running merge you always get the info of the first file. So in this case:

cjio random10_1.json merge random10_2.json info

the result is:

{
  "cityjson_version": "1.0",
  ...
  "cityobjects_total": 10,
  ...
}
{
  "cityjson_version": "1.0",
 ...
  "cityobjects_total": 20,
  ...
}

and in this case:

cjio random10_1.json merge random10_2.json save merged.json

the result is:

{
  "cityjson_version": "1.0",
  ...
  "cityobjects_total": 10,
  ...
}

So my question is whether there is any point in having the info of the first file in the output? In the first case it looks messy and redundant and in the second case it makes it look like the merge was unsuccessful, which was not the case.

export textures when subsetting a file

Also export the relevant texture files when subseting a file. The texture files are then saved into the <output file name>_textures folder in the same directory as the output file.

a more intelligent merge()

Right now it doesn't merge 2 CO having the same ID. But in practice say we have 2 datasets with the same objects, one is LoD1 and the other LoD2, and we would like to merge them into the same dataset.

So a smarter merge is necessary.

Analysing the geom for duplicates is probably out of scope, but just different LoD1 would help.

TypeError: unsupported operand type(s) for +=: 'dict' and 'list'

When running
cjio no_bld.json merge buildings.json save LOD2.json
I get the following error;

Parsing no_bld.json
v0.6 is not the latest version, and not everything will work.
Upgrade the file with 'upgrade_version' command: 'cjio input.json upgrade_version save out.json'
Merging files
{
"cityjson_version": "0.6",
"epsg": null,
"extensions": false,
"cityobjects_total": 214,
"cityobjects_present": [
"Building"
],
"vertices_total": 13766,
"transform/compressed": true,
"geom_primitives_present": [
"MultiSurface"
],
"materials": false,
"textures": true
}
Traceback (most recent call last):
File "C:\OSGeo4W64\apps\Python36\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "C:\OSGeo4W64\apps\Python36\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\OSGeo4W64\apps\Python36\Scripts\cjio.exe_main
.py", line 9, in
File "C:\OSGeo4W64\apps\Python36\lib\site-packages\click\core.py", line 764, in call
return self.main(*args, **kwargs)
File "C:\OSGeo4W64\apps\Python36\lib\site-packages\click\core.py", line 717, in main
rv = self.invoke(ctx)
File "C:\OSGeo4W64\apps\Python36\lib\site-packages\click\core.py", line 1164, in invoke
return _process_result(rv)
File "C:\OSGeo4W64\apps\Python36\lib\site-packages\click\core.py", line 1102, in _process_result
**ctx.params)
File "C:\OSGeo4W64\apps\Python36\lib\site-packages\click\core.py", line 555, in invoke
return callback(*args, **kwargs)
File "C:\OSGeo4W64\apps\Python36\lib\site-packages\cjio\cjio.py", line 96, in process_pipeline
cm = processor(cm)
File "C:\OSGeo4W64\apps\Python36\lib\site-packages\cjio\cjio.py", line 265, in processor
cm.merge(lsCMs)
File "C:\OSGeo4W64\apps\Python36\lib\site-packages\cjio\cityjson.py", line 997, in merge
self.j["appearance"]["vertices-texture"] += cm.j["appearance"]["vertices-texture"]
TypeError: unsupported operand type(s) for +=: 'dict' and 'list'

Level of Details as String

CJIO does not validate files in which LOD are not numbers but strings. Integers and floats are ok.

Accordingly to this issue on the CityJSON Github, it should validate it.

Add reason for invalidity to `validate`

Currently validate only prints that the file is invalid and prints the whole cityobject. It would be better to show why is the file invalid. Same is true for warnings.

Add CityObject class (and others?)

I've been looking into the implementation of #1 and I think it would make our life easier if we followed the CityJSON specification more closely with the class implementations. It would include for example creating a CityObject class with its methods for operating on it, eg update the path to the texture file. Once we have the classes properly set up, extending cjio would be a "breeze" as it would mainly require adding new methods to the classes. But all this would require refactoring the current codebase to accomodate the changes. What do you think @hugoledoux?

BBOX subset in a loop has unreliable behaviour

The following code finds a different nr of COs in grid_idx[15] at each run.

from os import path
import json

from cjio import cityjson
from cjio import tiling

p = './cjio/example_data/delft.json'
dout = './cjio/tmp/data'
with open(p, 'r') as f:
    cm = cityjson.CityJSON(file=f)

bbox = cm.update_bbox()
grid_idx = tiling.create_grid(bbox, 2)

for idx, bbox in grid_idx.items():
    print(idx, bbox)
    print("cm", len(cm.j['CityObjects']))
    s = cm.get_subset_bbox((bbox[0], bbox[1], bbox[3], bbox[4]), invert=False)
    print("s", len(s.j['CityObjects']))
    print("---------------")
    pout = path.join(dout,'{}.json'.format(idx))
    with open(pout, 'w') as fo:
        json_str = json.dumps(s.j, indent=2)
        fo.write(json_str)

Ẁhile this below, always returns 83.

bbox = cm.update_bbox()
grid_idx = tiling.create_grid(bbox, 2)
bbox = grid_idx[15]
s = cm.get_subset_bbox((bbox[0], bbox[1], bbox[3], bbox[4]), invert=False)

Create executables for the CLI with PyInstaller

cjio_dbexport has a Travis build+deploy setup, we can use the same. Having a single file executable could help a lot in adoption, since the user doesn't even need to have python installed. For linux, mac, windows

BBOX subset unhashable type dict

$ cjio /data/citymodels/Zurich/Building_LoD2_V10.json subset --bbox 2681133 1246440 2682862 1247462 save example_data/zurich_subset.json
Parsing /data/citymodels/Zurich/Building_LoD2_V10.json
Subset of CityJSON
Traceback (most recent call last):
  File "/home/balazs/.local/share/virtualenvs/cjio-xu4bWm9P/bin/cjio", line 11, in <module>
    load_entry_point('cjio', 'console_scripts', 'cjio')()
  File "/home/balazs/.local/share/virtualenvs/cjio-xu4bWm9P/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/balazs/.local/share/virtualenvs/cjio-xu4bWm9P/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/balazs/.local/share/virtualenvs/cjio-xu4bWm9P/lib/python3.6/site-packages/click/core.py", line 1164, in invoke
    return _process_result(rv)
  File "/home/balazs/.local/share/virtualenvs/cjio-xu4bWm9P/lib/python3.6/site-packages/click/core.py", line 1102, in _process_result
    **ctx.params)
  File "/home/balazs/.local/share/virtualenvs/cjio-xu4bWm9P/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/balazs/Development/cjio/cjio/cjio.py", line 92, in process_pipeline
    cm = processor(cm)
  File "/home/balazs/Development/cjio/cjio/cjio.py", line 468, in processor
    s = s.get_subset_bbox(bbox, exclude=exclude)
  File "/home/balazs/Development/cjio/cjio/cityjson.py", line 606, in get_subset_bbox
    re.add(self.j['CityObjects'][each])
TypeError: unhashable type: 'dict'

Split cjio into plugins

The idea is to keep the core minimal and ideally using only the standard library. This makes it much much more easy to integrate cjio with other libraries,apps. Extra/interface functions eg. format conversion, reprojection would go into plugins and the user would only install what they exactly need. What do you think @hugoledoux ?

Add materials to API

Works the same way as the semantics but at the moment the materials are ignored.

cjio functions to add

  • info
  • validate
  • save
  • subset 1) id; 2) bbox; 3) cotype
  • subset with polygon from .geojson, .stl, .obj
  • remove_textures
  • remove_materials
  • update_bbox
  • update_crs
  • decompress
  • merge [list-files]
  • remove_duplicate_vertices
  • remove_orphan_vertices
  • compress
  • convert2obj
  • reproject (with proj4)
  • upgrade_version
  • convert2gltf
  • convert2stl
  • convert2infragml
  • update_metadata (does everything!)
  • apply_material(rule, color=red)

CityJSON.validate() complains about vertices as tuples

The API casts the coordinates to tuples, not lists. Tuples should be allowed as they are bit more efficient on the memory.

(False,
 False,
 ["(0.0, 0.0, 0.0) is not of type 'array'",
  "(0.0, 0.0, 10.0) is not of type 'array'",
  "(0.0, 10.0, 0.0) is not of type 'array'",
  "(0.0, 10.0, 10.0) is not of type 'array'",
  "(10.0, 0.0, 0.0) is not of type 'array'",
  "(10.0, 0.0, 10.0) is not of type 'array'",
  "(10.0, 10.0, 0.0) is not of type 'array'",
  "(10.0, 10.0, 10.0) is not of type 'array'"],
 [])

KeyError: appearance when removing textures

From #23

@cfrank0214 could you please attach the test.json file to a reply so we can take a look? You'll probably need to zip it first.

cjio test.json remove_textures export test.obj
Parsing test.json
Remove all textures
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/bin/cjio", line 8, in
sys.exit(cli())
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 764, in call
return self.main(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 1164, in invoke
return _process_result(rv)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 1102, in _process_result
**ctx.params)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cjio/cjio.py", line 96, in process_pipeline
cm = processor(cm)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cjio/cjio.py", line 406, in processor
cm.remove_textures()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cjio/cityjson.py", line 839, in remove_textures
if self.j["appearance"] is None or len(self.j["appearance"]) == 0:
KeyError: 'appearance'

OBJ export fails due to missing module mapbox_earcut

I tried to convert a cityjson file to obj and got the error message:

OBJ export skipped: Python module 'mapbox_earcut' missing (to triangulate faces)

After investigating on mapbox_earcut, I got to this page:

#23 (comment)

and tried to install it, but the installation procedure is not well (if at all) described. Could you give me a hand? I got as far as getting cmake to configure the mapbox_earcut:

-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /usr/bin/python3.6 (found version "3.6.9")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.6m.so
-- Performing Test HAS_CPP14_FLAG
-- Performing Test HAS_CPP14_FLAG - Success
-- pybind11 v2.2.2
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- LTO enabled
-- Configuring done
-- Generating done

What is the last step to get cijo getting to recognise the mapbox_earcut? Any other way of getting the mapbox_earcut into cijo?
Thanks!

Unable to convert CityJSON to glb

I'm trying to convert CityJSON to glb with cjio version 0.6.
I've tried to convert the sampe dataset Neighbourhood 'Delfshaven'. Buildings in LoD2
The command:
cjio 3-20-DELFSHAVEN.json export --format glb 3-20-DELFSHAVEN.glb
produces

Parsing 3-20-DELFSHAVEN.json
Exporting CityJSON to glb /home/mattes/vmware/share/Daten/LOD2/3-20-DELFSHAVEN.glb
/home/mattes/.local/lib/python3.8/site-packages/cjio/geom_help.py:39: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  if((n==x3).all()):
Traceback (most recent call last):
  File "/home/mattes/.local/bin/cjio", line 11, in <module>
    load_entry_point('cjio==0.6', 'console_scripts', 'cjio')()
  File "/home/mattes/.local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/mattes/.local/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/mattes/.local/lib/python3.8/site-packages/click/core.py", line 1290, in invoke
    return _process_result(rv)
  File "/home/mattes/.local/lib/python3.8/site-packages/click/core.py", line 1224, in _process_result
    value = ctx.invoke(self.result_callback, value, **ctx.params)
  File "/home/mattes/.local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/mattes/.local/lib/python3.8/site-packages/cjio/cjio.py", line 103, in process_pipeline
    cm = processor(cm)
  File "/home/mattes/.local/lib/python3.8/site-packages/cjio/cjio.py", line 266, in processor
    exporter(cm)
  File "/home/mattes/.local/lib/python3.8/site-packages/cjio/cjio.py", line 164, in exporter
    glb = cm.export2gltf()
  File "/home/mattes/.local/lib/python3.8/site-packages/cjio/cityjson.py", line 1562, in export2gltf
    glb = convert.to_glb(self.j)
  File "/home/mattes/.local/lib/python3.8/site-packages/cjio/convert.py", line 188, in to_glb
    tri = geom_help.triangulate_face(face, vertexlist)
  File "/home/mattes/.local/lib/python3.8/site-packages/cjio/geom_help.py", line 88, in triangulate_face
    xy = to_2d(p, n)
  File "/home/mattes/.local/lib/python3.8/site-packages/cjio/geom_help.py", line 39, in to_2d
    if((n==x3).all()):
AttributeError: 'bool' object has no attribute 'all'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.