Comments (30)
bweeding
commented on June 24, 2024
1
Interestingly the update doesn't hang when I run it in QGIS on my desktop (as opposed to a linux server in the example above)!
from umep-processing.
ebocher
commented on June 24, 2024
1
A workaround to fix this issue is to create a multicolumn index
CREATE INDEX IF NOT EXISTS id_ID_POINT_TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447 ON TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447(ID_POINT, ID_Z);
CREATE INDEX IF NOT EXISTS id_ID_POINT_TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 ON
TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447(ID_POINT, ID_Z);
from umep-processing.
j3r3m1
commented on June 24, 2024
What do you mean by occasionally ? For every wind direction ? once every 10 wind direction ?
Did you get this problem with any kind of spatial data or a specific one ?
If you have a simple unit test case that I can debug I would opt for it.
from umep-processing.
bweeding
commented on June 24, 2024
Apologies for the lack of detail, here's an example:
urock_vectors_generalised_1.5m.zip
Wind speed: 8.04m/s at 10m
Wind direction: 296.37
Horizontal res: 1m
Vertical res: 2m
1st attempt fails reporting the error below, 2nd attempt works fine. Sometimes I can run 20+ URock models in a row without it happening, other times it will happen 3 times out of 10.
Connected!
Spatial functions added!
exited H2GIS
Load input data
Load table 'build_pre_srid_20240119130022'
Load table 'veg_pre_srid_20240119130022'
Creates blocks and stacked blocks
Rotates geometries from 296.37 degrees
Identify block base height and block cavity base
Calculates obstacle properties
Calculates zone properties
Initializes upwind facades
Update upwind facades base height
Initializes downwind facades
Calculates study area properties
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Creates displacement zones
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Creates cavity and wake zones
debug1
debug2
debug3
debug4
debug5
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Creates street canyon zones
Rotates geometries from -296.37 degrees
Creates rooftop zones (perpendicular and corner)
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Creates built-up and open vegetation zones
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Identify the buildings concerned by the impacted zone chosen by the user
Creates the grid of points
Affects each grid point to a building Rockle zone and calculates needed
variables for 3D wind speed
Affects each grid point to a vegetation Rockle zone and calculates
needed variables for 3D wind speed
Remove some of the Röckle zone points
Creates backward zones
Calculates the 3D wind speed factor value for each point of each BUILDING zone
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Rotates geometries from -296.37 degrees
Calculates the 3D wind speed factor value for each point of each VEGETATION zone
Rotates geometries from -296.37 degrees
Deals with superimposition (keeps only 1 value per 3D point)
Deals with superimposition (keeps only 1 value per 3D point)
Identify upstreamer points in TEMPO_WEIGHTING table
Identify upstreamer points in TEMPO_PRIORITIES table
Identify upstreamer points in TEMPO_PRIORITIES_WEIGHT table
Deals with superimposition (keeps only 1 value per 3D point)
Identify upstreamer points in TEMPO_WEIGHTING table
Identify upstreamer points in TEMPO_PRIORITIES table
Identify upstreamer points in TEMPO_PRIORITIES_WEIGHT table
Rotates geometries from -296.37 degrees
Identify grid points intersecting buildings
Set the initial 3D wind speed field
Time spent for wind speed initialization: 454.79771852493286 s
Shape: (558, 835, 40) - Nb cells: 18637200
Start to apply the wind solver
Time spent by the wind speed solver: 580.928831577301 s
Rotates geometries from -296.37 degrees
Logged critical: Traceback (most recent call last):
File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute
java.lang.java.lang.NumberFormatException: java.lang.NumberFormatException: For input string: ".061885983537887324"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "DbException.java", line 435, in org.h2.message.DbException.getJdbcSQLException
org.h2.jdbc.org.h2.jdbc.JdbcSQLDataException: org.h2.jdbc.JdbcSQLDataException: Data conversion error converting ".061885983537887324" [22018-200]
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute
org.h2.message.org.h2.message.DbException: org.h2.message.DbException: Data conversion error converting ".061885983537887324" [22018-200]
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute
Exception: Java Exception
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/init.py", line 534, in execute
is_rs = self._prep.execute()
org.h2.jdbc.org.h2.jdbc.JdbcSQLDataException: org.h2.jdbc.JdbcSQLDataException: Data conversion error converting "'.061885983537887324' (TEMPO_HORIZ: ""ID_POINT"" INTEGER)"; SQL statement:
CREATE TABLE TEMPO_HORIZ(ID_POINT INTEGER, HWS DOUBLE, HWD DOUBLE, VWS DOUBLE, WS DOUBLE)
AS SELECT ID_POINT, HWS, HWD, VWS, WS FROM CSVREAD('/tmp/tempo_horiz_wind.csv'); [22018-200]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/share/qgis/python/plugins/processing_umep/processor/urock_processing_algorithm.py", line 448, in processAlgorithm
MainCalculation.main(javaEnvironmentPath = javaEnvVar,
File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/MainCalculation.py", line 826, in main
saveData.saveBasicOutputs(cursor = cursor , z_out = z_out,
File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/saveData.py", line 105, in saveBasicOutputs
cursor.execute(
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/init.py", line 536, in execute
_handle_sql_exception()
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/init.py", line 165, in _handle_sql_exception_jpype
reraise(exc_type, exc_info[1], exc_info[2])
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/init.py", line 57, in reraise
raise value.with_traceback(tb)
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/init.py", line 534, in execute
is_rs = self._prep.execute()
jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLDataException: Data conversion error converting "'.061885983537887324' (TEMPO_HORIZ: ""ID_POINT"" INTEGER)"; SQL statement:
CREATE TABLE TEMPO_HORIZ(ID_POINT INTEGER, HWS DOUBLE, HWD DOUBLE, VWS DOUBLE, WS DOUBLE)
AS SELECT ID_POINT, HWS, HWD, VWS, WS FROM CSVREAD('/tmp/tempo_horiz_wind.csv'); [22018-200]
from umep-processing.
j3r3m1
commented on June 24, 2024
Here is a piece of code using your data but that does not reproduce the error you are logging. Can you test it to see if you get an error or change it in order to get an error ?
import os
from processing_umep.functions.URock.GlobalVariables import *
########################################
# Input definition #####################
########################################
plugin_directory_calc = "/home/decide/Code/Python/processing_umep/processor"
outputDirectory = "/tmp"
inputbuild = "/home/decide/Data/TestsUMEP/Ben/issue63/build_height_gen15.shp"
vegetationFilePath = "/home/decide/Data/TestsUMEP/Ben/issue63/veg_height_gen15.shp"
srid = 28355
z_ref = 10
v_ref = 2
windDirection = 200
prefix = ''
meshSize = 3
dz = 3
alongWindZoneExtend = ALONG_WIND_ZONE_EXTEND
crossWindZoneExtend = CROSS_WIND_ZONE_EXTEND
verticalExtend = VERTICAL_EXTEND
cadTriangles = ""
cadTreesIntersection = ""
tempoDirectory = TEMPO_DIRECTORY
onlyInitialization = ONLY_INITIALIZATION
maxIterations = MAX_ITERATIONS
thresholdIterations = THRESHOLD_ITERATIONS
idFieldBuild = None,
buildingHeightField = 'ROOF_HEIGH'
vegetationBaseHeight = ''
vegetationTopHeight = "VEG_HEIGHT"
idVegetation = None,
vegetationAttenuationFactor = ""
saveRockleZones = SAVE_ROCKLE_ZONES
outputRaster = None
feedback = None
saveRaster = True
saveVector = True
saveNetcdf = True
z_out = "1"
profileType = "power"
verticalProfileFile = None
########################################
# Call QGIS in Python ##################
########################################
from qgis.core import QgsApplication
# # Starts the qgis application without the graphical user interface
gui_flag = False
app = QgsApplication([], gui_flag)
app.initQgis()
import processing
from processing.core.Processing import Processing
Processing.initialize()
from processing_umep.processing_umep_provider import ProcessingUMEPProvider
umep_provider = ProcessingUMEPProvider()
QgsApplication.processingRegistry().addProvider(umep_provider)
########################################
# Start calculations ####################
########################################
for wd in [296.37, 256.32, 12.32,56.32, 12.68, 13.65, 245.36, 353.12, 8.05, 78.65, 77.32, 12.63]:
processing.run("umep:Urban Wind Field: URock",
{'BUILDINGS':inputbuild,
'HEIGHT_FIELD_BUILD':buildingHeightField,
'VEGETATION':vegetationFilePath,
'VEGETATION_CROWN_TOP_HEIGHT':vegetationTopHeight,
'VEGETATION_CROWN_BASE_HEIGHT':vegetationBaseHeight,
'ATTENUATION_FIELD':vegetationAttenuationFactor,
'INPUT_PROFILE_FILE':verticalProfileFile,
'INPUT_PROFILE_TYPE':prof_type,
'INPUT_WIND_HEIGHT':z_ref,
'INPUT_WIND_SPEED':v_ref,
'INPUT_WIND_DIRECTION':wd,
'RASTER_OUTPUT':outputRaster,
'HORIZONTAL_RESOLUTION':meshSize,
'VERTICAL_RESOLUTION':dz,
'WIND_HEIGHT':z_out,
'UROCK_OUTPUT':outputDirectory,
'OUTPUT_FILENAME':'urock_output',
'SAVE_RASTER':saveRaster,
'SAVE_VECTOR':saveVector,
'SAVE_NETCDF':saveNetcdf,
'LOAD_OUTPUT':False})from umep-processing.
j3r3m1
commented on June 24, 2024
@bweeding any outputs from what I have proposed ?
from umep-processing.
bweeding
commented on June 24, 2024
Apologies Jeremy, I'll try running it tomorrow!
…On Thu, 22 Feb 2024, 8:25 pm jeremy-b, ***@***.***> wrote:
@bweeding <https://github.com/bweeding> any outputs from what I have
proposed ?
—
Reply to this email directly, view it on GitHub
<#63 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANYGKRHWXOEEON2FACSVH73YU4FJ3AVCNFSM6AAAAABBXXK65SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJZGAZTAOJZGA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
from umep-processing.
bweeding
commented on June 24, 2024
This time I received a different intermittent error. The first time I ran the code I received the error below for a wind direction of 78.65, but when I restarted my kernel and ran it again, there was no problem...
I also had to make a couple of modifications to your code to make it run on my system:
########################################
# Input definition #####################
########################################
# plugin_directory_calc = "/home/decide/Code/Python/processing_umep/processor"
# outputDirectory = "/tmp"
current_folder = '/mnt/bweeding_workspace/phd_jupyter/paper3_workflows/p3_digital_models/'
inputbuild = current_folder+'build_height_gen15.shp'
vegetationFilePath =current_folder+'veg_height_gen15.shp'
srid = 28355
z_ref = 10
v_ref = 2
windDirection = 200
prefix = ''
meshSize = 3
dz = 3
# alongWindZoneExtend = ALONG_WIND_ZONE_EXTEND
# crossWindZoneExtend = CROSS_WIND_ZONE_EXTEND
# verticalExtend = VERTICAL_EXTEND
cadTriangles = ""
cadTreesIntersection = ""
# tempoDirectory = TEMPO_DIRECTORY
# onlyInitialization = ONLY_INITIALIZATION
# maxIterations = MAX_ITERATIONS
# thresholdIterations = THRESHOLD_ITERATIONS
idFieldBuild = None,
buildingHeightField = 'ROOF_HEIGH'
vegetationBaseHeight = ''
vegetationTopHeight = "VEG_HEIGHT"
idVegetation = None,
vegetationAttenuationFactor = ""
# saveRockleZones = SAVE_ROCKLE_ZONES
outputRaster = None
feedback = None
saveRaster = True
saveVector = True
saveNetcdf = True
z_out = "1"
prof_type = "power"
verticalProfileFile = None for wd in [296.37, 256.32, 12.32,56.32, 12.68, 13.65, 245.36, 353.12, 8.05, 78.65, 77.32, 12.63]:
processing.run("umep:Urban Wind Field: URock",
{'BUILDINGS':inputbuild,
'HEIGHT_FIELD_BUILD':buildingHeightField,
'VEGETATION':vegetationFilePath,
'VEGETATION_CROWN_TOP_HEIGHT':vegetationTopHeight,
'VEGETATION_CROWN_BASE_HEIGHT':vegetationBaseHeight,
'ATTENUATION_FIELD':vegetationAttenuationFactor,
'INPUT_PROFILE_FILE':verticalProfileFile,
'INPUT_PROFILE_TYPE':0,
'INPUT_WIND_HEIGHT':z_ref,
'INPUT_WIND_SPEED':v_ref,
'INPUT_WIND_DIRECTION':wd,
'RASTER_OUTPUT':outputRaster,
'HORIZONTAL_RESOLUTION':meshSize,
'VERTICAL_RESOLUTION':dz,
'WIND_HEIGHT':z_out,
'UROCK_OUTPUT':'/scratch/bweeding/urock_error_tests/',
'OUTPUT_FILENAME':'urock_output',
'SAVE_RASTER':saveRaster,
'SAVE_VECTOR':saveVector,
'SAVE_NETCDF':saveNetcdf,
'LOAD_OUTPUT':False})Error:
Logged critical: Traceback (most recent call last):
File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute
java.lang.java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Operation does not support GeometryCollection arguments
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute
Exception: Java Exception
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 534, in execute
is_rs = self._prep.execute()
org.h2.jdbc.org.h2.jdbc.JdbcSQLNonTransientException: org.h2.jdbc.JdbcSQLNonTransientException: Exception calling user-defined function: "validGeom(POLYGON ((527331 5252398, 527332 5252398, 527333 5252398, 527333 5252399, 527334 5252399, 527335.0000000009 5252398.999999999, 527335 5252400, 527333.9999999991 5252400.000000001, 527334 5252401, 527334 5252402.000000001, 527335.0000000009 5252402, 527335 5252403, 527336 5252403, 527336 5252403.999999999, 527336 5252405, 527336 5252406, 527334.9999999991 5252405.999999999, 527336 5252408, 527337.0000000009 5252408, 527338 5252408, 527338 5252407.000000001, 527338.9999999991 5252407.000000001, 527339 5252408, 527338.9999999991 5252409, 527339.9999999991 5252409, 527342 5252403, 527340.0000000009 5252402, 527340 5252403, 527339 5252403.000000001, 527339 5252402.000000001, 527339 5252400.999999999, 527337.9999999991 5252400.000000001, 527337 5252400, 527337.0000000009 5252399, 527336 5252396, 527336.1632530484 5252396.489759147, 527331.3598316666 5252394.999999999, 527333 5252395, 527331 5252394.999999999, 527331 5252398), (527334 5252397, 527333 5252397, 527333 5252396, 527334.0000000009 5252396.000000001, 527334 5252397), (527334 5252397, 527334.9999999991 5252396.999999999, 527334.9999999991 5252398, 527334 5252398.000000001, 527334 5252397))): Operation does not support GeometryCollection arguments"; SQL statement:
DROP TABLE IF EXISTS OPEN_VEGETATION_ZONES_ROTATED; [90105-200]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/share/qgis/python/plugins/processing_umep/processor/urock_processing_algorithm.py", line 448, in processAlgorithm
MainCalculation.main(javaEnvironmentPath = javaEnvVar,
File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/MainCalculation.py", line 378, in main
saveData.saveTable(cursor = cursor , tableName = vegetationOpenZoneTable,
File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/saveData.py", line 332, in saveTable
tableName = windRotation(cursor = cursor,
File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/Obstacles.py", line 81, in windRotation
cursor.execute(";".join(sqlRotateQueries))
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 536, in execute
_handle_sql_exception()
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 165, in _handle_sql_exception_jpype
reraise(exc_type, exc_info[1], exc_info[2])
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 57, in reraise
raise value.with_traceback(tb)
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 534, in execute
is_rs = self._prep.execute()
jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLNonTransientException: Exception calling user-defined function: "validGeom(POLYGON ((527331 5252398, 527332 5252398, 527333 5252398, 527333 5252399, 527334 5252399, 527335.0000000009 5252398.999999999, 527335 5252400, 527333.9999999991 5252400.000000001, 527334 5252401, 527334 5252402.000000001, 527335.0000000009 5252402, 527335 5252403, 527336 5252403, 527336 5252403.999999999, 527336 5252405, 527336 5252406, 527334.9999999991 5252405.999999999, 527336 5252408, 527337.0000000009 5252408, 527338 5252408, 527338 5252407.000000001, 527338.9999999991 5252407.000000001, 527339 5252408, 527338.9999999991 5252409, 527339.9999999991 5252409, 527342 5252403, 527340.0000000009 5252402, 527340 5252403, 527339 5252403.000000001, 527339 5252402.000000001, 527339 5252400.999999999, 527337.9999999991 5252400.000000001, 527337 5252400, 527337.0000000009 5252399, 527336 5252396, 527336.1632530484 5252396.489759147, 527331.3598316666 5252394.999999999, 527333 5252395, 527331 5252394.999999999, 527331 5252398), (527334 5252397, 527333 5252397, 527333 5252396, 527334.0000000009 5252396.000000001, 527334 5252397), (527334 5252397, 527334.9999999991 5252396.999999999, 527334.9999999991 5252398, 527334 5252398.000000001, 527334 5252397))): Operation does not support GeometryCollection arguments"; SQL statement:
DROP TABLE IF EXISTS OPEN_VEGETATION_ZONES_ROTATED; [90105-200]
---------------------------------------------------------------------------
QgsProcessingException Traceback (most recent call last)
Cell In[10], line 2
1 for wd in [296.37, 256.32, 12.32,56.32, 12.68, 13.65, 245.36, 353.12, 8.05, 78.65, 77.32, 12.63]:
----> 2 processing.run("umep:Urban Wind Field: URock",
3 {'BUILDINGS':inputbuild,
4 'HEIGHT_FIELD_BUILD':buildingHeightField,
5 'VEGETATION':vegetationFilePath,
6 'VEGETATION_CROWN_TOP_HEIGHT':vegetationTopHeight,
7 'VEGETATION_CROWN_BASE_HEIGHT':vegetationBaseHeight,
8 'ATTENUATION_FIELD':vegetationAttenuationFactor,
9 'INPUT_PROFILE_FILE':verticalProfileFile,
10 'INPUT_PROFILE_TYPE':0,
11 'INPUT_WIND_HEIGHT':z_ref,
12 'INPUT_WIND_SPEED':v_ref,
13 'INPUT_WIND_DIRECTION':wd,
14 'RASTER_OUTPUT':outputRaster,
15 'HORIZONTAL_RESOLUTION':meshSize,
16 'VERTICAL_RESOLUTION':dz,
17 'WIND_HEIGHT':z_out,
18 'UROCK_OUTPUT':'/scratch/bweeding/urock_error_tests/',
19 'OUTPUT_FILENAME':'urock_output',
20 'SAVE_RASTER':saveRaster,
21 'SAVE_VECTOR':saveVector,
22 'SAVE_NETCDF':saveNetcdf,
23 'LOAD_OUTPUT':False})
File /opt/anaconda3/envs/qgis_bw/share/qgis/python/plugins/processing/tools/general.py:106, in run(algOrName, parameters, onFinish, feedback, context, is_child_algorithm)
91 """
92 Executes given algorithm and returns its outputs as dictionary object.
93
(...)
103 :rtype: Union[dict, None]
104 """
105 if onFinish or not is_child_algorithm:
--> 106 return Processing.runAlgorithm(algOrName, parameters, onFinish, feedback, context)
107 else:
108 # for child algorithms, we disable to default post-processing step where layer ownership
109 # is transferred from the context to the caller. In this case, we NEED the ownership to remain
110 # with the context, so that further steps in the algorithm have guaranteed access to the layer.
111 def post_process(_alg, _context, _feedback):
File /opt/anaconda3/envs/qgis_bw/share/qgis/python/plugins/processing/core/Processing.py:176, in Processing.runAlgorithm(algOrName, parameters, onFinish, feedback, context)
172 if not alg.validateInputCrs(parameters, context):
173 feedback.pushInfo(
174 Processing.tr('Warning: Not all input layers use the same CRS.\nThis can cause unexpected results.'))
--> 176 ret, results = execute(alg, parameters, context, feedback, catch_exceptions=False)
177 if ret:
178 feedback.pushInfo(
179 Processing.tr('Results: {}').format(results))
File /opt/anaconda3/envs/qgis_bw/share/qgis/python/plugins/processing/gui/AlgorithmExecutor.py:70, in execute(alg, parameters, context, feedback, catch_exceptions)
68 return False, {}
69 else:
---> 70 results, ok = alg.run(parameters, context, feedback, {}, False)
71 return ok, results
QgsProcessingException: Traceback (most recent call last):
File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute
java.lang.java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Operation does not support GeometryCollection arguments
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "JdbcPreparedStatement.java", line 237, in org.h2.jdbc.JdbcPreparedStatement.execute
Exception: Java Exception
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 534, in execute
is_rs = self._prep.execute()
org.h2.jdbc.org.h2.jdbc.JdbcSQLNonTransientException: org.h2.jdbc.JdbcSQLNonTransientException: Exception calling user-defined function: "validGeom(POLYGON ((527331 5252398, 527332 5252398, 527333 5252398, 527333 5252399, 527334 5252399, 527335.0000000009 5252398.999999999, 527335 5252400, 527333.9999999991 5252400.000000001, 527334 5252401, 527334 5252402.000000001, 527335.0000000009 5252402, 527335 5252403, 527336 5252403, 527336 5252403.999999999, 527336 5252405, 527336 5252406, 527334.9999999991 5252405.999999999, 527336 5252408, 527337.0000000009 5252408, 527338 5252408, 527338 5252407.000000001, 527338.9999999991 5252407.000000001, 527339 5252408, 527338.9999999991 5252409, 527339.9999999991 5252409, 527342 5252403, 527340.0000000009 5252402, 527340 5252403, 527339 5252403.000000001, 527339 5252402.000000001, 527339 5252400.999999999, 527337.9999999991 5252400.000000001, 527337 5252400, 527337.0000000009 5252399, 527336 5252396, 527336.1632530484 5252396.489759147, 527331.3598316666 5252394.999999999, 527333 5252395, 527331 5252394.999999999, 527331 5252398), (527334 5252397, 527333 5252397, 527333 5252396, 527334.0000000009 5252396.000000001, 527334 5252397), (527334 5252397, 527334.9999999991 5252396.999999999, 527334.9999999991 5252398, 527334 5252398.000000001, 527334 5252397))): Operation does not support GeometryCollection arguments"; SQL statement:
DROP TABLE IF EXISTS OPEN_VEGETATION_ZONES_ROTATED; [90105-200]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/share/qgis/python/plugins/processing_umep/processor/urock_processing_algorithm.py", line 448, in processAlgorithm
MainCalculation.main(javaEnvironmentPath = javaEnvVar,
File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/MainCalculation.py", line 378, in main
saveData.saveTable(cursor = cursor , tableName = vegetationOpenZoneTable,
File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/saveData.py", line 332, in saveTable
tableName = windRotation(cursor = cursor,
File "/usr/share/qgis/python/plugins/processing_umep/functions/URock/Obstacles.py", line 81, in windRotation
cursor.execute(";".join(sqlRotateQueries))
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 536, in execute
_handle_sql_exception()
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 165, in _handle_sql_exception_jpype
reraise(exc_type, exc_info[1], exc_info[2])
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 57, in reraise
raise value.with_traceback(tb)
File "/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py", line 534, in execute
is_rs = self._prep.execute()
jaydebeapi.DatabaseError: org.h2.jdbc.JdbcSQLNonTransientException: Exception calling user-defined function: "validGeom(POLYGON ((527331 5252398, 527332 5252398, 527333 5252398, 527333 5252399, 527334 5252399, 527335.0000000009 5252398.999999999, 527335 5252400, 527333.9999999991 5252400.000000001, 527334 5252401, 527334 5252402.000000001, 527335.0000000009 5252402, 527335 5252403, 527336 5252403, 527336 5252403.999999999, 527336 5252405, 527336 5252406, 527334.9999999991 5252405.999999999, 527336 5252408, 527337.0000000009 5252408, 527338 5252408, 527338 5252407.000000001, 527338.9999999991 5252407.000000001, 527339 5252408, 527338.9999999991 5252409, 527339.9999999991 5252409, 527342 5252403, 527340.0000000009 5252402, 527340 5252403, 527339 5252403.000000001, 527339 5252402.000000001, 527339 5252400.999999999, 527337.9999999991 5252400.000000001, 527337 5252400, 527337.0000000009 5252399, 527336 5252396, 527336.1632530484 5252396.489759147, 527331.3598316666 5252394.999999999, 527333 5252395, 527331 5252394.999999999, 527331 5252398), (527334 5252397, 527333 5252397, 527333 5252396, 527334.0000000009 5252396.000000001, 527334 5252397), (527334 5252397, 527334.9999999991 5252396.999999999, 527334.9999999991 5252398, 527334 5252398.000000001, 527334 5252397))): Operation does not support GeometryCollection arguments"; SQL statement:
DROP TABLE IF EXISTS OPEN_VEGETATION_ZONES_ROTATED; [90105-200]
from umep-processing.
j3r3m1
commented on June 24, 2024
Thanks for reporting. It should now be OK using the last snapshot version (2.0.19) which has been pushed right now. Please let me know
from umep-processing.
bweeding
commented on June 24, 2024
Ok great! I'll update and do some testing.
from umep-processing.
bweeding
commented on June 24, 2024
Have updated and run some trials. The update seems to have removed the errors, but when I try and run it at 1m horizontal resolution, it hangs at "Identify upstreamer points in TEMPO_PRIORITIES_WEIGHT table". I've included the code below:
for wd in np.arange(270,360,0.5):
processing.run("umep:Urban Wind Field: URock",
{'BUILDINGS':inputbuild,
'HEIGHT_FIELD_BUILD':buildingHeightField,
'VEGETATION':vegetationFilePath,
'VEGETATION_CROWN_TOP_HEIGHT':vegetationTopHeight,
'VEGETATION_CROWN_BASE_HEIGHT':vegetationBaseHeight,
'ATTENUATION_FIELD':vegetationAttenuationFactor,
'INPUT_PROFILE_FILE':verticalProfileFile,
'INPUT_PROFILE_TYPE':0,
'INPUT_WIND_HEIGHT':10,
'INPUT_WIND_SPEED':v_ref,
'INPUT_WIND_DIRECTION':str(wd),
'RASTER_OUTPUT':outputRaster,
'HORIZONTAL_RESOLUTION':1,
'VERTICAL_RESOLUTION':2,
'WIND_HEIGHT':2,
'UROCK_OUTPUT':'/scratch/bweeding/urock_error_tests/',
'OUTPUT_FILENAME':'urock_output',
'SAVE_RASTER':saveRaster,
'SAVE_VECTOR':saveVector,
'SAVE_NETCDF':saveNetcdf,
'LOAD_OUTPUT':False})
cur_rx = rx.open_rasterio('/scratch/bweeding/urock_error_tests/z2_0/urock_outputWS.Gtiff')
cur_rx.plot()
plt.show()
cur_rx.close()
Output:
Connecting to database
->/tmp/myDbH2
/usr/share/qgis/python/plugins/processing_umep/functions/URock/h2gis-standalone/h2gis-dist-2.2.1.jar
/opt/anaconda3/envs/qgis_bw/lib/python3.9/site-packages/jaydebeapi/__init__.py:200: DeprecationWarning: jpype._core.isThreadAttachedToJVM is deprecated, use java.lang.Thread.isAttached instead
if not jpype.isThreadAttachedToJVM():
Connected!
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
Spatial functions added!
Load input data
Load table 'build_pre_srid_20240227163818'
Load table 'veg_pre_srid_20240227163818'
Creates blocks and stacked blocks
Rotates geometries from 270.0 degrees
Identify block base height and block cavity base
Calculates obstacle properties
Calculates zone properties
Initializes upwind facades
Update upwind facades base height
Initializes downwind facades
Calculates study area properties
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Creates displacement zones
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Creates cavity and wake zones
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Creates street canyon zones
Rotates geometries from -270.0 degrees
Creates rooftop zones (perpendicular and corner)
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Creates built-up and open vegetation zones
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Creates the grid of points
Affects each grid point to a building Rockle zone and calculates needed
variables for 3D wind speed
Affects each grid point to a vegetation Rockle zone and calculates
needed variables for 3D wind speed
Remove some of the Röckle zone points
Creates backward zones
Calculates the 3D wind speed factor value for each point of each BUILDING zone
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Rotates geometries from -270.0 degrees
Calculates the 3D wind speed factor value for each point of each VEGETATION zone
Rotates geometries from -270.0 degrees
Deals with superimposition (keeps only 1 value per 3D point)
Deals with superimposition (keeps only 1 value per 3D point)
Identify upstreamer points in TEMPO_WEIGHTING table
Identify upstreamer points in TEMPO_PRIORITIES table
Identify upstreamer points in TEMPO_PRIORITIES_WEIGHT table
Deals with superimposition (keeps only 1 value per 3D point)
Identify upstreamer points in TEMPO_WEIGHTING table
Identify upstreamer points in TEMPO_PRIORITIES table
Identify upstreamer points in TEMPO_PRIORITIES_WEIGHT table
from umep-processing.
bweeding
commented on June 24, 2024
I've found the code that causes my linux machine to hang, in InitWindField.py. The calls to DataUtil.createIndex all go through successfully, but the code then appears to get stuck.
# Join the downstream weigthted points to the non downstream weighted ones
cursor.execute("""
{12};
{13};
{10};
{11};
DROP TABLE IF EXISTS {9};
CREATE TABLE {9}
AS SELECT a.{2}, a.{3}, a.{4}, a.{5}, a.{6}, a.{7}, a.{8}
FROM {0} AS a LEFT JOIN {1} AS b
ON a.{2} = b.{2} AND a.{3} = b.{3}
WHERE b.{2} IS NULL
UNION ALL
SELECT c.{2}, c.{3}, c.{4}, c.{5}, c.{6}, c.{7}, c.{8}
FROM {1} AS c
""".format( tempoPrioritiesWeightedAllPlusBack , tempoUpstreamAndDownstream,
ID_POINT , ID_POINT_Z,
HEIGHT_FIELD , U,
V , W,
REF_HEIGHT_FIELD , initializedWindFactorTable,
DataUtil.createIndex(tableName=tempoUpstreamAndDownstream,
fieldName=ID_POINT,
isSpatial=False),
DataUtil.createIndex(tableName=tempoUpstreamAndDownstream,
fieldName=ID_POINT_Z,
isSpatial=False),
DataUtil.createIndex(tableName=tempoPrioritiesWeightedAllPlusBack,
fieldName=ID_POINT,
isSpatial=False),
DataUtil.createIndex(tableName=tempoPrioritiesWeightedAllPlusBack,
fieldName=ID_POINT_Z,
isSpatial=False)))
from umep-processing.
j3r3m1
commented on June 24, 2024
OK, quite difficult to debug when it becomes machine related. I can have a try to split this SQL query into 2 in a specific branch on my repo and then you can have a try if it solves the problem on your server. If yes we will merge to the current master.
from umep-processing.
bweeding
commented on June 24, 2024
Ah that sounds like a good idea!
I'm very much out of my depth when it comes to the SQL/Java stuff. I the machine issue might be to do with where the temporary files were being stored on the linux machine and the transfer speeds to that drive, but that isn't the case. Have you tried running the call at 1m resolution on your machine/s?
from umep-processing.
j3r3m1
commented on June 24, 2024
I have not try running the code at 1 m resolution, I will try now.
Here you can also try the new version of the code on your server to see if it solves the issue: https://github.com/j3r3m1/UMEP-processing/tree/modifBenServer
from umep-processing.
bweeding
commented on June 24, 2024
Ok great, I'll try running it!
I also noticed today when running at 2m resolution, direction 269.44 and a wind speed of 9.4m/s at 10m that it took a very long time, due to modelling going almost 7km up!
Time spent for wind speed initialization: 1873.6515927314758 s
Shape: (265, 397, 3376) - Nb cells: 355172080
10
Start to apply the wind solver
Time spent by the wind speed solver: 10153.252950429916 s
from umep-processing.
bweeding
commented on June 24, 2024
Hi Jeremy, I've tried the new code and it's still getting stuck in the same spot, at
# Join the downstream weigthted points to the non downstream weighted ones
I set the createIndex function in dataUtil.py to print the table names, and it gets to:
TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305201659
TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305201659
TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305201659
TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305201659
from umep-processing.
j3r3m1
commented on June 24, 2024
I also noticed today when running at 2m resolution, direction 269.44 and a wind speed of 9.4m/s at 10m that it took a very long time, due to modelling going almost 7km up!
Good catch. It might be related to issue (there is trouble with the rooftop corner scheme) #41
Is it still for the same area ? If so I should investigate this case and solve that.
from umep-processing.
bweeding
commented on June 24, 2024
Yep I'm just using the same area.
from umep-processing.
j3r3m1
commented on June 24, 2024
I have the same problem than you for the endless SQL query. How long did it take on your regular labtop and what are the characteristics of your regular labtop ?
It seems that the number of cells for this area and this resolution is quite high (> 20 M grid cells) which might also explain the time spent by the SQL query (but not an endless time as it seems to be however...).
@ebocher any idea how to investigate what's wrong with the query (except that the two tables are huge) ?
Here is the SQL query:
CREATE INDEX IF NOT EXISTS id_ID_POINT_TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447 ON TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447(ID_POINT);;
CREATE INDEX IF NOT EXISTS id_ID_Z_TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447 ON TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447(ID_Z);;
CREATE INDEX IF NOT EXISTS id_ID_POINT_TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 ON
TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447(ID_POINT);;
CREATE INDEX IF NOT EXISTS id_ID_Z_TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 ON TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447(ID_Z);;
DROP TABLE IF EXISTS INITIALIZED_WIND_FACTOR_FIELD;
CREATE TABLE INITIALIZED_WIND_FACTOR_FIELD
AS SELECT a.ID_POINT, a.ID_Z, a.HEIGHT_ROO, a.U, a.V, a.W, a.REF_HEIGHT
FROM TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447 AS a LEFT JOIN TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 AS b
ON a.ID_POINT = b.ID_POINT AND a.ID_Z = b.ID_Z
WHERE b.ID_POINT IS NULL
UNION ALL
SELECT c.ID_POINT, c.ID_Z, c.HEIGHT_ROO, c.U, c.V, c.W, c.REF_HEIGHT
FROM TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 AS cHere are the corresponding tables, saved as CSV (do not contain a geometry field):
from umep-processing.
ebocher
commented on June 24, 2024
Please give me the H2 database version you are using .
from umep-processing.
ebocher
commented on June 24, 2024
and give the result of
EXPLAIN SELECT a.ID_POINT, a.ID_Z, a.HEIGHT_ROO, a.U, a.V, a.W, a.REF_HEIGHT
FROM TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447 AS a LEFT JOIN TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 AS b
ON a.ID_POINT = b.ID_POINT AND a.ID_Z = b.ID_Z
WHERE b.ID_POINT IS NULL
UNION ALL
SELECT c.ID_POINT, c.ID_Z, c.HEIGHT_ROO, c.U, c.V, c.W, c.REF_HEIGHT
FROM TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447 AS c
from umep-processing.
j3r3m1
commented on June 24, 2024
The DB is definitely too big (1.9 Go) to be shared simply.
The result of the query is:
(SELECT
"A"."ID_POINT",
"A"."ID_Z",
"A"."HEIGHT_ROO",
"A"."U",
"A"."V",
"A"."W",
"A"."REF_HEIGHT"
FROM "PUBLIC"."TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447" "A"
/* PUBLIC.TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447.tableScan */
LEFT OUTER JOIN "PUBLIC"."TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447" "B"
/* PUBLIC.ID_ID_Z_TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447: ID_Z = A.ID_Z */
ON ("A"."ID_POINT" = "B"."ID_POINT")
AND ("A"."ID_Z" = "B"."ID_Z")
WHERE "B"."ID_POINT" IS NULL)
UNION ALL
(SELECT
"C"."ID_POINT",
"C"."ID_Z",
"C"."HEIGHT_ROO",
"C"."U",
"C"."V",
"C"."W",
"C"."REF_HEIGHT"
FROM "PUBLIC"."TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447" "C"
/* PUBLIC.TEMPO_UPSTREAM_AND_DOWNSTREAM_20240305165447.tableScan */)from umep-processing.
ebocher
commented on June 24, 2024
It seems that the index of the table TEMPO_PRIORITY_WEIGHTED_ALL_PLUS_BACK_20240305165447 is not used. Because H2 runs a tableScan.
This bug is present in the latest release of H2.
We must investigate the current snapshot.
from umep-processing.
j3r3m1
commented on June 24, 2024
OK I did not know about this. Good that it is a known bug.
from umep-processing.
j3r3m1
commented on June 24, 2024
Yep I'm just using the same area.
OK I will try to solve the issue when I will have more time.
from umep-processing.
j3r3m1
commented on June 24, 2024
Should now be solved thanks to the last commit (fa8a4ec). Have a try to the last version and report (close if solved)
Concerning the very high domain it is definitely related to the rooftop zone so it will be fixed in the other issue.
from umep-processing.
bweeding
commented on June 24, 2024
Seems to be working now. However, things are running much slower than previously and generating very large databases. Partially to do with the high domains I think.
from umep-processing.
j3r3m1
commented on June 24, 2024
Seems to be working now. However, things are running much slower than previously and generating very large databases. Partially to do with the high domains I think.
Do you observe this phenomenon on cases you had previously run before ? Because it is supposed to be faster...
from umep-processing.
bweeding
commented on June 24, 2024
I'll do some testing, it seems to vary a lot.
…On Thu, 7 Mar 2024, 11:44 pm jeremy-b, ***@***.***> wrote:
Seems to be working now. However, things are running much slower than
previously and generating very large databases. Partially to do with the
high domains I think.
Do you observe this phenomenon on cases you had previously run before ?
Because it is supposed to be faster...
—
Reply to this email directly, view it on GitHub
<#63 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANYGKRG3QTY3HRIKHROKNRTYXBOLPAVCNFSM6AAAAABBXXK65SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBTGQZTEOBTGM>
.
You are receiving this because you modified the open/close state.Message
ID: ***@***.***>
from umep-processing.
Related Issues (20)
- Urock prepare: fails when including DEM HOT 11
- Error when URock attempts to rotate geometries HOT 2
- Different limits for clothing parameter in PET and SOLWEIG / SpatialTC functions HOT 2
- URock: Use bilinear interpolation instead of averaging when outputRaster is used
- Urock issue: simplifed vs generalised vectors HOT 7
- Urock: Processing issue HOT 2
- UMEP for Processing not installing on QGIS 3.22 HOT 3
- URock Prepare doesn't create BUILDINGS_WITH_HEIGHT HOT 14
- Python error when running the Morphometric Calculator (Point) algorithm HOT 6
- Error when installing UMEP for processing HOT 6
- Urock prepare: problems tmp/ folder? HOT 13
- URock problem HOT 7
- SOLWEIG warning HOT 7
- UMEP Processing / URock v2023a - crashes with Java Exception No transformation found from epsg:31255 to epsg:4326 HOT 13
- Urock prepare: building footprint optional? HOT 7
- SystemExit: 'jaydebeapi' Python package is missing HOT 10
- Trouble with Numba using Python 3.12 HOT 29
- UMEP for processing and UMEP error installing HOT 10
- UMEP > Post-Processor > Solar Radiation > SEBE (Visualisation). HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from umep-processing.