Hello,
I am interested in restoring original colours of the image i have uploaded to my project.
From what I've read so far there is a way to manipulate image colours using colormaps, however it is a bit unclear on how to implement them. Also it is mentioned in the beginning of the documentation that django-raster only uses a single band. In short could you confirm if it is possible to render image in its original colour, and perhaps give an example on how to implement colour maps.
I know that asking of this is fairly rude but i can't imagine that I am the only noob trying to use this project .

Hi I made some changes in here:
https://github.com/justRishi/django-raster/tree/improve-working-with-big-raster-files-needs-lot-of-ram

I put it here as an issue , as not sure If you would like my pull request.

Description:

Problem statement

Big raster files in AWS issues

• /tmp storage gets full with 20GB docker image limit in AWS
• it takes 7 /8 hours to process a 10GB Sentinel 2 tif

Query in parser.py to be written to db , is to big

in process_quadrant bulk_create a result which is a to big query to be fired off by postgres, query string is just to big and results in postgres firing off an out of memory error.

Can not remove raster layers from django admin

Changes

• GDAL vsimem for creating tiles in memory (and not in tmp files) in parser.py
  dest_file_name = os.path.join('/vsimem/', '{}.tif'.format(uuid.uuid4()))
• added to .bulk_create second parameter with with default 50 , so query to be written to db is not to big, 2nd parameter(for
  write bulk in batches of 2nd param) is new since Django (X?).
• removed from admin.py the following:

            def has_delete_permission(self, request, obj=None):
                return False

Solves

• processing time reduced to 1.5 hours of processing for 10GB S2 raster files (when using 16GB RAM and 2 CPU)
• no "query string buffer is to big" errors from postgres

Drawbacks using vsimem:

• vsimem needs a lot of more memory to process files , when not enough RAM celery crashes
• vsimem seems only to work nice with all_in_one parameter set (RASTER_PARSE_SINGLE_TASK = True )

Drawback using max batch-size parameter in bulkcreate:

will not work with old Django versions .

Hi,I have new problem。
http://localhost:8000/raster/algebra/{z}/{x}/{y}.png?layers=a=78,b=0,c=1,d=2
Is this code correct?
a:layerid
b:band1
c:band2
d:band3
Thank you

Hello, i am completely new regarding GDAL and Django GIS coul you give an example of code on how to pass raster stored in raster database to leaflet (since you mention it in documentation)

Im wrote simple demo project for django-raster based on docker.
If this is of interest to you, I can transfer the repository.

https://github.com/winsento/django-raster-demo

I want to warn in the project there are parts that need improvement.

Is there any example project or demo raster layer data for me to explore more?

I'am opening an issue for Django Raster

My working env:

OS version : macos 10.12.5
Dependency: 
 - libgeotiff 1.4.2
 - gdal 1.11.5_2
Python version : 3.5
Package List: 
 - django 1.10.5
 - django-raster 0.5

I was stuck in creating a new raster layer (segmentation fault) when i submit a new raster tiff. And I search for a while and got nothing.

Can you give me a demo of raster layer of geotiff or tutorial so that i can try to explore more and produce the right things?

just ignore my poorly-written english.

I'm looking forward for your comments. Thanks

First, let me thank you for this great django app I've just discovered. This is brilliant. In my project, I am using Celery a lot. I have a lot of celery queues. That would be great to have the ability to change the task queue that will be used (when it is not defined, the 'celery' named queue will be used).

You might want to have a setting similar as RASTER_USE_CELERY (for example RASTER_CELERY_QUEUE).

What do you think?

Would you like me to do a PR?

Hi,

I have some difficulties with generating tiles for some of my tif pictures. Tiles are not correctly generated for some reason. I attach screen with situation from Leaflet. I do not use legend and in raster layer log looks everything fine. I am using Django 1.10 and it is testing version, so it is on django development server

Some functions require multiple arguments to be useful, such as max(), min(), sum() etc. But it seems the algebra parser is unable to handle formulas like max(a,b). I get a 400 bad request on such formula's. Is there a different syntax I should be using?

Hello,

It looks like a migration file is missing. I installed django-raster from the master with pip and added raster to my INSTALLED_APPS. Running the makemigrations command creates a new migration file for raster:

::::| ./manage.py makemigrations --dry-run
Migrations for 'raster':
  0032_auto_20160425_1328.py:
    - Alter field expression on legendentry
    - Alter field datatype on rasterlayer
    - Alter field max_zoom on rasterlayer
    - Alter field nodata on rasterlayer
    - Alter field rasterfile on rasterlayer
    - Alter field srid on rasterlayer
    - Alter field log on rasterlayerparsestatus
    - Alter field status on rasterlayerparsestatus
    - Alter field rasterfile on rasterlayerreprojected

numpy.ma.masked_values() crashes on the None that can be in the nodata_value.

Hi,I have a problem about algebra.

Hi,

It would be nice to have a new release that support django 1.10.

Thanks

Hello,

I'm getting the following error when calling the pixel_value_from_point method:

File "/usr/local/lib/python3.6/site-packages/raster/utils.py", line 153, in pixel_value_from_point
    bbox = OGRGeometry.from_bbox(raster.extent)
  File "/usr/local/lib/python3.6/site-packages/django/contrib/gis/gdal/geometries.py", line 158, in from_bbox
    x0, y0, x1, y1 = bbox
TypeError: 'method' object is not iterable

I'm afraid the error comes from the line 153 where instead of

bbox = OGRGeometry.from_bbox(raster.extent)

You should do:

bbox = OGRGeometry.from_bbox(raster.extent())

Hope this make sense.

I'm not able to install django-raster with GDAL 1.9.2 (which is the gdal build supplied with my OS of CentOS 6.4) It looks like django-raster is dependent on GDAL 1.10 or higher from the errors I'm getting, is that correct? Any possible workarounds, besides doing a re-install to GDAL 1.10?
When I try to install, it appears the build tries to install GDAL 1.10 but cannot find the proper header files.
Open to advice I'd really like to build this, it looks like a fantastic toolkit for what I need. Thank you, and apologies if this is not posted in the proper place (as it is a request more than an 'issue'.)

When opening signed 8bit rasters, GDALRaster handles it as unsigned 8bit raster.

Hi,

I am using django-raster to save raster data into postgis database.
Here is my code

   resistanceTile = rm.RasterLayer(id=resistanceMap[0].id, name="Resistance", rasterfile=tmp_path, datatype='co')
   resistanceTile.save()

where 'tmp_path' is a path to tiff file with 2960x2580 resolution and the maximum zoom level is 15 by default.
Before saving tile, app is using 110MB of memory,but after calling resistanceTile.save() the memory increase to 1.7GB and not free.

Regards,
David Vardanyan

I have GeoTIFF with 2-bands from Landsat 8. Im try to generate NDVI via raster algebra.

GET parameters for layer and formula like this:

'layers=nir:1=5,red:0=5&formula=(nir-red)/(nir+red)'

But (nir-red) - return empty raster tile.

Raster here:
LC08_L1TP_177024_20170922_20170922_01_RT.TIF.zip

Hi
I want to use WMS.But i don't how to import them.Can you give me some example?Thank you.

Problem

If RASTER_USE_CELERY = True and (RASTER_PARSE_SINGLE_TASK = False or not set) then
a temp file is created multiple times in def open_raster_file in RasterLayerParser in parser.py
Also when not in the right reprojection, the projection is done multiple times.

Why problem

Big raster files are copied in my case 4 times, processed by GDAL 4 times . and sometimes (when not in the right projection) 4 times reprojected.

How tested

by adding self.log to print out tmp file creation resulting in:

How to mitigate

put RASTER_PARSE_SINGLE_TASK = True in settings , but meaning will not use concurrency to process raster file

Possible solution to process parallel and not duplicate work

1. check that only 1 tmp folder is created :
   so this line in parser.py should change self.tmpdir = tempfile.mkdtemp(dir=raster_workdir (as always unique)
2. self.dataset in parser.py (in class RasterLayerParser) should be shared by all parallel tasks for same raster file

Hi, I am new in webgis development. I am trying to load, display and show raster files using geodjango as the base framework. Thus, I am looking for the solution by using django-raster.

My env version info is like:

Python-3.6
Django-3.0.3
postgresql-9.5
postgresql-9.5-postgis-2.4
django-raster-0.8

I try to set django-raster as the installation shown, but I get an error when I migrate raster to database.

Operations to perform:
  Apply all migrations: admin, auth, contenttypes, raster, sessions, world
Running migrations:
  Applying raster.0002_auto_20140925_0723...Traceback (most recent call last):
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedTable: relation "raster_rastertile" does not exist


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 21, in <module>
    main()
  File "manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/core/management/base.py", line 328, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 233, in handle
    fake_initial=fake_initial,
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/migrations/operations/fields.py", line 249, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 565, in alter_field
    old_db_params, new_db_params, strict)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/backends/postgresql/schema.py", line 154, in _alter_field
    new_db_params, strict,
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 756, in _alter_field
    self.execute(self._create_index_sql(model, [new_field]))
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 142, in execute
    cursor.execute(sql, params)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/johnlyu/anaconda3/envs/geodjango/lib/python3.6/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "raster_rastertile" does not exist

Could you please help me?

Hi,

Thanks for developing this nice software!

I noticed that the display of rasters with a continuous colormap was not including values exactly at the boundaries of the raster data range. This can be an issue for e.g. rasters having a lot of zeros as lower limit. If this is a bug, it can be fixed by modifying utils.py line 82 from:

        # Compute alpha channel from mask if available.
        if numpy.ma.is_masked(dat):
            alpha = 255 * numpy.logical_not(dat.mask) * (norm >= 0) * (norm <= 1)
        else:
            alpha = 255 * (norm > 0) * (norm < 1)

to

        # Compute alpha channel from mask if available.
        if numpy.ma.is_masked(dat):
            alpha = 255 * numpy.logical_not(dat.mask) * (norm >= 0) * (norm <= 1)
        else:
            alpha = 255 * (norm >= 0) * (norm <= 1)    #Just changed this line

Sorry for not making a proper merge request, but it is just a small change.

I'm struggling to get this up and running in a virtualenv! It's entirely due to GDAL not installing on a mac using this line:

pip install GDAL

Could this be omitted with GDAL just listed as a required package? I do have GDAL (although an older version) which I installed using the kyngchaos installer: http://www.kyngchaos.com/software/frameworks#gdal_complete

Default colormap (if not setting up a legend) is not working properly (no range attribute set).
This leads to color artefacts like this:

This is related to code in

hi
i get error After I adjust:
1-pip install django-raster
2-INSTALLED_APPS = (
...
'raster',
)
3-path('raster/', include('raster.urls')),
4-python manage.py migrate

version my python 38 -64bit
get error after add layer:
ModuleNotFoundError: No module named 'celery.task'
"POST /admin/raster/rasterlayer/add/ HTTP/1.1" 500 164659

Hi.

I have to upload to server raster file and convert it when saving in file system if necessary.
This is algorithm.

import sys, os, uuid, subprocess
from osgeo import gdal, gdalconst, osr


SUPPORTED_GDT = (gdalconst.GDT_Byte, )
SUPPORTED_GDT_NAMES = ', '.join([
    gdal.GetDataTypeName(i) for i in SUPPORTED_GDT
])


# os.chdir(r'C:\Users\Галина Геннадьевна\YandexDisk\Бизнес\ООО ГеоПризма\Разработка\Примеры данных\Тестовые данные\epsg3857 (pseudo mercator)')
os.chdir(r'C:\Users\Sergey\YandexDisk\Бизнес\ООО ГеоПризма\Разработка\Примеры данных\Тестовые данные\epsg3857 (pseudo mercator)')

input_file = "raster.tif"

gdal_source_ds = gdal.Open(input_file, gdalconst.GA_ReadOnly)
if not gdal_source_ds:
    sys.exit('GDAL library failed to open file {0}.'.format(input_file))

if gdal_source_ds.RasterCount not in (1, 3, 4):
    sys.exit('Only RGB, RGBA and single-band rasters are supported.')

for band_index in range(1, gdal_source_ds.RasterCount + 1):
    band = gdal_source_ds.GetRasterBand(band_index)
    if band.DataType not in SUPPORTED_GDT:
        sys.exit("Band #%(band)d has type '%(type)s', however only following types are supported: %(all_types)s."
                 % dict(band=band_index, type=gdal.GetDataTypeName(band.DataType), all_types=SUPPORTED_GDT_NAMES))

raster_xsize = gdal_source_ds.RasterXSize
raster_ysize = gdal_source_ds.RasterYSize
band_count = gdal_source_ds.RasterCount

print("InputRasterXSize=%(InputRasterXSize)d  InputRasterYSize=%(InputRasterYSize)d  InputBandCount=%(InputBandCount)d"
    % dict(InputRasterXSize=raster_xsize, InputRasterYSize=raster_ysize, InputBandCount=band_count))


# get the projection (SRS)
gdal_source_ds_projection = gdal_source_ds.GetProjection()
# provides the origin coordinates and pixel sizes, along with rotation values if the image isn’t situated so the top faces north
gdal_source_ds_transform = gdal_source_ds.GetGeoTransform()

if not gdal_source_ds_projection or not gdal_source_ds_transform:
    sys.exit('Only raster files with projection info are supported')

gdal_source_ds_osr = osr.SpatialReference()
gdal_source_ds_osr.ImportFromWkt(gdal_source_ds_projection)
gdal_target_ds_osr = osr.SpatialReference()
gdal_target_ds_osr.ImportFromEPSG(3857)  # 3857 - target spatial reference

reproject = not gdal_source_ds_osr.IsSame(gdal_target_ds_osr)

uuid = str(uuid.uuid4().hex)

# Separate in two folder levels by first id characters
levels = (uuid[0:2], uuid[2:4])
# path = os.path.join(os.getcwd(), 'tiled rasters', *levels)
path = os.path.join(r'c:\\', 'tiled rasters', *levels)

# Create folders if needed
if not os.path.isdir(path):
    os.makedirs(path)

output_file = os.path.join(path, str(uuid) + '.tif')

if reproject:
    cmd = ['gdalwarp',
           '-of', 'GTiff',
           '-t_srs', 'EPSG:%d' % 3857]
    if gdal_source_ds.RasterCount == 3:
        cmd.append('-dstalpha')
else:
    cmd = ['gdal_translate', '-of', 'GTiff']

cmd.extend(('-co', 'TILED=YES',
            '-co', 'BIGTIFF=YES',
            input_file, output_file))

subprocess.check_call(cmd)

gdal_target_ds = gdal.Open(output_file, gdalconst.GA_Update)
if not gdal_target_ds:
    sys.exit('GDAL library failed to open file {0}.'.format(output_file))

# Build overviews/pyramid layers
gdal_target_ds.BuildOverviews('average', [2, 4, 8, 16, 32])

raster_xsize = gdal_target_ds.RasterXSize
raster_ysize = gdal_target_ds.RasterYSize
band_count = gdal_target_ds.RasterCount

print("OutputRasterXSize=%(OutputRasterXSize)d  OutputRasterYSize=%(OutputRasterYSize)d  OutputBandCount=%(OutputBandCount)d"
    % dict(OutputRasterXSize=raster_xsize, OutputRasterYSize=raster_ysize, OutputBandCount=band_count))

del gdal_source_ds
del gdal_target_ds

How can I use django-raster to implement this?

Thank you.

Django-raster is not compatible with upstream version of celery.

The changelog of the 4.0 release states:

• The experimental celery.contrib.methods feature has been removed, as there were far many bugs in the implementation to be useful.

I am planning to drop the use of celery to parallelize import of tiles as follows (this passes all the test suite locally) since I do the map imports in an async celery task anyway.

Would a PR with such change make it to master?

diff --git a/raster/tiles/parser.py b/raster/tiles/parser.py
index 41c4c16..35e27e1 100644
--- a/raster/tiles/parser.py
+++ b/raster/tiles/parser.py
@@ -8,7 +8,6 @@ import zipfile
 
 import numpy
 from celery import current_app, group
-from celery.contrib.methods import task_method
 
 from django.conf import settings
 from django.contrib.gis.gdal import GDALRaster, OGRGeometry
@@ -290,9 +289,8 @@ class RasterLayerParser(object):
 
         self.log('Creating {0} tiles in {1} quadrants at zoom {2}.'.format(self.nr_of_tiles(zoom), len(quadrants), zoom))
 
-        # Process quadrants in parallell
-        quadrant_task_group = group(self.process_quadrant.si(indexrange, zoom) for indexrange in quadrants)
-        quadrant_task_group.apply()
+        for indexrange in quadrants:
+            self.process_quadrant(indexrange, zoom)
 
         # Store histogram data
         if zoom == self.max_zoom:
@@ -305,7 +303,6 @@ class RasterLayerParser(object):
 
     _quadrant_count = 0
 
-    @current_app.task(filter=task_method)
     def process_quadrant(self, indexrange, zoom):
         """
         Create raster tiles for a quadrant of tiles defined by a x-y-z index

When adding a raster I am getting the following error:

InternalError at /admin/raster/rasterlayer/add/
None of the constraints specified could be added.  Is the schema name, table name or column name incorrect?
CONTEXT:  PL/pgSQL function addrasterconstraints(name,name,name,boolean,boolean,boolean,boolean,boolean,boolean,boolean,boolean,boolean,boolean,boolean,boolean) line 53 at RETURN
Request Method: POST
Request URL:    http://127.0.0.1:8000/admin/raster/rasterlayer/add/
Django Version: 1.7
Exception Type: InternalError
Exception Value:    
None of the constraints specified could be added.  Is the schema name, table name or column name incorrect?
CONTEXT:  PL/pgSQL function addrasterconstraints(name,name,name,boolean,boolean,boolean,boolean,boolean,boolean,boolean,boolean,boolean,boolean,boolean,boolean) line 53 at RETURN

I've connected to the database I am using with pgAdmin, and run

SELECT DropRasterConstraints('raster_rastertile'::name,'rast'::name);
SELECT AddRasterConstraints('raster_rastertile'::name,'rast'::name);

This runs fine; any thoughts on why it is breaking within django? I've tried including 'public'::name, as an additional parameter, incase it was a schema issues, but that does not make a difference.

Hi,

is there a way to annotate a RasterLayer queryset with pixel value for given coordinates? If not, how would you do to retrieve pixel value for a bunch of RasterLayer in an effective manner?

Thanks.

Hello,

I'm trying to show digital surface model on Mapbox, but I couldn't apply the continuous colour scheme on the raster

The raster include only 1 band.

I tried below URL but didn't work very well. I see the colourized pixels but they are not related with elevation information.

'http://127.0.0.1:8080/raster/algebra/{z}/{x}/{y}.png?layers=r:0=1&formula=r&scale=1964,6490&colormap={"continuous":true,"range":[0,255],"from":[165,0,38],"to":[0,104,55]}'

Thank you

Hello,

When I applied VARI formula on the tails, returns 400 bad request error. How can i solve this problem?
Min and max functions may solve this problem; however, they are not implemented right now.

VARI Formula:
(Green - Red) / (Green + Red - Blue)

My Tile URL:
http://127.0.0.1:8080/raster/algebra/{z}/{x}/{y}.png?layers=r:0=1,g:1=1,b:2=1&alpha&formula=(r-g)/(r+g-b)

Bands:

Thank you.

I'm trying to add raster into Raster layers using admin page,
but everytime i tried it always give [WinError 32],
I already tried to change permission on admin page and remove the file on temp data but still got the error.

I'm using Windows 10, PostgreSQL 11.4, PostGIS 2.5.3 and Python 3 using virtual env from conda.
My data only have 1 band (grayscale)

Here's the traceback error:

Environment:

Request Method: POST
Request URL: http://127.0.0.1:8000/admin/raster/rasterlayer/add/

Django Version: 2.2.6
Python Version: 3.7.3
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.gis',
'raster',
'ras.apps.RasConfig']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback:

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\raster\tasks.py" in create_tiles
40. parser.reproject_rasterfile()

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\raster\tiles\parser.py" in reproject_rasterfile
206. dest_zip = zipfile.ZipFile(dest.name, 'w', allowZip64=True)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\zipfile.py" in init
1204. self.fp = io.open(file, filemode)

During handling of the above exception ([Errno 13] Permission denied: 'C:\Users\DELL\AppData\Local\Temp\tmpaadzqk14\tmpmp_6awy2.zip'), another exception occurred:

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\core\handlers\exception.py" in inner
34. response = get_response(request)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\core\handlers\base.py" in _get_response
115. response = self.process_exception_by_middleware(e, request)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\core\handlers\base.py" in _get_response
113. response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\contrib\admin\options.py" in wrapper
606. return self.admin_site.admin_view(view)(*args, **kwargs)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\utils\decorators.py" in _wrapped_view
142. response = view_func(request, *args, **kwargs)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\views\decorators\cache.py" in _wrapped_view_func
44. response = view_func(request, *args, **kwargs)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\contrib\admin\sites.py" in inner
223. return view(request, *args, **kwargs)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\contrib\admin\options.py" in add_view
1634. return self.changeform_view(request, None, form_url, extra_context)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\utils\decorators.py" in _wrapper
45. return bound_method(*args, **kwargs)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\utils\decorators.py" in _wrapped_view
142. response = view_func(request, *args, **kwargs)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\contrib\admin\options.py" in changeform_view
1522. return self._changeform_view(request, object_id, form_url, extra_context)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\contrib\admin\options.py" in _changeform_view
1561. self.save_model(request, new_object, form, not add)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\contrib\admin\options.py" in save_model
1088. obj.save()

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\db\models\base.py" in save
741. force_update=force_update, update_fields=update_fields)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\db\models\base.py" in save_base
790. update_fields=update_fields, raw=raw, using=using,

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\dispatch\dispatcher.py" in send
175. for receiver in self._live_receivers(sender)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\django\dispatch\dispatcher.py" in
175. for receiver in self._live_receivers(sender)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\raster\models.py" in parse_raster_layer_if_status_is_unparsed
243. parse(instance.id)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\raster\tasks.py" in parse
131. all_in_one(rasterlayer_id, zoom_range)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\celery\local.py" in call
191. return self._get_current_object()(*a, **kw)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\celery\app\task.py" in call
392. return self.run(*args, **kwargs)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\raster\tasks.py" in all_in_one
77. create_tiles(rasterlayer_id, zoom_range, True)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\celery\local.py" in call
191. return self._get_current_object()(*a, **kw)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\celery\app\task.py" in call
392. return self.run(*args, **kwargs)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\site-packages\raster\tasks.py" in create_tiles
50. shutil.rmtree(parser.tmpdir)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\shutil.py" in rmtree
513. return _rmtree_unsafe(path, onerror)

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\shutil.py" in _rmtree_unsafe
397. onerror(os.unlink, fullname, sys.exc_info())

File "C:\Users\DELL\Anaconda3\envs\earth-analytics-python\lib\shutil.py" in _rmtree_unsafe
395. os.unlink(fullname)

Exception Type: PermissionError at /admin/raster/rasterlayer/add/
Exception Value: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\DELL\AppData\Local\Temp\tmpaadzqk14\test.tif_copy.GTiff'

Hi,I want to use admin to build legend,I failed.

Looking forward to your reply. Thank you.

Hello!

This is a pretty great package, but I seem not to be able to put multiple input rasters into the same RasterLayer. This would be useful for cases such as loading input tiles from satellite imagery of the entire Earth without first stitching them all together into one jumbo file (with associated memory requirements). Before I go ahead and add such functionality, I wanted to check if I'm missing some obvious way of doing this? It does seem like I could just give multiple layers the same name, but that seems like a dirty hack (and I haven't checked how the views treat that case).

What's the correct approach? Or do I just have to write my own?

Thanks!

https://github.com/geodesign/django-raster/blob/master/raster/models.py#L303

The defaults should be changed: from [] to list

https://docs.djangoproject.com/en/dev/ref/contrib/postgres/fields/#django.contrib.postgres.fields.ArrayField

If you give the field a default, ensure it’s a callable such as list (for an empty default) or a callable that returns a list (such as a function). Incorrectly using default=[] creates a mutable default that is shared between all instances of ArrayField.

Newer Django throws warnings:
WARNINGS:
raster.RasterLayerParseStatus.tile_levels: (postgres.E003) ArrayField default should be a callable instead of an instance so that it's not shared between all field instances.
HINT: Use a callable instead, e.g., use list instead of [].

Can it be that the latest update also added support for python 3.8? Before I was not able to parse raster with 3.8 and had to downgrade to 3.6. But then again I am also using now latest gdal docker image(osgeo/gdal:ubuntu-small-latest) .

Hi @yellowcap, sorry for the inconvenience, but I need to know if there is any way to capture the color of the map when I click on any point, or at least some information from the legends where I put the colors?
this is my map with all color:

you su when click i capture the lat and lng, but the color yellow no :/
the question is:
How can I do to capture the color when I click on the map?

or capture information from the legends entry either the name, the code or the expression?

thanks, if you can help me with any idea, if that is possible or some link on how to do it

Problem when adding raster layer, when processing a Large raster file(2.4GB) with max zoom level 14 the creation of tiles runs into an error at level 13 with:

psycopg2.errors.ProgramLimitExceeded: out of memory 
DETAIL: Cannot enlarge string buffer containing 0 bytes by 1233179600 more bytes.   
....   
File "/opt/venv/lib/python3.8/site-packages/raster/tiles/parser.py", line 412, in process_quadrant   
RasterTile.objects.bulk_create(batch)   
File "/opt/venv/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method   

(see complete stack at the end)

So code where it goes wrong is here at line 412:
/raster/tiles/parser.py

 # Commit batch to database and reset it
                if len(batch) == self.batch_step_size:
                    RasterTile.objects.bulk_create(batch)
                    batch = []
        # Commit remaining objects
        if len(batch):
            RasterTile.objects.bulk_create(batch)

last line is 412. Batch string is to long, to many objects.

Complete error log:

[2021-06-15 12:07:39] Creating 968 tiles in 1 quadrants at zoom 14.
[2021-06-15 12:07:39] Starting tile creation for quadrant 1 at zoom level 14
[2021-06-15 12:07:39] Traceback (most recent call last):
File "/opt/venv/lib/python3.8/site-packages/raster/tasks.py", line 39, in create_tiles
parser.create_tiles(zoom)
File "/opt/venv/lib/python3.8/site-packages/raster/tiles/parser.py", line 287, in create_tiles
self.populate_tile_level(zoom_levels)
File "/opt/venv/lib/python3.8/site-packages/raster/tiles/parser.py", line 313, in populate_tile_level
self.process_quadrant(indexrange, zoom)
File "/opt/venv/lib/python3.8/site-packages/raster/tiles/parser.py", line 347, in process_quadrant
snapped_dataset = self.dataset.warp({
File "/opt/venv/lib/python3.8/site-packages/django/contrib/gis/gdal/raster/source.py", line 410, in warp
target = GDALRaster(ds_input, write=True)
File "/opt/venv/lib/python3.8/site-packages/django/contrib/gis/gdal/raster/source.py", line 143, in __init__
self._ptr = capi.create_ds(
File "/opt/venv/lib/python3.8/site-packages/django/contrib/gis/gdal/prototypes/errcheck.py", line 127, in check_pointer
raise GDALException('Invalid pointer returned from "%s"' % func.__name__)
django.contrib.gis.gdal.error.GDALException: Invalid pointer returned from "GDALCreate"

[2021-06-15 12:07:51] Traceback (most recent call last):
File "/opt/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.ProgramLimitExceeded: out of memory
DETAIL: Cannot enlarge string buffer containing 0 bytes by 1233179600 more bytes.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/venv/lib/python3.8/site-packages/raster/tasks.py", line 39, in create_tiles
parser.create_tiles(zoom)
File "/opt/venv/lib/python3.8/site-packages/raster/tiles/parser.py", line 287, in create_tiles
self.populate_tile_level(zoom_levels)
File "/opt/venv/lib/python3.8/site-packages/raster/tiles/parser.py", line 313, in populate_tile_level
self.process_quadrant(indexrange, zoom)
File "/opt/venv/lib/python3.8/site-packages/raster/tiles/parser.py", line 412, in process_quadrant
RasterTile.objects.bulk_create(batch)
File "/opt/venv/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/venv/lib/python3.8/site-packages/django/db/models/query.py", line 514, in bulk_create
returned_columns = self._batched_insert(
File "/opt/venv/lib/python3.8/site-packages/django/db/models/query.py", line 1287, in _batched_insert
inserted_rows.extend(self._insert(
File "/opt/venv/lib/python3.8/site-packages/django/db/models/query.py", line 1270, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
File "/opt/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql
cursor.execute(sql, params)
File "/opt/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/opt/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/opt/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/opt/venv/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/opt/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
django.db.utils.OperationalError: out of memory
DETAIL: Cannot enlarge string buffer containing 0 bytes by 1233179600 more bytes.

Hi i faced with problem of installation and using(i installed all dependencies)
after executing
python3 manage.py migrate
i receive error
django.core.exceptions.ImproperlyConfigured: Raster fields require backends with raster support.

can somebody help me?

Hi

I'm getting such an error:

urlresolvers.py", line 203, in regex
    (regex, six.text_type(e)))
ImproperlyConfigured: "^algebra/(?P<z>[0-9]+)/(?P<x>[0-9]+)/(?P<y>[0-9]+)(?P<format>\.jpg|\.png)\.(?P<format>[a-z0-9]+)/?$" is not a valid regular expression: redefinition of group name u'format' as group 5; was group 4

It also show this on Pythex

Hi,

I have an issue with the use of programmatically created legends, but I am not sure if this is a bug or a misuse of my part.

First, I create programmatically a grayscale legend:

        legendGrayscale = Legend.objects.filter(title='grayscale').first()
        if legendGrayscale is None:
            LegendEntry.objects.all().delete()
            LegendSemantics.objects.all().delete()
            Legend.objects.all().delete()
            legendGrayscale = Legend.objects.create(title='grayscale', description='White to black continuous')
            fromSemantics = LegendSemantics.objects.create(name='from')
            toSemantics = LegendSemantics.objects.create(name='to')
            continuousSemantics = LegendSemantics.objects.create(name='continuous')
            fromEntry = LegendEntry.objects.create(semantics=fromSemantics, legend=legendGrayscale, expression='from', color='#FFFFFF', code='#FFFFFF')
            toEntry = LegendEntry.objects.create(semantics=toSemantics, legend=legendGrayscale, expression='to', color='#000000', code='#000000')
            continuousEntry = LegendEntry.objects.create(semantics=continuousSemantics, legend=legendGrayscale, expression='continuous', code='True')

But when I try to render tiles, it complains that the colors cannot be converted from hex to rgb. This is due to the colormap function in models.py where it tries to convert the color associated with the continuous expression to a rgb color:

    @property
    def colormap(self):
        legend = json.loads(self.json)
        cmap = {}
        for leg in legend:
            cmap[leg['expression']] = hex_to_rgba(leg['color'])
        return cmap

My workaround was to update the colormap function, but I am not sure if this was the actual issue, or if my legend was not correctly created.

    @property
    def colormap(self):
        legend = json.loads(self.json)
        cmap = {}
        for leg in legend:
            if leg['expression'] == 'continuous':
                cmap[leg['expression']] = leg['code']
            else:
                cmap[leg['expression']] = hex_to_rgba(leg['color'])
        return cmap

Package six has been removed from Django3

File "/usr/local/lib/python3.8/site-packages/raster/models.py", line 18, in <module> from raster.utils import hex_to_rgba File "/usr/local/lib/python3.8/site-packages/raster/utils.py", line 7, in <module> from django.utils import six ImportError: cannot import name 'six' from 'django.utils' (/usr/local/lib/python3.8/site-packages/django/utils/__init__.py)

i want to add raster functionality in django using postgis database using django-raster package and this docs.

here my code :

models.py

from django.contrib.gis.db import models
class RasterWithName(models.Model):
raster = models.RasterField()
name = models.TextField()
add raster using console :

from django.contrib.gis.gdal import GDALRaster
from myapp.models import RasterWithName
gdal_raster = GDALRaster('C:/Users/username/Desktop/image.tif')
rast = RasterWithName(name='one', raster=gdal_raster)
rast.save()
but i am very confused with Rendering tiles and urls.

docs say :

/raster/tiles/layer_id/{z}/{x}/{y}.png where the layer_id is the primary key of a raster layer. This structure can be used directly in online mapping software such as OpenLayers or Leaflet. An example request could look like this: /raster/tiles/23/8/536/143.png, returning a tile in png format of the layer with ID pk=23 at zoom level z=8 and indexes x=536 and y=143

but indexes x,y mean some random x,y coordinate in image ?anyway i using this url http://127.0.0.1:8000/raster/tiles/1/8/536/143.png or http://127.0.0.1:8000/raster/tiles/1/8/20/40.png(origin x,y in my image) and i take a black window in my browser,any idea where i have wrong in my url rendering ?

I'm trying to implement panoramic view of big images (not geo-related).
This is an example of what I would like to do: http://www.precipoint.com/o8-oil-microscope-scanner/

From the example above I see that they use leaflet.js and images with .gtif format.
It seems that all of the machinery from django-raster could be reused.
The first question: is it the right choice to use django raster for this?

The second question. When uploading an image via the admin website (I set up srid to 3857), the following exception occurs:

  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/raster/models.py", line 241, in parse_raster_layer_if_status_is_unparsed
    parse(instance.id)
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/raster/tasks.py", line 129, in parse
    all_in_one(rasterlayer_id, zoom_range)
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/celery/local.py", line 191, in __call__
    return self._get_current_object()(*a, **kw)
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/celery/app/task.py", line 392, in __call__
    return self.run(*args, **kwargs)
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/raster/tasks.py", line 75, in all_in_one
    create_tiles(rasterlayer_id, zoom_range, True)
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/celery/local.py", line 191, in __call__
    return self._get_current_object()(*a, **kw)
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/celery/app/task.py", line 392, in __call__
    return self.run(*args, **kwargs)
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/raster/tasks.py", line 39, in create_tiles
    parser.create_tiles(zoom)
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/raster/tiles/parser.py", line 286, in create_tiles
    self.populate_tile_level(zoom)
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/raster/tiles/parser.py", line 309, in populate_tile_level
    self.process_quadrant(indexrange, zoom)
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/raster/tiles/parser.py", line 348, in process_quadrant
    'height': (indexrange[3] - indexrange[1] + 1) * self.tilesize,
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/django/contrib/gis/gdal/raster/source.py", line 413, in warp
    c_void_p(), c_void_p(), c_void_p()
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/django/contrib/gis/gdal/prototypes/errcheck.py", line 118, in check_errcode
    check_err(result, cpl=cpl)
  File "~/.virtualenvs/testsite/lib/python3.7/site-packages/django/contrib/gis/gdal/error.py", line 59, in check_err
    raise e(msg)
django.contrib.gis.gdal.error.GDALException: FileIO

Before that I usually see:

GDAL_ERROR 1: b'The transformation is already "north up" or a transformation between pixel/line and georeferenced coordinates cannot be computed for /tmp/tmpb3y4vimd/almost-there_XFSV2t1.tiff. There is no affine transformation and no GCPs
. Specify transformation option SRC_METHOD=NO_GEOTRANSFORM to bypass this check.'

And after the server returns 500 I see sometimes (probably, because the directory is deleted):

GDAL_ERROR 1: b'Unable to save auxiliary information in /tmp/abcdef/almost-there_2mg1OYo.tif.aux.xml.'

How to overcome that?

Calling legend.delete() raises

DoesNotExist: Legend matching query does not exist.

it seem could not open the datasource

On several rasters I get a SoftTimeLimitExceeded Error from celery, is this normal and do I need to change a setting somewhere? The files are tiffs ~30MB

I succesfully installed django_raster with pip
Then I add 'raster' to installed_apps but
starting server (django 1.9 on Ubuntu 14.04) I get the following exception:

Unhandled exception in thread started by <function check_errors..wrapper at 0x7f23e2d3e950>
Traceback (most recent call last):
File "/home/enrico/.env/lib/python3.4/site-packages/django/utils/autoreload.py", line 226, in wrapper
fn(_args, *_kwargs)
File "/home/enrico/.env/lib/python3.4/site-packages/django/core/management/commands/runserver.py", line 109, in inner_run
autoreload.raise_last_exception()
File "/home/enrico/.env/lib/python3.4/site-packages/django/utils/autoreload.py", line 249, in raise_last_exception
six.reraise(__exception)
File "/home/enrico/.env/lib/python3.4/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/home/enrico/.env/lib/python3.4/site-packages/django/utils/autoreload.py", line 226, in wrapper
fn(_args, **kwargs)
File "/home/enrico/.env/lib/python3.4/site-packages/django/init.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/enrico/.env/lib/python3.4/site-packages/django/apps/registry.py", line 108, in populate
app_config.import_models(all_models)
File "/home/enrico/.env/lib/python3.4/site-packages/django/apps/config.py", line 202, in import_models
self.models_module = import_module(models_module_name)
File "/home/enrico/.env/lib/python3.4/importlib/init.py", line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 2254, in _gcd_import
File "", line 2237, in _find_and_load
File "", line 2226, in _find_and_load_unlocked
File "", line 1200, in _load_unlocked
File "", line 1129, in _exec
File "", line 1471, in exec_module
File "", line 321, in _call_with_frames_removed
File "/home/enrico/.env/lib/python3.4/site-packages/raster/models.py", line 11, in
from raster.fields import RasterField
File "/home/enrico/.env/lib/python3.4/site-packages/raster/fields.py", line 3, in
from raster.ogrraster import OGRRaster
File "/home/enrico/.env/lib/python3.4/site-packages/raster/ogrraster.py", line 7, in
from utils import convert_pixeltype, HEADER_STRUCTURE, HEADER_NAMES,
ImportError: No module named 'utils'