juliomalegria / django-chunked-upload Goto Github PK

Due to the updates on django 4.0> you can no longer import ugettext and this raises the following error.
ImportError: cannot import name 'ugettext' from 'django.utils.translation

Offset is a PositiveIntegerField with a max value of 2147483647 when PostgreSQL is used (https://code.djangoproject.com/ticket/12030#comment:14). This prevents uploads of files that are size of 2GB and larger.

A new file security checkup in Django 2.2.21 throws SuspiciousFileOperation.

For reference see:
https://docs.djangoproject.com/en/dev/releases/2.2.21/
django/django@04ac162

Django now prevents empty file name:

# Remove potentially dangerous names if name in {'', '.', '..'}: raise SuspiciousFileOperation("Could not derive file name from '%s'" % name)

The class ChunkedUploadView initializes the file with an empty name:

def create_chunked_upload(self, save=False, **attrs): """ Creates new chunked upload instance. Called if no 'upload_id' is found in the POST data. """ chunked_upload = self.model(**attrs) # file starts empty chunked_upload.file.save(name='', content=ContentFile(''), save=save) return chunked_upload

The name needs to be changed to something not empty to fix this issue.

Until this issue is fixed, it is possible to override create_chunked_upload with a custom class:

`
class MyChunkedUploadView(ChunkedUploadView):
"""
This view receives the posted chunk
"""

model = ChunkedUploadedFile
field_name = 'the_file'

def create_chunked_upload(self, save=False, **attrs):
    """
    Creates new chunked upload instance. Called if no 'upload_id' is
    found in the POST data.
    """
    chunked_upload = self.model(**attrs)
    # file starts empty
    chunked_upload.file.save(name='tmp', content=ContentFile(''), save=save)
    return chunked_upload`

Many thanks for your excellent library! I am trying to use it in my own case, but am finding that I am bumping up against an error where I'm hitting a dead end. In my view.py file, I continually am seeing the following error:

Internal Server Error: /upload/chunked_upload Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/views/generic/base.py", line 97, in dispatch return handler(request, *args, **kwargs) File "/Users/jonathanmartin/git/absFileNav/absFileNav/upload/views.py", line 308, in post return self._post(request, *args, **kwargs) File "/Users/jonathanmartin/git/absFileNav/absFileNav/upload/views.py", line 406, in _post chunked_upload = self.create_chunked_upload(save=False, **attrs) File "/Users/jonathanmartin/git/absFileNav/absFileNav/upload/views.py", line 357, in create_chunked_upload chunked_upload = self.model(**attrs) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/db/models/base.py", line 483, in __init__ _setattr(self, field.name, rel_obj) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/django/db/models/fields/related_descriptors.py", line 205, in __set__ if value is not None and not isinstance(value, self.field.remote_field.model._meta.concrete_model): AttributeError: 'str' object has no attribute '_meta'

The error occurs when I am calling self.model(**attrs). However, the value of my attrs variable is something like the following:

{'filename': 'filename.extension', 'user': <SimpleLazyObject: <function AuthenticationMiddleware.process_request.. at 0x10819f730>>}

Am I missing some attributes in the ChunkedUploadView object? Do you have any pointers for why I'm seeing this error? I am happy to provide more information if you have any time to give me some pointers. Am I supposed to be including the file in memory in my attributes? I have not changed much of the code from your original work.

Thank you again for developing this package and for any advice you might have! I’ve been stuck on this for a while now, and verbose logging isn’t getting me any further.

Hi There!

the method get_response_data() returns a Success message. Is there a way to include more information like:

messages = {}
messages.update({'message': ('Success')})
messages.update({'message': ('Whatever...')})
...
return messages

The actual code prints only the last entry of this return.

Hi!

Is not always, but sometimes during the chunks upload, the process simply frozen and nothing happens.

Anybody have any idea what could cause this bahavior?

Thanks!

It would be great if you could dispatch parallel chunk uploads. Currently, each chunk has to be sent sequentially, waiting for a response from the server to send the next chunk. Would be cool if you could specify the chunk start and end so that chunks could be sent out of order and retried if necessary.

If I find time I'll try to make a PR but just filing this here for now.

Hi Julio, I hope you had a good Christmas.

With the final call to ChunkedUploadCompleteView, I assume the checksum has to be the md5 for the whole file which has been uploaded? I am looking to upload very large files (like 10s of GB). Creating an md5 on the client side seems to be rather a slow process. Do you have any suggestions for the best way to accomplish this? Using https://github.com/satazor/SparkMD5, it takes about 40 seconds to compute an md5 hash for a 5GB file!

We have a use case where uploaded files are 30GB+ and we would like to display the checksums to users. However, the checksums take a few minutes to calculate every time. Also, the final POST request to ChunkedUploadCompleteView can time out depending on the web server configuration. It looks like this can be implemented by making _md5 a field on BaseChunkedUpload model.

Also, it would be great to have an option to run the checksum calculation as a Celery task but that might be a bit more involved.

Hi!

Thank you for sharing this package. It's really useful.

I've found a problem upgrading to the 1.1.2 release. Using runserver works great, but at least on uWSGI and gunicorn, it fails with the following error just after receiving the first chunk.

[2017-06-22 10:03:41,866] ERROR django.request-132 Internal Server Error: /api/chunked_upload/
Traceback (most recent call last):
  File "/home/juan/.virtualenvs/demo/lib/python3.5/site-packages/django/core/handlers/exception.py", line 42, in inner
    response = get_response(request)
  File "/home/juan/.virtualenvs/demo/lib/python3.5/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/juan/.virtualenvs/demo/lib/python3.5/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/juan/.virtualenvs/demo/lib/python3.5/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/juan/.virtualenvs/demo/lib/python3.5/site-packages/django/views/generic/base.py", line 88, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/juan/.virtualenvs/demo/lib/python3.5/site-packages/chunked_upload/views.py", line 93, in post
    return self._post(request, *args, **kwargs)
  File "/home/juan/.virtualenvs/demo/lib/python3.5/site-packages/chunked_upload/views.py", line 214, in _post
    chunked_upload.append_chunk(chunk, chunk_size=chunk_size, save=False)
  File "/home/juan/.virtualenvs/demo/lib/python3.5/site-packages/chunked_upload/models.py", line 69, in append_chunk
    self.file.write(chunk.read())
io.UnsupportedOperation: write

Same code with release 1.1.1 works as expected
I've tested it with uWSGI-2.0.14 and gunicorn-19.7.1

will this download the whole file to local disc first and then send it to s3?

You need to add a line like this to setup.py:

    packages=find_packages(),

Or otherwise explicitly specify that the management and management.commands packages should be included.

See: http://setuptools.readthedocs.io/en/latest/setuptools.html#using-find-packages

Hi,

I am thinking of using/pulling/copying chunked-upload in a project that I am working on right now as I need something very similar (http://github.com/sdaps/sdaps_web). I was wondering if you could clarify the license before I pull it in and screw up by e.g. accidentally relicensing it to AGPL.

Benjamin

I use user id in the chunked upload filepath. I couldn't figure out a way to cleanly replace upload_to=generate_filename without tampering with the code (if there is one feel free to close the issue). I ended up cloning the project and made some other changes as well, but this is something I think could be beneficial for many users.

Basic idea is:
file = models.FileField(max_length=255, upload_to=UPLOAD_PATH,
storage=STORAGE)

Then users are more free to alter the upload_to path.

For example i did this in my settings.py:
def generate_filename(instance, filename):
filename = str(instance.upload_id) + '.part'
return "{0}/{1}".format(instance.user.id, filename)

CHUNKED_UPLOAD_PATH = generate_filename

For it to be backwards compatible, a new setting could be added be where the default implementation is the old generate_filename.

Dear Sir,
I am trying to build a service where you could upload several files with django_chunked_upload. If I try to upload two files. The coleteUploadView is called twice but in you demo implementation md5 checksum is called on data.files[0]. so the second time the uploadCompleteView is called the error message 'checksum do not match' which is an rather obvious answer as md5 checksum is only calculated on the first file. Could it be possible to iterate over the files and callculate md5 and call completeUploadView separately for each file, or at least reset and recalculate md5?
Any help would be appreciated!

Yours truly

Soren Norrby

In my code I've extended BaseChunkedUpload without adding a user field. If the django request is authenticated, an error is thrown after injecting a user field into the attrs dict here:

https://github.com/juliomalegria/django-chunked-upload/blob/master/chunked_upload/views.py#L184

There should be an extra condition which verifies that a user field exists on self.model before injecting it into attrs.

Hello,

Import Error:

$ ... delete_expired_uploads --help
Traceback (most recent call last):
  ...
  File "/.../django-chunked-upload/chunked_upload/management/commands/delete_expired_uploads.py", line 7, in <module>
    from .settings import EXPIRATION_DELTA
ImportError: No module named settings

Guess it should be from chunked_upload.settings import EXPIRATION_DELTA

PS: Thanks for this product!

Hi,
I tested your app, using your demo, and when the download finishes the file extension keeps as ".part".
Am I doing something wrong?
Thanks for this product!

i start django server and get answer:

Watching for file changes with StatReloader Exception in thread django-main-thread: Traceback (most recent call last): File "/Users/vladimir/.pyenv/versions/3.11.2/lib/python3.11/threading.py", line 1038, in _bootstrap_inner self.run() File "/Users/vladimir/.pyenv/versions/3.11.2/lib/python3.11/threading.py", line 975, in run self._target(*self._args, **self._kwargs) File "/Users/vladimir/.pyenv/versions/project_name/lib/python3.11/site-packages/django/utils/autoreload.py", line 64, in wrapper fn(*args, **kwargs) File "/Users/vladimir/.pyenv/versions/project_name/lib/python3.11/site-packages/django/core/management/commands/runserver.py", line 125, in inner_run autoreload.raise_last_exception() File "/Users/vladimir/.pyenv/versions/project_name/lib/python3.11/site-packages/django/utils/autoreload.py", line 87, in raise_last_exception raise _exception[1] File "/Users/vladimir/.pyenv/versions/project_name/lib/python3.11/site-packages/django/core/management/__init__.py", line 394, in execute autoreload.check_errors(django.setup)() File "/Users/vladimir/.pyenv/versions/project_name/lib/python3.11/site-packages/django/utils/autoreload.py", line 64, in wrapper fn(*args, **kwargs) File "/Users/vladimir/.pyenv/versions/project_name/lib/python3.11/site-packages/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/Users/vladimir/.pyenv/versions/project_name/lib/python3.11/site-packages/django/apps/registry.py", line 116, in populate app_config.import_models() File "/Users/vladimir/.pyenv/versions/project_name/lib/python3.11/site-packages/django/apps/config.py", line 269, in import_models self.models_module = import_module(models_module_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/vladimir/.pyenv/versions/3.11.2/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1206, in _gcd_import File "<frozen importlib._bootstrap>", line 1178, in _find_and_load File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 690, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 940, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/Users/vladimir/.pyenv/versions/project_name/lib/python3.11/site-packages/chunked_upload/models.py", line 10, in <module> from .constants import CHUNKED_UPLOAD_CHOICES, UPLOADING File "/Users/vladimir/.pyenv/versions/project_name/lib/python3.11/site-packages/chunked_upload/constants.py", line 1, in <module> from django.utils.translation import ugettext as _ ImportError: cannot import name 'ugettext' from 'django.utils.translation' (/Users/vladimir/.pyenv/versions/project_name/lib/python3.11/site-packages/django/utils/translation/__init__.py)

I was setting up some custom client-side code to interact with the chunked upload API and during the debugging process I discovered the requirement for setting the Content-Range header in the chunk uploading process only after digging through the source code. Would be great to add this more explicitly in the docs in Step 3 of Typical Usage

Hey is there a example of the fileupload javascript somewhere to make use of this?

Hi,
is the following code correct? strftime for filename looks a bit strange for me :)
https://github.com/juliomalegria/django-chunked-upload/blob/master/chunked_upload/models.py#L23

Django 1.9 removes support for automatic loading of initial_data fixtures and initial SQL data. Because of this, it doesn't perform any migrations for django-chunked-upload.

Hi there. I hope you don't mind me asking a support question! I've been trying for a few days to get you code working in my project.

I am simply trying to add django-chunked-upload to my django project. I have downloaded the source code, created the egg file and have added it to my INSTALLED_APPS. However, I can't get the model migrated into my database. Everytime I run makemigrations chunked_upload, it states that there are no changes and doesn't do anything. Do you know what I need to do to get this working with django 1.7?

Thanks and Happy Christmas!

Hi,
I wanted to suggest to use the JSONResponseMixin or other mixins to render the response instead of the Response class, It seems more flexible to me.

Cheers,
qwattash

patch:

https://pastebin.com/7DGmhsd5

hi,
I want to add a new feature,
If we want our service more efficient, we can use CRC. And someone cares checksum collision very much, he will choose sha384 if he want to rename his uploaded file with the checksum(it is said that it is more secure. http://security.stackexchange.com/questions/33108/why-does-some-popular-software-still-use-md5)
So I suggest that django-chunked-upload should provide more checksum algorithms.

I dont know whether someone tried to deploy a website using this upload?
Locally it works, but when I try to upload it on a deployed server, then this mistakes occurs:

BytesReceived="0", ErrorCode="Reached the end of the file. (0x80070026)"

Any clues

?

Hi,

My chunked upload parts are not been cleaned after the time set in CHUNKED_UPLOAD_EXPIRATION_DELTA.

Any idea?

Thanks.

In views.py, the method 'check_permissions' uses user.is_authenticated(). is_authenticated as a function call is no longer supported in Django 2.0, and is throwing an exception. See: https://docs.djangoproject.com/en/2.0/ref/contrib/auth/#django.contrib.auth.models.User.is_authenticated

doesn't support changing settings at runtime, eg with django.test.modify_settings

https://docs.djangoproject.com/en/2.0/_modules/django/test/utils/#modify_settings

hey @juliomalegria ! I'm familiar with the JQuery uploads for which this is intended (from a form POST) and am wondering if you have examples of using the module for restful POST (and if this is supported?) I'm hoping to add chunked upload to a POST from a command line client (not a form) and am hoping this could do the trick :) Thanks!

Hello

I may have missed this somewhere. But is it possible to get the chunked upload directory to automatically clean up at upload completion?

My upload completion assigns the file to a model file field, which results in the file being copied to a new location. However, the chunked_upload '.part' data accumulates.

Thanks!

Hi guys,

I'm running python 3.6.x with django 1.11.11 on gunicorn 19.7.1.

When I upload a file, I get the following error:

"File was not opened in write mode."

Sentry points me to this line:

    self.file.open(mode='ab')  # mode = append+binary
    # We can use .read() safely because chunk is already in memory
    self.file.write(chunk.read())

I already checked this issue but it was related to an old version of django.

Any ideas?

Thanks
Ron

PS: Maybe this is related to my storage of media files on AWS S3? Chunked parts are created but have size zero.

If something goes wrong and a ChunkedUpload object is created, but the FileField does not have an underlying file on storage then delete() is impossible. Any attempt to delete the ChunkedUpload from the DB will raise an error:

   upload.delete()
  File "/home2/dtrudgian/Git/astrocyte/.venv/lib/python2.7/site-packages/chunked_upload/models.py", line 62, in delete
    storage, path = self.file.storage, self.file.path
  File "/home2/dtrudgian/Git/astrocyte/.venv/lib/python2.7/site-packages/django/db/models/fields/files.py", line 63, in _get_path
    self._require_file()
  File "/home2/dtrudgian/Git/astrocyte/.venv/lib/python2.7/site-packages/django/db/models/fields/files.py", line 46, in _require_file
    raise ValueError("The '%s' attribute has no file associated with it." % self.field.name)
ValueError: The 'file' attribute has no file associated with it.

See PR #24 for a possible solution?

I used django-chunked-upload-demo with Django 1.7 version

Traceback:
File "/Users/macmini1/Workspace/django-chunked-upload-demo/django-chunked-upload-demo-env/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  111.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/macmini1/Workspace/django-chunked-upload-demo/django-chunked-upload-demo-env/lib/python2.7/site-packages/django/views/generic/base.py" in view
  69.             return self.dispatch(request, *args, **kwargs)
File "/Users/macmini1/Workspace/django-chunked-upload-demo/django-chunked-upload-demo-env/lib/python2.7/site-packages/django/views/generic/base.py" in dispatch
  87.         return handler(request, *args, **kwargs)
File "/Users/macmini1/Workspace/django-chunked-upload-demo/django-chunked-upload-demo-env/lib/python2.7/site-packages/chunked_upload/views.py" in post
  93.             return self._post(request, *args, **kwargs)
File "/Users/macmini1/Workspace/django-chunked-upload-demo/django-chunked-upload-demo-env/lib/python2.7/site-packages/chunked_upload/views.py" in _post
  210.                         status=http_status.HTTP_200_OK)
File "/Users/macmini1/Workspace/django-chunked-upload-demo/django-chunked-upload-demo-env/lib/python2.7/site-packages/chunked_upload/response.py" in __init__
  15.             *args, **kwargs
File "/Users/macmini1/Workspace/django-chunked-upload-demo/django-chunked-upload-demo-env/lib/python2.7/site-packages/django/http/response.py" in __init__
  318.         super(HttpResponse, self).__init__(*args, **kwargs)

Exception Type: TypeError at /api/chunked_upload
Exception Value: __init__() got an unexpected keyword argument 'mimetype'

It could be resolved not passing the mimetype to django.http.HttpResponse class

I came here because I need an online interface to a Python data pipe (Excel Anonymiser Script) Motorrat/anonyxel#2

Data Pipe is a rather standard task. Many existing scripts would profit from being exposed on the web. For example I have another data pipe project autosklearn-zeroconf

So I would like to propose to pack your project in a drop-in app with a two hooks/callbacks - file received (to start processing pipeline) and return new file to the user (once processed). The programmer would then just have to configure the program (data pipe) name to be called upon file upload and write the resulting file out in a specified way (callback function?)

It could serve as a standard go-to technology to expose python scripts to internal users or where authentication etc. is not applicable, so no further functionality is required.

Hi,

I was trying to understand how to make the ChunkedUploadCompleteView handle a whole file upload as a last chunk, il could be useful to have a method called when the file upload is not chunked.
I have seen that someone checks for existence of the HTTP_X_FILE_NAME header to distinguish between the two, but I don't know how clean this solution may be.

Thanks!
Keep up the good work!

I struggled to POST to a subclass of ChunkedUploadCompleteView

While the jquery .ajax() post seems to form the request perfectly, I could not get the right combination of headers on my fetch post.

I believe this is because the view is expecting a form to be involved because in _post() the request is expected to have request.POST.get('upload_id') to retrieve upload_id

As a result, POSTing to ChunkedUploadCompleteView with fetch did not see the upload_id or md5 in the body of my fetch-based POSTs.

I ended up overriding _post to change how the view found the two variables like this:

class MyChunkedUploadCompleteView(ChunkedUploadCompleteView):
	# ...

    def _post(self, request, *args, **kwargs):
        request_body = json.loads(request.read().decode('utf-8'))
        upload_id = request_body['upload_id']
        md5 = request_body['md5']

        error_msg = None
        if self.do_md5_check:
		# ...

My fetch method on the javascript side looks like this:

const postData = (url, data) => {
  const csrfToken = getCSRFTokenCookie('csrftoken');
  return fetch(url, {
    method: 'POST',
    credentials: 'same-origin',
    headers: {
      'X-CSRFToken': csrfToken,
      "Accept": "application/json",
      "Content-Type": "application/json"
    },
    body: JSON.stringify(data)
  });
};

This works just fine for posting to ChunkedUploadView which does not require accessing request.POST.

Here's .ajax() post that does supply a request.POST

  $.ajax({
    type: "POST",
    url: url,
    data: data,
    dataType: "json",
    success: function (data) {
      console.log(data);
    }
  });

Is there a better way to handle getting data to this view using fetch apart from overriding this internal function?

I have the following code in my project:

settings.py:
CHUNKED_UPLOAD_STORAGE_CLASS = 'sstraffic.storage_backends.PublicMediaStorage'

storage_backends.py:

class PublicMediaStorage(S3Boto3Storage):
    location = 'media'
    default_acl = 'public-read'
    file_overwrite = False

I have an error:

[DJANGO] ERROR 2020-10-28 14:42:52,843 log django.request.log_response:228: Internal Server Error: /instructors/create-program/6cebcf3a-ae80-483c-8c40-b4dbf5179858/upload-media-chunks/
Traceback (most recent call last):
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/django/views/generic/base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/django/utils/decorators.py", line 45, in _wrapper
    return bound_method(*args, **kwargs)
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/django/utils/decorators.py", line 45, in _wrapper
    return bound_method(*args, **kwargs)
  File "/home/alex/code/sstraffic/app/instructors/decorators.py", line 18, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/django/views/generic/base.py", line 97, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/chunked_upload/views.py", line 100, in post
    return self._post(request, *args, **kwargs)
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/chunked_upload/views.py", line 221, in _post
    chunked_upload.append_chunk(chunk, chunk_size=chunk_size, save=False)
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/chunked_upload/models.py", line 65, in append_chunk
    with open(self.file.path, mode='ab') as file_obj:  # mode = append+binary
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/django/db/models/fields/files.py", line 57, in path
    return self.storage.path(self.name)
  File "/home/alex/.local/share/virtualenvs/sstraffic-kPpfodf3/lib/python3.7/site-packages/django/core/files/storage.py", line 109, in path
    raise NotImplementedError("This backend doesn't support absolute paths.")
NotImplementedError: This backend doesn't support absolute paths.
[DJANGO] ERROR 2020-10-28 14:42:52,846 basehttp django.server.log_message:154: "POST /instructors/create-program/6cebcf3a-ae80-483c-8c40-b4dbf5179858/upload-media-chunks/ HTTP/1.1" 500 26323

I fixed it by adding a function path to my PublicMediaStorage class.

    def path(self, name):
        return name

Is not that too hucky? Will not I break something by this?
It would be nice to have that working out of the box.

class LoggingException(object):

    def __init__(self, get_response):
        print('__init__')
        self.get_response = get_response

    def __call__(self, request):
        print('--process_request--')
        response = self.get_response(request)
        print('--process_reponse--')
        print(dir(response))
        return response

    def process_exception(self, request, exception):
        print(request)
        print("logging exception")
        print(exception)

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'myadmin.loggingException.LoggingException',
]

when I upload large file（200M），return 400 status， but I cannot log error

I am combining django_filer with django_chunked_upload, and I am trying to upload a file to the folder location my url is currently pointed to. Instead, all chunked uploads go to an unsorted folder.

Is there a way I can add a folder_id to the chunked_upload views.py file?

Hello I'm trying to work an integration using django-storages (sftp or ftp), but it is not working.

Can someone help me, please?

When I work using django.core.files.storage.FileSystemStorage it works normally.

Should it work using django-storages (sftp)? Or not?

Thanks.

class AbstractChunkedUpload(models.Model):
    def append_chunk(self, chunk, chunk_size=None, save=True):
        self.file.close()
        with open(self.file.path, mode='ab') as file_obj:  # mode = append+binary
            file_obj.write(chunk.read())  # We can use .read() safely because chunk is already in memory

        if chunk_size is not None:
            self.offset += chunk_size
        elif hasattr(chunk, 'size'):
            self.offset += chunk.size
        else:
            self.offset = self.file.size
        self._md5 = None  # Clear cached md5
        if save:
            self.save()
        self.file.close()  # Flush

api_1 | This backend doesn't support absolute paths. Traceback (most recent call last):
api_1 | File "/usr/share/scapole/upload/models.py", line 12, in append_chunk
api_1 | print(self.file.path)
api_1 | File "/home/scapole/.local/lib/python3.9/site-packages/django/db/models/fields/files.py", line 57, in path
api_1 | return self.storage.path(self.name)
api_1 | File "/home/scapole/.local/lib/python3.9/site-packages/django/core/files/storage.py", line 116, in path
api_1 | raise NotImplementedError("This backend doesn't support absolute paths.")
api_1 | NotImplementedError: This backend doesn't support absolute paths.

1. Can this work with s3?

2. What is the max file size , that can be uploaded with this

3. Is it compatible with django rest api?

Hi,

sorry for this stupid question, but someone can help me?
how can I handle errors like OSError: [Errno 28] No space left on device etc?

In the ChunkedUploadCompleteView class in the post method there is the following call on line 286:
self.on_completion(chunked_upload.get_uploaded_file(), request)
This opens the uploaded file and returns it to the on_completion user method.

This prevented me from moving the uploaded file to a separate database model as the file was open.
I also couldn't do this work in the post_save method as it causes an error as the file is no longer present when the get_uploade_file attempts to open the file.

In the end I shifted the model shifting to the get_response_data method which allowed me to do what I want without modifying this project.