upload module only recognizes headers if they are in a single line.. the most problematic on is Content-Disposition

This one works:

--FormBoundary_dNC89KxOWaziZb1mJpHL2f1m2
Content-Disposition: form-data; name="details"
Content-Type: application/json

{"uri":"fooo:\/\/fn","time":"2011-12-19T09:06:26.000"}
--FormBoundary_dNC89KxOWaziZb1mJpHL2f1m2
Content-Disposition: form-data; name="file"; filename="data taken from.txt"
Content-Type: application/octet-stream

oijgdifojgodfjgodjfogdfo
--FormBoundary_dNC89KxOWaziZb1mJpHL2f1m2--

This one doesn't. Please notice that the filename part is in new line and indented in this case by one space, but by RFC there can be multiple white spaces

--FormBoundary_dNC89KxOWaziZb1mJpHL2f1m2
Content-Disposition: form-data; name="details"
Content-Type: application/json

{"uri":"fooo:\/\/fn","time":"2011-12-19T09:06:26.000"}
--FormBoundary_dNC89KxOWaziZb1mJpHL2f1m2
Content-Disposition: form-data; name="file"; 
 filename="data taken from.txt"
Content-Type: application/octet-stream

oijgdifojgodfjgodjfogdfo
--FormBoundary_dNC89KxOWaziZb1mJpHL2f1m2--

We use this module for uploading larger files into our systems. The module works great over regular SSL mode. Recently we have been experimenting with SPDY and now that 1.3 is out with nginx 1.5.10 we would like to try to use both.

I have compiled 1.5.10 successfully with SPDY and the upload module. Again uploads work as expected under both SSL and non-SSL connections. When SPDY is enabled we see the following error (or a derivation thereof):

[alert] 19832#0: *2 epoll_ctl(1, 15) failed (17: File exists), client: 10.0.0.10, server: my.local.server, request: "POST ....

This happens for every chunk we try to upload to the server via POST. The server is running nginx 1.5.10 on CentOS 6.3 that is updated as of this issue. Using the latest commit from your 2.2 branch.

I did a quick on Google and found something similar but it was related to both ipv4 and v6 being enabled at the same time with SPDY enabled. Did not help any with this issue. Please let me know if you need any more information.

nginx_upload_module-2.0.10/ngx_http_upload_module.c:216:30: error: invalid digit "9" in octal constant
nginx_upload_module-2.0.10/ngx_http_upload_module.c:1188:30: error: invalid digit "9" in octal constant
nginx_upload_module-2.0.10/ngx_http_upload_module.c:1713:30: error: invalid digit "9" in octal constant

Any ideas?

Jeff

I'm getting this error: could not find named location "http://localhost:81". The value, http://localhost:81, is the identical location I use for proxy_pass and it works fine.

My full config is:

server {
    listen       80;

    location ~* \/(favicon.ico$|apple-tou.*|robots.txt$) {
        root /ria/src/views/images/favicons;
    } 

    location ~ /(common/|views/(js|css)/|views/images/|views/pages/[^/]+_client\.js$|notification/images/).* {
        root /ria/src;
    }

    location ~ \/ria\/.*(\.jpg|\.jpeg|\.gif|\.png)$$ {
        proxy_pass          http://localhost:5984;
        proxy_set_header    X-Real-IP  $remote_addr;
        proxy_set_header    Authorization "Basic cm9vdDp0Ymdjb21wczI=";
    }

    location / {
        proxy_pass          http://localhost:81;
        proxy_set_header    X-Real-IP  $remote_addr;
    }

    location /upload {
        client_max_body_size    0;
        upload_max_file_size    0;
        upload_pass             http://localhost:81;
        upload_store            /mnt/uploads 1;

        upload_set_form_field $upload_field_name.name "$upload_file_name";
        upload_set_form_field $upload_field_name.path "$upload_tmp_path";
        upload_aggregate_form_field "$upload_field_name.sha1" "$upload_file_sha1";
        upload_aggregate_form_field "$upload_field_name.size" "$upload_file_size";
        #upload_pass_form_field "^fallback$|^login$|^usession$";
        upload_cleanup 400 404 499 500-505;
    }

    # Stop Image and Document Hijacking
    location ~* (\.jpg|\.jpeg|\.gif|\.png)$ {
        if ($http_referer !~ ^(https?:\/\/[^\/]*comps2.info) ) {
            return 444;
        }
    }

    error_page 500 502 503 504  /50x.html;
    location = /50x.html {
        root   /ria/etc/nginx/html;
    }
    #error_page  404              /404.html;
    #charset koi8-r;
}

When posting a file to url via:
curl -i -F "[email protected]" http://localhost/file_upload

I get a "503 Service Temporarily Unavailable"

Here's the nginx.conf excerpt:

  location /file_upload {
    upload_pass /upload;
    upload_store /tmp 1;
    upload_store_access user:rw group:rw all:r;
    upload_set_form_field $upload_field_name.original_name "$upload_file_name";
    upload_set_form_field $upload_field_name.content_type "$upload_content_type";
    upload_set_form_field $upload_field_name.filepath "$upload_tmp_path";

    upload_pass_form_field "^authenticity_token$";
    upload_cleanup 400 404 499 500-505;
  }

  location /upload {
    # passenger_enabled on;
      proxy_pass http://127.0.0.1:81;
      proxy_redirect default;
  }

(nginx-0.7.67 with passenger-2.2.15 and nginx-upload-progress-module and nginx-upload-module 2.0.12)

Hi

I am using upload module in ngx-openresty-1.2.6.6. but its give me "405 Not Allowed".

Modules I have used:-
nginx-upload-module-2.2
nginx-upload-progress-module-0.9.0
ngx_cache_purge-2.1
ngx_openresty-1.2.6.6

   location /games {
       upload_pass @process_redirects;
        upload_pass_args on;
        upload_pass_form_field ".*";
        upload_store /tmp/;
        upload_set_form_field $upload_field_name.name "$upload_file_name";
        upload_set_form_field $upload_field_name.content_type "$upload_content_type";
        upload_set_form_field $upload_field_name.path "$upload_tmp_path";
        upload_cleanup 400 404 499 500-505;
        }
    location @process_redirects {
       try_files /$host$uri @backend_app;
        }

if i use following code, request go to my app.

   location /games {
          try_files /$host$uri @backend_app;
         }

Hi.
During chunked upload I uploaded all but last chunk, one by one.
In the nginx conf I have:

upload_cleanup 404 500-505;

On the last chunk upload:

POST /upload_ext_946/ HTTP/1.1
Content-Disposition: attachment; filename="75c23a0ab76c930f559cdbdf144a4f00.apk"
Content-Type: application/octet-stream
X-Content-Range: bytes 11827200-11832954/11832955
Session-ID: 402746096
Content-Length: 5755
Connection: Keep-Alive

I see 499 error in access log and the following in the state-file:

11827200-11832954/11832955

So, chunk was uploaded correctly and server has all file, but state file seems to be corrupt.
Is this correct?

Problem uploading file name - test"try.txt (with double quotes inside file name) under firefox browser.
It pass to backend file content as a file name.

{"note": [""], "upfile": ["erthtyjt\nrthy\ntrh\nrt\nh\nrt\nh\nrt\nh\nrth\nr\nth\nr\nth\n\nt\n\n"]}

here is debug from nginx.
http://pastebin.com/P6nkNNj3

even more on http://forum.nginx.org/read.php?2,238613

I just tried building on a fresh ubuntu-11.10 instance (on Rackspace) and got this error. It builds fine on 11.04 (gcc-4.5.2).

I also tried building the current head (5b5d008), but it fails with "ngx_garbage_collector_temp_handler undeclared" at ngx_http_upload_module.c:1104:31 on both 11.04 and 11.10.

Here's the 2.2.0 output on 11.10:

root@host:~/build# tar -zxvf ~/downloads/nginx-1.0.10.tar.gz 
root@host:~/build# tar -zxvf ~/downloads/nginx_upload_module-2.2.0.tar.gz 
root@host:~/build# cd nginx-1.0.10/
root@host:~/build/nginx-1.0.10# ./configure --add-module=../nginx_upload_module-2.2.0
checking for OS
 + Linux 3.0.0-12-virtual x86_64
checking for C compiler ... found
 + using GNU C compiler
 + gcc version: 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3) 

[...]

adding module in ../nginx_upload_module-2.2.0
 + ngx_http_upload_module was configured
checking for PCRE library ... found
checking for system md library ... not found
checking for system md5 library ... not found
checking for OpenSSL md5 crypto library ... found
checking for sha1 in system md library ... not found
checking for OpenSSL sha1 crypto library ... found
checking for zlib library ... found
creating objs/Makefile

Configuration summary
  + using system PCRE library
  + OpenSSL library is not used
  + md5: using system crypto library
  + sha1: using system crypto library
  + using system zlib library

  nginx path prefix: "/usr/local/nginx"
  nginx binary file: "/usr/local/nginx/sbin/nginx"
  nginx configuration prefix: "/usr/local/nginx/conf"
  nginx configuration file: "/usr/local/nginx/conf/nginx.conf"
  nginx pid file: "/usr/local/nginx/logs/nginx.pid"
  nginx error log file: "/usr/local/nginx/logs/error.log"
  nginx http access log file: "/usr/local/nginx/logs/access.log"
  nginx http client request body temporary files: "client_body_temp"
  nginx http proxy temporary files: "proxy_temp"
  nginx http fastcgi temporary files: "fastcgi_temp"
  nginx http uwsgi temporary files: "uwsgi_temp"
  nginx http scgi temporary files: "scgi_temp"

root@host:~/build/nginx-1.0.10# make
make -f objs/Makefile
make[1]: Entering directory `/root/build/nginx-1.0.10'
gcc -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Wunused-function -Wunused-variable -Wunused-value -Werror -g  -I src/core -I src/event -I src/event/modules -I src/os/unix -I objs \
        -o objs/src/core/nginx.o \
        src/core/nginx.c

[...]

gcc -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Wunused-function -Wunused-variable -Wunused-value -Werror -g  -I src/core -I src/event -I src/event/modules -I src/os/unix -I objs -I src/http -I src/http/modules \
        -o objs/src/http/modules/ngx_http_upstream_ip_hash_module.o \
        src/http/modules/ngx_http_upstream_ip_hash_module.c
gcc -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Wunused-function -Wunused-variable -Wunused-value -Werror -g   -I src/core -I src/event -I src/event/modules -I src/os/unix -I objs -I src/http -I src/http/modules -I src/mail \
        -o objs/addon/nginx_upload_module-2.2.0/ngx_http_upload_module.o \
        ../nginx_upload_module-2.2.0/ngx_http_upload_module.c
../nginx_upload_module-2.2.0/ngx_http_upload_module.c: In function ‘ngx_http_upload_merge_ranges’:
../nginx_upload_module-2.2.0/ngx_http_upload_module.c:1681:18: error: variable ‘result’ set but not used [-Werror=unused-but-set-variable]
cc1: all warnings being treated as errors

make[1]: *** [objs/addon/nginx_upload_module-2.2.0/ngx_http_upload_module.o] Error 1
make[1]: Leaving directory `/root/build/nginx-1.0.10'
make: *** [build] Error 2

is your module resolve conflicts in temporary file names ? i have a couple upload servers which writes to the same mounted directory(NFS)

When compiling nginx with this module I get the following errors - please advice:
/root/src/nginx-upload-module//ngx_http_upload_module.c: In function ngx_http_upload_merge_loc_conf:/root/src/nginx-upload-module//ngx_http_upload_module.c:1104:31: error: ngx_garbage_collector_temp_handler undeclared (first use in this function)
/root/src/nginx-upload-module//ngx_http_upload_module.c:1104:31: note: each undeclared identifier is reported only once for each function it appears in /root/src/nginx-upload-module//ngx_http_upload_module.c:1104:31: error: passing argument 1 of ngx_conf_merge_path_value from incompatible pointer type [-Werror]
src/core/ngx_file.h:140:7: note: expected struct ngx_conf_t * but argument is of type struct ngx_path_t *
/root/src/nginx-upload-module//ngx_http_upload_module.c:1104:31: error: passing argument 2 of ngx_conf_merge_path_value from incompatible pointer type [-Werror]
src/core/ngx_file.h:140:7: note: expected struct ngx_path_t ** but argument is of type struct ngx_path_t *
/root/src/nginx-upload-module//ngx_http_upload_module.c:1104:31: error: passing argument 3 of ngx_conf_merge_path_value from incompatible pointer type [-Werror]
src/core/ngx_file.h:140:7: note: expected struct ngx_path_t * but argument is of type char *
/root/src/nginx-upload-module//ngx_http_upload_module.c:1104:31: error: passing argument 4 of ngx_conf_merge_path_value makes pointer from integer without a cast [-Werror]
src/core/ngx_file.h:140:7: note: expected struct ngx_path_init_t * but argument is of type int
/root/src/nginx-upload-module//ngx_http_upload_module.c:1104:31: error: too many arguments to function ngx_conf_merge_path_value
src/core/ngx_file.h:140:7: note: declared here
/root/src/nginx-upload-module//ngx_http_upload_module.c: In function ngx_http_upload_pass_form_field:
/root/src/nginx-upload-module//ngx_http_upload_module.c:1490:5: error: passing argument 1 of ngx_regex_compile from incompatible pointer type [-Werror]
src/core/ngx_regex.h:45:11: note: expected struct ngx_regex_compile_t * but argument is of type struct ngx_str_t *
/root/src/nginx-upload-module//ngx_http_upload_module.c:1490:5: error: too many arguments to function ngx_regex_compile
src/core/ngx_regex.h:45:11: note: declared here
/root/src/nginx-upload-module//ngx_http_upload_module.c:1490:14: error: assignment makes pointer from integer without a cast [-Werror]
/root/src/nginx-upload-module//ngx_http_upload_module.c:1497:5: error: implicit declaration of function ngx_regex_capture_count [-Werror=implicit-function-declaration]
/root/src/nginx-upload-module//ngx_http_upload_module.c:1501:28: error: ngx_regex_capture_count_n undeclared (first use in this function)
/root/src/nginx-upload-module//ngx_http_upload_module.c:1501:54: error: expected) before string constant
cc1: all warnings being treated as errors

make[1]: *** [objs/addon/nginx-upload-module/ngx_http_upload_module.o] Error 1
make[1]: Leaving directory `/root/src/nginx-1.0.1'
make: *** [build] Error 2

There is a bug with the Nginx Upload Module when the HTTP boundary header contains a double quote, e.g.:

Content-Type: multipart/form-data; boundary="MyBoundary"

without the double quotes, the request works fine, e.g.:

Content-Type: multipart/form-data; boundary=MyBoundary

The upload module includes the double quotes in the actual boundary, thus failing to parse the multipart body content.

../nginx-upload-module/ngx_http_upload_module.c
../nginx-upload-module/ngx_http_upload_module.c: In function ‘ngx_http_upload_merge_loc_conf’:
../nginx-upload-module/ngx_http_upload_module.c:1104: error: ‘ngx_garbage_collector_temp_handler’ undeclared (first use in this function)
../nginx-upload-module/ngx_http_upload_module.c:1104: error: (Each undeclared identifier is reported only once
../nginx-upload-module/ngx_http_upload_module.c:1104: error: for each function it appears in.)
cc1: warnings being treated as errors
../nginx-upload-module/ngx_http_upload_module.c:1104: error: passing argument 1 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
src/core/ngx_file.h:142: note: expected ‘struct ngx_conf_t ’ but argument is of type ‘struct ngx_path_t *’
../nginx-upload-module/ngx_http_upload_module.c:1104: error: passing argument 2 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
src/core/ngx_file.h:142: note: expected ‘struct ngx_path_t *’ but argument is of type ‘struct ngx_path_t ’
../nginx-upload-module/ngx_http_upload_module.c:1104: error: passing argument 3 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
src/core/ngx_file.h:142: note: expected ‘struct ngx_path_t *’ but argument is of type ‘char *’
../nginx-upload-module/ngx_http_upload_module.c:1104: error: passing argument 4 of ‘ngx_conf_merge_path_value’ makes pointer from integer without a cast
src/core/ngx_file.h:142: note: expected ‘struct ngx_path_init_t *’ but argument is of type ‘int’
../nginx-upload-module/ngx_http_upload_module.c:1104: error: too many arguments to function ‘ngx_conf_merge_path_value’
../nginx-upload-module/ngx_http_upload_module.c: In function ‘ngx_http_upload_pass_form_field’:
../nginx-upload-module/ngx_http_upload_module.c:1490: error: passing argument 1 of ‘ngx_regex_compile’ from incompatible pointer type
src/core/ngx_regex.h:50: note: expected ‘struct ngx_regex_compile_t *’ but argument is of type ‘struct ngx_str_t *’
../nginx-upload-module/ngx_http_upload_module.c:1490: error: too many arguments to function ‘ngx_regex_compile’
../nginx-upload-module/ngx_http_upload_module.c:1490: error: assignment makes pointer from integer without a cast
../nginx-upload-module/ngx_http_upload_module.c:1497: error: implicit declaration of function ‘ngx_regex_capture_count’
../nginx-upload-module/ngx_http_upload_module.c:1501: error: ‘ngx_regex_capture_count_n’ undeclared (first use in this function)
../nginx-upload-module/ngx_http_upload_module.c:1501: error: expected ‘)’ before string constant
make[1]: ** [objs/addon/nginx-upload-module/ngx_http_upload_module.o] Error 1
make[1]: Leaving directory `/usr/src/nginx/nginx-1.1.16'
make: *** [build] Error 2

Hello... i installed nginx/0.8.54 and nginx_upload_module-2.2.0 with nginx-upload-progress-module... i place standart configuration file with

worker_processes 20;

error_log logs/error.log notice;

working_directory /usr/local/nginx;

events {
worker_connections 1024;
}

http {
include mime.types;
default_type application/octet-stream;

server {
    listen       80;
    client_max_body_size 100m;

    # Upload form should be submitted to this location
    location /upload {
        # Pass altered request body to this location
        upload_pass   @test;

        # Store files to this directory
        # The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist
        upload_store /tmp 1;

        # Allow uploaded files to be read only by user
        upload_store_access user:r;

        # Set specified fields in request body
        upload_set_form_field "${upload_field_name}_name" $upload_file_name;
        upload_set_form_field "${upload_field_name}_content_type" $upload_content_type;
        upload_set_form_field "${upload_field_name}_path" $upload_tmp_path;

        # Inform backend about hash and size of a file
        upload_aggregate_form_field "${upload_field_name}_md5" $upload_file_md5;
        upload_aggregate_form_field "${upload_field_name}_size" $upload_file_size;

        upload_pass_form_field "^submit$|^description$";
    }

    # Pass altered request body to a backend
    location @test {
        proxy_pass   http://localhost:8080;
    }
}

}

and then i start nginx next command

/etc/init.d/nginx start

and i have a trouble :

Starting nginx: /usr/sbin/nginx: /usr/lib/libxml2.so.2: no version information available (required by /usr/sbin/nginx)
[emerg]: unknown directive "upload_pass" in /etc/nginx/nginx.conf:23
configuration file /etc/nginx/nginx.conf test failed

Please help me...

It seems that this module no longer works on nginx 1.3.12.

gcc -c -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall -DFORTIFY_SOURCE=2 -fstack-protector -D_FORTIFY_SOURCE=2  -DNDK_SET_VAR  -I src/core -I src/event -I src/event/modules -I src/os/unix -I /build/buildd/nginx-1.3.12/debian/modules/nginx-development-kit/objs -I objs/addon/ndk -I /build/buildd/nginx-1.3.12/debian/modules/nginx-http-push/src -I /usr/include/lua5.1 -I /build/buildd/nginx-1.3.12/debian/modules/nginx-lua/src/api -I /usr/include/libxml2 -I objs -I src/http -I src/http/modules -I src/http/modules/perl -I /build/buildd/nginx-1.3.12/debian/modules/nginx-development-kit/src -I src/mail \
                -o objs/addon/nginx-upload-module/ngx_http_upload_module.o \
                /build/buildd/nginx-1.3.12/debian/modules/nginx-upload-module/ngx_http_upload_module.c
/build/buildd/nginx-1.3.12/debian/modules/nginx-upload-module/ngx_http_upload_module.c: In function 'ngx_http_upload_merge_ranges':
/build/buildd/nginx-1.3.12/debian/modules/nginx-upload-module/ngx_http_upload_module.c:1681:18: warning: variable 'result' set but not used [-Wunused-but-set-variable]
/build/buildd/nginx-1.3.12/debian/modules/nginx-upload-module/ngx_http_upload_module.c: In function 'ngx_http_read_upload_client_request_body':
/build/buildd/nginx-1.3.12/debian/modules/nginx-upload-module/ngx_http_upload_module.c:2628:15: error: 'ngx_http_request_body_t' has no member named 'to_write'
/build/buildd/nginx-1.3.12/debian/modules/nginx-upload-module/ngx_http_upload_module.c:2687:7: error: 'ngx_http_request_body_t' has no member named 'to_write'
/build/buildd/nginx-1.3.12/debian/modules/nginx-upload-module/ngx_http_upload_module.c: In function 'ngx_http_do_read_upload_client_request_body':
/build/buildd/nginx-1.3.12/debian/modules/nginx-upload-module/ngx_http_upload_module.c:2769:57: error: 'ngx_http_request_body_t' has no member named 'to_write'
/build/buildd/nginx-1.3.12/debian/modules/nginx-upload-module/ngx_http_upload_module.c:2785:19: error: 'ngx_http_request_body_t' has no member named 'to_write'
/build/buildd/nginx-1.3.12/debian/modules/nginx-upload-module/ngx_http_upload_module.c:2877:45: error: 'ngx_http_request_body_t' has no member named 'to_write'
make[2]: *** [objs/addon/nginx-upload-module/ngx_http_upload_module.o] Error 1

Build fails with nginx-0.8.53.

Configure options:

./configure --prefix='/opt/nginx' --with-http_ssl_module --with-http_flv_module  --with-http_realip_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-http_stub_status_module --with-poll_module --add-module=/usr/local/src/nginx-upload-module  --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --user=www-data --group=www-data

Trace:

/usr/local/src/nginx-upload-module/ngx_http_upload_module.c: In function ‘ngx_http_upload_merge_loc_conf’:
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1104: error: ‘ngx_garbage_collector_temp_handler’ undeclared (first use in this function)
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1104: error: (Each undeclared identifier is reported only once
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1104: error: for each function it appears in.)
cc1: warnings being treated as errors
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1104: error: passing argument 1 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
src/core/ngx_file.h:140: note: expected ‘struct ngx_conf_t *’ but argument is of type ‘struct ngx_path_t *’
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1104: error: passing argument 2 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
src/core/ngx_file.h:140: note: expected ‘struct ngx_path_t **’ but argument is of type ‘struct ngx_path_t *’
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1104: error: passing argument 3 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
src/core/ngx_file.h:140: note: expected ‘struct ngx_path_t *’ but argument is of type ‘char *’
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1104: error: passing argument 4 of ‘ngx_conf_merge_path_value’ makes pointer from integer without a cast
src/core/ngx_file.h:140: note: expected ‘struct ngx_path_init_t *’ but argument is of type ‘int’
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1104: error: too many arguments to function ‘ngx_conf_merge_path_value’
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c: In function ‘ngx_http_upload_pass_form_field’:
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1490: error: passing argument 1 of ‘ngx_regex_compile’ from incompatible pointer type
src/core/ngx_regex.h:45: note: expected ‘struct ngx_regex_compile_t *’ but argument is of type ‘struct ngx_str_t *’
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1490: error: too many arguments to function ‘ngx_regex_compile’
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1490: error: assignment makes pointer from integer without a cast
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1497: error: implicit declaration of function ‘ngx_regex_capture_count’
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1501: error: ‘ngx_regex_capture_count_n’ undeclared (first use in this function)
/usr/local/src/nginx-upload-module/ngx_http_upload_module.c:1501: error: expected ‘)’ before string constant
make[1]: *** [objs/addon/nginx-upload-module/ngx_http_upload_module.o] Error 1
make[1]: *** Waiting for unfinished jobs....
cc1: warnings being treated as errors
/usr/local/src/nginx-upload-module/ngx_upload_unzip_filter_module.c: In function ‘ngx_http_unzip_inflate_process_chain’:
/usr/local/src/nginx-upload-module/ngx_upload_unzip_filter_module.c:1175: error: comparison between signed and unsigned integer expressions
make[1]: *** [objs/addon/nginx-upload-module/ngx_upload_unzip_filter_module.o] Error 1

Hi

HTML5 does support file uploading using xmlhttprequest, which will post the raw content of file in POST body. I wonder if upload module can also support this new kind of file uploading method.

Just like this example here, but he didnt mention anything on how to modify upload module to accept PUT method.

"For our part, we’ve made some modifications to the venerable nginx upload module, allowing it to also accept PUT requests with raw file contents in the body."

http://onehub.com/blog/posts/designing-an-html5-drag-drop-file-uploader-using-sinatra-and-jquery-part-1/

Thx
Andrey

When compiling nginx 1.3.9 and using upload module 2.2.0 as an addon module, the compilation fails (at least for me on Linux using gcc). The error is as follows:

gcc -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g   -I src/core -I src/event -I src/event/modules -I src/os/unix -I /usr/include/libxml2 -I objs -I src/http -I src/http/modules -I src/mail \
        -o objs/addon/nginx_upload_module-2.2.0/ngx_http_upload_module.o \
        ../nginx_upload_module-2.2.0/ngx_http_upload_module.c
../nginx_upload_module-2.2.0/ngx_http_upload_module.c: In function ‘ngx_http_read_upload_client_request_body’:
../nginx_upload_module-2.2.0/ngx_http_upload_module.c:2628: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
../nginx_upload_module-2.2.0/ngx_http_upload_module.c:2687: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
../nginx_upload_module-2.2.0/ngx_http_upload_module.c: In function ‘ngx_http_do_read_upload_client_request_body’:
../nginx_upload_module-2.2.0/ngx_http_upload_module.c:2769: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
../nginx_upload_module-2.2.0/ngx_http_upload_module.c:2785: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
../nginx_upload_module-2.2.0/ngx_http_upload_module.c:2877: error: ‘ngx_http_request_body_t’ has no member named ‘to_write’
make[1]: *** [objs/addon/nginx_upload_module-2.2.0/ngx_http_upload_module.o] Error 1

Thanks for your work on this software!

It would be nice to allow upload_cleanup with HTTP success statuses like 200, so we can benefit of the advantages of autocleanup after parsing a temporary file.

using nginx 1.4.4 and 2.2 upload module am getting segfault using args in upload_pass

conf:
worker_processes 2;
worker_rlimit_core 500M;
working_directory /tmp/;

error_log /tmp/error.log notice;

events {
worker_connections 1024;
}

http {
default_type application/octet-stream;
variables_hash_max_size 1024;
variables_hash_bucket_size 128;

server {
    listen       80;
    client_max_body_size 7m;
    root /tmp/nginx-upload/web;

    # Upload form should be submitted to this location
    location /admin {
        location ~ ^/admin/upload/(\d+) {
            # Pass altered request body to this location
            upload_pass   /test/$1;

            upload_store /tmp/pica;

            upload_store_access all:rw;

            # Set specified fields in request body
            upload_set_form_field $upload_field_name.name "$upload_file_name";
            upload_set_form_field $upload_field_name.content_type "$upload_content_type";
            upload_set_form_field $upload_field_name.path "$upload_tmp_path";

            # Inform backend about hash and size of a file
            upload_aggregate_form_field $upload_field_name.md5 "$upload_file_md5";
            upload_aggregate_form_field $upload_field_name.size "$upload_file_size";

            upload_pass_form_field "^submit$|^description$";
        }

        proxy_pass   http://localhost:8080/;
    }

    location / {
        proxy_pass   http://localhost:8080/;
    }

    # Pass altered request body to a backend
    location /test/2 {
        proxy_pass   http://localhost:8080/;
    }
}

}

logs:
2013/12/05 15:09:30 [alert] 8745#0: worker process 8759 exited on signal 11 (core dumped)
2013/12/05 15:09:30 [notice] 8745#0: start worker process 8793
2013/12/05 15:09:30 [notice] 8745#0: signal 29 (SIGIO) received

Program terminated with signal 11, Segmentation fault.
#0 0x0000000000502f33 in ngx_http_upload_eval_state_path (r=0x929270)

at /root/src/szn-nginx/szn-nginx-extras/../addons/nginx-upload-module/ngx_http_upload_module.c:967

967 if(ulcf->state_store_path->is_dynamic) {
(gdb) bt
#0 0x0000000000502f33 in ngx_http_upload_eval_state_path (r=0x929270)

at /root/src/szn-nginx/szn-nginx-extras/../addons/nginx-upload-module/ngx_http_upload_module.c:967

#1 0x0000000000502bf2 in ngx_http_upload_handler (r=0x929270)

at /root/src/szn-nginx/szn-nginx-extras/../addons/nginx-upload-module/ngx_http_upload_module.c:865

#2 0x000000000044c1ae in ngx_http_core_content_phase (r=0x929270, ph=0x943210)

at src/http/ngx_http_core_module.c:1408

#3 0x000000000044acf8 in ngx_http_core_run_phases (r=0x929270)

at src/http/ngx_http_core_module.c:888

#4 0x000000000044ac6f in ngx_http_handler (r=0x929270)

at src/http/ngx_http_core_module.c:871

#5 0x0000000000459ebe in ngx_http_process_request (r=0x929270)

at src/http/ngx_http_request.c:1851

#6 0x00000000004587bc in ngx_http_process_request_headers (rev=0x94af00)

at src/http/ngx_http_request.c:1282

#7 0x0000000000457b36 in ngx_http_process_request_line (rev=0x94af00)

at src/http/ngx_http_request.c:963

#8 0x0000000000456a51 in ngx_http_wait_request_handler (rev=0x94af00)

at src/http/ngx_http_request.c:486

#9 0x000000000042f605 in ngx_event_process_posted (cycle=0x91e510, posted=0x8ed220)

at src/event/ngx_event_posted.c:40

#10 0x000000000042d170 in ngx_process_events_and_timers (cycle=0x91e510)

at src/event/ngx_event.c:276

#11 0x000000000043a979 in ngx_worker_process_cycle (cycle=0x91e510, data=0x1)

at src/os/unix/ngx_process_cycle.c:807

#12 0x0000000000437286 in ngx_spawn_process (cycle=0x91e510,

proc=0x43a7b3 <ngx_worker_process_cycle>, data=0x1, name=0x61810b "worker process",
respawn=1) at src/os/unix/ngx_process.c:198

#13 0x000000000043a3c2 in ngx_reap_children (cycle=0x91e510)

at src/os/unix/ngx_process_cycle.c:619

#14 0x00000000004390a9 in ngx_master_process_cycle (cycle=0x91e510)

at src/os/unix/ngx_process_cycle.c:180

#15 0x00000000004076f6 in main (argc=3, argv=0x7fffffffeae8) at src/core/nginx.c:412

(gdb) print u
Display all 288 possibilities? (y or n)
(gdb) print ulc
ulcf ulckpwdf
(gdb) print ulc
ulcf ulckpwdf
(gdb) print ulcf->state_store_path
$1 = (ngx_http_upload_path_t *) 0x0
(gdb) print ulcf->state_store_path

I have successfully configured nginx with this command:
./configure --prefix=/opt/nginx --with-debug --without-mail_pop3_module --without- mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --add-module=/usr/local/src/nginx_upload_module-2.0.8

But when I do 'make' I get this compile error:
gcc -c -O -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Wunused-function -Wunused-variable -Wunused-value -Werror -g -I src/core -I src/event -Isrc/event/modules -I src/os/unix -I objs -I src/http -I src/http/modules -I src/mail \ -o objs/addon/nginx_upload_module-2.0.8/ngx_http_upload_module.o \ /usr/local /src/nginx_upload_module-2.0.8/ngx_http_upload_module.c
/usr/local/src/nginx_upload_module-2.0.8/ngx_http_upload_module.c: In function ‘ngx_http_upload_merge_loc_conf’:
/usr/local/src/nginx_upload_module-2.0.8/ngx_http_upload_module.c:1151: error: ‘ngx_garbage_collector_temp_handler’ undeclared (first use in this function)
/usr/local/src/nginx_upload_module-2.0.8/ngx_http_upload_module.c:1151: error: (Each undeclared identifier is reported only once
/usr/local/src/nginx_upload_module-2.0.8/ngx_http_upload_module.c:1151: error: for each function it appears in.)
cc1: warnings being treated as errors
/usr/local/src/nginx_upload_module-2.0.8/ngx_http_upload_module.c:1151: error: passing argument 1 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
/usr/local/src/nginx_upload_module-2.0.8/ngx_http_upload_module.c:1151: error: passing argument 2 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
/usr/local/src/nginx_upload_module-2.0.8/ngx_http_upload_module.c:1151: error: passing argument 3 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
/usr/local/src/nginx_upload_module-2.0.8/ngx_http_upload_module.c:1151: error: passing argument 4 of ‘ngx_conf_merge_path_value’ makes pointer from integer without a cast
/usr/local/src/nginx_upload_module-2.0.8/ngx_http_upload_module.c:1151: error: too many arguments to function ‘ngx_conf_merge_path_value’
make[1]: *** [objs/addon/nginx_upload_module-2.0.8/ngx_http_upload_module.o] Error 1
make[1]: Leaving directory `/usr/local/src/nginx-0.7.61'
make: *** [build] Error 2

We've been getting segfaults for our upload urls after reloading the server a few times - this is reproducible. I've installed from chris-lea's ubuntu ppa (nginx-extras, version 1.5.10-1chl1~precise1) which is a pretty up to date version of the upload module (only missing the last commit). After a few reloads, nginx will start to segfault on one of the two upload urls configured (generally only one of them).

I've been looking through the core file with gdb and it appears to be segfaulting in: (from src/http/ngx_http_script.c)

void
ngx_http_script_flush_complex_value(ngx_http_request_t *r,
    ngx_http_complex_value_t *val)
{
    ngx_uint_t *index;

    index = val->flushes;

    if (index) {
        while (*index != (ngx_uint_t) -1) {

            if (r->variables[*index].no_cacheable) {
                r->variables[*index].valid = 0;
                r->variables[*index].not_found = 0;
            }

            index++;
        }
    }
}

index (val->flushes) doesn't look like a pointer value - it is 1 in the core that I have - but then I'm not familiar enough with the nginx code to know exactly what's happening.

Going to do more investigation cutting down on modules loaded and config used etc, but in the meantime, would someone have some clues as to what's happening?

Thanks!

Chris

I've got the following information:

Segfault:

syslog:

Feb 19 16:44:35 cbtsof7 kernel: [1392441.036355] nginx[20783]: segfault at 1 ip 000000000048bba9 sp 00007fff0824fe38 error 4 in nginx[400000+2d8000]

error.log:

2014/02/19 16:44:35 [alert] 17040#0: worker process 20783 exited on signal 11 (core dumped)

Backtrace:

(gdb) bt
#0  ngx_http_script_flush_complex_value (r=0xca55a0, val=0xdc2728) at src/http/ngx_http_script.c:44
#1  0x000000000048bc15 in ngx_http_complex_value (r=0xca55a0, val=0xdc2728, value=0x7fff0824ff00) at src/http/ngx_http_script.c:71
#2  0x00000000004f778b in ngx_http_upload_eval_state_path (r=0xca55a0) at /build/buildd/nginx-1.5.10/debian/modules/nginx-upload-module-2.2/ngx_http_upload_module.c:975
#3  ngx_http_upload_handler (r=0xca55a0) at /build/buildd/nginx-1.5.10/debian/modules/nginx-upload-module-2.2/ngx_http_upload_module.c:865
#4  ngx_http_upload_handler (r=0xca55a0) at /build/buildd/nginx-1.5.10/debian/modules/nginx-upload-module-2.2/ngx_http_upload_module.c:771
#5  0x0000000000478a02 in ngx_http_core_content_phase (r=0xca55a0, ph=0xdc4050) at src/http/ngx_http_core_module.c:1410
#6  0x00000000004732f5 in ngx_http_core_run_phases (r=0xca55a0) at src/http/ngx_http_core_module.c:888
#7  0x000000000047f2a2 in ngx_http_process_request (r=0xca55a0) at src/http/ngx_http_request.c:1842
#8  0x00000000004801e6 in ngx_http_process_request_line (rev=0xf5ab98) at src/http/ngx_http_request.c:954
#9  0x0000000000480842 in ngx_http_ssl_handshake_handler (c=0x1165b48) at src/http/ngx_http_request.c:743
#10 ngx_http_ssl_handshake_handler (c=0x1165b48) at src/http/ngx_http_request.c:694
#11 0x0000000000460098 in ngx_event_process_posted (cycle=0xc630a0, posted=0x908f08) at src/event/ngx_event_posted.c:40
#12 0x0000000000466f99 in ngx_worker_process_cycle (cycle=0xc630a0, data=<optimised out>) at src/os/unix/ngx_process_cycle.c:816
#13 0x0000000000465955 in ngx_spawn_process (cycle=0xc630a0, proc=0x466ee0 <ngx_worker_process_cycle>, data=0x0, name=0x62fd47 "worker process", respawn=-4) at src/os/unix/ngx_process.c:198
#14 0x00000000004671d4 in ngx_start_worker_processes (cycle=0xc630a0, n=4, type=-4) at src/os/unix/ngx_process_cycle.c:364
#15 0x000000000046809a in ngx_master_process_cycle (cycle=0xc630a0) at src/os/unix/ngx_process_cycle.c:249
#16 0x000000000044636d in main (argc=<optimised out>, argv=<optimised out>) at src/core/nginx.c:407 

version:

nginx version: nginx/1.5.10
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/var/run/nginx.pid --with-pcre-jit --with-debug --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-http_spdy_module --with-ipv6 --with-mail --with-mail_ssl_module --with-openssl=/build/buildd/nginx-1.5.10/debian/openssl-1.0.1f --add-module=/build/buildd/nginx-1.5.10/debian/modules/nginx-auth-pam --add-module=/build/buildd/nginx-1.5.10/debian/modules/headers-more-nginx-module --add-module=/build/buildd/nginx-1.5.10/debian/modules/nginx-development-kit --add-module=/build/buildd/nginx-1.5.10/debian/modules/nginx-echo --add-module=/build/buildd/nginx-1.5.10/debian/modules/nginx-http-push --add-module=/build/buildd/nginx-1.5.10/debian/modules/nginx-upload-progress --add-module=/build/buildd/nginx-1.5.10/debian/modules/nginx-upload-module-2.2 --add-module=/build/buildd/nginx-1.5.10/debian/modules/nginx-upstream-fair --add-module=/build/buildd/nginx-1.5.10/debian/modules/nginx-dav-ext-module --add-module=/build/buildd/nginx-1.5.10/debian/modules/nginx-cache-purge

config:

Our config is long and convoluted - I'll work on a cut down test case and post that here. In the meantime, here are the appropriate bits for the upload server:
nginx.conf

user vault;
worker_processes  4;
worker_rlimit_core  500M;
working_directory   /data/nginxcores;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
  worker_connections  2048;
}

http {
  include       /etc/nginx/mime.types;
  default_type  application/octet-stream;

  types_hash_max_size 2048;
  types_hash_bucket_size 64;

  access_log    /var/log/nginx/access.log;
  log_format  fluentd '"hostname":"$hostname","server":"$server_name", "method":"$request_method", "remoteaddr":"$remote_addr", "uri":"$request_uri", "status":$status, "referer":"$http_referer", "useragent":"$http_user_agent","reqtime":$request_time, "bytes":$bytes_sent, "virtualhost":"$http_host", "msec":$msec';
  access_log  /var/log/nginx/fluent.log fluentd;

  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;

  keepalive_timeout  0;
  client_body_timeout 300;
  client_header_timeout 300;
  send_timeout 300;

  proxy_read_timeout 300;

  gzip  off;

  server_names_hash_bucket_size 64;
  variables_hash_max_size 1024;

  include /etc/nginx/conf.d/*.conf;
  include /etc/nginx/sites-enabled/*;
}

virtual host config (there are 2 of these, exactly the same, except for different URIs and upload_stores):

....
   # Upload form should be submitted to this location
   location = /upload/1 {

        # Pass altered request body to this location
        upload_pass   @test;

        # Store files to this directory
        # The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist
        upload_store /mnt1/assets/tmp 1;

        # Allow uploaded files to be read only by user
        upload_store_access user:rw group:rw all:rw;
#       upload_store_access 066

        # Set specified fields in request body
        upload_set_form_field $upload_field_name.name "$upload_file_name";
        upload_set_form_field $upload_field_name.content_type "$upload_content_type";
        upload_set_form_field $upload_field_name.path "$upload_tmp_path";

        # Inform backend about hash and size of a file
        upload_aggregate_form_field "$upload_field_name.sha1" "$upload_file_sha1";
        upload_aggregate_form_field "$upload_field_name.size" "$upload_file_size";

        upload_pass_form_field "^submit$|^description$|^token$|^qqfile$|^file1$|^chunk$|^chunks$|^chunk_base64_sha_hash$|^storage_path$|^storage_unit$|^server$";

        if ($request_method = 'OPTIONS') {
           add_header Access-Control-Allow-Origin '*';
           add_header Access-Control-Allow-Methods 'POST, OPTIONS';
           add_header Access-Control-Max-Age '1728000';
           add_header Content-Type: 'text/plain; charset=UTF-8';
           add_header Access-Control-Allow-Headers 'X-CSRF-Token';
           return 200;
        }

        # FILE UPLOADS
        if ($request_method = 'POST') {
           add_header Access-Control-Allow-Origin: '*';
        }

        upload_cleanup 400 404 499 500-505;
    }

    location @test {
        proxy_pass   http://127.0.0.1:8090;
    }

.....

I followed the config in http://www.grid.net.ru/nginx/upload.en.html
and I got the error webpage: 503 Service Temporarily Unavailable

I tried passenger_enabled, which is
http://blog.joshsoftware.com/2010/10/20/uploading-multiple-files-with-nginx-upload-module-and-upload-progress-bar/

Then I get another error : The page is temporarily unavailable

I am using nginx-0.8.54, upload-module-2.2.0
Thank you so much.

The state file is updated after all request body writed into file. If file upload process is interupted by some errors, the content already uploaded and writen must be uploaded again.

Given a file size is 100MB, there are two upload strategy.

1. Upload less data(256KB...500kB) every time, clent need to issue 100MB/(segments size) times POST request.
2. Upload as many as possible data every time, it's the best to upload whole file by one POST request.

So, if server write data into temp file and record current range into state file at the same time, client could send the rest data at the next request.

How do you think about this?

I am using resumable upload and I would like to know if it is possible to upload empty (0 B) file. I don't know how to fill Content-Range HTTP header.

Is it possible and compliant with HTTP standards?

It seems that most of the aggregate form field functions do not work with upload_resumable on; like upload_aggregate_form_field crc32 $upload_file_crc32;.

Only an empty string is passed to the backend for me.

Is this a known issue or is this expected to work?

Is it possible to get the time it took for the file to upload successfully. Maybe it could be an upload_aggregate_form_field value. Or at least the time file started uploading. With that passed to the back end, the upload time could be computed within a few milliseconds.

Hi all,

When I try to upload a file then to pass the request to my cgi script, cgi fails to parse correctly the params... giving me somthing fully not usable. basically i use a fastcgi_pass. I checked on tcpdump in order to see the request sent by nginx to fcgi and here is the thing :

[...]#HTTP_ACCEPT_LANGUAGEfr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4.............(..------WebKitFormBoundaryfiThjfB0gmEGwJSS.....(..
Content-Disposition: form-data; name="........path............"

........ ../var/www/............*..
------WebKitFormBoundaryfiThjfB0gmEGwJSS...........(..
Content-Disposition: form-data; name="........test............"

...........upp..........*..
------WebKitFormBoundaryfiThjfB0gmEGwJSS...........(..
Content-Disposition: form-data; name="........submit.........."

...........Submit.........
------WebKitFormBoundaryfiThjfB0gmEGwJSS--
..........

As you see, there are many useless spaces before boundaries and in parameters, for name and value... If I disable upload_module I get :

[...]#HTTP_ACCEPT_LANGUAGEfr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4................------WebKitFormBoundaryY4Vvfs7Ju8Q2zTvi
Content-Disposition: form-data; name="path"

/var/www/
------WebKitFormBoundaryY4Vvfs7Ju8Q2zTvi
Content-Disposition: form-data; name="file"; filename="text.txt"
Content-Type: text/plain

------WebKitFormBoundaryY4Vvfs7Ju8Q2zTvi
Content-Disposition: form-data; name="test"

upp
------WebKitFormBoundaryY4Vvfs7Ju8Q2zTvi
Content-Disposition: form-data; name="submit"

Submit
------WebKitFormBoundaryY4Vvfs7Ju8Q2zTvi--
..........

Which seems usual.

I've tried with many versions of nginx and upload_module and result is the same, I'm surprised that nobody got this problem before... but I didn't find any solution. I work on ubuntu 13.04... according to the difference between with and without the use of upload_module I think problem is coming from it.

Even if the different ranges of the file are sent in ascending order.

Thanks,
Pierre-Yves

client_max_body_size seems to be inaccurated and in-efficient.

curl 192.168.191.40/upload/ -d @access.log -H "Content-Type: text/xml" -H "Content-

Disposition: attachment; filename=big.TXT" -H "X-Content-Range: bytes 105-106/511920" -H "Session-ID: 1111215056" -v

• About to connect() to 192.168.191.40 port 80
• Trying 192.168.191.40... connected
• Connected to 192.168.191.40 (192.168.191.40) port 80

  POST /upload/ HTTP/1.1
  User-Agent: curl/7.15.1 (x86_64-suse-linux) libcurl/7.15.1 OpenSSL/0.9.8a zlib/1.2.3 libidn/0.6.0
  Host: 192.168.191.40
  Accept: /
  Content-Type: text/xml
  Content-Disposition: attachment; filename=big.TXT
  X-Content-Range: bytes 105-106/511920
  Session-ID: 1111215056
  Content-Length: 11533545
  Expect: 100-continue

  HTTP/1.1 201 Created
  < Server: nginx/0.8.53
  < Date: Sat, 23 Apr 2011 08:14:54 GMT
  < Content-Length: 14
  < Connection: close
  < Range: 100-106/511920
  Closing connection #0
  100-106/511920

Here is the output from a core dump using gdb:

Core was generated by `nginx: worker process '.
Program terminated with signal 11, Segmentation fault.
#0 0x00000000004b8eb4 in ngx_http_do_read_upload_client_request_body (r=0x1fe6bc70) at nginx-upload-module/ngx_http_upload_module.c:3031

3031 if (u->limit_rate) {
(gdb) bt
#0 0x00000000004b8eb4 in ngx_http_do_read_upload_client_request_body (r=0x1fe6bc70) at nginx-upload-module/ngx_http_upload_module.c:3031
#1 0x00000000004b8af4 in ngx_http_read_upload_client_request_body_handler (r=0x1fe6bc70) at nginx-upload-module/ngx_http_upload_module.c:2976
#2 0x0000000000450b08 in ngx_http_request_handler (ev=0x1ff06ba0) at src/http/ngx_http_request.c:1867
#3 0x000000000042ad74 in ngx_event_process_posted (cycle=0x1fdc3750, posted=0x89bbc8) at src/event/ngx_event_posted.c:40
#4 0x0000000000428a39 in ngx_process_events_and_timers (cycle=0x1fdc3750) at src/event/ngx_event.c:274
#5 0x0000000000435a69 in ngx_worker_process_cycle (cycle=0x1fdc3750, data=0x0) at src/os/unix/ngx_process_cycle.c:806
#6 0x00000000004325c1 in ngx_spawn_process (cycle=0x1fdc3750, proc=0x4358bd <ngx_worker_process_cycle>, data=0x0, name=0x5dfc7e "worker process", respawn=-3) at src/os/unix/ngx_process.c:198
#7 0x000000000043496f in ngx_start_worker_processes (cycle=0x1fdc3750, n=4, type=-3) at src/os/unix/ngx_process_cycle.c:365
#8 0x0000000000434026 in ngx_master_process_cycle (cycle=0x1fdc3750) at src/os/unix/ngx_process_cycle.c:137
#9 0x0000000000404499 in main (argc=3, argv=0x7fffbce8ac88) at src/core/nginx.c:410

This is the make error:
cc -c -I /home/mimiko/src/nginx-modules/ngx_http_js_module/src/ -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I/opt/GeoIP/include -I/opt/gperftools/include -I/opt/ffmpeg/include -I/opt/ImageMagick/include/ImageMagick-6 -Wno-error=unused-but-set-variable -DMAGICKCORE_QUANTUM_DEPTH=32 -DMAGICKCORE_HDRI_ENABLE=0 -DNDK_SET_VAR -I/home/mimiko/src/nginx-modules/nginx-rtmp-module -I/home/mimiko/src/nginx-modules/ngx_http_lower_upper_case -DNDK_SET_VAR -DNDK_UPSTREAM_LIST -I/home/mimiko/src/nginx-modules/ngx_http_php_session -DNDK_SET_VAR -DNDK_SET_VAR -DNDK_SET_VAR -DNDK_SET_VAR -DNDK_SET_VAR -Wno-unused-function -Wno-missing-field-initializers -D_POSIX_C_SOURCE=200112L --std=c99 -Isrc -I src/core -I src/event -I src/event/modules -I src/os/unix -I /home/mimiko/src/nginx-modules/ngx_devel_kit/objs -I objs/addon/ndk -I /opt/libpq/include -I /opt/spidermonkey/include -I /home/mimiko/src/nginx-modules/nginx-video-thumbextractor-module/src -I /home/mimiko/src/nginx-modules/nginx-video-thumbextractor-module/include -I /opt/sregex/include -I /opt/luajit/include/luajit-2.0 -I /opt/luajit/include/luajit-2.0 -I /home/mimiko/src/nginx-modules/lua-nginx-module/src/api -I /home/mimiko/src/nginx-modules/nginx-push-stream-module/src -I /home/mimiko/src/nginx-modules/nginx-push-stream-module/include -I /home/mimiko/src/nginx-modules/nginx_http_push_module/src -I /home/mimiko/src/nginx-modules/nginx-selective-cache-purge-module/src -I /home/mimiko/src/nginx-modules/nginx-selective-cache-purge-module/include -I ../pcre -I ../openssl/.openssl/include -I ../zlib -I /usr/include/libxml2 -I objs -I src/http -I src/http/modules -I src/http/modules/perl -I /home/mimiko/src/nginx-modules/ngx_devel_kit/src -I /home/mimiko/src/nginx-modules/nginx-upload-module -I /home/mimiko/src/nginx-modules/healthcheck_nginx_upstreams -I /home/mimiko/src/nginx-modules/nginx-http-sysguard -I src/mail
-o objs/addon/nginx-upload-module/ngx_http_upload_module.o
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c: In function Б-?ngx_http_upload_merge_loc_confБ-?:
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1104:31: error: Б-?ngx_garbage_collector_temp_handlerБ-? undeclared (first use in this function)
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1104:31: note: each undeclared identifier is reported only once for each function it appears in
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1104:31: error: passing argument 1 of Б-?ngx_conf_merge_path_valueБ-? from incompatible pointer type [-Werror]
In file included from src/core/ngx_core.h:58:0,
from /home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:7:
src/core/ngx_file.h:145:7: note: expected Б-?struct ngx_conf_t _Б-? but argument is of type Б-?struct ngx_path_t *Б-?
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1104:31: error: passing argument 2 of Б-?ngx_conf_merge_path_valueБ-? from incompatible pointer type [-Werror]
In file included from src/core/ngx_core.h:58:0,
from /home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:7:
src/core/ngx_file.h:145:7: note: expected Б-?struct ngx_path_t *_Б-? but argument is of type Б-?struct ngx_path_t Б-?
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1104:31: error: passing argument 3 of Б-?ngx_conf_merge_path_valueБ-? from incompatible pointer type [-Werror]
In file included from src/core/ngx_core.h:58:0,
from /home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:7:
src/core/ngx_file.h:145:7: note: expected Б-?struct ngx_path_t *Б-? but argument is of type Б-?char *Б-?
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1104:31: error: passing argument 4 of Б-?ngx_conf_merge_path_valueБ-? makes pointer from integer without a cast [-Werror]
In file included from src/core/ngx_core.h:58:0,
from /home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:7:
src/core/ngx_file.h:145:7: note: expected Б-?struct ngx_path_init_t *Б-? but argument is of type Б-?intБ-?
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1104:31: error: too many arguments to function Б-?ngx_conf_merge_path_valueБ-?
In file included from src/core/ngx_core.h:58:0,
from /home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:7:
src/core/ngx_file.h:145:7: note: declared here
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c: In function Б-?ngx_http_upload_pass_form_fieldБ-?:
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1490:5: error: passing argument 1 of Б-?ngx_regex_compileБ-? from incompatible pointer type [-Werror]
In file included from src/core/ngx_core.h:63:0,
from /home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:7:
src/core/ngx_regex.h:50:11: note: expected Б-?struct ngx_regex_compile_t *Б-? but argument is of type Б-?struct ngx_str_t *Б-?
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1490:5: error: too many arguments to function Б-?ngx_regex_compileБ-?
In file included from src/core/ngx_core.h:63:0,
from /home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:7:
src/core/ngx_regex.h:50:11: note: declared here
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1490:14: error: assignment makes pointer from integer without a cast [-Werror]
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1497:5: error: implicit declaration of function Б-?ngx_regex_capture_countБ-? [-Werror=implicit-function-declaration]
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1501:28: error: Б-?ngx_regex_capture_count_nБ-? undeclared (first use in this function)
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1501:54: error: expected Б-?)Б-? before string constant
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c: In function Б-?ngx_http_read_upload_client_request_bodyБ-?:
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1922:7: error: Б-?ngx_http_request_body_tБ-? has no member named Б-?to_writeБ-?
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c: In function Б-?ngx_http_do_read_upload_client_request_bodyБ-?:
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1991:56: error: Б-?ngx_http_request_body_tБ-? has no member named Б-?to_writeБ-?
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:1997:19: error: Б-?ngx_http_request_body_tБ-? has no member named Б-?to_writeБ-?
/home/mimiko/src/nginx-modules/nginx-upload-module/ngx_http_upload_module.c:2063:44: error: Б-?ngx_http_request_body_tБ-? has no member named Б-?to_writeБ-?
cc1: all warnings being treated as errors
make[1]: ** [objs/addon/nginx-upload-module/ngx_http_upload_module.o] Error 1
make[1]: Leaving directory `/home/mimiko/src/nginx'
make: *** [build] Error 2

I'm on Debian Wheezy x86_64. Nginx and module latest from git.

cc -c -I src/core -I src/event -I src/event/modules -I src/os/unix -I objs -I src/http -I src/http/modules -I /Users/batuhangoksu/Desktop/nginx-upload-module-master -I src/mail
-o objs/addon/nginx-upload-module-master/ngx_http_upload_module.o
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload_module.c
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload_module.c:706:9: warning:
'MD5_Init' is deprecated: first deprecated in OS X 10.7 [-Wdeprecated-declarations]
MD5Init(&u->md5_ctx->md5);
^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload.h:15:21: note: expanded from
macro 'MD5Init'

define MD5Init MD5_Init

                ^

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/openssl/md5.h:113:5: note:
'MD5_Init' declared here
int MD5_Init(MD5_CTX *c) DEPRECATED_IN_MAC_OS_X_VERSION_10_7_AND_LATER;
^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload_module.c:709:9: warning:
'SHA1_Init' is deprecated: first deprecated in OS X 10.7 [-Wdeprecated-declarations]
SHA1_Init(&u->sha1_ctx->sha1);
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/openssl/sha.h:121:5: note:
'SHA1_Init' declared here
int SHA1_Init(SHA_CTX *c) DEPRECATED_IN_MAC_OS_X_VERSION_10_7_AND_LATER;
^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload_module.c:750:13: warning:
'MD5_Final' is deprecated: first deprecated in OS X 10.7 [-Wdeprecated-declarations]
MD5Final(u->md5_ctx->md5_digest, &u->md5_ctx->md5);
^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload.h:17:21: note: expanded from
macro 'MD5Final'

define MD5Final MD5_Final

                ^

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/openssl/md5.h:115:5: note:
'MD5_Final' declared here
int MD5_Final(unsigned char *md, MD5_CTX *c) DEPRECATED_IN_MAC_OS_X_VERSION_10_7_AND_LATER;
^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload_module.c:753:13: warning:
'SHA1_Final' is deprecated: first deprecated in OS X 10.7 [-Wdeprecated-declarations]
SHA1_Final(u->sha1_ctx->sha1_digest, &u->sha1_ctx->sha1);
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/openssl/sha.h:123:5: note:
'SHA1_Final' declared here
int SHA1_Final(unsigned char *md, SHA_CTX *c) DEPRECATED_IN_MAC_OS_X_VERSION_10_7_AND_LATER;
^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload_module.c:850:13: warning:
'MD5_Update' is deprecated: first deprecated in OS X 10.7 [-Wdeprecated-declarations]
MD5Update(&u->md5_ctx->md5, cl->buf->pos, cl->buf->last - cl->buf->pos);
^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload.h:16:21: note: expanded from
macro 'MD5Update'

define MD5Update MD5_Update

                ^

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/openssl/md5.h:114:5: note:
'MD5_Update' declared here
int MD5_Update(MD5_CTX c, const void *data, size_t len) DEPRECATED_IN_MAC_OS_X_VERSION_10_7...
^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload_module.c:853:13: warning:
'SHA1_Update' is deprecated: first deprecated in OS X 10.7 [-Wdeprecated-declarations]
SHA1_Update(&u->sha1_ctx->sha1, cl->buf->pos, cl->buf->last - cl->buf->pos);
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/openssl/sha.h:122:5: note:
'SHA1_Update' declared here
int SHA1_Update(SHA_CTX *c, const void *data, size_t len) DEPRECATED_IN_MAC_OS_X_VERSION_10...
^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload_module.c:1104:31: error: use
of undeclared identifier 'ngx_garbage_collector_temp_handler'
ngx_garbage_collector_temp_handler, cf);
^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload_module.c:1922:9: error: no
member named 'to_write' in 'ngx_http_request_body_t'
rb->to_write = rb->bufs;
~~ ^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload_module.c:1991:58: error: no
member named 'to_write' in 'ngx_http_request_body_t'
ngx_http_process_request_body(r, rb->to_write));
~~ ^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload_module.c:1997:21: error: no
member named 'to_write' in 'ngx_http_request_body_t'
rb->to_write = rb->bufs->next ? rb->bufs->next : rb->bufs;
~~ ^
/Users/batuhangoksu/Desktop/nginx-upload-module-master/ngx_http_upload_module.c:2063:46: error: no
member named 'to_write' in 'ngx_http_request_body_t'
ngx_http_process_request_body(r, rb->to_write));
~~ ^
6 warnings and 5 errors generated.
make[1]: *_* [objs/addon/nginx-upload-module-master/ngx_http_upload_module.o] Error 1
make: *** [build] Error 2

I have an issue which occurs frequently, but I can't seem to replicate it consistently. When uploading a batch of files, occasionally I end up with an extra copy or two of the file in the upload directory. For example, if I upload ten files in a row, maybe one or two of them will leave an extra copy or two that doesn't get removed by processing.

Let's say one of those files uploaded is called xxx.pdf. I will end up with something like 000000222,000000223 in my upload directory after all the uploading and processing. Each of those files are identical to the uploaded file (xxx.pdf). The successfully processed version of the file is 000000224. So in this case, three copies of the uploaded file were put in the upload directory, the first two did not get processed and the third one did. The other nine files in that ten file group processed without any issues.

My processing script (doc_upload.php) simply checks for file validity, sets up a processing job in a job queue to have processing done asynchronously from a different script controlled by a daemon (not nginx) and then terminates. The moving of the uploaded file from the upload directory is not done by the doc_upload.php script, but the separate asynchronous daemon. I don't appear to have an issue of where a file is uploaded and does not get processed at all.

doc_upload.php (launched by upload plugin to process file) --> sets up a processing job and places it a queue for processing by a daemon (does not remove file from upload directory)

daemon --> reads the job queue, processes the uploaded file, removes it from the upload directory

This is the only thing I see that seems to apply from the nginx error.log (I get these occasionally)

2012/03/08 13:27:52 [alert] 12907#0: *92047 aborted uploading file "xxx.pdf" to "/usr/local/nginx/uploads/0000000181", dest file removed, client: 0.0.0.0, server: www.xxxx.com, request: "POST /xxxx HTTP/1.1", host: "www.xxxx.com"

I am running nginx 1.1.11 with the upload being done with SSL. I am using the 2.2 version of the plugin. Here is my conf script section involving the uploading.

client_max_body_size 50m; # maximum file upload size

upload module specific parameters

upload_pass /doc_upload.php;
upload_store /usr/local/nginx/html/uploads;
upload_pass_args on;
upload_limit_rate 1m; # limit the upload to

Set specified fields in request body

upload_set_form_field $upload_field_name.name "$upload_file_name";
upload_set_form_field $upload_field_name.content_type "$upload_content_type";
upload_set_form_field $upload_field_name.path "$upload_tmp_path";
upload_aggregate_form_field "$upload_field_name.size" "$upload_file_size";
upload_aggregate_form_field "$upload_field_name.number" "$upload_file_number";
upload_aggregate_form_field "$upload_field_name.external" "$1";
upload_aggregate_form_field "$upload_field_name.args" "$args";
upload_pass_form_field "^name$"; # this if for the unique_name field in routing
upload_cleanup 400 404 499 500-505;

Any suggestions?

I am dealing with a standalone scanner that uploads scans over HTTP/HTTPS to a IP Address or URL

It starts its request with a directory request through POST variables (to make sure that there aren't any files in the destination directory with the same name) and then POSTs the file (using multipart-encoding) after that request (to the same cgi/url).

In its request header on the first request, it is sending an "Expect: 100 - Continue".

Does this module currently support this sequence of operations?. If not, will it be able to do it soon or ever?

# curl localhost:8081/upload/ -d a -H "Content-Type: text/xml" -H "Content-Disposition: attachment; filename=big.TXT" -H "X-Content-Range: bytes 1-1/111414930" -H "Session-ID: 1111215057" -v
* About to connect() to localhost port 8081 (#0)
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8081 (#0)
> POST /upload/ HTTP/1.1
> User-Agent: curl/7.21.3 (i686-pc-linux-gnu) libcurl/7.21.3 zlib/1.2.3
> Host: localhost:8081
> Accept: */*
> Content-Type: text/xml
> Content-Disposition: attachment; filename=big.TXT
> X-Content-Range: bytes 1-1/111414930
> Session-ID: 1111215057
> Content-Length: 1
>
* Empty reply from server
* Connection #0 to host localhost left intact
curl: (52) Empty reply from server
* Closing connection #0

Is this a bug?

Would it be possible, that the upload module would respond only to certain http verbs like PUT, POST and pass the data on unsupported verbs to a custom backend.

Our API is RESTfull, and because of that we'd like to have the same uri for download/upload, but with different http verb. We looked at the code a bit and it seems that its not possible to do so because upload module is registered as content handler. Adding try_files into the same section doesn't work because try_files takes precedence.

День добрый!

Прошу прощения за мой русский, но на английском как-то сложновато все это описать.

На нашем сервере с nginx 1.2.5 стоит Ваш модуль версии 2.2.0.
На этот сервер идет upload с приложения (php с помощью php-curl). Все хорошо, если мы шлем из скрипта файлы в одном запросе, но как только мы включаем отложенную загрузку (вместо одного запроса к upload-серверу происходит уже два отдельных, из одного скрипта, с одного ip), на графиках начинает постоянно расти Waiting connections (см.скриншот ниже).

(спад на графике - перезапуск nginx)

Скрипт для воспроизведения:

<?php

$handle = curl_init($url);
curl_setopt_array($handle, array(
    CURLOPT_POST => true,
    CURLOPT_POSTFIELDS => array(
         'file1' => '@somefile' 
    )
));
curl_exec($handle);
curl_close($handle);

echo 'something';
fastcgi_finish_request();   // <- не причина, без этой строки тот же результат

$handle = curl_init($url);
curl_setopt_array($handle, array(
    CURLOPT_POST => true
    CURLOPT_POSTFIELDS => array(
         'file2' => '@somefile2',
         'file3' => '@somefile3',
         'file4' => '@somefile4',
    )
));
curl_exec($handle);
curl_close($handle);

Есть какие-нибудь предложения по поводу того, почему это можно происходить?

Видели в интернетах такую же проблему в ранних версиях и патч, который это дело исправляет, но в 2.2.0 эти изменения тоже есть. Пробовали отключить keep-alive (ставили keepalive_timeout 0) - тоже не помогло.

nginx has for sometime supported defining variables with set directive, this is really handy as you don't have to write the absolute path throughout the config file.

I have defined page_root variable in my server section e.g set $page_root /srv/www/......;
This variable expands just fine throughout of config, but the upload_store doesn't seem to recognize it.

gcc -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Wunused-function -Wunused-variable -Wunused-value -Werror -g   -I src/core -I src/event -I src/event/modules -I src/os/unix -I /usr/include/libxml2 -I objs -I src/http -I src/http/modules -I src/http/modules/perl -I ./nginx-upload-module -I src/mail \
        -o objs/addon/nginx-upload-module/ngx_http_upload_module.o \
        ./nginx-upload-module/ngx_http_upload_module.c
./nginx-upload-module/ngx_http_upload_module.c: In function ‘ngx_http_upload_merge_loc_conf’:
./nginx-upload-module/ngx_http_upload_module.c:1104: error: ‘ngx_garbage_collector_temp_handler’ undeclared (first use in this function)
./nginx-upload-module/ngx_http_upload_module.c:1104: error: (Each undeclared identifier is reported only once
./nginx-upload-module/ngx_http_upload_module.c:1104: error: for each function it appears in.)
cc1: warnings being treated as errors
./nginx-upload-module/ngx_http_upload_module.c:1104: error: passing argument 1 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
src/core/ngx_file.h:140: note: expected ‘struct ngx_conf_t *’ but argument is of type ‘struct ngx_path_t *’
./nginx-upload-module/ngx_http_upload_module.c:1104: error: passing argument 2 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
src/core/ngx_file.h:140: note: expected ‘struct ngx_path_t **’ but argument is of type ‘struct ngx_path_t *’
./nginx-upload-module/ngx_http_upload_module.c:1104: error: passing argument 3 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
src/core/ngx_file.h:140: note: expected ‘struct ngx_path_t *’ but argument is of type ‘char *’
./nginx-upload-module/ngx_http_upload_module.c:1104: error: passing argument 4 of ‘ngx_conf_merge_path_value’ makes pointer from integer without a cast
src/core/ngx_file.h:140: note: expected ‘struct ngx_path_init_t *’ but argument is of type ‘int’
./nginx-upload-module/ngx_http_upload_module.c:1104: error: too many arguments to function ‘ngx_conf_merge_path_value’
./nginx-upload-module/ngx_http_upload_module.c: In function ‘ngx_http_upload_pass_form_field’:
./nginx-upload-module/ngx_http_upload_module.c:1490: error: passing argument 1 of ‘ngx_regex_compile’ from incompatible pointer type
src/core/ngx_regex.h:45: note: expected ‘struct ngx_regex_compile_t *’ but argument is of type ‘struct ngx_str_t *’
./nginx-upload-module/ngx_http_upload_module.c:1490: error: too many arguments to function ‘ngx_regex_compile’
./nginx-upload-module/ngx_http_upload_module.c:1490: error: assignment makes pointer from integer without a cast
./nginx-upload-module/ngx_http_upload_module.c:1497: error: implicit declaration of function ‘ngx_regex_capture_count’
./nginx-upload-module/ngx_http_upload_module.c:1501: error: ‘ngx_regex_capture_count_n’ undeclared (first use in this function)
./nginx-upload-module/ngx_http_upload_module.c:1501: error: expected ‘)’ before string constant
make[1]: *** [objs/addon/nginx-upload-module/ngx_http_upload_module.o] Error 1
make[1]: Leaving directory `/root/dist/nginx-0.8.52'

The scenario:

1. The last chunk of resumable upload is sent to server.
2. Full range is done and status file is deleted (this is cause of problem!)
3. Call to upload_pass back-end is performed.
4. Back-end fails temporally with negative error code, for example 502.
5. Client retries last chunk.
6. The upload module returns "201 Created" for last chunk. But?! it should call back-end again and give something other than 201, because whole file is uploaded already. However response content informs that server has got only the last chunk of uploaded file - the one uploaded by the attempt in step 5.

So the problem is removal of state file in step 2.
That should be done once dispatched action returns any positive 2xx/3xx status code (step 4). Eventually status file path could be sent to back-end as request param then back-end would take care of that file same way as uploaded content file.

All URLs on http://www.grid.net.ru/ currently return an Nginx 404 Not Found page.

If I post an invalid data, like content-length less than content-Range, this module will pass to backend rather than return a 41* error.

test case: curl localhost:8081/upload/ -d abc -H "Content-Type: text/xml" -H "Content-Dispos
ition: attachment; filename=big.TXT" -H "X-Content-Range: bytes 6-10/511920" -H "Session-ID: 1111215056"

logs:
2011/04/08 15:26:57 [debug] 5216#0: 19 write: 18, 080E7F46, 3, 6
2011/04/08 15:26:57 [error] 5216#0: *19 file offset at the end of a part 9 does not match the end specified range 6-10
/511920, client: 127.0.0.1, server: localhost, request: "POST /upload/ HTTP/1.1", host: "localhost:8081"
2011/04/08 15:26:57 [debug] 5216#0: *19 test location: "@test"
2011/04/08 15:26:57 [debug] 5216#0: *19 using location: @test "/upload/?"
2011/04/08 15:26:57 [debug] 5216#0: *19 rewrite phase: 2
2011/04/08 15:26:57 [debug] 5216#0: *19 post rewrite phase: 3
2011/04/08 15:26:57 [debug] 5216#0: *19 generic phase: 4
2011/04/08 15:26:57 [debug] 5216#0: *19 generic phase: 5
2011/04/08 15:26:57 [debug] 5216#0: *19 generic phase: 6
2011/04/08 15:26:57 [debug] 5216#0: *19 http init upstream, client timer: 0
2011/04/08 15:26:57 [debug] 5216#0: *19 epoll add event: fd:17 op:3 ev:80000005
2011/04/08 15:26:57 [debug] 5216#0: *19 http script copy: "Host: "
2011/04/08 15:26:57 [debug] 5216#0: *19 http script var: "10.6.2.51"
2011/04/08 15:26:57 [debug] 5216#0: *19 http script copy: "^M
"
2011/04/08 15:26:57 [debug] 5216#0: *19 http script copy: "Connection: close^M
"
2011/04/08 15:26:57 [debug] 5216#0: *19 http proxy header: "User-Agent: curl/7.21.3 (i686-pc-linux-gnu) libcurl/7.21.3
zlib/1.2.3"
2011/04/08 15:26:57 [debug] 5216#0: *19 http proxy header: "Accept: */"
2011/04/08 15:26:57 [debug] 5216#0: *19 http proxy header: "Content-Type: multipart/form-data; boundary=00000123477"
2011/04/08 15:26:57 [debug] 5216#0: *19 http proxy header: "Content-Disposition: attachment; filename=big.TXT"
2011/04/08 15:26:57 [debug] 5216#0: *19 http proxy header: "X-Content-Range: bytes 6-10/511920"
2011/04/08 15:26:57 [debug] 5216#0: *19 http proxy header: "Session-ID: 1111215056"
2011/04/08 15:26:57 [debug] 5216#0: *19 http proxy header: "Content-Length: 19"
2011/04/08 15:26:57 [debug] 5216#0: *19 http proxy header:

I'd like to be able to defined in the from what upload_pass will be.

ie

<form ...>
 <input type='hidden' name='upload_pass' value='/admin/form'>
</form>

<form ..>
  <input type='hidden' name='upload_pass' value='/admin/productimage'>
</form>

So that I don't have to configure multiple locations for each upload form. Just one.

location /admin/fileupload {
    upload_pass $upload_pass;

    upload_set_form_field filename "$upload_file_name";
    upload_set_form_field filecontenttype "$upload_content_type";
    upload_set_form_field filepath "$upload_tmp_path";

    #form & product field
    upload_pass_form_field "submit";
    upload_pass_form_field "Description";
    upload_pass_form_field "Name";
    upload_pass_form_field "Id";

    #product fields
    upload_pass_form_field "Price";
    upload_pass_form_field "defaultImage";
  }

Is it possible to pass an array to the upload_pass_form_field
see you have

second file ...
<input type='text' name='colors[]>

Thank you for the great work already

For instance, using a simple curl script would produce the following (one of a few 256KB parts):

Connected to localhost (127.0.0.1) port 80 (#0)
POST /rupload?seal=ee0e115a9c254797aa9f17af3b95e5c4&uuid=8d866e5c63fcc7f73da7f378f3fc776f HTTP/1.1
Host: localhost
Accept: /
Content-Disposition: attachment; filename="infile.flv"
Content-Type: application/octet-stream
X-Content-Range: bytes 1310720-1572863/1800417
X-Session-ID: 8d866e5c63fcc7f73da7f378f3fc776f
Content-Length: 262144

HTTP/1.1 201 Created
Date: Mon, 18 Apr 2011 08:27:11 GMT
Content-Length: 17
Connection: keep-alive
Range: 0-1572863/1800417

Send failure: Connection reset by peer
Closing connection #0

Also, some strange messages appear in the Nginx error log:

26 2011/04/18 00:22:30 [info] 13581#0: *6 started uploading part 1310720-1572863/1800417 of file "infile.flv" to "/data/xupload/incoming/64c02dc28705ed16fed72cb6d72ab643" (field "", content type "application/octet-stream"), client: 127.0.0.1, server: , request: "POST /rupload?seal=0 7f500299b2ccb2afb42709b40df8361&uuid=64c02dc28705ed16fed72cb6d72ab643 HTTP/1.1", host: "localhost"
27 2011/04/18 00:22:30 [info] 13581#0: *6 finished uploading part 1310720-1572863/1800417 of a file "infile.flv" to "/data/xupload/incoming/64c02dc28705ed16fed72cb6d72ab643", client: 127.0.0.1, server: , request: "POST /rupload?seal=07f500299b2ccb2afb42709b40df8361&uuid=64c02dc28705 ed16fed72cb6d72ab643 HTTP/1.1", host: "localhost"
28 2011/04/18 00:22:30 [info] 13581#0: *6 client sent invalid method while reading client request line, client: 127.0.0.1, server: , request: "©`ê íìRÖ0É<94>îK¢ú^_»?òX^@<92><9b><82>A£<94><81>(Y!˦I^T"

As if the request context was corrupted somehow. This doesn't happen with non-resumable (standard multipart/form-data) uploads.

Thanks,
Pierre-Yves

The "upload_state_store" configuration directive does not seem to have any effect when used in the configuration file.
A quick look at the module's code shows that the state_store_path context variable is not used to build the state file path.

Thanks,
Pierre-Yves

gcc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Wunused-function -Wunused-variable -Wunused-value -Werror -g -I src/core -I src/event -I src/event/modules -I src/os/unix -I objs -I src/http -I src/http/modules -I contrib/nginx-upload-module -I src/mail
-o objs/addon/nginx-upload-module/ngx_http_upload_module.o
contrib/nginx-upload-module/ngx_http_upload_module.c
contrib/nginx-upload-module/ngx_http_upload_module.c: In function ‘ngx_http_upload_merge_loc_conf’:
contrib/nginx-upload-module/ngx_http_upload_module.c:1104: error: ‘ngx_garbage_collector_temp_handler’ undeclared (first use in this function)
contrib/nginx-upload-module/ngx_http_upload_module.c:1104: error: (Each undeclared identifier is reported only once
contrib/nginx-upload-module/ngx_http_upload_module.c:1104: error: for each function it appears in.)
cc1: warnings being treated as errors
contrib/nginx-upload-module/ngx_http_upload_module.c:1104: warning: passing argument 1 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
contrib/nginx-upload-module/ngx_http_upload_module.c:1104: warning: passing argument 2 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
contrib/nginx-upload-module/ngx_http_upload_module.c:1104: warning: passing argument 3 of ‘ngx_conf_merge_path_value’ from incompatible pointer type
contrib/nginx-upload-module/ngx_http_upload_module.c:1104: warning: passing argument 4 of ‘ngx_conf_merge_path_value’ makes pointer from integer without a cast
contrib/nginx-upload-module/ngx_http_upload_module.c:1104: error: too many arguments to function ‘ngx_conf_merge_path_value’
contrib/nginx-upload-module/ngx_http_upload_module.c: In function ‘ngx_http_upload_pass_form_field’:
contrib/nginx-upload-module/ngx_http_upload_module.c:1490: warning: passing argument 1 of ‘ngx_regex_compile’ from incompatible pointer type
contrib/nginx-upload-module/ngx_http_upload_module.c:1490: error: too many arguments to function ‘ngx_regex_compile’
contrib/nginx-upload-module/ngx_http_upload_module.c:1490: warning: assignment makes pointer from integer without a cast
contrib/nginx-upload-module/ngx_http_upload_module.c:1497: warning: implicit declaration of function ‘ngx_regex_capture_count’
contrib/nginx-upload-module/ngx_http_upload_module.c:1501: error: ‘ngx_regex_capture_count_n’ undeclared (first use in this function)
contrib/nginx-upload-module/ngx_http_upload_module.c:1501: error: expected ‘)’ before string constant
make[1]: *** [objs/addon/nginx-upload-module/ngx_http_upload_module.o] Error 1
make: *** [install] Error 2

Please fix this issue as it seems that nginx Makefile specified -Werror