Git Product home page Git Product logo

laravel-chunk-upload-example's Introduction

Laravel Chunked Upload Example

This example covers all supported versions of Laravel. This implementation allows testing across all versions simultaneously.

This repository contains a full example for the laravel-chunk-upload package. You can either explore an example app or run tests against all supported versions of Laravel.

Install

git clone [email protected]:pionl/laravel-chunk-upload-example.git --recurse-submodules
npm install

Requirements

  • PHP Installed if running locally
  • Node.js
  • Docker and Docker Compose installed

Usage

To install and run a specific version of Laravel, use the X.* version format.

node run.js "8.*"

Pass --ignore-example to ignore updating and building the example.

Pass --verbose for debug information.

Open http://localhost:8000

Compatibility

The current version of laravel-chunk-upload is tested against:

Laravel PHP Run Tests
10.* 8.2-node-20, 8.2-node-20 node run "11.*" node tests.js "11.*"
10.* 8.1-node-17, 8.2-node-18 node run "10.*" node tests.js "10.*"
9.* 8.0-node-17, 8.1-node-17, 8.2-node-18 node run "9.*" node tests.js "9.*"
8.* 7.4-node-17, 8.0-node-17, 8.1-node-17 node run "8.*" node tests.js "8.*"
7.* 7.4-node-17 node run "7.*" node tests.js "7.*"

Tests

Pass --verbose for debug information.

Run tests on all Laravel versions (install them before using it) using docker

node tests.js

Run tests on a desired Laravel version

node tests.js "8.*"

Running Tests Locally

  • In one terminal, run php 7.\*/artisan serve --host=0.0.0.0 --port=8000.
  • In a second terminal, run tests with ./node_modules/.bin/codeceptjs run --steps.
  • npm run test-debug will show the browser while tests are running.

Docker-Compose

  • Environment variables should be passed: IMAGE_VERSION (PHP version) and LARAVEL_VERSION.
  • abort-on-container-exit needs to be used to stop the Laravel server.
IMAGE_VERSION=7.4 LARAVEL_VERSION=8.\* docker-compose -f docker-compose.yml -f docker-compose-tests.yml up --abort-on-container-exit

Testing Your Contribution

Do not commit your changes - use a pull request in the main repo.

  • Replace the laravel-chunk-upload folder with your repository (clone your fork).
  • Or you can pull changes from your fork with with given bash script:
pr.sh <your-github-username> <branch = default is master> <repository-name = laravel-chunk-upload>
  • Run tests on all versions with node tests.js to ensure backward compatibility.

Adding Support for a New Laravel Release

  • Docker and Docker Compose installed.
  • Node.js installed.

Do not commit your changes - use a pull request in the main repo.

  • Add a new version to the versions.json file.
  • Add your changes to the laravel-chunk-upload folder.
  • Test the Laravel version with node tests.js "10.\*".
  • Run tests on all versions with node tests.js to ensure backward compatibility.

Example Code

Uploading to Amazon S3 (or Any Other Cloud Storage)

  • It is recommended to upload the file in a separate queue JOB. Uploading can take time (for large files), and the request could get timed out.
  • It is important to stream the file to the cloud (not using file_get_contents), which would consume your memory.
  • Also, don't forget to remove the uploaded file after the upload.

Example code found at ./example/src/Http/Controllers/UploadController.php.

Contribution

  • If you make changes in assets, do not forget to call node compile.js to compile the changes and publish them to all versions (you can also pass the desired version).

TODO

  • Improve example controller.
  • Add tests to Dropzone.
  • Clean storage after tests.
  • Add tests for parallel save.

Running Tests on the Latest Laravel Release

I did not find a way to install Laravel on master with the latest changes from the framework. Let me know if you know how.

Copyright and License

laravel-chunk-upload-example was written by Martin Kluska and is released under the MIT License.

Copyright (c) 2017 and beyond Martin Kluska and all the contributors (Thank you ❤️)

laravel-chunk-upload-example's People

Contributors

dependabot[bot] avatar pionl avatar r0aringthunder avatar seki98 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

laravel-chunk-upload-example's Issues

How to merge PART chuks into single file

Hello Everyone,
I'm using resumable.js with laravel-chunk-upload-example, App front-end is in NuxtJS I configure the resumable successfully and use the example code of laravel-chunk things are working fine but on server side part chunks are not merging into single file should I need to do some extra setup for this.
Please help me
Thanks

Success Event Response Text Empty...

Hey,

Whenever I hook into the success event, the response is always empty.

this.$refs.sectionDropzone[0].dropzone.on('success', (file, responseText) => {
    console.log(file, responseText);

    if (responseText) {
        this.getTicket()
            .then(ticket => {
                this.ticket = ticket;

                this.setProgressMeter();
                this.checkProgress(file);
                this.uploadVideo(file, responseText);
            });
    }
});

Here's my PHP code:

<?php

namespace App\Http\Controllers\API;

use Storage;
use App\Models\Video;
use Illuminate\Http\Request;
use Illuminate\Http\UploadedFile;
use App\Http\Controllers\Controller;
use App\Models\CourseChapterSection as Section;
use Pion\Laravel\ChunkUpload\Exceptions\UploadMissingFileException;
use Pion\Laravel\ChunkUpload\Handler\AbstractHandler;
use Pion\Laravel\ChunkUpload\Handler\HandlerFactory;
use Pion\Laravel\ChunkUpload\Receiver\FileReceiver;

class SectionVideoController extends Controller
{
    protected $section;
    protected $video;

    public function __construct(Section $section, Video $video)
    {
        $this->section = $section;
        $this->video = $video;
    }

    public function index()
    {
        $section = $this->section->find(request()->section_id);

        return $section->videos;
    }

    /**
     * Handles the file upload
     *
     * @param Request $request
     *
     * @return \Illuminate\Http\JsonResponse
     *
     * @throws UploadMissingFileException
     * @throws \Pion\Laravel\ChunkUpload\Exceptions\UploadFailedException
     */
    public function store(Request $request) {
        // create the file receiver
        $receiver = new FileReceiver('video', $request, HandlerFactory::classFromRequest($request));

        // check if the upload is success, throw exception or return response you need
        if ($receiver->isUploaded() === false) {
            throw new UploadMissingFileException();
        }

        // receive the file
        $save = $receiver->receive();

        // check if the upload has finished (in chunk mode it will send smaller files)
        if ($save->isFinished()) {
            $file = $save->getFile();

            // save the file and return any response you need, current example uses `move` function. If you are
            // not using move, you need to manually delete the file by unlink($save->getFile()->getPathname())
            $filename = str_slug(basename(time(), '.'.$file->getClientOriginalExtension())).'.'.$file->getClientOriginalExtension();;
        
            // move the file name
            $file->move(storage_path().'/app/media/video/', $filename);

            return storage_path().'/app/media/video/'.$filename;
        }

        // we are in chunk mode, lets send the current progress
        /** @var AbstractHandler $handler */
        $handler = $save->handler();

        return response()->json([
            'done' => $handler->getPercentageDone(),
            'status' => true
        ]);
    }

    public function destroy($id)
    {
        $video = $this->video->find($id);
        $video->sections()->detach(request()->section_id);
        $video->delete();
    }
}

No matter what I do, I cannot get the response after successfully uploading all chunks.

The success event is fired and a can log the file paramter, but the responseText argument is always empty.

Any ideas?

Add sample using resumable.js

I have problem when tried to use resumable.js, I tried using default UploadController.php with jquery-file-upload and dropzone and it works fine but not with resumable.js it always return (in Chrome console):
The page has expired due to inactivity.
Please refresh and try again.

Here is my view code (script section, other are UI related):

@section('body-end')
    <script src="{{ asset('vendor/resumable/resumable.js') }}"></script>
    <script>
        var r = new Resumable({
            target:'{{ url("upload") }}',
            chunkSize:5*1024*1024,
            simultaneousUploads:4,
            testChunks:false,
            throttleProgressCallbacks:1
          });
        // Resumable.js isn't supported, fall back on a different method
        if(!r.support) {
          $('.resumable-error').show();
        } else {
          // Show a place for dropping/selecting files
          $('.resumable-drop').show();
          r.assignDrop($('.resumable-drop')[0]);
          r.assignBrowse($('.resumable-browse')[0]);

          // Handle file add event
          r.on('fileAdded', function(file){
              // Show progress pabr
              $('.resumable-progress, .resumable-list').show();
              // Show pause, hide resume
              $('.resumable-progress .progress-resume-link').hide();
              $('.resumable-progress .progress-pause-link').show();
              // Add the file to the list
              $('.resumable-list').append('<li class="resumable-file-'+file.uniqueIdentifier+'">Uploading <span class="resumable-file-name"></span> <span class="resumable-file-progress"></span>');
              $('.resumable-file-'+file.uniqueIdentifier+' .resumable-file-name').html(file.fileName);
              // Actually start the upload
              r.upload();
            });
          r.on('pause', function(){
              // Show resume, hide pause
              $('.resumable-progress .progress-resume-link').show();
              $('.resumable-progress .progress-pause-link').hide();
            });
          r.on('complete', function(){
              // Hide pause/resume when the upload has completed
              $('.resumable-progress .progress-resume-link, .resumable-progress .progress-pause-link').hide();
            });
          r.on('fileSuccess', function(file,message){
              // Reflect that the file upload has completed
              $('.resumable-file-'+file.uniqueIdentifier+' .resumable-file-progress').html('(completed)');
            });
          r.on('fileError', function(file, message){
              // Reflect that the file upload has resulted in error
              $('.resumable-file-'+file.uniqueIdentifier+' .resumable-file-progress').html('(file could not be uploaded: '+message+')');
            });
          r.on('fileProgress', function(file){
              // Handle progress for both the file and the overall upload
              $('.resumable-file-'+file.uniqueIdentifier+' .resumable-file-progress').html(Math.floor(file.progress()*100) + '%');
              $('.progress-bar').css({width:Math.floor(r.progress()*100) + '%'});
            });
          r.on('cancel', function(){
            $('.resumable-file-progress').html('canceled');
          });
          r.on('uploadStart', function(){
              // Show pause, hide resume
              $('.resumable-progress .progress-resume-link').hide();
              $('.resumable-progress .progress-pause-link').show();
          });
        }
      </script>
@endsection

Install error on Xampp server (windows 10)

Problem 1
- Installation request for sebastian/environment 3.1.x-dev -> satisfiable by sebastian/environment[3.1.x-dev].
- sebastian/environment 3.1.x-dev requires ext-posix * -> the requested PHP extension posix is missing from your system.
Problem 2
- Installation request for sebastian/environment dev-master -> satisfiable by sebastian/environment[dev-master].
- sebastian/environment dev-master requires ext-posix * -> the requested PHP extension posix is missing from your system.
Problem 3
- sebastian/environment 3.1.x-dev requires ext-posix * -> the requested PHP extension posix is missing from your system.
- phpunit/phpunit 6.5.x-dev requires sebastian/environment ^3.1 -> satisfiable by sebastian/environment[3.1.x-dev].
- Installation request for phpunit/phpunit 6.5.x-dev -> satisfiable by phpunit/phpunit[6.5.x-dev].

To enable extensions, verify that they are enabled in your .ini files:
- F:\xampp\php\php.ini
You can also run php --ini inside terminal to see which files are used by PHP in CLI mode.

errors when running npm dev

Hi I was just trying to run it again for some testing purposes and realized some errors!

fallbackLoader option has been deprecated - replace with "fallback"
loader option has been deprecated - replace with "use"
Invalid configuration object. Webpack has been initialised using a configuration object that does not match the API schema.
 - configuration.output.path: The provided value "public" is not an absolute path!

npm ERR! Darwin 16.5.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "dev"
npm ERR! node v7.7.1
npm ERR! npm  v4.1.2
npm ERR! code ELIFECYCLE
npm ERR! @ dev: `node node_modules/cross-env/dist/bin/cross-env.js NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @ dev script 'node node_modules/cross-env/dist/bin/cross-env.js NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the  package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node node_modules/cross-env/dist/bin/cross-env.js NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/chrisbreuer/code/test/npm-debug.log

Remove "minimum-stability" from composer config

In the config file of composer the "minimum-stability": "dev", line causes the following issues:

  • For some reason it makes composer to downloads the developer version all of the packagis. (Not just pionl/laravel-chunk-upload)
  • Composer shows no matching package found for ramsey/uuid.

Once that line is removed everithing works as expected.

Incorrect example?

How are all demo's supposed to work with chunking, if only 1 is setup to do chunking?
Since:
-Dropzone
-Resumable

Only link to the original files, without any aditional configuration for chunking, so how is this "chunking" example going to do chunking, without any configuration?

Or am I missing something?

Remove merged chunk file

Hi
I have a merged chunk file : NAMEVIDEO-oKqxtMjDY6c3Q63PrHzQjvzBmWKhSZPjflAjk4nn.part in storage/app/chunks folder.
I want to remove it after upload on Cloud server.
How can i do that. Thanks so much

The request is missing a file

I am using dropzone to upload file to AWS, and i am having this 'The request is missing a file.' error.

` $aws=Storage::disk('s3');

        $receiver = new FileReceiver("media", $request, HandlerFactory::classFromRequest($request));
        if ($receiver->isUploaded() ) {
            // receive the file
            $save = $receiver->receive();
            // check if the upload has finished (in chunk mode it will send smaller files)
            if ($save->isFinished()  === true) {
                // save the file and return any response you need
                return $aws->put('test.mp3',$save->getFile(),'public');
            } else {
                // we are in chunk mode, lets send the current progress
                /** @var AbstractHandler $handler */
                $handler = $save->handler();
                return response()->json([
                    "done" => $handler->getPercentageDone(),
                ]);
            }
        } else {
            throw new UploadMissingFileException();
        }`

and this is my dropzone code, here i am using angular 4

this.dropzoneConfig ={ url : environment.apiurl+'/project/driveupload/?section=1&projectid='+this.projectId, acceptedFiles:"audio/*", autoReset:1, chunking: true, paramName: 'media', chunkSize:4000000 };

why i am getting this error? what is i am doing wrong?

chunks folder can't merge into 1 file

hello I was trying to upload the big files about (5M)
and i chunked into 2M.

when I looked on my upload folder, I only got 1 chunked file(it isn't merged file)
does the folder structure has an effect on it?

my structure folder ( the directory is storage/app/)

  1. testfolders (i put tmp,delete folder here)
    testfolders/tmp -> temporary upload
    testfolders/delete -> if user delete the file
  2. chunks folders

and the extension is only allow .zip

source is the same as an example code one.

thank you very much

Errors when running 'npm run dev'

I tried cloning this project, and after composer install and npm install running successfully I get the error message when I try to run npm run dev.

Am I missing anything?

chunk [master] npm run dev

> @ dev /Users/chrisbreuer/code/chunk
> node node_modules/cross-env/bin/cross-env.js NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js

module.js:472
    throw err;
    ^

Error: Cannot find module '/Users/chrisbreuer/code/chunk/node_modules/cross-env/bin/cross-env.js'
    at Function.Module._resolveFilename (module.js:470:15)
    at Function.Module._load (module.js:418:25)
    at Module.runMain (module.js:605:10)
    at run (bootstrap_node.js:425:7)
    at startup (bootstrap_node.js:146:9)
    at bootstrap_node.js:540:3

npm ERR! Darwin 16.4.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "dev"
npm ERR! node v7.7.1
npm ERR! npm  v4.1.2
npm ERR! code ELIFECYCLE
npm ERR! @ dev: `node node_modules/cross-env/bin/cross-env.js NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @ dev script 'node node_modules/cross-env/bin/cross-env.js NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the  package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node node_modules/cross-env/bin/cross-env.js NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/chrisbreuer/code/chunk/npm-debug.log

and the npm-debug.log:

0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'run', 'dev' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'predev', 'dev', 'postdev' ]
5 info lifecycle @~predev: @
6 silly lifecycle @~predev: no script for predev, continuing
7 info lifecycle @~dev: @
8 verbose lifecycle @~dev: unsafe-perm in lifecycle true
9 verbose lifecycle @~dev: PATH: /usr/local/lib/node_modules/npm/bin/node-gyp-bin:/Users/chrisbreuer/code/chunk/node_modules/.bin:/usr/local/sbin:/Users/chrisbreuer/bin:/usr/local/bin:/Users/chrisbreuer/.composer/vendor/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
10 verbose lifecycle @~dev: CWD: /Users/chrisbreuer/code/chunk
11 silly lifecycle @~dev: Args: [ '-c',
11 silly lifecycle   'node node_modules/cross-env/bin/cross-env.js NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js' ]
12 silly lifecycle @~dev: Returned: code: 1  signal: null
13 info lifecycle @~dev: Failed to exec dev script
14 verbose stack Error: @ dev: `node node_modules/cross-env/bin/cross-env.js NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js`
14 verbose stack Exit status 1
14 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/lib/utils/lifecycle.js:279:16)
14 verbose stack     at emitTwo (events.js:106:13)
14 verbose stack     at EventEmitter.emit (events.js:194:7)
14 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/lib/utils/spawn.js:40:14)
14 verbose stack     at emitTwo (events.js:106:13)
14 verbose stack     at ChildProcess.emit (events.js:194:7)
14 verbose stack     at maybeClose (internal/child_process.js:899:16)
14 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)
15 verbose pkgid @
16 verbose cwd /Users/chrisbreuer/code/chunk
17 error Darwin 16.4.0
18 error argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "dev"
19 error node v7.7.1
20 error npm  v4.1.2
21 error code ELIFECYCLE
22 error @ dev: `node node_modules/cross-env/bin/cross-env.js NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js`
22 error Exit status 1
23 error Failed at the @ dev script 'node node_modules/cross-env/bin/cross-env.js NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js'.
23 error Make sure you have the latest version of node.js and npm installed.
23 error If you do, this is most likely a problem with the  package,
23 error not with npm itself.
23 error Tell the author that this fails on your system:
23 error     node node_modules/cross-env/bin/cross-env.js NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js
23 error You can get information on how to open an issue for this project with:
23 error     npm bugs
23 error Or if that isn't available, you can get their info via:
23 error     npm owner ls
23 error There is likely additional logging output above.
24 verbose exit [ 1, true ]

only saves last chunk to upload folder

I just tried it with this fix out of curiosity and realized that only the last chunk gets moved to the upload folder. Later tonight I should have some free time to look at the code, in case I don't I might get back to it at a later time

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.