Git Product home page Git Product logo

zippy's Introduction

HELLO

Support my work on Patreon

zippy's People

Contributors

3b avatar christophejunke avatar daewok avatar scymtym avatar shinmera avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

zippy's Issues

can not extract this file

Invalid index 4096 for (SIMPLE-ARRAY (UNSIGNED-BYTE 8) (4096)), should be a non-negative integer below 4096.
[Condition of type SB-INT:INVALID-ARRAY-INDEX-ERROR]

Restarts:
0: [RETRY] Retry SLIME REPL evaluation request.
1: [*ABORT] Return to SLIME's top level.
2: [ABORT] abort thread (#<THREAD "new-repl-thread" RUNNING {10038CD1E3}>)

Backtrace:
0: (3BZ::DECOMPRESS-DEFLATE/VECTOR #<3BZ::OCTET-VECTOR-CONTEXT {10054D8DF3}> #S(ORG.SHIRAKUMO.ZIPPY::DEFLATE-STATE :CURRENT-STATE :DECODE-COMPRESSED-DATA :LAST-BLOCK-FLAG NIL :DYNAMIC-HUFFMAN-TREE #1=(#S..
1: ((:METHOD ORG.SHIRAKUMO.ZIPPY:CALL-WITH-DECOMPRESSED-BUFFER (T T T T 3BZ::DEFLATE-STATE)) #<FUNCTION (FLET ORG.SHIRAKUMO.ZIPPY:OUTPUT :IN ORG.SHIRAKUMO.ZIPPY:ENTRY-TO-FILE) {100554862B}> #(74 243 249 ..
2: ((FLET ORG.SHIRAKUMO.ZIPPY:OUTPUT :IN ORG.SHIRAKUMO.ZIPPY:CALL-WITH-DECRYPTED-BUFFER) 0 4096)
3: ((:METHOD ORG.SHIRAKUMO.ZIPPY:CALL-WITH-DECRYPTED-BUFFER (T STREAM T ORG.SHIRAKUMO.ZIPPY::STREAM-DECRYPTION-STATE)) #<FUNCTION (FLET ORG.SHIRAKUMO.ZIPPY::DECOMPRESS :IN ORG.SHIRAKUMO.ZIPPY:DECODE-ENTR..
4: (ORG.SHIRAKUMO.ZIPPY:ENTRY-TO-FILE #P"/tmp/OEBPS/Images/image00731.jpeg" #<ORG.SHIRAKUMO.ZIPPY:ZIP-ENTRY "OEBPS/Images/image00731.jpeg"> :IF-EXISTS :ERROR :PASSWORD NIL :RESTORE-ATTRIBUTES T)
5: (ORG.SHIRAKUMO.ZIPPY:EXTRACT-ZIP #<ORG.SHIRAKUMO.ZIPPY:ZIP-FILE #P"/tmp/pingfan.zip"> "/tmp/pingfan" :IF-EXISTS :ERROR :PASSWORD NIL)
6: (ORG.SHIRAKUMO.ZIPPY::CALL-WITH-INPUT-ZIP-FILE #<FUNCTION (LAMBDA (ORG.SHIRAKUMO.ZIPPY::ZIP) :IN ORG.SHIRAKUMO.ZIPPY:EXTRACT-ZIP) {1004FDA71B}> "/tmp/pingfan.zip" :START 0 :END NIL)
7: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ORG.SHIRAKUMO.ZIPPY:EXTRACT-ZIP "/tmp/pingfan.zip" "/tmp/pingfan") #)
8: (EVAL (ORG.SHIRAKUMO.ZIPPY:EXTRACT-ZIP "/tmp/pingfan.zip" "/tmp/pingfan"))
--more--
pingfan.zip

zip-file or zip-entry?

Hello,
I think you meant to use
(setf (aref entries 0) (make-instance 'zip-entry :content file :file-name "-"))
instead of
(setf (aref entries 0) (make-instance 'zip-file :content file :file-name "-"))
@ zippy.lisp, line 123
(sorry, I am not familiar yet with pull requests).
Kind regards

Cannot "store" files larger than 1024 bytes

Hi,

I've been trying to use Zippy to create a zip where multiple entries are "stored" instead of compressed with deflate. It looks like this can be done by creating a zip-entry with :compression-method nil, but any entries with an uncompressed size larger than 1024 bytes are compressed with deflate, as if :compression-method :deflate was set.

Setup and tests

My setup has two files:

$ ls -l moby-dick.txt alphabet.txt
-rw-r--r-- 1 iukibkim iukibkim      52 Nov 28 21:22 alphabet.txt
-rw-r--r-- 1 iukibkim iukibkim 1276235 Nov 28 21:09 moby-dick.txt

Here's my control created using Infozip:

$ zip -0 infozip-test.zip moby-dick.txt alphabet.txt 
  adding: moby-dick.txt (stored 0%)
  adding: alphabet.txt (stored 0%)

Here's the minimal test case, showing the behavior:

(ql:quickload "zippy")
(setf *zip* (make-instance 'org.shirakumo.zippy:zip-file))

(setf
 (org.shirakumo.zippy:entries *zip*)
 (vector
  ;; create a entry for a file larger than 1024 bytes
  (make-instance 'org.shirakumo.zippy:zip-entry
		 :content "~/tmp/moby-dick.txt"
		 ;; expecting this to be "0" or store
		 :compression-method nil)
  ;; create a entry for a file less than 1024 bytes
  (make-instance 'org.shirakumo.zippy:zip-entry
		 :content "~/tmp/alphabet.txt"
		 :compression-method nil)))

;; write zip file, expecting both to be stored
(org.shirakumo.zippy:compress-zip *zip* "~/tmp/zippy-test.zip")

Results

Expected results (created with Infozip), both files are "Stored".

$ unzip -v infozip-test.zip 
Archive:  infozip-test.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
 1276235  Stored  1276235   0% 2021-11-28 21:09 3a80500b  moby-dick.txt
      52  Stored       52   0% 2021-11-28 21:22 ed99fb6c  alphabet.txt
--------          -------  ---                            -------
 1276287          1276287   0%                            2 files

Actual results (created with sample code). Only one file, the file less than 1024 bytes is "Stored". The other is "Defl:N" or deflate compressed.

$ unzip -v zippy-test.zip 
Archive:  zippy-test.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
 1276235  Defl:N   744379  42% 2064-09-28 10:27 3a80500b  moby-dick.txt
      52  Stored       52   0% 2065-06-02 10:27 ed99fb6c  alphabet.txt
--------          -------  ---                            -------
 1276287           744431  42%                            2 files

Findings

I've found the code in encode-file where files with :compression-method nil and larger than 1024 bytes are set to have deflate compression enabled:

zippy/encode.lisp

Lines 50 to 53 in 19a4b43

(when (and content
(null (compression-method entry))
(< 1024 (or (uncompressed-size entry) 1025)))
(setf (compression-method entry) :deflate))))

The nil being used to "store" files is ignored because of the check in encode-file.

The actual problem seems to be in this table below, maybe "store" compressions should have it's own entry that isn't nil or have two entries for the key 0?

zippy/tables.lisp

Lines 40 to 42 in 8603b36

(defparameter *compression-method-map*
(alist-vector '((0 . NIL)
(1 . :shrink)

Thanks.

Versions

Quicklisp version and dist:

CL-USER> (ql:dist-version "quicklisp")
"2021-10-21"
CL-USER> (ql:client-version)
"2021-02-13"

Clozure CL version:

$ ccl64 --version
Version 1.12.1  LinuxX8664

Infozip version:

$ zip -v
Copyright (c) 1990-2008 Info-ZIP - Type 'zip "-L"' for software license.
This is Zip 3.0 (July 5th 2008), by Info-ZIP.
Currently maintained by E. Gordon.  Please send bug reports to
the authors using the web page at www.info-zip.org; see README for details.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip,
as of above date; see http://www.info-zip.org/ for other sites.

Compiled with gcc 10.2.1 20210110 for Unix (Linux ELF).

Zip special compilation options:
	USE_EF_UT_TIME       (store Universal Time)
	BZIP2_SUPPORT        (bzip2 library version 1.0.8, 13-Jul-2019)
	    bzip2 code and library copyright (c) Julian R Seward
	    (See the bzip2 license for terms of use)
	SYMLINK_SUPPORT      (symbolic links supported)
	LARGE_FILE_SUPPORT   (can read and write large files on file system)
	ZIP64_SUPPORT        (use Zip64 to store large files in archives)
	UNICODE_SUPPORT      (store and read UTF-8 Unicode paths)
	STORE_UNIX_UIDs_GIDs (store UID/GID sizes/values using new extra field)
	UIDGID_NOT_16BIT     (old Unix 16-bit UID/GID extra field not used)
	[encryption, version 2.91 of 05 Jan 2007] (modified for Zip 3)

Encryption notice:
	The encryption code of this program is not copyrighted and is
	put in the public domain.  It was originally written in Europe
	and, to the best of our knowledge, can be freely distributed
	in both source and object forms from any country, including
	the USA under License Exception TSU of the U.S. Export
	Administration Regulations (section 740.13(e)) of 6 June 2002.

Zip environment options:
             ZIP:  [none]
          ZIPOPT:  [none]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.