jgm / zip-archive Goto Github PK
View Code? Open in Web Editor NEWNative Haskell library for working with zip archives
License: Other
Native Haskell library for working with zip archives
License: Other
With zip-archive 0.2.2 (commit a6eaa6e ) I'm having this kind of issues:
$ echo "contents" > somefile
$ Zip somefile.zip somefile
adding: somefile (stored 0%)
$ rm somefile
$ unzip somefile.zip
Archive: somefile.zip
file #1 (somefile):
mismatch between local and central GPF bit 11 ("UTF-8"),
continuing with central flag (IsUTF8 = 1)
extracting: somefile
Moreover file-roller allows me to see the contents of archives created with zip-archive, but not to actually open its files.
On the other hand, using the built-in Zip binary to extract these archives works well.
Software used:
Configuring zip-archive-0.3.2.3...
setup.exe: The program 'unzip' is required but it could not be found.
got this from switch from stackage LTS-10.5 to 10.6
I've got the following message:
"Building zip-archive-0.1.1.8...
Preprocessing library zip-archive-0.1.1.8...
[1 of 1] Compiling Codec.Archive.Zip ( Codec/Archive/Zip.hs, dist/build/Codec/Archive/Zip.o )
Codec/Archive/Zip.hs:413:13: Not in scope: `lookAheadM'
Codec/Archive/Zip.hs:486:10: Not in scope: `lookAhead'
Codec/Archive/Zip.hs:514:33: Not in scope: `lookAhead'
Codec/Archive/Zip.hs:575:10: Not in scope: `lookAhead'
Failed to install zip-archive-0.1.1.8
cabal: Error: some packages failed to install:
zip-archive-0.1.1.8 failed during the building phase. The exception was:
ExitFailure 1"
Filling of CVE-2019-13232
introduced a check against a zipbomb and many (Debian, NixOS) distributions patched against it.
It seems that one of the test of zip-archive is using an archive that triggers this check and the test fails. Please see a test failed in NixOS https://logs.nix.ci/?key=nixos/nixpkgs.64909&attempt_id=cc70c8f9-1d57-4b64-8073-42691767eeda
More information about the attach https://www.bamsoftware.com/hacks/zipbomb/
And the patch applied can be found at madler/unzip@47b3cea
I noticed that Zip --version
still says version 0.1.1.4
. Is the version numbering for the Zip executable independent of the library version (0.3.0.1
)?
Issue tested in both mac os x and linux (ubuntu 20.04): in a ZIP archive, a filename that contains a colon (":", eg "video-list: best.xlsx" ) isn't reported by getEntries; other files in the zip archive do show up. Changing the filename to something that doesn't contain a colon (eg "video-list, best.xlsx") results in the getEntries reporting the file and everything else working as expected.
Packages:
zip-archive 0.4.1
zip 1.7.0
base 4.14.1.0
and ghc 8.6.5.
extractFilesFromArchive is very handy however it doesn't take a target folder and simply extracts in the current folder. This forced me to copy-paste this function and writeEntry to my project and add them a target folder parameter.
Adding this functionnality in the library would be trivial and IMHO very useful.
Hello, I am having trouble unzipping this archive http://rghost.net/8pkKrx4sY , it fails with Did not find end of central directory signature
, while unzip
decompress it just fine.
This is the output of zipinfo:
u@m:~/folder$ zipinfo nonsense.zip
Archive: nonsense.zip
Zip file size: 239899 bytes, number of entries: 3
-rw-rw-rw- 2.0 fat 82798 b- defX 00-Nov-25 20:34 NONSENSE.MZX
-rw-rw-rw- 2.0 fat 110702 b- defN 80-Jan-01 00:00 tomsdine.mod
-rw-rw-rw- 2.0 fat 200302 b- defN 96-Oct-08 10:59 CD_STAR.MOD
3 files, 393802 bytes uncompressed, 239574 bytes compressed: 39.2%
So defX
and defN
, which are deflate methods (normal & maximum compression).
Documentation states:
[t]here is no support for [...] compression methods other than Deflate.
So I decided to open a bug report.
I'm trying to cross-compile zip-archive
as a build dependency to pandoc
.
I use my specially compiled ghc
which produce armhf
binaries, through cabal
using command bellow:
cabal new-build \
--builddir=dist/arm-linux-gnueabihf \
--with-ghc=arm-linux-gnueabihf-ghc \
--with-ghc-pkg=arm-linux-gnueabihf-ghc-pkg \
--with-gcc=arm-linux-gnueabihf-clang \
--with-ld=arm-linux-gnueabihf-ld \
--hsc2hs-options=--cross-compile \
--disable-shared \
--configure-option=--host=arm-linux-gnueabihf
The build failes, right after successfully compiling the setup
binary from Setup.hs
, by following errors:
`./dist/arm-linux-gnueabihf/build/arm-linux/ghc-8.6.3/zip-archive-0.3.3/setup/setup: 1: ./dist/arm-linux-gnueabihf/build/arm-linux/ghc-8.6.3/zip-archive-0.3.3/setup/setup: Syntax error: word unexpected (expecting ")")`
(just for simplicity I replaced full path to zip-archive-0.3.3
by .
)
no more information I can find using -v
flag to cabal
, however here is the full ouput: pastebin
I'm guessing the setup
binary is part of build life-cycle, and it fails to run on build machine as it cross-compiled, however I don't know how to check and/or resolve this, may be manual compile and installing?!
$ file ./dist/arm-linux-gnueabihf/build/arm-linux/ghc-8.6.3/zip-archive-0.3.3/setup/setup
./dist/arm-linux-gnueabihf/build/arm-linux/ghc-8.6.3/zip-archive-0.3.3/setup/setup: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, with debug_info, not stripped
$ uname -m
x86_64
$ arm-linux-gnueabihf-ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.6.3
$ cabal --version
cabal-install version 2.4.1.0
compiled using version 2.4.1.0 of the Cabal library
af8c8f1 broke the build with Nix. Reading the commit message, I am slightly puzzled about what your motivation to make that change might have been.
Failed to build zip-archive-0.4.2.
cabal.exe: Failed to build zip-archive-0.4.2 (which is required by
Build log (
pandoc-2.17.1.1). See the build log above for details.
C:\cabal\logs\ghc-8.10.7\zip-archive-0.4.2-ebb[976](https://github.com/lierdakil/pandoc-crossref/runs/5394825626?check_suite_focus=true#step:11:976)8932adbdb43ef91090f80ebcf911d5c65e.log
):
Preprocessing library for zip-archive-0.4.2..
Building library for zip-archive-0.4.2..
[1 of 1] Compiling Codec.Archive.Zip ( src\Codec\Archive\Zip.hs, dist\build\Codec\Archive\Zip.o )
src\Codec\Archive\Zip.hs:492:31: error:
Error: Variable not in scope: isLetter :: Char -> Bool
|
492 | (c:':':d:xs) | isLetter c
| ^^^^^^^^
Error: Process completed with exit code 1.
https://github.com/lierdakil/pandoc-crossref/runs/5394825626
GHC 8.10.7.
Thereafter a patch to manage special character the same way than for an url :
import Storage.URL( encString )
import Data.Char( isAlpha, isAscii, isDigit )
-- | Writes contents of an 'Entry' to a file.
writeEntry :: [ZipOption] -> Entry -> IO ()
writeEntry opts entry = do
let path = case [d | OptDestination d <- opts] of
(x:_) -> x </> eRelativePath entry
_ -> eRelativePath entry
-- create directories if needed
let dir = takeDirectory path
exists <- doesDirectoryExist dir
unless exists $ do
createDirectoryIfMissing True dir
when (OptVerbose `elem` opts) $
hPutStrLn stderr $ " creating: " ++ dir
if length path > 0 && last path == '/' -- path is a directory
then return ()
else do
when (OptVerbose `elem` opts) $ do
hPutStrLn stderr $ case eCompressionMethod entry of
Deflate -> " inflating: " ++ path
NoCompression -> "extracting: " ++ path
+ #ifdef _WINDOWS
+ let encPath = (encString False (\x -> (isAscii x && isAlpha x) || isDigit x || (elem x "_-. /~")) path)
+ B.writeFile encPath (fromEntry entry)
+ #else
+ B.writeFile path (fromEntry entry)
+ #endif
-- Note that last modified times are supported only for POSIX, not for
-- Windows.
setFileTimeStamp path (eLastModified entry)
I've getting this error on some archives that work fine in other tools.
zip-archive/src/Codec/Archive/Zip.hs
Line 796 in fa62708
This is an example zip file that doesn't work.
https://releases.hashicorp.com/vault/1.15.6/vault_1.15.6_darwin_amd64.zip (sorry about the size).
Everything works in 4.2, which makes me suspect d78c534
Hackage version has binary < 0.6
and binary-0.6 is 1.5 months old. Github version does not have this restriction.
Time to publish new package version?
[1 of 1] Compiling Codec.Archive.Zip ( Codec/Archive/Zip.hs, dist/build/Codec/Archive/Zip.o )
Codec/Archive/Zip.hs:202:4:
Couldn't match expected type time-1.4.0.1:Data.Time.Clock.UTC.UTCTime' with actual type
ClockTime'
In the pattern: TOD modEpochTime _
In a stmt of a 'do' block:
(TOD modEpochTime _) <- getModificationTime path
In the expression:
do { isDir <- doesDirectoryExist path;
let path' = zipifyFilePath $ normalise $ path ++ ...;
contents <- if isDir then return B.empty else B.readFile path;
(TOD modEpochTime _) <- getModificationTime path;
.... }
the time update issue! :)
That path shouldn't be hard-coded. IMHO, the best way to remedy this issue is to call zip
and rely on $PATH
to find the program. Also, the zip
utility should be added to the build-tools
stanza of the test suite.
I tried to build pandoc
from source:
cd ~/oss
git clone https://github.com/jgm/pandoc
cd pandoc
cabal update
cabal install pandoc-cli
The install failed with this error:
cabal: Failed to download zip-archive-0.4.3 (which is required by exe:pandoc
from pandoc-cli-0.1). The exception was:
Unexpected response 403 for
http://objects-us-east-1.dream.io/hackage-mirror/package/zip-archive-0.4.3.tar.gz
I tried cabal get
with verbose output:
ben@zagreus:~/oss/pandoc$ cabal get zip-archive -v3
no user package environment file found at /home/ben/oss/pandoc
Reading available packages of hackage.haskell.org...
Using most recent state specified from most recent cabal update
index-state(hackage.haskell.org) = 2023-03-15T01:34:46Z
Downloading zip-archive-0.4.3
Writing
/home/ben/.cabal/packages/hackage.haskell.org/zip-archive/0.4.3/zip-archive-0.4.3.tar.gz
Selected mirror http://hackage.haskell.org/
Downloading package zip-archive-0.4.3
Searching for curl in path.
Found curl at /usr/bin/curl
Searching for powershell in path.
Cannot find powershell on the path
Searching for wget in path.
Found wget at /usr/bin/wget
Selected http transport implementation: curl
/usr/bin/curl 'http://hackage.haskell.org/package/zip-archive-0.4.3.tar.gz' --output /tmp/transportAdapterGet39717-1 --location --write-out '%{http_code}' --user-agent 'cabal-install/3.6.2.0 (linux; x86_64)' --silent --show-error --dump-header /tmp/curl-headers39717-2.txt
Exception Unexpected response 403 for
http://hackage.haskell.org/package/zip-archive-0.4.3.tar.gz when using mirror
http://hackage.haskell.org/
Selected mirror http://hackage.fpcomplete.com/
Downloading package zip-archive-0.4.3
/usr/bin/curl 'http://hackage.fpcomplete.com/package/zip-archive-0.4.3.tar.gz' --output /tmp/transportAdapterGet39717-4 --location --write-out '%{http_code}' --user-agent 'cabal-install/3.6.2.0 (linux; x86_64)' --silent --show-error --dump-header /tmp/curl-headers39717-5.txt
Exception Unexpected response 403 for
http://hackage.fpcomplete.com/package/zip-archive-0.4.3.tar.gz when using
mirror http://hackage.fpcomplete.com/
Selected mirror http://objects-us-east-1.dream.io/hackage-mirror/
Downloading package zip-archive-0.4.3
/usr/bin/curl 'http://objects-us-east-1.dream.io/hackage-mirror/package/zip-archive-0.4.3.tar.gz' --output /tmp/transportAdapterGet39717-7 --location --write-out '%{http_code}' --user-agent 'cabal-install/3.6.2.0 (linux; x86_64)' --silent --show-error --dump-header /tmp/curl-headers39717-8.txt
Unexpected response 403 for http://objects-us-east-1.dream.io/hackage-mirror/package/zip-archive-0.4.3.tar.gz
For now I worked around the issue by putting the following in my pandoc/cabal.project
:
source-repository-package
type: git
-- Normally this is a git URL, but a local path works too (but I think it must be absolute?):
location: [email protected]:jgm/zip-archive.git
-- Replace this with the commit you want to check out:
tag: fa62708e998bf1b4b5da4bc83fed0607fce5fbd4
I'm not sure if this is a problem with zip-archive
or with the Hackage mirrors. I am located in Japan but also tried with a US VPN, and received the same error. Other packages seem to install just fine. This is just an FYI, feel free to close it if you think it's not a zip-archive
issue.
Hi,
info: UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.
problem: i'm having this issue every while extracting with unzip under linux
Fetched 162 kB in 0s (584 kB/s) (Reading database ... 17571 files and directories currently installed.) Preparing to unpack .../unzip_6.0-16+deb8u3_amd64.deb ... Unpacking unzip (6.0-16+deb8u3) over (6.0-16+deb8u2) ... Processing triggers for mime-support (3.58) ... Setting up unzip (6.0-16+deb8u3) ... Archive: /VOCBENCH.zip inflating: /vocbench/README.txt inflating: /vocbench/vocbench-2.4.4.war inflating: /vocbench/semanticturkey-0.12.2+vb-bundle-2.4.4.zip Archive: /vocbench/semanticturkey-0.12.2+vb-bundle-2.4.4.zip file #1 (semanticturkey-0.12.2-2017-03-02/): mismatch between local and central GPF bit 11 ("UTF-8"), continuing with central flag (IsUTF8 = 1) file #2 (semanticturkey-0.12.2-2017-03-02/bin/): mismatch between local and central GPF bit 11 ("UTF-8"), continuing with central flag (IsUTF8 = 1) ......
PS: i've tested this process on another PC and i don't have this issue. Is this archive corruption/rights OR unzip crypto related ?
Thanks
Consider this program, which creates and then unpacks a zip file evil.zip
that contains a file with filepath "../evil"
:
module Main (main) where
import qualified Codec.Archive.Zip as Zip
import qualified Data.ByteString.Lazy as LBS
main :: IO ()
main = writeIt >> readIt
where writeIt = do
let entry = Zip.toEntry "../evil" 0 mempty
archive = Zip.addEntryToArchive entry Zip.emptyArchive
LBS.writeFile "evil.zip" $ Zip.fromArchive archive
readIt = do
archive <- Zip.toArchive <$> LBS.readFile "evil.zip"
Zip.extractFilesFromArchive [Zip.OptVerbose] archive
When run, this program will indeed create an empty file at "../evil"
. This means that extractFilesFromArchive
is dangerous to use on untrusted zip files. The unzip
tool on *nix has some form of safety mechanism here:
$ unzip evil.zip
Archive: evil.zip
warning: skipped "../" path component(s) in ../evil
extracting: evil
I think the best solution is to fail in a noisy way when such dangerous entries are encountered.
GHC 7.6 uses binary-0.5.x, so constraining to binary >= 0.7 has the rough effect of making this package only work with GHC 7.8. Since 7.8 has only been out for a few months, it seems way to soon to be imposing this restriction.
zip-archive-0.3.2 is failing tests on nightly-2018-01-17 with the following output:
[1 of 1] Compiling Main ( tests/test-zip-archive.hs, dist/build/test-zip-archive/test-zip-archive-tmp/Main.o )
tests/test-zip-archive.hs:12:1: warning: [-Wunused-imports]
The import of ‘Control.Applicative’ is redundant
except perhaps to import instances from ‘Control.Applicative’
To import instances alone, use: import Control.Applicative()
|
12 | import Control.Applicative
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
Linking dist/build/test-zip-archive/test-zip-archive ...
> /tmp/stackage-build12/zip-archive-0.3.2$ dist/build/test-zip-archive/test-zip-archive
^MCases: 8 Tried: 0 Errors: 0 Failures: 0^MCases: 8 Tried: 1 Errors: 0 Failures: 0^MCases: 8 Tried: 2 Errors: 0 Failures: 0^M ^M### Error in: 2
tests/test_dir_with_symlinks: getSymbolicLinkStatus: does not exist (No such file or directory)
^MCases: 8 Tried: 3 Errors: 1 Failures: 0^MCases: 8 Tried: 4 Errors: 1 Failures: 0 adding: LICENSE (deflated 46%)
adding: src/ (stored 0%)
adding: LICENSE (deflated 46%)
adding: src/ (stored 0%)
adding: src/Codec/ (stored 0%)
adding: src/Codec/Archive/ (stored 0%)
adding: src/Codec/Archive/Zip.hs (deflated 74%)
^M ^M### Error in: 4
tests/test_dir_with_symlinks: getSymbolicLinkStatus: does not exist (No such file or directory)
^MCases: 8 Tried: 5 Errors: 2 Failures: 0^MCases: 8 Tried: 6 Errors: 2 Failures: 0 creating: test-zip-archive.1078/dir1
extracting: test-zip-archive.1078/dir1/hi
creating: test-zip-archive.1078/dir1/dir2
inflating: test-zip-archive.1078/dir1/dir2/hello
^MCases: 8 Tried: 7 Errors: 2 Failures: 0 creating: test-zip-archive.1078/dir3
extracting: test-zip-archive.1078/dir3/hi
^M ^MCases: 8 Tried: 8 Errors: 2 Failures: 0
We have added it to the expected-test-failures
list — please let us know when you think the tests should pass.
Contrary to the advertised compatibility with base >= 3 && < 5
, this can be falsified with base < 4.5
:
Configuring library for zip-archive-0.4.1..
Preprocessing library for zip-archive-0.4.1..
Building library for zip-archive-0.4.1..
/opt/ghc/7.0.4/lib/ghc-7.0.4/ghc -B/opt/ghc/7.0.4/lib/ghc-7.0.4 -pgmc /usr/bin/gcc -pgma /usr/bin/gcc -pgml /usr/bin/gcc -pgmP /usr/bin/gcc -E -undef -traditional --make -fbuilding-cabal-package -O -outputdir /tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/build -odir /tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/build -hidir /tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/build -stubdir /tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/build -i -i/tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/build -isrc -i/tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/build/autogen -i/tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/build/global-autogen -I/tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/build/autogen -I/tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/build/global-autogen -I/tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/build -optP-include -optP/tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/build/autogen/cabal_macros.h -package-name zip-archive-0.4.1 -hide-all-packages -no-user-package-conf -package-conf /matrix/.cabal/store/ghc-7.0.4/package.db -package-conf /tmp/matrix-worker/1556000046/dist-newstyle/packagedb/ghc-7.0.4 -package-conf /tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/package.conf.inplace -package-id array-0.3.0.2-143060371bda4ff52c270d1067551fe8 -package-id base-4.3.1.0-bafbc7ad22c91044397c91929f8c61bc -package-id binary-0.8.2.1-ac4ef41f35971e6efc0b383682cc4c1776f9e97874255e7ce158e0fa30195ea4 -package-id bytestring-0.10.8.2-707db3366704fdefee7401a214abc04cfcdd0571cd35723d20009af0db52f6fa -package-id containers-0.4.0.0-b4885363abca642443ccd842502a3b7e -package-id digest-0.0.1.2-8431de2ba8fa0d8e505368bffcd5070e8c318cea390158d4549e1a81c559765f -package-id directory-1.2.0.1-5947aca0b601aa0a61df7df7c298bb1c4ce348f22e43d9d739072ee661ebf849 -package-id filepath-1.2.0.0-b4f4cf7e95546b00f075372f0ccb0653 -package-id mtl-2.2.2-70c5331ff6878818104cc879650655684df8a02a0dd5a02d0904aff9725b5377 -package-id pretty-1.0.1.2-f8dc299a95cc94a3e513e27c5b80f951 -package-id text-1.2.3.1-2597b4605275010fa1b4ef1079216e319be3138e738bdfceda02687ee7d3d0da -package-id time-1.2.0.3-3493203919ef238f1a58223fa55b1ceb -package-id unix-2.4.2.0-7643ffafb38b84c18b4316347085103e -package-id zlib-0.6.2-5ad36fd829f4d0a9c75b45a9d116b60b5c9f80c22d6921ba1c479e6581d6b2af -XHaskell98 Codec.Archive.Zip -Wall -hide-all-packages +RTS -M1750M -t
[1 of 1] Compiling Codec.Archive.Zip ( src/Codec/Archive/Zip.hs, /tmp/matrix-worker/1556000046/dist-newstyle/build/x86_64-linux/ghc-7.0.4/zip-archive-0.4.1/build/Codec/Archive/Zip.o )
src/Codec/Archive/Zip.hs:424:20:
Not in scope: data constructor `CMode'
<<ghc: 323951264 bytes, 544 GCs, 9873329/28553800 avg/max bytes residency (6 samples), 59M in use, 0.00 INIT (0.00 elapsed), 0.11 MUT (0.12 elapsed), 0.17 GC (0.17 elapsed) :ghc>>
See also https://matrix.hackage.haskell.org/package/zip-archive@1555999843
I've already fixed up the meta-data accordingly (see https://hackage.haskell.org/package/zip-archive-0.4.1/revisions/)
It looks like there is a problem building zip-archive with cabal's new-build. See here: haskell/cabal#3394
Compiling with ghc-8.2.2 results in a valid build plan but fails with this error.
[1 of 1] Compiling Codec.Archive.Zip ( src/Codec/Archive/Zip.hs, dist/build/Codec/Archive/Zip.o )
src/Codec/Archive/Zip.hs:809:26: error:
• Variable not in scope: (<>) :: [Char] -> [Char] -> String
• Perhaps you meant one of these:
‘<$>’ (imported from Control.Applicative),
‘<=’ (imported from Prelude), ‘</>’ (imported from System.FilePath)
|
809 | <> "expected %d, got %d bytes")
Suggested solution: raise lower bound on base package.
It would be nice to keep hackage releases and git tags in sync :)
build-tool: unzip
should already verify that unzip
is there for tests. Having build-type: Simple
will simplify some things.
EDIT: also to be sure, test-suite
could try to run unzip --version
to test it's there, and make test fail early.
One of the things is development of cabal-install
, which is sadly affected build-type: Custom
packages. Cross compiling is other. And in general build-type: Custom
packages are difficult. cc @hvr
I'd be happy to make a PR, if this sounds right.
I think it would be useful to be able to modify Entry safely. I need to read all the entries out of an archive and change the pathnames. Currently I'm doing this with eRelativePath, but that seems to be unsafe, since the library internally uses the trailing "/" as a marker for directories.
This is important in my use case, because I am using OptRecursive to add a directory full of files, and then trimming all the paths in the entries. When using OptRecursive with addaddEntryToArchive on a directory, the directory is given an entry in the archive of its own. During my path trimming, the trailing slash happens to be eliminated, which -- as far as the library is concerned -- turns the directory entry into a file entry. Then, when I write out the archive, I end up with a blank file with the same path and filename as the directory.
[1 of 1] Compiling Codec.Archive.Zip ( src/Codec/Archive/Zip.hs, /tmp/zip-archive-0.3.1.1/dist-newstyle/build/x86_64-linux/ghc-7.4.2/zip-archive-0.3.1.1/build/Codec/Archive/Zip.o )
src/Codec/Archive/Zip.hs:265:36:
Not in scope: `<$>'
Perhaps you meant one of these:
`</>' (imported from System.FilePath),
`<.>' (imported from System.FilePath)
This broke several install plans for pandoc
(specifically, cabal install pandoc
on GHC 7.6 would run into a build-failure), as can be seen in the build-matrix screenshot below (the dark red cells for GHC 7.6 and 7.4).
After a bit of investigation I was able to track this down do the change below in the latest release, which had the effect of rendering install-plans with binary < 0.6
and base < 4.8
unsound:
--- a/src/Codec/Archive/Zip.hs
+++ b/src/Codec/Archive/Zip.hs
@@ -262,7 +262,7 @@ readEntry opts path = do
_ -> p)
contents <- if isDir
then return B.empty
- else B.readFile path
+ else B.fromStrict <$> S.readFile path
#if MIN_VERSION_directory(1,2,0)
modEpochTime <- fmap (floor . utcTimeToPOSIXSeconds)
$ getModificationTime path
where a use of <$>
was added in a scope where it's not guaranteed that <$>
is visible.
I've already fixed up the meta-data at https://hackage.haskell.org/package/zip-archive-0.3.1.1/revisions/ accordingly; so there's no immediate need for a new release of zip-archive
, but please make sure the next release either restores compatibility with pre-AMP base
or includes a tighter lower bound on binary
, so this doesn't regress again.
Hi,
on systems without which
(which is non-standard), the test suite fails with:
Running 1 test suites...
Test suite test-zip-archive: RUNNING...
test-zip-archive: which: rawSystem: posix_spawnp: does not exist (No such file or directory)
Please consider using something different, e.g., one of the fixes suggested by shellcheck: command -v
.
Thanks!
Regards
itd
There seems to be no way to test whether an archive is invalid...
I downloaded a zip from a server. Modes (in particular, executable bits) are not preserved by zip-archive
, but they are by unzip
.
I believe that the condition here is incorrect: https://github.com/jgm/zip-archive/blob/master/src/Codec/Archive/Zip.hs#L321
I modified it to read:
when (modes /= 0) $ setFileMode path modes
and decompressed again and permissions are preserved.
I created a zip file with the zip
tool with just one executable file inside with the contents:
#!/bin/bash
echo "Hello world!"
The same thing happens: executable bit not preserved by zip-archive
as is; everything ok if I apply that change.
For info: I'm on a Linux machine. I have 'Zip 3.0 (July 5th 2008), by Info-ZIP' installed.
I'm having trouble batch processing zip files with the lib. I'm getting unicode exceptions like this one:
*** Exception: Cannot decode byte '\x94': Data.Text.Encoding.decodeUtf8: Invalid UTF-8 stream
The problem is the parser assumes, filenames are encoded as UTF-8. But according to the specification the default is CP437 which is incompatible to UTF-8, examples are 'ä', 'ö', 'ü'. Which encoding is used should be indicated by the 11th bit (language encoding flag (EFS)) from the general purpose bit flag, says the spec. Currently the EFS is just ignored by the parser. For me the question is, are there specific reasons to implement it like that or would you merge a pull request implementing it faithful to the spec? For now I helped myself silently ignoring decoding errors with decodeUtf8With and ignore, but that's rather ugly.
Here we go again ... I report this error for, I dunno, the third time, maybe? Why is that necessary? Do you even care whether this packages compiles in Nix (and other environments with proper sandboxing)?
Anyhow, for the record:
Test suite test-zip-archive: RUNNING...
Cases: 11 Tried: 4 Errors: 0 Failures: 0 adding: LICENSE (deflated 46%)
adding: src/ (stored 0%)
adding: LICENSE (deflated 46%)
adding: src/ (stored 0%)
adding: src/Codec/ (stored 0%)
adding: src/Codec/Archive/ (stored 0%)
adding: src/Codec/Archive/Zip.hs (deflated 74%)
adding: test-zip-archive.1418/test_dir_with_symlinks2/ (stored 0%)
adding: test-zip-archive.1418/test_dir_with_symlinks2/link_to_directory (deflated 13%)
adding: test-zip-archive.1418/test_dir_with_symlinks2/link_to_directory/file.txt (stored 0%)
adding: test-zip-archive.1418/test_dir_with_symlinks2/link_to_file (deflated 14%)
adding: test-zip-archive.1418/test_dir_with_symlinks2/1/ (stored 0%)
adding: test-zip-archive.1418/test_dir_with_symlinks2/1/file.txt (stored 0%)
adding: test-zip-archive.1418/test_dir_with_symlinks2/ (stored 0%)
adding: test-zip-archive.1418/test_dir_with_symlinks2/link_to_directory (deflated 13%)
adding: test-zip-archive.1418/test_dir_with_symlinks2/link_to_file (deflated 14%)
adding: test-zip-archive.1418/test_dir_with_symlinks2/1/ (stored 0%)
adding: test-zip-archive.1418/test_dir_with_symlinks2/1/file.txt (stored 0%)
Cases: 11 Tried: 6 Errors: 0 Failures: 0 creating: test-zip-archive.1418/dir1
creating: test-zip-archive.1418/dir1/dir2
inflating: test-zip-archive.1418/dir1/dir2/hello
extracting: test-zip-archive.1418/dir1/hi
Cases: 11 Tried: 7 Errors: 0 Failures: 0 creating: test-zip-archive.1418/dir3
extracting: test-zip-archive.1418/dir3/hi
Cases: 11 Tried: 10 Errors: 0 Failures: 0/bin/sh: unzip: command not found
### Error in: 10
callCommand: unzip ./test-zip-archive.1418/testUnzip.zip (exit 127): failed
Cases: 11 Tried: 11 Errors: 1 Failures: 0
test-zip-archive.1418/test_dir_with_symlinks2/
test-zip-archive.1418/test_dir_with_symlinks2/link_to_directory
test-zip-archive.1418/test_dir_with_symlinks2/link_to_directory/file.txt
test-zip-archive.1418/test_dir_with_symlinks2/link_to_file
test-zip-archive.1418/test_dir_with_symlinks2/1/
test-zip-archive.1418/test_dir_with_symlinks2/1/file.txt
test-zip-archive.1418/test_dir_with_symlinks2/
test-zip-archive.1418/test_dir_with_symlinks2/link_to_directory
test-zip-archive.1418/test_dir_with_symlinks2/link_to_file
test-zip-archive.1418/test_dir_with_symlinks2/1/
test-zip-archive.1418/test_dir_with_symlinks2/1/file.txt
Test suite test-zip-archive: FAIL
Test suite logged to: dist/test/zip-archive-0.3.2.2-test-zip-archive.log
Explain the problem.
pandoc cannot read this epub file. It doesn't give any more information than that one line. I can extract the same epub without issues as a zip file as well as with ebook-convert
from Calibre.
❯ pandoc -i s.epub --verbose
Couldn't extract ePub file
Pandoc version?
pandoc 3.1, Arch Linux
Input epub file: Software Design X-Rays Fix Technical Debt with Behavioral Code Analysis by Adam Tornhill.zip
Hello,
I'm using zip-archive
version 0.4.3.
I have this pseudocode:
import "zip-archive" Codec.Archive.Zip qualified as ZArch
import Data.Time.Clock.System (getSystemTime, systemSeconds)
systime <- liftBase getSystemTime
let lastModified = fromIntegral $ systemSeconds systime
let e = ZArch.toEntry fpath lastModified bscContents
ZArch.fromArchive (ZArch.addEntryToArchive e ZArch.emptyArchive)
So basically this should create a zip file with some entry where last modified
is current local time. However, from what I see in my archive viewer, the last modified
time is set to UTC.
To be more precise:
st <- getSystemTime
systemToUTCTime st
returns 2024-04-04 10:05:34 UTC
currently. The local time is 12:05:34
. So the st
value is correct. However, the zip file shows 10:05:34
LOCAL time.
I see that the C library differentiates between zip_file_set_mtime
and zip_file_set_dostime
(https://libzip.org/documentation/zip_file_set_mtime.html):
Following historical practice, the zip_file_set_mtime() function translates the time from the zip archive into the local time zone. If you want to avoid this, use the zip_file_set_dostime() function instead.
Since zip-archive
is a native Haskell library, I'm guessing that there are 2 different fields for this? Maybe we set the wrong one?
I noticed the license in 0.1.1.8 seems to have changed from GPL to BSD?
However the GPL COPYING file is still included and both Zip.hs files still say GPL in their headers.
Was the change fully intended or perhaps done incompletely?
I'm not sure what the correct solution here is, whether to modify zip-archive.cabal or change Stackage semantics. Currently, Stackage uses the build-tools fields to force the correct ordering of building Haskell packages. For example, to make sure alex and happy are built before things that depend on them.
The addition of the zip
requirement to the zip-archive test suite causes confusion here, since there is no executable zip
provided by any Haskell packages, which causes your test suite to be listed as failing. Like I said above, I'm not sure what the correct solution is. Temporarily, I can just disable the test suite for zip-archive, but that's a sub-optimal solution.
I use zip-archive as a library in ampersand. We need to extract the file https://github.com/AmpersandTarski/Prototype/archive/v1.1.1.zip programatticaly. Since we upgraded to zip-archive v0.4, we get an error message:
Generating prototype...
Error encountered during deployment of prototype framework:
Failed to extract the archive found at https://github.com/AmpersandTarski/Prototype/archive/v1.1.1.zip
UnsafePath "C:\\Bitnami\\wampstack-7.1.26-0\\apache2\\htdocs\\AmpersandPrototypes\\Arbeidsduur.proto\\.bowerrc"
I think it is totally valid to extract a zipfile that contains only files and directories that are in scope, such as .bowerrc
I expected that the extraction of the archive would go without any problem. Has something gone wrong with the implementation of Issue #50 ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.