brorfred / pixz Goto Github PK
View Code? Open in Web Editor NEWThis project forked from vasi/pixz
Parallel, indexed xz compressor
Home Page: http://sourceforge.net/projects/pixz/files/
License: BSD 2-Clause "Simplified" License
This project forked from vasi/pixz
Parallel, indexed xz compressor
Home Page: http://sourceforge.net/projects/pixz/files/
License: BSD 2-Clause "Simplified" License
Pixz (pronounced 'pixie') is a parallel, indexing version of XZ Repository: https://github.com/vasi/pixz Downloads: https://sourceforge.net/projects/pixz/files/ The existing XZ Utils ( http://tukaani.org/xz/ ) provide great compression in the .xz file format, but they have two significant problems: * They are single-threaded, while most users nowadays have multi-core computers. * The .xz files they produce are just one big block of compressed data, rather than a collection of smaller blocks. This makes random access to the original data impossible. With pixz, both these problems are solved. The most useful commands: $ pixz foo.tar foo.tpxz # Compress and index a tarball, multi-core $ pixz -l foo.tpxz # Very quickly list the contents of the compressed tarball $ pixz -d foo.tpxz foo.tar # Decompress it, multi-core $ pixz -x dir/file < foo.tpxz | tar x # Very quickly extract a file, multi-core. # Also verifies that contents match index. $ tar -Ipixz -cf foo.tpxz foo # Create a tarball using pixz for multi-core compression $ pixz bar bar.xz # Compress a non-tarball, multi-core $ pixz -d bar.xz bar # Decompress it, multi-core Specifying input and output: $ pixz < foo.tar > foo.tpxz # Same as 'pixz foo.tar foo.tpxz' $ pixz -i foo.tar -o foo.tpxz # Ditto. These both work for -x, -d and -l too, eg: $ pixz -x -i foo.tpxz -o foo.tar file1 file2 ... # Extract the files from foo.tpxz into foo.tar $ pixz foo.tar # Compress it to foo.tpxz, removing the original $ pixz -d foo.tpxz # Extract it to foo.tar, removing the original Other flags: $ pixz -1 foo.tar # Faster, worse compression $ pixz -9 foo.tar # Better, slower compression $ pixz -p 2 foo.tar # Cap the number of threads at 2 $ pixz -t foo.tar # Compress but don't treat it as a tarball (don't index it) $ pixz -d -t foo.tpxz # Decompress foo, don't check that contents match index $ pixz -l -t foo.tpxz # List the xz blocks instead of files For even more tuning flags, check the manual page. Compare to: plzip * About equally complex, efficient * lzip format seems less-used * Version 1 is theoretically indexable...I think ChopZip * Python, much simpler * More flexible, supports arbitrary compression programs * Uses streams instead of blocks, not indexable * Splits input and then combines output, much higher disk usage pxz * Simpler code * Uses OpenMP instead of pthreads * Uses streams instead of blocks, not indexable * Uses temp files and doesn't combine them until the whole file is compressed, high disk/memory usage Comparable tools for other compression algorithms: pbzip2 * Not indexable * Appears slow * bzip2 algorithm is non-ideal pigz * Not indexable dictzip, idzip * Not parallel Requirements: * libarchive 2.8 or later * liblzma 4.999.9-beta-212 or later (from the xz distribution)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.