I'm using ffi-archive and raise error when i start server:

Could not open library 'archive': archive: cannot open shared object file: No such file or directory. (LoadError)
Could not open library 'libarchive.so': libarchive.so: cannot open shared object file: No such file or directory.
Could not open library 'libarchive.so.2': libarchive.so.2: cannot open shared object file: No such file or directory

Describe the Enhancement:

The type of an archive may not be known before reading, but libarchive has methods for exposing this information: format, format_name, compression, and compression_name. Please expose them.

Describe the Need:

The type of an archive being read is unknown.

Current Alternative

No current alternative. This is a blocker to adopters previously using libarchive-ruby-swig.

Can We Help You Implement This?:

I have a PR ready! ✨

It would be nice to have support for archive_write_disk and friends.

Version:

Ruby 2.6.3p62
Rails 6.0.2.2
gem ffi-libarchive 1.0.0

Environment:

MacOs
rails on development

Scenario:

The error came after "bundle install" and Rails app is not starting with the gem.

There are quite a few releases:

https://github.com/chef/ffi-libarchive/tags

which look like they haven't made it to rubygems:

https://rubygems.org/gems/ffi-libarchive/versions

Could you please push the missing versions? 🙏

Would you expect this binding library to already support this? (I'm new here, and trying to sort out where to dig in)

Ref: libarchive/libarchive#586

This manifests if archive_read_data returns an error code, but the code can't handle it because it expects a negative return value in those cases:

/var/lib/gems/2.3.0/gems/ffi-libarchive-0.2.0/lib/ffi-libarchive/reader.rb:111:in `get_bytes': bignum too big to convert into `long' (RangeError)
        from /var/lib/gems/2.3.0/gems/ffi-libarchive-0.2.0/lib/ffi-libarchive/reader.rb:111:in `read_data'
[...]

ChefDK version 0.10.0 installs libarchive.so.13.1.2, which causes problems with ffi-libarchive when used through the libarchive Chef cookbook at https://github.com/reset/libarchive-cookbook. Here's the output for the failure that occurs while trying to extract an archive for the consul cookbook:

==> default: ================================================================================
==> default:         
==> default: Error executing action `extract` on resource 'libarchive_file[/var/chef/cache/consul-0.5.2.zip]'
==> default:         
==> default: ================================================================================
==> default:         
==> default: 
==> default: 
==> default:         
==> default: Archive::Error
==> default:         
==> default: --------------
==> default:         
==> default: Unsupported ZIP compression method (deflation)
==> default:         
==> default: 
==> default: 
==> default:         
==> default: Cookbook Trace:
==> default:         
==> default: ---------------
==> default:         
==> default: /var/chef/cache/cookbooks/libarchive/libraries/helper.rb:47:in `block (2 levels) in extract'
==> default: 
==> default:         /var/chef/cache/cookbooks/libarchive/libraries/helper.rb:39:in `block in extract'
==> default:         /var/chef/cache/cookbooks/libarchive/libraries/helper.rb:36:in `chdir'
==> default:         /var/chef/cache/cookbooks/libarchive/libraries/helper.rb:36:in `extract'
==> default:         /var/chef/cache/cookbooks/libarchive/providers/file.rb:18:in `block in class_from_file'
==> default:         /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
==> default:         /var/chef/cache/cookbooks/libartifact/libraries/libartifact_file.rb:44:in `block in <class:ArtifactFile>'
==> default:         /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
==> default:         /var/chef/cache/cookbooks/consul/libraries/consul_service.rb:97:in `action_enable'

And here's the Vagrantfile that produced the above output. You should be able to run it in isolation to reproduce the issue:

Vagrant.configure('2') do |config|
  config.vm.box = 'precise64'
  config.vm.box_url = 'http://files.vagrantup.com/precise64.box'

  config.vm.provision :shell, inline: <<BASH
sudo apt-get update

wget -qO - https://www.chef.io/chef/install.sh | sudo bash -s -- -P chefdk

# https://github.com/mitchellh/vagrant/issues/6110#issuecomment-152646177
mkdir -p /tmp/vagrant-chef/cookbooks/nodes

cat > /tmp/Berksfile <<BERKSFILE
source 'https://supermarket.chef.io'
cookbook 'consul'
BERKSFILE

berks vendor -b /tmp/Berksfile /tmp/vagrant-chef/cookbooks/cookbooks

# Remove the libarchive.so.13* that comes with Chef, which breaks
# the ffi-libarchive gem, in favor libarchive.so.12* that comes with Vagrant.
# UNCOMMENT ME.
# sudo rm /opt/chefdk/embedded/lib/libarchive*
BASH

  config.vm.provision :chef_zero do |chef_zero|
    chef_zero.install = false
    chef_zero.binary_path = '/opt/chefdk/bin'

    chef_zero.add_recipe 'consul'
  end
end

If you uncomment the line with sudo rm after "UNCOMMENT ME", the problem no longer reproduces because ffi-libarchive uses an older libarchive at /usr/lib/x86_64-linux-gnu/libarchive.so.12.

We believe this is the ultimate underlying cause of an issue with the consul cookbook: sous-chefs/consul#241

Gem Version

0.4.6

Description

Reader::save_data doesn't write to a file because IO.sysopen never accepted a &block.
So code:

    def save_data(file_name)
      IO.sysopen(file_name, "wb") do |fd|
        raise Error, @archive if C.archive_read_data_into_fd(archive, fd) != C::OK
      end
    end

Should be changed to something like:

    def save_data(file_name)
      File.open(file_name, "wb") do |fd|
        raise Error, @archive if C.archive_read_data_into_fd(archive, fd.fileno) != C::OK
      end
    end

Thanks.

Describe the Enhancement:

Hi! 👋 I'd like to add support for libarchive's custom read functions, described in https://github.com/libarchive/libarchive/wiki/Examples#List_contents_of_Archive_with_custom_read_functions and https://github.com/libarchive/libarchive/wiki/LibarchiveIO.

Describe the Need:

The primary motivation is to allow streaming extraction from a source such as a HTTP download without having to save the file on disk or buffer it entirely in memory, which may be infeasible for large archives.

Current Alternative

None that I'm aware of.

Can We Help You Implement This?:

I have an implementation of this at master...jatoben:streaming-reader. It adds Archive.read_open_stream which accepts either a Proc or a custom object; the Proc or object's #call method will be used for the read callback, and if the object responds to #seek and/or #skip, those callbacks will also be added.

I have a couple of working examples here:

• Threaded streaming reader with a Proc: https://gist.github.com/jatoben/4331886f3fdcfb3979c526826bd14c78
• A custom object using HTTP byte range requests to seek and skip: https://gist.github.com/jatoben/ea6feed8f4c7bf5a8471a15ed1f2ca70

#7 Appears to have also bumped the required_ruby_version in ffi-libarchive.gemspec.

I believe this had unintended consequences as I've found a number of third party and Chef cookbooks that don't pin this gem, and are therefore requiring and Ruby versions to be >= 2.4.0.

Dependabot couldn't parse the Gemfile found at /Gemfile.

The error Dependabot encountered was:

Dependabot only supports uninterpolated string arguments to eval_gemfile. Got `__FILE__ + ".local"`

Describe the Enhancement:

When iterating through an archive with Archive#next_header (or via Archive#each_entry), libarchive's archive_read_next_header() function is used to create the entry objects. This function is documented to re-use its internal buffer for each call.

archive_read_next_header()
	     Read the header for the next entry	and return a pointer to	a
	     struct archive_entry.  This is a convenience wrapper around
	     archive_read_next_header2() that reuses an	internal struct
	     archive_entry object for each request.

For me, the reuse lead to surprising behavior when inspecting the Archive::Entry objects separately from the archive iteration, as the results may change unexpectedly, or return nil (when #next_header reaches EOF).

archive = Archive.read_open_filename("test/data/test.tar.gz")
entry = archive.next_header
entry.pathname
=> "test/"

archive.next_header
entry.pathname
=> "test/b/"

loop { break if archive.next_header.nil? }
entry.pathname
=> nil

To avoid this, I'd like to add the option to use either archive_entry_clone() on the returned entry, or archive_read_next_header2(), which does not re-use a buffer. I chose the first option in my implementation (see below).

Describe the Need:

I encountered this when collecting entries in an array for file presence and comparison checks between two archives.

Current Alternative:

Specific attributes (e.g. pathname & size) can be retrieved from the entry and stored before continuing the iteration with #next_header. This is somewhat more awkward in my case, though.

Can We Help You Implement This?:

I have an implementation at master...jatoben:archive-entry-clone which adds a clone_entry: parameter to #next_header. It's working well for me, and I'm happy to open a PR if it looks acceptable (perhaps after adding the clone_entry: option to #each_entry as well).

clone_entry could also be set on the Archive::Reader object, which is a little less flexible but perhaps makes for a nicer interface. I would be happy with that option too 🙂