Git Product home page Git Product logo

glide-vc's Introduction

Glide vendor cleaner

Build Status Build status

Important Note!!!

Before using this tool be sure that cleaning and commiting vendored directories to VCS does not violate the licenses of the packages you're vendoring.

For a detailed explanation on why Glide doesn't do this see here

Description

This tool will help you removing from the project vendor directories all the files not needed for building your project. By default it'll keep only needed packages (returned by the glide list command) and all the files inside them. If you want to keep only source code (including tests) files you can provide the --only-code option. If you want to remove also the go test files you can add the --no-tests option.

By default glide-vc doesn't remove:

  • files that are likely to contain some type of of legal declaration or licensing information (to remove them use the --no-legal-files option)
  • nested vendor directories. Doing this will change compilation and runtime behavior of your project because only the top level vendored dependencies will be used for compilation. If these are at a different revision (from the one provided inside nested vendor directories) they can cause compilation problems or runtime misbehiaviours. On the other side, keeping nested vendor directories can cause compilation problems like this one.

Vendoring additional tools

In order to vendor tools, some projects will specify packages in their glide.yaml that aren't imported by the project. These packages will be installed and registered in the glide.lock file (but this behavior may change in the future).

By default glide-vc uses the output of the glide list command to retrieve the packages needed by your project. Since glide list uses your projects imports it will not include the tools contained in glide.lock.

Using the --use-lock-file option will make glide-vc use the packages list from glide.lock instead of the one provided by glide list, preserving the tool packages.

Instead of vendoring these tools using glide and using the glide-vc --use-lock-file option, a suggestion (since there isn't a common accepted practice) is to vendor additional project tools using other scripts/tools and perhaps not inside the vendor directory but in another project's path and use the vendor directory just for go dependencies (or if you want to keep them inside vendor then run your tool after glide-vc). See also this discussion.

Install

go get github.com/sgotti/glide-vc

Run

glide vendor cleaner

Usage:
  glide-vc [flags]

Flags:
      --dryrun            just output what will be removed
      --keep value        A pattern to keep additional files inside needed packages. The pattern match will be relative to the deeper vendor dir. Supports double star (**) patterns. (see https://golang.org/pkg/path/filepath/#Match and https://github.com/bmatcu
k/doublestar). Can be specified multiple times. For example to keep all the files with json extension use the '**/*.json' pattern. (default [])
      --no-legal-files    remove also licenses and legal files
      --no-test-imports   remove also testImport vendor directories. Works only with --use-lock-file
      --no-tests          remove also go test files (requires --only-code)
      --only-code         keep only source code files (including go test files)
      --use-lock-file     use glide.lock instead of glide list to determine imports

You have to run glide-vc, or (if glide is installed) glide vc inside your current project root directory.

To see what it'll do use the --dryrun option.

Examples

Tests removal of all unneeded packages.

glide-vc --dryrun

Do it

glide-vc

Keep only source code (including tests) files.

glide-vc --only-code

Keep only source code without go tests.

glide-vc --only-code --no-tests

Keep only source code without go tests and also remove all licenses and legal files.

glide-vc --only-code --no-tests --no-legal-files

glide-vc's People

Contributors

ericchiang avatar fugitech avatar runcom avatar sgotti avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

glide-vc's Issues

Unneeded vendor packages in glide.lock are kept

The glide.lock file may contain unneeded packages. This can happen with unneeded entries in glide.yaml or with not fine grained ones (for example having just github.com/coreos/etcd when only the github.com/coreos/etcd/client pkg is needed will have a glide.lock entry with a lot of etcd packages not needed by the project)

A way to get the real needed pkgs list will be to use the glide list json output. But, since we want to also keep the base package license files, there's the need to also get the base package path. Actually this isn't exported so a solution will be to recycle the glide functions that do this, but this is suboptimal.

A few ideas

Thanks for putting this together. I know it will be useful.

I have a few thoughts that may help you along.

  1. Don't strip non-Go files by default. Instead opt-in to that with a flag. The reason is those documents can contain license information. Sometimes it's in a license file. Sometimes it's in a readme. You never can be sure.

A good default for licenses is important. Take the BSD license. If someone removes the license and someone redistributes the parent project on GitHub that person is in violation of the BSD license. There are a bunch of things like that.

Would you be open to opting in on that rather than opting out? That way those who can do it, such as a proprietary codebase, can opt-in. It spares accidents for open source projects.

  1. This project deletes things. So, can you add some testing in? Maybe a testdata directory and some tests. Maybe even cross platform (POSIX and Windows)? If you're looking for an example you can see what I did on rmvcsdir. I'm happy to talk more about this.

It's reassuring to know the tests make sure it works right and for those looking to use it.

  1. If you're not going to keep the dependencies in the vendor/ folder, consider adding vendor/ to the .gitignore file. I was considering a pull request but I ran out of time when I was in my dev environment.

  2. This is entirely optional but, glide has a plugin system like git. If this were named glide-vc (or something else) you could use the command glide vc when the plugin is installed. Just a thought. I'm not all that opinionated on this. I just like to raise awareness.

Nice work. I'm happy to see you working on this.

Don't remove main packages explicitly declared in glide.yaml

Issue:

Projects sometime vendor main packages no directly imported by any code in the project. For example one of my projects vendors a go protoc plugin[0].

However, because these aren't imported directly, glide-vc removes them from the vendor directory,

Proposal:

If mains are explicitly declared as packages or sub packages, include them when determining if a package is imported.

[0] https://github.com/coreos/dex/blob/3c9ab50ddf05b07b74cda1777df6addf9ff44dfe/glide.yaml#L125

License?

What license is this available under? Without a license it's technically not open source. Glide itself is under the MIT.

If glide.lock package has empty subpackages, glide-vc cleans subdirs of the repo.

Ex given a glide.lock (not complete, trimmed to demonstrate):

hash: a92787bfb8b070e832ea4315aa20b5220ac3c530383ee531462dc75c1478c7fc
updated: 2016-08-29T11:04:56.268632692-07:00
- name: github.com/coreos/pkg
  version: 160ae6282d8c48a33d8c150e4e4817fdef8a5cde
  subpackages:
  - flagutil

Glide-vc will clean all things that are not
github.com/coreos/pkg/flagutil as expected:

glide vc --dryrun --only-code | grep coreos\/pkg
Removing unused file: github.com/coreos/pkg/.gitignore
Removing unused file: github.com/coreos/pkg/.travis.yml
Removing unused file: github.com/coreos/pkg/CONTRIBUTING.md
Removing unused file: github.com/coreos/pkg/DCO
Removing unused file: github.com/coreos/pkg/MAINTAINERS
Removing unused file: github.com/coreos/pkg/README.md
Removing unused file: github.com/coreos/pkg/build
Removing unused dir: github.com/coreos/pkg/capnslog
Removing unused dir: github.com/coreos/pkg/cryptoutil
Removing unused dir: github.com/coreos/pkg/dlopen
Removing unused dir: github.com/coreos/pkg/health
Removing unused dir: github.com/coreos/pkg/httputil
Removing unused dir: github.com/coreos/pkg/multierror
Removing unused dir: github.com/coreos/pkg/netutil
Removing unused dir: github.com/coreos/pkg/progressutil
Removing unused file: github.com/coreos/pkg/test
Removing unused dir: github.com/coreos/pkg/timeutil
Removing unused dir: github.com/coreos/pkg/yamlutil

Given the following glide.lock I would expect every subpackage in github.com/coreos/pkg to be retained:

hash: a92787bfb8b070e832ea4315aa20b5220ac3c530383ee531462dc75c1478c7fc
updated: 2016-08-29T11:04:56.268632692-07:00
- name: github.com/coreos/pkg
  version: 160ae6282d8c48a33d8c150e4e4817fdef8a5cde

However, it won't keep any packages within github.com/coreos/pkg in this case:

glide vc --dryrun --only-code | grep coreos\/pkg
Removing unused file: github.com/coreos/pkg/.gitignore
Removing unused file: github.com/coreos/pkg/.travis.yml
Removing unused file: github.com/coreos/pkg/CONTRIBUTING.md
Removing unused file: github.com/coreos/pkg/DCO
Removing unused file: github.com/coreos/pkg/MAINTAINERS
Removing unused file: github.com/coreos/pkg/README.md
Removing unused file: github.com/coreos/pkg/build
Removing unused dir: github.com/coreos/pkg/capnslog
Removing unused dir: github.com/coreos/pkg/cryptoutil
Removing unused dir: github.com/coreos/pkg/dlopen
Removing unused dir: github.com/coreos/pkg/flagutil
Removing unused dir: github.com/coreos/pkg/health
Removing unused dir: github.com/coreos/pkg/httputil
Removing unused dir: github.com/coreos/pkg/multierror
Removing unused dir: github.com/coreos/pkg/netutil
Removing unused dir: github.com/coreos/pkg/progressutil
Removing unused file: github.com/coreos/pkg/test
Removing unused dir: github.com/coreos/pkg/timeutil
Removing unused dir: github.com/coreos/pkg/yamlutil

Using --keep has no useful effect here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.