Git Product home page Git Product logo

velocita-proxy's Introduction

Velocita

Docker Image Version (latest semver) Docker Pulls License

Velocita is a caching reverse proxy for Composer repositories and package distribution locations, such as Packagist or GitHub.

  • Speeds up downloads for package metadata and dist files
  • Serves cached files even if the source location is unreachable
  • Can be used as a shared cache by multiple developers, CI, etc.
  • No changes required to your project's files!

Installation

There are two parts to Velocita:

  • Velocita Proxy, a caching reverse proxy
  • Composer-velocita, a plugin which instructs Composer to use a Velocita Proxy instance

Running Velocita Proxy

Velocita is available as a Docker image. There are two supported ways to run this image:

Using docker compose

  1. Clone this repository:

    git clone https://github.com/isaaceindhoven/velocita-proxy
    cd velocita-proxy
    
  2. Copy .env.dist to .env

  3. Edit .env and set:

    • VELOCITA_URL: the URL (e.g. https://mydomain.tld) on which Velocita is hosted
    • VELOCITA_TLS_ENABLED: set to true to enable HTTPS
    • VELOCITA_TLS_CERT_FILE: the path to your X.509 PEM-encoded certificate (or chain) for the domain
    • VELOCITA_TLS_KEY_FILE: the path to the private key associated with the certificate
  4. Start Velocita:

    docker compose -f docker-compose.yml -f docker-compose.https.yml up -d
    
  5. Done!

Using docker

Alternatively, you can run the Docker image directly: see the image's usage instructions.

Using Composer-velocita

Composer-velocita is a Composer plugin that redirects downloads to your Velocita instance for all repositories it supports.

Run the following commands on the machine where you want to enable Velocita, replacing https://url.to.your.velocita.tld/ with the URL of your instance:

composer global config allow-plugins.isaac/composer-velocita true
composer global require isaac/composer-velocita
composer velocita:enable https://url.to.your.velocita.tld/

And you're all set!

Configuration

See the Docker image instructions for environment variables that you can use to configure the proxy's behavior.

Performance

Velocita can give you major performance improvements when a package is not present in the local cache. For example, installing the PHPUnit dependencies from composer.lock:

Configuration Duration Relative
Composer 2.2s ± 0.1s 100%
Composer + Velocita 0.5s ± 0.1s 23%

Command: composer install --profile after composer require phpunit/phpunit:9.5.19 and clearing both the local cache and the vendor folder.

Velocita works great together with Symfony Flex:

Configuration Duration Relative
Composer + Symfony Flex 4.7s ± 0.7s 100%
Composer + Symfony Flex + Velocita 1.6s ± 0.2s 35%

Command: composer create-project symfony/skeleton:v6.0.99 symfony --profile after clearing the local cache.

Benchmark setup:

  • Velocita is configured with mirrors for Packagist and GitHub Codeload
  • PHP version 8.1.4
  • Composer version 2.2.10
  • Symfony Flex 2.1.6

Authors

Contributing

Raise an issue or submit a pull request on GitHub.

License

This project is licensed under the MIT License - see the LICENSE file for details.

velocita-proxy's People

Contributors

gdxbsv avatar gmta avatar nurtext avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

velocita-proxy's Issues

[Request] Dependency monitoring

Hello,

I started using your proxy because I quite often had the problem that github/packagist messed up some dependencies for a short time (which is already quite annoying).

Now, I'm caching dependencies nearly forever. This solved my issue with broken dependencies.

Caching dependencies forever has the bad taste of not knowing what happens with the dependency on the remote site. I would like to know what happens to the dependency.
Is it possible to add some kind of monitoring to the proxy which informs the plugin during an install, whenever a dependency gets deleted or is not available for some reason?

Regards
Max

If MIRROR_{name}_CACHE_EXPIRY is not specified, is the repository specified expiry used?

Thanks for this great project.

I had a question around expiry of repo packages.

I have not set the MIRROR_{name}_CACHE_EXPIRY parameter on the Docker image but although I downloaded some packages from a repo yesterday, when I go to fetch them today via the cache, they are all listed as upstream ... [EXPIRED].

Would specifying MIRROR_{name}_CACHE_EXPIRY override the repo set expiry?

GitLab Authentication

Hey,

I experience issues with GitLab again. I'm not sure if that's due to a GitLab change or a setup change on my side.

I get a 404 all the time when I try to require a private gitlab-provided package. GitLab returns a 404 whenever the authentication fails.
So I tried it by myself. Adding basic auth -> 404. I had to create a new access token and had to add it as a get parameter or header:

private_token=XYZ
--header "PRIVATE-TOKEN: XYZ"

Could you take a look at this, please?

Thanks!

Issue downloading private package from gitlab.com

Hi all.

First at all thank you for this aweseome project.

My setup:

  • Proxy is running as docker image
  • It is using https
  • Secured by basic auth
  • I activated the gitlab mirror and added my bearer token

When opening the mirrors.json page the following json is returned:

{
    "mirrors": [
        {
            "url": "//gitlab.com/",
            "path": "/mirror/gitlab/"
        },
        {
            "url": "//codeload.github.com/",
            "path": "/mirror/github-codeload/"
        },
        {
            "url": "//repo.packagist.org/",
            "path": "/mirror/packagist/"
        }
    ]
}

For packages downloading from packagist there is no issue
But when trying to download my private package from gitlab it always fails with 404 not found.

When I try to download it manually via postman providing the bearer token, everything works.

Url which I get in composer when doing composer install:
Failed to download package/mypackage from dist: The "https://myproxy.domain.com/mirror/gitlab/api/v4/projects/my-project/mypackage/repository/archive.zip?sha=12342523" ...

And when I try to open the url directly in the browser following json is returned:

{"error":"404 Not Found"}

When I try it via postman directly:
https://gitlab.com/api/v4/projects/my-project/mypackage/repository/archive.zip?sha=12342523

So I checked the proxy loggs and found where it tries to download the package:

proxy_1  | X.X.X.X - myuser [22/Nov/2023:08:19:12 +0000] "GET /mirror/gitlab/api/v4/projects/my-project/mypackage/repository/archive.zip?sha=12342523 HTTP/1.1" 404 25 "-" "Mozilla/5.0 (X11; Linux x86_64) ..." "X.X.X.X"

proxy_1  | X.X.X.X - myuser [22/Nov/2023:08:19:12 +0000] upstream gitlab.com - [STALE] GET /mirror/gitlab/api/v4/projects/my-project/mypackage/repository/archive.zip?sha=12342523 HTTP/1.1 [-] -ms -b

The request to gitlab always ends in the status "STALE".

Please help me!

Thank you in advance!

Proxy headers to origin

Some private repositories (eg. composer.amasty.com) also require that the User-Agent (eg. Composer/1.9) is sent to the origin in order to grant access. Otherwise the request is rejected with 401.

So the http headers need to be proxied to the origin as well.

Define content encoding headers for 3rd party mirrors

Hey,

gitlab.com has adjusted the API interface with one of the latest releases. If no Accept-Encoding header is defined, GitLab uses the Content-Encoding header with the value "br", which is used by Composer or libcurl.

curl error 61 while downloading URL: Unrecognized content encoding type. libcurl understands deflate, gzip content encodings.

If gitlab.com receives the Accept-Encoding header with the value gzip, then the correct Content-Encoding header with gzip is also returned.

GitLab:
image
Response Header:
image

Velocita -> Gitlab:
image
Response Header:
image

Composer error:
image

Could you help me with this issue, please?

Regards
Max

Some sort of auth for hosting on a public URL

Do you have any thoughts on how authentication could be approached? I am considering putting velocita on a public facing URL so that cloud builds can take easy advantage of it.

Possibly creating a Docker image that includes some of the vouch-proxy setup?

Just throwing this out there at the moment for discussion.

Mirror Basic Authorization with special chars

When the password contains special chars (eg. = or :), auth fails and velocita responds with 401.
When having = in the password, whole password in docker-compose must be quoted (").
As the password is working (request works with postman and with composer without velocita), I think there is a bug in velocita.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.