laserphaser / gitlab-group-clone Goto Github PK

View Code? Open in Web Editor NEW

15.0 2.0 8.0 211 KB

Tool for recursive cloning whole gitlab group tree

License: MIT License

Python 100.00%

gitlab gitlab-api

gitlab-group-clone's Introduction

gitlab-clone

Tool for easy cloning whole gitlab structure to your local machine.

Free software: MIT license

Requirements

Requests
Python >= 3.6

Installation

You can install "gitlab-clone" via pip:

$ pip install gitlab-clone

Usage

>>> gitlab-clone:: optional arguments: -h, --help show this help message and exit --group_id group_id Id of a group in gitlab --branch branch Branch to clone in all repos [by default master] --http Clone via http instead of ssh --gitlab-url gitlab Gitlab address [by default gitlab.com] --token token Gitlab Token

Example

$ gitlab-clone --group=123 --token=MySecretToken --gitlab-url=gitlab.organization.com

For example if you clone this group https://gitlab.com/lmaolmaolmao which id is 6849299

you will have absolutely the same structure locally:

>>> tree

.

└── lmaolmaolmao

└── subgroup1

└── subgroup2: └── subgroup3

If you will use this command again gitlab-clone will just pull the updated to the existing repos, and will download new repos from the group.

gitlab-group-clone's People

Stargazers

Watchers

Forkers

f9n johannges nickstatka777 bogdaan teamcfml-in phuthuycoding spirkaa eulenleber

gitlab-group-clone's Issues

fail to display help

Hello Arseni

I was able to install your contribution

gilcot@localhost $ pip install gitlab-clone
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Defaulting to user installation because normal site-packages is not writeable
Collecting gitlab-clone
  Using cached gitlab_clone-0.1.3-py2.py3-none-any.whl (4.3 kB)
Requirement already satisfied: requests in /Users/gilcot/Library/Python/2.7/lib/python/site-packages (from gitlab-clone) (2.23.0)
Requirement already satisfied: idna<3,>=2.5 in /Users/gilcot/Library/Python/2.7/lib/python/site-packages (from requests->gitlab-clone) (2.9)
Requirement already satisfied: chardet<4,>=3.0.2 in /Users/gilcot/Library/Python/2.7/lib/python/site-packages (from requests->gitlab-clone) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /Users/gildascotomale/Library/Python/2.7/lib/python/site-packages (from requests->gitlab-clone) (1.25.9)
Requirement already satisfied: certifi>=2017.4.17 in /Users/gildcot/Library/Python/2.7/lib/python/site-packages (from requests->gitlab-clone) (2020.4.5.1)
Installing collected packages: gitlab-clone
Successfully installed gitlab-clone-0.1.3

But I can't make it work

gilcot@localhost $ gitlab-clone -h
Traceback (most recent call last):
  File "/Users/gilcot/Library/Python/2.7/bin/gitlab-clone", line 5, in <module>
    from gitlab_clone.clonner import main
  File "/Users/gilcot/Library/Python/2.7/lib/python/site-packages/gitlab_clone/clonner.py", line 30
    f"https://{gitlab_url}/api/v4/groups/{group_id}/projects?private_token={token}&include_subgroups=True&per_page=100&page={page}&with_shared=False")
                                                                                                                                                    ^
SyntaxError: invalid syntax

It seems to be related to argument parsing, but I may be wrong.

Why groupId is needed ?

Hy,

I got this error because i dont have any groupId :

gitlab-clone: error: argument --group_id:

Thanks.

Cloning of a group fails with MissingSchema error

Fresh installation of gitlab-clone:

pip install --user gitlab-clone 
Collecting gitlab-clone
  Downloading gitlab_clone-1.1.0-py2.py3-none-any.whl (4.4 kB)
Requirement already satisfied: requests in /Users/miguel/Library/Python/3.9/lib/python/site-packages (from gitlab-clone) (2.25.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/miguel/Library/Python/3.9/lib/python/site-packages (from requests->gitlab-clone) (1.26.3)
Requirement already satisfied: certifi>=2017.4.17 in /Users/miguel/Library/Python/3.9/lib/python/site-packages (from requests->gitlab-clone) (2020.12.5)
Requirement already satisfied: idna<3,>=2.5 in /Users/miguel/Library/Python/3.9/lib/python/site-packages (from requests->gitlab-clone) (2.10)
Requirement already satisfied: chardet<5,>=3.0.2 in /Users/miguel/Library/Python/3.9/lib/python/site-packages (from requests->gitlab-clone) (4.0.0)
Installing collected packages: gitlab-clone
Successfully installed gitlab-clone-1.1.0

Then trying to clone a group from gitlab.com produces this error:

gitlab-clone --group=123 --token=ABC --gitlab-url gitlab.com
Traceback (most recent call last):
  File "/Users/miguel/Library/Python/3.9/bin/gitlab-clone", line 8, in <module>
    sys.exit(main())
  File "/Users/miguel/Library/Python/3.9/lib/python/site-packages/gitlab_clone/clonner.py", line 24, in main
    clone(**request_param)
  File "/Users/miguel/Library/Python/3.9/lib/python/site-packages/gitlab_clone/clonner.py", line 32, in clone
    response = requests.get(
  File "/Users/miguel/Library/Python/3.9/lib/python/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/miguel/Library/Python/3.9/lib/python/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/miguel/Library/Python/3.9/lib/python/site-packages/requests/sessions.py", line 528, in request
    prep = self.prepare_request(req)
  File "/Users/miguel/Library/Python/3.9/lib/python/site-packages/requests/sessions.py", line 456, in prepare_request
    p.prepare(
  File "/Users/miguel/Library/Python/3.9/lib/python/site-packages/requests/models.py", line 316, in prepare
    self.prepare_url(url, params)
  File "/Users/miguel/Library/Python/3.9/lib/python/site-packages/requests/models.py", line 390, in prepare_url
    raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL 'gitlab.com/api/v4/groups/123/projects?private_token=ABC&include_subgroups=True&per_page=100&page=1&with_shared=False': No schema supplied. Perhaps you meant http://gitlab.com/api/v4/groups/123/projects?private_token=ABC&include_subgroups=True&per_page=100&page=1&with_shared=False?

Tokens should not be stored in repository remotes, at least by default

When cloning using --http --token <MyToken>, all git repository remotes include the token:

https://token:<MyToken>@gitlab.mycompany.com/path/to/repo.git

I think, at least by default, remotes should be created without the auth part (and we rely on git credential.helper=store to hold the token), and writing the token in each and every repo remotes should be an explicit request from the user.

Update is not working

OS: Ubuntu linux
Python: 3.8

First call works: gitlab-clone --group_id <my-id> --token <my-token> --gitlab-url <my-url>
When I want to update the repo by the repeated call of the previous command it fails with error:
[Errno 2] No such file or directory: 'cd'
The issues are the lines https://github.com/LaserPhaser/gitlab-group-clone/blob/master/gitlab_clone/clonner.py#L52-L53
Popen cannot execute shell internal cd <dir>,
but the following construct which changes directory before the command execution can be used:

command = shlex.split(f"git pull")
result_code = subprocess.Popen(command, cwd=path)

laserphaser / gitlab-group-clone Goto Github PK

gitlab-group-clone's Introduction

gitlab-clone

Requirements

Installation

Usage

Example

gitlab-group-clone's People

Stargazers

Watchers

Forkers

gitlab-group-clone's Issues

fail to display help

Why groupId is needed ?

Cloning of a group fails with MissingSchema error

Tokens should not be stored in repository remotes, at least by default

Update is not working

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent