Git Product home page Git Product logo

git-cvs-fast-import's Introduction

git-cvs-fast-import

git-cvs-fast-import provides a restartable, performant Git importer for CVS repositories, focused on providing continuously updated mirrors of CVS repositories so that they can be analysed using existing tools that can only handle Git natively, such as Sourcegraph.

To support continuous updates, git-cvs-fast-import makes some tradeoffs compared to other tools that convert CVS repositories into Git: most notably, considerably less effort is made to preserve precise history, especially on branches and tags. This is not intended to be a general purpose, one time converter when migrating from a CVS setup to Git: see the comparison to other tools for suggestions on what to use in that case.

Installation

Linux binaries are provided on the releases page, including RPM and DEB packages for RHEL/CentOS and Debian/Ubuntu installs, respectively. These binaries have been tested back to CentOS 7 and Ubuntu 16.04.

You will also need git installed, as git-cvs-fast-import uses the git fast-import command internally when operating. Any version released in the last decade should be sufficient.

Usage

You will need access to the CVSROOT of the CVS repository you wish to import, as git-cvs-fast-import parses the RCS files in the root to import the history of each file. In practice, this means you should expect to see a tree of files ending in ,v.

You will also need a valid Git repository. This means that you need to git init your target repository before running git-cvs-fast-import for the first time.

Full help is available through git-cvs-fast-import --help, but for most uses, you only need to provide the CVSROOT, Git repository, metadata store, and (optionally) the CVS directories to be imported. For example, to import the project and src directories from a CVS repository at /cvs, and write to a Git repository at /git, and store the metadata at /tmp/import.db, you would run the following:

git-cvs-fast-import -c /cvs -g /git -s /tmp/import.db project src

By default, all branches will be imported, but this can be controlled by only specifying the branches of interest with --branch.

Comparison to other tools

We know of three other tools that allow for CVS-to-Git conversion:

  • git-cvsimport: this ships with Git, and has support for incremental updates.
  • cvs-fast-export: this is a standalone tool that parses CVS repositories and exports data in the git fast-import stream format, but does not support incremental updates.
  • cvs2git is referenced in the git-cvsimport manpage, but no longer appears to have a home page.

We would suggest trying cvs-fast-export first for one-time conversions where the CVS repository will not be used thereafter, and then falling back to git-cvsimport if cvs-fast-export fails (which can happen with complex Git histories).

Known issues

  • Tag history can be misleading: CVS tags are applied on a per-file basis, whereas Git tags are per-repository. As a result, git-cvs-fast-import makes a fake commit for each tag: this ensures that the actual content of the tag is correct, but may be misleading in terms of the history of the tag if the same CVS tag was applied to different files at different times, as commits may appear in the Git log that weren't logically part of the CVS history for a specific file.

Development

Please refer to DEVELOPMENT.md for more detail on how this tool is structured, why some choices were made, and how to contribute.

git-cvs-fast-import's People

Contributors

lawngnome avatar renovate-bot avatar renovate[bot] avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Forkers

david-sandy

git-cvs-fast-import's Issues

Allow mark files > 2³¹-1 to be written to the database

I think this is a theoretical concern right now, but it's probably close enough that we should at least have a ticket.

There's a fairly obvious path forward here, roughly covered in Connection::set_raw_marks():

  1. Split mark files into chunks of no larger than 231-1 and write each chunk to a new marks record.
  2. Implement a reader on top of the Blob Read implementation that can handle multiple records in get_raw_marks().

Add non-HEAD branch support

Right now, only the HEAD branch is turned into a Git branch. We should use the same patchset detection on other branches and insert accordingly.

The major challenge here is ensuring that commits that are shared across multiple branches are only represented once in the Git repo, and not multiple times.

CVS: investigate non-deterministic import behaviour

I observed while testing HEAD last week that importing larger CVS repositories could be non-deterministic, presumably due to random commit ordering causing issues with patchset detection. I don't believe this affects 0.1, but this blocks the 0.2 release I'd like to do!

Investigate git-fast-import exit code 1

It looks like imports that don't actually import anything might result in exit code 1. I need to verify where that's coming from, what it means (since it's not in the manpage), and then special case as appropriate.

Re-enable clippy in CI

Clippy is disabled for now while this is still under heavy development, but should be re-enabled before this becomes public.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository. View logs.

  • WARN: Fallback to renovate.json file as a preset is deprecated, please use a default.json file instead.

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

cargo
Cargo.toml
  • anyhow 1.0.53
  • async-recursion 1.0.0
  • flexi_logger 0.22.3
  • flume 0.10.10
  • log 0.4.14
  • num_cpus 1.13.1
  • parse_duration 2.1.1
  • structopt 0.3.26
  • tempfile 3.3.0
  • thiserror 1.0.30
  • tokio 1.16.1
  • walkdir 2.3.2
  • tokio-test 0.4.2
comma-v/Cargo.toml
  • chrono 0.4.19
  • derive_more 0.99.17
  • itertools 0.10.3
  • nom 7.1.0
  • thiserror 1.0.30
  • anyhow 1.0.53
  • structopt 0.3.26
eq-macro/Cargo.toml
  • quote 1.0.15
  • syn 1.0.86
git-fast-import/Cargo.toml
  • derive_more 0.99.17
  • nom 7.1.0
  • rev_lines 0.2.1
  • serde 1.0.136
  • thiserror 1.0.30
internal/process/Cargo.toml
  • log 0.4.14
  • structopt 0.3.26
  • thiserror 1.0.30
  • tokio 1.16.1
internal/state/Cargo.toml
  • bincode 1.3.3
  • derive_more 0.99.17
  • log 0.4.14
  • serde 1.0.136
  • speedy 0.8.1
  • thiserror 1.0.30
  • tokio 1.16.1
  • zstd 0.10.0
patchset/Cargo.toml
  • binary-heap-plus 0.4.1
  • thiserror 1.0.30
rcs-ed/Cargo.toml
  • anyhow 1.0.53
  • nom 7.1.0
  • thiserror 1.0.30
  • structopt 0.3.26
dockerfile
Dockerfile
  • rust 1.58.1-alpine@sha256:b61698ea823c6f9bc726272d7783867d89e79ca87e9944998739ce619da7699a
  • alpine 3.15@sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300
github-actions
.github/workflows/goreleaser.yaml
  • actions/checkout v2
  • actions/setup-go v2
  • actions-rs/toolchain v1
  • goreleaser/goreleaser-action v2
.github/workflows/rust.yaml
  • actions/checkout v2
  • actions-rs/toolchain v1
  • actions-rs/cargo v1
  • actions/checkout v2
  • actions-rs/toolchain v1
  • actions-rs/cargo v1
  • actions/checkout v2
  • actions-rs/toolchain v1
  • actions-rs/cargo v1

  • Check this box to trigger a request for Renovate to run again on this repository

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.