itchyny / github-migrator Goto Github PK

GitHub repository migrator

License: MIT License

Makefile 0.22% Go 99.78%

github-migrator's Introduction

github-migrator

This tool migrates a GitHub repository to another. This is especially useful to move a repository from GitHub Enterprise to github.com.

Usage

export GITHUB_MIGRATOR_SOURCE_API_TOKEN=xxx
export GITHUB_MIGRATOR_SOURCE_API_ENDPOINT=http://localhost/api/v3 # This might be the endpoint of GitHub Enterprise
export GITHUB_MIGRATOR_TARGET_API_TOKEN=yyy
# export GITHUB_MIGRATOR_TARGET_API_ENDPOINT=https://api.github.com # No need to specify the endpoint of github.com
# export GITHUB_MIGRATOR_TARGET_PROXY_URL=http://proxyIp:proxyPort # If you need proxy URL
go run . [old-owner]/[source] [new-owner]/[target]

Be sure to use this tool before pushing the git tree to the new origin (otherwise the links in the merged commits are lost).

Other options

Sometimes same user has different user id on GitHub and Enterprise.

export GITHUB_MIGRATOR_USER_MAPPING=user-before1:user-after1,user-before2:user-after2,user-before3:user-after3

Requirements

Go 1.17+
API tokens to access the source and target repositories.

Features

Issues
- Issue description with the link to the original repository
- Issue comments with the user name and icon (within the comment)
- Created dates, Labels
- Issue numbers are same as the original repository
- Various events (including title changes, issue locking, assignments, review requests and branch deletion in a pull request)
Pull requests
- A pull request is converted to an issue
- Comments and review comments are migrated as issue comments
- Created dates, Labels
- Pull request numbers (issue numbers) are same as the original repository
- Number of changed files, insertions and deletions
- Entire diff (excluding large file diffs)
- Commits list and link to the corresponding /compare/ page
Repository information
- Description, Homepage (only when the target repository has blank description, homepage)
Labels
- Label name, description and colors
- Label changes in issue and pull request
Projects
- Projects, columns and cards
- Note that column automations are not migrated (cannot be set via API)
Milestones
- Milestone titles, description and due date
- Connect issues to milestones
Webhooks
- Webhook URL, content type and events the hooks is trigger for.
All the other things will be lost
- Images posted to issue and pull request comments.
- Emoji reactions to issue and pull request comments
- Diffs (split) view of pull requests
- Wiki
- Default branch, Protection rules
- Notifications, Integrations

Bug Tracker

Report bug at Issues・itchyny/github-migrator - GitHub.

Author

itchyny (https://github.com/itchyny)

License

This software is released under the MIT License, see LICENSE.

Previous works and references

fastlane/monorepo: Scripts to migrate to a monorepo
- This tool greatly influenced me, especially for investigating the usage of the import api.
aereal/migrate-gh-repo: migrate GitHub (incl. Enterprise) repositories with idempotent-like manner
- For the idea of keeping the issue and pull request numbers.
Complete issue import API walkthrough with curl
- Comprehensive tutorial for using the import api (which is not listed in the official api document yet).

github-migrator's People

Contributors

Stargazers

Watchers

Forkers

yopita ykzts chrolis 00mjk everpeace hirox

github-migrator's Issues

Fails when /hooks returns 404

For some reason this returns 404:

https://api.github.com/repos/bergie/passport-saml/hooks?

The URL is correct, but either there are no hooks or my API token doesn't have permission to see them so the URL is 404.

Either way, github-migrator crashes at this point.

Instead, it should treat the case the same as "no hooks" and proceed from there.

I think this is really a bug in the Github API design, though. Returning "403 Forbidden" in this case seems more appropriate, or "204 No Content".

Thanks!

fails due to deleted Github user

Example commit which triggers it:

node-saml/passport-saml@3ba0158

Example of what output looks like when the the failure is triggered:

===> GET: https://api.github.com/users/Herbert%20Voj%C4%8D%C3%ADk
<=== 404 Not Found: GET: https://api.github.com/users/Herbert%20Voj%C4%8D%C3%ADk
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x698452]

goroutine 1 [running]:
github.com/itchyny/github-migrator/migrator.(*builder).buildImageTag(0xc0000a9980, 0x0, 0x23, 0xc0001dbf08, 0x0)
        /home/mark/RAgit/github-migrator/migrator/builder.go:240 +0x42
github.com/itchyny/github-migrator/migrator.(*builder).buildUserActionBody(0xc0000a9980, 0x0, 0x74fd1a, 0x27, 0x0, 0x0, 0x0, 0x0)
        /home/mark/RAgit/github-migrator/migrator/builder.go:234 +0x75

panic: runtime error: invalid memory address or nil pointer dereference

To reproduce, try importing the bergie/passport-saml repo to another empty repo on Github.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x69a399]

goroutine 1 [running]:
github.com/itchyny/github-migrator/migrator.groupEventsByCreated(0xc00008eea0, 0x4, 0x4, 0xc0002351c0, 0x702a60, 0xc0002351a0)
        /home/mark/RAgit/github-migrator/migrator/builder_events.go:75 +0x1b9
github.com/itchyny/github-migrator/migrator.(*builder).buildImportEventComments(0xc0004a1980, 0xc0000fb280, 0x4, 0x4, 0x6d54c0, 0xc0002771c0)
        /home/mark/RAgit/github-migrator/migrator/builder_events.go:15 +0x92
github.com/itchyny/github-migrator/migrator.(*builder).buildImportComments(0xc0004a1980, 0xc000182870, 0x9dd528, 0x0, 0x0, 0xc000130e38)
        /home/mark/RAgit/github-migrator/migrator/builder.go:151 +0x72
github.com/itchyny/github-migrator/migrator.(*builder).build(0xc0004a1980, 0x0, 0x77, 0x77)
        /home/mark/RAgit/github-migrator/migrator/builder.go:71 +0x1c7
github.com/itchyny/github-migrator/migrator.(*migrator).buildImport(0xc00020fd60, 0xc0004b4340, 0xc0003a5cc0, 0xc0000fafe0, 0x4, 0x4, 0xc00008eea0, 0x4, 0x4, 0xc00012e270, ...)
        /home/mark/RAgit/github-migrator/migrator/builder.go:43 +0x190
github.com/itchyny/github-migrator/migrator.(*migrator).migrateIssue(0xc00020fd60, 0xc0004b4340, 0xc0004a1d20, 0x0, 0x0, 0x0, 0x0)
        /home/mark/RAgit/github-migrator/migrator/issues.go:136 +0x8d8
github.com/itchyny/github-migrator/migrator.(*migrator).migrateIssues(0xc00020fd60, 0x0, 0x0)
        /home/mark/RAgit/github-migrator/migrator/issues.go:42 +0x1bc
github.com/itchyny/github-migrator/migrator.(*migrator).Migrate(0xc00020fd60, 0x14, 0x7fff11f4f169)
        /home/mark/RAgit/github-migrator/migrator/migrator.go:71 +0x2fa
main.run(0xc000114160, 0x2, 0x2, 0x44316a, 0x971fe0)
        /home/mark/RAgit/github-migrator/main.go:31 +0x96
main.main()
        /home/mark/RAgit/github-migrator/main.go:17 +0x6c