babel / phabricator-to-github Goto Github PK

https://github.com/babel-bot

Could potentially also be used for codecov messages or other automated stuff.

The current MySQL dump seems to be exported with the encoding utf8 instead of utf8mb4.

This makes emojis render as question marks. (😂 => ?) I think. I'm not entirely sure but I would like to try this solution and see if it works.

I try to reach @kittens to give me access to mysql so I can try exporting the dump with the correct charset.

http://stackoverflow.com/questions/20216849/mysqldump-with-utf8-can-not-export-the-right-emojis-string

Seems the import into SQLite is dropping all newlines from texts (description and comments)

The dump itself contains them.

blocked by #9

CONTRIBUTING: https://github.com/babel/phabricator-to-github/blob/master/CONTRIBUTING.md

Just an issue to track what we can/should do to migrate

Info

Old issue id range: 1 - 3086 (maniphest_task)
New issue range: > 6637
Number of new issues: ~871
Number of new comments: ~2940
Github ratelimit is 5000 requests/hour (https://developer.github.com/v3/#rate-limiting)

Open questions:

• How to compare/merge issues? Someone needs to check if the github id is somewhere in the phabricator dump. Otherwise we need to retrieve all comments beforehand. Phabricator ids match the github ids.
• What was the last phabricatorID for the import from github? IF we know that we can easily know which to merge and which to create (3086)
• Do we want to migrate issues that do not exist yet in github but are already closed? YES
• Do we need to migrate attachments? (There are 41 attachments that can be migrated by hand)
• The issue import API is async, which means that issues could get imported in random order. We could wait till the import of one issue is finished and then send the next, but then the import might take several hours (I've seen delays of up to a minute between sending request and issue being imported.)

Tasks:

• phabricator db dump -> more consumable format #1
• script to create a new github issue with the issues api
• script to create all the comments for this new issues.
• script to merge existing issues that had more comments, changes, ...
• script to close finished issues
• handle the edge case that someone really posts an issue during migration and log these issue numbers (should be easy by just looking if the returned issue number from the api is +1 the last issue number), otherwise they might get lost in the flood of issues.
• create a new dummy user that will actually "report" all the migrated stuff in github (maybe babel-bot)
• setup a simple issue/pr template in the babel repo babel/babel#3690
• Move parser issues to babylon. (16 Issues, https://phabricator.babeljs.io/maniphest/query/5cfBx2lNVIJt/#R)

• We don't want to spam watchers on github with notifications.
  • Maybe notify all watchers of babel at what time we will do the migration, so they could unwatch the project for that period.
  • Or maybe we can globally disable notifications?
  • Or contact github about that? Maybe also useful for the api rate limiting
• make sure usernames transfer over correctly
  • some people don't sign up with github (can we link everything?)
  • is it possible to transfer subscribers?

So that in case someone wants to reuse this it is documented how it works.

with the github import api

See if we can get the github username, otherwise we should at least be able to get the phabricator username.

Add more logging so in case something on the import fails we know what was the last thing successfully migrated.

blocked by #9

• Change phabricator.babeljs.io to be a CNAME to babel.github.io.
• Add phabricator.babeljs.io as custom domain for github-pages in https://github.com/babel/phabricator-redirects/settings

@thejameskyle @kittens Can someone of you do this? That would be nice and should not be that hard.

For reference: Currently the domain is pointing to:

❯ nslookup phabricator.babeljs.io
Server:     192.168.127.253
Address:    192.168.127.253#53

Non-authoritative answer:
Name:   phabricator.babeljs.io
Address: 104.27.143.153
Name:   phabricator.babeljs.io
Address: 104.27.142.153

This is a ordered list of things that need to be done for the migration

• Reach out to github for support with abuse rate limit and notifications @danez
• Disable phabricator to track github repo and send mails about commits merges.
• Put phabricator into readonly mode (login will probably not work anymore)
• Merge PR and Issue template for babel babel/babel#3690
• Enable issues on github
• Migrate old and new issues to github
• Push redirects to (https://github.com/babel/phabricator-redirects)
• Add phabricator.babeljs.io for github pages to this repo
• SITE: Lower DNS TTL to 1 minute if possible ($TTL 60) for babeljs.io (Keep note of the current value)
• SITE: add new additional subdomain for phabricator phabricatorbackup.babeljs.org (or similar)
• SITE: Adjust DNS for phabricator.babeljs.io to be a CNAME to babel.github.io. (https://help.github.com/articles/setting-up-a-custom-subdomain/)
• SITE: Set DNS TTL back to what it was before the first task
• As soon as phabricator.babeljs.io is pointing to github, we could disable readonly mode in phabricator so we can login again in case

I hope I did not miss anything :)

Like babel version, etc.