Ignoring revisions by username (e.g. spam) can taint following revisions about mediawiki_to_git_md HOT 4 CLOSED

One approach is to give up on removing spam edits, and take the history as is - warts and all. This does at least credit the cleanup work fairly.

Another idea is to do a cleanup after parsing the XML into our intermediate SQLite3 database, where we can add a "hide" boolean. The idea here would be when hitting an unwanted revision, consider the next few revisions of that page, and look for a revert back to the last clean version. If found, then hide all those revisions to the page.

This would handle:

• Spam from A
• More spam from A or A2
• Revert all from B

And also:

• Spam from A
• More spam from A or A2
• Revert some by B
• Revert rest by C

The problematic cases would would be left in the history and require manual intervention to remove.

from mediawiki_to_git_md.

Perhaps the best plan is leave the unwanted commits in the git history, although giving them a clear comment (e.g. "UNWANTED COMMIT") and perhaps also indicate this in the author field.

They can then be removed fairly easily with git rebase -i ... (thanks to the clear comment) possibly with --empty=drop to automatically drop the simple reverts.

from mediawiki_to_git_md.

The interactive rebase is working pretty well to remove spam, coupled with this to find a good preceding commit to start from:

$ git log --oneline | grep -C 1 UNWANTED

This assumes the spam accounts were all flagged before starting the XML dump to git conversion.

from mediawiki_to_git_md.

The default comment from a MediaWiki reversion via the interface seems to start "Reverted edits by " so this is also worth reviewing:

$ git log --oneline --grep revert -i

I am finding the rebases quite slow, but it may be partly down to the filesystem (sadly I can't run this locally on a case in-sensitive Mac).

from mediawiki_to_git_md.