16

Is there a way to show "who contributed to this line" of each line of a page of Wikipedia or of a MediaWiki site?

It would show per page and would be similar to the subversion blame tool.

5
  • Do you want to find the first occurrence of a specific word or phrase in a page's revision history, or is this question about something else? There must be some way to download the full revision history for a specific page, but I haven't found it yet. Nov 26, 2012 at 17:59
  • Yeah that's about right.
    – rogerdpack
    Nov 26, 2012 at 18:01
  • 2
    It's very easy to do this using WikiBlame, as explained here: webapps.stackexchange.com/a/35914/20087 Nov 26, 2012 at 19:19
  • Wow that's pretty close. Looks like that one drills in to find the committer of "given words" in the wiki page, maybe it could be modified to show the entire page :)
    – rogerdpack
    Nov 26, 2012 at 19:55
  • It already shows the revision where the text was added, so you can just click on that link to see what the entire page looked like when the text was added. Nov 26, 2012 at 20:02

5 Answers 5

3

I often need something like this as well, but it looks like there's no ready solution.

What I've done is written a script that helps me grab revisions using the MediaWiki API and import them to the Bazaar version control system. bzr qblame article.wiki, then gives a nice view of who changed what. The script is not really ready for release, but you can find it below or on Pastebin. The script adds to a mercurial repo, which can then be converted to Bazaar.

# I hereby place this script into the Public Domain!
import os, sys
import time

import mwclient

import mercurial.ui
from mercurial import localrepo
from mercurial import commands

article = 'Love'
#start_time = None
start_time = '2011-01-01T00:00:00Z'

# set up mercurial repo
ui = mercurial.ui.ui()
repo_dir = article
repo = localrepo.localrepository(ui, path=repo_dir, create = not os.path.isdir(repo_dir))
#if not os.path.isdir(article):
#   os.mkdir(article)
#os.chdir(article)
print "rep in", repo.root
content_path = os.path.join(repo.root, article + '.wiki')

site = mwclient.Site('en.wikipedia.org')
page = site.Pages[article]

for rev in page.revisions(start=start_time, limit=50,dir='newer', prop='ids|timestamp|flags|comment|user|content'):
    content = rev['*']
    timestamp = time.asctime(rev['timestamp'])
    comment = rev['comment'].encode('utf8')
    if len(comment) == 0: comment = "blank"
    print "writing revision from", timestamp

    f = open(content_path, 'wb')
    f.write(content.encode('utf8'))
    f.close()

    commands.addremove(ui, repo)
    commands.commit(ui, repo, message=comment, user=rev['user'].encode('utf8'), date=timestamp)
4

Inspired by previous answers, I've adapted the Mercurial script (Pastebin) shared in @eug's solution to instead use Git as backend.

mediawiki2git * GitLab

Simple Python script to convert MediaWiki article history to Git commits.
gitlab.com/andreascian/mediawiki2git

import os
import time

import mwclient

from git import Repo
from git import Actor

article = 'AxelEVB-Lite'
start_time = '2011-01-01T00:00:00Z'

repo_dir = os.path.join(os.getcwd(), article)
repo = Repo.init(repo_dir)

file_name = article + ".wiki"
content_path = os.path.join(repo_dir, file_name)

# site = mwclient.Site('en.wikipedia.org')
site = mwclient.Site('wiki.dave.eu', path='/')
page = site.Pages[article]

print("repository created, now start getting revisions")

for rev in page.revisions(
        start=start_time, limit=50,
        dir='newer',
        prop='ids|timestamp|flags|comment|user|content'):

    content = rev['*']
    timestamp = time.asctime(rev['timestamp'])
    comment = rev['comment'].encode('utf8')
    if len(comment) == 0:
        comment = "blank comment"

    print "writing revision from", timestamp

    print("processig revision for " + timestamp)

    f = open(content_path, 'wb')
    f.write(content.encode('utf8'))
    f.close()

    index = repo.index
    index.add([file_name])

    actor = Actor(rev['user'].encode('utf8'), "[email protected]")
    index.commit(comment, author=actor, committer=actor)

print("done")
3

http://search.cpan.org/~daxim/Mediawiki-Blame-0.0.3/lib/Mediawiki/Blame.pm appears to support exactly this. Unfortunately, it's just a perl module, not a user-friendly command, so it's going to be a bit harder to use than one might want …

1
2

You might prefer to convert the history of the page to Git, and then use your favourite IDE/command-line tools to search the history.

Python Script

Save the following script in mwdump on your PATH:

#!/usr/bin/env python3
import os
import subprocess
import sys

import mwclient
from datetime import datetime
from time import mktime

print("getting page...")
site = mwclient.Site("en.wikipedia.org", scheme="https")
name = sys.argv[1]
page = site.pages[name]
    
    
def conv_time(t):
return datetime.fromtimestamp(mktime(t))


os.mkdir(name)
os.chdir(name)


subprocess.check_output(["git", "init"])
print("extracting revisions (may take a really long time, depending on the page)...")
for i, revision in enumerate(
    page.revisions(prop="ids|timestamp|flags|comment|user|content", dir="newer")
):
    if "*" in revision:
        timestamp = conv_time(revision["timestamp"])
        print(timestamp)
        with open("page", "w") as stream:
            stream.write(revision["*"])
        subprocess.check_output(["git", "add", "page"])
        subprocess.check_output(
            [
                "git",
                "commit",
                "--allow-empty",
                f"--date={timestamp.isoformat()}",
                "-m",
                f"{revision['user']} {revision['revid']}",
            ]
        )

Command

Run the following command which will create a directory called Main_Page containing the Git repository with page history:

mwdump Main_Page 
1
  • In os.mkdir(name) if page has parents (i.e. includes slashes in name), then here will be an error.
    – Ashark
    Sep 28 at 21:38
1

Another not very user friendly solution would be the mw-to-git extension for git, which would presumably allow you to run git blame on a page.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.