Is there a way to show "who contributed to this line" of each line of a page of Wikipedia or of a MediaWiki site?
It would show per page and would be similar to the subversion blame tool.
Web Applications Stack Exchange is a question and answer site for power users of web applications. It only takes a minute to sign up.
Sign up to join this communityIs there a way to show "who contributed to this line" of each line of a page of Wikipedia or of a MediaWiki site?
It would show per page and would be similar to the subversion blame tool.
I often need something like this as well, but it looks like there's no ready solution.
What I've done is written a script that helps me grab revisions using the MediaWiki API and import them to the Bazaar version control system. bzr qblame article.wiki
, then gives a nice view of who changed what. The script is not really ready for release, but you can find it below or on Pastebin. The script adds to a mercurial repo, which can then be converted to Bazaar.
# I hereby place this script into the Public Domain!
import os, sys
import time
import mwclient
import mercurial.ui
from mercurial import localrepo
from mercurial import commands
article = 'Love'
#start_time = None
start_time = '2011-01-01T00:00:00Z'
# set up mercurial repo
ui = mercurial.ui.ui()
repo_dir = article
repo = localrepo.localrepository(ui, path=repo_dir, create = not os.path.isdir(repo_dir))
#if not os.path.isdir(article):
# os.mkdir(article)
#os.chdir(article)
print "rep in", repo.root
content_path = os.path.join(repo.root, article + '.wiki')
site = mwclient.Site('en.wikipedia.org')
page = site.Pages[article]
for rev in page.revisions(start=start_time, limit=50,dir='newer', prop='ids|timestamp|flags|comment|user|content'):
content = rev['*']
timestamp = time.asctime(rev['timestamp'])
comment = rev['comment'].encode('utf8')
if len(comment) == 0: comment = "blank"
print "writing revision from", timestamp
f = open(content_path, 'wb')
f.write(content.encode('utf8'))
f.close()
commands.addremove(ui, repo)
commands.commit(ui, repo, message=comment, user=rev['user'].encode('utf8'), date=timestamp)
Inspired by previous answers, I've adapted the Mercurial script (Pastebin) shared in @eug's solution to instead use Git as backend.
Simple Python script to convert MediaWiki article history to Git commits.
gitlab.com/andreascian/mediawiki2git
import os
import time
import mwclient
from git import Repo
from git import Actor
article = 'AxelEVB-Lite'
start_time = '2011-01-01T00:00:00Z'
repo_dir = os.path.join(os.getcwd(), article)
repo = Repo.init(repo_dir)
file_name = article + ".wiki"
content_path = os.path.join(repo_dir, file_name)
# site = mwclient.Site('en.wikipedia.org')
site = mwclient.Site('wiki.dave.eu', path='/')
page = site.Pages[article]
print("repository created, now start getting revisions")
for rev in page.revisions(
start=start_time, limit=50,
dir='newer',
prop='ids|timestamp|flags|comment|user|content'):
content = rev['*']
timestamp = time.asctime(rev['timestamp'])
comment = rev['comment'].encode('utf8')
if len(comment) == 0:
comment = "blank comment"
print "writing revision from", timestamp
print("processig revision for " + timestamp)
f = open(content_path, 'wb')
f.write(content.encode('utf8'))
f.close()
index = repo.index
index.add([file_name])
actor = Actor(rev['user'].encode('utf8'), "[email protected]")
index.commit(comment, author=actor, committer=actor)
print("done")
http://search.cpan.org/~daxim/Mediawiki-Blame-0.0.3/lib/Mediawiki/Blame.pm appears to support exactly this. Unfortunately, it's just a perl module, not a user-friendly command, so it's going to be a bit harder to use than one might want …
You might prefer to convert the history of the page to Git, and then use your favourite IDE/command-line tools to search the history.
Save the following script in mwdump
on your PATH
:
#!/usr/bin/env python3
import os
import subprocess
import sys
import mwclient
from datetime import datetime
from time import mktime
print("getting page...")
site = mwclient.Site("en.wikipedia.org", scheme="https")
name = sys.argv[1]
page = site.pages[name]
def conv_time(t):
return datetime.fromtimestamp(mktime(t))
os.mkdir(name)
os.chdir(name)
subprocess.check_output(["git", "init"])
print("extracting revisions (may take a really long time, depending on the page)...")
for i, revision in enumerate(
page.revisions(prop="ids|timestamp|flags|comment|user|content", dir="newer")
):
if "*" in revision:
timestamp = conv_time(revision["timestamp"])
print(timestamp)
with open("page", "w") as stream:
stream.write(revision["*"])
subprocess.check_output(["git", "add", "page"])
subprocess.check_output(
[
"git",
"commit",
"--allow-empty",
f"--date={timestamp.isoformat()}",
"-m",
f"{revision['user']} {revision['revid']}",
]
)
Run the following command which will create a directory called Main_Page
containing the Git repository with page history:
mwdump Main_Page
os.mkdir(name)
if page has parents (i.e. includes slashes in name), then here will be an error.
Another not very user friendly solution would be the mw-to-git extension for git, which would presumably allow you to run git blame
on a page.