I've noticed that many paragraphs and article sections have been copied and pasted from one Wikipedia article to another, leading to excessive amounts of redundant text on Wikipedia. Do any tools, scripts, or APIs exist that would make it possible to automatically identify these duplicate sections and paragraphs (so that they can be removed)?
asked Aug 18, 2012 at 17:48
-
I want to do this programmatically because doing it manually would be extremely tedious. For example, I once found that the article "Nutrition in animals" was mostly copied and pasted from the "Nutrition" article, and now they have both been modified, they will need to be merged together again.– Anderson GreenAug 18, 2012 at 17:51
-
On Wikipedia, text duplication is not necessarily considered a bad thing by itself.– nicSep 19, 2013 at 3:10
Add a comment
|
1 Answer
I'm afraid there's not any way to do this using the API or anything like that. However, you could probably do something with the Wikimedia dumps to find the sort of duplication you're looking for. The people already doing research might also be able to help you out.
