1

I've noticed that many paragraphs and article sections have been copied and pasted from one Wikipedia article to another, leading to excessive amounts of redundant text on Wikipedia. Do any tools, scripts, or APIs exist that would make it possible to automatically identify these duplicate sections and paragraphs (so that they can be removed)?

  • I want to do this programmatically because doing it manually would be extremely tedious. For example, I once found that the article "Nutrition in animals" was mostly copied and pasted from the "Nutrition" article, and now they have both been modified, they will need to be merged together again. – Anderson Green Aug 18 '12 at 17:51
  • On Wikipedia, text duplication is not necessarily considered a bad thing by itself. – nic Sep 19 '13 at 3:10
3

I'm afraid there's not any way to do this using the API or anything like that. However, you could probably do something with the Wikimedia dumps to find the sort of duplication you're looking for. The people already doing research might also be able to help you out.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.