Is there a tool out there that will crawl one of your friend's status updates and download them to a file? I'm doing some research and it's a pain to keep hitting "older posts.."

  • 1
    "I'm doing some stalking" FTFY. Jokes aside, I don't know of any way to do this now that they stopped piping this stuff to RSS but maybe there's some backdoor functionality I'm unaware of.
    – Shinrai
    Feb 8 '11 at 23:14
  • I suggest you elaborate a bit about what you're specifically trying to do and since Super User isn't about websites, it should be more general regarding crawling any site.
    – Ivo Flipse
    Feb 8 '11 at 23:22
  • I would like to collect a friend's status updates for a period of time. Specifically, from the present back one year, I want to collect the text of the status updates from his profile.
    – Caveatrob
    Feb 8 '11 at 23:47
  • PS - Stalking would seem to be easier done with some other method. This is data collection.
    – Caveatrob
    Feb 8 '11 at 23:48
  • 1
    Facebook has an official API for retrieving user data. Any other scraping would be considered a violation of their terms of use. See developers.facebook.com for more info.
    – nhinkle
    Feb 9 '11 at 9:17

You may want to look at the questions on StackOverflow about the Facebook Graph API. Your solution for instance, may involve something like wget and a URL of the form (or similar): https://graph.facebook.com/USERNAME_HERE/feed.

E.g.: https://graph.facebook.com/cocacola/feed

In fact by examining the bottom previous/next links from this page, I can see that adding a query similar to ?limit=25 and perhaps even &since=2011-02-09T04 may be just what you need.

E.g.: https://graph.facebook.com/cocacola/feed?limit=100


Try fb-crawl.

It's a Perl script that will insert wall posts, user information, and friends into a MySQL database. From there you can sift through the data using MySQL SELECT statements. It doesn't make any API calls, so if you can see it on Facebook, fb-crawl can get it.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.