3

Is there a tool out there that will crawl one of your friend's status updates and download them to a file? I'm doing some research and it's a pain to keep hitting "older posts.."

6
  • 1
    "I'm doing some stalking" FTFY. Jokes aside, I don't know of any way to do this now that they stopped piping this stuff to RSS but maybe there's some backdoor functionality I'm unaware of.
    – Shinrai
    Commented Feb 8, 2011 at 23:14
  • I suggest you elaborate a bit about what you're specifically trying to do and since Super User isn't about websites, it should be more general regarding crawling any site.
    – Ivo Flipse
    Commented Feb 8, 2011 at 23:22
  • I would like to collect a friend's status updates for a period of time. Specifically, from the present back one year, I want to collect the text of the status updates from his profile.
    – Caveatrob
    Commented Feb 8, 2011 at 23:47
  • PS - Stalking would seem to be easier done with some other method. This is data collection.
    – Caveatrob
    Commented Feb 8, 2011 at 23:48
  • 1
    Facebook has an official API for retrieving user data. Any other scraping would be considered a violation of their terms of use. See developers.facebook.com for more info.
    – nhinkle
    Commented Feb 9, 2011 at 9:17

2 Answers 2

1

You may want to look at the questions on StackOverflow about the Facebook Graph API. Your solution for instance, may involve something like wget and a URL of the form (or similar): https://graph.facebook.com/USERNAME_HERE/feed.

E.g.: https://graph.facebook.com/cocacola/feed

In fact by examining the bottom previous/next links from this page, I can see that adding a query similar to ?limit=25 and perhaps even &since=2011-02-09T04 may be just what you need.

E.g.: https://graph.facebook.com/cocacola/feed?limit=100

3

Try fb-crawl.

It's a Perl script that will insert wall posts, user information, and friends into a MySQL database. From there you can sift through the data using MySQL SELECT statements. It doesn't make any API calls, so if you can see it on Facebook, fb-crawl can get it.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.