Tell me more ×
Web Applications Stack Exchange is a question and answer site for power users of web applications. It's 100% free, no registration required.
up vote 4 down vote favorite
1

I am building a small news aggregation service where in it collects news from some 1600 sources across the web and analyses the retrieved articles. (I only fetch rss feeds.) In this regard, I want to use Google Reader as a platform. Meaning, My application fetches all the feedentries from a single source(Google Reader account) and my Google Reader account collects the feedentries from the 1600 sources I subscribe to. Can I use Google Reader for commercial usage like this? I have already raised a question in Google Reader forums and looking for an answer. If anyone here have used Google Reader for a similar usage, please let me know.

share|improve this question
If you don't get a positive answer to Google Reader, then note that there are other feed aggregation services out there that may work differently for you, Bloglines and Feedburner spring to mind, but I don't know how well either work as a platform. – GAThrawn Jul 28 '10 at 13:32
superfeedr.com is another alternative to look at. – Craig Aug 4 '10 at 1:21
I think you need to rephrase what you're looking for. If you're "collecting" the data from the feeds, you're doing exactly what Google Reader already does. It sounds like you're looking for a service that is always running and collecting the data and your application/service will then perform analysis of that data. Assuming that each use of your app/service will be going off the same sources, trying to use Google Reader (which is meant for end user reading of RSS) is like trying to fit a very big square into a round hole. – Agent_9191 Aug 17 '10 at 2:36
You might find out how feedly.com works. – Grayside Aug 18 '10 at 20:39

1 Answer

up vote 6 down vote

Does interfacing with Reader provide processing improvements over pulling the feeds directly?

If you're analyzing (and enriching?) feeds, I'm guessing you're running something like Calais and Solr as part of your processing stack. With 1600 feeds you're talking about a serious number of entries to deal with. Does Reader provide enough bandwidth to pull the entries fast enough to keep your service relevant?

I've gotta imagine having your processing and enriching machines virtualized on scalable cloud space (an EC2 account or something) and setting a processing schedule would be much more efficient than drudging such a massive stream of posts from Google.

That aside, the Google Terms of Service has a few entries which may be bad news:

5.3 You agree not to access (or attempt to access) any of the Services by any means other than through the interface that is provided by Google, unless you have been specifically allowed to do so in a separate agreement with Google. You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers) and shall ensure that you comply with the instructions set out in any robots.txt file present on the Services.

If you use scripts to crawl your 1600-feed account, that may be a violation.

5.4 You agree that you will not engage in any activity that interferes with or disrupts the Services (or the servers and networks which are connected to the Services).

Making Google parse and process your feeds, then taking up that bandwidth, might interfere with the normal operation of Reader.

5.5 Unless you have been specifically permitted to do so in a separate agreement with Google, you agree that you will not reproduce, duplicate, copy, sell, trade or resell the Services for any purpose.

This last one seems to speak pretty directly to your question.

share|improve this answer
Guess that should be the answer to the question. You need to have a separate agreement with Google if you intend to use the output for commercial purposes. – thunderror Aug 18 '10 at 8:30
Thats an awesome answer :D, looks like I need to revisit a few things. – None-da Aug 19 '10 at 3:31
I will say, the amount of information you're looking to process is enormous. We looked at managing even 100 feeds for a newspaper we were supporting last year, and even that started to get expensive considering the necessary infrastructure to do all the enriching. – Matt Aug 20 '10 at 5:07

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.