We are building a system to analyze the results of Google Alerts for certain medical topics.
We are testing two different methods. Let's take the topics pregnancy and vaccines in Italian. Our methods are:
vaccini e gravidanzawith results delivered by mail once per day.
~vaccini, ~gravidanzawith results delivered by RSS feed.
Our hypothesis is that query 1 should give a set of results that is a subset of query 2, since query 2 uses ~ to include the synonyms of the search terms.
However, we are getting different results. In one simulation along one month we found that just 7 results from query 1 were in common with query 2 from 18 total results for query 1. That's a ratio of similitude of ~37% while we expected 100%.
So what could be the problem? Is it normal that queries so similar return such different results? Or is it the delivery system that causes different results? Or are we misunderstanding the inner workings of Google and the system is not able to return a consistent set of results for a given query over a given time?
(Maybe this is not the right stackexchange site for such a question, feel free to move it)
UPDATE: I made a comparative search between "~vaccini, ~gravidanza" and "~gravidanza". results are discouraging... With 924 results from "~gravidanza" and 18 from "~vaccini, ~gravidanza", only 3 results are in common!