2

When I run a query for "hers" in Google Ngram Viewer, I get back the word's frequency of occurrence as a percentage. We know the outcome percentage; what's the denominator on the other side? Is it 100 million words? More?

2

Currently the total number of words is 468,491,999,592. (Approaching half a trillion)

You can obtain that figure by downloading the latest dataset.
In that dataset, the total_counts file(txt) lists the number of unigrams, pages, and books for each year.
Summing up the unigram counts yields the 468,491,999,592 figure.

Important notes:

  1. Currently (Nov 2015), the latest Ngram data is the Version 20120701 set. It was compiled in 2012, but covers books from 1505 to 2008.

  2. A unigram is mostly the same as a word. Details of Google's parsing may yield differences in (hopefully) rare cases.

  3. Only words within sentences are counted. Reference: Syntactic Annotations for the Google Books Ngram Corpus(PDF), section 3.2.

  4. The percent displayed on the graph is normalized per year. This means that there is no one "denominator" if you are trying to figure the real counts of a given word. (But you can get that data from the downloaded datasets.) See below.


How the graph is calculated:

Consider the word "circumvallate" (all lower case). From the data download, we look up the 1978 and 1979 figures and get:

Word            Count in 1978 books   Count in 1979 books
-------------   -------------------   -------------------
circumvallate                  335                   261
*All words*          4,365,839,878         4,528,331,460

Which calculates out as:

  • 0.0000076732% in 1978
  • 0.0000057637% in 1979

If we then look at the corresponding graph, and hover over the respective points, we get:

Sample graph

Which agrees with what we calculate.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.