Gmail bulk Emails manipulation

Question

I am trying to deal with 600,000+ emails in inbox, my goal is to either delete or archive emails before 1/1/2021.

Doing a search and select all search results and then delete or archive as most people know will result in the action being applied to a couple of thousands emails at a time 2,000-6,000 sometimes as high as 25,000 but it's not consistent.

I tried to use the script provided here and it worked for like 200,000 emails then I reached the "Service invoked too many times for one day" Which means That I hit the daily limit for execution, when I tried in another day I got the "Exceeded maximum execution time" error after a short while and only about 5,000 emails were processed.

The script I am using

function batchArchiveEmail() {
  var batchSize = 100 // Process up to 100 threads at once
  var searchSize = 500 // Limit search result to a max of 500 threads.
  //Use this if you encounter the "Exceeded maximum execution time" error "funny enough it didn't help that much".
  
  var threads = GmailApp.search('label:inbox before:2021/1/1', 0, searchSize);
  for (j = 0; j < threads.length; j+=batchSize) {
    GmailApp.moveThreadsToArchive(threads.slice(j, j+batchSize));
  }
}

In the comments user "acm" mentioned the usage of 'paged' call "Google's preferred method", but unfortunately didn't post his script.

Google Apps script doesn't have a detailed documentation about this call and all I could find was this answer, but I fail to understand how to implement/ use this call to process emails based on date since it appears to me it retrieves a list of emails starting from the most recent and I can't figure out how to use it along with a date like user "acm" did.

If anyone have another way to achieve what I want to do please do tell.

Answer 1

Try the following script

/**
 * Creates the first trigger to call batchArchiveEmail. 
 */
function init(){
   ScriptApp
    .newTrigger('batchArchiveEmail')
    .timeBased()
    .after(60 * 1000)
    .create();
    console.log(`trigger created`)
}

/**
 * Archive emails by batches preventing controlling limiting the execution time and  
 * creating a trigger if there are still threads pending to be archived.
 */ 
function batchArchiveEmail(){
  const start = Date.now();
  /** 
   * Own execution time limit for the search and archiving operations to prevent an 
   * uncatchable error. As the execution time check is done in do..while condition there  
   * should be enough time to one search and archive operation and to create a trigger 
   * to start a new execution. 
   */ 
  const maxTime = 25 * 60 * 1000; // Instead of 25 use 3 for Google free accounts
  const batchSize = 100;
  let threads, elapsedTime;
  /** Search and archive threads, then repeat until the search returns 0 threads or the 
   * maxTime is reached
   */ 
  do {
    threads = GmailApp.search('label:inbox before:2021/1/1');
    // console.log(`${threads[0] ? threads[0].getFirstMessageSubject() : 'No more threads'}`)
    for (let j = 0; j < threads.length; j += batchSize) {
      GmailApp.moveThreadsToArchive(threads.slice(j, j + batchSize));
    };
    /**
     * Used to prevent to have too many calls in a short time, might not be 
     * necessary with a large enough batchSize
     */
    Utilities.sleep(`2000`); 
    elapsedTime = Date.now() - start;
  } while (threads.length > 0 &&  elapsedTime < maxTime);
  if(threads.length > 0){
    /** Delete the last trigger */
    deleteTriggers();

    /** Create a one-time new trigger */
    ScriptApp
    .newTrigger('batchArchiveEmail')
    .timeBased()
    .after(60 * 1000)
    .create();
    console.log(`trigger created`)
  } else {
    /** Delete the last trigger */
    deleteTriggers();
    console.log(`No more threads to archive`);
  }
}

/**
 * As there is a limit on the number of triggers that a user might have for each 
 * project, delete the triggers.
 */
function deleteTriggers(){
  var triggers = ScriptApp.getProjectTriggers();
  for (var i = 0; i < triggers.length; i++) {
  ScriptApp.deleteTrigger(triggers[i]);
  }
}

Please note the use of do...while, if statement that creates a time-driven trigger, and the deleteTriggers function which is needed to prevent the archiving process being interrupted by a "This script has too many triggers" error.

---

Your are trying to find how to handle a large number email threads / messages and you get stuck on how to deal with Google Apps Script quotas.

Regarding "Service using too much computer time for one day" (from questions's rev 1), according to the documentation, this error usually is caused by scripts being called by a trigger but might also get this error when running manually computer time intensive scripts.

Unfortunately there is no way to get more computer time for the account that got this message, the only workaround is to use a second account, but this will lead to use a more complex script, starting by by using UrlFetchApp Service to call the Gmail API instead of the Google Apps Script Gmail Service and the Gmail Advanced Service, and implies to use Google OAuth. There is GitHub repository / library published by Google to help with this but it's very likely that this is beyond of what you are looking, so it's very likely that the best will be to 1, improve the efficiency of your script, and 2 use a time-driven trigger and appropriate algorithm to run your script daily until all the messages were processed.

NOTE: It looks that is not feasible to migrate the account to a Google Workspace Account which have bigger quotas. I have not reviewed yet Google Workspace Individual plan, I'm wondering if it offers to increase the Google Apps Script quotas.

Regarding "Service invoked too many times for one day", according to the documentation the most common cause is to call the function from a Google Sheets custom function. It's unclear why is this happening to the user who reported this to you.

Regarding using "'paged' call", this means to use search(query, start, max) instead of search(query) which is already used on your code but it's not taking advantage of how it works. The script above is not using it because I found that this is not really needed as nowadays `search(query) returns a maximum of 500 threads.