I need a google apps script which can do the following:
if I send an email to my alias email, say to myemail+expenses#gmail.com, it adds a line in a specific google spreadsheet with data from that email. E.g. timestamp, subject, the first line of a body - whatever.
I found interesting article describing similar process of sending data to google spreadsheet.
May be there is no any direct way of doing that, however, it should exist some workaround.
P.S. It should be done only by using a google echosystem. No php, servers etc.
There are two step for this.
#1
At first create a trigger to run the mail checker.
function switchTrigger() {
var isExist = false;
var triggers = ScriptApp.getProjectTriggers();
for (var i = 0; i < triggers.length; i++) {
if ( /*<CONDITION>*/ ) {
isExist = true;
ScriptApp.deleteTrigger(triggers[i]);
}
}
if (!isExist) {
ScriptApp.newTrigger( /*<CONDITION>*/ ).create();
Logger.log('create');
}
}
#2
Then you have to check emails. Something like that
function checkMail() {
var sh = SpreadsheetApp.openById(properties.targetSheetId).getSheets()[0];
var query = /* properties.queryString */ ;
var threads = GmailApp.search(query);
if (threads.length < 1) return;
for (var i = 0; i < threads.length; i++) {
var messages = threads[i].getMessages();
for (var j = 0; j < messages.length; j++) {
if (messages[j].isStarred()) {
sh.appendRow([
new Date(),
messages[j].getFrom(),
messages[j].getPlainBody()
]);
messages[j].unstar();
}
}
}
}
Be careful. You have to set up Gmail filters so that all incoming to myemail+expenses#gmail.com would be marked with an star at the beginning.
Working Example
I wrote the code to update my table (SecurityQuestionAnswer) with new security password questions and move to old questions to another table (SecurityQuestionAnswersArchives). Total no of security questions is 3. I am able to update the current table, but when I add the same rows to history table, it shows weird data: only two records are added instead of 3 and the data is also duplicated. My code is as follows:
if (oldQuestions.Any())
{
var oldquestionstoarchivelist = new List<SecurityQuestionAnswersArchives>();
var oldquestionstoarchive =new SecurityQuestionAnswersArchives();
for (int i = 0; i < 3; i++)
{
oldquestionstoarchive.Id = oldQuestions[i].Id;
oldquestionstoarchive.SecurityQuestionId = oldQuestions[i].SecurityQuestionId;
oldquestionstoarchive.Answer = oldQuestions[i].Answer;
oldquestionstoarchive.UpdateDate = oldQuestions[i].UpdateDate;
oldquestionstoarchive.IpAddress = oldQuestions[i].IpAddress;
oldquestionstoarchive.SecurityQuestion = oldQuestions[i].SecurityQuestion;
oldquestionstoarchive.User = oldQuestions[i].User;
oldquestionstoarchivelist.Add(oldquestionstoarchive);
}
user.SecurityQuestionAnswersArchives = oldquestionstoarchivelist;
//await Store.UpdateAsync(user);
_dbContext.ArchiveSecurityQuestionAnswers.AddRange(oldquestionstoarchivelist);
_dbContext.SecurityQuestionAnswers.RemoveRange(oldQuestions);
await _dbContext.SaveChangesAsync();
oldquestionstoarchivelist.Clear();
}
UPDATE 1
The loop looks fine, It iterates three times(0,1,2), which is expected. First issue is with AddRange function to which I was passing a list , but it takes an IEnumerable input, I rectified it using following code.
IEnumerable<SecurityQuestionAnswersArchives> finalArchiveses = oldquestionstoarchivelist;
_dbContext.ArchiveSecurityQuestionAnswers.AddRange(finalArchiveses);
The other issue is duplicate data , which I am unable to figure out where the issue is. Please help me in finding this out.
Your help is much appreciated !
Got it ! Just sharing in case anybody has same issue.
The problem was with initialization at wrong place. I moved
var oldquestionstoarchive =new SecurityQuestionAnswersArchives();
in side the Forloop, now the variable will hold the unique values over each iteration.
var oldquestionstoarchivelist = new List<SecurityQuestionAnswersArchives>();
for (int i = 0; i < 3; i++)
{
var oldquestionstoarchive = new SecurityQuestionAnswersArchives();
oldquestionstoarchive.SecurityQuestionId = oldQuestions[i].SecurityQuestionId;
oldquestionstoarchive.Answer = oldQuestions[i].Answer;
oldquestionstoarchive.UpdateDate = oldQuestions[i].UpdateDate;
oldquestionstoarchive.IpAddress = oldQuestions[i].IpAddress;
oldquestionstoarchive.SecurityQuestion = oldQuestions[i].SecurityQuestion;
oldquestionstoarchive.User = oldQuestions[i].User;
oldquestionstoarchivelist.Add(oldquestionstoarchive);
}
I've tried Observable.Create
waits to finish before showing any results.
Possibly because the example I'm trying to follow is a changing live value, not a changing live collection.
and
ObservableCollection<FileAnalysisResult> fileAnalysisResults = new ObservableCollection<FileAnalysisResult>();
I can't seem to apply because .DumpLive() isn't applicable to an ObservableCollection.
Short answer: use LINQPad's DumpContainer:
var dc = new DumpContainer().Dump();
for (int i = 0; i < 100; i++)
{
dc.Content = i;
Thread.Sleep(100);
}
Long answer: DumpContainer writes to LINQPad's standard HTML results window, so you can see the value change in place while the main thread is blocked, whereas calling DumpLive on an IObservable uses a WPF control to render the updates, so the main thread must remain unblocked in order to see updates as they occur.
It's also possible to dump a WPF or Windows Forms control and update it in place:
var txt = new TextBox().Dump();
for (int i = 0; i < 100; i++)
{
txt.Text = i.ToString();
await Task.Delay(100);
}
Just as with DumpLive, you must be careful not to block the main thread. If you replaced await Task.Delay with Thread.Sleep, you'd block the UI thread and nothing would appear until the end.
Gmail has a issue where conversation labels are not applied to new messages that arrive in the conversation thread. issue details here
We found a Google Apps Script that fixes the labels on individual messages in the Gmail Inbox to address this issue. The script is as follows:
function relabeller() {
var labels = GmailApp.getUserLabels();
for (var i = 0; i < labels.length; i++) {
Logger.log("label: " + i + " " + labels[i].getName());
var threads = labels[i].getThreads(0,100);
for (var j = 1; threads.length > 0; j++) {
Logger.log( (j - 1) * 100 + threads.length);
labels[i].addToThreads(threads);
threads = labels[i].getThreads(j*100, 100);
}
}
}
However this script times out on email boxes with more than 20,000 messages due to the 5 mins execution time limit on Google Apps Script.
Can anyone please suggest a way to optimize this script so that it doesn't timeout?
OK, I've been working on this for a few days because I was really frustrated with the strange way that Gmail labels/doesn't label messages in conversations.
I'm flabbergasted actually that labels aren't automatically applied to new messages in a conversation. This is not reflected at all in the Gmail UI. There's no way to look at a thread and determine that the labels only apply to some messages in the thread, and you cannot add labels to a single message in the UI. As I was working through my script below, I noticed that you can't even programmatically add labels to a single message. So there really is no reason for the current behavior.
With my rant out of the way, I have a few notes about the script.
I sort of combined Saqib's code with Serge's code.
The script has two parts: an initial run that relabels all threads that have a user label attached, and a maintenance run that labels recent emails (currently looks back 4 days). Only one part executes during a single run. Once the initial run is completed, only the maintenance part will run. You can set a trigger to it run once per day, or more or less often, depending on your needs.
The initial run halts after 4 minutes to avoid being terminated by the 5 minute script time limit. It sets a trigger to run again after 4 minutes (both of these times can be changed using constants in the script). The trigger gets deleted at the next run.
There is no run-time check in the maintenance section. If you have lots of emails in the last 4 days, the maintenance section might hit the script time limit. I could probably change the script to be more efficient here, but so far it's worked for me so I am not really motivated to improve on it.
There's a try/catch statement in the initial run to try to catch the Gmail "write quota error" and exit gracefully (i.e. writing the current progress so it can be picked up again later), but I don't know if it works because I couldn't get the error to happen.
You'll get an email when the time limit is reached, and when the initial run is finished.
For some reason, the log doesn't always clear fully between runs, even when using the Logger.clear() command. So the status logs that it emails to the user have more than just the most recent run info. I don't know why this occurs.
I have used this to process 20,000 emails in around half an hour (including wait times). I actually ran it twice, so it processed 40,000 emails in one day. I guess the Gmail read/write limit of 10,000 isn't what is being applied here (maybe applying a label to 100 threads at a time counts as a single write event instead of 100?). It gets through about 5,000 threads in a 4 minute run, according to the status email it sends.
Sorry for the long lines. I blame the widescreen monitors. Let me know what you think!
function relabelGmail() {
var startTime= (new Date()).getTime(); // Time at start of script
var BATCH=100; // total number of threads to apply label to at once.
var LOOKBACKDAYS=4; // Days to look back for maintenance section of script. Should be at least 2
var MAX_RUN_TIME=4*60*1000; // Time in ms for max execution. 4 minutes is a good start.
var WAIT_TIME=4*60*1000; // Time in ms to wait before starting the script again.
Logger.clear();
// ScriptProperties.deleteAllProperties(); return; // Uncomment this line and run once to start over completely
if(ScriptProperties.getKeys().length==0){ // this is to create keys on the first run
ScriptProperties.setProperties({'itemsProcessed':0, 'initFinished':false, 'lastrun':'20000101', 'itemsProcessedToday':0,
'currentLabel':'null-label-NOTREAL', 'currentLabelStart':0, 'autoTrig':0, 'autoTrigID':'0'});
}
var itemsP = Number(ScriptProperties.getProperty('itemsProcessed')); // total counter
var initTemp = ScriptProperties.getProperty('initFinished'); // keeps track of when initial run is finished.
var initF = (initTemp.toLowerCase() == 'true'); // Make it boolean
var lastR = ScriptProperties.getProperty('lastrun'); // String of date corresponding to itemsProcessedToday in format yyyymmdd
var itemsPT = Number(ScriptProperties.getProperty('itemsProcessedToday')); // daily counter
var currentL = ScriptProperties.getProperty('currentLabel'); // Label currently being processed
var currentLS = Number(ScriptProperties.getProperty('currentLabelStart')); // Thread number to start on
var autoT = Number(ScriptProperties.getProperty('autoTrig')); // Number to say whether the last run made an automatic trigger
var autoTID = ScriptProperties.getProperty('autoTrigID'); // Unique ID of last written auto trigger
// First thing: google terminates scripts after 5 minutes.
// If 4 minutes have passed, this script will terminate, write some data,
// and create a trigger to re-schedule itself to start again in a few minutes.
// If an auto trigger was created last run, it is deleted here.
if (autoT) {
var allTriggers = ScriptApp.getProjectTriggers();
// Loop over all triggers. If trigger isn't found, then it must have ben deleted.
for(var i=0; i < allTriggers.length; i++) {
if (allTriggers[i].getUniqueId() == autoTID) {
// Found the trigger and now delete it
ScriptApp.deleteTrigger(allTriggers[i]);
break;
}
}
autoT = 0;
autoTID = '0';
}
var today = dateToStr_();
if (today == lastR) { // If new day, reset daily counter
// Don't do anything
} else {
itemsPT = 0;
}
if (!initF) { // Don't do any of this if the initial run has been completed
var labels = GmailApp.getUserLabels();
// Find position of last label attempted
var curLnum=0;
for ( ; curLnum < labels.length; curLnum++) {
if (labels[curLnum].getName() == currentL) {break};
}
if (curLnum == labels.length) { // If label isn't found, start over at the beginning
curLnum = 0;
currentLS = 0;
itemsP=0;
currentL=labels[0].getName();
}
// Now start working through the labels until the quota is hit.
// Use a try/catch to stop execution if your quota has been hit.
// Google can actually automatically email you, but we need to clean up a bit before terminating the script so it can properly pick up again tomorrow.
try {
for (var i = curLnum; i < labels.length; i++) {
currentL = labels[i].getName(); // Next label
Logger.log('label: ' + i + ' ' + currentL);
var threads = labels[i].getThreads(currentLS,BATCH);
for (var j = Math.floor(currentLS/BATCH); threads.length > 0; j++) {
var currTime = (new Date()).getTime();
if (currTime-startTime > MAX_RUN_TIME) {
// Make the auto-trigger
autoT = 1; // So the auto trigger gets deleted next time.
var autoTrigger = ScriptApp.newTrigger('relabelGmail')
.timeBased()
.at(new Date(currTime+WAIT_TIME))
.create();
autoTID = autoTrigger.getUniqueId();
// Now write all the values.
ScriptProperties.setProperties({'itemsProcessed':itemsP, 'initFinished':initF, 'lastrun':today, 'itemsProcessedToday':itemsPT,
'currentLabel':currentL, 'currentLabelStart':currentLS, 'autoTrig':autoT, 'autoTrigID':autoTID});
// Send an email
var emailAddress = Session.getActiveUser().getEmail();
GmailApp.sendEmail(emailAddress, 'Relabel job in progress', 'Your Gmail Relabeller has halted to avoid termination due to excess ' +
'run time. It will run again in ' + WAIT_TIME/1000/60 + ' minutes.\n\n' + itemsP + ' threads have been processed. ' + itemsPT +
' have been processed today.\n\nSee the log below for more information:\n\n' + Logger.getLog());
return;
} else {
// keep on going
var len = threads.length;
Logger.log( j * BATCH + len);
labels[i].addToThreads(threads);
currentLS = currentLS + len;
itemsP = itemsP + len;
itemsPT = itemsPT + len;
threads = labels[i].getThreads( (j+1) * BATCH, BATCH);
}
}
currentLS = 0; // Reset LS counter
}
initF = true; // Initial run is done
} catch (e) { // Clean up and send off a notice.
// Write current values back to ScriptProperties
ScriptProperties.setProperties({'itemsProcessed':itemsP, 'initFinished':initF, 'lastrun':today, 'itemsProcessedToday':itemsPT,
'currentLabel':currentL, 'currentLabelStart':currentLS, 'autoTrig':autoT, 'autoTrigID':autoTID});
var emailAddress = Session.getActiveUser().getEmail();
var errorDate = new Date();
GmailApp.sendEmail(emailAddress, 'Error "' + e.name + '" in Google Apps Script', 'Your Gmail Relabeller has failed in the following stack:\n\n' +
e.stack + '\nThis may be due to reaching your daily Gmail read/write quota. \nThe error message is: ' +
e.message + '\nThe error occurred at the following date and time: ' + errorDate + '\n\nThus far, ' +
itemsP + ' threads have been processed. ' + itemsPT + ' have been processed today. \nSee the log below for more information:' +
'\n\n' + Logger.getLog());
return;
}
// Write current values back to ScriptProperties. Send completion email.
ScriptProperties.setProperties({'itemsProcessed':itemsP, 'initFinished':initF, 'lastrun':today, 'itemsProcessedToday':itemsPT,
'currentLabel':currentL, 'currentLabelStart':currentLS, 'autoTrig':autoT, 'autoTrigNumber':autoTID});
var emailAddress = Session.getActiveUser().getEmail();
GmailApp.sendEmail(emailAddress, 'Relabel job completed', 'Your Gmail Relabeller has finished its initial run.\n' +
'If you continue to run the script, it will skip the initial run and instead relabel ' +
'all emails from the previous ' + LOOKBACKDAYS + ' days.\n\n' + itemsP + ' threads were processed. ' + itemsPT +
' were processed today. \nSee the log below for more information:' + '\n\n' + Logger.getLog());
return; // Don't run the maintenance section after initial run finish
} // End initial run section statement
// Below is the 'maintenance' section that will be run when the initial run is finished. It finds all new threads
// (as defined by LOOKBACKDAYS) and applies any existing labels to all messages in each thread. Note that this
// won't miss older threads that are labeled by the user because all messages in a thread get the label
// when the label action is first performed. If another message is then sent or received in that thread,
// then this maintenance section will find it because it will be deemed a "new" thread at that point.
// You may need to search further back the first time you run this if it took more than 3 days to finish
// the initial run. For general maintenance, though, 4 days should be plenty.
// Note that I have not implemented a script-run-time check for this section.
var threads = GmailApp.search('newer_than:' + LOOKBACKDAYS + 'd', 0, BATCH); //
var len = threads.length;
for (var i=0; len > 0; i++) {
for (var t = 0; t < len; t++) {
var labels = threads[t].getLabels();
for (var l = 0; l < labels.length; l++) { // Add each label to the thread
labels[l].addToThread(threads[t]);
}
}
itemsP = itemsP + len;
itemsPT = itemsPT + len;
threads = GmailApp.search('newer_than:' + LOOKBACKDAYS + 'd', (i+1) * BATCH, BATCH);
len = threads.length;
}
// Write the property data
ScriptProperties.setProperties({'itemsProcessed':itemsP, 'initFinished':initF, 'lastrun':today, 'itemsProcessedToday':itemsPT,
'currentLabel':currentL, 'currentLabelStart':currentLS, 'autoTrig':autoT, 'autoTrigID':autoTID});
}
// Takes a date object and turns it into a string of form yyyymmdd
function dateToStr_(dateObj) { //takes in a date object, but uses current date if not a date
if (!(dateObj instanceof Date)) {
dateObj = new Date();
}
var dd = dateObj.getDate();
var mm = dateObj.getMonth()+1; //January is 0!
var yyyy = dateObj.getFullYear();
if(dd<10){dd='0'+dd};
if(mm<10){mm='0'+mm};
dateStr = ''+yyyy+mm+dd;
return dateStr;
}
Edit: 3/24/2017
I guess I should turn on notifications or something, because I never saw the question from user29020. In case anyone ever has the same question, here's what I do: I run it as a maintenance function by setting a daily trigger to run each night between 1 and 2 AM.
An additional note: It seems that at some point in the last year or so, labeling calls to Gmail have slowed down significantly. It now takes around 0.2 seconds per thread, so I would expect an initial run of 20k emails to take at least 20 runs or so before it makes it all the way through. This also means that if you typically receive more than 100-200 emails a day, the maintenance section might also start to take too long and start to fail. Now that's a lot of emails, but I bet there are some people that receive that many, and it seems much more likely that you would hit that than the 1000 or so daily emails that would have been needed for failure back when I first wrote the script.
Anyway, one mitigation would be to reduce the LOOKBACKDAYS to less than 4, but I wouldn't recommend putting it less than 2.
From the documentation :
method getInboxThreads()
Retrieve all Inbox threads irrespective of labels
This call will fail when the size of all threads is too large for the system to handle. Where the thread size is unknown, and potentially very large, please use the 'paged' call, and specify ranges of the threads to retrieve in each call.*
So you should handle a certain number of threads, label the messages and set up a time trigger to run each "page" every 10 minutes or so until all the messages are labelled.
EDIT : I have given this a try , please consider as a draft to start with :
The script will process 100 threads at a time and send you an email to inform you on its progress and show the log.
When it's finished it will warn you with an email as well. It uses scriptProperties to store its state. (don't forget to update the mail adress at the end of the script). I tried it with a time trigger set to 5 minutes and it seems to run smoothly for now...
function inboxLabeller() {
if(ScriptProperties.getKeys().length==0){ // this is to create keys on the first run
ScriptProperties.setProperties({'threadStart':0, 'itemsprocessed':0, 'notF':true})
}
var items = Number(ScriptProperties.getProperty('itemsprocessed'));// total counter
var tStart = Number(ScriptProperties.getProperty('threadStart'));// the value to start with
var notFinished = ScriptProperties.getProperty('notF');// the "main switch" ;-)
Logger.clear()
while (notFinished){ // the main loop
var threads = GmailApp.getInboxThreads(tStart,100);
Logger.log('Number of threads='+Number(tStart+threads.length));
if(threads.length==0){
notFinished=false ;
break
}
for(t=0;t<threads.length;++t){
var mCount = threads[t].getMessageCount();
var mSubject = threads[t].getFirstMessageSubject();
var labels = threads[t].getLabels();
var labelsNames = '';
for(var l in labels){labelsNames+=labels[l].getName()}
Logger.log('subject '+mSubject+' has '+mCount+' msgs with labels '+labelsNames)
for(var l in labels){
labels[l].addToThread(threads[t])
}
}
tStart = tStart+100;
items = items+100
ScriptProperties.setProperties({'threadStart':tStart, 'itemsprocessed':items})
break
}
if(notFinished){
GmailApp.sendEmail('mymail', 'inboxLabeller progress report', 'Still working, '+items+' processed \n - see logger below \n \n'+Logger.getLog());
}else{
GmailApp.sendEmail('mymail', 'inboxLabeller End report', 'Job completed : '+items+' processed');
ScriptProperties.setProperties({'threadStart':0, 'itemsprocessed':0, 'notF':true})
}
}
This will find individual messages that do not have a label and apply the label of the associated thread. It takes much less time because it's not relabeling every single message.
function label_unlabeled_messages() {
var unlabeled = GmailApp.search("has:nouserlabels -label:inbox -label:sent -label:chats -label:draft -label:spam -label:trash");
for (var i = 0; i < unlabeled.length; i++) {
Logger.log("thread: " + i + " " + unlabeled[i].getFirstMessageSubject());
labels = unlabeled[i].getLabels();
for (var j = 0; j < labels.length; j++) {
Logger.log("labels: " + i + " " + labels[j].getName());
labels[j].addToThread(unlabeled[i]);
}
}
}
I've been trying to whip up a quick google script to count rsvps for the invite response spreadsheet for a wedding. The script worked perfectly for a week as new entries were added to the spreadsheet, then suddenly stopped working with the following error message in each cell:
Error: Service Times Out: Apps Script
The script itself is simple. It queries the relevant column (there are multiple events) and then checks to see whether there is some response spefied by the user - "YES", "NO", or a blank, typically.
What does this error mean, and does anyone have any suggestions for fixes?
function sumRSVP(response, rsvpType) {
var rsvpCol = 7;
if (rsvpType == "rehearsal") rsvpCol = 8;
if (rsvpType == "brunch") rsvpCol = 9;
var mySum = 0;
var sh = SpreadsheetApp.getActiveSheet();
for( i=2; i<177; i++){
var rsvp = sh.getRange(i, rsvpCol).getValue();
var nguests = sh.getRange(i, 6).getValue();
if(nguests != "" && rsvp == response){
mySum = mySum + parseFloat(nguests);
}
}
return mySum;
}
Hopefully the wedding went well. This was asked some time ago but has been viewed over 300 times at this post and I believe is important:
Data should not be extracted from a spreadsheet in a loop. The data needed should be extracted in a batch to an array and the array evaluated in the loop.
See docs reference at:
https://developers.google.com/apps-script/guide_common_tasks#OptimizeScripts
You can write scripts to take maximum advantage of the built-in caching, by minimizing the number of reads and writes. Alternating read and write commands is slow. To speed up a script, read all data into an array with one command, perform any operations on the data in the array, and write the data out with one command.
function sumRSVP(response, rsvpType) {
var rsvpCol = 7;
if (rsvpType == "rehearsal") rsvpCol = 8;
if (rsvpType == "brunch") rsvpCol = 9;
var mySum = 0;
var sh = SpreadsheetApp.getActiveSheet();
// start at row 2 - uses columns 6-9
var data = sh.getRange(2, 6, 177 - 1 , 4).getValues();
for(var i=0; i<data.length; i++){
var rsvp = data[i][rsvpCol - 6];
var nguests = data[i][0];
if(nguests != "" && rsvp == response){
mySum = mySum + parseFloat(nguests);
}
}
return mySum;
}