Extract portions of Google Docs form data and write it on separate lines - forms

Let's say I have a Google Docs Form that gathers the following info:
Timestamp (default field)
Names
Ref#
The form data then appears on the spreadsheet as follows:
4/10/2013 16:20:31 | Jack, Jill, Oscar | Ref6656X
(Note: the number of names may be anywhere from 1 to many)
I need the data to appear on the spreadsheet as follows:
4/10/2013 16:20:31 | Jack | Ref6656X
4/10/2013 16:20:31 | Jill | Ref6656X
4/10/2013 16:20:31 | Oscar | Ref6656X
I can often decipher and edit Google Apps Script (JavaScript?), but I don't know how to think in that language in order to create it for myself (especially with an unknown number of names in the Name field). How can I get started on solving this?

First of all, you've got some choices to make before you start writing your code.
Do you want to modify the spreadsheet that's accepting form input, or produce a separate sheet that has the modified data? If you want to have a record of what was actually input by a user, you'd best leave the original data alone. If you're using a second sheet for the massaged output, the presence of multiple tabs might be confusing to your users, unless you take steps to hide it.
Do you want to do the modifications as forms come in, or (in bulk) at some point afterwards? If you already have collected data, you'll have to have the bulk processing, and that will involve looping and having to handle insertions of new rows in the middle of things. To handle forms as they come in, you'll need to set up a function that is triggered by form submissions, and only extend the table further down... but you've got more learning to do - see Container-Specific Triggers, Understanding Triggers and Understanding Events for background info.
Will you primarily use Spreadsheet service functions, or javascript Arrays? This choice is often about speed - the more you can do in javascript, the faster your script will be, but switching between the two can be confusing at first.
Here's an example function to do bulk processing. It reads all existing data into an array, goes through that and copies all rows into a new array, expanding multiple names into multiple rows. When done, the existing sheet data is overwritten. (Note - not debugged or tested.)
function bulkProcess() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var dataIn = ss.getDataRange().getValues();
var dataOut = [];
for (var row in dataIn) { // Could use: for (var row = 0; row < dataIn.length; row++)
var names = dataIn[row][1].split(','); // array of names in second column
var rowOut = dataIn[row];
for (var i in names) {
rowOut[1] = names[i]; // overwrite with single name
dataOut.push(rowOut); // then copy to dataOut array
}
}
// Write the updated array back to spreadsheet, overwriting existing values.
ss.getRange(1,1,dataOut.length,dataOut[0].length).setValues(dataOut);
}

Related

mirth connect Database Reader automatic column mapping

Please could somebody confirm the following..
I am using Mirth Connect 3.5.08232.
My Source Connector is a Database Reader.
Say, I am using a query that returns multiple rows, and return the result (via JavaScript), as documentation suggests, so that Mirth would treat each row as a separate message. I also use a couple of mappers as source transformers, and save the mapped fields in my channel map (which ends up to contain only those fields that I define in transformers)
In the destination, and specifically, in destination response transformer (or destination body, if it is a JavaScript writer), how do I access the source fields?
the only way I found by trial and error is
var rawMsg = connectorMessage.getRawData();
var xmlMsg = new XML(rawMsg);
logger.info(xmlMsg.some_field); // ignore the root element of rawMsg
Is this the right way to do this? I thought that maybe the fields that were nicely automatically detected would be put in some kind of a map, like sourceMap - but that doesn't seem to be the case, right?
Thank you
If you are using Mapper steps in your transformer to extract the data and put it into a variable map (like the channel map), then you can use any of the following methods to retrieve it from a subsequent JavaScript context (including a JavaScript Writer, and your response transformer):
var value = channelMap.get('key');
var value = $c('key');
var value = $('key');
Look at the Variable Maps section of the User Guide for more information.
So to recap, say you're selecting a column "mycolumn" with a Database Reader. The XML sent to the channel will be something like this:
<result>
<mycolumn>value</mycolumn>
</result>
Then you can choose to extract pieces of that message into specific variables for later use. The transformer allows you to easily drag-and-drop pieces of the sample inbound message.
Finally in your JavaScript Writer (or in any subsequent filter, transformer, or response transformer), just drag the value into the field you want:
And the corresponding JavaScript code will automatically be inserted:
One last note, if you are selecting a lot of variables and don't want to make Mapper steps for each one individually, you can use a JavaScript Step to iterate through the message and extract each column into a separate map variable:
for each (child in msg.children()) {
channelMap.put(child.localName(), child.toString());
}
Or, you can just reference the columns directly from within the JavaScript Writer:
var msg = new XML(connectorMessage.getEncodedData());
var column1 = msg.column1.toString();
var column2 = msg.column2.toString();
...

updating existing data on google spreadsheet using a form?

I want to build kind of an automatic system to update some race results for a championship. I have an automated spreadsheet were all the results are shown but it takes me a lot to update all of them so I was wondering if it would be possible to make a form in order to update them more easily.
In the form I will enter the driver name and the number o points he won on a race. The championship has 4 races each month so yea, my question is if you guys know a way to update an existing data (stored in a spreadsheet) using a form. Lets say that in the first race, the driver 'X' won 10 points. I will insert this data in a form and then call it from the spreadsheet to show it up, that's right. The problem comes when I want to update the second race results and so on. If the driver 'X' gets on the second race 12 points, is there a way to update the previous 10 points of that driver and put 22 points instead? Or can I add the second race result to the first one automatically? I mean, if I insert on the form the second race results can it look for the driver 'X' entry and add this points to the ones that it previously had. Dunno if it's possible or not.
Maybe I can do it in another way. Any help will be much appreciated!
Thanks.
Maybe I missed something in your question but I don't really understand Harold's answer...
Here is a code that does strictly what you asked for, it counts the total cumulative value of 4 numbers entered in a form and shows it on a Spreadsheet.
I called the 4 questions "race number 1", "race number 2" ... and the result comes on row 2 so you can setup headers.
I striped out any non numeric character so you can type responses more freely, only numbers will be retained.
form here and SS here (raw results in sheet1 and count in Sheet2)
script goes in spreadsheet and is triggered by an onFormSubmit trigger.
function onFormSubmit(e) {
var sh = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Sheet2');
var responses = []
responses[0] = Number(e.namedValues['race number 1'].toString().replace(/\D/g,''));
responses[1] = Number(e.namedValues['race number 2'].toString().replace(/\D/g,''));
responses[2] = Number(e.namedValues['race number 3'].toString().replace(/\D/g,''));
responses[3] = Number(e.namedValues['race number 4'].toString().replace(/\D/g,''));
var totals = sh.getRange(2,1,1,responses.length).getValues();
for(var n in responses){
totals[0][n]+=responses[n];
}
sh.getRange(2,1,1,responses.length).setValues(totals);
}
edit : I changed the code to allow you to change easily the number of responses... range will update automatically.
EDIT 2 : a version that accepts empty responses using an "if" condition on result:
function onFormSubmit(e) {
var sh = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Sheet2');
var responses = []
responses[0] = Number((e.namedValues['race number 1']==null ? 0 :e.namedValues['race number 1']).toString().replace(/\D/g,''));
responses[1] = Number((e.namedValues['race number 2']==null ? 0 :e.namedValues['race number 2']).toString().replace(/\D/g,''));
responses[2] = Number((e.namedValues['race number 3']==null ? 0 :e.namedValues['race number 3']).toString().replace(/\D/g,''));
responses[3] = Number((e.namedValues['race number 4']==null ? 0 :e.namedValues['race number 4']).toString().replace(/\D/g,''));
var totals = sh.getRange(2,1,1,responses.length).getValues();
for(var n in responses){
totals[0][n]+=responses[n];
}
sh.getRange(2,1,1,responses.length).setValues(totals);
}
I believe you can found everything you want here.
It's a form url, when you answer this form you'll have the url of the spreadsheet where the data are stored. One of the information stored is the url to modify your response, if you follow the link it will open the form again and update the spreadsheet in consequence. the code to do this trick is in the second sheet of the spreadsheet.
It's a google apps script code that need to be associated within the form and triggered with an onFormSubmit trigger.
It may be too late now. I believe we need a few things (I have not tried it)
A unique key to map each submitted response, such as User's ID or email.
Two Google Forms:
a. To request the unique key
b. To retrieve relevant data with that unique key
Create a pre-filled URL (See http://www.cagrimmett.com/til/2016/07/07/autofill-google-forms.html)
Open the URL from your form (See Google Apps Script to open a URL)

Manipulating form input values after submission causes multiple instances

I'm building a form with Yii that updates two models at once.
The form takes the inputs for each model as $modelA and $modelB and then handles them separately as described here http://www.yiiframework.com/wiki/19/how-to-use-a-single-form-to-collect-data-for-two-or-more-models/
This is all good. The difference I have to the example is that $modelA (documents) has to be saved and its ID retrieved and then $modelB has to be saved including the ID from $model A as they are related.
There's an additional twist that $modelB has a file which needs to be saved.
My action code is as follows:
if(isset($_POST['Documents'], $_POST['DocumentVersions']))
{
$modelA->attributes=$_POST['Documents'];
$modelB->attributes=$_POST['DocumentVersions'];
$valid=$modelA->validate();
$valid=$modelB->validate() && $valid;
if($valid)
{
$modelA->save(false); // don't validate as we validated above.
$newdoc = $modelA->primaryKey; // get the ID of the document just created
$modelB->document_id = $newdoc; // set the Document_id of the DocumentVersions to be $newdoc
// todo: set the filename to some long hash
$modelB->file=CUploadedFile::getInstance($modelB,'file');
// finish set filename
$modelB->save(false);
if($modelB->save()) {
$modelB->file->saveAs(Yii::getPathOfAlias('webroot').'/uploads/'.$modelB->file);
}
$this->redirect(array('projects/myprojects','id'=>$_POST['project_id']));
}
}
ELSE {
$this->render('create',array(
'modelA'=>$modelA,
'modelB'=>$modelB,
'parent'=>$id,
'userid'=>$userid,
'categories'=>$categoriesList
));
}
You can see that I push the new values for 'file' and 'document_id' into $modelB. What this all works no problem, but... each time I push one of these values into $modelB I seem to get an new instance of $modelA. So the net result, I get 3 new documents, and 1 new version. The new version is all linked up correctly, but the other two documents are just straight duplicates.
I've tested removing the $modelB update steps, and sure enough, for each one removed a copy of $modelA is removed (or at least the resulting database entry).
I've no idea how to prevent this.
UPDATE....
As I put in a comment below, further testing shows the number of instances of $modelA depends on how many times the form has been submitted. Even if other pages/views are accessed in the meantime, if the form is resubmitted within a short period of time, each time I get an extra entry in the database. If this was due to some form of persistence, then I'd expect to get an extra copy of the PREVIOUS model, not multiples of the current one. So I suspect something in the way its saving, like there is some counter that's incrementing, but I've no idea where to look for this, or how to zero it each time.
Some help would be much appreciated.
thanks
JMB
OK, I had Ajax validation set to true. This was calling the create action and inserting entries. I don't fully get this, or how I could use ajax validation if I really wanted to without this effect, but... at least the two model insert with relationship works.
Thanks for the comments.
cheers
JMB

Is there a way to get the original value of a cell in onEdit()?

I'd like to know if there's a way to get the original value of the cell from within the onEdit() event.
For example:
Cell A1's current value is "hello"
I edit A1 it by changing it to "world"
Right now, when I try to get the value from the active range, I'd get "world"
But I would also like to have the possibility to get the original value, i.e. "hello". Is that currently possible?
You can use onEdit(e) and e.oldValue if you're dealing with a single cell. The example below adds the original value to A1 when any cell is edited. 'undefined' if the cell was blank.
function onEdit(e) {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getActiveSheet();
var cell = sheet.getActiveCell();
var origValue = e.oldValue
sheet.getRange('A1').setValue(origValue);
}
The only code that would come close to this is to use an onOpen exit to save
the initial value. The initial value could be saved in several ways. There
are "user properties". I don't know how they work, yet. I am new to this.
You could also use a "hidden" sheet to save the value, in the "onOpen" exit.
There may be other ways, that I don't know about.
The problem becomes more difficult with an increasing number of cells for which
you want to save the original value.
By-the-way, you should understand that the "onOpen" routine fires at the time
the spreadsheet is opened. It so happens, that the end-user also has access and
can change cell values before the onOpen handler finishes its execution. You may
not capture all of your initial values.
One final thing you should know. The onEdit trigger is NOT fired when an UNDO or REDO
event occurs. The cell's contents could change and you will not know it.
I don't know how a validation routine works. If the routine rejects a value, will
the spreadsheet restore the original value? If it does, then this might get around
the onOpen problem. If it only tells the user the value is invalid, it will not be
of much help.
A really round about way that may work, but is very complicated is to save the image
before the spreadsheet closes. You post all the "after" images to a second spreadsheet.
Then in your onEdit handler you look at the corresponding cell in the other spreadsheet.
You then decide to restore the previous image or allow the new image to proceed.
Lastly a wild idea of using a data table in place of the second spreadsheet.
I am just learning about all of these concepts, so don't ask me how to implement them.
I just understand that they MIGHT be possible. For coding and support purposes they
may not be the best options. But since the current script service does not provide
before image access, it is about the best I could do. You have to understand that this
google interface is a client-server application. Your scripts run on the server. The data changes occur in the "clients" (end-users) browser.
One final note: the onEdit trigger does not fire for an UNDO or REDO change to a cell.
So the cell could change and your script is not aware of it.
I don't think that's possible.
I imagine you could get that functionality by having a exact copy of your sheet on a second sheet that updates automatically when your 'onEdit' functions ends.
Until that update, data on the second sheet will have the former value.
A bit tricky but why not ?-)
EDIT : seems to be the 'question of the day', see this post and Henrique Abreu's pertinent comment.
When you change the value of a cell diagrammatically, you can use the setComment method to store the original value as a comment in that cell.
What you basically need to do is to create a shadow sheet (which you can protect and hide or even have it in a totally separate spreadsheet) and use the IMPORTRANGE function to get the values of the original sheet into the shadow sheet (This gives enough delay time to get the old value that was in the cell before it got edited).
=IMPORTRANGE("enter your original sheet's ID","enter the range you wish to get from the sheet")
Please note that using this code, when editing multiple cells at the same time, the function will only work on the first cell.
function onEdit(){
var sheet = SpreadsheetApp.getActiveSpreadsheet();
var originalSheet = sheet.getSheetByName('Original');
var shadowSheet = sheet.getSheetByName('Shadow');
var editedCell = originalSheet.getActiveCell();
var cellColumn = editedCell.getColumn();
var cellRow = editedCell.getRow();
var oldValue = shadowSheet.getRange(cellRow, cellColumn).getValue();
var cellValue = editedCell.getValue();
var editorMail = Session.getActiveUser().getEmail(); \\getting the editor email
if ("The condition you want to meet for the onEdit function to activate"){
editedCell.setNote(editorMail + "\n" + new Date + "\n" + oldValue + " -> " + cellValue);
}
}

Generate unique 3 letter/number code and compare to existing ones in PHP/MySQL

I'm making a code generation script for UN/LOCODE system and the database has unique 3 letter/number codes in every country. So for example the database contains "EE TLL", EE being the country (Estonia) and TLL the unique code inside Estonia, "AR TLL" can also exist (the country code and the 3 letter/number code are stored separately). Codes are in capital letters.
The database is fairly big and already contains a huge number of locations, the user has also the possibility of entering the 3 letter/number him/herself (which will be checked against the database before submission automatically).
Finally neither 0 or 1 may be used (possible confusion with O and I).
What I'm searching for is the most efficient way to pick the next available code when none is provided.
What I've came up with:
I'd check with AAA till 999, but then for each code it would require a new query (slow?).
I could store all the 40000 possibilities in an array and subtract all the used codes that are already in the database... but that uses too much memory IMO (not sure what I'm talking about here actually, maybe 40000 isn't such a big number).
Generate a random code and hope it doesn't exist yet and see if it does, if it does start over again. That's just risk taking.
Is there some magic MySQL query/PHP script that can get me the next available code?
I will go with number 2, it is simple and 40000 is not a big number.
To make it more efficient, you can store a number representing each 3-letter code. The conversion should be trivial because you have a total of 34 (A-Z, 2-9) letters.
I would for option 1 (i.e. do a sequential search), adding a table that gives the last assigned code per country (i.e. such that AAA..code are all assigned already). When assigning a new code through sequential scan, that table gets updated; for user-assigned codes, it remains unmodified.
If you don't want to issue repeated queries, you can also write this scan as a stored routine.
To simplify iteration, it might be better to treat the three-letter codes as numbers (as Shawn Hsiao suggests), i.e. give a meaning to A-Z = 0..25, and 2..9 = 26..33. Then, XYZ is the number X*34^2+Y*34+Z == 23*1156+24*34+25 == 27429. This should be doable using standard MySQL functions, in particular using CONV.
I went with the 2nd option. I was also able to make a script that will try to match as close as possible the country name, for example for Tartu it will try to match T** then TA* and if possible TAR, if not it will try TAT as T is the next letter after R in Tartu.
The code is quite extensive, I'll just post the part that takes the first possible code:
$allowed = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ23456789';
$length = strlen($allowed);
$codes = array();
// store all possibilities in a huge array
for($i=0;$i<$length;$i++)
for($j=0;$j<$length;$j++)
for($k=0;$k<$length;$k++)
$codes[] = substr($allowed, $i, 1).substr($allowed, $j, 1).substr($allowed, $k, 1);
$used = array();
$query = mysql_query("SELECT code FROM location WHERE country = '$country'");
while ($result = mysql_fetch_array($query))
$used[] = $result['code'];
$remaining = array_diff($codes, $used);
$code = $remaining[0];
Thanks for your opinion, this will be the key to transport codes all over the world :)