I have a spreadsheet with three sheets. Two are called 2012 and 2011 and have a bunch of similar data. The last sheet does comparisons between the data.
To be able to choose year, I'm using a cell (D1) where I can I can write either 2011 or 2012. The formulas then use the INDIRECT function to include this cell as part of the reference.
INDIRECT(CHAR(39)&$D$1&CHAR(39)&"!F:F")
This is not a pretty solution and makes the formula quite long and complex.
=IFERROR(SUM(FILTER( INDIRECT(CHAR(39)&$D$1&CHAR(39)&"!M:M") ; (INDIRECT(CHAR(39)&$D$1&CHAR(39)&"!B:B")=$A4)+(INDIRECT(CHAR(39)&$D$1&CHAR(39)&"!B:B")=$A5)+(INDIRECT(CHAR(39)&$D$1&CHAR(39)&"!B:B")=$A6)+(INDIRECT(CHAR(39)&$D$1&CHAR(39)&"!B:B")=$A7)+(INDIRECT(CHAR(39)&$D$1&CHAR(39)&"!B:B")=$A8); MONTH(INDIRECT(CHAR(39)&$D$1&CHAR(39)&"!D:D"))=$B$1 ; INDIRECT(CHAR(39)&$D$1&CHAR(39)&"!F:F")=D$3));0)
Is there a better way of doing this?
I've tried to create a separate spreadsheet for the calculations sheet and importing (IMPORTRANGE) the data from the two sheets together on one sheet with VMERGE (custom function from the script gallery) but there is quite a lot of of data in these two sheets and the import takes a long time. Any changes (like changing year) also take a long time to recalculate.
Database functions tend to be cleaner when doing this kind of thing.
https://support.google.com/docs/bin/static.py?hl=en&topic=25273&page=table.cs&tab=1368827
Database functions take a while to learn, but they are powerful.
Or
You could put INDIRECT(CHAR(39)&$D$1&CHAR(39)&"!B:B") in a cell on its own.
I think that you have two years of information where the schema is identical (column C has the same type of information on both sheets). Also, I'm assuming that column B tracks the year.
If so, consider holding all of your information on one sheet and and use the spreadsheet function "QUERY" to create views.
For instance, this formula returns all the cells between A1:E from a sheet named "DataSheet" where the values in column B = 2010.
=QUERY(DataSheet!A1:E; "SELECT * WHERE B = 2010";1)
Sometimes there is a really good reason to have the data stored on two sheets. If so, use one of the vMerge functions in the script gallery to assemble a working sheet. Then create views and reports from the working sheet.
function VMerge() {
var maxw=l=0;
var minw=Number.MAX_VALUE;
var al=arguments.length ;
for( i=0 ; i<al ; i++){
if( arguments[i].constructor == Array )l =arguments[i][0].length ;
else if (arguments[i].length!=0) l = 1 ; // literal values count as array with a width of one cell, empty cells are ignored!
maxw=l>maxw?l:maxw;
minw=l<minw?l:minw;
}
if( maxw==minw) { /* when largest width equals smallest width all are equal */
var s = new Array();
for( i=0 ; i<al ; i++){
if( arguments[i].constructor == Array ) s = s.concat( arguments[i].slice() )
else if (arguments[i].length!=0) s = s.concat( [[arguments[i]]] )
}
if ( s.length == 0 ) return null ; else return s //s
}
else return "#N/A: All data ranges must be of equal width!"
}
Hope this helps.
Related
I know how to 'select' a range in LO (7.2.4.1) Calc BASIC ....
ThisComponent.CurrentController.ActiveSheet.getCellRangeByName("D1:H6")
But how to write a value, e.g. "1", into that range using BASIC?
myRange = ThisComponent.CurrentController.ActiveSheet.getCellRangeByName("D1:H6")
myRange.Value = 1
Gives an "property or method not found" error. But I can't find any properties or values to go after Range to allow me to do what I want. Flailing around and trying
myRange.setValue = 1
myRange.writeValue = 1
myRange.setString = "1"
and numerous other variants don't work either.
Would really appreciate the solution. Thanks.
You can edit the value of an individual cell, but not the entire range. You will have to iterate over all the cells in the range one at a time, changing the value of each of them.
Sub Set1ToD1H6
myRange = ThisComponent.CurrentController.ActiveSheet.getCellRangeByName("D1:H6")
For i = 0 To myRange.getRows().getCount()-1
For j = 0 To myRange.getColumns().getCount()-1
myRange.getCellByPosition(j, i).setValue(1)
Next j
Next i
End Sub
But since the read-write operation to a cell is comparable in time to the read-write operation to a whole range, it is preferable to use another method - to prepare data in an array and write from it to a range in one operation:
Sub Set1ToRange
myRange = ThisComponent.CurrentController.ActiveSheet.getCellRangeByName("D1:H6")
dataOfRange = myRange.getData()
For i = LBound(dataOfRange) To UBound(dataOfRange)
For j = LBound(dataOfRange(i)) To UBound(dataOfRange(i))
dataOfRange(i)(j) = 1
Next j
Next i
myRange.setData(dataOfRange)
End Sub
(For your example, this will be approximately 30 times faster, for a larger range the time winnings will be even more significant)
The .getData() and .setData() methods work on numeric range values. To work with text strings (and numbers), use .getDataArray() and .setDataArray(), for working with cell formulas use .getFormulaArray() and .setFormulaArray()
I have two Google sheets workbooks.
One is the "master" source of lookup data with a key based on manufacturer item #, which could be anything from 1234 to A-01/234-Name_1. This sheet, referenced via SpreadsheetApp.openByUrl, has 18,000 rows and 13 columns. The key column has been converted to plain text and the sheet is sorted by this column.
The second is the "template" where people enter item #s that they need to look up against the master, typically 20 - 1500 items at a time.
The script is in the template. It is very slow and routinely times out after 30 minutes. It was written by someone else and I am new to App Script, but I think I've managed to understand what the script is doing and where the bottleneck is occurring.
It does a bunch of stuff, but this is the meat of the lookup:
var numrows = master.getDataRange().getNumRows();
var masterdata = master.getDataRange().getValues();
var itemnumberlist = template.getDataRange().getValues();
var retreiveddata = [];
// iterate through the manf item number list to find all matches in the
// master and return those matches to another sheet
for (i = 1; i < template.getDataRange().getValues().length; i++) {
for (j = 0; j < numrows; j++) {
if (masterdata[j][1].toString() === itemnumberlist[i][1].toString()) {
retreiveddata.push(data[j]);
anothersheet.appendRow(data[j]);
}
}
}
I used Logger.log() to determine that each time through the i loop is taking 11 - 19 seconds, which just seems insane.
I've been doing some google searching and I've tried a couple of different things...
First I tried moving the writing of found data out of the for loop so the script would be doing all of its reading first and then writing in one big chunk, but I couldn't get it exactly right. My two attempts are below.
var mycounter = 0;
for (i = 0; i < template.getDataRange().getValues().length; i++) {
for (j = 0; j < numrows; j++) {
if (masterdata[j][0].toString() === itemnumberlist[i][0].toString()) {
retreiveddata.push(masterdata[j]);
mycounter = mycounter + 1;
}
}
}
// Attempt 1
// var myrange = retreiveddata.length;
// for(k = 0; k < myrange; k++) {
// anothersheet.appendRow(retreiveddata.pop([k]);
// }
//Attempt 2
var myotherrange = anothersheet.getRange(2,1,myothercounter, 13)
myotherrange.setValues(retreiveddata);
I can't remember for sure, because this was on Friday, but I think both attempts resulted in the script trying to write the entire master file into "anothersheet".
So I temporarily set this aside and decided to try something else. I was trying to recreate the issue in a couple of sample spreadsheets, but I was unable to do so. The same script is getting through my 15,000 row sample "master" file in less than 1 second per lookup. The only thing I can think of is that I used a random number as my key instead of a weird text string.
That led me to think that maybe I could use a hash algorithm on both the master data and the values to be looked up, but this is presenting a whole other set of issues.
I borrowed these functions from another forum post:
function GetMD5Hash(value) {
var rawHash = Utilities.computeDigest(Utilities.DigestAlgorithm.MD5,
value);
var txtHash = '';
for (j = 0; j <rawHash.length; j++) {
var hashVal = rawHash[j];
if (hashVal < 0)
hashVal += 256;
if (hashVal.toString(16).length == 1)
txtHash += "0";
txtHash += hashVal.toString(16);
Utilities.sleep(100);
}
return txtHash;
}
function RangeGetMD5Hash(input) {
if (input.map) { // Test whether input is an array.
return input.map(GetMD5Hash); // Recurse over array if so.
Utilities.sleep(100);
} else {
return GetMD5Hash(input)
}
}
It literally took me all day to get the hash value for all 18,000 item #s in my master spreadsheet. Neither GetMD5Hash nor RangeGetMD5Hash will return a value consistently. I can only do a few rows at a time. Sometimes I get "Loading..." indefinitely. Sometimes I get "#Name" with a message about GetMD5Hash being undefined (despite the fact that it worked on the previous row). And sometimes I get "#Error" with a message about an internal error.
This method actually reduces the lookup time of each item to 2 - 3 seconds (much better, but not great). However, I can't get the hash function to consistently work on the input data.
At this point I'm so frustrated and behind on my other work that I thought I'd reach out to the smart people on these forums and hope for some sort of miracle response.
To summarize, I'm looking for suggestions on these three items:
What am I doing wrong in my attempt to move the write out of the for loop?
Is there a way to get my hash value faster or utilize a different method to accomplish the same goal?
What else can I try to help speed up the script?
Any suggestions you can offer would be greatly appreciated!
-Mandy
It sounds like you hit on the right approach with attempting to move the appendRow() call out of the loop. Anytime you are reading or writing to a spreadsheet you can expect the individual call to take 1 to 2 seconds, so this will eat up a lot of time when you get matches. Storing the matches in an array and writing them all at once is the way to go.
Another thing I notice is that your script calls getValues() in the actual for loop condition statement. The condition statement is executed each time on each iteration of the loop, so this is potentially wasting a lot of time even when you don't have matches.
A final tweak that may be helpful depending on your desired behaviour. You can stop the inner for loop after it finds the first match, which, if you only care about the first match or know there will only be one match, will save you a lot of iterations. To do this, put "break" immediately after the retreiveddata.push(masterdata[j]); line.
To fix the getValues issue, Change:
for (i = 1; i < template.getDataRange().getValues().length; i++) {
To:
for (i = 1; i < itemnumberlist.length; i++) {
And that fix along with the appendRow issue, and including the break call:
for (i = 1; i < itemnumberlist.length; i++) {
for (j = 0; j < numrows; j++) {
if (masterdata[j][0].toString() === itemnumberlist[i][0].toString()) {
retreiveddata.push(masterdata[j]);
break; //stop searching after first match, move on to next item
}
}
}
//make sure you have data to write before trying to write it.
if(retreiveddata.length > 0){
var myotherrange = anothersheet.getRange(2,1,retreiveddata.length, retreiveddata[0].length);
myotherrange.setValues(retreiveddata);
}
If you are re-using the same sheet for "anothersheet" on each execution, you may also want to call anothersheet.clear() to erase any existing data before you write your fresh results.
I would pass on the hashing approach altogether, comparing strings is comparing strings, so whether they are hashes or actual part numbers I wouldn't expect a significant difference.
I have a Google Sheet where information from a Google Form is dumped. Two of the columns create a date range (columns C and G) and I would like for the sheet to automatically create a new row of information for every date of the range and copy all the other information from the original row for every row that is created. In the end, every date in the range has it's own row regardless of it being 2 days or 25 and all the the information gathered through the form be present for each day. If there is not a date in column G, it is only a one day trip and there is no need for additional rows. To make things more difficult when someone submits a form, the information is entered into the row directly beneath the last one that it filled, so these new rows filled by the date range will need to be down the sheet, possibly beginning at row 2000 or more as this sheet will have a lot of information in a few months. As you may see in the sample, there is another sheet in the workbook that performs all the sorting. Thanks for any help.
Sample Document
You will need to create a form submit event and attach the following code to it. Also you'll need to create a sheet name 'ResponseReview'.
function formSubmitEvent1(e)
{
var ss=SpreadsheetApp.openById('SpreadsheetID');
var sht=ss.getSheetByName('ResponseReview');
sht.appendRow(e.values);
}
The above code will need the SpreadsheetId in the openById Method. This code will append any new rows to the end of the ResponseReview sheet.
The code below will expand any entrees that have a date in column 3 and 7 and it will also remove the end date in column 7 from that first row. I use the fact that if column 3 is not empty and column 7 is not and column 7 is no equal to column 3 then that's a row that needs to be expanded. So I have to remove the end date so that it won't continue to get expanded when it's run again in the future. We could figure something else out if you need to keep than end date. We could add a don't expand column at the end.
function convertRangetoRows()
{
var ss=SpreadsheetApp.getActive();
var sht=ss.getSheetByName('ResponseReview');
var rng=sht.getDataRange();
var rngA=rng.getValues();
var rngB=[];
var day=86400000;
rngB.push(rngA[0]);
for(var i=1;i<rngA.length;i++)
{
rngB.push(rngA[i]);
if(rngA[i][2] && rngA[i][6] && rngA[i][2]!=rngA[i][6])
{
var row=rngA[i].slice();//returns a new copy of the array by value
rngA[i][6]='';//deletes the end date by reference so it also deletes the one thats already been pushed into rngB
var dt0=new Date(row[2]);
var dt1=new Date(row[6]);
var days=(dt1.valueOf()-dt0.valueOf())/day;
var dt=dt0.valueOf();
for(j=0;j<days;j++)
{
dt+=day;
row[2]=Utilities.formatDate(new Date(dt), Session.getScriptTimeZone(), "MM/dd/yyyy");//original array unchanged
row[6]='';//original array unchanged
rngB.push(row.slice());//push in a copy
}
}
var intermediate='nothing';
}
var outrng=sht.getRange(1,1,rngB.length,rngB[0].length);
outrng.setValues(rngB);
var end='the end is near';
}
This is what my spreadsheet looks like before running the expansion function:
And After:
And now you can leave the sheet linked to the form alone and let it be an archive for submitted data.
Ahab stated in 2010: the complex looking number based on the Timestamp has one important property, the number can not change when rows are deleted or inserted.
As long as the submitted data is not changed by inserting deleting rows the simple formula =ArrayFormula(ROW(A2:A) - 1) may be the easiest one to use.
For other situations there is no nice reliable solution. :(
Now we live in 2015. Maybe times have changed?
I need a reliable way to number entries using a form.
Maybe a script can do the trick? A script that can add 1 to each entry?
That certain entry has to keep that number even when rows are deleted or inserted.
I created this simple spreadsheet in which I added 1,2, and 3 manually,please have a look:
https://docs.google.com/spreadsheets/d/1H9EXns8-7m9oLbCrTyIZhLKXk6TGxzWlO9pOvQSODYs/edit?usp=sharing
The script has to find the maximum of the former entries, which is 3, and then add 1 automatically.
Who can help me with this?
Grtz, Bij
Maybe a script can do the trick? A script that can add 1 to each
entry?
Yes, that would be what you need to resort to. I took the liberty of entering this in your example ss:
function onEdit(e) {
var watchColumns = [1, 2]; //when text is entered in any of these columns, auto-numbering will be triggered
var autoColumn = 3;
var headerRows = 1;
var watchSheet = "Form";
var range = e.range;
var sheet = range.getSheet();
if (e.value !== undefined && sheet.getName() == watchSheet) {
if (watchColumns.indexOf(range.getColumn()) > -1) {
var row = range.getRow();
if (row > headerRows) {
var autoCell = sheet.getRange(row, autoColumn);
if (!autoCell.getValue()) {
var data = sheet.getDataRange().getValues();
var temp = 1;
for (var i = headerRows, length = data.length; i < length; i++)
if (data[i][autoColumn - 1] > temp)
temp = data[i][autoColumn - 1];
autoCell.setValue(temp + 1);
}
}
}
}
}
For me the best way is to create a query in a second sheet pulling everything from form responses in to second column and so on. then use the first column for numbering.
In your second sheet B1 you would use:
=QUERY(Form!1:1004)
In your second sheet A2 you would use:
=ARRAYFORMULA(if(B2:B="",,Row(B2:B)-1))
I made a second sheet in your example spreadsheet, have a look at it.
I've got a form done in x++ (formBuild) and I managed to display different grids in different tabs. However, when I do a right-click record info on any of the grids other than the first one, the details are that of the first grid. Eg. The second row of grid 2 when I do a record info is actually the second row of grid 1.
One thing is all the grids are actually using the same table, just having different query ranges for each.
Any way to fix this?
Added code snippets
Making the grid:
for (counter = 0; counter < locations.lastIndex(); counter++)
{
formBuildDatasource = form.addDataSource(tableStr(SomeTable));
formBuildTabPageControl = formBuildTabControl.addControl(FormControlType::TabPage, locations.value(counter+1));
formBuildTabPageControl.caption(locations.value(counter+1));
formBuildGridControl = formBuildTabPageControl.addControl(FormControlType::Grid, locations.value(counter+1));
formBuildGridControl.allowEdit(0);
formBuildGridControl.dataSource(formBuildDatasource);
formBuildGridControl.height(500,-1);
formBuildGridControl.width(550,-1);
formBuildGridControl.addDataField(formBuildDatasource.id(), fieldNum(SomeTable, MachineId));
formBuildGridControl.addDataField(formBuildDatasource.id(), fieldNum(SomeTable, MachineStatus));
}
Adding the query:
for (counter = 0; counter < locations.lastIndex(); counter++)
{
fds = formRun.dataSource(counter+1);
qbds = fds.query().dataSourceNo(1);
qbr = Qbds.addRange(fieldnum(SomeTable, MachineLocation));
qbr.value(locations.value(counter+1));
}
This answer to your prior question applies here as well:
Adding view/temporary table records to Form Grid
You will have to use more than one datasource (using the same table). Remember to change the datasource attribute of the grids to match the correct one. My guess would be that they currently all reference the same datasource.
Can you make a query + view of the table, and have that as the 'child' entity?
I don't know why you can't have the same table referenced twice in the same form data sources, however. Ensure that the link between the tables are defined correctly, and you have no confusion with the datasource names that you use for them.
You actually need to set the datasource on the grid object using the id() method on the FormDataSource object, not just the full object.
Change from:
formBuildGridControl.dataSource(formBuildDatasource);
to:
formBuildGridControl.dataSource(formBuildDatasource.id());