Iterate over files in random order - swift

I am working on an educational script in swift. The aim is to facilitate collaboration between peers in a university level class on statistics. Each student is required to make suggestions (anonymously) to one of their colleagues' work. The script I have been writing works so that all markdown files in the directory are anonymized (ie, author and email yaml keys are removed) and new files with the contents are created and renamed with an email address.
So far I was able to remove author: and email: yaml keys from each file and I was also able to rename the files in the directory. However, I am struggling to create a way of randomizing files and emails. The point is that the original file should be passed on to another student, but without the last one knowing the author. The receiving student should be decided randomly. This randomization is what I am struggling with.
The MWE
It is a linux project and the easiest way I found to create a MWE was to write the script using the excellent John Sundell's marathon script structure. Below is the code:
import Yaml // marathon: https://github.com/behrang/YamlSwift.git
import Files // marathon:https://github.com/JohnSundell/Files.git
import Foundation
var email: String = ""
var document: String = ""
for file in Folder.current.files {
guard file.extension == "md" else {
continue
}
let content = try file.readAsString()
let pattern = "(?s)(?<=---).*(?=---)"
if let range = content.range(of: pattern, options: .regularExpression) {
let text = String(content[range])
let value = try! Yaml.load(text)
email = value["email"].string!
let author = value["author"].string!
let emailline = "email: " + email
let authorline = "author: " + author
document = content.replacingOccurrences(of: "\n\(emailline)", with: "")
document = content.replacingOccurrences(of: "\n\(authorline)", with: "")
}
// Creating new file with name from email and copying contents into it
//let folder = try Folder()
let file = try Folder.current.createFile(named: email)
try file.write(string: document)
}
An example md file:
---
# Metadata
title: hum
author: jecatatu
email: email#gmail.com
---
This is more text outside the yaml block
email: zwe#gmail.com
A second file:
# Metadata
title: My essay
author: brauser
email: brauser#gmail.com
---
Extra text
A third file:
# Metadata
title: My third essay
author: bestuser
email: bestuser#gmail.com
---
Extra text
Question
To start off, I don't need code. But code is welcome. Note that one can run the above example (provided you have marathon installed) with:
marathon run lf-araujo/collab
I guess I can solve this problem by iterating over the files in the directory in a random order. How can do that? My first thought was to create a dict with emails and filenames, and scramble these two.

What you need is a deterministic way of assigning one reviewer to one peer. You can achieve that by creating a two maps of (random -> article). Sort them by the random key and match the articles by index. An example in pseudo code:
/* Note that you may want to be sure that random keys are unique */
map reviewers = [random_integer(): article_one,
random_integer(): article_two,
...]
map articles = [random_integer(): article_one,
random_integer(): article_two,
...]
sort reviewers by key
sort articles by key
for (index in 0..length(reviewers)) {
assign_reviewer(author_from: reviewers[index], to: articles[index])
}

Related

How can I create a function that automatically takes data from Google Sheets and replaces the tags in a Slides template?

I am new to Google Apps Script and coding in general and wasn't entirely sure how to do this. I want to create code that allows me to create a new set of Google Slides based on a Slides template using the relevant rows from a Google Sheets document.
function generateNewSlides() {
var wsID = "would insert worksheet URL ID here";
var ws = SpreadsheetApp.openById(wsID).getSheetByName("Data");
var data = ws.getRange(2, 1, ws.getLastRow()-1, 5).getValues();
>the above should get the relevant table from the sheet
data.forEach(function(info){
if(info[0]){
var firstname = info[0];
var surname = info[1];
var email = info[2];
var phone = info[3];
var image = info[4];
var presName = info[5];
>the above are columns where the different pieces of data would be taken from for the placeholders in the Slides template
var slidesTemplateID = "would insert slides template URL ID here";
var slidesTemplate = SlidesApp.openById(slidesTemplateID);
var template = slidesTemplate.getSlides();
var folderID = "would insert desired folder ID for saving in here";
>the above should get me the Slides template
template.makeCopy(presName,DriveApp.getFolderById(folderID)); **>line where error occurred**
var newPresentation = DriveApp.getFilesByName(presName).next().getUrl();
var Presentation = SlidesApp.openByUrl(newPresentation);
>the above should create a copy and then open it
var shapes = (Presentation.getShapes());
shapes.forEach(function(shape){
shape.getText().replaceAllText('{{firstname}}',firstname);
shape.getText().replaceAllText('{{surname}}',surname);
shape.getText().replaceAllText('{{email}}',email);
shape.getText().replaceAllText('{{phone}}',phone);
shape.getText().replaceAllText('{{presname}}', presName)
});
>the above should replace all the placeholder tags in the template with the row data
}
});
}
Above is the code I have so far. The worksheet I am extracting data from has columns: first name, surname, email address, phone number, image (URL), and presentation name. When I try to run it I encounter an error on line 37 where it says template.makeCopy is not a function, however I am certain .makeCopy should be able to create a copy for it, no?
My main questions are:
1) What should I change to make it work, generating a new set slides for each row in the worksheet?
2) How can I add images to it replacing placeholder tags I've added in squares (not textboxes) in the template?
Thanks in advance!
Issue 1. makeCopy:
makeCopy(name, destination) is a method of the class File, which belongs to the Drive Service, not to the Slides Service. In your code, template is a list of Slides (you retrieve it by calling the method getSlides() from a Presentation). makeCopy cannot work here.
In order to make a copy of a Presentation, you should be using the Drive Service instead. You should replace these lines:
var slidesTemplate = SlidesApp.openById(slidesTemplateID);
var template = slidesTemplate.getSlides();
With this one:
var template = DriveApp.getFileById(slidesTemplateID);
Issue 2. Iterating through all shapes:
Next, you want to iterate through all shapes in your Presentation, and replace all placeholder tags with your desired text. In order to do that, you are using Presentation.getShapes(), which cannot work, since getShapes() is not a method of Presentation, but of Slide.
You should first iterate through all Slides in the Presentation, and for each Slide, iterate through all Shapes. You should replace these lines:
var shapes = (Presentation.getShapes());
shapes.forEach(function(shape){
// Replacing text lines
});
With these ones:
Presentation.getSlides().forEach(function(slide) {
slide.getShapes().forEach(function(shape) {
// Replacing text lines
})
});
Note:
In order to retrieve the copied presentation, you are currently doing this:
template.makeCopy(presName,DriveApp.getFolderById(folderID));
var newPresentation = DriveApp.getFilesByName(presName).next().getUrl();
var Presentation = SlidesApp.openByUrl(newPresentation);
There is no need to do this, you can just retrieve the ID of the created template, and open by ID, like this:
var copiedTemplate = template.makeCopy(presName,DriveApp.getFolderById(folderID));
var Presentation = SlidesApp.openById(copiedTemplate.getId());
Reference:
Slides Service
Drive Service

google apps script: file.setOwner() Not Transferring Ownership in Google Drive

I am trying to transfer ownership of all my .pdf files to another account with more space. I am testing the code with a single folder in my drive.
function transfer() {
var user = Session.getActiveUser();
var folder = DriveApp.getFolderById('123folder-id456789-VxdZjULVQkPAaJ');
var files = folder.getFilesByType(MimeType.PDF);
while (files.hasNext()) {
var file = files.next();
if (file.getOwner() == user) file.setOwner('example#gmail.com');
}
}
When I run the code, none of the files change ownership.
How about this modification?
Modification points:
In your script, it tries to compare the objects of Session.getActiveUser() and file.getOwner(). I think that this is the reason of your issue.
So how about this modification? Please think of this as just one of several answers.
Modified script:
function transfer() {
var user = Session.getActiveUser().getEmail(); // Modified
var folder = DriveApp.getFolderById('123folder-id456789-VxdZjULVQkPAaJ');
var files = folder.getFilesByType(MimeType.PDF);
while (files.hasNext()) {
var file = files.next();
if (file.getOwner().getEmail() == user) file.setOwner('example#gmail.com'); // Modified
}
}
In this modification, the emails are compared.
References:
getActiveUser()
getOwner()
Class User
If this didn't resolve your issue, I apologize.
Currently drive can't transefere the ownership of file that are not build-in, like pdf, zip, etc. So you must download them and reupload from the other account. I wrote a colab to do that without consume my bandwith. It can recursively transfere an entire folder with both build-in file types and other file types.

What is the easiest way to import excel/google sheet spreadsheet to a Cloud Firestore database?

I need to import a large table of data into my database in one go. I currently have it as an Excel file but I am happy to copy it to Google sheets etc.
So far I've added a few entries manually directly via cloud firestore.
Is there a solution already out there to achieve this?
I think the easiest way to export table data into Firestore is to use a Google Apps Script Library (for Google Sheets).
Step 1
Make a Copy of THIS example Google Spreadsheet I created as an example
Step 2
From the menu of YOUR copy of the Example Google Spreadsheet from step 1, click Tools > Script Editor. This should open up the example Google App Script associated with the example spreadsheet.
Step 3
Follow the Steps for installing this library and then update the script with the following:
email
key
projectID
These variables are generated by going to the Google Service Accounts page. This will require that you already have a Firebase or Google Cloud account setup. I won't repeat all the steps that are already iterated in in the aforementioned Github writeup. Just follow them carefully, and realize that the private_key is THE ENTIRE KEY starting with -----BEGIN PRIVATE KEY-----\n, EVERYTHING in between, and ending with \n-----END PRIVATE KEY-----\n
Step 4
Insert a page on your spreadsheet that contains your data, and EDIT the script to use your new sheet name and your data. I have HEAVILY commented the script so it's pretty clear what almost every line of code is doing. For those of you that just want to peek at the Google App Script that's behind this spreadsheet, here's the code:
// Note this Script uses an external library as per this page:
// https://github.com/grahamearley/FirestoreGoogleAppsScript
// This solution requires a Google Spreadhseet and a Firebase Account
// FOLLOW THE INSTRUCTIONS ON THAT GITHUB REPO TO SETUP NEEDED API KEYS!!!
//Global Variables
const ss = SpreadsheetApp.getActiveSpreadsheet(); // Gets the active "workbook"
const sheet = ss.getSheetByName('Restaurants'); // CHANGE TO YOUR SHEET NAME
const headerRowNumber = 1; // If you have more than one row for your header, then change this value to number of header rows
// If you want to mark modified cells, then set up a trigger for the following function:
// Edit > Current Project Triggers > (+ Add Trigger) > On Edit Spreadsheet etc
function onEdit(e) {
var cell = ss.getActiveCell(); //This will also effectively get our row
var dataRange = sheet.getDataRange(); //This checks for all rows/columns with data
var modifiedCol = dataRange.getLastColumn()-1; //Our "modified" column should be the second to last
if (cell.getColumn() < modifiedCol && cell.getRow() > headerRowNumber) { //If we edit any cells to the left of our modified column and below our header...
var celltoMark = sheet.getRange(cell.getRowIndex(),modifiedCol) //Get the R/C cordinates of cell to place modified time
celltoMark.setValue(new Date()); //write timestamp to that cell
}
};
// This will parse any comma separated lists you create in any of your fields (useful for search words, or attributes, etc)
function listToArray(list) {
var ogArray = list.split(","); //Input is a comma separated list
let trimmedArr = ogArray.map(string => string.trim()); //Let's strip out the leading/trailing whitespaces if any
return trimmedArr; //return the cleaned array
}
function writeToFireStore() {
const email = 'sheets#yourprojectid.iam.gserviceaccount.com'; // CHANGE THIS!!!
const key = '-----BEGIN PRIVATE KEY-----\nYOURPRIVATEKEY\n-----END PRIVATE KEY-----\n'; // CHANGE THIS!!!
const projectID = 'yourprojectid'; // CHANGE THIS!!!
var firestore = FirestoreApp.getFirestore(email, key, projectID);
const collection = "MySpreadsheetData"; // Name of your Firestore Database "Collection"
var dataRange = sheet.getDataRange().offset(headerRowNumber, 0, sheet.getLastRow() - headerRowNumber); //this is your data range
var data = dataRange.getValues(); // this is an array of your datarange's values
var lastCol = dataRange.getLastColumn(); // this is the last column with a header
var newDoc = {}; // Instantiate your data object. Each one will become the data for your firestore documents
// r = row number in this case
for (let r = 0; r <= dataRange.getLastRow(); r++) {
//Logger.log("R = ",r);
var cellMod = dataRange.getCell(r+1, lastCol-1);
var cellFS = dataRange.getCell(r+1, lastCol);
var cellModVal = cellMod.getValue();
var cellFSVal = cellFS.getValue();
//
// IMPORTANT READ THIS IMPORTANT READ THIS IMPORTANT READ THIS IMPORTANT READ THIS IMPORTANT READ THIS!!!
// Well, read the line below...
if (r > 2) break; //Comment Out this line after you're done testing otherwise you'll write all your rows to firestore after every run
newDoc[r] = {
name : data[r][1],
category : data[r][2],
cuisine : data[r][3],
address: {
add1: data[r][4],
add2: data[r][5],
city: data[r][6],
state: data[r][7],
zip: data[r][8]
},
tel: data[r][9],
searchterms: listToArray(data[r][10]) //Let's turn a csv list into an array
}
// For the sake of efficiency and to save $, we WON'T create documents that have already been created...
// ...and we won't update documents that have a fireStore Timestamp that's newer than a Modified Timestamp
// If there's not firestore timestamp in our spreadsheet, then let's create firestore document and update firestore stamp:
if (!cellFSVal) {
var now = new Date(); //Generate timestamp right now
try {
firestore.createDocument(collection + "/" + data[r][0], newDoc[r]); // To Use Your Own Document ID
//Now let's insert a timestamp in our Firestore TS column of the sheet so we know it's been added to Firestore
cellFS.setValue(now);
Logger.log("Row ",r,"(",data[r][1],") is NEW and was added to FireStore Successfully");
} catch (e) {
Logger.log("Error: ",e," : Document with same name already existed in Firestore.");
}
}
//var if FS Timestamp exists but, the modified time stamp is greater, let's update the Firstore Document
else if ((cellFSVal) && (cellModVal > cellFSVal)) {
try {
firestore.updateDocument(collection + "/" + data[r][0], newDoc[r]);
//Now let's insert a timestamp in our Firestore TS column of the sheet so we know it's been updated to Firestore
cellFS.setValue(now);
Logger.log("Row ",r,"(",data[r][1],") updated/edited.");
} catch (e) {
Logger.log("Error: ",e," : Document existed, we tried updating it, but jack shit happened.");
}
}
else {
Logger.log("Row ",r,"(",data[r][1],") Already in Firestore & hasn't been modified. Skipped.");
}
}
}
Step 5
Once your script is modified to your needs, it's time to run the script. Simply save it (File > Save), then choose the function "writeToFireStore" from the "Select function" dropdown selector in the menu bar (in between the icon of the bug, and the lightbulb), then hit the PLAY icon (to the left of the bug icon). At this point, you will likely be prompted to accept permissions to run the script, (which you need to accept if you want to run the script). Once you've accepted the permissions, then run the "writeToFireStore" function again if it hasn't already run, and voila!
NOTES:
I created a function that automatically writes a Modified Timestamp to the second to last column in the target sheet, and when you run the function, writes a Firestore Timestamp (so you know which rows have been successfully exported to Firestore). This way, if you run the firestore function again, and you haven't changed the data on your sheet, it won't bother updating the database with the same data (and will save you money and/or server resources). For this functionality to work, you must setup project Triggers (which is explained inside the script in the comments).

Making text into an array - Swift

For fun I'm helping my school out by creating an app which has all class cancellations for student use. From my IT technician I got a quite complex structure containing class name, teacher, and other information looking like this:
3818,"20170217",5,752,64,"Rh",,"fr_2",,,,"iV5",,,"IS10a~IS10b~IS10c~IS10d","Z",,1,"IS10a~IS10b~IS10c~IS10d",C,201702161517,"-"
3819,"20170217",6,752,102,"Rh",,"fr",,,,"iB3","iB3",,"IT10a","Z",,0,"IT10a",,201702161517,"-"
3820,"20170217",8,752,119,"Rh",,"fr",,,,"iC1.2","iC1.2",,"IS6a","Z",,0,"IS6a",,201702161517,"-"
3821,"20170227",2,753,207,"Dd","Kru","sc",,,,"iB8","iB8",,"IS9b","Z",,2097152,"IS9b",,201702270804,"+~-"
3822,"20170227",3,753,8,"Dd",,"phH_1",,,,"iB8",,,"IS12~IT12","Z",,2097153,"IS12~IT12",C,201702270804,"-"
3823,"20170227",4,753,29,"Dd",,"phH_1",,,,"iB8",,,"IS11~IT11","Z",,2097153,"IS11~IT11",C,201702270804,"-"
3824,"20170227",5,753,30,"Dd",,"phH_1",,,,"iB8",,,"IS11~IT11","Z",,2097153,"IS11~IT11",C,201702270804,"-"
3825,"20170227",6,753,7,"Dd",,"phH_1",,,,"iB8",,,"IS12~IT12","Z",,2097153,"IS12~IT12",C,201702270804,"-"
3826,"20170227",7,753,327,"Dd",,"COV",,,,"AC1",,,,"Z",,2097153,,,201702270803,
3827,"20170227",8,753,46,"Dd",,"ph_1",,,,"iB8",,,"IS10a~IS10b~IS10c~IS10d~IT10a~IT10b","Z",,2097153,"IS10a~IS10b~IS10c~IS10d~IT10a~IT10b",C,201702270804,"-"
From this data I need to get various pieces, such as "20170217" and put them into an array for later use. How would I best do this? For anyone who cares, I added the full snippet below!
https://jsfiddle.net/pztwfsq1/
Since there is one dataset per line you can iterate through all lines. Split each line at , and you'll have an array of the information.
Similar to this (to give you an idea):
let row = "1,Peter,5,92,,Brooklyn"
let data = row.components(separatedBy: ",")
let name = data[1] // Peter
let location = data[5] // Brooklyn

Protovis - dealing with a text source

lets say I have a text file with lines as such:
[4/20/11 17:07:12:875 CEST] 00000059 FfdcProvider W com.test.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:/Prgs/testing/WebSphere/AppServer/profiles/ProcCtr01/logs/ffdc/server1_3d203d20_11.04.20_17.07.12.8755227341908890183253.txt com.test.testserver.management.cmdframework.CmdNotificationListener 134
[4/20/11 17:07:27:609 CEST] 0000005d wle E CWLLG2229E: An exception occurred in an EJB call. Error: Snapshot with ID Snapshot.8fdaaf3f-ce3f-426e-9347-3ac7e8a3863e not found.
com.lombardisoftware.core.TeamWorksException: Snapshot with ID Snapshot.8fdaaf3f-ce3f-426e-9347-3ac7e8a3863e not found.
at com.lombardisoftware.server.ejb.persistence.CommonDAO.assertNotNull(CommonDAO.java:70)
Is there anyway to easily import a data source such as this into protovis, if not what would the easiest way to parse this into a JSON format. For example for the first entry might be parsed like so:
[
{
"Date": "4/20/11 17:07:12:875 CEST",
"Status": "00000059",
"Msg": "FfdcProvider W com.test.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I",
},
]
Thanks, David
Protovis itself doesn't offer any utilities for parsing text files, so your options are:
Use Javascript to parse the text into an object, most likely using regex.
Pre-process the text using the text-parsing language or utility of your choice, exporting a JSON file.
Which you choose depends on several factors:
Is the data somewhat static, or are you going to be running this on a new or dynamic file each time you look at it? With static data, it might be easiest to pre-process; with dynamic data, this may add an annoying extra step.
How much data do you have? Parsing a 20K text file in Javascript is totally fine; parsing a 2MB file will be really slow, and will cause the browser to hang while it's working (unless you use Workers).
If there's a lot of processing involved, would you rather put that load on the server (by using a server-side script for pre-processing) or on the client (by doing it in the browser)?
If you wanted to do this in Javascript, based on the sample you provided, you might do something like this:
// Assumes var text = 'your text';
// use the utility of your choice to load your text file into the
// variable (e.g. jQuery.get()), or just paste it in.
var lines = text.split(/[\r\n\f]+/),
// regex to match your log entry beginning
patt = /^\[(\d\d?\/\d\d?\/\d\d? \d\d:\d\d:\d\d:\d{3} [A-Z]+)\] (\d{8})/,
items = [],
currentItem;
// loop through the lines in the file
lines.forEach(function(line) {
// look for the beginning of a log entry
var initialData = line.match(patt);
if (initialData) {
// start a new item, using the captured matches
currentItem = {
Date: initialData[1],
Status: initialData[2],
Msg: line.substr(initialData[0].length + 1)
}
items.push(currentItem);
} else {
// this is a continuation of the last item
currentItem.Msg += "\n" + line;
}
});
// items now contains an array of objects with your data