I have integrated with Rally which downloads test cases .Every Test case has its own test data in excel spread sheet form.
I am planning to consolidate all test cases excel data into single excel sheet and read the test data from this consolidated excel as part of data driven testing.
So I would like to know how to read from excel and write into excel in protractor.
Hope i am clear .
Thank you.
You can use one of these node packages.
https://www.npmjs.com/package/xlsx
https://www.npmjs.com/package/edit-xlsx
I think the second one would be ideal for you as you need to edit existing excel files.
I'm using Exceljs to make my test cases data driven.
https://www.npmjs.com/package/exceljs
Code sample for reading from excel:
var Excel = require('exceljs');
var wrkbook = new Excel.Workbook();
wrkbook.xlsx.readFile('Auto.xlsx').then(function() {
var worksheet = wrkbook.getWorksheet('Master');
worksheet.eachRow(function (Row, rowNumber) {
console.log("Row " + rowNumber + " = " + JSON.stringify(Row.values));
});
});
Related
I have a quick question that maybe someone can help me with. I am fairly new to MongoDB and I made it pretty far on my own so far.
I have created a database that contains all my employees, emails, job titles.
I uploaded these from a CSV file but the CSV file is pretty huge to edit. I was wondering if there a way I can insert a row into it so I can label them such as: Name:Email:Title
I havent been able to figure this out and any help would be great.
In PHP you could do something like this :
$manager = new MongoDB\Driver\Manager("mongodb://localhost:27017");
$insRec = new MongoDB\Driver\BulkWrite;
$insRec->insert(['name' =>'Max CodeSmith', 'email'=>'info#maxcodesmith.com', 'title'=>'Solutions Architect']);
$writeConcern = new MongoDB\Driver\WriteConcern(MongoDB\Driver\WriteConcern::MAJORITY, 1000);
$result = $manager->executeBulkWrite('your.collection', $insRec, $writeConcern);
Is there a way to read an Excel file stored in a GCS bucket using Dataflow?
And I would also like to know if we can access the metadata of an object in GCS using Dataflow. If yes then how?
CSV files are often used to read files from excel. These files can be split and read line by line so they are ideal for dataflow. You can use TextIO.Read to pull in each line of the file, then parse them as CSV lines.
If you want to use a different binary excel format, then I believe that you would need to read in the entire file and use a library to parse it. I recommend using CSV files if you can.
As for reading the GCS metadata. I don't think that you can do this with TextIO, but you could call the GCS API directly to access the metadata. If you only do this for a few files at the start of your program then it will work and not be too expensive. If you need to read many files like this, you'll be adding an extra RPC for each file.
Be careful to not read the same file multiple times, I suggest reading each file's metadata once once and then writing the metadata out to a side input. Then in one of your ParDo's you can access the side input for each file.
Useful links:
ETL & Parsing CSV files in Cloud Dataflow
https://cloud.google.com/dataflow/java-sdk/JavaDoc/com/google/cloud/dataflow/sdk/io/TextIO.Read
https://cloud.google.com/dataflow/model/par-do#side-inputs
private static final int BUFFER_SIZE = 64 * 1024;
private static void printBlob(com.google.cloud.storage.Storage storage, String bucketName, String blobPath) throws IOException, InvalidFormatException {
try (ReadChannel reader = ((com.google.cloud.storage.Storage) storage).reader(bucketName, blobPath)) {
InputStream inputStream = Channels.newInputStream(reader);
Workbook wb = WorkbookFactory.create(inputStream);
StringBuffer data = new StringBuffer();
for(int i=0;i<wb.getNumberOfSheets();i++) {
String fName = wb.getSheetAt(i).getSheetName();
File outputFile = new File("D:\\excel\\"+fName+".csv");
FileOutputStream fos = new FileOutputStream(outputFile);
XSSFSheet sheet = (XSSFSheet) wb.getSheetAt(i);
Iterator<Row> rowIterator = sheet.iterator();
data.delete(0, data.length());
while (rowIterator.hasNext())
{
// Get Each Row
Row row = rowIterator.next();
data.append('\n');
// Iterating through Each column of Each Row
Iterator<Cell> cellIterator = row.cellIterator();
while (cellIterator.hasNext())
{
Cell cell = cellIterator.next();
// Checking the cell format
switch (cell.getCellType())
{
case Cell.CELL_TYPE_NUMERIC:
data.append(cell.getNumericCellValue() + ",");
break;
case Cell.CELL_TYPE_STRING:
data.append(cell.getStringCellValue() + ",");
break;
case Cell.CELL_TYPE_BOOLEAN:
data.append(cell.getBooleanCellValue() + ",");
break;
case Cell.CELL_TYPE_BLANK:
data.append("" + ",");
break;
default:
data.append(cell + ",");
}
}
}
fos.write(data.toString().getBytes());
}
}
}
You should be able to read the metadata of a GCS file by using the GCS API. However you would need the filenames. You can do this by doing a ParDo or other transform over a list of PCollection<string> which holds the filenames.
We don't have any default readers for excel files. You can parse from a CSV file by using a text input:(ETL & Parsing CSV files in Cloud Dataflow)
I'm not very knowledgeable on excel, and how the file format is stored. If you want to process one file at a time, you can use a PCollection<string> of files. And then use some library to parse the excel file at a time.
If an excel file can be split into easily-parallelizable parts, I'd suggest you take a look at this doc (https://beam.apache.org/documentation/io/authoring-overview/). (If you are still using Dataflow SDK, it should be similar.) It may be worth splitting into smaller chunks before reading to get more parallelization out of your pipeline. In this case you could use IOChannelFactory to read from the file.
I'm trying to build a simple dashboard app using google apps script to pull data from a google spreadsheet and then display that data visually using charts on the dashboard.
I've managed to carry out those initial steps but am having problems trying to group the data to prevent multiple slices in the pie chart being assigned to the same task. In the below source spreadsheet you'll see numerous entries of the "Support" task (column E) linked to different Clients (column B). What I need is to group the data based on the "Task" column (E) so the time entries recorded in column G (Hours) are totaled and displayed accordingly (i.e. one segment of the pie chart for each task regardless of client).
I've searched the Apps Script Documentation and have essentially got lost in the noise and can't find any definitive way of doing this directly in Apps Script. I've found some info about potentially using Google Visualization to achieve something similar but am struggling with how to implement as there is so much conflicting documentation out there.
Essentially I'm asking is this possible within Apps Script directly without 1st Querying the Data and producing another sheet (as a basis for the chart) or if not then what the most straightforward and/or best approach would be?
Data Source:
https://docs.google.com/spreadsheets/d/1Gvqe89ytJxrKWOvGJWX5heYwrj5L-MP_Pe2jpTqqB5o/edit?usp=sharing
Current App displays the below:
Apps Script Code:
function doGet() {
var ss = SpreadsheetApp.openById('1Gvqe89ytJxrKWOvGJWX5heYwrj5L-MP_Pe2jpTqqB5o');
var data = ss.getDataRange();
var taskFilter = Charts.newCategoryFilter().setFilterColumnIndex(4).build();
var clientFilter = Charts.newStringFilter().setFilterColumnIndex(1).build();
var tableChart = Charts.newTableChart()
.setDataViewDefinition(Charts.newDataViewDefinition().setColumns([1,2,4,6]))
.build();
var pieChart = Charts.newPieChart()
.setDataViewDefinition(Charts.newDataViewDefinition().setColumns([4,6]))
.build();
var dashboard = Charts.newDashboardPanel().setDataTable(data)
.bind([taskFilter, clientFilter], [tableChart, pieChart])
.build();
var app = UiApp.createApplication();
var filterPanel = app.createVerticalPanel();
var chartPanel = app.createHorizontalPanel();
filterPanel.add(taskFilter).add(clientFilter).setSpacing(10);
chartPanel.add(tableChart).add(pieChart).setSpacing(10);
dashboard.add(app.createVerticalPanel().add(filterPanel).add(chartPanel));
app.add(dashboard);
return app;
}
I have a .rpt file that I have created by setting it's data source as a text (csv) file using the (Access/Excel (DAO) ) option.
Now I want the same .rpt file loaded using a C# code and each time my C# code will change the input file and I want a new report to be generated based on the data in the new text file.
I am doing the following code and when I export the file to a pdf document, it still displays the data according to the data in the old input file.
I have checked off the option in the .rpt file that says "save data with report" and "verify on first refresh".
What am I missing here?
CODE:
cryRpt = new ReportDocument();
cryRpt.Load(reportfile);
Tables tables = cryRpt.Database.Tables;
TableLogOnInfo tableLogonInfo;
foreach (Table table in cryRpt.Database.Tables)
{
tableLogonInfo = table.LogOnInfo;
tableLogonInfo.TableName = "MYdata_BS_NEW#csv";
table.Location = "MYdata_BS_NEW#csv";
table.ApplyLogOnInfo(tableLogonInfo);
}
cryRpt.Refresh();
// After this I export the report to pdf document.
How to convert excel data into xml file using ado.net?
You can use the Microsoft Jet OLEDB 4.0 Data Provider to read the Excel file. Information about how to establish a connection to an Excel file can be found here.
This article explains how to read an Excel file using the provider. Once you have read the data, you can compose your XML document using LINQ to XML or the System.Xml classes.
In Excel, you can save the file to XML by using the File menu and changing the saved file type to XML spreadsheet.
If you want to read an Excel XML file with ADO.Net, try the XmlReader.
Or see this step-by-step example from Microsoft.
I've not used ado.net, but I've used xquery very successfully for this. Use excel export to create an XML file, then write xquery/xpath commands to convert as you want. Excel XML export format is pretty gnarly but it does do the job. Use the Oxygen 30 day eval license to lighten the xquery debug job.
use this code :
public static DataSet exceldata(string filelocation)
{
DataSet ds = new DataSet();
OleDbCommand excelCommand = new OleDbCommand();OleDbDataAdapter excelDataAdapter = new OleDbDataAdapter();
string excelConnStr = "Provider=Microsoft.Jet.OLEDB.4.0; Data Source=" + filelocation + "; Extended Properties =Excel 8.0;";
OleDbConnection excelConn = new OleDbConnection(excelConnStr);
excelConn.Open();
DataTable dtPatterns = new DataTable();excelCommand = new OleDbCommand("SELECT UUID, `PATTERN` as PATTERN, `PLAN` as PLAN FROM [PATTERNS$]", excelConn);
excelDataAdapter.SelectCommand = excelCommand;
excelDataAdapter.Fill(dtPatterns);
dtPatterns.TableName = "Patterns";
ds.Tables.Add(dtPatterns);
return ds;
}
and then convert returned datatable to xml with DataTable.WriteXml()