how to append Multiple sheets from multiple workbooks - append

I have 5 workbooks with hundreds of sheets with same name and same columns in each workbook, I want to append the data of each sheet of all workbooks separately and export to CSV or XLS, i am appending one by one
dir_path = r'D:\Input\\'
res = []
for path in os.listdir(dir_path):
if os.path.isfile(os.path.join(dir_path, path)):
res.append(path)
FilePath_1=dir_path+ res[0]
FilePath_2=dir_path+ res[1]
Data1 = pd.read_excel (FilePath_1,sheet_name='Data',header=None)
Data1.drop(Data1.head(1).index, inplace=True)
Data2 = pd.read_excel (FilePath_2,sheet_name='Data',header=None)
Data2.drop(Data2.head(2).index, inplace=True)
Data=pd.concat([Data1, Data2])
Data.to_excel(r'D:\Data.xlsx', index = False,header=None)

Related

Exporting output results from a model into the input of a different model

I'm trying to build a model of a factory using the personal learning edition of AnyLogic. Since this version has a limited number of blocks per model, building the full factory on a single model is presenting itself as an impossible task. In order to surpass this issue I want to split the factorys main processes into different models, which means I'll have to feed the output of process A into the input of process B.
My question is: how can I export a time stamped output of a model into the input of a different model?
Thank you in advance.
You have 2 options
Option 1: Through an Excel file (or txt file)
Simply link an Excel file in your model, using the object from the connectivity palette
Then you can get the data using code similar to below
int excelRow = 2;
String sheetName = "Sheet1!";
String cellName = sheetName + "A" + excelRow;
while (excelFile.cellExists( cellName )) {
int x = (int)excelFile.getCellNumericValue( sheetName + "A" + excelRow);
int b = (int)excelFile.getCellNumericValue( sheetName + "B" + excelRow);
int c = (int)excelFile.getCellNumericValue( sheetName + "C" + excelRow);
boolean d = excelFile.getCellBooleanValue( sheetName + "D" + excelRow);
excelRow ++; // Increase the row that we will lookup in the Excel
}
Just a while loop where you go from one excel line to the next as long as the line exists, and then do what ever is needed with the data
Option 2: AnyLogic Internal DB
Simply import your excel sheet to the AnyLogic DB and then loop over the entries in the table using a for loop
List<Tuple> rows = selectFrom(db_table).list();
for (Tuple row : rows) {
traceln(
row.get( db_table.db_column )
);
}

Bulk copy filtered rows from one google sheet to another google sheet

I have a spreadsheet where I am able to filter the sheet based on a value on a column. I can copy the filtered data using isRowHiddenByFilter. But this does it one row at a time. I am looking for some input on how I can copy say 200 rows that I obtain after using a filter to be copied to another spreadsheet all at once and not evaluating 200 rows.
This is what I have working:
var criteria = SpreadsheetApp.newFilterCriteria().whenTextContains("I do NOT
plan").build();
ss.getActiveSheet().getFilter().setColumnFilterCriteria(4,criteria);
var nrsheet = ss.getSheetByName("Not Returning");
for (var i = 2; i < sheet.getLastRow(); i++)
{
if(!sheet.isRowHiddenByFilter(i))
{
row_data = sheet.getRange(i, 1, 1, sheet.getLastColumn()).getValues();
one_arr_nr = row_data.join().split(",");
nrsheet.appendRow(one_arr_nr);
}
}
This is what I would like to do:
Remove the for loop to evaluate each row and be able to copy what I see on the spreadsheet to be copied to the Not Returning sheet. Any help on how I can proceed?

Merging tbl_svysummary and stacked tbl_regression tables with different variable names but same labels

Follow up question to (Renaming Rows in gtsummary, tbl_regression/tbl_stack):
I am now trying to merge the renamed, stacked table (Table 1) with a tbl_summary table that includes the prevalence for each of the outcomes (Table 2). However, because each renamed line of Table 1 is, in reality, just the same variable repeated over and over again, it doesn't merge with Table 2, instead creating a (Table 3) that has duplicated outcome names stacked onto one another. Any way to merge these tables so that the lines of Table 1 match seamlessly with those from Table 2?
UPDATE:
As of gtsummary v 1.4.0, tbl_uvregression() now accepts survey objects.
library(gtsummary)
packageVersion("gtsummary")
#> [1] '1.4.0'
# convert trial data frame to survey object
tbl <-
survey::svydesign(
data = trial[c("response", "death", "age", "marker")],
ids = ~1,
weights = ~1
) %>%
# build univariate regression models
tbl_uvregression(
x = age,
method = survey::svyglm,
method.args = list(family = binomial),
exponentiate = TRUE,
formula = "{y} ~ {x} + marker",
label = list(response = "Response", death = "Death"),
hide_n = TRUE,
include = -marker
) %>%
add_n() %>%
add_nevent() %>%
modify_header(
label = "**Outcome**",
estimate = "**Age OR**"
)
Created on 2021-04-14 by the reprex package (v2.0.0)

Create multiple RDDs from single file based on row value ( header record in sample file) using Spark scala

I am trying to create multiple RDDs to process independently from below file based on the similar format of data .
Please find the file with different data formats
custid,starttime,rpdid,catry,auapp,sppp,retatype,status,process,fileavil
4fgdfg,00:56:30.034,BM_-unit1,GEN,TRUE,FALSE,NONE,A,45,TRUE
X95GEK,00:56:32.083,CBM_OMDD_RSVCM0CBM-unit0,GEN,TRUE,FALSE,NONE,A,GWC,TRUE
XWZ08K,00:57:01.947,GWC-0-UNIT-1,GEN,TRUE,FALSE,NONE,A,GWC,TRUE
custid,relstatus
fg3-03,R
dfsdf4-01,V
56fbfg,R
devid,reg,hold,devbrn,lname,lcon
CTUTANCM0CBM,TRUE,FALSE,13:17:36.934,CBM_BMI_25_5_2,13:43:21.370
In the above file, we have three different type of data formats exist and I want to split the file into three different RDDs as per the format.
Could you please suggest how to implement using Spark (Scala)?
Your file looks like it has 3 different csv files in it.
You can read it as a single file and extract 3 RDDs from it based on the number of fields you have in each row.
// Caching because you'll be filtering it thrice
val topRdd = sc.textFile("file").cache
topRdd.count
//res0: Long = 10
val rdd1 = topRdd.filter(_.split(",", -1).length == 10 )
val rdd2 = topRdd.filter(_.split(",", -1).length == 2 )
val rdd3 = topRdd.filter(_.split(",", -1).length == 6 )
rdd1.collect.foreach(println)
// custid,starttime,rpdid,catry,auapp,sppp,retatype,status,process,fileavil
// 4fgdfg,00:56:30.034,BM_-unit1,GEN,TRUE,FALSE,NONE,A,45,TRUE
// X95GEK,00:56:32.083,CBM_OMDD_RSVCM0CBM-unit0,GEN,TRUE,FALSE,NONE,A,GWC,TRUE
// XWZ08K,00:57:01.947,GWC-0-UNIT-1,GEN,TRUE,FALSE,NONE,A,GWC,TRUE
rdd2.collect.foreach(println)
// custid,relstatus
// fg3-03,R
// dfsdf4-01,V
// 56fbfg,R
rdd3.collect.foreach(println)
// devid,reg,hold,devbrn,lname,lcon
// CTUTANCM0CBM,TRUE,FALSE,13:17:36.934,CBM_BMI_25_5_2,13:43:21.370

Drill down column chart with highcharts4gwt

I want to drill down column chart but using highcharts4Gwt . I am doing
Array<Data> data1 = series1.dataAsArrayObject();
Data dat = highchartsFactory.createSeriesColumnData();
dat.y(15);
dat = dat.selected(false);
Data drillDownSeries = dat.drilldown("drill");
drillDownSeries.y(5);
drillDownSeries = drillDownSeries.selected(false);
data1.setValue(0, dat);
options.series().addToEnd(series1);
but column gets clicked bydefault. I don't want using json , but by the above way with 2 level. Is there any way to do this?
Thanks,