I have a BIRT report with multiple tables with different datasets and number of columns on it. I generate output in .xls and convert into .csv using ssconvert utility on Unix. But in the .csv file I see extra delimiter for tables where there are fewer columns. For example, here is the .csv output with extra "," in .csv file:
table1-- this has only 10 columns
5912,,,0,,,0,,0,,,0,,,0,,,0,,,
tables2 --this has 20 columns
'12619493',28/03/2018 17:27:40,sdfsdfasd,'61901492478'1.08,,,1.08,sdfs,,dsf,,sdfadfs,'738331',,434,,,,,,,333,
I try to put grid but still I see extra ",". I have opened the .xls file and I see it has same issue. The cells in Excel are merged.
Related
I have got a very large table with around 28 columns and 900k records.
I converted it to CSV file (Pipe separated) and then tried to use that file for feeding another table using ADF itself.
When I tried to use that file, it keeps triggering an error saying some column datatype mismatch.
So excavating more into the data I have found few rows having Pipe (|) symbol in their text itself. So at the time coverting it back, the text after the pipe been considered for the next column and thus the error.
So how to handle the conversion into CSV efficiently when there are texts with delimiters in their columns.
Option1: If there is a possibility, I would suggest changing the delimiter to other than pipe(|), as the column value also contains pipe in its text.
Option2: In the CSV dataset, select a Quote character to identify the columns.
Step1: Copying data from table1 to CSV.
Source:
Sink CSV dataset:
Output:
Step2: Loading same CSV data to table2 with a copy activity.
CSV output file of Step1.
Source CSV dataset:
Sink dataset:
Output:
I have a huge dataset comprising of about 600000000 rows. I need to export all these rows in pipe separated values format, and data should be in multiple files instead of 1 huge file, with each file containing 100000000 rows.
I tried exporting this file with right click --> Export --> Delimited --> Multiple files, but there is no option to specify number of rows in each file, and data is exported in .dsv format not .psv format.
Is there any way to achieve this?
I have thousands of csv files and they basically have 2 formats. One type of 2 formats is that in those csv files there are 100 rows and 2 columns. The other type of csv files has 50 columns and 5 rows. The numbers are given just to provide an example.
What I want to do is to write a Matlab code that will extract the complete second row of the csv files with the first format and make it the first row of the csv files with the second format. The number of the csv files with the first and second format is equal.
Any help is appreciated.
When exporting a TSQL select result to CSV the values show a strange thousandseparator. Partial code is:
CONVERT(DECIMAL(10,2),i.UserNumber_04) as CAP
The query results show perfect values, for example: 1470.00 but the CSV or txt file show strange values like 1,470,00. How can I prevent the first comma?
At first I thought it was just the formatting style in excel, but it does the same in txt files.
I have a set of pivot tables that use external csv files as their data sources. The csv files originally contained dates in the format dd/mm/yy (e.g. 31/01/13). The pivot tables did not recognise these as dates. I converted the dates in the csv files to dd/mm/yyyy (e.g. 31/01/2013) but these were still not recognised as dates by the pivot tables.
I tried setting up a calculated field =DATEVALUE(date_from_csv) but when used in the pivot table (I'm using the Max option to select the most recent date) I get #VALUE! errors.
I have tried converting the csv file to xlsx and also importing the data into the workbook that contains the pivot table - but I can't change from the external connection to use the internal data. I don't want to rebuild the pivots as there are a lot of variables and formatting that would take ages to redo.
Any ideas??
The problem was caused by the date column being blank for some rows and I found that if I moved a row to the top (after the header line) that had all the fields filled in, then Excel got the formats correct and the pivot tables now work!