I am trying to extract certain fields from a csv file, having comma separated values.
The issues is , some of the fields also contains comma and the fields are not enclosed within quotes. Given the scenario, how can i extract the fields.
also,only one of the field contains comma within values, and i don't need that. e.g: I want to extract the first 2 columns and the last 5 columns from the data set of 8 columns , where the third column contains values with comma
PS: Instead of down voting i would suggest to come ahead and post your
brilliant ideas if you have any.
Solution:
$path = "C:\IE3BW0047A_08112017133859.csv"
Get-Content $path| Foreach {"$($_.split(',')[0,1,-8,-7,-6,-5,-4,-3,-2,-1] -join '|')"} | set-content C:\IE3BW0047A_08112017133859_filter.csv
Related
I have a csv delimited with spaces that can change to 9-10-11, it s there a way for split the column in two with Azure data factory?
Examples:
This is my CSV
I try using dataflows:
but when I execute the dataflow, it throw me this error:
PD: the csv has 4.000.000 rows
Solve the problem using azure data factory, the csv needs to finish in my DW
I have the following data in my sample csv file with either 9, 10 or 11 spaces in between the value.
Now, after reading the column, use derived column transformation to split the required column using 9 spaces (minimum number of spaces).
req1 : split(col1,' ')[1]
req2 : split(col1,' ')[2]
This will split the data into an array of 2 elements, where 1st index will have no spaces in its value and the 2nd index element has trailing spaces.
Now apply ltrim on the req2 column. Check the length of this column before and after the transformation to confirm that we are eliminating the trailing spaces.
req2 : ltrim(req2)
After doing this, you can check the length of the req2 and it would be 1.
Now, select only the required columns and then write it to required sink.
I have to combine a lot of files , mostly CSV, already have code to combine however I need first to trim the desired csv files so I can get the data that I want. Each CSV has first 2 columns of 8 rows which contain data that I want. and then just below those there is a row that generates 8 columns. I am only having issue grabbing data from the first 8 rows of the 2 columns.
Example of the csv first 3 rows:
Target Name: MIAW
Target OS: Windows
Last Updated On: June 27 2019 07:35:11
This is the data that I want, the first 3 rows are like this, with 2 columns. My idea is to store the 3 values of the 2nd column each into a variable and then use it with the rest of my code.
As I only have a problem extracting the data, and since the way the CSV are formated there is no header at all, it is hard to come up with an easy way to read the 2nd column data. Below is an example, this of course will be used to process several files so it will be a foreach, but I want to come up first with the simple code for 1 file so I can adapt it myself to a foreach.
$a = MIAW-Results-20190627T203644Z.csv
Write-Host "$a[1].col2"
This would work if and only if I had a header called col2, I could name it with the first value on the 2nd column but the issue is that that value will change for CSV file. So the code I tried would not work for example if I were to import several files using:
$InFiles = Get-ChildItem -Path $PSScriptRoot\ *.csv -File |
Where-Object Name -like *results*
Each CSV will have a different value for the first value on the 2nd column.
Is there an easier way to just grab the 3 rows of the second column that I need? I need to grab each one and store each in a different variable.
I have two fields that contain concatenated strings. The first field contains medical codes and the second field contains the descriptions of those codes. I don't want to break these into multiple fields because some of them would contain hundreds of splits. Is there any way to break them into a row each like below? The code and description values are separated by a semicolon (;)
code description
----- ------------
80400 description1
80402 description2
A sample of the data:
One way is you can custom split two columns at ; which will create separate columns for every entry then you can pivot code columns and description columns separately.
One issue will be you can't guarantee if every code is mapped to correct description.
One more way is export data to excel sheet and then split and pivot the columns and then match the code and description, Then take the excel as datasource to the tableau.
I have a CSV file that has a static number of columns (with headers) and a dynamic number of rows and I need convert it to another CSV which has a static number of rows and a dynamic number of columns (with or without headers).
Here is an example of what the before and after would look like in a spreadsheet. Please keep in mind that the top section would represent my input data and the bottom section is the desired state.
Of course this is a small subset as I could have a user that has 50 groups and a user which may only have 1 group associated with it. This leaves me with not knowing the number of columns in advance so it has to be dynamic.
I have tried using what I found on this post, but I am running into an issue trying to modify the code to fit my needs. The closest I have been able to do is create a row for each unique user ID but instead of having the corresponding group names in columns next to each user, they are set as the headers and no values for the users.
If you don't require headers or defined columns you could simply collect the values from the second column in a hashtable of arrays where you use the values from the first column as keys:
$data = #{}
Import-Csv 'C:\path\to\input.csv' | ForEach-Object {
$data[$_.UserID] += #($_.GroupName)
}
Then you can export the data to a text file like this:
$data.Keys | ForEach-Object {
$_ + ',' + ($data[$_] -join ',')
} | Set-Content 'C:\path\to\output.txt'
After running Execute query write results to file - the columns in my output file for datatype money get broken into two columns. e.g if my revenue is $500 it is displayed correctly. But, if my revenue is $1,500.00 - there is an issue. It gets broken into two columns $1 and $500.00
Can you please help me getting my results in a csv file in a single column for datatype money?
What is this command "execute query write results to file"? Do you mean COPY? If so, have a look at the FORCE QUOTE option http://www.postgresql.org/docs/current/static/sql-copy.html
Eg.
COPY yourtable to '/some/path/and/file.csv' CSV HEADER FORCE QUOTE *;
Note: if the application that is consuming the csv files still fails because of the comma, you can change the delimiter from "," to whatever works for you (eg. "|").
Additionally, if you do not want CSV, but you do want TSV, you can omit the CSV HEADER keywords and the results will output in tab-separated format.
Comma is the list separator of our computer for some regions, some region semicolon is the list separator. so I think you need to replace the comma when you write it to csv.