<EDIT: I kind of have it working, but in order to get it to work, my template csv has to have a blank line for every line I am going to be adding to it. So, if I could figure out how to add lines to the imported empty (just a header row) csv file, I could then use export-csv at the end. (It would be somewhat slower, but it would at least work.)>
I am creating a .csv file in PowerShell. The output file has 140 columns. Many of them are null.
I started out just doing
$out = 'S-'+$Snum+',,,,,TRUE,,,,,'+'S-'+$Snum+',"'
$out = $out + '{0:d9}' -f $item.SupplierCode2
until I had filled all the columns with the correct value. But, the system that is reading the output keeps changing the column locations. So, I wanted to take the header row from the template for the system and use that to name the columns. Then, if the columns change location, it won't matter because I will be referring to it by name.
Because there are so many columns, I'm trying to avoid a solution that has me enter all the column names. By using a blank .csv with just the headers, I can just paste that into the csv whenever it changes and I won't have to change my code.
So, I started by reading my csv file in so I can use the headers.
$TempA = Import-Csv -Path $Pathta -Encoding Default
Then I was hoping I could do something like this:
$TempA.'Supplier Key' = "S-$Snum"
$TempA.'Auto Complete' = "TRUE"
$TempA.'Supplier ID' = "S-$Snum"
$tempA.'Supplier Data - Supplier Reference ID' = '{0:d9}' -f $item.SupplierCode2
I would only need to fill in the fields that have values, everything else would be null.
Then I was thinking I could write out this record to a file. My old write looked like this
$writer2.WriteLine($out)
I wanted to write the line from the new csv line instead
$writer2.WriteLine($TempA)
I'd rather use streams if I can because the files are large and using add-Content really slows things down.
I know I need to do something to add a line to $TempA and I would like each loop to start with a new line (with all nulls) because there are times when certain lines only have a small subset of the values populated.
Clearly, I'm not taking the correct approach here. I'd really appreciate any advice anyone can give me.
Thank you.
If you only want to fill in certain fields, and don't mind using Export-Csv you can use the -append and -force switches, and it will put the properties in the right places. For example, if you had the template CSV file with only the column names in it you could do:
$Output = ForEach($item in $allItems){
[PSCustomObject]#{
'Supplier Key' = "S-$Snum"
'Auto Complete' = "TRUE"
'Supplier ID' = "S-$Snum"
'Supplier Data - Supplier Reference ID' = '{0:d9}' -f $item.SupplierCode2
}
}
$Output | Export-Csv -Path $Pathta -Append -Force
That would create objects with only the four properties that you are interested in, and then output them to the CSV in the correct columns, adding commas as needed to create blank values for all other columns.
Related
long time listener first time caller.
Normally I am pretty good at finding and digging and getting what I need and then modifying it to suit. This one seems to be a little trickier than what I have managed to pull off before. I am self taught in PowerShell mostly out of curiosity to see what I can do.
I am trying to create a report from data from 2 CSVs, and "most" of the data in the 2 CSVs are identical. There is simply 1 column of data in one of the CSVs that I want to add to the other one. I live regularly in the world of excel and I can do this with a formula in a matter of seconds [=VLOOKUP(H8,C:C,2,FALSE)] but accomplishing the same goal in PowerShell seems to be eluding me.
As I mentioned, I tend to try and find others who have done similar things and modify it. The best sounding one I found here ( Combine data from 2 CSV files into 1 new CSV using Powershell ) and I am still trying to play with the code on that site. Sometimes I find something and I try and stick with it too long where there might be another command that I am not familiar with that is better suited to what I should be looking at and might just need a pointer in that direction.
But here is a visual representation of what I am trying to do.
And every email address in File 2, is present in File 1.
Use Import-Csv to parse both CSV input files into arrays of [pscustomobject] instances for OOP processing.
For file 2, build a hashtable that maps the Email column values to their License values.
Then use a calculated property with Select-Object to append a property to the objects parsed from file 1, using the hashtable to map each Email property to the License value from file 2; if there is no hashtable entry for a given Email property value, $null is returned, which in the context of exporting to CSV (with Export-Csv) amounts to an empty field (column value).
# Import file 2 and create a hashtable that maps each Email
# column value to the License column value.
$ht = #{}
Import-Csv File2 | ForEach-Object { $ht[$_.Email] = $_.License }
# Import file 1 and append a License column that contains
# the license value from file 2 if the Email column value matches.
Import-Csv File1 |
Select-Object *, #{ Name='License'; Expression={ $ht[$_.Email] } }
# | Export-Csv ... # complete as needed
I have a CSV column with alpha numerical combinations in a column.
I am later going to use this csv file in a PowerShell script by importing the data.
Examples: 1A01, 1C66, 1E53.
Now before putting these values in, I made sure to format the column as text.
Now at first it works. I input the data, save. I test in PowerShell to import it and
all data shows up valid including 1E53. But lets say I edit the file again later to add data and then save and close. I re-import into PowerShell and 1E53 comes in as 1.00E+53. How can I prevent this permanently? Note that the column is filled with codes and there are lots of #E##.
Your issue is not with PowerShell, its with Excel. For a demonstration, take 1E53 and enter it into Excel and then save that excel file as a CSV file. You will see that the value is now changed to 1.00E+53.
How to fix this?
There are a few ways of disabling scientific notation:
https://superuser.com/questions/452832/turn-off-scientific-notation-in-excel
https://www.logicbroker.com/excel-scientific-notation-disable-prevent/
I hope some of them work for you.
I think you can rename the file to .txt instead of .csv and excel may treat it differently.
Good Luck
As commented:
You will probably load the csv from file:
$csv = Import-Csv -Path 'X:\original.csv' -UseCulture
The code below uses a dummy csv in a Here-String here:
$csv = #'
"Column1","Column2","ValueThatMatters"
"Something","SomethingElse","1E53"
"AnotherItem","Whatever","4E12"
'# | ConvertFrom-Csv
# in order to make Excel see the values as Text and not convert them into scientific numbers
$csv | ForEach-Object {
# add a TAB character in front of the values in the column
$_.ValueThatMatters = "`t{0}" -f $_.ValueThatMatters
}
$csv | Export-Csv -Path 'X:\ExcelFriendly.csv' -UseCulture -NoTypeInformation
I would like to split the following csv into two csvs
StartOrder,1,SupplierName,
Line,2,12345,2,5,5.50,
Line,3,12345,3,6,5.20,
Line,4,12345,3,7,1.99,
EndOrder,5,booked as soon as possible to deliver.
StartOrder,6,SupplierName
Line,7,100015,2,5,5.50,
Line,8,100015,3,6,5.20,
Line,9,100015,3,7,1.99,
EndOrder,10,booked as soon as possible to deliver.
in order to be:
1st file
StartOrder,1,SupplierName,
Line,2,12345,2,5,5.50,
Line,3,12345,3,6,5.20,
Line,4,12345,3,7,1.99,
EndOrder,5,booked as soon as possible to deliver.
2nd file
StartOrder,6,SupplierName
Line,7,100015,2,5,5.50,
Line,8,100015,3,6,5.20,
Line,9,100015,3,7,1.99,
EndOrder,10,booked as soon as possible to deliver.
I have tried using GroupBy but is not working as I am expecting.
Any help?
This is something i would do with Regular Expressions.
$orders = (get-content -path C:\temp\orders.txt)
$orders = [string]::Join("`n",$orders) # this is to make sure you keep your lines
$output = [regex]::Matches($orders,'(?s)(StartOrder,(\d{0,}).*?deliver.)') # added regex option S
foreach($c in $output){
$order = $c.groups[2].value #order name that will serve as filename
""
$c.groups[0].value # content of order
$c.groups[0].value | out-file C:\temp\$order.txt -Force
}
This wil create a 1.txt and a 6.txt with its needed content.
EDIT : The only issue is that it doesn't keep the enters. -> FIXED THAT
The Regex is fairly simple, more detail on the regex : https://regex101.com/r/J0Xsu7/1
This will give you file 1.txt with
StartOrder,1,SupplierName,
Line,2,12345,2,5,5.50,
Line,3,12345,3,6,5.20,
Line,4,12345,3,7,1.99,
EndOrder,5,booked as soon as possible to deliver.
This will give you file 6.txt with
StartOrder,6,SupplierName
Line,7,100015,2,5,5.50,
Line,8,100015,3,6,5.20,
Line,9,100015,3,7,1.99,
EndOrder,10,booked as soon as possible to deliver.
I am having problem with filtering a specific value from my csv file. The csv file looks as follows:
"1","19/Oct/2016","15:03:58","19/Oct/2014","15:03:58","0:00:00","---","---","nice_meme#help.com","---","sip","1232Kbps","---","Out","1140","1","---","---","---","user:---","---","---","---","---","---","---","---","Failed Attempt; ""Your call could not be completedOver.""","3","---","---","---","---","---","---","---","---","---","---","---","---","---"
As you can see there are multiple values with '---', I tried a lot of ways to remove these three dashes. I do not know how to filter them out using PowerShell. I want to get the rows that are not equal to the three dashes.
Something like this:
$a = Import-CSV -Path "C:\Transformed\test.csv" | Where-Object {$_.Header -neq "---"}
Anyone have any ideas on how to rename files by finding an association with an index file?
I have a file/folder structure like the following:
Folder name = "Doe, John EO11-123"
Several files under this folder
The index file(MS Excel) has several columns. It contains the names in 2 columns(First and Last). It also has a column containing the number EO11-123.
What I would like to do is write maybe a script to look at the folder names in a directory, compare/find an associated value in the index file(like that number EO11-123) and then rename all the files under the folder using a 4th column value in the index.
So,
Folder name = "Doe, John EO11-123", index column1 contains same value "EO11-123", use column2 value "111111_000000" and rename all the files under that directory folder to "111111_000000_0", "111111_000000_1", "111111_000000_2" and so on.
This possible with powershell or vbscript?
Ok, I'll answer your questions in your comment first. Importing the data into PowerShell allows you to make an array in powershell that you can match against, or better yet make a HashTable to reference for your renaming purposes. I'll get into that later, but it's way better than trying to have PowerShell talk to Excel and use Excel's search functions because this way it's all in PowerShell and there's no third party application dependencies. As for importing, that script is a function that you can load into your current session, so you run that function and it will automatically take care of the import for you (it opens Excel, then opens the XLS(x) file, saves it as a temp CSV file, closes Excel, imports that CSV file into PowerShell, and then deletes the temp file).
Now, you did not state what your XLS file looks like, so I'm going to assume it's got a header row, and looks something like this:
FirstName | Last Name | Identifier | FileCode
Joe | Shmoe | XA22-573 | JS573
John | Doe | EO11-123 | JD123
If that's not your format, you'll need to either adapt my code, or your file, or both.
So, how do we do this? First, download, save, and if needed unblock the script to Import-XLS. Then we will dot source that file to load the function into the current PowerShell session. Once we have the function we will run it and assign the results to a variable. Then we can make an empty hashtable, and for each record in the imported array create an entry in the hashtable where the 'Identifier' property (in your example above that would be the one that has the value "EO11-123" in it), make that the Key, then make the entire record the value. So, so far we have this:
#Load function into current session
. C:\Path\To\Import-XLS.ps1
$RefArray = Import-XLS C:\Path\To\file.xls
$RefHash = #{}
$RefArray | ForEach( $RefHash.Add($_.Identifier, $_)}
Now you should be able to reference the identifier to access any of the properties for the associated record such as:
PS C:\> $RefHash['EO11-123'].FileCode
JD123
Now, we just need to extract that name from the folder, and rename all the files in it. Pretty straight forward from here.
Get-ChildItem c:\Path\to\Folders -directory | Where{$_.Name -match "(?<= )(\S+)$"}|
ForEach{
$Files = Get-ChildItem $_.FullName
$NewName = $RefHash['$($Matches[1])'].FileCode
For($i = 1;$i -lt $files.count;$i++){
$Files[$i] | Rename-Item -New "$NewName_$i"
}
}
Edit: Ok, let's break down the rename process here. It is a lot of piping here, so I'll try and take it step by step. First off we have Get-ChildItem that gets a list of folders for the path you specify. That part's straight forward enough. Then it pipes to a Where statement, that filters the results checking each one's name to see if it matches the Regular Expression "(?<= )(\S+)$". If you are unfamiliar with how regular expressions work you can see a fairly good breakdown of it at https://regex101.com/r/zW8sW1/1. What that does is matches any folders that have more than one "word" in the name, and captures the last "word". It saves that in the automatic variable $Matches, and since it captured text, that gets assigned to $Matches[1]. Now the code breaks down here because your CSV isn't laid out like I had assumed, and you want the files named differently. We'll have to make some adjustments on the fly.
So, those folder that pass the filter will get piped into a ForEach loop (which I had a typo in previously and had a ( instead of {, that's fixed now). So for each of those folders it starts off by getting a list of files within that folder and assigning them to the variable $Files. It also sets up the $NewName variable, but since you don't have a column in your CSV named 'FileCode' that line won't work for you. It uses the $Matches automatic variable that I mentioned earlier to reference the hashtable that we setup with all of the Identifier codes, and then looks at a property of that specific record to setup the new name to assign to files. Since what you want and what I assumed are different, and your CSV has different properties we'll re-work both the previous Where statement, and this line a little bit. Here's how that bit of the script will now read:
Get-ChildItem c:\Path\to\Folders -directory | Where{$_.Name -match "^(.+?), .*? (\S+)$"}|
ForEach{
$Files = Get-ChildItem $_.FullName
$NewName = $Matches[2] + "_" + $Matches[1]
That now matches the folder name in the Where statement and captures 2 things. The first thing it grabs is everything at the beginning of the name before the comma. Then it skips everything until it gets tho the last piece of text at the end of the name and captures everything after the last space. New breakdown on RegEx101: https://regex101.com/r/zW8sW1/2
So you want the ID_LName, which can be gotten from the folder name, there's really no need to even use your CSV file at this point I don't think. We build the new name of the files based off the automatic $Matches variable using the second capture group and the first capture group and putting an underscore between them. Then we just iterate through the files with a For loop basing it off how many files were found. So we start with the first file in the array $Files (record 0), add that to the $NewName with an underscore, and use that to rename the file.