Using a wildcard to replace CSV cell contents with PowerShell - powershell

I'm trying to get PowerShell to manipulate some CSV data.
The column I’m trying to manipulate in my CSV contains stock levels as text, containing mostly "High", "Good", "Medium", "Low" and "Out Of Stock" which I want to convert to a numerical format before importing the CSV to the website at the end of the day.
My code currently is:
#Declare Stock Level Words To Value
$StockLevelWordToValue = #{
"High" = "70"
"Good" = "50"
"Medium" = "30"
"Low" = "10"
"Out Of Stock" = "0"
}
# import the CSV file
$csv = Import-Csv "C:\temp\temp.csv";
# for each row, replace the Stock Level Word field with a value
foreach($row in $csv) {
$row."Stock Level" = $StockLevelWordToValue[$row."Stock Level"];
}
# export the modified CSV
$csv | Export-Csv "C:\temp\temp2.csv" -NoTypeInformation;
The problem I have is in my "Stock Level" column, some of the entries are like "Good: Last update August 2019" or “High: Last update March 2019”
Currently, when running this code, there is no value returned for these entries and the cell is blank.
I was hoping a wildcard would work like:
"Good*" = "50"
But that returns a blank cell too.
How can check a cell "contains" a word and returns a numerical value, no matter what other words are in that cell?

You can use Select-String to check the value of your attribute by patterns like "High*". This will return null if doesn't find any, true if there is a string matching this pattern.
foreach($row in $csv) {
If ($row."Stock Level" | Select-String "High*) {
$row."Stock Level" = $StockLevelWordToValue["High"];
}
}

Related

Powershell - How to split a string based on characters?

I have a list of pdf filenames that need to be parsed and ultimately sent to a sql table, with the parse out pieces each in their own column. How would I split based on a dash '-' and ultimately get it into a table.
What cmdlets would you start with to split on a character? I need to split based on the dash '-'.
Thanks for the help.
Example File Names:
tester-2458-full_contact_snapshot-20200115_1188.pdf
tester-2458-limited_contact_snapshot-20200119_9330.pdf
Desired Results:
There is also a -split operator.
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_split
basic example:
if you have file names in $FilePaths array.
foreach($filepath in $FilePaths)
{
$parts = $filepath -split '-';
[pscustomobject]#{"User" = $parts[0]; "AppID" = $parts[1]; "FileType" = $parts[2]; "FilePath"=$filepath }
}
Use $variable.split('-') which will return a string array with a length equal to however many elements are produced by the split operation.
yet another way is to use regex & named capture groups. [grin]
what it does ...
creates a set of file name strings to work with
when ready to use real data, remove the entire #region/#endregion block and use either (Get-ChildItem).Name or another method that gives you plain strings.
iterates thru the collection of file name strings
uses $Null = to suppress the False/True output of the -match call
does a regex match with named capture groups
uses the $Match automatic variable to plug the captured values into the desired properties of a [PSCustomObject]
sends that PSCO out to the $Results collection
displays that on screen
sends it to a CSV for later use
the code ...
#region >>> fake reading in a list of file names
# in real life, use (Get-ChildItem).Name
$InStuff = #'
tester-2458-full_contact_snapshot-20200115_1188.pdf
tester-2458-limited_contact_snapshot-20200119_9330.pdf
'# -split [System.Environment]::NewLine
#endregion >>> fake reading in a list of file names
$Results = foreach ($IS_Item in $InStuff)
{
$Null = $IS_Item -match '^(?<User>.+)-(?<AppId>.+)-(?<FileType>.+)-(?<Date>.+)\.pdf$'
[PSCustomObject]#{
User = $Matches.User
AppId = $Matches.AppId
FileType = $Matches.FileType
Date = $Matches.Date
FileName = $IS_Item
}
}
# display on screen
$Results
# send to CSV file
$Results |
Export-Csv -LiteralPath "$env:TEMP\JM1_-_FileReport.csv" -NoTypeInformation
output to screen ...
User : tester
AppId : 2458
FileType : full_contact_snapshot
Date : 20200115_1188
FileName : tester-2458-full_contact_snapshot-20200115_1188.pdf
User : tester
AppId : 2458
FileType : limited_contact_snapshot
Date : 20200119_9330
FileName : tester-2458-limited_contact_snapshot-20200119_9330.pdf
content of the C:\Temp\JM1_-_FileReport.csv file ...
"User","AppId","FileType","Date","FileName"
"tester","2458","full_contact_snapshot","20200115_1188","tester-2458-full_contact_snapshot-20200115_1188.pdf"
"tester","2458","limited_contact_snapshot","20200119_9330","tester-2458-limited_contact_snapshot-20200119_9330.pdf"

PowerShell loop through a CSV column and change the values according to criteria from another CSV

I've been attempting to write a PowerShell script that loops through a CSV column and changes the old values to new values that are mapped to search criteria.
CSV1:
Column1 Column2
TEST ANT = Test Ant,
ALPHA = Alpha,
OMEGA = Omega,
CSV2:
001234-Alpha-Bravo,
TTTTTTTTTT,
DATA:TEST-(RANDOM DATA)ANT,
ere393,
OMEGA-333,
Basically
Import a CSV1 that contains search criteria in column1 mapped to approved values in column2.
Import a CSV2 that contains a column of raw data.
Loop through column1 of search criteria.
For each row in CSV1, I want to determine if that string exists in one or more rows (as a whole string, or as a substring) from the raw data column in CSV2.
If the string exists (whole or substring), then replace the cells raw data with the approved value.
If there is no match, then continue to the next row.
Export a CSV with the new approved values and values that were not changed.
I've searched and tried many of the examples on this site, and each one seems to address a part of the problem I'm trying to solve, but not completely. A hash table seems to be the closest to a solution, but the raw data can be dynamic and can't seem to match the key to any part of the string in the raw data.
One example I've tried
$s = Import-Csv C:\SearchCriteriaCSV.csv
$searchCrit = $($s.Name)
$Imported = Import-Csv 'C:\ImportCSV.csv'
$Output = foreach ($i in $Imported) {
foreach ($c in $searchCrit) {
if ($c.Name -like "*$i*") {
$i.Name = $c.Name
}
}
$i
}
$Output
$Output | Export-Csv 'C:\ExportCSV.csv' -NoTypeInformation
Another example tried:
$RawNameFormatName = #{
"ALPHA" = "Alpha";
"OMEGA" = "Omega"
}
$csv = Import-Csv C:\ImportCSV.csv;
foreach($row in $csv) {
$row.Name = $RawNameFormatName[$row.Name];
}
$csv | Export-Csv "C:\ExportCSV.csv" -NoTypeInformation;
Exported CSV:
Alpha
TTTTTTTTTT
Test Ant
ere393
Omega

Read a CSV in powershell with a variable number of columns

I have a CSV that contains a username, and then one or more values for the rest of the record. There are no headers in the file.
joe.user,Accounting-SG,CustomerService-SG,MidwestRegion-SG
frank.user,Accounting-SG,EastRegion-SG
I would like to read the file into a powershell object where the Username property is set to the first column, and the Membership property is set to either the remainder of the row (including the commas) or ideally, an array of strings with each element containing a single membership value.
Unfortunately, the following line only grabs the first membership and ignores the rest of the line.
$memberships = Import-Csv -Path C:\temp\values.csv -Header "username", "membership"
#{username=joe.user; membership=Accounting-SG}
#{username=frank.user; membership=Accounting-SG}
I'm looking for either of these outputs:
#{username=joe.user; membership=Accounting-SG,CustomerService-SG,MidwestRegion-SG}
#{username=frank.user; membership=Accounting-SG,EastRegion-SG}
or
#{username=joe.user; membership=string[]}
#{username=frank.user; membership=string[]}
I've been able to get the first result by enclosing the "rest" of the data in the csv file in quotes, but that doesn't really feel like the best answer:
joe.user,"Accounting-SG,CustomerService-SG,MidwestRegion-SG"
Well, the issue is that what you have isn't really a (proper) CSV. The CSV format doesn't support that notation.
You can "roll your own" and just process the file yourself, something like this:
$memberships = Get-Content -LiteralPath C:\temp\values.csv |
ForEach-Object -Process {
$user,$membership = $_.Split(',')
New-Object -TypeName PSObject -Property #{
username = $user
membership = $membership
}
}
You could do a half and half sort of thing. Using your modification, where the groups are all a single field in quotes, do this:
$memberships = Import-Csv -Path C:\temp\values.csv -Header "username", "membership" |
ForEach-Object -Process {
$_.membership = $_.membership.Split(',')
$_
}
The first example just reads the file line by line, splits on commas, then creates a new object with the properties you want.
The second example uses Import-Csv to create the object initially, then just resets the .membership property (it starts as a string, and we split the string so it's now an array).
The second way only makes sense if whatever is creating the "CSV" can create it that way in the first place. If you have to modify it yourself every time, just skip this and process it as it is.

cleanup improperly formatted csv file

I am downloading a xlsx file from a sharepoint, and then convert it into a csv file. However, since the xlsx file contained empty columns that were not deleted, it exports those to a csv file like follows...
columnOne,columnTwo,columnThree,,,,
valueOne,,,,,,
,valueTwo,,,,,
,,valueThree,,,,
As you can see, Import-Csv cmdlet will fail with that file because of the extra null titles. I want to know how to count the extra commas at the end. The number of columns are always changing, and the name of the columns are also always changing. So we start the count based from the last non-null title number.
Right now, I'm doing the following...
$csvFileEdited = Get-Content $csvFile
$csvFileEdited[0] = $csvFileEdited[0].TrimEnd(',')
$csvFileEdited | Set-Content "$csvFile-temp"
Move-Item "$csvFile-temp" $csvFile -Force
Write-Host "Trim Complete."
This will make the file output like this...
columnOne,columnTwo,columnThree
valueOne,,,,,,
,valueTwo,,,,,
,,valueThree,,,,
The naming is now accepted for Import-Csv, but as you can see there is still extra null values that are not necessary since they are null for every row.
If I did the following code...
$csvFileWithExtraCommas = Get-Content $csvFile
$csvFileWithoutExtraCommas = #()
FOrEach ($line in $csvFileWithExtraCommas)
{
$line = $line.TrimEnd(',')
$csvFileWithoutExtraCommas += $line
{
$csvFileWithoutExtraCommas | Set-Content "$csvFile-temp"
Move-Item "$csvFile-temp" $csvFile -Force
Write-Host "Trim Complete."
Then it would remove a null value that should be null because it belongs to a non-null title-name. Such is the output....
columnOne,columnTwo,columnThree
valueOne
,valueTwo
,,valueThree
Here is the desired output:
columnOne,columnTwo,columnThree
valueOne,,
,valueTwo,
,,valueThree
Can anyone help with this?
Update
I'm using the following code to count the extra null titles...
$csvFileWithCommas = Get-Content $csvFile
[int]$csvFileWithExtraCommasNumber = $csvFileWithCommas[0].Length
$csvFileTitlesWithoutExtraCommas = $csvFileWithCommas[0].TrimEnd(',')
[int]$csvFileWithoutExtraCommasNumber = $csvFileTitlesWithoutExtraCommas.Length
$numOfCommas = $csvFileWithExtraCommasNumber - $csvFileWithoutExtraCommasNumber
The output of value of $numOfCommas is 4. Now the question is how can I use $line.TrimEnd(',') to only do so 4 times??
Ok.... If you really need to do this you can count the trailing commas from the header and use regex to remove as many the from the end of each line. There are other string manipulation approaches but the regex in this case is pretty clean.
Note that what Bluecakes answer shows should suffice. Perhaps there is some other hidden characters that are not being copied in the question or perhaps an encoding issue with your real file.
$file = Get-Content "D:\temp\text.csv"
# Number of trailing commas. Compare the length before and after the trim
$numberofcommas = $file[0].Length - $file[0].TrimEnd(",").Length
# Use regex to remove as many commas from the end of each line and convert to csv object.
$file -replace ",{$numberofcommas}$" | ConvertFrom-Csv
Regex is looking for X commas at the end of of each line where X is $numberofcommas. In our case it would look like ,{4}$
Source file used with above code was generated as such
#"
columnOne,columnTwo,columnThree,,,,
valueOne,,,,,,
,valueTwo,,,,,
,,valueThree,,,,
"# | set-content D:\temp\text.csv
Are you getting an error when trying to Import-csv? The cmdlet is smart enough to ignore columns without a heading without any additional code needed.
I copied your csv file to my H:\ drive:
columnOne,columnTwo,columnThree,,,,
valueOne,,,,,,
,valueTwo,,,,,
,,valueThree,,,,
and then ran $nullcsv = Import-Csv -Path H:\nullcsv.csv and this is what i got
PS> $nullcsv
columnOne columnTwo columnThree
--------- --------- -----------
valueOne
valueTwo
valueThree
The imported csv only contains 3 values as you would expect:
PS> $nullcsv.count
3
The cmdlet is also orrectly accounting for null values in each of the columns:
PS> $nullcsv | Format-List
columnOne : valueOne
columnTwo :
columnThree :
columnOne :
columnTwo : valueTwo
columnThree :
columnOne :
columnTwo :
columnThree : valueThree

PowerShell CSV comparison

I am very, very, very new to Powershell. I was wondering if any one an help me with the following script:
The idea is to have two excel spreadsheets.
1.csv
QCODE
PC1009
PC1009
PC1011
PC1012
2.csv
QCODE
PC1009
PC1009
PC1009
PC1012
I am trying to compare values between the two CSV documents. If the value in cell1 in 1.csv is equal to any cell in 2.csv the script must perform a certain action, once the action is finished it must loop over to cell2 in 1.csv and compare it again with all the values in 2.csv
This is about as far as I have managed yo get:
$CSV=Import-Csv C:\1.csv
$COMP=Import-Csv C:\2.csv
$count=0
$cnt=0
while($count -le $CSV.Count)
{
while($validator -eq $false)
{
if($CSV[$count].QCODE -eq $COMP[$cnt].QCODE)
{
Write-Host "Exiting"
$validator=$true
}
else{
$cnt++
}
}
$count++
}
It's a mess, I apologize. Any help would be greatly appreciated
Here is a solution for you. I have created two CSV files with matching headers. The column names are:
Prop1
Prop2
Prop3
Prop4
Prop5
When these lines are imported into PowerShell, it will automatically create a PSObject for each line. The property names on the PSObject will be the column headers. These two CSV files exist in the folder named c:\test.
NOTE: There is a single, mismatching value between the two files, in the dead middle. This will be our test.
The code looks like this. There are some in-line comments to help guide you. Basically, we're dynamically querying all of the property (column) names, getting the value of each one (the cell values), and comparing them. If they do not match, we throw a warning. Based on the single, mismatching "cell" in this example, the output I get is in a screenshot below. It seems to be working quite well in my testing.
NOTE: Even though it says that line #1 is mismatching, and you might think it's line #2, that's because arrays are zero-based. Therefore, in array terminology, #1 is actually #2, because it starts counting at zero.
# Import both CSV files
$Csv1 = Import-Csv -Path C:\test\csv1.csv;
$Csv2 = Import-Csv -Path C:\test\csv2.csv;
# For each line in CSV1 ...
foreach ($Line1 in $Csv1) {
$LineNumber = $Csv1.IndexOf($Line1);
# Get the same line from CSV2
$Line2 = $Csv2[$LineNumber];
# For each property (column) ...
foreach ($Property in (Get-Member -InputObject $Line1 -MemberType NoteProperty)) {
# Get the property's name
$PropertyName = $Property.Name;
# If the value of the property doesn't match each CSV file ..
if ($Line1.$PropertyName -ne $Line2.$PropertyName) {
# Warn the user
Write-Warning -Message ('Value of property {0} did not match for line # {1}' -f $PropertyName, $LineNumber);
# PERFORM SOME CUSTOM ACTION HERE
};
}
}