I'm working in PowerShell with a CSV with 13 incremented columns "access1, access2,..", and I am trying to step through and export "crednum" from each row where the access column is not blank, with a new CSV for each column. I've tried (-ne $null) and (-ne " ") both will just give me the entire content of the crednum column and I have no idea what I am doing wrong.
$a = 1
do {
$column = ("access" + $a)
$outpath = "C:\Test\"
$access = Import-Csv 'C:\import.csv' |
where {$_.$column -ne " "} |
select crednum
$a
$access | Export-Csv ($outpath + $column + ".csv") -NoTypeInformation
$a++
} while ($a -le 13)
An empty string doesn't equal $null and it doesn't equal a single space " ". It equals "", '', or [String]::Empty. However, it's usually best to use the [String]::IsEmptyOrWhitespace() function for this sort of thing because it covers more corner cases such as when the column is null, or a single space, or an empty string.
You're also importing the entire CSV on each loop. That's a waste of effort because the entire file has to be processed every iteration.
Try something like this:
$Columns = 1..12 | ForEach-Object { "access$_" }
$Access = Import-Csv 'C:\import.csv'
foreach ($Column in $Columns) {
$Access |
Where-Object { -not [String]::IsNullOrWhiteSpace($_.$Column) } |
Select-Object -Property crednum |
Export-Csv -LiteralPath ($outpath + $Column + ".csv") -NoTypeInformation
}
Related
I have been asked to create a script to put comma (,) after every word in csv cell using powershell. However the actual csv has not been shared with me due to some security reason.
I have created a test csv and tried to put comma (,) after every word (Screen shot 1). I am able to create the script and it is working 100% fine (Screen shot 2). However the actual file size is approx 250 MB. I am assuming if that file has 100s of columns, do I have to mention each column name in the script as I mentioned in the foreach loop. Is there any easy way to accomplish this task without mentioning each column name. Any help would be appreciated. Below is my script that is working fine:
$csv = Import-Csv -Path C:\temp\test.csv
$outcsv = Read-Host "Enter the name of the output csv"
$outfile = "$env:USERPROFILE\Downloads" + "" + $outcsv + ".csv"
$data = #()
foreach ($item in $csv) {
$col1 = $item.Name + ","
$col2 = $item.Location + ","
$col3 = $item.Company + ","
$col4 = $item.Profile + ","
$col5 = $item.Shift + ","
$Properties = [ordered]#{
'Name' = $col1
'Location' = $col2
'Company' = $col3
'Profile' = $col4
'Shift' = $col5
}
$data += New-Object -TypeName PSObject -Property $Properties
}
$data | Export-Csv -Path $outfile -NoTypeInformation
Screen shot 1:
Screen shot 2 with comma (,):
This is an easier way to go about it that could handle the logic dynamically by accessing the object's PSObject.Properties.
$outcsv = Read-Host "Enter the name of the output csv"
if(-not [System.IO.Path]::GetExtension($outcsv)) {
$outcsv = $outcsv + '.csv'
}
$outfile = Join-Path "$env:USERPROFILE\Downloads" -ChildPath $outcsv
Import-Csv -Path C:\temp\test.csv | ForEach-Object { $firstObject = $true } {
if($firstObject) {
# get the property names of the first object
$properties = $_.PSObject.Properties.Name
$firstObject = $false
}
# enumerate each property of the object
foreach($property in $properties) {
# update the value
$_.$property = $_.$property + ','
}
# output the updated object
$_
} | Export-Csv $outfile
I based my own solution on How to separate one row into multiple rows based on a single cell containing multiple values in the same row in a CSV file using PowerShell
Trying to iterate over one row, retrieve the zips, split it by ' ' and create a new PSCustomObject for each using the zipcodes and place the zipcode in the column with new rows in the same csv.
Sample (yes, hood = no value)
City,State,hood,zip,lat,lng
Carmel,IN,,"46290 46032 46033 46280 46082",39.9783711,-86.1180435
My non working solution is a modification of the solution in the link sited
$x = Get-Content 'D:\Carmel-IN.csv'
$y = $x | ConvertFrom-Csv -Delimiter ' '
$y | Foreach {
$current = $_
$current.zip -split ' ' | foreach {
[PSCUstomObject]#{
zip = $_
}
}
} | ConvertTo-Csv -NoTypeInformation -Delimiter ' ' | % {
$_ -replace '"',''
}
The output is the same on ISE and console ... the one word zip and nothing is written to file.
I am very new to PS, using 5.1. I have a feeling the correction is very simple but I cant see it. Looking for a result file as ...
City,State,hood,zip,lat,lng
Carmel,IN,,"46290 46032 46033 46280 46082",39.9783711,-86.1180435
,,,46290,,
,,,46032,,
,,,46033,,
,,,46280,,
,,,46082,,
UPDATE:
I managed to rework the script to get result I can use
$x = Get-Content 'D:\temp\Carmel-IN.csv'
$exportlocation = 'D:\temp\Carmel-IN.csv'
$y = $x | ConvertFrom-Csv -Delimiter ','
$y | Foreach {
$current = $_
$current.zip -split ' ' | foreach {
[PSCUstomObject]#{
City = $null
State = $null
hood = $null
zip = $_
lat = $null
lng = $null
}
}
} | Export-CSV -Append -NoTypeInformation -Delimiter ',' -Force $exportlocation | % { $_ -replace '"',''}
and the result is
City,State,hood,zip,lat,lng
Carmel,IN,,"46290 46032 46033 46280 46082",39.9783711,-86.1180435
,,,"46290",,
,,,"46032",,
,,,"46033",,
,,,"46280",,
,,,"46082",,
I am going to care of the extra "'s in another script
The | % { $_ -replace '"',''} doesn't seem to be removing them
Thanks for all suggestions
Use Select-Object:
$x = Get-Content 'D:\Carmel-IN.csv'
$y = $x | ConvertFrom-Csv -Delimiter ' '
foreach($row in $y){
foreach($zip in -split $row.zip){
$row |select *,#{Name='zip';Expression={$zip}} -ExcludeProperty zip
}
}
Alternatively, if you want to preserve the order of columns, set up the property list of properties to select prior to the loop:
$columns = $y[0].psobject.Properties.Name |ForEach-Object {
if($_ -eq 'zip'){
$_ = #{Name = 'zip';Expression = { -split $zip }}
}
$_
}
foreach($row in $y){
foreach($zip in -split $row.Zip){
$row |select $columns
}
}
I want to process a csv file in powershell, but I don't know what the column headings in the CSV file will be when it is processed.
For example:
$path = "d:\scratch\export.csv"
$csv = Import-csv -path $path
foreach($line in $csv)
{
foreach ($head in $line | get-member | where-object {$_.MemberType -eq "NoteProperty"} | select Definition)
{
#pseudocode...
doSomething($head.columnName, $head.value)
}
}
How do I loop through the line in the csv file, getting the name of the column and the value? Or is there another way I should be doing this (like not using Import-csv)?
Import-Csv $path | Foreach-Object {
foreach ($property in $_.PSObject.Properties)
{
doSomething $property.Name, $property.Value
}
}
A slightly other way of iterating through each column of each line of a CSV-file would be
$path = "d:\scratch\export.csv"
$csv = Import-Csv -path $path
foreach($line in $csv)
{
$properties = $line | Get-Member -MemberType Properties
for($i=0; $i -lt $properties.Count;$i++)
{
$column = $properties[$i]
$columnvalue = $line | Select -ExpandProperty $column.Name
# doSomething $column.Name $columnvalue
# doSomething $i $columnvalue
}
}
so you have the choice: you can use either $column.Name to get the name of the column, or $i to get the number of the column
$header3 = #("Field_1","Field_2","Field_3","Field_4","Field_5")
Import-Csv $fileName -Header $header3 -Delimiter "`t" | select -skip 3 | Foreach-Object {
$record = $indexName
foreach ($property in $_.PSObject.Properties){
#doSomething $property.Name, $property.Value
if($property.Name -like '*TextWrittenAsNumber*'){
$record = $record + "," + '"' + $property.Value + '"'
}
else{
$record = $record + "," + $property.Value
}
}
$array.add($record) | out-null
#write-host $record
}
I'm looking for a way (if possible) to find any hits within any column that contain a ";" semicolon character and return the column/field name.
I'm basically loading in a DAT delimited text file (or csv). The headers will be different each time, but I'm basically trying to figure out if I will be expecting any of the columns to contain multi-delimited values within the column such as email CC or BCC.
I'm using a form with a text box to input the DAT/CSV.
$form.Topmost = $True
$form.Add_Shown({$textBox.Select()})
$result = $form.ShowDialog()
if ($result -eq [System.Windows.Forms.DialogResult]::OK)
{
$x = $textBox.Text
$x
}
Here is my code for output file:
Get-Content $x |
foreach {$_ -replace "รพ", '"'} |
ConvertFrom-Csv -Delimiter "" |
Out-GridView
I have been able to search a hit on the entire CSV by using:
$FileContent = Get-Content $x
$Matches = Select-String -InputObject $FileContent -Pattern ';' -AllMatches
$Matches.Matches.Count
The above part does giving me the total number of ";" hits. But I'd rather see which columns hit, I don't really need a total count, just the header name or column number.
I'm using powershell ISE v5.
I would use a Select-String to to find the initial hits which would be quicker than looping through each column and row. You can the loop through the results by converting each found row into a CSV and then an object. All you need to do then is loop though each property and output the results. Something like this:
$file = 'your_file.csv'
$head = Get-Content $file -TotalCount 1
$re = ';'
Select-String $file -Pattern $re | % {
$line = $_
$item = $head + "`n" + $_.line | ConvertFrom-Csv
$item | gm -MemberType NoteProperty | select -ExpandProperty Name | % {
if($item."$_" -match $re) {
New-Object psobject -property #{
Column = $_
Row = $line.Linenumber
Value = $item."$_"
}
}
}
}
I'm working with a big text file, I mean more than 100 MB big, and I need to loop through a specific number of lines, a kind of subset so I'm trying with this,
$info = Get-Content -Path $TextFile | Select-Object -Index $from,$to
foreach ($line in $info)
{
,,,
But it does not work. It is like if it only gets the first line in the subset.
I don't find documentation about the Index attribute, so is this possible or should I try using a different approach considering the file size?
PS> help select -param index
-Index <Int32[]>
Selects objects from an array based on their index values. Enter the indexes in a comma-separated list.
Indexes in an array begin with 0, where 0 represents the first value and (n-1) represents the last value.
Required? false
Position? named
Default value None
Accept pipeline input? false
Accept wildcard characters? false
Based on the above, '8,13' will get you just two lines. One thing you can do is pass an array of numbers, you can use the range operator:
Get-Content -Path $TextFile | Select-Object -Index (8..13) | Foreach-Object {...}
Are the rows of fixed length? If they are, you can seek to desired position by simply calculating offset*row length and using something like .Net FileStream.Seek(). If they are not, all you can do is to read file row by row.
To extract lines m, n, try something like
# Open text file
$reader = [IO.File]::OpenText($myFile)
$i=0
# Read lines until there are no lines left. Count the lines too
while( ($l = $reader.ReadLine()) -ne $null) {
# If current line is within extract range, print it
if($i -ge $m -and $i -le $n) {
$("Row {0}: {1}" -f $i, $l)
}
$i++
if($i -gt $n) { break } # Stop processing the file when row $n is reached.
}
# Close the text file reader
$reader.Close()
$reader.Dispose()
The below is working for me.
It extract all the content between 2 lines.
$name = "MDSinfo"
$MDSinfo = "$PSScriptRoot\$name.txt" #create text file
$MDSinfo = gc $MDSinfo
$from = ($MDSinfo | Select-String -pattern "sh feature" | Select-Object LineNumber).LineNumber
$to = ($MDSinfo | Select-String -pattern "sh flogi database " | Select-Object LineNumber).LineNumber
$i = 0
$array = #()
foreach ($line in $MDSinfo)
{
foreach-object { $i++ }
if (($i -gt $from) -and ($i -lt $to))
{
$array += $line
}
}
$array
The Get-Content cmdlet has a readcount and totalcount parameters. I would play around with those and try to set it up so that the lines your interested in get assigned to an object, then use that object for your loops.
Try this code:
Select-String $FilePath -pattern "FromHere" | Out-Null
$FromHereStartingLine = Select-String $FilePath -pattern "FromHere" | Select-Object LineNumber
$UptoHereStartingLine = Select-String $FilePath -pattern "UptoHere" | Select-Object LineNumber
for($i=$FromHereStartingLine.LineNumber; $i -lt $UptoHereStartingLine.LineNumber; $i+=1)
{
$HoldInVariable += Get-Content -Path $FilePath | Foreach-Object { ($_ -replace "`r*`n*","") } | Select-Object -Index $i
}
Write-Host "HoldInVariable : " $HoldInVariable