File data not displaying in table form using Powershell Script - powershell

I have been grabbing the file data from student_data.dat and trying to display it into tabular form.
The first 3 lines of the file are written like this:
Jamie Zawinski,78.8,81.0,77.3,80.0,80.0,77.0
Adam Douglas,86.2,69.0,77.8,81.0,87.5,88.0
Wallace Steven,66.2,68.0,91.3,78.6,80.3,86.4
I wish to set it up into a table with headers of Student Name, Assignment-1, Assignment-2, etc. I will later be manipulating the data to calculate the class averages for each assignment and the students overall average in the course so far. Each method of setting up the table results in the file being displayed as:
#{Name=Jamie Zawinski; Assignment-1=78.8; Assignment-2=81.0; Assignment-3=77.3; Assignment-4=80.0;
Midterm_Exam=80.0; Final_Exam=77.0} #{Name=Adam Douglas; Assignment-1=86.2; Assignment-2=69.0;
Assignment-3=77.8; Assignment-4=81.0; Midterm_Exam=87.5; Final_Exam=88.0} #{Name=Wallace Steven;
Assignment-1=66.2; Assignment-2=68.0; Assignment-3=91.3; Assignment-4=78.6; Midterm_Exam=80.3;
Final_Exam=86.4}
My coding has looked like this:
$file = Import-Csv C:\**real path**\student_data.dat -Delimiter ',' -Header 'Name', 'Assignment-1','Assignment-2','Assignment-3','Assignment-4','Midterm_Exam','Final_Exam'
Write-Host $file
I tried adding:
foreach ($line in $file){
$data += [pscustomobject]#{
Name = $line.name
Assignment-1 = $line.Assignment-1
Assignment-2 = $line.Assignment-2
}
}
and writing instead:
$filedata = Get-Content ./student_data.dat
$newline = $filedata.Split("`n")
$newline.count
foreach ($l in $newline){
$Names = $l.Split(",")[0].Trim()
$Assignment-1 = $l.Split(",").Trim()
[pscustomObject]#{
Names = $Names;
Assignment-1 = $Assignment-1
}
}
but errors occurred.

First of all, note that you get a more readable output if you use Write-Output instead of Write-Host, which tries to push the entire input into a single string. (Or just remove it altogether, $file is equivalent to Write-Output $file)
As Matthias R. Jessen commented, you are probably looking for the Format-Table command:
$path = "C:\**real path**\student_data.dat"
$header = 'Name', 'Assignment-1','Assignment-2','Assignment-3','Assignment-4','Midterm_Exam','Final_Exam'
$data = Import-Csv $path -Delimiter ',' -Header $header
$data | Format-Table
Note this just "pretty-prints" the data for display in the console. The data is not changed and you should not use this for any kind of file output. And it's not really necessary for working with the data. Once you're done manipulating your data, you can just export it back as CSV:
$outpath = "C:\**real path**\student_data_result.csv"
$data | Export-Csv $outpath -Delimiter "," -NoTypeInformation

Related

PowerShell string replacement with csv as search and replace pattern for a separate file

I'm trying to find a way to reliably replace all occurrences of a string found in a file with data from a column in a CSV using one column as the search pattern with data from the same row on the next column for the replace pattern. The new data is then written to a new file as to keep the original intact. The purpose of this is to simplify exchanging IDs between environments that are hardcoded into the Master pages of a SharePoint site collection. Here's what I have so far.
$file = "C:\Users\jeffery\Documents\ids.csv"
$csv = Import-Csv -Path $file -Delimiter `,
$prd2016 = $csv.'2016 PRD ID'
$stg2016 = $csv.'2016 STG ID'
$prd2010 = $csv.'2010 PRD ID'
$srcFile = "C:\Users\jeffery\Downloads\v5.master"
$dstFile = "C:\Users\jeffery\Downloads\v6.master"
Set-Variable 2010,2016
$content = Get-Content -Path $srcFile
For($i=0; $i -lt $prd2016.Count; $i++){
Clear-Variable 2010
Clear-Variable 2016
$2010 = $prd2010[$i]
$2016 = $prd2016[$i]
$content.replace("$2016", "$2010") | Set-Content -Path $dstFile -Force
}
I've also tried nested loops and using foreach loops to no avail as of yet. Any help will be greatly appreciated. Also, here's some sample data to assist with any answers.
CSV Data:
Navigation,a5a0c64c-17b1-4cba-a8ff-a6a61d8466f3,a66d1d48-ab5e-4aed-9eb9-e8763b88ff2a,2d3cd026-7e2a-4241-8500-abd9a83a0803
Source file data:
<WebPartPages:DataFormWebPart runat="server" IsIncluded="True" AsyncRefresh="false" NoDefaultStyle="TRUE" ViewFlag="8" Title="Navigation" PageType="PAGE_NORMALVIEW" __markuptype="vsattributemarkup" __WebPartId="{9CDA54AA-5C9F-4E62-A0D6-BE149C8B27F0}" partorder="2" id="g_9cda54aa_5c9f_4e62_a0d6_be149c8b27f0" listname="{a5a0c64c-17b1-4cba-a8ff-a6a61d8466f3}" pagesize="1" chrometype="None" __AllowXSLTEditing="true" WebPart="true" Height="" Width="">
<DataSources><SharePoint:SPDataSource runat="server" DataSourceMode="List" UseInternalName="true" UseServerDataFormat="true" selectcommand="<View><Query><Where><Eq><FieldRef Name="Title"/><Value Type="Text">Top Nav</Value></Eq></Where></Query></View>" id="dataformwebpart8"><SelectParameters><WebPartPages:DataFormParameter Name="ListID" ParameterKey="ListID" PropertyName="ParameterValues" DefaultValue="{a5a0c64c-17b1-4cba-a8ff-a6a61d8466f3}"/><asp:Parameter Name="MaximumRows" DefaultValue="1"/></SelectParameters><DeleteParameters><WebPartPages:DataFormParameter Name="ListID" ParameterKey="ListID" PropertyName="ParameterValues" DefaultValue="{a5a0c64c-17b1-4cba-a8ff-a6a61d8466f3}"/></DeleteParameters><UpdateParameters><WebPartPages:DataFormParameter Name="ListID" ParameterKey="ListID" PropertyName="ParameterValues" DefaultValue="{a5a0c64c-17b1-4cba-a8ff-a6a61d8466f3}"/></UpdateParameters><InsertParameters><WebPartPages:DataFormParameter Name="ListID" ParameterKey="ListID" PropertyName="ParameterValues" DefaultValue="{a5a0c64c-17b1-4cba-a8ff-a6a61d8466f3}"/></InsertParameters></SharePoint:SPDataSource></DataSources>
<datafields>#Title,Title;#Navigation,Navigation;#ID,ID;#ContentType,Content Type;#Modified,Modified;#Created,Created;#Author,Created By;#Editor,Modified By;#_UIVersionString,Version;#Attachments,Attachments;#File_x0020_Type,File Type;#FileLeafRef,Name (for use in forms);#FileDirRef,Path;#FSObjType,Item Type;#_HasCopyDestinations,Has Copy Destinations;#_CopySource,Copy Source;#ContentTypeId,Content Type ID;#_ModerationStatus,Approval Status;#_UIVersion,UI Version;#Created_x0020_Date,Created;#FileRef,URL Path;#ItemChildCount,Item Child Count;#FolderChildCount,Folder Child Count;#AppAuthor,App Created By;#AppEditor,App Modified By;</datafields>
<XSL><xsl:stylesheet xmlns:x="http://www.w3.org/2001/XMLSchema" xmlns:d="http://schemas.microsoft.com/sharepoint/dsp" version="1.0" exclude-result-prefixes="xsl msxsl ddwrt" xmlns:ddwrt="http://schemas.microsoft.com/WebParts/v2/DataView/runtime" xmlns:asp="http://schemas.microsoft.com/ASPNET/20" xmlns:__designer="http://schemas.microsoft.com/WebParts/v2/DataView/designer" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:msxsl="urn:schemas-microsoft-com:xslt" xmlns:SharePoint="Microsoft.SharePoint.WebControls" xmlns:ddwrt2="urn:frontpage:internal">
Thank you in advance for any and all help with this query.
You're jumping through some serious hoops to avoid using the objects Import-Csv gives you and foreach.
You're also overwriting the destination file every time you loop, which is unnecessary.
$srcFile = "C:\Users\jeffery.grantham\Downloads\v5.master"
$dstFile = "C:\Users\jeffery.grantham\Downloads\v6.master"
$file = "C:\Users\jeffery\Documents\ids.csv"
$csv = Import-Csv -Path $file
$content = Get-Content -Path $srcFile -Raw # Read the file into a single string.
foreach ($row in $csv) {
$content = $content.Replace($row.'2016 PRD ID', $row.'2010 PRD ID')
}
$content | Set-Content -Path $dstFile -Force
I found my own answer finally. The problem was that the string was getting replaced in realtime but it was not updating the variable prior to trying to write the result to the $dstFile.
So my final script looks like the following:
$file = "C:\Users\jeffery.grantham\Documents\peer-ids.csv"
$csv = Import-Csv -Path $file -Delimiter `,
$prd2016 = $csv.'2016 PRD ID'
$stg2016 = $csv.'2016 STG ID'
$prd2010 = $csv.'2010 PRD ID'
$srcFile = "C:\Users\jeffery.grantham\Downloads\v5.master"
$dstFile = "C:\Users\jeffery.grantham\Downloads\v6.master"
Set-Variable 2010,2016
$content = Get-Content -Path $srcFile
For($i=0; $i -lt $prd2016.Count; $i++){
Clear-Variable 2010
Clear-Variable 2016
$2010 = [string]$prd2010[$i]
$2016 = [string]$prd2016[$i]
$content = $content.replace("$2016", "$2010")
$content | Set-Content -Path $dstFile -Force
}
I hate when I find my own answer less than an hour after posting a question, but hopefully, finding this will help someone else in the future attempt to do the same thing or something similar.

Changing multiple lines in a text file based on a psobject

I'm working on a script which will add some additional informations to a txt file. These informations are stored in a CSV file which looks like this (the data will differs each time the script will launch):
Number;A;B;ValueOfB
FP01340/05/20;0;1;GTU_01,GTU_03
FP01342/05/20;1;0;GTU01
The txt file looks like this (data inside will of course differ each time):
1|1|FP01340/05/20|2020-05-02|2020-05-02|2020-05-02|166,91|203,23|36,32|nothing interesting 18|33333|63-111 somewhere|||||
2|zwol|9,00|9,00|0,00
2|23|157,91|194,23|36,32
1|1|FP01341/05/20|2020-05-02|2020-05-02|2020-05-02|12,19|14,99|2,80|Some info |2222222|blabla|11-111 something||||
2|23|12,19|14,99|2,80
1|1|FP01342/05/20|2020-05-02|2020-05-02|2020-05-02|525,36|589,64|64,28|bla|222222|blba 36||62030|something||
2|5|213,93|224,63|10,70
2|8|120,34|129,97|9,63
2|23|191,09|235,04|43,95
What I need to do is to find a line which contains 'Number' and then add value 'A' and 'B' from a CSV in a form: |0|1 and then on the first line below, at the end, add 'ValueofB' in a form |AAA_01,AAA_03
So the first two lines should look like this at the end:
1|1|FP01340/05/20|2020-05-02|2020-05-02|2020-05-02|166,91|203,23|36,32|nothing interesting 18|33333|63-111 somewhere||||||0|1
2|zwol|9,00|9,00|0,00|AAA_01,AAA_03
2|23|157,91|194,23|36,32
Rest of lines should not be touched.
I made a script which uses select-string method with context to find what I need to - put that into an object and then add to previously found strings what I need to and put that in to an another object.
My script is as follws:
$csvFile = Import-Csv -Path Somepath\file.csv -Delimiter ";"
$file = "Somepath2\SomeName.txt"
$LinesToChange = #()
$script:LinesToChange = $LinesToChange
$LinesOriginal = #()
$script:LinesOriginal = $LinesOriginal
foreach ($line in $csvFile) {
Select-String -Path $file -Pattern "$($Line.number)" -Encoding default -Context 0, 1 | ForEach-Object {
$1 = $_.Line
$2 = $_.Context.PostContext
}
$ListOrg = [pscustomobject]#{
Line_org = $1
Line_GTU_org = $2
}
$LinesOriginal = $LinesOriginal + $ListOrg
$lineNew = $ListOrg.Line_org | foreach { $_ + "|$($line.A)|$($line.B)" }
$GTUNew = $ListOrg.Line_GTU_org | foreach { $_ + "|$($line.ValueofB)" }
$ListNew = [pscustomobject]#{
Line_new = $lineNew
Line_GTU_new = $GTUNew
Line_org = $ListOrg.Line_org
Line_GTU_org = $ListOrg.Line_GTU_org
}
$LinesToChange = $LinesToChange + $ListNew
}
The output is an object $LinesToChange which have original lines and lines after the change. The issue is I have no idea how to use that to change the txt file. I tried few methods and ended up with file which contains updated lines but all others are doubbled (I tried foreach) or PS is using whole RAM and couldn't finish the job :)
My latest idea is to use something like that:
(Get-Content -Path $file) | ForEach-Object {
$line = $_
$LinesToChange.GetEnumerator() | ForEach-Object {
if ($line -match "$($LinesToChange.Line_org)") {
$line = $line -replace "$($LinesToChange.Line_org)", "$($LinesToChange.Line_new)"
}
if ($line -match "$($LinesToChange.Line_GTU_org)") {
$line = $line -replace "$($LinesToChange.Line_GTU_org)", "$($LinesToChange.Line_GTU_new)"
}
}
} | Set-Content -Path Somehere\newfile.txt
It seemed promising at first, but the variable $line contains all lines and as such it can't find the match.
Also I need to be sure that the second line will be directly below the first one (it is unlikely but it can be a case that there will be two or more lines with the same data while the "number" from CSV file is unique) so preferably while changing the txt file it would be needed to find a match for a two-liner; in short:
find this two lines:
1|1|FP01340/05/20|2020-05-02|2020-05-02|2020-05-02|166,91|203,23|36,32|nothing interesting 18|33333|63-111 somewhere|||||
2|zwol|9,00|9,00|0,00
change them to:
1|1|FP01340/05/20|2020-05-02|2020-05-02|2020-05-02|166,91|203,23|36,32|nothing interesting 18|33333|63-111 somewhere||||||0|1
2|zwol|9,00|9,00|0,00|AAA_01,AAA_03
Do that for all lines in a $LinesToChange
Any help will be much appreciated!
Greetings!
Some strange text file you have there, but anyway, this should do it:
# read in the text file as string array
$txt = Get-Content -Path '<PathToTheTextFile>'
$csv = Import-Csv -Path '<PathToTheCSVFile>' -Delimiter ';'
# loop through the items (rows) in the CSV and find matching lines in the text array
foreach ($item in $csv) {
$match = $txt | Select-String -Pattern ('|{0}|' -f $item.Number) -SimpleMatch
if ($match) {
# update the matching text line (array indices count from 0, so we do -1)
$txt[$match.LineNumber -1] += ('|{0}|{1}' -f $item.A, $item.B)
# update the line following
$txt[$match.LineNumber] += ('|{0}' -f $item.ValueOfB)
}
}
# show updated text on screen
$txt
# save updated text to file
$txt | Set-Content -Path 'Somehere\newfile.txt'

How to seperate CSV values within a CSV into new rows in PowerShell

I'm receiving an automated report from a system that cannot be modified as a CSV. I am using PowerShell to split the CSV into multiple files and parse out the specific data needed. The CSV contains columns that may contain no data, 1 value, or multiple values that are comma separated within the CSV file itself.
Example(UPDATED FOR CLARITY):
"Group","Members"
"Event","362403"
"Risk","324542, 340668, 292196"
"Approval","AA-334454, 344366, 323570, 322827, 360225, 358850, 345935"
"ITS","345935, 358850"
"Services",""
I want the data to have one entry per line like this (UPDATED FOR CLARITY):
"Group","Members"
"Event","362403"
"Risk","324542"
"Risk","340668"
"Risk","292196"
#etc.
I've tried splitting the data and I just get an unknown number of columns at the end.
I tried a foreach loop, but can't seem to get it right (pseudocode below):
Import-CSV $Groups
ForEach ($line in $Groups){
If($_.'Members'.count -gt 1, add-content "$_.Group,$_.Members[2]",)}
I appreciate any help you can provide. I've searched all the stackexchange posts and used Google but haven't been able to find something that addresses this exact issue.
Import-Csv .\input.csv | ForEach-Object {
ForEach ($Member in ($_.Members -Split ',')) {
[PSCustomObject]#{Group = $_.Group; Member = $Member.Trim()}
}
} | Export-Csv .\output.csv -NoTypeInformation
# Get the raw text contents
$CsvContents = Get-Content "\path\to\file.csv"
# Convert it to a table object
$CsvData = ConvertFrom-CSV -InputObject $CsvContents
# Iterate through the records in the table
ForEach ($Record in $CsvData) {
# Create array from the members values at commas & trim whitespace
$Record.Members -Split "," | % {
$MemberCount = $_.Trim()
# Check if the count is greater than 1
if($MemberCount -gt 1) {
# Create our output string
$OutputString = "$($Record.Group), $MemberCount"
# Write our output string to a file
Add-Content -Path "\path\to\output.txt" -Value $OutputString
}
}
}
This should work, you had the right idea but I think you may have been encountering some syntax issues. Let me know if you have questions :)
Revised the code as per your updated question,
$List = Import-Csv "\path\to\input.csv"
foreach ($row in $List) {
$Group = $row.Group
$Members = $row.Members -split ","
# Process for each value in Members
foreach ($MemberValue in $Members) {
# PS v3 and above
$Group + "," + $MemberValue | Export-Csv "\path\to\output.csv" -NoTypeInformation -Append
# PS v2
# $Group + "," + $MemberValue | Out-File "\path\to\output.csv" -Append
}
}

Storing output in CSV file

I have wriiten the following SQL script to execute using PowerShell.
cls
foreach ($svr in get-content "demo.txt")
{
$con = "server=MC1-PQ10X.RF.LILLY.COM\SQL01;database=mylilly_WSS_Content_INCTSK0014840;Integrated Security=sspi"
$cmd = "SELECT
Docs.DirName + '/' + Docs.LeafName AS 'Item Name',
DocVersions.UIVersion,
(DocVersions.UIVersion/512) as Version_Label, DocVersions.Level, DocVersions.TimeCreated
FROM DocVersions FULL OUTER JOIN Docs ON Docs.Id = DocVersions.Id
-- INNER JOIN Webs On Docs.WebId = Webs.Id
--INNER JOIN Sites ON Webs.SiteId = SItes.Id
WHERE (DirName LIKE '%globalcontentrepository%')
AND (IsCurrentVersion = '0')
AND (DocVersions.Id IN ('$svr'))"
$da = new-object System.Data.SqlClient.SqlDataAdapter ($cmd, $con)
$dt = new-object System.Data.DataTable
$da.fill($dt) |out-null
$dt | Export-Csv music.csv -Encoding ascii -NoTypeInformation
}
The problem I'm facing with the above code is regarding the output. For every $svr this code is creating a new CSV file. The input file is containing around 1000 inputs. My requirement is that all the output should get stored in the same file rather than creating new file.
There are several ways to achieve your goal. One is to append to the output file as #PeterMmm suggested in the comments to your question:
foreach ($svr in Get-Content "demo.txt") {
...
$dt | Export-Csv music.csv -Encoding ascii -NoTypeInformation -Append
}
Note, however, that Export-Csv doesn't support appending prior to PowerShell v3. Also, appending usually has a negative impact on performance, because you have to repeatedly open the output file. It's better to put the export cmdlet outside the loop and write the output file in one go. foreach loops don't play nicely with pipelines, though, so you'd have to assign the loop output to a variable first:
$music = foreach ($svr in Get-Content "demo.txt") {
...
$dt
}
$music | Export-Csv music.csv -Encoding ascii -NoTypeInformation
or run it in a subexpression:
(foreach ($svr in Get-Content "demo.txt") {
...
$dt
}) | Export-Csv music.csv -Encoding ascii -NoTypeInformation
Personally I'd prefer a ForEach-Object loop over a foreach loop, because the former does work well with pipelines:
Get-Content "demo.txt" | ForEach-Object {
...
$dt
} | Export-Csv music.csv -Encoding ascii -NoTypeInformation
Note that you must replace every occurrence of your loop variable $svr with the "current object" variable $_ for this to work, though.

Output to Pipeline from within IF ELSE statement

UPDATE: I just found out that the client I am running this on is PS V1. I can't use splatting.
I have a script for processing csv files. I don't know ahead of time if the file will have a header or not so I'm prompting the user to either input a header or use the one in the file:
$header = Read-Host 'Input your own header?'
What I want to do is be able to check whether the header variable has been set to decide if I want to use that flag when executing the Import-CSV commandlet.
I have tried a number of things along the lines of:
IF($Header){Import-Csv $File -delimiter $delimiter -Header $header }
ELSE
{Import-Csv $File -delimiter $delimiter} |
Or
IF($Header){Import-Csv $File -delimiter $delimiter -Header $header | %{$_} }
ELSE
{Import-Csv $File -delimiter $delimiter | %{$_}}
The first example results in complaints of an empty pipeline. The second results in the first column of the file just being ouptut to the console and then errors when the file is done processing because the rest of the pipeline is empty.
As always any help would be appreciated.
Thanks,
Bill
Eris already suggested splatting, but I wanted to give a more comprehensive example, using your code.
# Declare a hash containing the parameters you will always need
$csvParams = #{
File = $File;
delimiter = $delimiter;
}
# if the header is specified, add a Header to $csvParams
if ($Header) { $csvParams.Header = $header }
# Call Import-Csv, splatting $csvParams
Import-Csv #csvParams
Splatting is an extremely useful technique. Get-Help about_Splatting for more information about it.
The easiest way is just to use the -OutVariable option on Import-Csv
Import-Csv -Path:$File -Delimiter:$Delimiter -OutVariable:CSVContents will save it in $CSVContents
From the Import-CSV Technet page:
This cmdlet supports the common parameters: -Verbose, -Debug, -ErrorAction, -ErrorVariable, -OutBuffer, and -OutVariable. For more information, see about_CommonParameters (http://go.microsoft.com/fwlink/p/?LinkID=113216).
Another alternative is to use an args hash and "splat" it:
$myArgs = #{
Path = "$HOME\temp\foo"
}
Get-Content #myArgs
Update for Version 1 (untested):
( IF($Header) { Import-Csv $File -delimiter $delimiter -Header $header }
ELSE { Import-Csv $File -delimiter $delimiter} ) |
#More pipeline commands here, example:
Format-Table
Horrible disgusting version (untested):
$ImportCommand = "Import-Csv $File -delimiter $delimiter"
If($header -ne $null) { $ImportCommand += " -header $header" }
Invoke-Expression $ImportCommand |
Format-Table
You second approach is the way I would go. I'm not sure why, what you have shown for your second example, would be failing. Here is what I would do:
filter ProcessCsv {
$_ | ... further processing ...
}
$header = Read-Host 'Input your own header?'
if ($header) {
Import-Csv $file -Delimiter $delimiter -Header $header | ProcessCsv
else {
Import-Csv $file -Delimiter $delimiter | ProcessCsv
}