This question already has answers here:
Export-CSV exports length but not name
(6 answers)
Closed 4 years ago.
Operating System - Windows 10
Powershell version - 5.1.15063.1088
Ok, I'm really trying hard to think logically what can be wrong with this PowerShell script, but apparently can't get an idea and asking for some help. So here is what I'm trying to do, simple as 1+1
If I understood the tutorial correctly, creating an array in PowerShell is like this:
$someVariable = "PowerShell", "MowerShell", "HowerShell", "ZowerShell"
Then I'm simply trying to write this thing to csv file with comma as delimeter, but firstly give it a try in the console output
$someVariable | ConvertTo-Csv -NoTypeInformation
According to PowerShell 5.1 official documentation
...Specifies a delimiter to separate the property values. The default
is a comma (,).
So no additional writing that I would like to use comma as delimiter is not required. Once the command Write-Host $someVariable is executed, I see this weird output:
"Length" "10" "10" "10" "10"
What is this? Am I suppose to see the values of my variable separated with simple comma? So from the numbers I can guess that scripts calculates the amount of alphabet letters in each word -
P o w e r S h e l l
contains 10 letters.
Is this the suggested way to calculate the amount of letters in the string (in case I get PowerShell task on my next job interview) using ConvertTo-Csv command?
Writing this funky data to the csv file itself leads to more unexpected results:
Now I'm completely lost what those numbers are...
Is this possible to write my strings as STRINGS to the csv file in one line rather then silly numbers?
The desired output is this entry as headers in the csv file:
"PowerShell","MowerShell","HowerShell","ZowerShell"
The output reads "Length", and has a series of 10's. Each of your strings are 10 characters long (the double quotes aren't factored in).
Length can be calculated many ways. I wouldn't say there is one suggested way, only the ways that fit what you're trying to do.
To get the literal text of what you posted (no headers, etc.) in a csv, try:
$someVariable | Out-File foo.csv
Related
I have a source file which is in .txt format. It looks like a semi-colon separated file:
100;200;ThisisastringcolumnA;4;
101;400;Thisisastringc;lumnA;5;
102;600;ThisisastringcolumnB;6;
104;600;Thisisa;;ringcolumnB;6;
However, it is determined by length. So it is a length-delimited file.
Fist column for example is from first value to the third (100), then a semi-colon follows.
Second column starts at 5th position (including), until (including) 7th position. A string column can contain a semi-colon.
Now I want to import this length-delimited txt file with Powershell and export it as a csv file. This file should be really semi-colon separated. The result should look like
100;200;ThisisastringcolumnA;4;
101;400;"Thisisastringc;lumnA";5;
102;600;ThisisastringcolumnB;6;
104;600;"Thisisa;;ringcolumnB";6;
But I have simply no idea how to do it? I googled it, but I did not find that much useful code examples for importing length-delimited txt files with PowerShell.
Unfortunately, I cannot use Python. I am not sure, if this task is generally possible using Powershell? Because when exporting, Powershell also needs to recognize that there are string values containing the separator, so it has to pay attention to the quoting: "Thisisa;;ringcolumnB". I think it would be also ok for me, if the whole column is quoted, so every entry in a string column gets quotes added.
You can use regex to describe a string in which the 3rd "column" contains a ; and then inject the quotation marks with the -replace operator:
$lines = Get-Content path\to\file.txt
#($lines) -replace '(.{3});(.{3});(.{20}(?<=;.{0,19}));(.);', '$1;$2;"$3";$4;'
The expression (.{20}(?<=;.{0,19})) is going to match the 20-char 3rd column value only if it contains at least one semi-colon - so lines with no semicolon in that column will be left alone:
# let's try it out with your test data
$lines = #'
100;200;ThisisastringcolumnA;4;
101;400;Thisisastringc;lumnA;5;
102;600;ThisisastringcolumnB;6;
104;600;Thisisa;;ringcolumnB;6;
'# -split '\r?\n'
#($lines) -replace '(.{3});(.{3});(.{20}(?<=;.{0,19}));(.);', '$1;$2;"$3";$4;'
Which yields the following four strings:
100;200;ThisisastringcolumnA;4;
101;400;"Thisisastringc;lumnA";5;
102;600;ThisisastringcolumnB;6;
104;600;"Thisisa;;ringcolumnB";6;
To write the output back to file, use Set-Content:
#($lines) -replace '(.{3});(.{3});(.{20}(?<=;.{0,19}));(.);', '$1;$2;"$3";$4;' |Set-Content path\to\fixed_output.scsv
Hope all is well! I came across a bit of a tricky issue with a flat file that is exported from Oracle PBCS with some carriage return issues. End users - when inputting data into PBCS - will often press in a specific data field input screen. When the data gets exported representing a specific record with all the data elements representing that data point (intersection) - think like a SQL record - the record element where the user pressed enter causes that record to break at that point - shifting the rest of the data elements in that record to the next line. This is very bad as each record must have the same amount of elements - causing downstream issues in a mapping. In effect one unique record becomes two broken records.
I need a Powershell script that looks at the improper CR LF (Windows system) and reforms each unique record. However, the majority of the records in the flat file are fine so the code will have to be able to discern the "mostly good" from the "very bad" cases.
My flat file is pipe delimited and has a header element. The header element may not need to be considered as I am simply trying to address the fix - a solution could potentially look at the amount of property values for the header record to determine how to format broken records based off a property count using the pipe delimiter - but not sure that is necessary.
I will be honest - there are Jython scripts I tried to no avail - so I felt given that I have employed a couple Powershell scripts for other reasons in the past that I would use this again. I have a basis of a script for a csv file - but this isn't quite working.
$file = Get-Content 'E:\EPM_Cloud\Exports\BUD_PLN\Data\EXPORT_DATA_BUD_PLN.txt'
$file| Foreach-Object {
foreach ($property in $_.PSObject.Properties) {
$property.Value = ($property.Value).replace("`r","").replace("`n","")
}
}
$file|out-file -append 'E:\EPM_Cloud\Exports\BUD_PLN\Data\EXPORT_DATA_BUD_PLN_FINAL.txt'
Here are a few examples of what the before and after case would be if I could get this code to work.
This is supposed to be one record - as you see beginning with "$43K from... the user pressed enter several times. As you see it is pipe delimited - I use the numeric line numbers to show you what I mean since this isn't notepad++. The idea is this should all just be on 1.
Contract TBD|#missing|#missing|#missing|#missing|ORNL to Perform Radio-Chemical (RCA) Measurements|#missing|#missing|#missing|#missing|"$43K from above
$92,903
$14,907
The current $150K to be reprogrammed to XXX, plus another $150K from Fuel Fac for this item to be reprogrammed to RES."|#missing|#missing|#missing|"Summary|X0200_FEEBASED|No_BOC|O-xxxx-B999|xx_xxx_xx_xxx|Plan|Active|FY19|BegBalance"|COMMIT
This is what the output should look like (I have attached screenshots instead). All in 1.
Contract TBD|#missing|#missing|#missing|#missing|ORNL to Perform Radio-Chemical (RCA) Measurements|#missing|#missing|#missing|#missing|"$43K from above $92,903 $14,907 The current $150K to be reprogrammed to XXX, plus another $150K from Fuel Fac for this item to be reprogrammed to RES."|#missing|#missing|#missing|"Summary|X0200_FEEBASED|No_BOC|O-xxxx-B999|xx_xxx_xx_xxx|Plan|Active|FY19|BegBalance"|COMMIT
In other cases the line breaks just once - all defined just by how many times the user presses enter.enter image description here
As you see in the data image - you see how the line splits - this is the point of the powershell. As you see next to that screenshot image - other lines are just fine.
So after checking locally you should be able to just import the file as a csv, then loop through everything and remove CRLF from each property on each record, and output to a new file (or the same, but its safer to output to a new file).
$Records = Import-Csv C:\Path\To\File.csv -Delimiter '|'
$Properties = $Records[0].psobject.properties.name
ForEach($Record in $Records){
ForEach($Property in $Properties){
$Record.$Property = $Record.$Property -replace "[\r\n]"
}
}
$Records | Export-Csv C:\Path\To\NewFile.csv -Delimiter '|' -NoTypeInfo
I have a table that contains message_content. It looks like this:
message_content | WFUS54 ABNT 080344\r\r
| TORLCH\r\r
| TXC245-361-080415-\r
How would I extract only the 2nd line of that output(TORLCH)? I've tried to shorten the output to a certain number of characters but that ultimately doesn't provide what I want. I've also tried removing carriage returns and new lines. I am outputting my results to a CSV I could manipulate with Python, but was wondering if there's a way to do it in the query first.
Based on other examples, it seems like I could use a regular expression to maybe do this? Not sure where to start with learning that though.
you can split the line into an array, then take the second element:
(string_to_array(message_content, e'\r\r'))[2]
Online example: https://rextester.com/MDYLXB40812
I have an extremely large log file (max 1GB) which is appended to throughout the day. There are various strings within this log which I would like to search for (that I can already achieve using Select-String) however I am scanning the whole file on every sweep which is inefficient and a tad unnecessary.
Ideally I want to scan only the last 5 minutes of the log for these strings on each sweep. Unfortunately not every row of the log file contains a timestamp. I initially thought of doing a wildcard select-string for the last 5 mins timestamps combined with the strings of interest will miss some occurrences. My only other idea at the moment is to determine the line numbers of interest, $FromLineNumber (5 mins before current system time) and $ToLineNumber (the very last line number of log file) and then only Select-String between those two line number ranges.
As an example, to search between line 50 and the final line of the log. I am able to return the line number of $FromLineNumber but I'm struggling with grabbing $ToLineNumber for final row of log.
Q. How do I return only the line number of the final row of a log file?
So far I have tried returning this with Get-Content $path -tail -1 (object type linenumber) however this always returns blank values even with various switches and variations. I can only return line numbers via the Select-String cmdlet however I do not have a specific string to use that relates to the final row of the log. Am I misusing this cmdlet per its original design and if so...is there any other alternative to return the last line number?
Continued...Once I have determined the line number range to search between would I isolate those rows using a Get-Content loop between
$FromLineNumber and $ToLineNumber first to filter down to this smaller selection and then pipe this into a Select-String or is there a more efficient way to achieve this? I suspect that looping through thousands of lines would be demanding on resources so I'm keen to know if there is a better way.
Here is the answer to the first question
From https://blogs.technet.microsoft.com/heyscriptingguy/2011/10/09/use-a-powershell-cmdlet-to-count-files-words-and-lines/
If I want to know how many lines are contained in the file, I use the Measure->Object cmdlet with the line switch. This command is shown here:
Get-Content C:\fso\a.txt | Measure-Object –Line
I used the ">" operator to put the content of an array into a CSV file, here is what i got, the first line is empty
Computer IP
-------- --
IMPPRD1 172.22.30.33
IMPPRD2 172.22.30.31
IMPPRD3 172.22.30.32
IMPSR1 172.22.30.12
IMPPRD5 172.22.30.17
I would like it to be a normal CSV, so something like this :
Computer,IP
IMPPRD1,172.22.30.33
IMPPRD2,172.22.30.31
IMPPRD3,172.22.30.32
IMPSR1,172.22.30.12
How could I manage to do this using powershell?
Thanks !
Using > outputs as normal text - including the spaces between columns. Basically the same as Write-Output. You are looking for Export-Csv which outputs in CSV format.