Good day,
with the script below I would like to use the following input txt from my output txt.
Input:
Klaus;Müller;Straße;PLZ;Ort;;;;;DE12345;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE12345678;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE999999;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE7777777;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE7777779;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE777777987;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE7777779765;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE77777797634;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE7777779763465;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE77777797623435435;
Output:
Klaus;Müller;Straße;PLZ;Ort;;;;;DE12345;DE12345678;DE999999;DE7777777;DE7777779;DE777777987;DE7777779765;DE77777797634;DE7777779763465;DE77777797623435435;
The script takes the last value from the following lines and appends them to the first line at the end and adds semicolons:
Import-Csv input.txt -delimiter ";" -Header (1..20)
1..9 | %{$data[0].($_+10) = $data[$_].10}
($data[0] | convertto-csv -delimiter ";" -NoType | select -skip 1) -replace '"' | out-file output.txt
gc test_neu.txt
if i save this into a .ps1 file it doesn´t work. anyone could say me why?
You don't assign Import-Csv to anything. The first line should be: $data = Import-Csv input.txt -delimiter ";" -Header (1..20) Your last line should be gc output.txt. And use the dot notation to location the input.txt file in the current directory. With these fixes, your script works:
$data = Import-Csv .\input.txt -delimiter ";" -Header (1..20)
1..9 | %{$data[0].($_+10) = $data[$_].10}
($data[0] | convertto-csv -delimiter ";" -NoType | select -skip 1) -replace '"' | out-file output.txt
gc output.txt
this seems to do what you want. [grin] it expects that the source lines are all to be combined.
i presume you can handle saving things to a file, so i leave that to you.
what it does ...
fakes reading in a text file
when ready to work with real data, replace the entire #region/#endregion block with a call to Get-Content.
iterates thru the collection by index number
if the line is the 1st, set $NewString to that entire value
else, add the last data item of the line to the existing $NewString value with a trailing ;
the .Where({$_}) filters out any blank items.
display the string
the code ...
#region >>> fake reading in a text file
# in real life, use Get-Content
$InStuff = #'
Klaus;Müller;Straße;PLZ;Ort;;;;;DE12345;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE12345678;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE999999;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE7777777;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE7777779;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE777777987;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE7777779765;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE77777797634;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE7777779763465;
Klaus;Müller;Straße;PLZ;Ort;;;;;DE77777797623435435;
'# -split [System.Environment]::NewLine
#endregion >>> fake reading in a text file
foreach ($Index in 0..$InStuff.GetUpperBound(0))
{
if ($Index -eq 0)
{
$NewString = $InStuff[$Index]
}
else
{
$NewString += $InStuff[$Index].Split(';').Where({$_})[-1] + ';'
}
}
$NewString
output ...
Klaus;Müller;Straße;PLZ;Ort;;;;;DE12345;DE12345678;DE999999;DE7777777;DE7777779;DE777777987;DE7777779765;DE77777797634;DE7777779763465;DE77777797623435435;
Just in case you don't know how many lines there are going to be on the input file:
$fmt='$1$2'
gc .\input.txt | %{$_ -replace '(^.*;)(.*;$)',$fmt;$fmt='$2'} | sc output.txt -NoNewline
gc output.txt
Related
I need to convert strings in a csv file to strings with quotation marks around it.
My csv file looks like this:
Description;AllowHosts;SPNs;Owner
Description1;server1$, server2$, server3$;MSSQLSvc/PD01.dom1.com:1521,MSSQLSvc/PD01.dom1;Owner JDOE
Description2;server4$, server5$, server6$;MSSQLSvc/PD02.dom2.com:1521,MSSQLSvc/PD02.dom2;Owner JDOE
Description3;server7$, server8$, server9$;MSSQLSvc/PD03.dom1.com:1521,MSSQLSvc/PD03.dom1;Owner JDOE
I tried to search for header "AllowHosts" and replace with quotation mark in start and end,
$csv = #(
Import-Csv -Path $New -Delimiter ';' -Encoding UTF8
)
$data = ConvertFrom-Csv $csv
$Data[0].AllowHosts = '"'
$Data | where AllowHosts -Like '*$' | foreach {
$_.AllowHosts = '*$"'
}
$Data | where AllowHosts -Like 'SF' | foreach {
$_.AllowHosts = '"SF*'
}
$Data | ConvertTo-Csv -NoTypeInformation
but it did not work as expected....
I would like to have quotation mark around each string
in column "AllowHosts" (servernames)
in column "SPNs"
I am hoping for a result like this:
Description;AllowHosts;SPNs;Owner
Description1;"server1$", "server2$", "server3$";"MSSQLSvc/PD01.dom1.com:1521","MSSQLSvc/PD01.dom1";Owner JDOE
Description2;"server4$", "server5$", "server6$";"MSSQLSvc/PD02.dom2.com:1521","MSSQLSvc/PD02.dom2";Owner JDOE
Description3;"server7$", "server8$", "server9$";"MSSQLSvc/PD03.dom1.com:1521","MSSQLSvc/PD03.dom1";Owner JDOE
But how?
I have a powershell script that imports csv-file and creates json-files. My problem is that this line
" ""PrincipalsAllowedToRetrieveManagedPassword"""+": [" | Out-File $filepath1 -Append
gives this result
"PrincipalsAllowedToRetrieveManagedPassword": [ "server1$, server2$, server3$" ],
instead of
"PrincipalsAllowedToRetrieveManagedPassword": [ "server1$", "server2$", "server3$" ],
Use the -replace operator to add "'s around each "word" in the string:
# read data into memory
$csv = Import-Csv -Path $New -Delimiter ';' -Encoding UTF8
# modify all `AllowHosts` and `SPN` cells
$csv |ForEach-Object {
$_.AllowHosts = $_.AllowHosts -replace '([^\s,]+)','"$1"'
$_.SPNs = $_.SPNs -replace '([^\s,]+)','"$1"'
}
# re-export
$csv |Export-Csv -Path path\to\export.csv -NoTypeInformation
The pattern ([^\s,]+) matches (and captures) any consecutive sequence of characters not containing , or whitespace, and the substitution string "$1" expands to "".
Beware that this introduces ambiguity, as "'s are also used as value qualifiers in CSVs - so Export-Csv will escape the quotation marks you've added to retain them, and the resulting file will look like this:
"Description","AllowHosts","SPNs","Owner"
"Description1","""server1$"", ""server2$"", ""server3$""","""MSSQLSvc/PD01.dom1.com:1521"",""MSSQLSvc/PD01.dom1""","Owner JDOE"
"Description2","""server4$"", ""server5$"", ""server6$""","""MSSQLSvc/PD02.dom2.com:1521"",""MSSQLSvc/PD02.dom2""","Owner JDOE"
"Description3","""server7$"", ""server8$"", ""server9$""","""MSSQLSvc/PD03.dom1.com:1521"",""MSSQLSvc/PD03.dom1""","Owner JDOE"
I have a text file. Each line record an event and the fields are delimited by '|'.
Is it possible using some cmdlet like "get-content event.log | export-csv event.csv" to convert text file to csv file?
xxx(1365)|2016-09-29 06:00:00.0|2016-09-29 06:30:00.0|
bbb(110)|2016-09-29 06:30:00.0|2016-09-29 07:00:00.0|
ccc(5243)|2016-09-29 07:00:00.0|2016-09-29 07:30:00.0|
ddd(1950)|2016-09-29 07:30:00.0|2016-09-29 08:00:00.0|
eee(10)|2016-09-29 08:00:00.0|2016-09-29 09:00:00.0|
fff(464)|2016-09-29 09:00:00.0|2016-09-29 10:00:00.0|
dddd(874)|2016-09-29 10:00:00.0|2016-09-29 11:00:00.0|
ggggg(6)|2016-09-29 11:00:00.0|2016-09-29 12:00:00.0|
kkkk(272)|2016-09-29 12:00:00.0|2016-09-29 12:30:00.0|
The Import-Csv cmdlet allows you to specify a delimiter
$file = Import-Csv .\event.log -Delimiter '|'
so in your case, it can be as simple as
Import-Csv .\event.log -Delimiter "|" | Export-Csv .\event.csv -NoTypeInformation
If the file isn't too big (under a few megabytes) you could do it with a straight-forward String.Replace operation:
$content = ( $fileName | Get-Content )
$content = $content -replace '|' ','
$content | Out-File $fileName
Or more succinctly:
( $fileName | Get-Content ) -replace '|' ',' | Out-File $fileName
This won't work very well for large files (more than a few megabytes) because the entire file is loaded into memory as a System.String instance, then the Replace operation will then create a new instance (thus doubling the memory requirement).
A faster version might read from the input file line-by-line and perform the -replace operation for each line - or even character-by-character.
I tried the following to turn a text file into a document by leveraging import-csv where each item in the original document was a new line
Sample file.txt
James Cameron
Kirk Cobain
Linda Johnson
Code:
$array = import-csv file.txt | ConvertFrom-Csv -Delim `r
foreach ($Data in $array)
{
if (sls $Data Master.txt -quiet)
{Add-Content file.txt $Data}
}
It never created the document
Import-Csv takes a CSV and outputs PSCustomObjects. It's intended for when the file has a header row, and it reads that as the properties of the objects. e.g.
FirstName,LastName
James,Cameron
Kirk,Cobain
# ->
#{FirstName='James';LastName='Cameron'}
#{FirstName='Kirk';LastName='Cobain'}
etc.
If your file has no header row, it will take the first row and then ruin everything else afterwards. You need to provide the -Header 'h1','h2',... parameter to fix that. So you could use -Header Name, but your data only has one property, so there's not much benefit.
ConvertFrom-Csv is intended to do the same thing, but from CSV data in a variable instead of a file. They don't chain together usefully. It will try, but what you end up with is...
A single object, with a property called '#{James=Kirk}' and a value of '#{James=Linda}', where 'James' was taken from line 1 as a column header, and the weird syntax is from forcing those objects through a second conversion.
It's not at all clear why you are reading in from file.txt and adding to file.txt. But since you don't have a CSV, there's no benefit from using the CSV cmdlets.
$lines = Get-Content file.txt
$master = Get-Content master.txt
foreach ($line in $lines)
{
if ($master -contains $line)
{
Add-Content file2.txt $line
}
}
or just
gc file.txt |? { sls $_ master.txt -quiet } | set-content file2.txt
Auto-generated PS help links from my codeblock (if available):
gc is an alias for Get-Content (in module Microsoft.PowerShell.Management)
? is an alias for Where-Object
sls is an alias for Select-String (in module Microsoft.PowerShell.Utility)
Set-Content (in module Microsoft.PowerShell.Management)
I have a text file in the following format:
.....
ENTRY,PartNumber1,,,
FIELD,IntCode,123456
...
FIELD,MFRPartNumber,ABC123,,,
...
FIELD,XPARTNUMBER,ABC123
...
FIELD,InternalPartNumber,3214567
...
ENTRY,PartNumber2,,,
...
...
the ... indicates there is other data between these fields. The ONLY thing I can be certain of is that the field starting with ENTRY is a new set of records. The rows starting with FIELD can be in any order, and not all of them may be present in each group of data.
I need to read in a chunk of data
Search for any field matching the
string ABC123
If ABC123 found, search for the existence of the
InternalPartNumber field & return that row of data.
I have not seen a way to use Get-Content that can read in a variable number of rows as a set & be able to search it.
Here is the code I currently have, which will read a file, searching for a string & replacing it with another. I hope this can be modified to be used in this case.
$ftype = "*.txt"
$fnames = gci -Path $filefolder1 -Filter $ftype -Recurse|% {$_.FullName}
$mfgPartlist = Import-Csv -Path "C:\test\mfrPartList.csv"
foreach ($file in $fnames) {
$contents = Get-Content -Path $file
foreach ($partnbr in $mfgPartlist) {
$oldString = $mfgPartlist.OldValue
$newString = $mfgPartlist.NewValue
if (Select-String -Path $file -SimpleMatch $oldString -Debug -Quiet) {
$stringData = $contents -imatch $oldString
$stringData = $stringData -replace "[\n\r]","|"
foreach ($dataline in $stringData) {
$file +"|"+$stringData+"|"+$oldString+"|"+$newString|Out-File "C:\test\Datachanges.txt" -Width 2000 -Append
}
$contents = $contents -replace $oldString $newString
Set-Content -Path $file -Value $contents
}
}
}
Is there a way to read & search a text file in "chunks" using Powershell? Or to do a Read-ahead & determine what to search?
Assuming your fine isn't too big to read into memory all at once:
$Text = Get-Content testfile.txt -Raw
($Text -split '(?ms)^(?=ENTRY)') |
foreach {
if ($_ -match '(?ms)^FIELD\S+ABC123')
{$_ -replace '(?ms).+(^Field\S+InternalPartNumber.+?$).+','$1'}
}
FIELD,InternalPartNumber,3214567
That reads the entire file in as a single multiline string, and then splits it at the beginning of any line that starts with 'ENTRY'. Then it tests each segment for a FIELD line that contains 'ABC123', and if it does, removes everything except the FIELD line for the InternalPartNumber.
This is not my best work as I have just got back from vacation. You could use a while loop reading the text and set an entry flag to gobble up the text in chunks. However if your files are not too big then you could just read up the text file at once and use regex to split up the chunks and then process accordingly.
$pattern = "ABC123"
$matchedRowToReturn = "InternalPartNumber"
$fileData = Get-Content "d:\temp\test.txt" | Where-Object{$_ -match '^(entry|field)'} | Out-String
$parts = $fileData | Select-String '(?smi)(^Entry).*?(?=^Entry|\Z)' -AllMatches | Select-Object -ExpandProperty Matches | Select-Object -ExpandProperty Value
$parts | Where-Object{$_ -match $pattern} | Select-String "$matchedRowToReturn.*$" | Select-Object -ExpandProperty Matches | Select-Object -ExpandProperty Value
What this will do is read in the text file, drop any lines that are not entry or field related, as one long string and split it up into chunks that start with lines that begin with the work "Entry".
Then we drop those "parts" that do not contain the $pattern. Of the remaining that match extract the InternalPartNumber line and present.
I need to read in a CSV file and find replace certain characters from the first line of the file only. I have used foreach-object however this processes the entire file. Any thought on how this can best be achieved.
Here is the code :
Get-Content c:\output.csv | ForEach-Object { $_ -replace "objectGUID", 'StudentID' } | Set-Content c:\output2.csv
This won't fix the problem of having to process the entire file, but should substantially reduce the time it takes to do it if it's a substantially large file.
$Updated = $false
Get-Content c:\output.csv -ReadCount 1000 |
ForEach-Object {
if ($Updated)
{
$_ | Add-Content c:\output2.csv
}
else {
$_[0] = $_[0] -replace "objectGUID", 'StudentID'
$_ | Set-Content c:\output2.csv
$Updated = $true
}
}
Edit: if it's only 3000 rows this should be sufficient:
$FileContent = Get-Content c:\output.csv
$FileContent[0] = $FileContent[0] -replace 'objectGUID', 'StudentID'
$FileContent | Set-Content c:\output2.csv
Ok, Get-Content makes this simple enough if all you want to do is change the first line of a text file.
GC c:\output.csv|select -first 1|%{$_ -replace "objectGUID", 'StudentID'}|Out-File C:\Output2.csv
GC C:\output.csv -readcount 1000|Select -skip 1|Out-File C:\Output2.csv -Append
That will pull the first line only, replacing the text you wanted and write it to a new file (assuming you don't already have an Output2.csv file). After that it reads in the rest of the file skipping the first line and adds that to the same file. You can delete the original file after that and rename the output file if you feel the need.