I have a csv file that contains fields with values that begin with "$" that are representative of variables in different powershell scripts. I am attempting to import the csv and then replace the string version of the variable (ex. '$var1') with the actual variable in the script. I have been able to isolate the appropriate strings from the input but I'm having difficulty turning the corner on modifying the value.
Example:
CSV input file -
In Server,Out Server
\\$var1\in_Company1,\\$var2\in_Company1
\\$var1\in_Company2,\\$var2\in_Company2
Script (so far) -
$Import=import-csv "C:\temp\test1.csv"
$Var1="\\server1"
$Var2="\\server2"
$matchstring="(?=\$)(.*?)(?=\\)"
$Import| %{$_ | gm -MemberType NoteProperty |
%{[regex]::matches($Import.$($_.name),"$matchstring")[0].value}}
Any thoughts as to how to accomplish this?
The simplest way to address this that I could think of was with variable expansion as supposed to replacement. You have the variables set in the CSV so lets just expand them to their respective values in the script.
# If you dont have PowerShell 3.0 -raw will not work use this instead
# $Import=Get-Content $path | Out-String
$path = "C:\temp\test1.csv"
$Import = Get-Content $path -Raw
$Var1="\\server1"
$Var2="\\server2"
$ExecutionContext.InvokeCommand.ExpandString($Import) | Set-Content $path
This will net the following output in $path
In Server,Out Server
\\\\server1\in_Company1,\\\\server2\in_Company1
\\\\server1\in_Company2,\\\\server2\in_Company2
If the slashes are doubled up here and you do not want them to be then just change the respective variable values in your script of the data in the CSV.
Caveat
This has the potential to execute malicious code you did not mean too. Have a look at this thread for more on Expanding variables in file contents. If you are comfortable with the risks then this solution as presented should be fine.
Maybe I misunderstood, but do you need this?
$Import=import-csv "C:\temp\test1.cvs"
$Var1="\\server1"
$Var2="\\server2"
Foreach ($row in $Import )
{
$row.'In Server' = ($row.'In Server' -replace '\\\\\$var1', "$var1")
$row.'Out Server' = ($row.'Out Server' -replace '\\\\\$var2', "$var2")
}
$import | set-content $path
Related
I recently started using Powershell and I'm trying out some code.
I have a .cfg file with several rules of code. The code is written like this:
ad.name=1
ad.virtual=active
ad.set=none
ad.partition=78
Now I want to export the value of ad.partition, which is 78, to a new file. I don't want to export ad.partition or = but only the number 78.
So far I got this:
Get-Content -Path C:\file.cfg | Where-Object {$_ -like 'ad.partition=78'}
But then I -obviously- just get the variable and the value. Not sure how to continue...
I hope someone has a way of achieving what I want.
After saving the value in a new file, would it be possible to add spaces? For example, the value consists out of 9 digits, e.g. 123456789. The desired output result would be 123 456 789.
Use ConvertFrom-StringData cmdlet to create a hash table from your file, then simply index the key you are after:
$h=(Get-Content -Path C:\file.cfg | ConvertFrom-StringData)
$h.("ad.partition") -replace ('^(\d{1,3})(\d{1,3})?(\d{1,3})?','$1 $2 $3') > C:\out.cfg
You can use the Select-String cmdlet to capture your desired value using a regex. Then just pipe the result to the Out-File cmdlet. To get your desired output with spaces, you can use a simple format string:
"{0:### ### ###}" -f [int](Select-string 'ad\.partition=(.*)' -Path C:\file.cfg).Matches.Groups[1].Value |
Out-File C:\result.cfg
Basically I have a huge csv of phishing links and I'm trying to trim off https://www. and anything after .com .edu etc. so basically the ideal ouput of the powershell script would be a long list of urls all of which look something like google.com or microsoft.com so far I have imported the csv but everything I have tried either doesn't work or leaves the www on the beggining. Any help would be great. The csv im using is this: http://data.phishtank.com/data/online-valid.csv
$urls = Import-Csv -Path .\online-valid.csv | select -ExpandProperty "url"
The below will take your CSV and do magic for you. Have a play around with [Uri], it is very useful when parsing web links.
$csv = import-csv C:\temp\verified_online.csv
Foreach($Site in $csv) {
$site | Add-Member -MemberType NoteProperty -Name "Host" -Value $(([Uri]$Site.url).Host -replace '^www\.')
}
$csv | Export-Csv C:\temp\verified_online2.csv -NoTypeInformation
Adjusted based on recommendation from Mklement0.
A concise and fast alternative to Drew's helpful answer based on casting the URL strings directly to an array of [uri] (System.Uri) instances, and then trimming prefix www., if present, from their .Host (server name) property:
([uri[]] (Import-Csv .\online-valid.csv).url).Host -replace '^www\.'
Note that the -replace operator is regex-based, and regex ^www\. makes sure what www is only replaced at the start (^) of the string, and only if followed by a literal . (\.), in which case this prefix is removed (replaced with the implied empty string); if no such prefix is present, the input string is passed through as-is.
The solution reads the entire CSV file into memory at once, for convenience and speed, and outputs just the trimmed server names, as an array of strings.
I have two csv files i need to compare them, then output a new file that will have the new values of the prophetess that changes and will mark(highlight) those properties some how. I already have the resulting file with all the values and all the properties but i dont know how to mark the specific properties that changed. Is there a way to do that is powershell?
Here is a sample of my code:
[CmdletBinding()]
param(
[Parameter(Mandatory=$true)]
$file1,
[Parameter(Mandatory=$true)]
$file2
)
$content1 = Import-Csv -Path $file1
$content2 = Import-Csv -Path $file2
$props = $content1 | Get-Member -MemberType NoteProperty | Select-Object -ExpandProperty Name
$comparedLines = Compare-Object $content1 $content2 -Property $props -PassThru | Where-Object {$_.SideIndicator -eq "=>" }
$comparedLines | Export-csv -Path C:\FullPUF\Difference.csv –NoTypeInformation
So, if I understand this correctly, $content2 is the newer file, right?
If so, then $comparedlines is going to contain all of the values which changed. If that's right, then we're in business, because Doug Finke wrote an absolutely awesome PowerShell module called importExcel that is going to make this trivially easy. But first, how did I do this?
I needed some files to test against, and I didn't have your source file so I couldn't guess at the values. I just made a simple CSV with a Name and Position value, and changed the Position numbers between Content1 and Content2.
How to solve this problem, first off, download Import-Excel right now. If you're on PowerShell version 4.0 or higher you can install it from PowerShell like so:
Find-Module ImportExcel | Install-Module
This module has the concept of ConditionalText filter rules. You can create a new one like this.
1..5 | Export-Excel C:\temp\test.xslx -Show`
-ConditionalText (New-ConditionalText -ConditionType GreaterThan 3)
This will output and display a new spreadsheet with the numbers 1 through 5 in it, and highlight the ones higher than 3.
We can also highlight rows that ContainsText which matches a value we know.
So, we take the value of $comparedLines using ForEach-Object, and create a New-ConditionalText rule to highlight the line if it contains one of the properties.
$highlight =$comparedLines | % {New-ConditionalText -ConditionalType ContainsText $_.Name}
Next, we will echo the contents of $content2, and use Export-Excel to create a new .xslx file and apply a conditional formatting rule to it.
$content2 | Export-Excel r:\test.xlsx -show -ConditionalText ($highlight)
And the output
You'll probably need to play with the code a tiny bit, and modify it to fit whatever your columns are called, but this is how you can export a csv and highlight the differences using PowerShell. Except there's no such thing as highlighting or text format in a .csv so you have to use an Excel SpreadSheet instead.
Lemme know if you need me to dig deeper anywhere :)
I've looked all around this site and can't quite seem to find anything that fits my situation. Basically, I am trying to write an addition to the NETLOGON file that will replace text in a text file on all of our users' desktops. The current text is static across the board.
The text I want it changed to will be unique to each user. I want to change the current text (user1) to the users AD username (i.e. johnd, janed, etc.). I am using Windows Server 2008 R2 and all the workstations are Windows 7 Professional SP1 64 bit.
Here's what I have tried so far (with a few variables, which none have worked for one reason or the other):
gc c:\Users\%USERNAME%\desktop\VPN.txt' -replace "user1",$env:username | out-file c:\Users\%USERNAME%\desktop\VPN.txt
I didn't get an error, but it also did not go back to the normal "PS C:>" prompt, just ">>>" and the file did not change as anticipated.
If that is how you have the code exactly then I suppose it is because you have an opening single quote without a closing one. You are still going to have two other problems and you have one answer in your code. The >>> is the line continuation characters because the parser knows that the code is not complete and giving you the option to continue with the code. If you were purposely coding a single line on multiple lines you would consider this a feature.
$path = "c:\Users\$($env:username)\desktop\VPN.txt"
(Get-Content $path) -replace "user1",$env:username | out-file $path
Closed the path in quotes and used a variable since you called the path twice.
%name% is used in command prompt. Environment variables in PowerShell use the $env: provider which you did you once in your snippet.
-replace is a regex replaced tool that can work against Get-Content but you need to capture the result in a sub expression first.
Secondly with -replace is for regex and your string is not regex based you could just use .Replace() as well.
Set-Content is generally preferred over Out-File for performance reasons.
All that being said...
you could also try something like this.
$path = "c:\Users\$($env:username)\desktop\VPN.txt"
(Get-Content $path).Replace("user1",$env:username) | Set-Content $path
Do you want to only replace the first occurrence?
You could use a little regex here with a tweak in how you get the use Get-Content
$path = "c:\Users\$($env:username)\desktop\VPN.txt"
(Get-Content $path | Out-String) -replace "(.*?)user1(.*)",('$1{0}$2' -f $env:username) | out-file $path
Regex will match the entire file. There are two groups which it captures.
(.*?) - Up until the first "user1"
(.*) - Everything after that
Then we use the format operator to sandwich the new username in between those capture groups.
Use:
(Get-Content $fileName) | % {
if ($_.ReadCount -eq 1) {
$_ -replace "$original", "$content"
}
else {
$_
}
} | Set-Content $fileName
I have a piece of code that should grab all the .CSV files out of a directory and import them, using pipe character delimiters.
$apeasy = dir .\APEasy\*.csv | Import-CSV -delimiter '|'
The problem is this returns null. Without exception, no matter what I do.
The weird thing is that this works:
dir .\APEasy\*.csv
It returns a FileInfo object, which SHOULD be getting piped into Import-CSV as the file to import. In addition, these two commands work:
$csvFiles = dir .\Processed_Data_Review -Filter *.txt | Import-CSV -header(1..19) -delimiter "$([char]0x7C)"
dir .\LIMS -Filter *.csv | Import-CSV | ? {$_.SampleName -like "????-*"}| Export-CSV -Path .\lims_output.txt -NoTypeInformation
I really have no idea what's going on here. I'm dealing with a basic pipe-delimited file, quotations around every field (which is fine, I can import the data with those). Nothing special going on here. The file is THERE, Import-CSV just isn't GETTING it for some reason.
So my question is this: What could cause a file grabbed by 'dir' to fail to be piped into Import-CSV?
EDIT: The overall goal of this is to read the CSV files in a directory without knowing their name in advance, and output specific columns into a variety of output files.
EDIT: This is the line of code as it stands right now:
$apeasy = Get-ChildItem .\APEasy\*.csv | Select-Object -ExpandProperty FullName | Import-CSV -delimiter "$([char]0x7C)"
Isolating the Get-ChildItem statement, and isolating Get-Child and Select-Object both return what they should. A list of csv files in the directory, and an array of their full paths, respectively. Still, when they get piped into Import-CSV, they dissappear. Get-Member on the variable returns that it's empty.
Import-Csv accepts only strings (path) from the pipeline so in order to pipe directly to it you need to first expand the paths:
dir .\APEasy\*.csv |
select -expand fullname |
Import-CSV -delimiter '|'
Although cmdlets like Get-Content work that way in that they can accept the Path parameter by property name (and LiteralPath by value, which makes sense), Import-Csv is a little inconsistent. It only accepts the path to import by value:
-Path <String[]>
Specifies the path to the CSV file to import. You can also pipe a path to Import-Csv.
Required? false
Position? 1
Default value None
Accept pipeline input? true (ByValue)
Accept wildcard characters? false
So you could use
Get-ChildItem ... | Select-Object -ExpandProperty FullName | Import-Csv
but it won't work out of the pipeline directly.
This is a known bug in Import-Csv - even though you should be able to pipe Get-ChildItem (a.k.a dir) output directly to Import-Csv, that is still broken as of Windows PowerShell v5.1 / PowerShell Core v6.0.2.
Outputting the files' .PSPath property value is a way of working around the problem (.FullName works too) : The input objects' .PSPath property is what should be bound to Import-Csv's -LiteralPath parameter, but currently isn't.
Since you're collecting all input in memory anyway, you can simply use member-access enumeration (PSv3+) to access the .PSPath property on all matching files:
$apeasy = (Get-ChildItem .\APEasy\*.csv).PSPath | Import-CSV -delimiter '|'