lines = pipeline | 'ReadFromText' >> beam.io.ReadFromText(
'path/to/input-*.csv')
What is the significance of string 'ReadFromText' in the above code?
A string after the pipe | is an optional way to name the step, and it can be named anything not necessarily matching the name of the beam.io.
The above code can also be written without naming the step like the following:
lines = pipeline | beam.io.ReadFromText(
'path/to/input-*.csv')
The redirect key >> is only to be used when naming the step.
Related
I have a file1 with several variables defined:
'$GateWay_HostName'='Biscuits'
'$DomainName'='AND'
'$DC_Name'='Gravy'
I have another file (File2) with line by line commands to send to Cisco devices. I do not want to read file, replace variables, then save file, because the passwords will be entered in cleartext. I can't seem to figure out how to import the variables and replace any matching string with the value.
I can pull in the variables and call them in the script and reference $GateWay_HostName for example with:
$ReplaceVars = Get-Content "C:\folder\file1.csv" | ConvertFrom-StringData
But I can't seem to find anything about going through the imported string to replace all of the variables (some appear once, some appear many, some don't exist).
$CommandstoSend = (Get-Content -Path "C:\folder\File2" -raw)
Because the code to execute the commands are passed as raw, it won't read the variables on the fly. I need to import the raw data otherwise plink won't pass the commands, but I don't care when it gets converted to "raw". Also, if I'm going to end up using a search and replace, I know I don't need to have $VARIABLEname format.
Thanks
You can do the following:
$ReplaceVars = Get-Content "C:\folder\file1.csv" -Raw | ConvertFrom-StringData
$replaceString = ($ReplaceVars.Keys |% {[regex]::Escape($_)}) -join '|'
[regex]::Replace((Get-Content file2 -raw),$replaceString,{param($m) $ReplaceVars[$m.Value]})
Using -Raw on Get-Content creates a single string. This is important for returning a single hash table after piping into ConvertFrom-StringData. Without -Raw, ConvertFrom-StringData outputs an array of hash tables, making lookups more complex.
[regex]::Escape() is to escape special regex characters like $ that could be in your variable names. Joining on | creates a regex alternation (equivalent to logical OR).
The [regex]::Replace() method allows for a script block to be used in the replacement string parameter. Using a parameter $m, we can reference the object that contains the matched text (stored in the Value property) and manipulate it. Here we use it as a key name ($m.Value) for the $replaceVars hash table.
Effectively, this solution looks for text that matches a hash table key name and then replaces that name with the corresponding hash table value.
IMO, the better solution is to configure file2 to be a template file. You can use string format placeholders where the substitutions need to happen. Then just use the string format operator -f. See below for an example:
# file2 contents
$file2 = #'
My DC Name = {0}
Domain Name = {1}
I don't want to edit this line.
Gateway Host = {2}
'#
$file2 -f 'Gravy','AND','Biscuits'
Output:
My DC Name = Gravy
Domain Name = AND
I don't want to edit this line.
Gateway Host = Biscuits
i've written a simple PowerShell script that is designed to take a file name and then move the file into a particular folder.
The files in question are forms scanned in as PDF documents.
Within the file name, I have defined a unique string of characters used to help identify which form it is and then my script will sort the file into the correct folder.
I've captured the file name as a string variable.
I am using -match to evaluate the file name string variable and my issue is that match is acting like...well -like.
For example, my script looks for (F1) in the string and if this returns true my script will move the file into a folder named IT Account Requests.
This all works well until my script finds a file with (F10) in the name, as 'match' will evaluate the string and find a match for F1 also.
How can I use 'match' to return true for an exact string block?
I know this may sound like a fairly basic newbie question to ask, but how do I use -match to tell the different between the two file types?
I've scoured the web looking to learn how to force -match to do what I would like but maybe I need a re-think here and use something other than 'match' to gain the result I need?
I appreciate your time reading this.
Code Example:
$Master_File_name = "Hardware Request (F10).pdf"
if ($Master_File_name -match "(F1)"){Write-Output "yes"}
if ($Master_File_name -match "(F10)"){Write-Output "yes"}
Both if statements return 'yes'
-match does a regular expression based match against your string, meaning that the right-hand side argument is a regex pattern, not a literal string.
In regex, (F1) means "match on F and 1, and capture the substring as a separate group".
To match on the literal string (F1), escape the pattern either manually:
if($Master_File_Name -match '\(F1\)'){Write-Output 'yes'}
or have it done for you automatically using the Regex.Escape() method:
if($Master_File_Name -match [regex]::Escape('(F1)')){Write-Output 'yes'}
I am trying to automate Active Directory installation on Windows Server 2008 using windows powershell. I created a text file with .tmpl extension and added:
[DCINSTALL]
ReplicaOrNewDomain=_ReplicaOrNewDomain__
Then I created an answer file in a text format:
[DCINSTALL]
ReplicaOrNewDomain=$env:ReplicaOrNewDomain
Now I want to be able to write a script in PowerShell which will use the template file to get the value of variable ReplicaOrNewDomain from environment and replace $env:ReplicaOrNewDomain by that value in the text file so that I can use that answer file for AD installation.
You have a few options to do this. One is Environment.ExpandEnvironmentVariables. This uses a %variable% syntax (instead of $env:variable), so it would be simpler if you only want to substitute environment variables:
gc input.tmpl | foreach { [Environment]::ExpandEnvironmentVariables($_) } | sc out.ini
A more complete expansion of PowerShell expressions can be achieve via ExpandString. This is more useful if you want to insert actual PowerShell expressions into the template:
gc input.tmpl | foreach { $ExecutionContext.InvokeCommand.ExpandString($_) } | sc out.ini
A third option would be something like a customized templating scheme that uses Invoke-Expression, which I implemented here.
You can do that with a simple replacement like this:
$f = 'C:\path\to\your.txt'
(Get-Content $f -Raw) -replace '\$env:ReplicaOrNewDomain', $env:ReplicaOrNewDomain |
Set-Content $f
or like this:
$f = 'C:\path\to\your.txt'
(Get-Content $f -Raw).Replace('$env:ReplicaOrNewDomain', $env:ReplicaOrNewDomain) |
Set-Content $f
Note that when using the -replace operator you need to escape the $ (because otherwise it'd have the special meaning "end of string"). When using the Replace() method you just need to use single quotes to prevent expansion of the variable in the search string.
However, why the intermediate step of replacing the template parameter _ReplicaOrNewDomain__ with a different template parameter $env:ReplicaOrNewDomain? You would make your life easier if you just kept the former and replaced that with the value of the environment variable ReplicaOrNewDomain.
One thing that I like to do with my template files is something like this.
[DCINSTALL]
ReplicaOrNewDomain={0}
OtherVariable={1}
Then in my code I can use the format operator -f to make the changes.
$pathtofile = "C:\temp\test.txt"
(Get-Content $pathtofile -Raw) -f $env:ReplicaOrNewDomain, "FooBar" | Set-Content $pathtofile
It can help if you have multiple things that you need to update at once. Update your file with as many place holders as you need. You can use the same one multiple times if need be in the file.
[DCINSTALL]
ReplicaOrNewDomain={0}
SimilarVariable={0}
Caveat
If your actual file is supposed to contain curly braces you need to double them up to the are escaped.
You can use the ExpandString function, like this:
$ExecutionContext.InvokeCommand.ExpandString($TemplVal)
(assuming $TemplVal has the template string).
I am currently trying to build a simple script that would report back a string from a filename
I have a directory full of files named as follows:
C:\[20141002134259308][302de103-6dc8-4e29-b303-5fdbd39c60c3][U0447744][10.208.15.40_54343][ABC_01].txt
C:\[20141002134239815][302de103-6dc8-4e29-b303-5fdbd39c60c3][U0011042][10.168.40.34_57350][ABC_01].txt
C:\[20141002134206386][302de103-6dc8-4e29-b303-5fdbd39c60c3][u1603381][10.132.171.132_54385][ABC_01].txt
C:\[2014100212260259][302de103-6dc8-4e29-b303-5fdbd39c60c3][U0010217][10.173.0.132_49921][ABC_01].txt
So, I'd like to extract from each filename the user ids that are identified starting with a letter U and seven digits, then create a new txt o csv and have all these Ids listed. That's it.
As Patrice pointed out, you really should try and do it yourself and come to us with the relevant piece of code you tried and the error that you are getting. That said, I'm bored, and this is really easy. I'd use a regex match against the name of the file, and then for each one that matched I'd output the captured string:
Get-ChildItem 'C:\Path\To\Files\*.txt' | Where{$_.Name -match "\[(U\d{7})\]"} | ForEach{$Matches[1]}
That will return:
U0447744
U0011042
u1603381
U0010217
If you want to output it to a file, just pipe that to Out-File, and specify the full path and name of the file you want to save that in.
just wondering if i could do this in powershell, or even a c#/vb.net command line program.
I have data that looks like this:
(source: kalleload.net)
I have a Teams Table. It looks like this:
| id | teamname | teamcity |
so for example, C2 has the value "Atlanta Braves". I need to split this up into "Atlanta" and "Braves". Data is consistent. for example "New York Mets" is actually "NewYork Mets".
So i need to go through column C and D and insert all the teams (no duplicates into the db).
One line of PowerShell will read in the CSV file and create a custom object for each home and away team listing (with a property for the city name and for the team name). The last command in the pipeline will eliminate the duplicates.
$TeamsAndCities = import-csv -path c:\mycsvfile.csv | foreach-object { $_.away, $_.home | select-object #{Name='City';Expression={$_.split(' ')[0]}}, #{Name='Team';Expression={$_.split(' ')[1]}} } | select-object -unique
You can do database access from PowerShell as well, but that might be suited to a new question with some more details about the database you are connecting to.
I rarely code in VBA/VB but...
Something like
Dim rngAwayTeam As Range, rngHomeTeam As Range
set rngAwayTeam = Worksheets("YourWorksheet").Range("C2")
set rngHomeTeam = Worksheets("YourWorksheet").Range("D2")
Dim rowOffset As Integer
rowOffset = 1
Do While (rngAwayTeam.Offset(rowOffset,1).Text <> "")
'Do something with rngAwayTeam.Offset(rowOffset,1).Text
'and rngHomeTeam.Offset(rowOffset,1).Text
rowOffset = rowOffset + 1
Loop
There are other ways I'm sure, but, here is what I would do.
Yes that is an excel macro. Again, I rarely use VBA or .Net, just trying to help you out the best I can. You could just use a C# COM object for the database side of things. (Still new, can't comment.)
You can do it in C# console application quite easily.
All you have to do is loop through each line in the file, adding it to an array using split on the comma (,).
Then you can use your array to display the values or retrieve a specific value on a row.