I'm attempting to write a Twitter Powershell script that will use community created interfaces PoshTwitter with the Twitter API to attempt and find a list of followers who are potential spammers.
I have a feeling that my problem lies not with the particular cmdlet I'm calling (Get-TwitterFollowers), but rather with the difference between assigning a variable:
If I try this:
$rawFol = get-twitterfollowers -page $page -raw 1
$rawFol is different than if I do this:
get-twitterfollowers -page $page -raw 1 > .\page$page.txt
$rawFol = gc .\page$page.txt
The Get-TwitterFollowers cmdlet returns an XML file converted to string.
What things can I try to determine the differences between these two assignments? They look like they'd result with same content.
The difference you're seeing is how powershell handles new lines in strings. When calling the get-twitterfollowers CmdLet, it is either returning a single string or an array of strings. My guess by your description is that it returns a string. So the $rawFol variable will have a single string value. Any new lines are simply embedded into the string value.
The second command you write the return to a file. Now all of the newlines in the string are represented as lines in the file. Later when you call gc on that file, each line will be returned as a separate string. So the $rawFol variable will now have an array of strings.
Related
While using powershell I struggle to build up a filename from two variables. When I originally creaded the powershell script, it was working fine. Now I have tried to move some repeatable steps into a function, but the string behaviour is different.
MWE:
$topa = "ABC"
$topb = "XYZ"
function Test-Fun{
param(
$a,
$b
)
echo "$($a)H$($b).csv"
}
echo "$($topa)H$($topb).csv"
Test-Fun($topa, $topb)
The output on my system is
ABCHXYZ.csv
ABC XYZH.csv
Originally, I wanted to use an underscore instead of H and thought that is causing issues, but its not. What did I miss or rather what is the difference between string expansion within a function and outside of it?
You are calling Test-Func wrong. The comma after $topa will create an array, so you basically pass []"ABC", "XYZ" as an array to $a. In that case $b is empty!
You can easily fix this by removing the comma (also the parentheses are not necessary):
Test-Fun $topa $topb
I know I can dot source a file:
. .\MyFunctions.ps1
But, I would like to dot source the commands in a string variable:
. $myFuctions
I see that this is possible:
.{$x=2}
And $x equals 2 after the script block is sourced.
But... .{$myFunctions} does not work.
I tried $myFunctions | Invoke-Expression, but it doesn't keep the source function in the current scope. The closest I have been able to come up with is to write the variable to a temporary file, dot source the file, and then remove the file.
Inevitably, someone will ask: "What are you trying to do?" So here is my use case:
I want to obfuscate some functions I intend to call from another script. I don't want to obfuscate the master script, just my additional functions. I have a user base that will need to adjust the master script to their network, directory structure and other local factors, but I don't want certain functions modified. I would also like to protect the source code. So, an alternate question would be: What are some good ways to protect PowerShell script code?
I started with the idea that PowerShell will execute a Base64-encoded string, but only when passed on the command line with -EncodedCommand.
I first wanted to dot source an encoded command, but I couldn't figure that out. I then decided that it would be "obfuscated" enough for my purposes if I converted by Base64 file into a decode string and dot sourced the value of the string variable. However, without writing the decoded source to a file, I cannot figure out how to dot source it.
It would satisfy my needs if I could Import-Module -EncodedCommand .\MyEncodedFile.dat
Actually, there is a way to achieve that and you were almost there.
First, as you already stated, the source or dot operator works either by providing a path (as string) or a script block. See also: . (source or dot operator).
So, when trying to dot-source a string variable, PowerShell thinks it is a path. But, thanks to the possibility of dot-sourcing script blocks, you could do the following:
# Make sure everything is properly escaped.
$MyFunctions = "function Test-DotSourcing { Write-Host `"Worked`" }"
. { Invoke-Expression $MyFunctions }
Test-DotSourcing
And you successfully dot-sourced your functions from a string variable!
Explanation:
With Invoke-Expression the string is evaluated and run in the child scope (script block).
Then with . the evaluated expressions are added to the current scope.
See also:
Invoke-Expression
About scopes
While #dwettstein's answer is a viable approach using Invoke-Expression to handle the fact that the function is stored as a string, there are other approaches that seem to achieve the same result below.
One thing I'm not crystal clear on is the scoping itself, Invoke-Expression doesn't create a new scope so there isn't exactly a need to dot source at that point...
#Define your function as a string
PS> $MyUselessFunction = "function Test-WriteSomething { 'It works!' }"
#Invoke-Expression would let you use the function
PS> Invoke-Expression $MyUselessFunction
PS> Test-WriteSomething
It works!
#Dot sourcing works fine if you use a script block
PS> $ScriptBlock = [ScriptBlock]::Create($MyUselessFunction)
PS> . $ScriptBlock
PS> Test-WriteSomething
It works!
#Or just create the function as a script block initially
PS> $MyUselessFunction = {function Test-WriteSomething { 'It works!' }}
PS> . $MyUselessFunction
PS> Test-WriteSomething
It works!
In other words, there are probably a myriad of ways to get something similar to what you want - some of them documented, and some of them divined from the existing documentation. If your functions are defined as strings, then Invoke-Expression might be needed, or you can convert them into script blocks and dot source them.
At this time it is not possible to dot source a string variable.
I stand corrected! . { Invoke-Expression $MyFunctions } definitely works!
I found the following code in a book about PowerShell scripts:
$Text = Read-Host -Prompt 'Input your text'
function FastTrain($text) {
$h = #{}
ForEach ($word in [regex]::split($text.ToLower(), '\W+'))
{
$h[$word] = ''
}
$h
}
FastTrain($Text)
I tried to run it, and got this:
What does it do? I think that it gets a string from the user, and then searches for characters, but I am not entirely sure.
Read a line of text from the user
$Text = Read-Host -Prompt 'Input your text'
Define a function
function FastTrain($text) {
Create a new hashtable. This is a data structure that maps keys to values.
$h = #{}
Lower-case the text argument and split it on successive non-word characters. This results in an array of “words” (however, since “word” characters for regular expressions are a quite arbitrary concept of little use anywhere, this will also include numbers, underscores, and a bunch of other things apart from letters).
ForEach ($word in [regex]::split($text.ToLower(), '\W+'))
{
Use the word as key in the hashtable and set the value to an empty string. This is merely a poor-man's version of a set, so the hashtable will contain all unique words from the input as keys (the values are irrelevant).
$h[$word] = ''
}
Return the hashtable
$h
}
Run above function on the input read earlier. This will also cause the hashtable from earlier to be printed on the screen since any object that is returned from a statement or pipeline will be output by default.
FastTrain($Text)
Note that this usage of PowerShell functions is technically incorrect and can easily lead to mistakes. A PowerShell function is invoked like any other PowerShell command, but not like a .NET method. So arguments are separated by spaces and there are no parentheses. In this case it works because there is only a single argument.
Given how atrocious this example is, I guess you should find a better book. This code looks nothing like how PowerShell code should look (in my opinion at least). The function performed by that code is essentially “Given a string, return all unique words from it”. A more PowerShell-ey version of that function would probably be:
function Get-UniqueWords($Text) {
$Text.ToLower() -split '\W+' | Select -Unique
}
No messing around with a hashtable, just to get a set of sorts. No unnecessary call to a .NET method where a PowerShell operator suffices. And using the pipeline to transform and/or filter a stream of data. Loops like that are often unnecessary since the pipeline is often easier to read and grasp (since you can just read how things are piped into another, instead of having to parse what happens to data structures to find out what happens to your data).
However, considering my gripe about \w/\W from earlier, the following regex would probably yield saner results for humans:
function Get-UniqueWords($Text) {
$Text.ToLower() -split '\P{L}+' | Select -Unique
}
This really only considers letters.
It is reading a sentence from the user:
$Text = Read-Host -Prompt 'Input your text'
It then creates an empty hashtable (a collection of key-value pairs):
$h = #{}
Then, splits the sentence into words:
[regex]::split($text.ToLower(), '\W+'))
And adds each one to the hashtable (with the word as the key, and nothing for the value):
$h[$word] = ''
Finally, it prints the hashtable:
$h
There is a function definition/call mixed in, but the above is what the code does.
I have the following json code in my powershell script.
I set the $variable to 1111111111
$jsonfile = '{"Version": "2012-10-17","Statement": {"Effect": "Allow","Action": "sts:AssumeRole","Resource": "arn:aws:iam::$variable:role/xxxxxx"}}'
The output gives ....arn:aws:iam::$variable:role/xxxxxx..... instead of ....arn:aws:iam::1111111111:role/xxxxxx
The problem is that I must use the single quote for the json string otherwise I will get an error. If I use single quote I wont be able to put the variables inside the string. How do I workaround this problem?
There are various ways to solve your problem, but perhaps the easiest approach is to use PowerShell's string interpolation:
use a double-quoted string overall to enable interpolation of embedded variable references and subexpressions ($(...)).
escape embedded " chars. as `" (using backticks)
disambiguate variable references by enclosing the variable name in {...}.
Simplified example:
PS> $variable='111'
PS> "{`"Version`": `"arn:aws:iam::${variable}:role/xxxxxx`"}}"
{"Version": "arn:aws:iam::111:role/xxxxxx"}}
Note that enclosing variable names in {...} in interpolated strings is only necessary if the following char. could be misinterpreted as part of the variable name.
A : following the variable name - as is the case here - is such a case, because PS variables can have a scope specifier preceding the variable name that is separated from the variable name with :, such as in $env:USERNAME.
DAXaholic's helpful answer shows an alternative based on PowerShell's binary -f operator, which is essentially the same as the .NET framework's String.Format method; as such:
it introduces additional complexity, such as needing to know what its escaping rules are ({ chars. must be escape as {{, and how to format its arguments specified on the RHS of -r ({0} refers to the 1st RHS argument, ...)
on the flip side, -f offers many sophisticated formatting options.
Also, consider use of the Convert*-Json cmdlets his answer demonstrates: even though they're less performant, they ultimately make manipulation of JSON much easier and more robust.
Alternatives in the realm of native PowerShell code:
String concatenation with the binary + operator:
'{"Version": "arn:aws:iam::' + $variable + ':role/xxxxxx"}}'
String templating with $ExecutionContext.InvokeCommand.ExpandString():
$variable='111'
$tmpl = '{"Version": "arn:aws:iam::${variable}:role/xxxxxx"}}' # string template *literal*
$ExecutionContext.InvokeCommand.ExpandString($tmpl) # performs on-demand interpolation
Another solution would be
$jsonfile = '{{"Version": "2012-10-17","Statement": {{"Effect": "Allow","Action": "sts:AssumeRole","Resource": "arn:aws:iam::{0}:role/xxxxxx"}}}}' -f $variable
So you have to escape the braces with another brace but in your case you have fewer braces than quotes so it is "less obfuscation" :)
In your case, maybe the simplest solution is just concatenating the strings together instead of using string formatting / interpolation.
In addition you could also go the way with the JSON cmdlets:
$jsonfile |
ConvertFrom-Json |
% { $_.Statement.Resource = "arn:aws:iam::${variable}:role/xxxxxx"; $_ } |
ConvertTo-Json
I am working on PowerShell script that requires me to pass string as param. The string is a comma seperated list of user names. I get the error when i have 100+ user names. But i get no error if the string has less then 100 users. See below. I have tried to pass this value using array with no luck. What is the character limitation for this param and how can i solve this. I am using this in ServiceNow Run PowerShell script. That value of the parameter is passed by ServiceNow using a comma seperated value.
Param(
[string]$itil_users_a = "A.Syafiq,Aaron.Brown,Aaron.Reynnie,Abd.Jalil,Abdu.Hijazi,Abdul.Onny,Abdullah.Ammar,Abel.Muataco"
)
You may be running into the maximum length for command lines - 8191 chars. See this KB article on max command line length.
You can use pipe to get around this.
I can insert a million rows to sqlite3.exe using pipe.
Rather than sqlite.exe .\ex1.db $sql which will return error. You can use this: $sql | sqlite.exe .\ex1.db
The $sql variable can have more than a million rows (so it's more than 8191 characters).