I know that a script block executed via start-job cannot see the variables outside of the script block. To pass variables in you use the -arguments paramater. From the doco I've read though, jobs can't pass objects to each other without serialising them. Apparently this is because of how jobs works - when using Start-job, PowerShell creates a new process and runs the commands there; in order to transfer the object to the other process it needs to serialise it, then the other exe will deserialise it when it gets imported. This poses a problem when you want to use objects with start-job.
NOTE: The below are examples to demonstrate my problem - the actual cmdlets and scripts I am running are completely different and more complex, so in case you are wondering why I would even run these commands this way just bear in mind I wouldn't, they are just easy commands to demonstrate my issue.
Here is an example of the syntax I've used for a simple Get-aduser command
$user = get-aduser samaccount
When we output $user we see that the object is of type ADUser
$user|GM
TypeName: Microsoft.ActiveDirectory.Management.ADUser
Let's now serialise the object (to simulate what start-job does)
$user| Export-Clixml C:\temp\test.xml
And now rebuild it (deserialise)
$user = Import-Clixml C:\temp\test.xml
Now when when we view it's type it is different. It has the word 'derserialzed' in front.
$user|GM
TypeName: Deserialized.Microsoft.ActiveDirectory.Management.ADUser
The problem I now have is that it isn't a true representation of the orignal object. Even if all the properties and settings were identical we still have the problem that the object type is different. Now to demonstrate why this is a problem.
Get-aduser accepts a string as the input (very first example) but it also will accept a valid ADuser object. But when we now run:
get-aduser $user
We get the following error:
"Cannot convert the "AD_distinguishedname_here. I have omitted this for security reasons" value of type
"Deserialized.Microsoft.ActiveDirectory.Management.ADUser" to type "Microsoft.ActiveDirectory.Management.ADUser"."
This error is because Get-aduser expects you to provide an object of type Microsoft.ActiveDirectory.Management.ADUser but we provided Deserialized.Microsoft.ActiveDirectory.Management.ADUserand it doesn't know how to convert it. This is exactly what happens when you run command via start-job and why you can't pass objects to a job.
As I said above, I am not using get-aduser in my real code, I am just using this simple command everyone has access to, to demonstrate the issue. In my real code I must provide an object to the job.
So my question is, does anyone know how you get around this or know how to rebuild the object in it's original form?
After spending 8 hours on this straight I finally figured out what is going on. It's a long story but thr TLDR version is that you cannot get around this with any native tools - you have to essentially rebuild the object inside the start-job script block, and if you have a large complex object this just isn't feasible. I ended up findind a module you can download which runs commands via start-thread instead of the start-job and this does not use serialisation. Guide here to DL and install it. This worked when I tested it
To get the PowerShell module, go to PowerShell gallery and search for "start-thread"
Related
I'm going to start by saying I'm still pretty much a rookie at PowerShell and hoping there is a way to do this.
We have a utils.ps1 script that contains just functions that we dot source with in other scripts. One of the functions returns back a default value if a value is not passed in. I know I could check $args and such but what I wanted was to use the function for the default value in the parameters.
param(
[string]$dbServer=$(Get-DefaultParam "dbServer"),
[string]$appServer=$(Get-DefaultParam "appServer")
)
This doesn't work since the Util script hasn't been sourced yet. I can't put the dot source first because then params doesn't work as it's not the top line. The utils isn't a module and I can't use the #require.
What I got working was this
param(
[ValidateScript({ return $false; })]
[bool]$loadScript=$(. ./Utils.ps1; $true),
[string]$dbServer=$(Get-DefaultParam "dbServer"),
[string]$appServer=$(Get-DefaultParam "appServer")
)
Create a parameter that loads the script and prevent passing a value into that parameter. This will load the script in the correct scope, if I load it in the ValidateScript it's not in the correct scope. Then the rest of the parameters have access to the functions in the Utils.ps1. This probably is not a supported side effect, aka hack, as if I move the loadScript below the other parameters fail since the script hasn't been loaded.
PowerShell guarantee parameters will always load sequential?
Instead should we put all the functions in Utils.ps1 in global scope? this would need to run Utils.ps1 before the other scripts - which seems ok in scripting but less than ideal when running the scripts by hand
Is there a more supported way of doing this besides modules and #require?
Better to not use default value of params and just code all the checks after sourcing and check $args if we need to run the function?
It would be beneficial to instead turn that script into a PowerShell Module, despite your statement that you desire to avoid one. This way, your functions are always available for use as long as the module is installed. Also, despite not wanting to use it, the #Require directive is how you put execution constraints on your script, such as PowerShell version or modules that must be installed for the script to function.
If you really don't want to put this into a module, you can dot-source utils.ps1 from the executing user's $profile. As long as you don't run powershell.exe with the -NoProfile parameter, the profile loads with each session and your functions will be available for use.
Using CmdletBinding, is there an easy way to regurgitate the exact parameters that a cmdlet was called with, so I can call another cmdlet with the exact same parameters?
I'm writing Powershell Cmdlets in Powershell. I'm using advanced functions. I have a cmdlet called Get-Environment, with several optional parameters like [string]EnvironmentName and [switch]Active. I have another cmdlet, called Get-Machine, with all of the same optional parameters; it calls Get-Environment. Originally, before I added the [switch]Active parameter, I simply called Get-Environment with all variables explicitly (see below).
I can't do the same thing now, because if I add "active" then it will be set. I don't want to have make a test in Get-Machine to see if Active is true and have two different versions of the Get-Environment call. I'd prefer to not have to trawl through the $PSBoundParameters hashtable and reconstruct the original strings, but that looks like the only feasible way forward (unless I'm missing something.)
Original code inside get-machine:
$environments = get-Environment -EnvironmentName $EnvironmentName
Oh for Pete's sake. I found it. I was missing the big stupid easy thing. I'll leave this up for others, and in case someone has an even better answer.
https://ss64.com/ps/psboundparameters.html
$PSBoundParameters can be used to call a subordinate function or cmdlet passing the same parameters - PowerShell will automatically splat the hash table's values instead of having to type each of the parameters:
get-otherthing #PSBoundParameters
In my new project team, for each powershell cmdlet they have written proxy function. When i asked the reason for this practice, they said that it is a normal way that automation framework would be written. They also said that If powershell cmdlet is changed then we do not need to worry ,we can just change one function.
I never saw powershell cmdlets functionality or names changed.
For example, In SQL powershell module they previously used snapin then they changed to module. but still the cmdlets are same. No change in cmdlet signature. May be extra arguments would have added.
Because of this proxy functions , even small tasks taking long time. Is their fear baseless or correct? Is there any incident where powershell cmdlets name or parameter changed?
I guess they want to be extra safe. Powershell would have breaking changes here and here sometimes but I doubt that what your team is doing would be impacted by those (given the rare nature of these events). For instance my several years old scripts continue to function properly up to present day (and they were mostly developed against PS 2-3).
I would say that this is overengineering, but I cant really blame them for that.
4c74356b41 makes some good points, but I wonder if there's a simpler approach.
Bear with me while I restate the situation, just to ensure I understand it.
My understanding of the issue is that usage of a certain cmdlet may be strewn about the code base of your automation framework.
One day, in a new release of PowerShell or that module, the implementation changes; could be internal only, could be parameters (signature) or even cmdlet name that changes.
The problem then, is you would have to change the implementation all throughout your code.
So with proxy functions, you don't prevent this issue; a breaking change will break your framework, but the idea is that fixing it would be simpler because you can fix up your own proxy function implementation, in one place, and then all of the code will be fixed.
Other Options
Because of the way command discovery works in PowerShell, you can override existing commands by defining functions or aliases with the same name.
So for example let's say that Get-Service had a breaking change and you used it all over (no proxy functions).
Instead of changing all your code, you can define your own Get-Service function, and the code will use that instead. It's basically the same thing you're doing now, except you don't have to implement hundreds of "empty" proxy functions.
For better naming, you can name your function Get-FrameworkService (or something) and then just define an alias for Get-Service to Get-FrameworkService. It's a bit easier to test that way.
One disadvantage with this is that reading the code could be unclear, because when you see Get-Service somewhere it's not immediately obvious that it could have been overwritten, which makes it a bit less straightforward if you really wanted to call the current original version.
For that, I recommend importing all of the modules you'll be using with -Prefix and then making all (potentially) overridable calls use the prefix, so there's a clear demarcation.
This even works with a lot of the "built-in" commands, so you could re-import the module with a prefix:
Import-Module Microsoft.PowerShell.Utility -Prefix Overridable -Force
TL;DR
So the short answer:
avoid making lots and lots of pass-thru proxy functions
import all modules with prefix
when needed create a new function to override functionality of another
then add an alias for prefixed_name -> override_function
Import-Module Microsoft.PowerShell.Utility -Prefix Overridable -Force
Compare-OverridableObject $a $b
No need for a proxy here; later when you want to override it:
function Compare-CanonicalObject { <# Stuff #> }
New-Alias Compare-OverridableObject Compare-CanonicalObject
Anywhere in the code that you see a direct call like:
Compare-Object $c $d
Then you know: either this intentionally calls the current implementation of that command (which in other places could be overridden), or this command should never be overridden.
Advantages:
Clarity: looking at the code tells you whether an override could exist.
Testability: writing tests is clearer and easier for overridden commands because they have their own unique name
Discoverability: all overridden commands can be discovered by searching for aliases with the right name pattern i.e. Get-Alias *-Overridable*
Much less code
All overrides and their aliases can be packaged into modules
I am writing a powershell module with a list of utilties that I use on a daily basis. However, my question is: How can I not repeat so much code?
For example if I have a function that gets a list of hostnames from a file, I have to create that parameter in every single function. How can I just create it once, and then have each function prompt for it, or grab it?
function CopyFiles {
param (
[parameter(Mandatory = $true, HelpMessage = "Enter the Path to the Machine List File (UNC Path or local). ")]
[ValidateScript({$_ -ne ""})]
[string] $MachineListFilename,
...Sometime later in the script...
$MachineList = Get-Content $MachineListFilename
}
function DoSomeOtherTask {
param (
[parameter(Mandatory = $true, HelpMessage = "Enter the Path to the Machine List File (UNC Path or local). ")]
[ValidateScript({$_ -ne ""})]
[string] $MachineListFilename,
...Sometime later in the script...
$MachineList = Get-Content $MachineListFilename
}
It just seems really in-efficient to cut and paste the same code over and over again. Especially for something like, domain-name, username, password, etc.
Ultimately, I'm trying to get to a point to where I just write wrapper scripts for these functions once I import the module. Then I can just pass parameters via the command line. However, with the current way I'm doing it, the module is going to be littered with a lot of repetitive code, like parameters for username and password, etc.
Is there a better way?
Make your cmdlets/functions as independent and flexible as you can. Sometimes a wrapper function is the way to go, other times consolidating things into one function and calling it differently is more workable.
In the example you've given here, give the caller two options - you can pass in the filename for the list of machines, or pass in the list of machines. That way, you can read the file once in the calling script, and pass the array of machine names into each function. This will be much more efficient as you're only reading from disk one time.
I strongly recommend reading up on advanced functions and parametersets to simplify things (you'll need this for my suggestion above).
As for "repetitive code" - as soon as you find yourself copying/pasting code, stop. Find a way to make that code generic and move it into its own function, then call that function wherever it's needed. This isn't a PowerShell notion - this is standard programming, the DRY Principle.
Even then, you'll still find yourself with some modicum of copypasta. It's going to happen just because of the nature of the PowerShell environment. Look at Microsoft's own cmdlets - you'll see evidence of it there too. The key is to minimize it.
Having 3 cmdlets that all take username & password (why not take a Credential object instead/as another option, BTW?) will result in copying & pasting those parameters in the function definition. You're not going to avoid that, and it's not necessarily a bad thing. You can create code snippets in most good editors (PowerShell ISE included) to automatically "generate" it for you if that makes it easier/faster.
I personally like to create intermediary functions that call my functions with specific parameters for things I do a lot of times. I manage these with a switch statement. This way, the backend driver does not change, and I have a nice interface I can give to others who want to use, but not develop on, the code I made.
function frontEnd {
call intermediary(typeA)
}
function intermediary (callType){
switch(callType){
case(typeA):
call backEnd(param1="get dns" param2="domain1" param3=True
case(typeB):
call backEnd(param1="add to dns" param2="domain" param3=False
case(other):
call backEnd(arg1, arg2, arg3)
}
Depending on what functionality you are looking for, this could help you. This is a very crude way of doing it, and I highly suggesting making it more robust and stable if you aren't going to be the only one using it.
I'm coming from a unix background where I've written some scripts using bash and bourne. But I need to write some scripts using powershell and I'm having a hard time finding information.
For example, in *nix, I can do man bash and read all about how to use bash and I can do man some_command to read about a specific command. So far, I found some powershell equivalents like get-command to see all available commands, but getting and using objects is really confusing me.
For instance, I'm trying to create a new scheduled task using powershell and found some sample code here on SO. Here is a snippit:
$schedule = new-object -com Schedule.Service
$schedule.connect()
$tasks = $schedule.getfolder("\").gettasks(0)
$tasks | select Name, LastRunTime
foreach ($t in $tasks) {
foreach ($a in $t.Actions) {
$a.Path
}
}
I understand what this script is doing, but without experience how would I know to do the following:
Know to use new-object -com Schedule.Service
Know that this object has a .connect method
Know that this object has a .getfolder and .gettasks object
A lot of the code seems ambiguous to me, so where would I find out the above information natively using powershell?
So you found Get-Command. That's a good start it will show you the available cmdlets. There may be even more available after importing snapins/modules. Use Get-PSSnapin -Registered and Get-Module -ListAvailable to see additional modules that may be imported to give you even more cmdlets.
The nice thing about PowerShell is that the creators built in an alias system. One of the goals of it was to make it easier to learn PowerShell when you have a bash/DOS background. For example if you type man Get-Process it will give you the documentation for the Get-Process cmdlet. To see all documentation for it use man Get-Process -Full. man doesn't actually exist, it is an alias for Get-Help which has the same functionality as man on UNIX/Linux. You can use the Get-Alias cmdlet to show the registered alias' and their definitions.
The script you found is working with a COM object. You can tell because of the -com parameter that was used for New-Object (which is actually short for -ComObject). Unlike .NET objects, COM objects are not built in to PowerShell however PowerShell has support for them the same way VBScript has support for them. The Get-Member cmdlet will unveil both .NET and COM type object members (properties and methods). More about Get-Member below.
The script you found uses the New-Object cmdlet to create an instance of the COM object named Schedule.Service. There are two main ways to find out more information about this object. The first is that you can list its properties and methods directly within PowerShell using the Get-Member cmdlet. This cmdlet works for both .NET and COM objects. It is an invaluable cmdlet that will show you what you can do with your objects. Use man or Get-Help Get-Member to learn about it. In fact you can use Get-Member to discover the object members you asked about such as the .connect method. The second way is to look up the documentation for the object on MSDN which is Microsoft's developer documentation website. This is probably the best page for that particular object.
I am not familiar with powershell scripting but found this, maybe some reference to use:
http://technet.microsoft.com/eng-us/scriptcenter/powershell%28en-us%29.aspx
http://technet.microsoft.com/en-us/library/hh857339.aspx#BKMK_wps4
On the first link are PowerShell Scripting Webcasts to find and more.
Scheduling Jobs with the Windows PowerShell API: http://msdn.microsoft.com/en-us/library/windows/desktop/jj150476%28v=vs.85%29.aspx
Guide to getting started with Windows PowerShell: http://technet.microsoft.com/library/ee221100.aspx
About Windows PowerShell, following help topics:
get-command : Gets information about cmdlets from the cmdlet code.
get-member : Gets the properties and methods of an object.
where-object : Filters object properties.
about_object : Explains the use of objects in Windows PowerShell.
about_remote : Tells how to run commands on remote computers.
Conceptual help files are named "about_", such as:
about_regular_expression.
The help commands also display the aliases of the cmdlets. These
are alternate names or nicknames that are often easier to type.
For example, the alias for the Invoke-Command cmdlet is "remote".
To get the aliases, type:
get-alias
Hopefully this will help a little.
The first hit on Google for "powershell create scheduled task" leads here, where one of the answers refers to the Schedule.Service COM object. That object's documentation gives you a list of all the methods and properties of the object.
You can also use get-member to discover all the methods & properties of any variable or object in your session.
$schedule = new-object -com Schedule.Service
TypeName: System.__ComObject#{2faba4c7-4da9-4013-9697-20cc3fd40f85}
Name MemberType Definition
---- ---------- ----------
Connect Method void Connect (Variant, Variant, Variant, Variant)
GetFolder Method ITaskFolder GetFolder (string)
GetRunningTasks Method IRunningTaskCollection GetRunningTasks (int)
NewTask Method ITaskDefinition NewTask (uint)
Connected Property bool Connected () {get}
ConnectedDomain Property string ConnectedDomain () {get}
ConnectedUser Property string ConnectedUser () {get}
HighestVersion Property uint HighestVersion () {get}
TargetServer Property string TargetServer () {get}
The Component Object Model is a core piece of Windows and there are hundreds if not thousands of COM objects available in default Windows installation for interacting with both the OS and other software installed (software can install its own set of objects as well). A lot of it can be replaced with .NET Framework assemblies and PowerShell modules, snap-ins and cmdlets now.
How do you discover COM objects? Usually via Google - running searches for the things you're trying to do, and typically you'll find someone has already posted something about similar, or your search will key off words in the object's own documentation online.
If you're using PowerShell 3, you don't need to use Schedule.Service at all - there's a set of cmdlets for working with scheduled tasks. See New-ScheduledTask for a starter.
If you're looking for a generic PowerShell tutorial, I usually point people at this one
You're on the right track in that Get-Command *foo* will list all Cmdlets containing the word foo, and Get-Help New-Object will show you the help file for the New-Object cmdlet.
However, you then go straight into using COM objects, which far predate Powershell. COM programming is old and can be quite archaic. Powershell lets you interface with COM, but it's not really the "Powershell way" of doing things.
In Powershell 3, I was able to find Register-ScheduledJob:
The Register-ScheduledJob cmdlet creates scheduled jobs on the local computer.
If possible I would say that is the preferred approach over using the COM interface, just because it's likely easier and more Powershelley.