I am using Azure DevOps to deploy PowerShell modules to a server. This release task deploys the modules to the directory C:\Windows\System32\WindowsPowerShell\v1.0\Modules\. I am able to use the modules once they are deployed to this folder successfully.
If I modify one of the modules and re-release it the file in C:\Windows\System32\WindowsPowerShell\v1.0\Modules\ gets updated, however the old version of the module is still used when running from a batch file using pwsh.
I discovered that the module file also exists in the following paths:
C:\Program Files\PowerShell\Modules\
C:\Program Files\PowerShell\6\Modules\
When deploying the new version using Azure DevOps the old version in the above two directories are not updated. Manually updating the module in those locations fixes the problem.
Why is the module file being copied into those two additional paths?
Should those copies be overwritten when a new version of the module is deployed?
What is the correct way of deploying a module in this scenario?
Powershell uses different paths to load modules. Use $env:PSModulePath -split ";" to know which are the paths being used.
The difference between each path is user scope and usage scope (e.g. made for custom modules or windows official modules).
Now, by default, PS looks for the latest version of each module across all the paths. So maybe the old version is being run because at the time you re-deploy. You are not updating the module version in the Module Manifest, so if PS see they are the "same" version it gets the last one loaded on the PSModulePath.
Take a look at this awesome post for more details: Everything you wanted to know about PowerShell's Module Path
Now to your questions.
Why is the module file being copied into those two additional paths?
This could be a server configuration or the script that you are using to deploy.
Should those copies be overwritten when a new version of the module is deployed?
Not necessarily, if the versions are maintained correctly. On the post shared says how to check the versions of each module.
Related
I have some powershell modules which contain classes I need to instantiate from various powershell scripts. To access the classes from the powershell scripts I am have statements like:
using module "..\..\Library\Mymodule.psm1"
But I dont know ahead of time where in the folder hierarchy my library will be relative to the script.
We dont want to use the standard powershell modules folders because these classes are under source control and deploying to user's folders would be nightmarish.
This is an extremely hard topic to google because, well "using" is used everywhere!
Tried provide multiple possible locations for the same file but of course we get an error for the alternate locations that do not exist:
The specified module 'C:...Mymodule.psm1' was not loaded because no valid module file was found in any module directory.
At line:0 char:0
using module "..\..\Library\Mymodule.psm1"
using module "..\Library\Mymodule.psm1"
using module ".\Library\Mymodule.psm1"
But I'd rather run a function to first determine the correct module path, and then use something like this
using module "$foundModulePath"
Is there any way to dynamically set the path of a module and then "using" it?
I would version my modules in a separate repository and write a function the clones that repository inside the workspace at runtime.
Using tags in that repo - would allow me to control the version of the modules I am loading for each script.
This will make the development of the modules possible, even when a script still has to use the old versions and I don't want to "invest" time in re-writing it , just because I've changed a dependency.
idea no 2.
Gradle is very good in handling messy dependencies lists in big projects.
You can build a gradle script that prepares the workspace and then, from inside of it, execute your powershell script.
See Link here
I've looked at this document, and it lists $Env:ProgramFiles\WindowsPowerShell\Modules (%ProgramFiles%\WindowsPowerShell\Modules) as a good place for installing modules for all users. About naming, it says:
Use the Correct Module Directory Name
A "well-formed" module is a module that is stored in a directory that
has the same name as the base name of at least one file in the module
directory. If a module is not well-formed, Windows PowerShell does not
recognize it as a module.
The "base name" of a file is the name without the file name extension.
In a well-formed module, the name of the directory that contains the
module files must match the base name of at least one file in the
module.
For example, in the sample Fabrikam module, the directory that
contains the module files is named "Fabrikam" and at least one file
has the "Fabrikam" base name. In this case, both Fabrikam.psd1 and
Fabrikam.dll have the "Fabrikam" base name.
C:\Program Files Fabrikam Technologies
Fabrikam Manager
Modules
Fabrikam
Fabrikam.psd1 (module manifest)
Fabrikam.dll (module assembly)
However, I'm still unclear; suppose I have a module composed ABC.psd1 and ABC.psm1. Am I understanding correctly that it should be installed in $Env:ProgramFiles\WindowsPowerShell\Modules\ABC, with the .psd1 and .psm1 directly under that directory?
Looking at some of the microsoft modules, I see, for example:
C:\Program Files
WindowsPowerShell
Modules
AzureRM
5.7.0
AzureRM[.psd1,psm1]
So at least based on the referenced document, it breaks the rule of "well-formed module" since the statement does not seem to allow for that 5.7.0 directory, or does not talk about a recursive search. Is that a hierarchy specifically intended for versioning, is it of the developer's choice? Even for versioning, it doesn't match the recommendations in the above document, for "Using Multiple Versions of a Module" which suggest it is added to the directory name, not as a subdirectory.
The gist of my question is this:
Is there a better explanation on how powershell searches for modules in this directory, with respect to nesting?
Suppose as my product grows, I have a second module DEF.psd1. Should I group under a company or product folder, and will all versions of powershell be happy with this (that is, without me altering the PSModulePath).
e.g.:
C:\Program Files
WindowsPowerShell
Modules
My Company
ABC
ABC[.psd1,psm1]
DEF
DEF[.psd1,psm1]
I am looking to support Powershell 3.0 and later. Also, I am noticing that the powershell ISE module browser uses different rules for searching modules, than powershell itself. For example, using my proposed hierarchy with a company name works fine in powershell, ISE, and user scripts, but breaks the ISE module browser which gives a "cannot find module" error when clicking on "Show details".
Looks like you pretty much have this answered. Not sure why/how the AzureRM module has a versioning folder...but the module was created by Microsoft so maybe they know a loop hole.
In my experience and in the documentation I've read, the proper file structure is what you have in your example, except WITHOUT the 'My Company' folder. How you named the folders and the .psm1/psd1 files (with the same base name) is correct. In fact, for a minimal script based module, all you need is the .psm1 file and it will work (load whatever functions are in the file), although generating a module file with New-ModuleManifest doesn't hurt. As mentioned in the comments, versioning can then be tracked and recognized by PowerShell in the .psd1 file.
Now, I've never put a module in the ProgramFiles folder for all users but I imagine it uses the same rules as the user based folders (they are just paths in an enviroment variable that powershell looks at).
I've also used the link below as a method or template for building PowerShell modules that are easy to maintain:
http://ramblingcookiemonster.github.io/Building-A-PowerShell-Module/
You can ignore the parts you aren't interested in or not using (folders for tests or views or using a Git repo)...the base structure works as well as the technique for loading individual functions with the .psm1 file.
In order to make install scripts and understanding existing ones, I would like to know what happens behind the scenes of the typical:
choco install notepadplusplus
I found the following here:
Installation
Chocolatey uses Nuget.Core to retrieve the package from the source.
Choco determines if it self-contained or has automation scripts - PowerShell scripts (*.ps1 files), and soon to be open to Scriptcs files in the 0.9.10.x timeframe (I know, right?!).
Choco takes a registry snapshot for later comparison.
If there are automation scripts, choco will run those. They can contain whatever you need to do, if they are PowerShell you have the full power of Posh (PowerShell), but you should try to ensure they are compatible with Posh v2+.
Choco compares the snapshot and determines uninstaller information and saves that to a .registry file.
Choco snapshots the folder based on all files that are currently in the package directory.
Choco looks for executable files in the package folder and generates shims into the $env:ChocolateyInstall\bin folder so those items are available on the path. Those could have been embedded into the package or brought down from somewhere (internet, ftp, file folder share, etc) and placed there.
That given,
How can I get the .nupkg package URL? In general it seems like this:
https://chocolatey.org/api/v2/package/package-name
Which is the .nupkg package download directory?
Where is the content of the .nupkg package extracted by default? This is important since chocolateyInstall.ps1 sometime uses Split-Path -Parent $MyInvocation.MyCommand.Definition.
"Scriptcs files in the 0.9.10.x timeframe" is rather cryptic. Can you give some references?
Is Posh v2+ simply short for Powershell or is a specific technology?
There are several executable files in $env:ChocolateyInstall\lib without a link in $env:ChocolateyInstall\bin. For example, the mpv.exe of the mpv player is not linked.
Yes, that download URL seems correct. The download directory is always into the Chocolatey installation folder, then lib\packageName, and this is where contents are extracted to.
Right now, installation scripts are only written in PowerShell. This comment is referring to the ability to write in installation scripts in C#, using the ScriptCS run time. Currently, this isn't yet supported.
Yes, this is just a short way of referring to PowerShell.
In the case of the mpv package, you will notice that there is an mpv.exe.ignore file. The presence of this file in the package prevents a shim being created.
I'm installing a PowerShell module via Octopus Deploy onto a number of different servers. For testing purposes, I went with the guidance of Microsoft's documentation for installing PowerShell Modules.
This worked fine, but as the documentation stated, my changes would be visible only for the current session. That is, if I were to do the following:
$modulePath = [Environment]::GetEnvironmentVariable("PSModulePath", [EnvironmentVariableTarget]::Machine)
# More practically, this would be some logic to install only if not present
$modulePath += ";C:\CustomModules"
[Environment]::SetEnvironmentVariable("PSModulePath", $modulePath, [EnvironmentVariableTarget]::Machine)
When running this installer automatically on tentacle servers, future PowerShell sessions do not appear to see the newly installed modules.
How can I install a PowerShell module in a profile agnostic way so that every PowerShell session started can see it?
PowerShell can only "see" modules installed in one of the directories listed in $env:PSModulePath. Otherwise you'll have to import the module with its full path.
To make a new module visible to all users you basically have two options:
Install the module to the default system-wide module directory (C:\Windows\system32\WindowsPowerShell\v1.0\Modules).
Modify the system environment so that PSModulePath variable already contains your custom module directory (e.g. via a group policy preference).
The latter will only become effective for PowerShell sessions started after the modification was made, though.
This profile applies to all users and all shells.
%windir%\system32\WindowsPowerShell\v1.0\profile.ps1
After taking the steps you spelled out in your question (which I think is the general way to go), I found two ways to get the new module source recognized by Powershell:
Restart the machine. (Works every time.)
Reset the PSModulePath in each open session.
$env:PSModulePath=[Environment]::GetEnvironmentVariable("PSModulePath", "Machine")
I found this was necessary to run in both normal and elevated prompts to get this to work without restarting in each type of prompt. (See also the conversation # Topic: PSModulePath.)
PowerCLI with .NET has some dependencies on dll's that you get only when you install PowerCLI on each machine you want to run.
I have a console app with command line arguments, which when deployed using the usual method doesn't work because of the unmet dependencies...these assemblies are part of the GAC.
Clickonce deployment proved to be useless..it didnt recognize my arguments although I passed them as query params.
Finally, I installed VMware PowerCLI on this remote machine. Then ran the .exe and it worked. Is there a way to avoid installing PowerCLI and be able to include all the dependencies with my exe during deployment?
Depending on where exactly you want to deploy your console application, you may be out of luck. According to this page and the most relevant forum post I could find, the PowerCLI assemblies are not redistributable. Your best bet if you want to distribute this application outside your company is to use the Web Services SDK - a huge pain since you've already developed the app.
Assuming your application is for internal use only and you just want to deploy it on its destination server, you can do the following to reference the assemblies locally:
On the development machine, copy whichever PowerCLI .dlls you reference from the GAC (in %WINDIR%\assembly) to your solutions local directory.
Change your references in the project to point to the local versions of the .dlls.
Open the 'Properties' view for each of the references, and make sure 'Copy Local' is set to True.
Compile and deploy your console application (and it's coresident .dlls) to the target machine, it should reference them in the local directory and run without external dependencies.
Hope that helps!
You could also automate PowerCLI installation with a silent installation one-line Powershell script, if the problem is hiding the installation from the users.
Invoke-Expression ("cmd /c '$powerCLIexeFilePath'/S /VADDLOCAL=ALL /V/qn")