How to load PowerShell Module from custom script on vNext build agent? - powershell

I am using the standard TFS vNext build step to execute a PowerShell script. Inside the script I am trying to take advantage of some of the functions within the standard TFS Agent modules.
Listed here:
http://blog.majcica.com/2015/11/14/available-modules-for-tfs-2015-build-tasks/
I have seen the following two lines in many PowerShell scripts found in the build steps:
Import-Module "Microsoft.TeamFoundation.DistributedTask.Task.Internal"
Import-Module "Microsoft.TeamFoundation.DistributedTask.Task.Common"
I have tried to use the same lines in my script, however I get the error:
VERBOSE: Loading module from path
'C:\TFS2015-Agent\Agent1\agent\agent\worker\Modules\Microsoft.TeamFoundation.DistributedTask.Task.Common\Microsoft.TeamFoundation.DistributedTask.Task.Common.dll'.
Import-Module : Could not load file or assembly 'Microsoft.TeamFoundation.DistributedTask.Agent.Interfaces, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
The system cannot find the file specified.
At C:\tfsVnBw1\3\s\Configuration\BuildScripts\CommonFunctions.ps1:25 char:5
+ Import-Module "Microsoft.TeamFoundation.DistributedTask.Task.Common" - Error ... + CategoryInfo : NotSpecified: (:) [Import-Module], FileNotFoundException + FullyQualifiedErrorId : System.IO.FileNotFoundException,Microsoft.PowerShell.Commands.ImportModuleCommand
If I try not to import, it writes something like:
The 'Find-Files' command was found in the module 'Microsoft.TeamFoundation.DistributedTask.Task.Common',
but the module could not be loaded. For more information, run 'Import-Module Microsoft.TeamFoundation.DistributedTask.Task.Common'.
At C:\tfsVnBw1\3\s\Configuration\BuildScripts\CommonFunctions.ps1:71 char:16
+ $files = #(FindFiles $filePattern)
+ ~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (Find-Files:String) [FindFiles], CommandNotFoundException + FullyQualifiedErrorId : CouldNotAutoloadMatchingModule
Is it not possible to use the Modules from 'normal' PowerShell scripts and not only from PowerShell scripts registered as an actual build step?

Try to specify build agent’s module path, such as:
# Import the Task.Common and Task.Internal dll that has all the cmdlets we need for Build
$agentWorkerModulesPathRoot = "$($env:AGENT_HOMEDIRECTORY)\agent\worker"
$agentDistributedTaskInterfacesModulePath = "$agentWorkerModulesPathRoot\Microsoft.TeamFoundation.DistributedTask.Agent.Interfaces.dll"
$agentWorkerModulesPath = "$($env:AGENT_HOMEDIRECTORY)\agent\worker\Modules"
$agentDistributedTaskCommonModulePath = "$agentWorkerModulesPath\Microsoft.TeamFoundation.DistributedTask.Task.Common\Microsoft.TeamFoundation.DistributedTask.Task.Common.dll"
Write-Host "Importing VSTS Module $agentDistributedTaskInterfacesModulePath"
Import-Module $agentDistributedTaskInterfacesModulePath
Write-Host "Importing VSTS Module $agentDistributedTaskCommonModulePath"
Import-Module $agentDistributedTaskCommonModulePath

Is it not possible to use the Modules from 'normal' PowerShell scripts and not only from PowerShell scripts registered as an actual build step?
It is possible to have a PowerShell script in build step reference a module. I prefer to use the inline PowerShell task as it means I don't have to go through the rigmarole of gated check-in just to change the way the code is packaged. I use it quite often to do things like...
download a common assembly versioning ps1 script from a shared project in TFVC and execute it to stamp assemblies, nuget packages, etc with the build number.
change the build number
Save-Module for a module in a private PS repository, and include it in an artifact (install helpers)
Save-Module again to a temp folder, then import the module and use it to do something to the code being built.
Generate release notes or similar docs
... but you can use the regular "Run a PowerShell script" to do all these too.
Perhaps simpler though is to put the module in the path and update PowerShell if it's < 5.1, If you have PowerShell 3 or later on the build server, and the module is already installed on the server and in the agent account's $PSModulePath, then you don't even need to Import-Module, you can invoke the commands in that module directly.

Related

How to resolve 'failoverclusters' was not loaded PowerShell script error

I am trying to run the Batch file which runs the SSIS package. The SSIS package call the PowerShell script to fetch the data from different servers. I am able to get the SSIS package working without any error. But whenever I try to run the batch file which calls the SSIS package, it fails with below error:
import-module : The specified module 'failoverclusters' was not loaded because no valid module file was found in any
module directory.
At E:\XYZ\SSISPckg\Hypv\BASE\XYZ70GTNPENG003.ps1:6 char:1
+ import-module failoverclusters
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ResourceUnavailable: (failoverclusters:String) [Import-Module], FileNotFoundException
+ FullyQualifiedErrorId : Modules_ModuleNotFound,Microsoft.PowerShell.Commands.ImportModuleCommand
The term 'Get-ClusterGroup' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
no vms in XYZ70GTNPENG003
Microsoft (R) SQL Server Execute Package Utility
Version 15.0.4223.1 for 32-bit
Copyright (C) 2019 Microsoft. All rights reserved.
I did not get such error when I executed the PowerShell script from SSIS.
I was able to identify FailoverClusters module in powershell folder. I was able to locate it by running Get-Module -ListAvailable command.
I have already executed Install-WindowsFeature "RSAT-Clustering-Powershell" to install the required module. Except this module, I was able to import the other Hyper-V module.
So, can someone please help with understanding where I am making mistake. I am able to run the SSIS package which calls the powershell script using Execute Process Task. But if I call the batch file which run the SSIS package, it fails with above error.
The issue is that the FailoverClusters module is only available from the x64 bit version of PowerShell. e.g. when you are opening PowerShell by yourself, it by default, opens the x64 bit version of PowerShell, and hence, is why your testing works. A 32 bit x86 version of the FailoverClusters module does not exist, and can't be run from a 32 bit environment.
You can see this is the case when you launch Windows PowerShell (x86) and try to load the module:
PS C:\> Import-Module failoverclusters
Import-Module : The specified module 'failoverclusters' was not loaded because no valid module file was found in any
module directory.
At line:1 char:1
+ Import-Module failoverclusters
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ResourceUnavailable: (failoverclusters:String) [Import-Module], FileNotFoundException
+ FullyQualifiedErrorId : Modules_ModuleNotFound,Microsoft.PowerShell.Commands.ImportModuleCommand
When running SSIS packages from a .bat file, it defaults to the x86 version, which will launch the 32-bit x86 version of PowerShell, and hence, will throw the error message.
What you have to do is in your .bat file, explicitly launch the SSIS package with the x64 version of DTExec.exe something like this:
& "C:\Program Files\Microsoft SQL Server\150\DTS\Binn\DTExec.exe" /FILE MyFile.dtsx
This will launch the x64 bit version of DTExec.exe which will in turn launch the x64 version of PowerShell, and make the FailoverClusters module available.

Automate Azure Data Factory triggers using release pipeline by using powershell task

I am new on DevOps, i am trying to change Azure Data Factory triggers starttime and endtime properties in azure data factory at deployment time using powershell.
I found this link and tried to follow in PowerShell. I am getting following error when i am running below command.
PS C:\> Set-AzDataFactoryV2Trigger -ResourceGroupName "ADF" -DataFactoryName "WikiADF" -Name "ScheduledTrigger" -DefinitionFile ".\scheduledTrigger.json"
Set-AzureRmDataFactoryV2 : The term 'Set-AzureRmDataFactoryV2' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:16
+ $DataFactory = Set-AzureRmDataFactoryV2 -ResourceGroupName $ResGrp.Re ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (Set-AzureRmDataFactoryV2:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
I searched about this issue and found this issue as a bug in Microsoft official github account
If anyone able to resolved this issue or have already resolved, please help me.
Note: Azure DataFactory Triggers tag is not available in StackOverflow, so i am not able to add it.
Very agreed with Nick's opinion:
Any time youe see The term 'xyz' is not recognized as the name of a
cmdlet in powershell, it means you haven't installed the module.
Actually, in your issue, the error also caused by this reason. I think you should haven't installed the Az module for your Powershell.
The Az module is a rollup module for the Azure PowerShell cmdlets. Installing it can download all of the available Azure Resource Manager modules, include the Set-AzDataFactoryV2Trigger you are using, and makes their cmdlets available for use.
Try this command to install the Az module for your Powershell:
Install-Module -Name Az -AllowClobber -Scope CurrentUser
Since as default, the PowerShell gallery isn't configured as a trusted repository for PowerShellGet, the first time you use the PSGallery you would see the prompt which need you ensure if you want to get the Az module from PSGallery. Just answer Yes or Yes to All is ok.
Then, you can execute the following command to check the Az module:
Get-InstalledModule -Name Az
When you see the following message, it means the Az module has exists in the Powershell:
Now, try this link again, it would be succeed.
You seemed to use PowerShell task instead of Azure PowerShell task (https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-powershell?view=azure-devops) in your release pipeline.
You won't have any issue if using Azure PowerShell task.

How do I import a PowerShell module for use by a pipeline task

I'm creating a Build Pipeline in Azure DevOps. I have a PowerShell task that invokes a script inside a file - it's not "inline" PowerShell. That script needs the Az.Accounts module so I added Import-Module Az.Accounts. When I run the pipeline, I get the following:
Import-Module : The specified module 'Az.Accounts' was not loaded because no valid module file was found in any module
directory.
At D:\a\1\s\XXX\XXX\XXX.ps1:14 char:1
+ Import-Module Az.Accounts
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ResourceUnavailable: (Az.Accounts:String) [Import-Module], FileNotFoundException
+ FullyQualifiedErrorId : Modules_ModuleNotFound,Microsoft.PowerShell.Commands.ImportModuleCommand
How do I add the missing module so my script can run?
By default, the hosted windows agent just include the AzureRM module, which is older.With the Az module, Azure PowerShell is now compatible with PowerShell 5.1 on Windows and PowerShell Core 6.x and later on all supported platforms - including Windows, macOS, and Linux.It is the biggest and most important change.
You can add an inline powershell task for installing the Az.Accounts module.
Please note that if you don't add the -force parameter in the script, you will get this information in process.
User declined to install module (Az.Accounts).
The hosted agents don't have the Az modules installed -- they have the older AzureRM modules.
You should be able to use Install-Module to install them. Or you can rewrite your scripts to use AzureRM.
When using Azure Devops with the hosted windows-2019 image, I recommend you use the "AzurePowerShell" task instead of the plain "PowerShell" task, with a version #4 or higher.
documentation of this task can be found here.. Scroll down to see the options related to versioning.
Syntax and arguments are almost identical to the PowerShell task, except that you no longer need to login in Azure (you provide a service connection in the parameters).
You don't need to do crazy stuff like installing the Az module or uninstalling AzureRM. If you use the AzurePowerShell task, it's available for you.
note: don't use Az commands and AzureMR (deprecated) commands in the same script.

Powershell AzureML Get-AmlWorkspace

Get-AmlWorkspace : One or more errors occurred.
At line:1 char:1
+ Get-AmlWorkspace
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Get-AmlWorkspace],
AggregateException
+ FullyQualifiedErrorId :
System.AggregateException,AzureML.PowerShell.GetWorkspace
I am trying to use Powershell to connect to Azure ML studio as it looks like an easier way to manage a workspace. I've downloaded the dll file from https://github.com/hning86/azuremlps and changed my config.json file, but get the error above if I try to run any AzureML commands. I've unblocked the DLL file and imported the AzureMLPS module, and I can see the module and commands I am trying to use have been imported by doing Get-Module and Get-Command
For info I've not used Powershell before.
Any suggestions much appreciated!
Have you installed Azure PowerShell Installer on your local machine?
Click here for more info.
Download the latest Azure PowerShell Installer (4.3.1), then install on your local machine. Then retry using Azure PowerShell module and commands.
I installed mine last May, using Azure PowerShell 4.0.1, and the command Get-AmlWorkspace is working.
# Set local folder location
Set-Location -Path "C:\Insert here the location of AzureMLPS.dll"
# Unblock and import Azure Powershell Module (leverages config.json file)
Unblock-File .\AzureMLPS.dll
Import-Module .\AzureMLPS.dll
# Get Azure ML Workspace info
Get-AmlWorkspace
The output on my side looks like this:

Error Running Powershell Script Using PSCX

We're running a Team City build server which already executes this script locally with no problem. However, when I try to run this script manually I have the following problem:
Windows PowerShell
Copyright (C) 2012 Microsoft Corporation. All rights reserved.
PowerShell Community Extensions Imported.
PS C:\Users\user.domain> cd D:\pkg\platform-2b0c7e3f71f9-BUILD01; powershell.exe -noprofile -executionpolicy Bypass -file D:\pkg\platform-2b0c7e3f71f9-BUILD01\stage.ps1 production \\192.168.x.x\staging \\testserver.domain.com\staging
D:\pkg\platform-2b0c7e3f71f9-BUILD01
Start Staging: 12/15/2014 11:18:14
Verifying target environment configuration and staging targets
Copying source and environment switch
...
Verifying configuration files are valid XML files
Test-Xml : The 'Test-Xml' command was found in the module 'Pscx', but the module could not be loaded. For more information, run 'Import-Module Pscx'.
At D:\pkg\platform-2b0c7e3f71f9-BUILD01\stage.ps1:72 char:9
+ if (!(Test-Xml $file))
+ ~~~~~~~~
+ CategoryInfo : ObjectNotFound: (Test-Xml:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CouldNotAutoloadMatchingModule
So I see the PowerShell Community Extenstions are getting imported (line 4) which makes me wonder what is going on. I am running the script as the exact same user that runs it under Team City. I've checked the build step I'm trying to replicate:
and I can't think of anything that I've missed (working directory option is set to the script file's directory). Any ideas would be appreciated...
Line 4 loads Pscx, Line 5 opens up a new powershell session with -NoProfile option, so it's unlikely that Pscx is loaded in that Powershell session.