I am running a Custom Activity in Azure Data Factory. We are using the Test-Json command to validate JSON data against a schema. This command works perfectly find in PowerShell version 7+ but will fail in version 5.1.
For testing purposes, if I run the below command (directly from the Microsoft documentation) it fails when using the Custom Activity in Data Factory.
"{'name': 'Ashley', 'age': 25}" | Test-Json
If I run the below command, it show that we are using PowerShell version 5.1.
$PSVersionTable
Is there anyway to upgrade to version 7.1 so these batch file will process successfully?
Two potential options:
Modify your task to use ConvertFrom-Json with a try-catch block instead of Test-Json, as described here.
As you've already pointed out, Test-Json comes with PowerShell 7.1 (part of PowerShell Core), but it doesn't come with PowerShell 5.1 (part of Windows PowerShell), which many Windows images have installed by default. ConvertFrom-Json, however, does come with PowerShell 5.1 and can provide the same functionality as Test-Json with a few tweaks.
Install PowerShell Core 7.1 as a start task, and then invoke your command as normal.
The start task syntax to complete this installation would be cmd /c "powershell.exe -c "iex ""&" { $(irm https://aka.ms/install-powershell.ps1) } -UseMSI -Quiet""", based off the one-liner described here and modified to fit the Batch syntax described here. This will install the latest version of PowerShell Core, which will have Test-Json included. When creating tasks, be sure to invoke PowerShell Core using pwsh.exe, and not powershell.exe, which is for Windows PowerShell.
I am using PowerShell and PowerShell Core in parallel.
Not all my scripts which I have wrote for the "big" PowerShell are working in PowerShell Core and vice versa.
To avoid chaos, I was thinking if I should use two file-extensions:
.ps1: PowerShell
.pwsh: PowerShell Core
In Windows the extension is more important then in Unix systems where the shebang (#!/bin/mytool) would help to solve the problem.
Is the usage of two extension "best practice" or there are better options?
Additional: I am not sure if a .pwsh script will be executed by PowerShell when I call a script from command line / terminal.
I found a similar question in Unix / Linux context.
https://unix.stackexchange.com/questions/182882/use-sh-or-bash-extension-for-bash-scripts
Add a line
#requires -PSEdition Core
at the beginning of your scripts for PowerShell Core, and a line
#requires -PSEdition Desktop
at the beginning of your scripts for regular PowerShell.
For scripts that run in both editions just omit the line.
From the documentation:
-PSEdition <PSEdition-Name>
Specifies a PowerShell edition that the script requires. Valid values are Core for PowerShell Core and Desktop for Windows PowerShell.
Do not use different extensions, as that will impact functionality. For instance, PowerShell will not dot-source scripts with an extension other than .ps1.
There are several features of Windows Powershell that are not supported in Powershell Core. Since Powershell Core is being actively developed in favor of Windows Powershell, on a Windows platform can Powershell Core use Windows Powershell modules that have not yet been ported to Powershell Core?
Most notably I am concerned with making sure Powershell Core can work with COM objects and Powershell Remoting (over WinRM).
To answer your specific questions first
PowerShell Core CAN use WinRM remoting for Windows to Windows remoting scenarios. Use it in exactly the same way as Windows PowerShell. WinRM remoting is available from Linux machines to Windows machines but is still a work in progress. if your remoting scenario involves a Linux system you're better off using SSH remoting
You can use COM objects (on Windows systems only) for instance this works:
$xl = New-Object -comobject 'Excel.Application'
$xl.visible = $true
$xlbooks =$xl.workbooks
$wkbk = $xlbooks.Add()
$sheet = $wkbk.WorkSheets.Item(1)
With regard to modules some modules will work under PowerShell core - for instance the networking modules and the storage modules. if its a binary module such as Active Directory then it won't work. At least some of the binary modules, including AD, are being converted to work with PowerShell core. if you look in the module folder and it has CDXML files it should work under PowerShell core.
You could also use implicit remoting to get a module to work. Create a powershell remoting session to the local machine and import the module through the remoting session.
PowerShell core is still very much a work in progress and the number of modules that work with it will increase with time. If in doubt you'll have to test
I am working on a number of Powershell cmdlets that form a relatively complex module. I have done so in the past, but this time a number of thing are changed...
I can not get a breakpoint to work. SOmehow VS does not seem to autoamtically want to connect to Powershell or Powershell Core runtime environments.
The Module is a class library that is made using the netstnadard2.0
Executable is set as
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
Arguments are
-NoLogo -NoExit -Command "Import-Module '.\Cli.dll'"
All works nice - Powershell windows opens, but breakpoints do not load. Period. Executing a Cmdlet also does not load them. I am attached to SOMETHING because i offers "detach from process".
The situation. On a Windows 7 SP1 machine, I have updated with Windows6.1-KB2819745-x64-MultiPkg.msu. Furthermore, in PowerShell $PSVersionTable now reports ‘PSVersion 4.0’.
At present, my conclusion is that many PowerShell 4 cmdlets such Test-NetConnection, will only work on Windows 8.1. However, I was wondering if there was a work-around whereby I could import PowerShell 4 modules on my Windows 7 machine.
You cannot. They rely on the underlying features of the newer OS (8.0 or 8.1) and cannot be ported back to Windows 7 . The alternative is to write your own functions / modules to replicate the new cmdlets using .NET framework methods.
For instance, the Get-FileHash cmdlet is a one-liner in PowerShell 4.0, but to replicate in 2.0 we have to use .NET.
PowerShell v4
Get-FileHash -Algorithm SHA1 "C:\Windows\explorer.exe"
PowerShell v2
$SHA1 = new-object -TypeName System.Security.Cryptography.SHA1CryptoServiceProvider
$file = [System.IO.File]::Open("C:\Windows\explorer.exe",[System.IO.Filemode]::Open, [System.IO.FileAccess]::Read)
[System.BitConverter]::ToString($SHA1.ComputeHash($file)) -replace "-",""
$file.Close()
At least Test-NetConnection can be ported back to Windows 7. Just copy folders NetTCPIP, DnsClient, and NetSecurity from the supported Windows machine with the same PowerShell version (Windows 8.1, Windows 10, etc). Folder - C:\Windows\System32\WindowsPowerShell\v1.0\Modules. Then Import-Module -Name C:\Windows\System32\WindowsPowerShell\v1.0\Modules\NetTCPIP -Verbose
Alternatively, you can import a module from a remote machine (say win2012):
$rsession = New-PSSession -ComputerName win2012
Import-Module NetTCPIP -PSSession $rsession
I have had the same problem on my Windows 7 x64 and both solutions worked for me as of PowerShell 5.1.
Adding to Anton Krouglov's answer. PowerShell modules are cross-platform compatible. So a module copied from Windows Server 2012 R2 x64 can be imported to Windows 7 x86, and even if you are running as standard user without rights to copy them to C:\Windows\System32\WindowsPowerShell\v1.0\Modules you can copy it to any local folder, and run.
Assuming you copied the NetTCPIP, DnsClient, and NetSecurity modules from a Windows Server 2012 or higher machine, and save them to a folder you can import them using
Get-ChildItem -Directory .\psmodules | foreach { Import-Module -Name $_.FullName -Verbose}
Test-NetConnection -InformationLevel "Detailed"
As far as I know, Windows Server 2008 R2/Windows 7 simply doesn't have the counters that the .NET methods use to implement get-netstuff.
A new PowerShell version can implement hash compare, etc. since this is not related to anything, just a piece of code. But if you want to use, for example, Get-NetTCPConnection there is nothing to show.
I see several responses which assert portability, and my testing confirms their assertions:
Import all of the required modules, either from file, or via a PSSession to a host which has the required modules.
The architecture of PowerShell Console (x86 or x64) you run will determine which module architecture you import.
For those who are:
still unable to make this work
AND
do need a reliable TCP test, but may not need everything else provided by Test-NetConnection
AND
Need it all to work even on PowerShell v2.0
You may wish to try this.
# Create a TCP Client using .Net & attempt connection to target
$TCPClient = New-Object .net.sockets.tcpclient("[IP address or hostname]",[port])
# At the above point you may see familiar-looking Windows error messages in
# red text. (You may want to use "try/catch" if you intend to loop).
# Otherwise, check your new object's status to see if it worked
$TCPClient.Connected
# The response is either "True" or False"
# Now to avoid leaving idle connections, run:
$TCPClient.Close()
Naturally, it should be possible to create a loop which tests multiple connections, and outputs the results by selecting the properties of the $TCPClient.
My initial testing shows you would want to Select these properties
The address you tested
$TCPClient.Client.RemoteEndPoint.Address.IPAddressToString
The port you tested
$TCPClient.Client.RemoteEndPoint.Port
The result
$TCPClient.Connected
HIH
While PowerShell 4.0 is available on Windows 7, as Knuckle-Dragger states certain features rely on newer operating system functionality. Unfortunately Test-NetConnection is not available in Windows 7 as stated in the documentation.
Test-Connection, which is present, is basically ping. Test-NetConnection offers much more functionality, allowing a choice of things such as TCP ports, protocols, route tracing, and information levels.
There is a Send-Ping script available from the ScriptCenter in the TechNet gallery, but I think this is only really useful if you are stuck on PowerShell 3.0 for some reason.
I can only assume you installed the wrong package. Make sure you download the proper package from here.
Below you will see in running Windows 7 Service Pack 1 with PowerShell 4 using Test-Connection and Get-FileHash: