I am working on Containerized CI/CD. I have a web API project developed in .NET Framework 4.6.2. This application enabled docker support.
Expected Working Scenario:
I have to configure CI which needs to build the application as a docker image and push to ECR repository.
I have to deploy the same to ECS as well. As a final, I have to run my SQL (RDS) also as a docker and deploy as part of this.
Currently, I need to build and push the docker image to ECR.
Current Progress:
In order to build my application, I have created a custom docker as the build server. Please find my current docker file used to build the docker image;
# escape=`
FROM microsoft/dotnet-framework:4.7.2-runtime
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
#Install NuGet CLI
ENV NUGET_VERSION 4.4.1
RUN New-Item -Type Directory $Env:ProgramFiles\NuGet; `
Invoke-WebRequest -UseBasicParsing https://dist.nuget.org/win-x86-commandline/v$Env:NUGET_VERSION/nuget.exe -OutFile $Env:ProgramFiles\NuGet\nuget.exe
#Install AWS CLI
RUN Invoke-WebRequest -UseBasicParsing https://s3.amazonaws.com/aws-cli/AWSCLI64PY3.msi -OutFile AWSCLI64PY3.msi; `
Start-Process "msiexec.exe" -ArgumentList '/i', 'AWSCLI64PY3.msi', '/qn', '/norestart' -Wait -NoNewWindow; `
Remove-Item -Force AWSCLI64PY3.msi; `
# Install VS Test Agent
Invoke-WebRequest -UseBasicParsing https://download.visualstudio.microsoft.com/download/pr/12210068/8a386d27295953ee79281fd1f1832e2d/vs_TestAgent.exe -OutFile vs_TestAgent.exe; `
Start-Process vs_TestAgent.exe -ArgumentList '--quiet', '--norestart', '--nocache' -NoNewWindow -Wait; `
Remove-Item -Force vs_TestAgent.exe; `
# Install VS Build Tools
Invoke-WebRequest -UseBasicParsing https://download.visualstudio.microsoft.com/download/pr/12210059/e64d79b40219aea618ce2fe10ebd5f0d/vs_BuildTools.exe -OutFile vs_BuildTools.exe; `
# Installer won't detect DOTNET_SKIP_FIRST_TIME_EXPERIENCE if ENV is used, must use setx /M
setx /M DOTNET_SKIP_FIRST_TIME_EXPERIENCE 1; `
Start-Process vs_BuildTools.exe -ArgumentList '--add', 'Microsoft.VisualStudio.Workload.MSBuildTools', '--add', 'Microsoft.VisualStudio.Workload.NetCoreBuildTools', '--add', 'Microsoft.VisualStudio.Workload.WebBuildTools;includeRecommended', '--quiet', '--norestart', '--nocache' -NoNewWindow -Wait; `
Remove-Item -Force vs_buildtools.exe; `
Remove-Item -Force -Recurse \"${Env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\"; `
Remove-Item -Force -Recurse ${Env:TEMP}\*; `
Remove-Item -Force -Recurse \"${Env:ProgramData}\Package Cache\"
# Set PATH in one layer to keep image size down.
RUN setx /M PATH $(${Env:PATH} `
+ \";${Env:ProgramFiles}\NuGet\" `
+ \";${Env:ProgramFiles(x86)}\Microsoft Visual Studio\2017\TestAgent\Common7\IDE\CommonExtensions\Microsoft\TestWindow\" `
+ \";${Env:ProgramFiles(x86)}\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\")
# Install Targeting Packs
RUN #('4.0', '4.5.2', '4.6.2', '4.7.2') `
| %{ `
Invoke-WebRequest -UseBasicParsing https://dotnetbinaries.blob.core.windows.net/referenceassemblies/v${_}.zip -OutFile referenceassemblies.zip; `
Expand-Archive -Force referenceassemblies.zip -DestinationPath \"${Env:ProgramFiles(x86)}\Reference Assemblies\Microsoft\Framework\.NETFramework\"; `
Remove-Item -Force referenceassemblies.zip; `
}
Here I have installed my prerequisites as below;
NuGet
AWS CLI
List item
VS Test Agent
VS Build Tools
Issue / Question:
As the application enabled docker-compose, I can use MSBuild command to build the docker image as specified below;
msbuild "%~dp0SampleDevOps.sln" /p:TargetFrameworkVersion=v4.6.2 /p:Configuration=Release /p:PackageAsSingleFile=false /p:OutDir=%~dp0\artifacts\ docker-compose.dcproj
This commands worked successfully in my local machine (Windows 10). But when I run code build, I got error like below;
C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\Sdks\Microsoft.Docker.Sdk\build\Microsoft.VisualStudio.Docker.Compose.targets(195,5):
error : MSB0001: Internal MSBuild Error: SampleDevOps.sln unexpectedly
not a rooted path
[C:\codebuild\tmp\output\src096397699\src\github.com\xxxx\test1\docker-compose.dcproj]
Here my thought was I need to install docker and docker-compose also in my build server.
Is my understanding, right? Am I going the correct direction?
If yes, please help me to do that from the docker file I poster earlier.
I also confused that, there is not have built-in docker which already contains this AWS CLI and dockers?
Related
Am I faced with such a problem that I need to update Widows 10 to next build (for example from 1903 to 1909) using PowerShell.
I use next methods:
1)
$Updates = Start-WUScan
Install-WUUpdates -Updates $Updates
2)
Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force
Install-Module -Name PSWindowsUpdate -Confirm:$False -Force
Get-Package -Name PSWindowsUpdate
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Unrestricted -Force
$WinUpdates=Get-WindowsUpdate -MicrosoftUpdate -Verbose
Install-WindowsUpdate -KBArticleID $WinUpdates.kb -AcceptAll
Both of these options work great, they find and install updates. BUT! All these updates are for the current build only. Updates to the transition to the next build are not found.
Does anyone know what I'm doing wrong or how to install an update that will allow me to upgrade to the next version of Windows?
I found an article about how you can do a feature upgrade with PowerShell:
$dir = 'C:\_Windows_FU\packages'
mkdir $dir
$webClient = New-Object System.Net.WebClient
$url = 'https://go.microsoft.com/fwlink/?LinkID=799445'
$file = "$($dir)\Win10Upgrade.exe"
$webClient.DownloadFile($url,$file)
Start-Process -FilePath $file -ArgumentList '/quietinstall /skipeula /auto upgrade /copylogs $dir'
I couldn't test it. Here is the source: https://social.technet.microsoft.com/Forums/en-US/51104081-4ed7-4fdd-8b12-5d1f5be532ae/windows-10-feature-update-via-cmd-powershell-or-gpo?forum=win10itprogeneral
using this manual I was able to update the test computer tutorial to get list of windows updates:
Add-WUServiceManager -ServiceID "7971f918-a847-4430-9279-4a52d1efe18d" -AddServiceFlag 7
Get-WUlist -MicrosoftUpdate
But remember in list can be wrong size of updates....
Notice that when installing a build upgrade with PSWindowsUpdate, restart does not yet finalize upgrade. When you have restarted from PowerShell after PSWindowsUpdate has downloaded and installed build upgrade, it is in fact only initialized, not installed.
You must restart once more from Start > Power > Update and restart
Also you can find usefull info here
Background
I have a .NET application that I would like to install inside a Nano Server Docker container, specifically build 1809. The app is basically a REST server which will receive a REST request and depending on the contents of the JSON, invoke a particular PowerShell cmdlet on a particular remote system and return the results in JSON format.
I was able to create a Nano Server Docker container with both PowerShell and .NET Core installed. However, I ultimately realized that there is no WinRM installed on the container and therefore it is impossible to invoke remote PowerShell cmdlets.
My host system is Windows Server 2019 Datacenter, Version 1809 (OS Build 17763.379). I am using Docker Desktop for Windows (Version 2.0.0.3) with Windows containers enabled.
Dockerfile
Here is the Dockerfile I am using. I created it by combining portions of the Dockerfile from here and here.
# escape=`
# Args used by from statements must be defined here:
ARG fromTag=1809
ARG InstallerVersion=nanoserver
ARG InstallerRepo=mcr.microsoft.com/powershell
ARG NanoServerRepo=mcr.microsoft.com/windows/nanoserver
# Use server core as an installer container to extract PowerShell,
# As this is a multi-stage build, this stage will eventually be thrown away
FROM ${InstallerRepo}:$InstallerVersion AS installer-env
# Arguments for installing PowerShell, must be defined in the container they are used
ARG PS_VERSION=6.2.0
ARG PS_PACKAGE_URL=https://github.com/PowerShell/PowerShell/releases/download/v$PS_VERSION/PowerShell-$PS_VERSION-win-x64.zip
SHELL ["pwsh", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
ARG PS_PACKAGE_URL_BASE64
RUN Write-host "Verifying valid Version..."; `
if (!($env:PS_VERSION -match '^\d+\.\d+\.\d+(-\w+(\.\d+)?)?$' )) { `
throw ('PS_Version ({0}) must match the regex "^\d+\.\d+\.\d+(-\w+(\.\d+)?)?$"' -f $env:PS_VERSION) `
} `
$ProgressPreference = 'SilentlyContinue'; `
if($env:PS_PACKAGE_URL_BASE64){ `
Write-host "decoding: $env:PS_PACKAGE_URL_BASE64" ;`
$url = [System.Text.Encoding]::Unicode.GetString([System.Convert]::FromBase64String($env:PS_PACKAGE_URL_BASE64)) `
} else { `
Write-host "using url: $env:PS_PACKAGE_URL" ;`
$url = $env:PS_PACKAGE_URL `
} `
Write-host "downloading: $url"; `
[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12; `
New-Item -ItemType Directory /installer > $null ; `
Invoke-WebRequest -Uri $url -outfile /installer/powershell.zip -verbose; `
Expand-Archive /installer/powershell.zip -DestinationPath \PowerShell
# -------------------------------------------------------------------------------------------------------------------------------------------------------
# Retrieve .NET Core SDK
USER ContainerAdministrator
ENV DOTNET_SDK_VERSION 2.2.401
RUN Invoke-WebRequest -OutFile dotnet.zip https://dotnetcli.blob.core.windows.net/dotnet/Sdk/$Env:DOTNET_SDK_VERSION/dotnet-sdk-$Env:DOTNET_SDK_VERSION-win-x64.zip; `
$dotnet_sha512 = 'ed83eb5606912cd78d7696fbdc8e8074afa95fda84eec57b078d7371848ad15fe91aaf521b85e77c69b844a7b036a2c0b7b6cac87a8e356643980d96b689af93'; `
if ((Get-FileHash dotnet.zip -Algorithm sha512).Hash -ne $dotnet_sha512) { `
Write-Host 'CHECKSUM VERIFICATION FAILED!'; `
exit 1; `
}; `
`
Expand-Archive dotnet.zip -DestinationPath dotnet; `
Remove-Item -Force dotnet.zip
# -------------------------------------------------------------------------------------------------------------------------------------------------------
# Install PowerShell into NanoServer
FROM ${NanoServerRepo}:${fromTag}
# Copy PowerShell Core from the installer container
ENV ProgramFiles="C:\Program Files" `
# set a fixed location for the Module analysis cache
LOCALAPPDATA="C:\Users\ContainerAdministrator\AppData\Local" `
PSModuleAnalysisCachePath="$LOCALAPPDATA\Microsoft\Windows\PowerShell\docker\ModuleAnalysisCache" `
# Persist %PSCORE% ENV variable for user convenience
PSCORE="$ProgramFiles\PowerShell\pwsh.exe" `
# Set the default windows path so we can use it
WindowsPATH="C:\Windows\system32;C:\Windows"
# Set the path
ENV PATH="$WindowsPATH;C:\Program Files\PowerShell;C:\Program Files\dotnet;"
COPY --from=installer-env ["\\PowerShell\\", "$ProgramFiles\\PowerShell"]
# intialize powershell module cache
RUN pwsh `
-NoLogo `
-NoProfile `
-Command " `
$stopTime = (get-date).AddMinutes(15); `
$ErrorActionPreference = 'Stop' ; `
$ProgressPreference = 'SilentlyContinue' ; `
while(!(Test-Path -Path $env:PSModuleAnalysisCachePath)) { `
Write-Host "'Waiting for $env:PSModuleAnalysisCachePath'" ; `
if((get-date) -gt $stopTime) { throw 'timout expired'} `
Start-Sleep -Seconds 6 ; `
}"
# -------------------------------------------------------------------------------------------------------------------------------------------------------
COPY --from=installer-env ["/dotnet", "/Program Files/dotnet"]
# -------------------------------------------------------------------------------------------------------------------------------------------------------
USER ContainerAdministrator
EXPOSE 80/tcp
EXPOSE 5985/tcp
EXPOSE 5986/tcp
EXPOSE 7777/tcp
EXPOSE 7778/tcp
CMD ["pwsh.exe"]
Docker Commands
Here are the Docker commands I am using to create and access the Docker container (note the directory contains a single Dockerfile with the contents above):
docker build C:\powershell-nanoserver1809-with-dotnet-2.2.401
docker create -t --name NanoServerHelloWorld -h NanoServer -i <ID_RETURNED_FROM_PREVIOUS_COMMAND>
docker start -i NanoServerHelloWorld
PowerShell and WinRM commands that fail
On other systems I can use the following PowerShell code to create a CimSession to a remote system and subsequently invoke a PowerShell cmdlet:
$u = "REMOTE_DOMAIN\REMOTE_USERNAME";
$pw = "REMOTE_PASSWORD";
$secStr = New-Object -TypeName System.Security.SecureString;
$pw.ToCharArray() | ForEach-Object {$secStr.AppendChar($_)};
$Cred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $u, $secStr;
$Session = New-CimSession -ComputerName 172.27.0.114 -Authentication Negotiate -Credential $Cred -OperationTimeoutSec 900
But in this container I get this error message:
New-CimSession : FAILED
At line:1 char:12
+ $Session = New-CimSession -ComputerName 172.27.0.114 -Authentication ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [New-CimSession], CimException
+ FullyQualifiedErrorId : Microsoft.Management.Infrastructure.CimException,Microsoft.Management.Infrastructure.CimCmdlets.NewCimSessionCommand
Further, if I try to configure WinRM in anyway, I get the following (from cmd):
C:\>winrm set winrm/config/client #{TrustedHosts="*"}
'winrm' is not recognized as an internal or external command,
operable program or batch file.
Also, if I look at services on the container, I don't see WinRM:
PS C:\> Get-Service
Status Name DisplayName
------ ---- -----------
Running cexecsvc Container Execution Agent
Running CryptSvc Cryptographic Services
Running DcomLaunch DCOM Server Process Launcher
Running Dhcp DHCP Client
Running DiagTrack Connected User Experiences and Teleme…
Running Dnscache DNS Client
Running EventLog Windows Event Log
Stopped KeyIso CNG Key Isolation
Stopped LanmanServer Server
Running LanmanWorkstation Workstation
Stopped lmhosts TCP/IP NetBIOS Helper
Stopped mpssvc Windows Defender Firewall
Stopped Netlogon Netlogon
Stopped NetSetupSvc Network Setup Service
Running nsi Network Store Interface Service
Stopped Power Power
Running ProfSvc User Profile Service
Running RpcEptMapper RPC Endpoint Mapper
Running RpcSs Remote Procedure Call (RPC)
Running SamSs Security Accounts Manager
Running Schedule Task Scheduler
Stopped seclogon Secondary Logon
Running SystemEventsBroker System Events Broker
Running TimeBrokerSvc Time Broker
Get-Service : Service 'TrustedInstaller (TrustedInstaller)' cannot be queried due to the following error:
At line:1 char:1
+ Get-Service
+ ~~~~~~~~~~~
+ CategoryInfo : PermissionDenied: (System.ServiceProcess.ServiceController:ServiceController) [Get-Service], ServiceCommandException
+ FullyQualifiedErrorId : CouldNotGetServiceInfo,Microsoft.PowerShell.Commands.GetServiceCommand
Stopped TrustedInstaller TrustedInstaller
Running UserManager User Manager
Stopped VaultSvc Credential Manager
Stopped WerSvc Windows Error Reporting Service
Stopped WinHttpAutoProxyS… WinHTTP Web Proxy Auto-Discovery Serv…
Stopped wisvc Windows Insider Service
Question
Is there a way to get WinRM installed and working on a Nano Server Docker container build 1809? If not, is there some workaround to connect to remote systems with PowerShell to invoke PowerShell cmdlets?
Perhaps there is some special Docker command I am missing, or some other Nano Server image available which has this missing feature?
Thanks very much in advance.
I am using docker with powershell. I want to download and install webdeploy .msi
I am using the following code:
RUN Invoke-WebRequest http://download.microsoft.com/download/0/1/D/01DC28EA-638C-4A22-A57B-4CEF97755C6C/WebDeploy_x86_en-US.msi -OutFile webdeploy.msi; \
Start-Process msiexec -Wait -ArgumentList /q, /i, -PassThru, webdeploy.msi ; \
Remove-Item -Force webdeploy.msi
When I run:
docker build -t test.
Everything goes fine.
When I then check the folder "Program Files (x86)" inside the container I dont see any folder "IIS".
I also tried:
RUN Invoke-WebRequest http://download.microsoft.com/download/0/1/D/01DC28EA-638C-4A22-A57B-4CEF97755C6C/WebDeploy_x86_en-US.msi -OutFile webdeploy.msi; \
Start-Process msiexec -Wait -ArgumentList /q, /i, webdeploy.msi ; \
Remove-Item -Force webdeploy.msi
Some help would be appreciated.
I'm trying to write a script to download an install Azure Service Fabric SDK, runtime and tools into a few servers.
My issue is that the installer provided here is a Web Installer, and does not support silent mode.
I found a guy that solved this issue here. His code:
# Install Service Fabric Runtime
Invoke-WebRequest "http://download.microsoft.com/download/3/2/1/3217654F-6882-4CEA-BD51-49287EDECE9B/MicrosoftServiceFabric.6.0.232.9494.exe" -OutFile "C:\ServiceFabricRuntime.exe" -UseBasicParsing; \
Start-Process "C:\ServiceFabricRuntime.exe" -ArgumentList '/AcceptEULA', '/QUIET' -NoNewWindow -Wait; \
rm "C:\ServiceFabricRuntime.exe"
# Install Service Fabric SDK
Invoke-WebRequest "http://download.microsoft.com/download/3/2/1/3217654F-6882-4CEA-BD51-49287EDECE9B/MicrosoftServiceFabricSDK.2.8.232.msi" -OutFile "C:\ServiceFabricSDK.msi" -UseBasicParsing; \
Start-Process "msiexec" -ArgumentList '/i', 'C:\ServiceFabricSDK.msi', '/passive', '/quiet', '/norestart', '/qn' -NoNewWindow -Wait; \
rm "C:\ServiceFabricSDK.msi"
As you can see, he's using direct links to the .msi installers (as well as other guys are doing in other threads like these two answers.).
So my questions is, how do i get a direct link to the msi with the latest version of these installers?
And the follow up question would be, is there a universal link that will automatically download the latest version of these tools?
Thanks in advance.
I know this is not exactly what you asked for but you can use Web Platform Installer Command Line to install WebPI products silently. The idea is to download WebPICMD and run installation of Service Fabric SDK from the cmd line. The powershell script can look like this:
Invoke-WebRequest "https://download.microsoft.com/download/C/F/F/CFF3A0B8-99D4-41A2-AE1A-496C08BEB904/WebPlatformInstaller_amd64_en-US.msi" -OutFile "C:\WebPlatformInstaller.msi" -UseBasicParsing;
Start-Process "msiexec" -ArgumentList '/i', 'C:\WebPlatformInstaller.msi', '/passive', '/quiet', '/norestart', '/qn' -NoNewWindow -Wait;
rm "C:\WebPlatformInstaller.msi"
WebPICMD.exe /Install /Products:MicrosoftAzure-ServiceFabric-CoreSDK /AcceptEULA
Product MicrosoftAzure-ServiceFabric-CoreSDK will install latest version of Service Fabric SDK and Service Fabric Runtime silently.
If you want to install something different from the WebPI run :
WebPICMD.exe /List /ListOption:All
This command will list all of the available products, just grab the id of the product and run install command.
More about WebPICMD here.
To add on the answer above, if WebPlatformCMD gives you Windows' UAC consent window issues, you can use PSEXEC tools to run the installer as system account, avoiding that problem.
Example code:
Invoke-WebRequest "https://go.microsoft.com/fwlink/?LinkId=287166" -OutFile "$env:temp\WebPlatformInstaller_amd64_en-US.msi" -UseBasicParsing
Start-Process "msiexec" -ArgumentList "/i $env:temp\WebPlatformInstaller_amd64_en-US.msi /passive /quiet /norestart /qn" -NoNewWindow -Wait
$psToolsPath = "$env:temp\pstools"
New-Item $psToolsPath -ItemType directory -force -erroraction silentlycontinue
Invoke-WebRequest -Uri https://download.sysinternals.com/files/PSTools.zip -OutFile $psToolsPath\PSTools.zip
Expand-Archive "$psToolsPath\PSTools.zip" $psToolsPath -force
cd $psToolsPath
Start-Process psexec64 -ArgumentList "-s /accepteula WebPICMD.exe /Install /Products:MicrosoftAzure-ServiceFabric-CoreSDK /AcceptEULA"
Small note on SteppingRazor's answer above.
You can ease out the ArgumentList Parameter value like this:
Start-Process "msiexec" -ArgumentList "/i C:\WebPlatformInstaller.msi /passive /quiet /norestart /qn -NoNewWindow -Wait
Instead of
Start-Process "msiexec" -ArgumentList '/i', 'C:\WebPlatformInstaller.msi', '/passive', '/quiet', '/norestart', '/qn' -NoNewWindow -Wait;
Then using variables in the string is also easier.
I am trying to script creation of a Windows Server 2012 R2 VM using PowerShell. The VM is to run IIS and I want to deploy ASP .Net web sites to the IIS on the VM using Web Deploy.
So far I have successfully created a Windows Server VM and configured IIS on. The part I am having trouble with is how to script deployment of Web Deploy to the VM. Any ideas?
Install the vm extension check the devpart at the end of this article https://msdn.microsoft.com/en-us/library/azure/dn606311.aspx
Here is a code snippet I use. I launch the custom script extension via arm template to fire off a powershell script
if (!(test-path c:\temp))
{
mkdir c:\temp\
}
#web deploy v3.6
$webdeployURL = 'https://download.microsoft.com/download/0/1/D/01DC28EA-638C-4A22-A57B-4CEF97755C6C/WebDeploy_amd64_en-US.msi'
$webdeployOutFileMSI = 'c:\temp\WebDeploy_amd64_en-US.msi'
wget -uri $webdeployURL -OutFile $webdeployOutFileMSI
$list =
#(
"/I `"$webdeployOutFileMSI`"",
"/passive",
"ADDLOCAL=ALL LISTENURL=http://+:8080/MSWEBDEPLOY/",
"/L*V `"c:\temp\wdlog.log`""
)
if ((get-windowsfeature Web-Mgmt-Service).InstallState -eq 'Installed')
{
Start-Process -FilePath "msiexec" -ArgumentList $list -Wait
}
else
{
Add-WindowsFeature Web-Mgmt-Service
Start-Process -FilePath "msiexec" -ArgumentList $list -Wait
}
Set-ItemProperty -Path HKLM:\SOFTWARE\Microsoft\WebManagement\Server -Name EnableRemoteManagement -Value 1
Restart-Service wmsvc
if ((get-service -Name 'msdepsvc').Status -ne 'Running')
{
net start msdepsvc
}
netsh advfirewall firewall add rule name="Allow Web Management (WMSVC)" dir=in action=allow service="WMSVC"