In our pipelines building .NET apps, we use SonarScanner for Azure DevOps version 4.23.1, using the MSBuild integration. Sometimes, the tasks leave some mess in the .sonarqube directory inside the pipeline's workspace. We use a PowerShell script to delete the folder. Today, it started failing on an agent in a very strange way. It seems the directory's removal is blocked by a file that cannot even be listed be deleted, moved, nor opened. It happens only on the specific agent, but it is 100% reproducible.
The script:
[string]$sqPath = "$(Agent.BuildDirectory)\.sonarqube\"
Write-Host $sqPath
if (Test-Path -Path $sqPath) {
Write-Host "Path exists!"
#Get-ChildItem $sqPath -Recurse -Force # DEBUG
Remove-Item $sqPath -Recurse -Force
} else {
Write-Host "Path doesn't exist."
}
The output:
F:\Agents\01-V2\_work\104\.sonarqube\
Path exists!
Remove-Item : Cannot remove item F:\Agents\01-V2\_work\104\.sonarqube\out\.sonar\mod37\ucfg2\js\F__Agents_01_V2__work_1
04_s_Cmpny_Prod_Plugins_SitePrototypes_Cmpny_Prod_Plugins_SitePrototypes_Importer_Modules_ModelCharacterGallery_carouse
l_init_js_62_9_FD_45_tryToGetLargeYoutubeThumbnail.ucfg: Could not find a part of the path 'F__Agents_01_V2__work_104_s
_Cmpny_Prod_Plugins_SitePrototypes_Cmpny_Prod_Plugins_SitePrototypes_Importer_Modules_ModelCharacterGallery_carousel_in
it_js_62_9_FD_45_tryToGetLargeYoutubeThumbnail.ucfg'.
At F:\Agents\01-V2\_work\_temp\eac8036c-a258-4f5e-b3c0-16ce3b6a00d9.ps1:9 char:5
+ Remove-Item $sqPath -Recurse -Force
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : WriteError: (F__Agents_01_V2...eThumbnail.ucfg:FileInfo) [Remove-Item], DirectoryNotFoun
dException
+ FullyQualifiedErrorId : RemoveFileSystemItemIOError,Microsoft.PowerShell.Commands.RemoveItemCommand
##[error]PowerShell exited with code '1'.
When I uncomment the Get-ChildItem line, the file mentioned by the error message is not listed anywhereI added -Force and it appeared. is among the listed files, in a hidden directory, yet the Remove-Item line still fails with the same error.
The agents are custom Azure VMs running Windows, managed by the infrastructure department of our company. We have little info on what is going on there and we can use them just through the pipelines. When we send a ticket to the infrastructure admins, it takes ages to get any response and much time is wasted.
Is there a way I can diagnose the issue through the pipeline script? I tried different ways of finding the process that holds the handle to the file but all failed. Most strangely, I found that, unlike *nix systems, Windows does not allow the deletion of a file that is open by a process. So is this an issue with rights? Why is Get-ChildItem silent about the file when Remove-Item fails during its deletion? Why does any operation with the file fail? Historically, we had issues with too-long paths, but this time, it seems there is no PathTooLongException.
I suspect that this issue is/was being caused by Windows Maximum Path Length limitations
As detailed in that article:
In the Windows API (with some exceptions discussed in the following paragraphs), the maximum length for a path is MAX_PATH, which is defined as 260 characters. A local path is structured in the following order: drive letter, colon, backslash, name components separated by backslashes, and a terminating null character.
If my counting ability is correct, I see that path length is 265 characters, just above the 260 character limit for a path.
Part of the path looks to me to be randomly generated, so could explain why the issue is not occurring all the time.
When running recent versions of Windows, it's possible to enable NTFS Long Path support which will allow this limit to be bypassed by some applications (namely those that have been built with long path awareness - as far as I can tell PowerShell is one such application)
If you'd like to enable NTFS Long Paths, there's an article you can find here that details the process, and just in case the link gets downed at any point, the key steps are:
Click Windows key and type gpedit.msc, then press the Enter key. This
launches the Local Group Policy Editor.
Navigate to Local Computer Policy > Computer Configuration >
Administrative Templates > System > Filesystem.
Double click Enable NTFS long paths.
Select Enabled, then click OK.
The error messages are misleading, likely because an old PowerShell is used (v1.0, according to the path 😱).
SonarQube somehow managed to create files whose path is longer than MAX_PATH. MAX_PATH is 260 and it includes the trailing NUL character (string terminator). Your path has a length of 261, in this sense, which is just over the limit.
To delete the directory, use this pipeline:
pool:
name: Build
demands: Agent.Name -equals Agent-SKDACLSADOBW01-01-V2
variables:
sq: \\?\F:\Agents\01-V2\_work\104\.sonarqube\
steps:
- checkout: none
- script: rd /s /q $(sq)
It executes the rd (a.k.a. rmdir) command in recursive (/s), quiet (/q) mode on the folder using cmd.exe. This evades many of the idiosyncrasies and bugs of PowerShell. If you need to execute it from within PowerShell, use &cmd.exe /c rd /s /q $(sq). The path needs to be specified as an extended-length path using the \\?\ prefix. Note that when the prefix is used, forward slashes are not converted to backslashes automatically anymore.
In my case, the one-time cleanup described above did not help as the pipeline still produces the files. I modified the script from the question so that it does not have to be modified to run diagnostics and so that it logs a warning and tags the build sonarqube-cleanup each time the cleanup logic is actually used. Based on the tag, I will later be able to find whether the cleanup logic still needs to be there.
if ($env:SYSTEM_DEBUG -eq 'true') {
Write-Host "##[debug]Processes on the agent:"
Get-Process
}
[string]$sqPath = "$(Agent.BuildDirectory)\.sonarqube\"
if (Test-Path -Path $sqPath) {
Write-Host "##vso[task.logissue type=warning]SonarQube folder exists! $sqPath"
Write-Host "##vso[build.addbuildtag]sonarqube-cleanup"
if ($env:SYSTEM_DEBUG -eq 'true') {
Write-Host "##[debug]Contents of the SonarQube folder:"
Get-ChildItem $sqPath -Recurse -Force
}
#Remove-Item $sqPath -Recurse -Force
&cmd.exe /c rd /s /q "\\?\$sqPath"
} else {
Write-Host "SonarQube folder does not exist. $sqPath"
}
I found an interesting GitHub issue about PowerShell support for extended-length paths.
There are many Q & A on Stack Overflow and Super User that detail how files or folders with long paths can be deleted:
Delete directory regardless of 260 char limit
https://superuser.com/q/78434/269404
Related
Sorry it's probably a dumb question: what is the difference between
Remove-Item -recurse -Force -Verbose and Remove-Item -Force -Verbose
it seems if we use -recurse for a folder and subfolders powershell delete file one by one inside the folder. and if we remove the -recurse powershell simply delete the main folder without checking inside the folder. technically isn't script will run faster without -recurse?
tl;dr
Pass both -Recurse and -Force to Remove-Item in order to (more) predictably delete (remove) a given folder, which - barring any permission and timing problems - deletes the folder and any contents it may have.
Caveat: This instantly deletes any contents inside the target folder, and, given that deleted items are not placed in the operating system's recycle bin, can only be recovered, potentially, with specialty software.
it seems if we use -recurse for a folder and subfolders powershell delete file one by one inside the folder.
More accurately, it deletes the target folder's subtree, i.e. it recursively deletes all files and subfolders located in the target folder, including their files and subfolders, recursively, before deleting the target folder itself. In effect, it deletes the target folder and all of its contents, if any, but note the caveats:
re "protected" items, which additionally require -Force, discussed below.
re intermittent failures due to the file-system APIs being asynchronous in older Windows versions, discussed in the bottom section.
In fact, deleting all the contents of a folder before deleting the folder itself is the only way to delete a nonempty folder, technically: the file-system APIs do not offer deletion of a nonempty folder as a single operation.
Because inadvertently deleting an entire subfolder tree can have disastrous consequences, as a safety mechanism PowerShell requires you to signal the intent to delete a nonempty folder explicitly - by passing -Recurse.
If you neglect to pass -Recurse and the target folder is nonempty, you get an interactive confirmation prompt - irrespective of whether -Force is specified or not. Choose [A] Yes to All (type a) to delete the folder and all its contents - but see the situational additional need for -Force below.
That said, you do also need -Force in order to (more) predictably remove a nonempty target folder, because -Force makes PowerShell also delete "protected" files and folders, which are hidden files and folders and files that have the ReadOnly and/or System attributes set (on Windows).
If you neglect to pass -Force when you use -Recurse or interactively choose [A] Yes to All in response to the confirmation prompt, the presence of at least one protected item will prevent removal of the target folder as a whole, though unprotected items inside the subtree will get deleted right away.
Each protected item will cause a non-terminating error to be emitted, concluded by a non-terminating error that the target folder cannot be removed, because it isn't empty (yet). Perhaps confusingly, in Windows PowerShell the per-protected-item error messages only talks about "[in]sufficient access rights", even though the real problem in this case isn't one of permissions; the error message has been amended in _PowerShell (Core) 7+ to explicitly mention hidden, system, and readonly items.
if we remove the -recurse powershell simply delete the main folder without checking inside the folder.
No: It follows from the above that you cannot delete a given nonempty folder unless you delete its contents first.
If you attempt that without -Recurse, you'll invariably get the confirmation prompt (or, in non-interactive scenarios, the call will fail outright).
technically isn't script will run faster without -recurse?
It also follows from the above that only an empty folder can be removed without -Recurse without triggering the confirmation prompt.
If you do also specify -Recurse when targeting an empty folder, hypothetically unnecessary work of testing whether child items exist could be performed. In practice, Remove-Item's implementation always performs this test, whether or not you pass -Recurse.
Even with both -Recurse and -Force specified, overall removal may fail:
... due to insufficient file-system permission held by the current user relative to the target folder and its contents.
... intermittently, due to running on Windows versions older than Windows 10 20H2 (I don't know that Windows Server version that corresponds to), because file-system item deletion there was inherently asynchronous(!), resulting in intermittent failure to fully delete a given target folder, namely if deletion of an item inside the folder hadn't completed yet by the time deletion of the folder itself was attempted: see this answer for details and a workaround.
when you use Remove-Item on a dir tree that holds files ... and do NOT use -Recurse, you will get the standard confirmation prompt. so using that parameter makes it run without the delay from the "do you really want to do this?" prompt.
this is one of the reasons that some folks prefer to pipe the output of Get-ChildItem -Recurse to Remove-Item.
So here is the situation, I am trying to automate the copy of some files that are in a network drive into a local folder on one of my servers. The task seems to be simple and when I try the code with PowerShell or with x copy in the command line both are working pretty great.
I've installed a Jenkins agent on this Windows server 2016 server and run the agent as a service. When I try to run the same code from the Jenkins agent, it is never working.
I tried starting the agent service as local system and as the windows network administrator who has all the right
I tried with PowerShell those lines :
Copy-Item -Path "\\server IP\directory\*" -Destination "D:\Directory\" -Verbose
and
Copy-Item -Path "z:\*" -Destination "D:\Directory\" -Verbose
Both return no error but did not copy the files, and when I tried the same code with x copy I just receive no file found and the file was not copied
xcopy "\\server IP\directory\*" "D:\Directory\" /f /s /h /y
xcopy "z:\*" "D:\Directory\" /f /s /h /y
With PowerShell, I also tried inserting the copy-file command into a script and only calling the script with the Jenkins agent, and it also didn't work
I am now running in a circle and wonder how are we supposed to work with the network drive with the Jenkins agent? Or what I am doing wrong ?
Note that other PowerShell code are working great locally.
I tried starting the agent service as local system and as the windows network administrator who has all the right
Local system doesn't have any network permissions by default. This is the machine account, so you would have to give the machine access to "\\server\share". It is not advisable though, because permissions should be granted on finer granularity. Also, local system has too many local rights, which Jenkins doesn't need.
I don't know what you mean by "Windows Network Administrator". It sounds like this one would also have too many rights.
Usually, one creates a dedicated Jenkins (domain) user. This user will be granted access to any network paths it needs. Then you have two options:
Always run Jenkins service as this user (easiest way).
Run Jenkins under another local user and connect to network drives as the domain user only on demand: net use \\server\share /user:YourDomain\Jenkins <Password>. This adds some security as you don't need to give any local permissions to the domain user.
Both return no error but did not copy the files
To improve error reporting, I suggest you set $ErrorActionPreference = 'Stop' at the beginning of your PowerShell scripts. This way the script will stop execution and show an error as soon as the first error happens. I usually wrap my PS scripts in a try/catch block to also show a script stack trace, which makes it much easier to pinpoint error locations:
$ErrorActionPreference = 'Stop' # Make the script stop at the 1st error
try {
Copy-Item -Path "\\server IP\directory\*" -Destination "D:\Directory\" -Verbose
# Possibly more commands...
# Indicate success to Jenkins
exit 0
}
catch {
Write-Host "SCRIPT ERROR: $($_.Exception)"
if( $_.ScriptStackTrace ) {
Write-Host ( "`t$($_.ScriptStackTrace)" -replace '\r?\n', "`n`t" )
}
# Indicate failure to Jenkins
exit 1
}
In this case the stack trace is not much helpful, but it will come in handy, when you call functions of other scripts, which in turn may call other scripts too.
I have a powershell script which needs to delete the directory passed as argument and the script should be started asynchronously.
Example:
Script_1.ps1
$Path = "C:\XYZ"
$tempDir = "C:\Temp"
Start-Process -FilePath "powershell.exe" -ArgumentList "$($tempDir)\Script_2.ps1","-PathToDelete","$Path"
Script_2.ps1
param(
# The path to delete
[Parameter(Mandatory=$True,ValueFromPipeline=$False,HelpMessage="The path to delete")]
[string]$PathToDelete,
)
Remove-Item -path $PathToDelete -Recurse -Force
exit 0
Currently ,Remove_Item throws exception saying path cannot deleted because it's in use. Process explorer says the path to be deleted is used by powershell process started in script_1.ps.
You say that the process running Script_1.ps1 is preventing the removal, and I'm assuming you've already ruled out the following causes:
You're running that script from a directory located inside the directory tree you're trying to remove.
Inside that script, before calling Script_2.ps1:
you're changing the current directory to a directory located inside the directory tree.
or you've opened a file located in that directory tree without having closed it yet.
This leaves the following potential - and obscure - cause:
The process-level working directory - which is distinct from PowerShell's current location - may be a directory inside the directory tree you're trying to remove, thereby preventing removal.
To rule that out, change it to C:\ from inside Script_2.ps1, before calling Remove-Item, as follows:
[Environment]::CurrentDirectory = 'C:\'
Note:
That PowerShell's working directory (as reflected in the automatic $PWD variable and the return value from Get-Location) differs from the process' is unfortunate, but cannot be avoided, because a single PowerShell process can host multiple runspaces (sessions) simultaneously, each of which must maintain their own working directory (location) - see GitHub issue #3428.
A more common pitfall that results from this discrepancy is owed to the fact that .NET APIs use the process' working directory, which means that you cannot pass relative paths to .NET methods without risking their resolution relative to a different working directory than PowerShell's - see this answer for more information.
I am having trouble with a seemingly simple script I have which essentially copies an item from the user's PC and places it on another computer; however, the destination file path contains a space in it. I have tried multiple methods of correcting this issue (some I don't completely understand) from using double quotation marks around the string to forcing it to run powershell.
To give a very brief precursor to this situation, I must add that I initially created this script using Powershell on Windows 10 and I should also add that it works completely fine in Powershell, just not as a .bat. I understand there may be some differences in the languages or what is interpreted through the programs.
Here is the string in the question:
Copy-Item $ENV:USERPROFILE\Desktop\VAST.accdb -destination "\\PRECDP19670\C$\Users\WAKE\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup" -Force -PassThru -Verbose
The destination contains the space in the filepath.
Any help is appreciated!
As Bill mentioned, your BAT file needs to call PowerShell as an executable.
PowerShell.exe -command 'Copy-Item $ENV:USERPROFILE\Desktop\VAST.accdb -destination "\\PRECDP19670\C$\Users\WAKE\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup" -Force -PassThru -Verbose
I'm using Chef Kitchen to do a POC. We have a PowerShell script which goes to a network path, sorts the build based on date [Install shield installer], copies to the VM in a folder.
This script works correctly when I run inside the VM manually using Power Shell. When I run using Chef Kitchen I don't get any error, the recipe is run but only the first line.I can see the folder called Build created in the VM.
I did an experiment and added a second line to create another folder, it worked correctly. So the issue is the logic I used to copy and paste the build doesn't seem to work with Chef.
The first line creates a new folder, the second and third sets the source and destination path. The fourth line copies the build from the network path [Sort by date to get the latest] to the VM. And the last one is to rename.
And also is there an efficient way to copy a file from a network path [Sorting by date] to a VM using Chef?
powershell_script 'CopyBuild' do
code <<-EOH
New-Item -ItemType directory -Path C:\Build
$source = gci \\BuildPath\Build | ? { $_.PSIsContainer } | sort CreationTime -Desc | select -f 1
$destination = "C:\Build"
Copy-Item "\\BuildPath\Build\$source\*" $destination
Get-ChildItem $destination -r -i "*app1*.exe" | Rename-Item -NewName {"Test.exe"}
EOH
end
I also tried this, calling the PS script:
cookbook_file 'getbuild.ps1' do
mode '0755'
end
This looks like a windows remoting problem, your powershell needs the right configuration to be allowed to access network shares.
To verify if this is the case: replace the UNC path and just copy a local directory - if this works you can dig further to setup the requirements for windows remoting, see the following links for further help:
Safely running windows automation operations that fail inside winrm or powershell remoting (see chapter No access to network resources)
Knife issue 655
enabling and using credssp
To sum these links up: you can schedule a local task in your PS which does the copy to achieve this, or setup credssp.