Powershell Remove-Item Cmdlet error - powershell

Question

I'm using powershell to remove a directory with the command:
Remove-Item $pathAsString -Recurse -Force
However, it gives me the following error:
Remove-Item : Cannot remove the item at 'C:\Path' because it is in use.
At line:1 char:1
+ Remove-Item "C:\Path" -Recurse
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [Remove-Item], PSInvalidOperationException
+ FullyQualifiedErrorId : InvalidOperation,Microsoft.PowerShell.Commands.RemoveItemCommand
I thought that was odd, because the directory shouldn't be in use. I can go into file explorer and delete the directory manually. I'm not really sure why this behavior is occurring. I'm fairly new to powershell, so don't understand exactly how it behaves.
My scripts interaction with the program includes:
Creating the folder
Downloading an MSI to the folder
Setting a variable to represent the MSI stored in the folder. Like so:
$MSIVariable = Get-ChildItem $Path | Where {&_.Name -eq "MSIName.msi"}
I'm assuming that something to do with the folder is within a stream of some sort, but don't know how I'd fix this issue.
EDIT:
Here is the code I use involving the folder:
Creating the Folder:
if(!(Test-Path $MSILocation))
{
New-Item $MSILocation -ItemType directory
}
Downloading the MSI:
$webClient = (New-Object System.Net.WebClient)
$downloadLocation = Join-Path $MSILocation $MSIName
$webClient.DownloadFile($downloadURL, $downloadLocation)