Powershell BitsTransfer does not complete - powershell

I'm sorry to keep asking about Powershell, my script-foo is not what it needs to be.
I'm writing a BitsTransfer .PS1 to automate the weekly download of an ASCII file.
It never seems to complete and reach a status of "Transferred" and seems stalled in a "Transferring" state. I can see a TMP file in my -Destination folder, with my ASCII data in it.
When I manually download the target file and compare it to the TMP file, they're the same size and appear to have the same first and last records. I assume the download is done.
If I manually run Get-BitsTransfer | Complete-BitsTransfer, the TMP file disappears but still no -Destination file.
My script is nothing sophisticated...
$date= Get-Date -format yyMMdd
$ntispasswd = ConvertTo-SecureString "*******" -AsPlainText -Force
$ntiscreds = New-Object System.Management.Automation.PSCredential ("*******", $ntispasswd)
$jobdescriptor = "DMFWA" + $date
$dmfpath = "C:\DMF"
# -Source https://dmf.ntis.gov/dmldata/weekly/WA$date `
Import-Module BitsTransfer
Start-BitsTransfer `
-DisplayName $jobdescriptor `
-Priority High `
-ProxyUsage Override `
-ProxyList mckwebfilt1:3128 `
-RetryInterval 60 `
-TransferType Download `
-Source https://dmf.ntis.gov/dmldata/weekly/WA130322 `
-Destination $dmfpath\TestWA$date.txt `
-Authentication Basic `
-Credential $ntiscreds `
-Asynchronous
$job = Get-BitsTransfer $displayname
While($Job.Jobstate -ne 'Transferred'){
$job
Start-Sleep -s 1
}
Complete-BitsTransfer $job
Can anybody help me understand what I'm doing wrong?

You did it the right way:
Import-Module BitsTransfer
Start-BitsTransfer -Source $url -Destination $output -Asynchronous
Get-BitsTransfer | Complete-BitsTransfer
Possible failures
the destination paramater is wrong$dmfpath\TestWA$date.txt
there are more than 60 BitTransfers running, end them with Get-BitsTransfer | Remove-BitsTransfer

I used too much time on BitsAdmin trying a transfer a file which was never finished because the file length was not given from the server.
Start-BitsTransfer : Die Dateigröße wurde vom Server nicht zurückgegeben.
Möglicherweise enthält die URL dynamischen Inhalt. Der
Inhaltslängenheader ist in der Server-HTTP-Antwort nicht verfügbar.
In Zeile:1 Zeichen:1
+ Start-BitsTransfer http://***/file c:\users\***\file.txt
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [Start-BitsTransfer],Exception
+ FullyQualifiedErrorId :
StartBitsTransferCOMException,Microsoft.BackgroundIntelligentTransfer.
Management.NewBitsTransferCommand
After trying the same from powershell with Start-BitsTransfer it was the same behavior.
This solution is really awesome and fixed also my problem! Thank you!
$request = New-Object System.Net.Webclient
$passwd = ConvertTo-SecureString "**" -AsPlainText -Force
$request.Credentials = New-Object System.Management.Automation.PSCredential ("**", $passwd)
$request.Downloadstring("https://my full target url")

Look at the last example on this web page about using Start-BitsTransfer and see if that helps.

This turned out to be a "Royal PITA".
Thank you Kieth, for the hint.
I couldn't get BitTransfer to work properly and resorted to something like this..
$request = New-Object System.Net.Webclient
$passwd = ConvertTo-SecureString "**" -AsPlainText -Force
$request.Credentials = New-Object System.Management.Automation.PSCredential ("**", $passwd)
$request.Downloadstring("https://my full target url")

Related

MI_RESULT_FAILED on Copy-Item in PowerShell Script

I am running PowerShell on CentOS 7.x. I converted working individual commands when running within PowerShell via pwsh to a PowerShell script and then it no longer works. Can someone please shed me some light on what I did wrong?
Here's the working individual commands when running within PowerShell via pwsh.
PS /home/user1/Downloads> $userPw = ConvertTo-SecureString -String "user1password" -AsPlainText -Force
PS /home/user1/Downloads> cd
PS /home/user1> $userCredential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList "user1win", $userPw
PS /home/user1> $s = New-PSSession -computerName 192.168.20.143 -credential $userCredential -Authentication Negotiate
PS /home/user1> Copy-Item -Path /home/user1/Downloads/gssntlmssp-0.7.0-1.el7.x86_64.rpm -Destination "C:\users\user1win\Desktop" -ToSession $s
PS /home/user1> exit
Here's the script when I converted to a PowerShell script so I can pass arguments into it. remote-copy.ps
$remoteHost = $args[0]
$username = $args[1]
$pwp = $args[2]
$source = $args[3]
$destination = $args[4]
Write-Host "Remote Host: '$remoteHost'"
Write-Host "Username: '$username'"
Write-Host "Password: '$pwp'"
Write-Host "Source: '$source'"
Write-Host "Destination: '$destination'"
$pw = ConvertTo-SecureString -String $pwp -AsPlainText -Force
$cred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $username, $pw
$s = New-PSSession -computerName $remoteHost -credential $cred -Authentication Negotiate
Copy-Item -Path $source -Destination $destination -ToSession $s
When I run the script, I got the following error.
[user1#rhel7-tm PowerShell]$ pwsh -File ./remote_copy.ps 192.168.20.143 user1win user1password /home/user1/Downloads/vte-0.28.2-10.el7.x86_64.rpm "C:\\users\user1win\Desktop"
Remote Host: '192.168.20.143'
Username: 'user1'
Password: 'user1password'
Source: '/home/user1/Downloads/vte-0.28.2-10.el7.x86_64.rpm'
Destination: 'C:\users\user1win\Desktop'
Copy-Item:
Line |
19 |
Copy-Item -Path "$source" -Destination "$destination" -ToSession $s |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
Starting a command on the remote server failed with the following error message : MI_RESULT_FAILED For more information, see the about_Remote_Troubleshooting Help topic.
Copy-Item: Line |
19 |
Copy-Item -Path "$source" -Destination "$destination" -ToSession $s |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
Failed to copy file /home/user1/Downloads/vte-0.28.2-10.el7.x86_64.rpm to remote target destination.
I would be greatly appreciate if someone can point me to where I did wrong. Thanks!
It's a false alarm. The script does work. The issue is by default the MaxEnvelopeSizekb in winrm/config is only 500kb. I need to set to a bigger value if I want to send a bigger file. To set MaxEnvelopSizekb to 500mb, you need to open PowerShell on Windows as Administrator and run the following command.
PS C:\WINDOWS\system32> Set-WSManInstance -ResourceUri winrm/config -ValueSet #{MaxEnvelopeSizekb = "500000"}

Power Shell File Upload to SFTP with File Existence Check

I have a requirement to upload multiple files to SFTP server from my machine using powershell.i have created the following scripts for the same . the script is working fine . but need to check file existence before upload to SFTP server .
the following are the script i used for upload
$username = "UserName"
$password = "Password" | ConvertTo-SecureString -AsPlainText -Force
$server = "xxxxx"
$credential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList
$username,$password
Get-SFTPSession | Remove-SFTPSession
$SFTPSession=New-SFTPSession -ComputerName $server -Credential $credential -Port xxxx
$FilePath = get-childitem "D:\Test\xx*"
$SftpPath = '/D:/xxx/xxx/Test'
ForEach ($LocalFile in $FilePath)
{
Set-SFTPFile -SessionId $SFTPSession.SessionID -LocalFile $LocalFile.fullname -RemotePath $SftpPath -Overwrite
}
Please any body advice me on to check multiple files existence before uploading and only upload files which is not exist in server .
Thanks in advance
Assuming you are using Posh-SSH, you want to use the Test-SFTPPath cmdlet:
if (Test-SFTPPath -SFTPSession $session -Path $Path)
{
...
}

Getting below error on running PowerShell script

I am using one PowerShell script to download content (a zip file) from a website which requires credentials to get authenticated for the website. Also, have used proxy and proxy credentials since it is an authenticated proxy. I am getting an error which I have pasted below.
I have tried giving the proxy and proxy credentials but no luck.
I have bypassed the execution policy using Powershell -ExecutionPolicy Bypass.
When I try accessing the website from my browser then I am able to access the zip file.
Please find the code snippet below:
#Path to Save data
$tipath = "C:\Somepath"
$localpath = "\somefile.zip"
$timepath = "\Zip_file.txt"
$unzippath = "\Someotherpath"
$storepath = $tipath + $localpath
$remotefilepath = $tipath + $timepath
$unzipfolder = $tipath + $unzippath
# Create Folder If Not Exists #
$FolderToCreate = $tipath
if (!(Test-Path $FolderToCreate -PathType Container)) {
New-Item -ItemType Directory -Force -Path $FolderToCreate
}
#Downloading File #
$username = "username"
$password = "Password" | ConvertTo-SecureString -AsPlainText -Force
$cred = New-Object System.Management.Automation.PSCredential($username, $password)
$secPasswd = "password123" | ConvertTo-SecureString -AsPlainText -Force
$wc = New-Object System.Net.WebClient
$wc.Credentials = New-Object System.Net.NetworkCredential("domain\user", $secPasswd)
# get remote file info
Invoke-WebRequest -Proxy "http://x.x.x.x:8080" -ProxyCredential $myCreds -Uri "https://somewebsite/checksum.txt" -OutFile $remotefilepath -Credential $cred
$remotefile = Get-Content -Path $remotefilepath
Write-Host Remote File Hash : $remotefile
# if the file exists already
if (Test-Path $storepath) {
# get local file info
$localfile = (Get-FileHash $storepath -Algorithm SHA256).Hash
Write-Host Local File Hash : $localfile
# if the remote file is newer than the local file
if ($remotefile -ne $localfile) {
Invoke-WebRequest -Proxy "http://x.x.x.x:8080" -ProxyCredential $myCreds -Uri "https://somewebsite/checksum.txt" -OutFile $remotefilepath -Credential $cred
Invoke-WebRequest -Proxy "http://x.x.x.x:8080" -ProxyCredential $myCreds -Uri "https://somewebsite/somezipfile.zip" -OutFile $storepath -Credential $cred
This code should check the hash and download directly download the zip file from the website and unzip it.
If it can download in the zip file then the later part I can troubleshoot.
PS C:\Users\Administrator\Desktop> powershell -ExecutionPolicy ByPass -File C:\Users\Administrator\Desktop\mypowershell.ps1
powershell : Invoke-WebRequest : .?? ??? ?????? ???????? ??? ????? ?????
At line:1 char:1
+ powershell -ExecutionPolicy ByPass -File C:\Users\Administrator\Desktop\mypowershell.p ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (Invoke-WebReque...??? ????? ?????:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
The site you are trying to access contains content that is prohibited.
If you believe the website you are trying to access does not contain any such
content, please click here.
Above is the error I am getting.

How to do looping FTP uploads and downloads with powershell

I have a project here that I am trying to automate. I have some load tests i need to run on a machine and during that time I need to have a constant in and out of FTP traffic. I have been trying to write a Powershell script to do it but have been coming up short. I can get the file to download every time but the upload always fails. I also need to make sure my looping mechanism is correct.
I need this to download and upload a single file until I tell it to stop. Here is the script.
$FTPServer = '172.20.2.124'
$FTPUsername = 'ftp'
$FTPPassword = 'ftpTransfers'
$FTPSecurePassword = ConvertTo-SecureString -String $FTPPassword -asPlainText -Force
$FTPCredential = New-Object System.Management.Automation.PSCredential($FTPUsername,$FTPSecurePassword)
$Path = '/'
$LocalPath = 'C:\Downloads'
#Open session
Set-FTPConnection -Credentials $FTPCredential -Server $FTPServer -Session MySession -UseBinary -KeepAlive
$Session = Get-FTPConnection -Session MySession
#download file
Get-FTPItem -Path /File1.mxf -LocalPath $LocalPath -Overwrite -Session $Session | out-null
#Upload the file
Add-FTPItem -LocalPath $LocalPath/File1.mxf -Path $Path -Overwrite -Session $Session | out-null
do {
response = Read-Host "repeat?"
}
while ($response -eq "Y")
This is the error I get whenever I try to do an upload with the script. I have tried to figure out how to close the stream but I have not had any luck.
Add-FTPItem : Exception calling "GetRequestStream" with "0"
argument(s): "Object reference not set to an instance of an object."
At C:\Powershell Scripts\FTPUploadandDownload.ps1:22 char:1
+ Add-FTPItem -LocalPath $LocalPath/File1.mxf -Path $Path -Overwrite -Session ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Add-FTPItem
Any help would be appreciated, I am fairly new to this level of powershell.

using PsFtp to upload a file to FTP Powershell

I've been wracking my brain on this issue and can't seem to fix it. I'm trying to upload a file to FTP using PSFTP.
The script I'm using:
#------------------------------------------------------
#local variables
$ftp_server = "SERVERNAME"
$ftp_path = "/FTPPATH/PATH"
$local = "C:\ftp\"
$local_in = Join-Path $local "In"
$local_out = Join-Path $local "Out"
$session = "my_ftp_session"
# set up credentials object
$username = "FFandP"
$password = Get-Content "$local_out\Credentials.txt" | ConvertTo-SecureString -AsPlainText -Force
$cred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $username, $password
Set-FTPConnection -Server $ftp_server -Credentials $cred -Session $session -KeepAlive -confirm -UseBinary
Get-ChildItem -Path $local_out |
% {
$ftp_file = "$ftp_path/$($_.Name)" # determine item fullname
Add-FTPItem -Path $ftp_file -LocalPath $_.FullName -Session $session -
}
# -------------------------------------------------
And the error I receive:
Add-FTPItem : Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (550) File
unavailable (e.g., file not found, no access)."
At line:22 char:1
+ Add-FTPItem -Path $ftp_file -LocalPath $_.FullName -Session $session
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Add-FTPItem
I've tried running the Add-FTPitem command by itself, but I get the same error.
I can upload to the FTP using FileZilla. I have also tried removing the variables and using hard-coded paths; I get the same error.
Any ideas?
The answer in #Josh's comment solved it for me. Run Add-FTPItem with the -Overwrite parameter.
Add-FTPItem -Path $remotePath -LocaPath $myPath -Overwrite
It took me a moment to figure out this problem, but here is my solution (I had the same problem).
When using Add-FTPItem the -Path parameter must not include the filename itself.
Add-FTPItem
-Path "ftp://SomeServer/SomeDir/"
-LocalPath "C:\SomeFilename.ext"
-Session $session
So in your example it should be:
Add-FTPItem -Path $ftp_path -LocalPath $_.FullName -Session $session
The filename will be added to the remote FTP path. In case you don't want to have the same name you must either rename the file locally first or remotely after.
Change block
Get-ChildItem -Path $local_out | %{ .... }
to one line
Get-ChildItem -Path $local_out | Add-FTPItem -Path $ftp_path