How to create directories from powershell on FTP server? - powershell

I want to run a PS script when I want to publish to FTP server. I took this script as structure : structure script.
I have very simple folder :
C:\Uploadftp\Files\doc.txt
C:\Uploadftp\Files\Files2
C:\Uploadftp\Files\Files2\doc2.txt
nothing fancy there.
Here is my script :
cd C:\Uploadftp
$location = Get-Location
"We are here: $location"
$user = "test" # Change
$pass = "test" # Change
## Get files
$files = Get-ChildItem -recurse
## Get ftp object
$ftp_client = New-Object System.Net.WebClient
$ftp_client.Credentials = New-Object System.Net.NetworkCredential($user,$pass)
$ftp_address = "ftp://test/TestFolder"
## Make uploads
foreach($file in $files)
{
$directory = "";
$source = $($file.DirectoryName + "/" + $file);
if ($file.DirectoryName.Length -gt 0)
{
$directory = $file.DirectoryName.Replace($location,"")
}
$directory = $directory.Replace("\","/")
$source = $source.Replace("\","/")
$directory += "/";
$ftp_command = $($ftp_address + $directory + $file)
# Write-Host $source
$uri = New-Object System.Uri($ftp_command)
"Command is " + $uri + " file is $source"
$ftp_client.UploadFile($uri, $source)
}
I keep getting this error :
Exception calling "UploadFile" with "2" argument(s): "An exception occurred during a WebClient request."
If I hardcode specific folder for $uri and tell source to be some specific folder on my computer, this script doesn't create directory, it creates a file. What am I doing wrong?
P.S. dont hit me too hard, its my fist time ever doing something in power shell.

Try the "Create-FtpDirectory" function from https://github.com/stej/PoshSupport/blob/master/Ftp.psm1
function Create-FtpDirectory {
param(
[Parameter(Mandatory=$true)]
[string]
$sourceuri,
[Parameter(Mandatory=$true)]
[string]
$username,
[Parameter(Mandatory=$true)]
[string]
$password
)
if ($sourceUri -match '\\$|\\\w+$') { throw 'sourceuri should end with a file name' }
$ftprequest = [System.Net.FtpWebRequest]::Create($sourceuri);
$ftprequest.Method = [System.Net.WebRequestMethods+Ftp]::MakeDirectory
$ftprequest.UseBinary = $true
$ftprequest.Credentials = New-Object System.Net.NetworkCredential($username,$password)
$response = $ftprequest.GetResponse();
Write-Host Upload File Complete, status $response.StatusDescription
$response.Close();
}

Related

Connecting to webapp with ftp in a release pipeline to download files

Ive tried connecting to an ftp with the following powershell script:
#FTP Server Information - SET VARIABLES
$ftp = "ftp://XXX.com/"
$user = 'UserName'
$pass = 'Password'
$folder = 'FTP_Folder'
$target = "C:\Folder\Folder1\"
#SET CREDENTIALS
$credentials = new-object System.Net.NetworkCredential($user, $pass)
function Get-FtpDir ($url,$credentials) {
$request = [Net.WebRequest]::Create($url)
$request.Method = [System.Net.WebRequestMethods+FTP]::ListDirectory
if ($credentials) { $request.Credentials = $credentials }
$response = $request.GetResponse()
$reader = New-Object IO.StreamReader $response.GetResponseStream()
while(-not $reader.EndOfStream) {
$reader.ReadLine()
}
#$reader.ReadToEnd()
$reader.Close()
$response.Close()
}
#SET FOLDER PATH
$folderPath= $ftp + "/" + $folder + "/"
$files = Get-FTPDir -url $folderPath -credentials $credentials
$files
$webclient = New-Object System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass)
$counter = 0
foreach ($file in ($files | where {$_ -like "*.txt"})){
$source=$folderPath + $file
$destination = $target + $file
$webclient.DownloadFile($source, $target+$file)
#PRINT FILE NAME AND COUNTER
$counter++
$counter
$source
}
But i keep getting 530 error (authentication error) when connecting. I know the username and password is working cause i've tested it in other ftp clients.
So, i think it might be a problem because the webapp demands a certain protocol, which isnt used in this script.
Ive been doing some research and i found something called Posh-SSH which might work. But is there a way to modify my script instead? When im connecting with winscp i use FTP protocol with TLS/SSL implicit encryption to port 990.
UPDATE: WORKS
I made the following work:
#FTP Server Information - SET VARIABLES
$ftp = "ftp://waws-prod-xxx.ftp.azurewebsites.windows.net"
$user = 'xxxxx\xxxxx#email.com'
$pass = '$FRnqxxpxxxxxxx'
$folder = 'site/wwwroot/wwwroot/images/uploaded'
$target = "C:\Folder\Folder1\"
#SET CREDENTIALS
$credentials = new-object System.Net.NetworkCredential($user, $pass)
function Get-FtpDir ($url,$credentials) {
$request = [Net.WebRequest]::Create($url)
$request.Method = [System.Net.WebRequestMethods+FTP]::ListDirectory
if ($credentials) { $request.Credentials = $credentials }
$response = $request.GetResponse()
$reader = New-Object IO.StreamReader $response.GetResponseStream()
while(-not $reader.EndOfStream) {
$reader.ReadLine()
}
#$reader.ReadToEnd()
$reader.Close()
$response.Close()
}
#SET FOLDER PATH
$folderPath= $ftp + "/" + $folder + "/"
$files = Get-FTPDir -url $folderPath -credentials $credentials
Write-Host($files)
$files
$webclient = New-Object System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass)
$counter = 0
foreach ($file in ($files)){
$source=$folderPath + $file
$destination = $target + $file
$webclient.DownloadFile($source, $target+$file)
#PRINT FILE NAME AND COUNTER
$counter++
$counter
$source
}
I ended up adding a new credential to the web app, and changing sftp in hostname to ftp, and now it works. Using the credentials from webpublish file works too.
I also made WinSCP work and im able to download the full folder with children.
#FTP Server Information - SET VARIABLES
$ftp = "waws-prod-xxx-xxx.ftp.azurewebsites.windows.net"
$user = 'xxxxxx\xxxx#xxxxx.no'
$pass = '$xxxxxxxxx'
$folder = 'site/wwwroot/wwwroot/images/uploaded/*'
$target = "C:\Folder\Folder1\*"
# Load WinSCP .NET assembly
Add-Type -Path "WinSCPnet.dll"
# Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Ftp
HostName = $ftp
UserName = $user
Password = $pass
}
$session = New-Object WinSCP.Session
try
{
# Connect
$session.Open($sessionOptions)
# Download files
$session.GetFiles($folder, $target).Check()
}
finally
{
# Disconnect, clean up
$session.Dispose()
}
FtpWebRequest (nor any other built-in .NET FTP API) does not support implicit TLS/SSL encryption.
See Does .NET FtpWebRequest Support both Implicit (FTPS) and explicit (FTPES)?
Posh-SSH is SSH/SFTP library. It has nothing to do with FTP.
And as you get an "authentication error", I believe your problem is actually different. Double check your credentials. You are probably using wrong ones in your code.
Also, if WinSCP works for you, you can use WinSCP .NET assembly from the PowerShell. WinSCP GUI can even generate a code template for you, based on your working GUI session.

SharePoint 2013 Powershell - Copy File From One Site Collection To Another

Please can someone assist in helping with the above subject?
I would like to copy one file from a specific folder in a sharepoint site collection to another library (of the same name) in a different sharepoint site collection (but still within the same Web Application).
I have very little Powershell experience and have tried a number of Google searches but cannot seem to find anything that works.
Below is an example of what i have tried to do (lots of Write-Host to try and figure out what is going on) with the error message at the bottom.
Add-PSSnapIn "Microsoft.SharePoint.PowerShell"
##
#Set Static Variables
##
$SourceWebURL = "http://WebAppURL/sites/Area/Master"
$SourceLibraryTitle = "Web"
$DestinationWebURL = "http://WebAppURL/sites/OtherSiteName"
$DestinationLibraryTitle = "Web"
$FileName = "Resources.aspx"
##
#Begin Script
##
$sWeb = Get-SPWeb $SourceWebURL
$sList = $sWeb.Lists | ? {$_.Title -eq $SourceLibraryTitle}
$dWeb = Get-SPWeb $DestinationWebURL
$dList = $dWeb.Lists | ? {$_.title -like $DestinationLibraryTitle}
$DestFolder = $dList.Files
$RootFolder = $sList.RootFolder
Write-Host " line 25 -- " $RootFolder
$collfiles1 = $RootFolder.Files
Write-Host " line 27 -- "$collfiles1
Write-Host " line 28 -- "$DestFolder
Write-Host " line 30 -- "$str = $DestinationWebURL"/"$DestinationLibraryTitle"/"$FileName
Write-Host " line 31 -- "$collfiles1.Count
for($i = 0 ; $i -lt $collfiles1.Count ; $i++)
{
Write-Host " line 34 -- "$collfiles1[$i].Name
##Write-Host $FileName
if($collfiles1[$i].Name -eq $FileName)
{
## $str = $DestinationWebURL.Url + $DestinationLibraryTitle + "/" + $FileName
$str = $DestinationWebURL+"/" +$DestinationLibraryTitle+"/"
Write-Host " line 40 -- "$str
Write-Host " line 41 -- "$collfiles1[$i]
$FiletoCopy = $collfiles1[$i].Name
Write-Host " line 43 -- " $FiletoCopy
$FiletoCopy.CopyTo($str,$true)
}
}
Write-Host "Script Completed"
The below example gives the error
Cannot find an overload for "CopyTo" and the argument count: "2".
At line:44 char:3
+ $FiletoCopy.CopyTo($str,$true)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodException
+ FullyQualifiedErrorId : MethodCountCouldNotFindBest
If someone could point me in the right direction that would be very helpful.
Thanks in advance,
Ian.
The following PowerShell for your reference, copy a file from one library in site collection to another library in another site collection with fields.
Add-PSSnapIn "Microsoft.SharePoint.PowerShell"
##
#Set Static Variables
##
$SourceWebURL = "http://WebAppURL/sites/Area/Master"
$SourceLibraryTitle = "Web"
$DestinationWebURL = "http://WebAppURL/sites/OtherSiteName"
$DestinationLibraryTitle = "Web"
$FileName = "Resources.aspx"
##
#Begin Script
##
$sWeb = Get-SPWeb $SourceWebURL
#$sList = $sWeb.Lists | ? {$_.Title -eq $SourceLibraryTitle}
$dWeb = Get-SPWeb $DestinationWebURL
#$dList = $dWeb.Lists | ? {$_.title -like $DestinationLibraryTitle}
$SourceFile=$sWeb.GetFile($SourceWebURL+"/"+$SourceLibraryTitle+"/"+$FileName)
$TargetFolder = $dWeb.GetFolder($DestinationLibraryTitle)
#Copy File from the Source
$NewFile = $TargetFolder.Files.Add($SourceFile.Name, $SourceFile.OpenBinary(),$True)
#Copy Meta-Data from Source
Foreach($Field in $SourceFile.Item.Fields)
{
If(!$Field.ReadOnlyField)
{
if($NewFile.Item.Fields.ContainsField($Field.InternalName))
{
$NewFile.Item[$Field.InternalName] = $SourceFile.Item[$Field.InternalName]
}
}
}
#Update
$NewFile.Item.UpdateOverwriteVersion()
Write-host "Copied File:"$SourceFile.Name
Reference: Copy Files Between Document Libraries in SharePoint using PowerShell
So in case of large files where file size is greater than 50MB. This script mentioned by #LZ_MSFT will never be able to copy that file may be. In that aspect, you need to chunk the file into small pieces.Here is the PS to copy from source to destination with chunking if file size is greater than 50MB. Plus point for this one script is, it is using Client so it can be used with SP online and on-prem.
Add-Type –Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type –Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"
Function UploadFileInSlice ($DestinationCtx, $SourceCtx, $SourceFileUrl, $DestinationFolderUrl, $fileName, $fileChunkSizeInMB) {
# Each sliced upload requires a unique ID.
$UploadId = [GUID]::NewGuid()
# Get File by Server Relative URL
$File = $SourceCtx.Web.GetFileByServerRelativeUrl($SourceFileUrl)
$SourceCtx.Load($File)
# Get file Steam with OpenBinarySteam
$StreamToUpload = $File.OpenBinaryStream()
$SourceCtx.ExecuteQuery()
# File size in bytes
$FileSize = ($File).length
# Get Destination Folder by Server Relative URL
$DestinationFolder =
$DestinationContext.Web.GetFolderByServerRelativeUrl($DestinationFolderUrl)
$DestinationCtx.Load($DestinationFolder)
$DestinationCtx.ExecuteQuery()
# Set Complete Destination URL with Destination Folder + FileName
$destUrl = $DestinationFolderUrl + "/" + $fileName
# File object.
[Microsoft.SharePoint.Client.File] $upload
# Calculate block size in bytes.
$BlockSize = $fileChunkSizeInMB * 1000 * 1000
Write-Host "File Size is: $FileSize bytes and Chunking Size is:$BlockSize bytes"
if ($FileSize -le $BlockSize)
{
# Use regular approach if file size less than BlockSize
Write-Host "File uploading with out chunking"
$upload =[Microsoft.SharePoint.Client.File]::SaveBinaryDirect($DestinationCtx, $destUrl, $StreamToUpload.Value, $true)
return $upload
}
else
{
# Use large file upload approach.
$BytesUploaded = $null
$Fs = $null
Try {
$br = New-Object System.IO.BinaryReader($StreamToUpload.Value)
#$br = New-Object System.IO.BinaryReader($Fs)
$buffer = New-Object System.Byte[]($BlockSize)
$lastBuffer = $null
$fileoffset = 0
$totalBytesRead = 0
$bytesRead
$first = $true
$last = $false
# Read data from file system in blocks.
while(($bytesRead = $br.Read($buffer, 0, $buffer.Length)) -gt 0) {
$totalBytesRead = $totalBytesRead + $bytesRead
# You've reached the end of the file.
if($totalBytesRead -eq $FileSize) {
$last = $true
# Copy to a new buffer that has the correct size.
$lastBuffer = New-Object System.Byte[]($bytesRead)
[array]::Copy($buffer, 0, $lastBuffer, 0, $bytesRead)
}
If($first)
{
$ContentStream = New-Object System.IO.MemoryStream
# Add an empty file.
$fileCreationInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
$fileCreationInfo.ContentStream = $ContentStream
$fileCreationInfo.Url = $fileName
$fileCreationInfo.Overwrite = $true
#Add file to Destination Folder with file creation info
$Upload = $DestinationFolder.Files.Add($fileCreationInfo)
$DestinationCtx.Load($Upload)
# Start upload by uploading the first slice.
$s = New-Object System.IO.MemoryStream(,$Buffer)
Write-Host "Uploading id is:"+$UploadId
# Call the start upload method on the first slice.
$BytesUploaded = $Upload.StartUpload($UploadId, $s)
$DestinationCtx.ExecuteQuery()
# fileoffset is the pointer where the next slice will be added.
$fileoffset = $BytesUploaded.Value
Write-Host "First patch of file with bytes"+ $fileoffset
# You can only start the upload once.
$first = $false
}
Else
{
# Get a reference to your file.
$Upload = $DestinationCtx.Web.GetFileByServerRelativeUrl($destUrl);
If($last) {
# Is this the last slice of data?
$s = New-Object System.IO.MemoryStream(,$lastBuffer)
# End sliced upload by calling FinishUpload.
$Upload = $Upload.FinishUpload($UploadId, $fileoffset, $s)
$DestinationCtx.ExecuteQuery()
Write-Host "File Upload Completed Successfully!"
# Return the file object for the uploaded file.
return $Upload
}
else {
$s = New-Object System.IO.MemoryStream(,$buffer)
# Continue sliced upload.
$BytesUploaded = $Upload.ContinueUpload($UploadId, $fileoffset, $s)
$DestinationCtx.ExecuteQuery()
# Update fileoffset for the next slice.
$fileoffset = $BytesUploaded.Value
Write-Host "File uploading is in progress with bytes: "+ $fileoffset
}
}
} #// while ((bytesRead = br.Read(buffer, 0, buffer.Length)) > 0)
}
Catch {
Write-Host $_.Exception.Message -ForegroundColor Red
}
Finally {
if ($Fs -ne $null)
{
$Fs.Dispose()
}
}
}
return $null
}
#URL to Configure, in this case Destination is SP Online site URL
#Adding up credentials hard-code, you can use Get-Credentails PS command too
$DestnationSiteUrl = "https://your-domain.sharepoint.com/sites/xyz"
$DestinationRelativeURL = "/sites/xyz/TestLibrary" #server relative URL here with library Name and Folder name
$DestinationUserName = "xyz#your-domain.com"
$DestinationPassword = Read-Host "Enter Password for Destination User:
$DestinationUserName" -AsSecureString
#URL to Configure, in this case Source is On-Prem site URL
#Adding up credentials hard-code, you can use Get-Credentails PS command too
$SourceSiteUrl = "http://intranet/sites/xyz"
$SourceRelativeURL = "/sites/xyz/TestLibrary/myfile.pptx" #server relative URL here with library Name and file name with extension
$SourceUsername = "domain\xyz"
$SourcePassword = Read-Host "Enter Password for Source User: $SourceUsername" -AsSecureString
#Set a file name with extension
$FileNameWithExt = "myfile.pptx"
#Get Source Client Context with credentials
$SourceContext = New-Object Microsoft.SharePoint.Client.ClientContext($SourceSiteUrl)
#Using NetworkCredentials in case of On-Prem
$SourceCtxcredentials = New-Object System.Net.NetworkCredential($SourceUsername, $SourcePassword)
$SourceContext.RequestTimeout = [System.Threading.Timeout]::Infinite
$SourceContext.ExecuteQuery();
#Get Destination Client Context with credentials
$DestinationContext = New-Object Microsoft.SharePoint.Client.ClientContext($DestnationSiteUrl)
#Using SharePointOnlineCredentials in case of SP-Online
$DestinationContext.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($DestinationUserName, $DestinationPassword)
$DestinationContext.RequestTimeout = [System.Threading.Timeout]::Infinite
$DestinationContext.ExecuteQuery();
#All Set up, now just call the UploadFileInSlice with parameters
$UpFile = UploadFileInSlice -DestinationCtx $DestinationContext -SourceCtx $SourceContext -DestinationFolderUrl $DestinationRelativeURL -SourceFileUrl $SourceRelativeURL -fileName $FileNameWithExt -fileChunkSizeInMB 10

Recursively downloading files from FTP server

Hi All i am new to power shell scripting, i am trying to download all files in root directory and files in sub folder based on file name from FTP server. As i am looking to download files which got matched on files names in like, as my path will always point to root directory and root directory is having text files and sub folders, the sub folders contains text files.
I found an example to get text files to download to local from ftp server, but the example i found is only downloading text files from root directory it is not scanning sub folders and not getting files in sub folders.
Here is the example i tried
# $url = "ftp://XXX.com/"
$user = 'UserName'
$pass = 'Password'
$folder = 'FTP_Folder'
$target = "C:\Folder\Folder1\"
#SET CREDENTIALS
$credentials = new-object System.Net.NetworkCredential($user, $pass)
function Get-FtpDir ($url,$credentials) {
$request = [Net.WebRequest]::Create($url)
$request.Method = [System.Net.WebRequestMethods+FTP]::ListDirectory
if ($credentials) { $request.Credentials = $credentials }
$response = $request.GetResponse()
$reader = New-Object IO.StreamReader $response.GetResponseStream()
$reader.ReadToEnd()
$reader.Close()
$response.Close()
}
#SET FOLDER PATH
$folderPath= $ftp + "/" + $folder + "/"
$Allfiles=Get-FTPDir -url $folderPath -credentials $credentials
$files = ($Allfiles -split "`r`n")
$files
$webclient = New-Object System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass)
$counter = 0
foreach ($file in ($files | where {$_ -like "*.txt"})){
$source=$folderPath + $file
$destination = $target + $file
$webclient.DownloadFile($source, $target+$file)
#PRINT FILE NAME AND COUNTER
$counter++
$counter
$source
}
I tried a module PSFTP module functions like Get-ftpchilditem function with recursive, but it not worked for me.
Please help me in this to get files from root folder and sub folders of FTP server to local system.
I have done it in the past using WinSCP and SFTP. The example is below. This uses SFTP SCP which is basically over SSH. I am sure you can change the WInSCP protocol to ftp and try different combination. Hope this helps you. Change the FileMasks to only include .txt and this will go through sub folders. Also for this to work you will require WinSCPnet.dll and WinSCP.exe which you can easily download from WinSCP site. Download the ".Net Assembly/COM Library" and keep the script and the required files in the same folder and update the path in the script. Good Luck.
# Load WinSCP .NET assembly
Add-Type -Path "C:\Users\administrator.AD\Documents\WinSCP Script\WinSCPnet.dll"
$hostname = "ftp.xyz.net"
$user = "me"
$pass = "mypassword"
$sfolder = "/home/me"
$dfolder = "C:\tmp"
$sshHostKeyFingerprint = "ssh-rsa 2048 eb:2c:f9:ab:19:2e:0c:22:02:d4:8f:64:44:75:ec:04"
$mask = "*.jpg;*.jpeg;*.png;*.tiff;*.txt"
####################### END USER CONFIGURABLE SETTINGS #########################
# Main script
# Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Scp
HostName = $hostname
UserName = $user
Password = $pass
SshHostKeyFingerprint = $sshHostKeyFingerprint
}
$session = New-Object WinSCP.Session
$transferOptions = New-Object WinSCP.TransferOptions
$transferOptions.FileMask = "*.jpg; *.jpeg; *.png; *.tiff;"
try
{
# Will continuously report progress of synchronization
$session.add_FileTransferred( { FileTransferred($_) } )
# Connect
$session.Open($sessionOptions)
# Synchronize files
$synchronizationResult = $session.SynchronizeDirectories(
[WinSCP.SynchronizationMode]::Local,$dfolder, $sfolder, $False, $False, 1, $transferOptions)
# Throw on any error
$synchronizationResult.Check()
}
finally
{
# Disconnect, clean up
$session.Dispose()
}

copying many files from sharepoint to other using powershell

I m trying to copy documents from list in one sharepoint to another :
this is my code :
*************************************************
$source= "\\...\s1"
$destination = "\\..\s2"
foreach ($result in $result )
{ copy-item -path $source -dest $destination}
*************************************************
-$result is the list of all documents that I got using web-services, it s type is system.array
-$source and $destination are UNC that refers to URL of the two sharepoint
the error is
" can not find the path \...\s1system.xml.Xml.XmlElement"
PS: I m not using server machine,it s just a client
Here is my code
*****************************************************
{
param (
[String]$Value,
[String]$Field,
[String]$RowLimit = "0",
[String]$Operator = "Contains",
[String]$WebURL = "https://.................../wer",
[String]$ListName = "Main documents",
[String]$ViewName,
[Switch]$Recurse
)
$ScriptDirectory = split-path $MyInvocation.MyCommand.Definition
$dllPath = "P:\SamlCookieAuth.dll" -f $ScriptDirectory
[void][System.Reflection.Assembly]::LoadFrom($dllPath)
$queryOptionsValue = ''
if ($Recurse)
{
$queryOptionsValue = '<ViewAttributes Scope="RecursiveAll"/>'
}
$WSUri = $WebURL + "/_vti_bin/Lists.asmx?wsdl"
$listsWebServiceReference = New-WebServiceProxy -Uri $WSUri -UseDefaultCredential
$listsWebServiceReference.Url = $webURL + "/_vti_bin/lists.asmx"
[System.Uri]$CookieUri = $WebURL
$listsWebServiceReference.CookieContainer = [ST.SamlCookieAuth.SamlCookieManager]::GetAuthenticatedCookiesContainer($CookieUri.AbsoluteUri, 0, 0)
[System.Xml.XmlDocument]$xmlDoc = New-Object -TypeName System.Xml.XmlDocument
[System.Xml.XmlElement]$queryOptions =$xmlDoc.CreateElement("QueryOptions")
$queryOptions.InnerXml = $queryOptionsValue
if ($PSBoundParameters.Keys.Contains("Value"))
{
[System.Xml.XmlElement]$query = $xmlDoc.CreateElement("Query")
$queryValue = "<Where><$Operator><FieldRef Name='$Field'/><Value Type='Text'>$Value</Value></$Operator></Where>"
$query.InnerXml = $queryValue
$result=$listsWebServiceReference.GetListItems($listName, $viewName, $query, $null, $rowLimit, $queryOptions, $null).data.row
}
else
{
$result=$listsWebServiceReference.GetListItems($listName, $viewName, $null, $null, $rowLimit, $queryOptions, $null).data.row
}
$destDirectory = "\\.............\TER\Main Documents"
foreach ($resul in $result)
{Copy-Item -path $resul -destination $destDirectory }
}
Perhaps the problem that you are having is the result of a conversion issue. You are trying to write content from one SP to another as a document, however, you cannot do so with an XmlElement.
Suggest checking out this post:
Converting system.xml.xmlelement to system.xml.xmldocument with PowerShell
It would be helpful to see the content of "-$result is the list of all documents that I got using web-services, it s type is system.array".
Also, for readability, I would suggest differentiating between $result and $result as follows:
foreach($document in $documentList){}
-or- (at minimum)
foreach($result in $results){}

Upload files with FTP using PowerShell

I want to use PowerShell to transfer files with FTP to an anonymous FTP server. I would not use any extra packages. How?
I am not sure you can 100% bullet proof the script from not hanging or crashing, as there are things outside your control (what if the server loses power mid-upload?) - but this should provide a solid foundation for getting you started:
# create the FtpWebRequest and configure it
$ftp = [System.Net.FtpWebRequest]::Create("ftp://localhost/me.png")
$ftp = [System.Net.FtpWebRequest]$ftp
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential("anonymous","anonymous#localhost")
$ftp.UseBinary = $true
$ftp.UsePassive = $true
# read in the file to upload as a byte array
$content = [System.IO.File]::ReadAllBytes("C:\me.png")
$ftp.ContentLength = $content.Length
# get the request stream, and write the bytes into it
$rs = $ftp.GetRequestStream()
$rs.Write($content, 0, $content.Length)
# be sure to clean up after ourselves
$rs.Close()
$rs.Dispose()
There are some other ways too. I have used the following script:
$File = "D:\Dev\somefilename.zip";
$ftp = "ftp://username:password#example.com/pub/incoming/somefilename.zip";
Write-Host -Object "ftp url: $ftp";
$webclient = New-Object -TypeName System.Net.WebClient;
$uri = New-Object -TypeName System.Uri -ArgumentList $ftp;
Write-Host -Object "Uploading $File...";
$webclient.UploadFile($uri, $File);
And you could run a script against the windows FTP command line utility using the following command
ftp -s:script.txt
(Check out this article)
The following question on SO also answers this: How to script FTP upload and download?
I'm not gonna claim that this is more elegant than the highest-voted solution...but this is cool (well, at least in my mind LOL) in its own way:
$server = "ftp.lolcats.com"
$filelist = "file1.txt file2.txt"
"open $server
user $user $password
binary
cd $dir
" +
($filelist.split(' ') | %{ "put ""$_""`n" }) | ftp -i -in
As you can see, it uses that dinky built-in windows FTP client. Much shorter and straightforward, too. Yes, I've actually used this and it works!
Easiest way
The most trivial way to upload a binary file to an FTP server using PowerShell is using WebClient.UploadFile:
$client = New-Object System.Net.WebClient
$client.Credentials =
New-Object System.Net.NetworkCredential("username", "password")
$client.UploadFile(
"ftp://ftp.example.com/remote/path/file.zip", "C:\local\path\file.zip")
Advanced options
If you need a greater control, that WebClient does not offer (like TLS/SSL encryption, etc), use FtpWebRequest. Easy way is to just copy a FileStream to FTP stream using Stream.CopyTo:
$request = [Net.WebRequest]::Create("ftp://ftp.example.com/remote/path/file.zip")
$request.Credentials =
New-Object System.Net.NetworkCredential("username", "password")
$request.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$fileStream = [System.IO.File]::OpenRead("C:\local\path\file.zip")
$ftpStream = $request.GetRequestStream()
$fileStream.CopyTo($ftpStream)
$ftpStream.Dispose()
$fileStream.Dispose()
Progress monitoring
If you need to monitor an upload progress, you have to copy the contents by chunks yourself:
$request = [Net.WebRequest]::Create("ftp://ftp.example.com/remote/path/file.zip")
$request.Credentials =
New-Object System.Net.NetworkCredential("username", "password")
$request.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$fileStream = [System.IO.File]::OpenRead("C:\local\path\file.zip")
$ftpStream = $request.GetRequestStream()
$buffer = New-Object Byte[] 10240
while (($read = $fileStream.Read($buffer, 0, $buffer.Length)) -gt 0)
{
$ftpStream.Write($buffer, 0, $read)
$pct = ($fileStream.Position / $fileStream.Length)
Write-Progress `
-Activity "Uploading" -Status ("{0:P0} complete:" -f $pct) `
-PercentComplete ($pct * 100)
}
$ftpStream.Dispose()
$fileStream.Dispose()
Uploading folder
If you want to upload all files from a folder, see
PowerShell Script to upload an entire folder to FTP
I recently wrote for powershell several functions for communicating with FTP, see https://github.com/AstralisSomnium/PowerShell-No-Library-Just-Functions/blob/master/FTPModule.ps1. The second function below, you can send a whole local folder to FTP. In the module are even functions for removing / adding / reading folders and files recursively.
#Add-FtpFile -ftpFilePath "ftp://myHost.com/folder/somewhere/uploaded.txt" -localFile "C:\temp\file.txt" -userName "User" -password "pw"
function Add-FtpFile($ftpFilePath, $localFile, $username, $password) {
$ftprequest = New-FtpRequest -sourceUri $ftpFilePath -method ([System.Net.WebRequestMethods+Ftp]::UploadFile) -username $username -password $password
Write-Host "$($ftpRequest.Method) for '$($ftpRequest.RequestUri)' complete'"
$content = $content = [System.IO.File]::ReadAllBytes($localFile)
$ftprequest.ContentLength = $content.Length
$requestStream = $ftprequest.GetRequestStream()
$requestStream.Write($content, 0, $content.Length)
$requestStream.Close()
$requestStream.Dispose()
}
#Add-FtpFolderWithFiles -sourceFolder "C:\temp\" -destinationFolder "ftp://myHost.com/folder/somewhere/" -userName "User" -password "pw"
function Add-FtpFolderWithFiles($sourceFolder, $destinationFolder, $userName, $password) {
Add-FtpDirectory $destinationFolder $userName $password
$files = Get-ChildItem $sourceFolder -File
foreach($file in $files) {
$uploadUrl ="$destinationFolder/$($file.Name)"
Add-FtpFile -ftpFilePath $uploadUrl -localFile $file.FullName -username $userName -password $password
}
}
#Add-FtpFolderWithFilesRecursive -sourceFolder "C:\temp\" -destinationFolder "ftp://myHost.com/folder/" -userName "User" -password "pw"
function Add-FtpFolderWithFilesRecursive($sourceFolder, $destinationFolder, $userName, $password) {
Add-FtpFolderWithFiles -sourceFolder $sourceFolder -destinationFolder $destinationFolder -userName $userName -password $password
$subDirectories = Get-ChildItem $sourceFolder -Directory
$fromUri = new-object System.Uri($sourceFolder)
foreach($subDirectory in $subDirectories) {
$toUri = new-object System.Uri($subDirectory.FullName)
$relativeUrl = $fromUri.MakeRelativeUri($toUri)
$relativePath = [System.Uri]::UnescapeDataString($relativeUrl.ToString())
$lastFolder = $relativePath.Substring($relativePath.LastIndexOf("/")+1)
Add-FtpFolderWithFilesRecursive -sourceFolder $subDirectory.FullName -destinationFolder "$destinationFolder/$lastFolder" -userName $userName -password $password
}
}
Here's my super cool version BECAUSE IT HAS A PROGRESS BAR :-)
Which is a completely useless feature, I know, but it still looks cool \m/ \m/
$webclient = New-Object System.Net.WebClient
Register-ObjectEvent -InputObject $webclient -EventName "UploadProgressChanged" -Action { Write-Progress -Activity "Upload progress..." -Status "Uploading" -PercentComplete $EventArgs.ProgressPercentage } > $null
$File = "filename.zip"
$ftp = "ftp://user:password#server/filename.zip"
$uri = New-Object System.Uri($ftp)
try{
$webclient.UploadFileAsync($uri, $File)
}
catch [Net.WebException]
{
Write-Host $_.Exception.ToString() -foregroundcolor red
}
while ($webclient.IsBusy) { continue }
PS. Helps a lot, when I'm wondering "did it stop working, or is it just my slow ASDL connection?"
You can simply handle file uploads through PowerShell, like this.
Complete project is available on Github here https://github.com/edouardkombo/PowerShellFtp
#Directory where to find pictures to upload
$Dir= 'c:\fff\medias\'
#Directory where to save uploaded pictures
$saveDir = 'c:\fff\save\'
#ftp server params
$ftp = 'ftp://10.0.1.11:21/'
$user = 'user'
$pass = 'pass'
#Connect to ftp webclient
$webclient = New-Object System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass)
#Initialize var for infinite loop
$i=0
#Infinite loop
while($i -eq 0){
#Pause 1 seconde before continue
Start-Sleep -sec 1
#Search for pictures in directory
foreach($item in (dir $Dir "*.jpg"))
{
#Set default network status to 1
$onNetwork = "1"
#Get picture creation dateTime...
$pictureDateTime = (Get-ChildItem $item.fullName).CreationTime
#Convert dateTime to timeStamp
$pictureTimeStamp = (Get-Date $pictureDateTime).ToFileTime()
#Get actual timeStamp
$timeStamp = (Get-Date).ToFileTime()
#Get picture lifeTime
$pictureLifeTime = $timeStamp - $pictureTimeStamp
#We only treat pictures that are fully written on the disk
#So, we put a 2 second delay to ensure even big pictures have been fully wirtten in the disk
if($pictureLifeTime -gt "2") {
#If upload fails, we set network status at 0
try{
$uri = New-Object System.Uri($ftp+$item.Name)
$webclient.UploadFile($uri, $item.FullName)
} catch [Exception] {
$onNetwork = "0"
write-host $_.Exception.Message;
}
#If upload succeeded, we do further actions
if($onNetwork -eq "1"){
"Copying $item..."
Copy-Item -path $item.fullName -destination $saveDir$item
"Deleting $item..."
Remove-Item $item.fullName
}
}
}
}
You can use this function :
function SendByFTP {
param (
$userFTP = "anonymous",
$passFTP = "anonymous",
[Parameter(Mandatory=$True)]$serverFTP,
[Parameter(Mandatory=$True)]$localFile,
[Parameter(Mandatory=$True)]$remotePath
)
if(Test-Path $localFile){
$remoteFile = $localFile.Split("\")[-1]
$remotePath = Join-Path -Path $remotePath -ChildPath $remoteFile
$ftpAddr = "ftp://${userFTP}:${passFTP}#${serverFTP}/$remotePath"
$browser = New-Object System.Net.WebClient
$url = New-Object System.Uri($ftpAddr)
$browser.UploadFile($url, $localFile)
}
else{
Return "Unable to find $localFile"
}
}
This function send specified file by FTP.
You must call the function with these parameters :
userFTP = "anonymous" by default or your username
passFTP = "anonymous" by default or your password
serverFTP = IP address of the FTP server
localFile = File to send
remotePath = the path on the FTP server
For example :
SendByFTP -userFTP "USERNAME" -passFTP "PASSWORD" -serverFTP "MYSERVER" -localFile "toto.zip" -remotePath "path/on/the/FTP/"
Goyuix's solution works great, but as presented it gives me this error: "The requested FTP command is not supported when using HTTP proxy."
Adding this line after $ftp.UsePassive = $true fixed the problem for me:
$ftp.Proxy = $null;
Simple solution if you can install curl.
curl.exe -p --insecure "ftp://<ftp_server>" --user "user:password" -T "local_file_full_path"