Is there any component in Talend that can create a Directory in FTP?
I need to copy the files from one directory in FTP to another Directory(New) in FPT, For that i need to create a new Directory?
You can use the tFTPCreateDirectory 0.2 from Talend marketplace.
Marc
We can use Tssh component in Talend and use unix command to create directory.
Related
I am connecting to SFTP via host, port, username and password using PowerShell. I want to count the number of files in a particular zip folder without having to download the zip folder on my local machine and count. Please share the piece of logic that would do this. I looked into this but it seems a bit tricky when it comes to do this in a zip folder.
That's not an easy task to do. There's no API in SFTP to do that completely remotely. There are basically two solutions:
Use SFTP to download only the ZIP central directory (basically the listing that is placed at the very end of the ZIP file). And decode the directory locally. For C#, this is covered in my answer to List files inside ZIP file located on SFTP server in C#. Though as mentioned there, there's a bug in SSH.NET that requires a workaround with implementing an interface. While that's probably doable in PowerShell too, I've never done that.
If you have an SSH shell access to the server, use remote zip command to list the contents of the file. Or build another API (like a web service).
Btw, note that there's nothing like ZIP "folder". ZIP is an archive file. It's only Windows that call ZIP files "folders".
I have a problem with my VPS server, more precisely with extracting a .zip file (5GB) through the file manager. I have limited support because it is a self-managed VPS. I need a command for a terminal to extract file "5.zip" to a specific server dir for ex: http:xxxxxxx.com/funny folder on my server. Can someone help me with this? Thanks.
You provide URL, not path to directory. But to extract zip file in particular directory you can use command:
cd /directory/funny
unzip /path/to/5.zip
Change /directory/funny with the real directory which your web service serve.
We have a customer with their SFTP site, and I would like to copy files from specific folder, by using any automated process.
One of the example which I found, is winscp.net, but I have not managed how to use it, for my purpose.
http://www.itworld.com/article/2928599/windows/how-to-automate-sftp-file-transfers-in-microsoft-windows.html
QUESTION: All I will need is not run script and the file should be copied from their directory to my local folder. Is it possible at all?
I found the way of using PSFTP(Putty) to connect to the server, but how to make it automatically, do not know.
I think its a good solution for your problem.
You have to install winscp and the code some files and it will do automaticaly. I used 3 or 4 times. Also you need the key for your SFTP to connect throught it.
Here a link to the guide step by step.
https://winscp.net/eng/docs/guides
Here is a link to the scripting webpage
https://winscp.net/eng/docs/scripting
I recommend you to create an ini file to load all of the characteristics of your ftp connection and then execute an script over this.
It can be launched like that
WinSCP.com /ini=[your ini file] /script=[Your script file(what you want to do when it is connect)]
Hope this helps!.
I want to download a zip file from this website, unzip it and then show it into an excel file.
http://www.belgianfootball.be/nl/downloads-competities
With which component can I do this?
You can use the tHTTRequest component.
https://help.talend.com/display/TalendOpenStudioComponentsReferenceGuide521EN/14.11+tHttpRequest
Here you can find a nice guide:
https://www.talendbyexample.com/talend-thttprequest-component.html
Hi I want to know how to copy files to HDFS from source file system(Local File system),if source file already copied to HDFS,then how to eliminate or ignore that file to copy again in HDFS using Talend.
Thanks
Venkat
To copy files from local file system to the HDFS, you need to use tHDFSPut components if you have Talend for big data. If you use Talend for data integration you can easily use tSystem component with the right command.
To avoid duplicated files, you need to create a table in a RDBMS and keep track of all copied files. Each time the job start copying file, it should check if it already exists in the table.