There is an FTP server. On that server there are two folders (Folder1 and Folder2). Folder1 contains 20 csv files (Total size more than 2 GB). I want to move all csv from Folder1 to Folder2. But I don't want to use TFTPGet and TFTPPut as it will take to too much time to upload.
Can anyone help me?
Yes, we can. You can use tFTPRename component and give fully specified file paths of different folders to the Filemask and New name fields.
There are two ways to accomplish this in Talend. If you wish to copy all contents in a directory, then you only need a tFileCopy component and check "Copy a Directory" specifying the source and destination directories.
If you need to copy only certain files in a directory, you can accomplish this in Talend using 2 components that work together. You need a tFileList and a tFileCopy, connecting them together with an Iterate flow.
Use the tFileList to generate your list of files from a specified directory. You can configure wildcards in the filemask section. For example, to only take .txt you would enter "*.txt" in the filemask section.
Then rightclick tFileList in the designer and click Row-->Iterate. Connect this to the FileCopy component. In FileCopy use this code in file name:
((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))
You have other options in the FileCopy component as well, including Remove Source File, and Create the Directory if it doesn't exist.
Select which of the two best suit your needs.
Related
I'm following the official guide for Parcel 2.
By default parcel takes your input files and dumps them all in the same output directory.
The vast majority of developers have specific directory layouts for our projects. So parcel must offer a way for its users to put files and directories where we want them to go.
But I can't documentation for configuring output file layout.
How do I control where output files are created? And how to rename them?
For example, if I'm working with projectroot/src/js/index.js, how can I output to projectroot/dist/js/bundle.js
Hi I want to know how to copy files to HDFS from source file system(Local File system),if source file already copied to HDFS,then how to eliminate or ignore that file to copy again in HDFS using Talend.
Thanks
Venkat
To copy files from local file system to the HDFS, you need to use tHDFSPut components if you have Talend for big data. If you use Talend for data integration you can easily use tSystem component with the right command.
To avoid duplicated files, you need to create a table in a RDBMS and keep track of all copied files. Each time the job start copying file, it should check if it already exists in the table.
Currently all of my TypeScript typings are located under Scripts/typings/Library. By default, TypeLite is just under Scripts. Is there a way to specify the output directory so it is under a custom Library name? ie Scripts/typings/MyCSharpProj?
TypeLite uses T4 templates for generating output files and I am afraid it isn't possible to change the output directory for the template. The output file is always generated alongside the template.
If you move Manager.ttinclude TypeLite.tt and TypeLite.Net4.tt to another directory and the output files will be generated there.
Is there a way to make Progress Developer Studio 3.7 (Eclipse) generate all the wrx files (from the ocx) and place them in for example the rcode folder?
Clarification:
I dont know even how to make one wrx file. Have heard this "They get automatically created as soon as you drop an OCX control onto an ABL frame". But if you have removed that file, can you create it anew without having to redrop the control? And how do you automatically place it in a certain folder?
wrx files contain the properties of an ActiveX you set in the appbuilder.
If you loose the wrx, those properties revert back to default values. You should check-in the wrx files into your version control system together with the source .
To copy the wrx to the rcode directory I use robocopy.
suppose your sources are in a directory named src then you can copy them using
robocopy src rcode *.wrx /s
The wrx-file is generated when compiling in the AppBuilder.
See this entry in the Progress Knowledge base
I've been desperately looking for the answer to this and I feel I'm missing something obvious.
I need to copy a folder full of data files into the TARGETDIR of my deployment project at compile time. I can see how I would add individual files (ie. right click in File System and go to Add->File) but I have a folder full of data files which constantly get added to. I'd prefer not to have to add the new files each time I compile.
I have tried using a PreBuildEvent to copy the files:
copy $(ProjectDir)..\Data*.* $(TargetDir)Data\
which fails with error code 1 when I build. I can't help but feel I'm missing the point here though. Any suggestions?
Thanks in advance.
Graeme
Went to this route.
Created a new project (deleted the default source file Class1)
Added the files/folders necessary to the project.
Added the project as project output in the installer, choosing the option content files.
This removes the complexity of having to zip/unzip the files as suggested earlier.
Try
xcopy $(ProjectDir)..\Data\*.* $(TargetDir)Data /e /c /i [/f] [/r] /y
/e to ensure tree structure fulfilment (use /s if you want to bypass empty folders)
/c to continue on error (let the build process finish)
/i necessary to create the destination folder if none exists
/y assume "yes" for overwrite in case of previously existing files
[optionnal]
/f if you wanna see the whole pathes resulting from the copy
/r if you want to overwrite even previously copied read-only files
The method is simpler on the project than on files, yes. Beside, on files, it copies only the modified/missing files on each build but forces you to maintain the project on each data pack modification. Depends on the whole data size and the variability of your data pack.
Also beware the remaining files if you remove some from your data pack and rebuild without emptying your target folder.
Good luck.
I solved the problem by a workaround:
Add a build action of packaging entire directory (could be filtered) to a ZIP file.
Add a reference to an empty ZIP file to deployment project.
Add a custom action to deployment project to extract the ZIP to destination folder.
It's simple and stable.
Your error is probably because your path has spaces in it and you don't have the paths in quotes.
ex copy "$(ProjectDir)..\Data*.*" "$(TargetDir)Data\"
I need to do a similar thing. Thinking a custom action...
I found a different workaround for this. I added a web project to my solution that points at the data directory I want included in the deployment project. The web project automatically picks up any new files in the data directory and you can refer to the project content in the deployment project.