How can I fetch a list of files from local Windows share (that requires credentials)? Is there anywhere I can specify a username/password to authenticate to that share location? using talend.
I don't know if it's an option for you because you didn't provide many context information.
But you could use the tSystem component to execute a command like this :
net use x: \\share\some /user:username password
Then use the tFileList component in order to retrieve the files from that directory.
I think you can mapped shared drive on your machine(where you want to execute the Talend job) then used that mapped drive location to get the files.
Related
I want to export a database file that is created through a query, from the AS400 machine to my pc in the form of a csv file.
Is there a way to create that connection of the AS400 and my pc through a cl program?
An idea of what I want to do can be derived from the following code:
CLRPFM DTABASENAME
RUNQRY QRY(QRYTEST1)
CHGVAR VAR(&PATH) VALUE('C:\TESTS')
CHGVAR VAR(&PATH1) VALUE('C:\TESTS')
CHGVAR VAR(&CMD) VALUE(%TRIM(&PATH) *CAT '/DTABASENAME.CSV' !> &PATH !> &PATH1)
STRPCO PCTA(*YES)
STRPCCMD PCCMD(&CMD) PAUSE(*YES)
where I somehow get my database file, give the path that I want it to be saved in, in my pc , and lastly run the pc command accordingly
Take a look at
Copy From Query File (CPYFRMQRYF)
Which will allow you to create a database physical file from the query.
You may also want to look at
Copy To Import File (CPYTOIMPF)
Which will copy data from a database physical file to an Integrated File System (IFS) stream file (such as .CSV); which are the type of files you'd find on a PC.
ex:
CPYTOIMPF FROMFILE(MYLIB/MYPF) TOSTMF('/home/myuser/DTABASENAME.CSV') RCDDLM(*CRLF) DTAFMT(*DLM) STRDLM(*DBLQUOTE) STRESCCHR(*STRDLM) RMVBLANK(*TRAILING)
FLDDLM(',')
However, there's no single command to transfer data to your PC. Well technically, I suppose that's not true. If you configure a (SMB or NFS) file share on your PC and configure the IBM SMB or NFS client; you could in fact CPYTOIMPF directly to that file share or use the Copy Object (CPY) command to copy from the IFS to the network share.
If your PC has an FTP server available, you could send the data via the IBM i's FTP client. Similarly, if you have a SSH server on your PC, OpenSSL is available via PASE and SFTP or SCP could be used. You could also email the file from the i.
Instead of trying to send the file to your PC from the i. An easier solution would be to kick off a process on the PC that runs the download. My preference would be a Access Client Solution (ACS) data transfer.
You configure and save (as a .dtfx file) the transfer
Then you can kick it off with a
STRPCCMD cmd('java -jar C:\ACS\acsbundle.jar /plugin=download C:\testacs.dtfx')
More detailed information can be found in the Automating ACS Data Transfer document
The ACS download compoent is SQL based, so you could probably remove the need to use Query/400 at all
Assuming that you have your IFS QNTC mapped to your network domain. You could use the command CPYTOIMPF to copy the data directly from an IBMI DB2 file to a network directory.
This sample would result in a CSV file.
CPYTOIMPF FROMFILE(file) TOSTMF('//QNTC/servername or ip/path/filename.csv') STMFCCSID(*PCASCII) RCDDLM(*CRLF) STRDLM(*NONE)
Use the FLDDLM(';') option in addition to make semicolon separated values, omit it to use comma as value separator.
I have a Talend job that searches a directory and then uploads it to our database.
It's something like this: dbconnection>twaitforfile>tfilelist>fileschema>tmap>db
I have a subjobok that then commits the data into the table iterates through the directory and movies files to another folder.
Recently I was instructed to change the directory to a shared network path using the same components as before (I originally thought of changing components to tftpfilelist, etc.)
My question being how to direct it to the shared network path. I was able to get it to go through using double \ but it won't read any of the new files arriving.
Thanks!
I suppose if you use tWaitForFile on the local filesystem Talend/Java will hook somehow into the folder and get a message if a new file is being put into it.
Now, since you are on a network drive first of all this is out of reach of the component. Second, the OS behind the network drive could be different.
I understand your job is running all the time, listening. You could change the behaviour to putting a tLoop first which would check the file system for new files and then proceed. There must be some delta check in how the new files get recognized.
I want to access files from other domain, so I'm using this command:
net use n: \\10.0.0.1\share /user:domain\user password
but as I have to map this drive to N: and then use it.
Is there any way that I can directly use file without mapping.
Let suppose I want to use get-content to open a .txt file.
New to this so please provide me way.
In the past, I've had success accessing resources on a non-domain machine by using this variant of the NET command:
net use \\nodename /user:.\username password
The above is for a machine local account, replace the . with domain name for a domain account.
It establishes credentials to access the machine without mapping a drive. After this, accessing files through powershell cmdlets using UNC naming should work.
I need to read and write(update) some remote machine file.I am able to find the remote file using WMI(System.Management) but not able to do read or updation on that.
Any help would be appreciated.
Thanks
Himanshu
The WMI doesn't have any class (or method) to read or write the content of files. You may only retrieve the metadata (FileName, Date, Size) of the files using CIM_DataFile, or do tasks like Copy, Rename, Delete or Compress files.
RRUZ is correct: WMI cannot copy or create files over a network. This is because it would require credential "hopping":
http://msdn.microsoft.com/en-us/library/windows/desktop/aa389288%28v=vs.85%29.aspx
However, a workaround was recently created by Stackoverflow.com user Frank White in C#, and the WMI logic ports directly to VBS. Here's his solution:
WMI remote process to copy file
I ported it to a fully working VBScript:
https://stackoverflow.com/a/11948096/1569434
First check your file access in premmisions and set user "Everyone" to Full Control
then try it again.
I have a .twb file created using the excel data source that is located in some network file path.
On publishing to server I get this error.
How to resolve this?
I need to provide some access for "Run as user" in my server machine,how could I do that ?
Ensure that when the workbook is created, a connection to the Excel file is created using the full UNC path
Ensure that the Tableau Server Run As user is able to access that file using the UNC path specified in the workbook. An easy way to test this would be to open Windows Explorer as the Run As user, then paste in the UNC path. If the Excel file opens, then you're good to go.
If you still have issues, test access to the file while logged into the Tableau server to make sure there is no firewall or port blocking access from that host. Often Excel isn't installed on servers for security reasons, so the test is not whether Excel opens the file, but whether you can view the contents from the server as the Run As User, even just using the type command at the console.
If you can't get your network access fixed, an alternative is to build and publish extracts to the server. There are multiple ways to accomplish that