I'm trying to ftp files from a directory containing subdirectories and I'd like the directory structure/hierarchy to be respected on the server.
The job looks like this :
With the various components set like this :
and I've got a NullPointer Exception when accessing the row1.localPath variable (even if when tLogRow I see the right information I need).
You could link the tFileList directly to the tFTPPut component(iterate link), using tFileList_1_CURRENT_FILEPATH as the local directory in tFTPPut.
Related
Through the Github API, I can access files at the root of the repository just fine, using this URL structure:
https://api.github.com/repos/kevindecent/decent-salesforce/contents/README.md
But let's say README.md is within a subdirectory called 'files'. This does not work (I get a 'not found' error):
https://api.github.com/repos/kevindecent/decent-salesforce/contents/files/README.md
How can I access files in subdirectories?
EDIT: hitting the following gives me all files at the root, and only one of the 10 directories:
https://api.github.com/repos/kevindecent/decent-salesforce/contents
Very odd.
You have to hit the raw file. It looks like your example is just that, so here's a real file:
https://raw.githubusercontent.com/stefangabos/Zebra_Mptt/master/composer.json
Hi I'm getting the following error....
GET METADATA works fine if I do WildCard like ASN to find list of files when I do for loop and pass each file name in the COPY Activity (Source)
ErrorCode=SftpPathNotFound,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Can't find SFTP path '/Receive/INX_XXXXXXXX_ASN_20210728012200817116546932367669276.xml'. Please check if the path exists. If the path you configured does not start with '/', note it is a relative path under the given user's default folder ''.,Source=Microsoft.DataTransfer.ClientLibrary.SftpConnector,''Type=Renci.SshNet.Common.SftpPathNotFoundException,Message=/opt/apps/uprd1bpn1/bpfs2/2021/July/0209/04/22/25565817aec33de9cnode2_WF128489406.dat (No such file or directory),Source=Renci.SshNet,'
Any ideas?
I was facing the same issue described above, the only difference was the type of my source file, .txt.
In my case the error message was misleading, the actual error message was getting hidden by the 'chunking' option. If you pay attention to the source tab of the copy activity, by default it is set to use 'chunking', which transfers your data at an increased speed.
By disabling chunking, I could notice the error was actually in a particular line of the file, then fixing the file, fixed the problem.
1. Check if "Copy behavior" on the sink side is marked as "None".
2. Check if you have permissions for uploading the files.
3. For wildcardFileName Dynamic content of fileName should be replaced with asterisk (*).
E.g. If 20210728012200817116546932367669276 is dynamic content in your filename
then wildcardFileName for INX_XXXXXXXX_ASN_20210728012200817116546932367669276.xml should be INX_XXXXXXXX_ASN_*.xml
Refer - SFTP as sink - getting can't find SFTP path error
I would like to be able to create a list ,array, of file names on a folder so that I can use PeopleCode to loop through them and delete files that match a pattern and are in a date range.
I'm pretty sure I have the last half of that, matching a pattern and in a date range, but I do not know how to get the list on remote servers. I can do it on our local servers, but not remote ones.
I had hoped that this would work:
Local object &files = CreateJavaObject("java.io.File", SFO_DEL_FTP_AET.FTPDIRECTORY | "*.*");
But I don't think it is working.
Can somebody help me?
Thanks,
JPS
You can use Java to access/modify the files in a directory. Try:
Local JavaObject instead of Local object
We created a PS component to view, upload, and delete files in an App Server directory. You can see how we did it here:
https://github.com/cy2hq/PeopleSoft-Directory-Viewer
I am configuring Blast+ on my mac (os sierra) and am having trouble configuring my nr and nt databases that I also downloaded locally. I am trying to follow NCBI's instructions here, and am getting hung up on the Configuration and Example Execution steps.
They say to change my .bash_profile so that it says:
export PATH=$PATH:$HOME/Documents/Luke/Research/Pedulla\ 17-18/blast/ncbi-blast-2.6.0+/bin
That works fine, and they say configure a path for BLASTDB "similarly" but to the file where my DB will be, so I have done this:
export BLASTDB=$BLASTDB:$HOME/Documents/Luke/Research/Pedulla\ 17-18/blast/blastdb/nt.00
which specifies the exact folder that I got when I unzipped the nt tar file from their FTP. With this path, if I run the command...
blastn -query test_query.fa -db nt.00 -task blastn -outfmt "7 qseqid sseqid evalue bitscore" -max_target_seqs 5
then it runs successfully and I get results, but I am worried that these are only being checked against the nt.00 section of the entire nt.00 database file, especially because if I run my test_query.fa sequence on the Web Blast, I get different results.
Also, their instructions say that the path only needs to point to the folder that contains the whole database folder nt.00, from the tar I unzipped--and not the specific nt.00 itself--, which in my case would just be "blastdb/" (As opposed to "blastdb/nt.00/" which then contains nt.00.nhd, nt.00.nal, etc.). That makes sense because when I am working I want to be able to blastn on the nt database but also blastp on the nr one, etc. by changing the -db flag on my command, and there shouldn't be a problem with having them all in this folder, right? But if I must specify the path for BLASTDB with the nt.00 DB added to the end, how could I ever use nr.00 in the same folder (blastdb/)? Essentially, I want to do as the instructions say, and just have this:
export BLASTDB=$BLASTDB:$HOME/Documents/Luke/Research/Pedulla\ 17-18/blast/blastdb/
And then depending on what database I want to use I could just say so after the -db flag on my command. But when I make the path like that above, it gives me this error:
BLAST Database error: No alias or index file found for nucleotide database [nt] in search path [/Users/LJStout::/Users/LJStout/Documents/Luke/Research/Pedulla 17-18/blast/blastdb:]
I have tried running that same blastn command from above and swapping out "nt" for "nt.00", and have tried these commands with the path for BLASTDB ending in both "blastdb/" and "blastdb/nt" and of course "blastdb/nt.00" which is the only one that runs without errors.
Here's an example of another thread I read where the OP is worried about his executions not checking the entire nt.00 folder, this was different than my problem however.
Thanks for you help!
This whole problem came down to having the nt.00 & nr.00 files, the original folders that result from unzipping their respective .tar.gz's, in the same parent folder when it should be that their contents are in the same parent folder. I simply deleted the folders they came in and copied the contents over to my new, singular parent. I was kind of mislead by the instructions, it was a simple mistake. Now, I have one folder, blastdb/ that contains all of the contents of every database I plan on using, including nt,nr, and refseq.
My team faces the need to encrypt all files in a repository with AES256. For this purpose, we decided we are going to zip all files with such encryption, using the same key for all of them.
The problem we have is that these files sit in a NAS, so from windows boxes they are accessible by \ to them.
The directory structure is something like this:
Original Structure:
Root
-1
|--folder1
|---file1.ext
|---file2.ext
|--folder2
|---filea.ext
|---fileb.ext
|--folder2.a
|---filec.ext
and so on...
Essentially, what we need is to have all the original files contained in a zip file, keeping their original names, which would be something like this:
Desired Outcome:
|-Root
|-1
|--folder1
|---file1.zip
|---file2.zip
|--folder2
|---filea.zip
|---fileb.zip
|--folder2a
|---filec.zip
and so on...
To accomplish this, we tried a batch script that calls 7zip, but it only works if it's run from the root directory, which is something we cannot use as the files are not in a server.
Here is the syntax of the batch script we came up with:
FOR /R %%i IN ("*.wmv") DO "C:\Program Files\7-Zip\7z.exe" a -mx0 -tzip -pPasswordHere "%%~dpni.zip" "%%i"
But, as wrote previously, it only works when run from the root folder, which is something we cannot do as files sit on a network location.
Mapping the drive or making a symbolic link to it doesn't do the trick either.
I've also checked on 7zip to do this, namely, making use of its "-r" operator, but I couldn't find a way to get the desired outcome (namely, recurse through all folders in the remote tree structure -there are a lot of them...- and keep the original file name).
I'm open to any suggestions as any kind of script, trick or guizmo that gets the job done will be more than welcome. =)
Thanks a million in advance!,
Sebas.
----SOLUTION----
I actually found a sollution here, mapping the drive in a different way (it's so simple it just made me feel stupid(er), but it's altogheter beautiful).
Using the batch script below, the remote share can be mapped like so:
You can map a drive using
net use X: \\server\directory
and then you can change to that directory using
pushd X:
(Post from which the answer was taken from: Batch File Iterating through files on a local network server)