Azure Data Factory SFTP Source Please check if the path exists. If the path you configured does not start - azure-data-factory

Hi I'm getting the following error....
GET METADATA works fine if I do WildCard like ASN to find list of files when I do for loop and pass each file name in the COPY Activity (Source)
ErrorCode=SftpPathNotFound,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Can't find SFTP path '/Receive/INX_XXXXXXXX_ASN_20210728012200817116546932367669276.xml'. Please check if the path exists. If the path you configured does not start with '/', note it is a relative path under the given user's default folder ''.,Source=Microsoft.DataTransfer.ClientLibrary.SftpConnector,''Type=Renci.SshNet.Common.SftpPathNotFoundException,Message=/opt/apps/uprd1bpn1/bpfs2/2021/July/0209/04/22/25565817aec33de9cnode2_WF128489406.dat (No such file or directory),Source=Renci.SshNet,'
Any ideas?

I was facing the same issue described above, the only difference was the type of my source file, .txt.
In my case the error message was misleading, the actual error message was getting hidden by the 'chunking' option. If you pay attention to the source tab of the copy activity, by default it is set to use 'chunking', which transfers your data at an increased speed.
By disabling chunking, I could notice the error was actually in a particular line of the file, then fixing the file, fixed the problem.

1. Check if "Copy behavior" on the sink side is marked as "None".
2. Check if you have permissions for uploading the files.
3. For wildcardFileName Dynamic content of fileName should be replaced with asterisk (*).
E.g. If 20210728012200817116546932367669276 is dynamic content in your filename
then wildcardFileName for INX_XXXXXXXX_ASN_20210728012200817116546932367669276.xml should be INX_XXXXXXXX_ASN_*.xml
Refer - SFTP as sink - getting can't find SFTP path error

Related

postgreSQL COPY TO is producing a corrupted file

COPY sqllearning.superstore_people
TO 'C:\Windows\Temp\sup_people.csv'
WITH DELIMITER ','
CSV HEADER;
When I open the csv file either in notepad or excel I get an Error that says:
Excel cannot access 'file.csv', the document may be read only or encrypted.
file.csv cannot be accessed. the file may be corrupted, located on a server that is not responding, or read- only
Any tips on how to resolve this would be appreciated. Thanks!
I faced similar issue before. I solved it by checking and setting ownership of the entire folder and all its subfolders and files for current user (in Windows - right click on the folder - properties - security). Maybe it would help for you also

Azure DevOps: 'Replace Tokens task' having problem setting variable files from JSON

I'm trying to setup a 'Replace Tokens Task' for my build pipeline from a JSON file that will hold environmental variables as follow:
And then my JSON environment file:
The JavaScript target file to replace the tokens in:
Unfortunately, I'm getting the following error:
What am I missing here?
What am I missing here?
I can reproduce same issue on my side, and I think it only indicates there's something wrong with your Variable files(JSON) input:
Please make sure the variables are successfully loaded from the variable json file.
1.Make sure your env-config.json file has content below:
{
"alambda.endpoint":"ForTest"
}
2.Check carefully about the path of your env-config.json file! (In my opinion, your issue is more like a path issue~) If the task can't find and load the file, it also throws variable not found. For this:
Make sure the relative path is correct.
Try cut the long path, I'm not sure if the issue results from the too long path. Try using **/env-config.json instead of the very long path.
If above tips can't work, I suggest you can move the env-config.json file to root folder $(System.DefaultWorkingDirectory) with Copy Task or what. (About System.DefaultWorkingDirectory see here). Then try it again with **/env-config.json.

How to overwrite existing files with ncftpget?

When getting a remote ftp file that exists in the local destination
ncftpget says that
local file appears to be the same as the remote file, download is not necessary.
What does appears mean? How does ncftpget check if this is the same file?
It seems that it compares time and size of the file. But does it compare content or at least checksum?
Is there a way to force to overwrite the existing file. Of course other than remove it first.

postgreSQL COPY command error

Hallo everyone once again,
I did various searches but couldn't gind a suitable/applicable answer to the simple problem below:
On pgAdminIII (Windows 7 64-bit) I am running the following command using SQL editor:
COPY public.Raw20120113 FROM 'D:\my\path\to\Raw CSV Data\13_01_2012.csv';
I tried many different variations for the path name and verified the path, but I keep getting:
ERROR: could not open file "D:\my\path\to\Raw CSV Data\13_01_2012.csv" for reading: No such file or directory
Any suggestions why this happens?
Thank you all in advance
Petros
UPDATE!!
After some tests I came to the following conclusion: The reason I am getting this error is that the path includes some Greek characters. So, while Windows uses codepage 1253, the console is using 727 and this whole thing is causing the confusion. So, some questions arise, you may answer them if you like or prompt me to other questions?
1) How can I permanently change the codepageof the console?
2) How can I define the codepage is SQL editor?
Thank you again, and sorry if the place to post the question was inappropriate!
Try DIR "D:\my\path\to\Raw CSV Data\13_01_2012.csv" from command line and see if it works - just to ensure that you got the directory, file name, extension etc correct.
The problem is that COPY command runs on server so it takes the path to the file from the server's scope.
To use local file to import you need to use \COPY command. This takes local path to the file into account and loads it correctly.

'findstr' is not recognized as an internal or external command,

I got the following error while starting JBoss from a command line prompt today:
'findstr' is not recognized as an internal or external command
Please google it, you can find a lot of answers. But do as below to fix it. Add the following value to Right Click My Compuer -> Advanced -> Environment Variables -> System Variables -> Select Path variable -> append the below value.
C:\WINDOWS\system32
It should work with that change.
As others pointed, issue is in wrong settings of PATH variable in Windows.
According to article this is most probably because some stupid installer wrongly modified PATH variable in Windows registry. Registry has 2 different string value types - REG_SZ and REG_EXPAND_SZ. Only the second one allows for expansion of %SystemRoot%.
So check your path by typing set path in command prompt. If you see unexpanded %SystemRoot% and other variables in Path, you are affected (PATH should show only plain directory names, not variables).
You need to edit Path variable in registry: HKEY_CURRENT_USER\Environment and HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment. As it is not possible to change the type of key, save the path value somewhere, delete the key and re-create it with type REG_EXPAND_SZ. You need to logout for changes to take effect.
for me it works when I've coped findstr(from windows/system32) to wildfly/bin
Please go throught the simplest steps:-
go to C:\Windows\system32\ and copy findstr.exe file.
paste this file into the location C:\Program Files\Java\jdk1.6.0_24\bin
Run your jboss again you will get out of this.....
Check to see if you %SystemRoot% is evaluating (type set path into a command prompt, you should not see %SystemRoot%, but instead that actual path). If your path variable's (user, or systems) first entry begins with an %(an environment variable) this can cause an issue.
To resolve this, simply swap this first entry with anything else in your path that does not lead with an environment variable.
You can also hard code the directory by replacing 'findstr' with 'C:\Windows\system32\findstr'. This is useful when using systems with restricted user permissions.
I have try to work with play framework but stuck with to run activator.bat file but solution is the same just copy file from windows/system32/findsr and past it to under stuck folder then run the respective file again.
thanks to andrewsiand Suryaprakash
Please beware that current Windows systems use a Capital "S" for the System directory, so:
C:\WINDOWS\System32
%SystemRoot%\System32
Omitting the capital S will result in a neglect of the line in the %PATH%
In my case (not JBoss related) the following helped to fixed this error.
Instead of:
SET path="%path%;C:\some\additional\path"
I used:
SET "path=%path%;C:\some\additional\path"
For IBM ACE solution for
'findstr' is not recognized as an internal or external command,
Go to the path C:\Windows\System32
Find the findstr.exe, copy it and then find the path where you bin file of your application is found. eg C:\Program Files\IBM\ACE\11.0.0.12\server\bin then past it inside the bin file
cancel the console of ace and re-open it.
Then run ACE toolkit command on ace console.
Then press enter, now it can open.