I have the following problem:
I am using the following command:
EXPORT TO "D:\ExportFiles\ACTIVATE_DICT.csv" OF DEL MODIFIED BY TIMESTAMPFORMAT="YYYY/MM/DD HH:MM:SS" STRIPLZEROS MESSAGES "D:\ExportFiles\FMessage.txt" SELECT * FROM DB2INST4.ACTIVATE_DICT;
In the Command Editor of the program, the Control Center successfully exported data from the ACTIVATE_DICT table to a CSV file ACTIVATE_DICT.csv.
But for a number of reasons, I need you to execute this command in the IBM Data Studio or DataGrip program, and there it cannot be executed in this form.
Therefore, I read the following manual enter link description here
and based on it wrote the following command:
CALL SYSPROC.ADMIN_CMD('EXPORT to /lotus/ExportFiles/ACTIVATE_DICT.csv OF DEL MODIFIED BY TIMESTAMPFORMAT="YYYY/MM/DD HH:MM:SS" STRIPLZEROS MESSAGES /lotus/ExportFiles/FMessage.txt SELECT * FROM DB2INST4.ACTIVATE_DICT');
Here is the message on the result of the command:
[2018-10-11 15:15:23] [ ][3107] There is at least one warning
message in the message file.. SQLCODE=3107, SQLSTATE= ,
DRIVER=4.23.42 [2018-10-11 15:15:23] 1 row retrieved starting from 1
in 75 ms (execution: 29 ms, fetching: 46 ms)
And in the / lotus / ExportFiles / directory there is no ACTIVATE_DICT.csv file and there is no FMessage.txt file in the / lotus / ExportFiles / directory.
Question: How then to correctly execute this command ??? Maybe I'm doing something wrong?
sqlcode 3107 is a warning message:
SQL3107W At least one warning message was encountered during LOAD processing.
Explanation
You can load data into a database from a file, tape, or named pipe using the LOAD command. You can specify that any warnings or errors from the LOAD processing be printed to a message file. If no message file is specified, the warnings or errors are printed to standard out (unless the database manager instance is configured as a partitioned-database environment.)
It is to tell you to read message log in the message file you specified. In your case: /lotus/ExportFiles/FMessage.txt
Please read into the file to see what error is logged and if you need help understand what is logged, please post the content of the file.
This message is returned when at least one warning was received during processing. If a message file is being used, the warnings and errors will be printed there.
This warning does not affect processing.
User response
Review the message file warning.
EXPORT command using the ADMIN_CMD procedure
See use of the 'MESSAGES ON SERVER' clause, and how to get these messages using the result set returned by this routine in this case.
Related
I have written PySpark code to hit a REST API and extract the contents in an XML format and later wrote to Parquet in a data lake container.
I am trying to add logging functionality where I not only write out errors but updates of actions/process we execute.
I am comparatively new to Spark I have been relying on online articles and samples. All explain the error handling and logging through "1/0" examples and saving logs in the default folder structure (not in ADLS account/container/folder) which do not help at all. Most of the code written in Pure Python doesn't run as-is.
Could I get some assistance with setting up the following:
Push errors to a log file under a designated folder sitting under a data lake storage account/container/folder hierarchy".
Catching REST specific exceptions.
This is a sample of what I have written:
''''
LogFilepath = "abfss://raw#.dfs.core.windows.net/Data/logging/data.log"
#LogFilepath2 = "adl://.azuredatalakestore.net/raw/Data/logging/data.log"
print(LogFilepath)
try:
1/0
except Exception as e:
print('My Error...' + str(e))
with open(LogFilepath, "a") as f:
f.write("An error occured: {}\n".format(e))
''''
I have tried it both ABFSS and ADL file paths with no luck. The log file is already available in the storage account/container/folder.
I have reproduced the above using abfss path in with open() function but it gave me the below error.
FileNotFoundError: [Errno 2] No such file or directory: 'abfss://synapsedata#rakeshgen2.dfs.core.windows.net/datalogs.logs'
As per this Documentation
we can use open() on ADLS file with a path like /synfs/{jobId}/mountpoint/{filename}.
For that, first we need to mount the ADLS.
Here I have mounted it using ADLS linked service. you can mount either by Storage account access key or SAS as per your requirement.
mssparkutils.fs.mount(
"abfss://<container_name>#<storage_account_name>.dfs.core.windows.net",
"/mountpoint",
{"linkedService":"<ADLS linked service name>"}
)
Now use the below code to achieve your requirement.
from datetime import datetime
currentDateAndTime = datetime.now()
jobid=mssparkutils.env.getJobId()
LogFilepath='/synfs/'+jobid+'/synapsedata/datalogs.log'
print(LogFilepath)
try:
1/0
except Exception as e:
print('My Error...' + str(e))
with open(LogFilepath, "a") as f:
f.write("Time : {}- Error : {}\n".format(currentDateAndTime,e))
Here I am writing date time along with the error and there is no need to create the log file first. The above code will create and append the error.
If you want to generate the logs daily, you can generate date file names log files as per your requirement.
My Execution:
Here I have executed 2 times.
I have been trying to import a csv file into a table I created. I have set each column with data type "text" and believe to have the parameters configured correctly (delimiter, etc..)
I keep receiving the "Failed: Exit code -6" but with no explicit error message. Any ideas?
enter image description here
I checked the log file and I think this is the part that caused the problem:
Setting up database
[15:30:54] Configuring pg11 to point to existing data dir D:\John's Files\My Documents\Code\Databases\PostgreSQL\data\pg11
Setting PostgreSQL port
= 5432
Executing C:\Installed Software\Developer Software\PostgreSQLv11.1/pgc config pg11 --datadir "D:\John's Files\My Documents\Code\Databases\PostgreSQL\data\pg11"
Script exit code: 1
Script output: ################################################
# FATAL SQL Error in check_release
# SQL Message = near "s": syntax error
# SQL Statement = SELECT r.component FROM releases r, versions v
WHERE r.component = v.component
AND r.component LIKE '%D:\John's Files\My Documents\Code\Databases\PostgreSQL\data\pg11%' AND v.is_current = 1
################################################
Script stderr: Program ended with an error exit code
Error with configuration or permissions. Please see log file for more information.
Problem running post-install step. Installation may not complete correctly
Error with configuration or permissions. Please see log file for more information.
I think the problem is the apostrophe in "John's". Does anyone know if that's right? Is there a fix to this problem? I don't want to rename my directory because Postgresql can't handle apostrophes.
I am using unoconv to convert an ods spreadsheet to a csv file.
Here is the command:
unoconv -vvv --doctype=spreadsheet --format=csv --output= ~/Dropbox
/mariners_site/textFiles/expenses.csv ~/Dropbox/Aldeburgh/expenses
/expenses.ods
It saves the output file in the same directory as the source file, not in the specified directory. The error message is:
Output file: /home/richard/Dropbox/mariners_site/textFiles/expenses.csv
unoconv: UnoException during export phase:
Unable to store document to file:///home/richard/Dropbox/mariners_site
/textFiles/expenses.csv (ErrCode 19468)
I'm sure that this worked initially, but it has since stopped.
I have checked for permissions and they are identical for both directories.
I translated ErrCode 19468 for you and it boils down to meaning ERRCODE_SFX_DOCUMENTREADONLY.
You can find more information about the specific meaning of LibreOffice ErrCode numbers from the unoconv documentation at: https://github.com/dagwieers/unoconv/blob/master/doc/errcode.adoc
The clue here is that you have a whitespace-character between --output= and the filename (--output= ~/Dropbox
/mariners_site/textFiles/expenses.csv) and because of that unoconv gets an empty output value (which means the current directory) and is given 2 files. And that explains why you get this specific error IMO
First, Let me thank for the log4cplus source code.
I am facing one issue as follows:
What I am trying to do?
I want to log the messages to event log on windows.
What did I do?
I could get the event logging enabled by in including following .h
#include <log4cplus/nteventlogappender.h>
And creating the appender as follows:
SharedAppenderPtr append_3(new NTEventLogAppender(LOG4CPLUS_TEXT("127.0.0.1"), LOG4CPLUS_TEXT("log"), LOG4CPLUS_TEXT("source")));
append_3->setName(LOG4CPLUS_TEXT("ToEventlog"));
Logger to_eventlog = Logger::getInstance(LOG4CPLUS_TEXT("to_eventlog"));
to_eventlog.addAppender(append_3);
to_eventlog.setLogLevel(log4cplus::ALL_LOG_LEVEL);
And logging as
Logger to_eventlog = Logger::getInstance(LOG4CPLUS_TEXT("to_eventlog"));
LOG4CPLUS_FATAL(to_eventlog, “Test Message.”);
When I log the message, I get following in the event log:
The description for Event ID 4096 from source source cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
Test Message.
the message resource is present but the message is not found in the string/message table
I tried to create a dll with name NTEventLogAppender.dll with string table resource and a string with id 4096 (as hardcoded in the nteventlogappender.cxx file in log4cplus) and compiled it with resource only dll (/NOENTRY) but I still keep getting above error.
I already tried question to log4cplus-devel#lists.sourceforge.net but did not get any answer yet.
Any help on how to create this dll would be greatly appreciated.
Thanks.
This is how I resolved this issue:
comipiled NTEventLogAppender.mc file using commands:
mc -U NTEventLogAppender.mc
rc -r NTEventLogAppender.rc
link -dll -noentry -out:NTEventLogAppender.dll NTEventLogAppender.res
where NTEventLogAppender.mc has following contents
MessageIdTypedef=DWORD
SeverityNames=(Success=0x0:STATUS_SEVERITY_SUCCESS
Informational=0x1:STATUS_SEVERITY_INFORMATIONAL
Warning=0x2:STATUS_SEVERITY_WARNING
Error=0x3:STATUS_SEVERITY_ERROR
)
FacilityNames=(System=0x0:FACILITY_SYSTEM
Runtime=0x2:FACILITY_RUNTIME
Stubs=0x3:FACILITY_STUBS
Io=0x4:FACILITY_IO_ERROR_CODE
)
LanguageNames=(English=0x409:MSG00409)
; // The following are message definitions.
MessageId=0x1000
SymbolicName=SVC_TEST
Language=English
A message for something.
.
; // A message file must end with a period on its own line
; // followed by a blank line.
copied the NTEventLogAppender.dll to c:\windows\system32
and ran the above mentioned test program and found that event log found the message resource correctly.
you need to register this dll by copying following in .reg file and running the same
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\eventlog\log\source]
"EventMessageFile"="C:\\windows\\system32\\NTEventLogAppender.dll"
"CategoryMessageFile"="C:\\windows\\system32\\NTEventLogAppender.dll"
"TypesSupported"=dword:00000007
"CategoryCount"=dword:00000005