VBS script not printing the value of a variable in a file - rman

I need to generate a RMAN backup script dynamically and stored in a file so it can be executed later to backup database and also delete the old backups older than defined retention period.
I am passing the value of retention period in a variable inside a script where it will generate RMAN script as shown below;
connect target /
set echo off;
run {
allocate channel c1 device type disk;
allocate channel c2 device type disk;
allocate channel c3 device type disk;
crosscheck archivelog all;
backup full
filesperset 5
format 'D:\Oracle\DB_Backup\P7app\daily_P7app_BACKUP_DBF__20221228_140306_%s_%p_%c'
tag daily_P7app_BACKUP
database
plus archivelog
filesperset 5
tag daily_P7app_BACKUP;
backup current controlfile
filesperset 5
format 'D:\Oracle\DB_Backup\P7app\daily_P7app_BACKUP_CF__20221228_140306_%s_%p_%c'
tag daily_P7app_BACKUP;
backup spfile
filesperset 5
format 'D:\Oracle\DB_Backup\P7app\daily_P7app_BACKUP_SP__20221228_140306_%s_%p_%c'
tag daily_P7app_BACKUP;
sql 'alter database backup controlfile to trace';
release channel c1;
release channel c2;
release channel c3;
}
delete noprompt obsolete;
delete noprompt expired backup;
The script assigns value of strretention based on the month end, year end.... but while generating the delete statement it is not printing the value of "stretention" hence the below statement is incomplete where it should be
delete noprompt backup completed before 'sysdate - "value of stretention" tag daily_P7app_BACKUP;
but unfortunately the value of stretention is not printed on the generated statement where it is an incomplete statement which is as;
delete noprompt backup completed before 'sysdate - tag daily_P7app_BACKUP;
release channel;
exit
Any idea why the value of a variable is missing?
I was expecting;
delete noprompt backup completed before 'sysdate -35';
but I got below incomplete statement with missing value of strretention;
delete noprompt backup completed before 'sysdate - ';

Related

Problems encountered in recovering three tables from a dump file

I'm trying to restore tables from a dump file. It's illustrated by a footnote in the paper "VCCFinder-Finding Potential Vulnerabilities in Open-Source Projects to Assist Code Audits", that the dump file that the team created with pg_dump could be read with pg_restore. As it's shown in paper footnote with red line to emphasize. That's where I've started.
1. Use pg_restore command
By typing the command mentioned in your paper: VCCFinder: Finding Potential Vulnerabilities in Open-Source Projects to Assist Code Audits:
pg_restore -f vcc_base I:\OneDrive\PractiseProject\x_prjs\m_firmware_scan\m_firmware_scan.ref\vcc-database\vccfinder-database.dump
Windows CMD had returned an error message:
pg_restore: error: input file appears to be a text format dump. Please use psql.
As I had tried the operation in different version, including v14.4, v9.6, v9.4 and v9.3, the outcome is the same error message.
2.Use psql command
Then I turned to another direction: using psql. After typing command,
psql -v ON_ERROR_STOP=1 -U postgres < I:\OneDrive\PractiseProject\x_prjs\m_firmware_scan\m_firmware_scan.ref\vcc-database\vccfinder-database.dump
apart from postgreSQL 14.4 environment, the returned error message is:
psql: SCRAM authentication requires libpq version 10 or above
Under postgreSQL 14.4 environment, the returned message became:
SET
SET
SET
SET
SET
SET
ERROR: schema "export" already exists
If I remove the -v ON_ERROR_STOP=1 option, and returned message would be like this:
SET
SET
SET
SET
SET
SET
ERROR: schema "export" already exists
SET
SET
SET
ERROR: type "public.hstore" does not exist
LINE 27: patch_keywords public.hstore
^
ERROR: relation "cves" already exists
ERROR: relation "repositories" already exists
ERROR: relation "commits" does not exist
invalid command \n
invalid command \N
invalid command \N
...
(Solved) I have tried to solve the unreadable code problem shown in above error messages by typing chcp 65001, chcp 437 and etc to change character set into UTF8 or American English in Windows CMD, but it's not helpful. But after viewing the source code of the dump file in Visual Studio, it's not difficult to infer that those error messages were caused by psql commands in the dump file.
After the error messages became understandable, I focused on one particular error message:
ERROR: type "public.hstore" does not exist
LINE 27: patch_keywords public.hstore
So I manually created a "hstore" type below the "pulic SCHEMA", after that error messages turned into these:
SET
SET
SET
SET
SET
SET
SET
ERROR: schema "export" already exists
SET
SET
SET
ERROR: relation "commits" already exists
ERROR: relation "cves" already exists
ERROR: relation "repositories" already exists
ERROR: malformed record literal: ""do"=>"1", "if"=>"0", "asm"=>"41", "for"=>"5", "int"=>"13", "new"=>"0", "try"=>"0", "auto"=>"0", "bool"=>"0", "case"=>"0", "char"=>"1", "else"=>"0", "enum"=>"0", "free"=>"0", "goto"=>"0", "long"=>"15", "this"=>"0", "true"=>"0", "void"=>"49", "alloc"=>"0", "break"=>"0", "catch"=>"0", "class"=>"0", "const"=>"0", "false"=>"0", "float"=>"0", "short"=>"0", "throw"=>"0", "union"=>"0", "using"=>"0", "while"=>"1", "alloca"=>"0", "calloc"=>"0", "delete"=>"0", "double"=>"0", "extern"=>"4", "friend"=>"0", "inline"=>"18", "malloc"=>"0", "public"=>"0", "return"=>"4", "signed"=>"1", "sizeof"=>"0", "static"=>"32", "struct"=>"4", "switch"=>"0", "typeid"=>"0", "default"=>"0", "mutable"=>"0", "private"=>"0", "realloc"=>"0", "typedef"=>"0", "virtual"=>"0", "wchar_t"=>"0", "continue"=>"0", "explicit"=>"0", "operator"=>"0", "register"=>"0", "template"=>"0", "typename"=>"0", "unsigned"=>"23", "volatile"=>"23", "namespace"=>"0", "protected"=>"0", "const_cast"=>"0", "static_cast"=>"0", "dynamic_cast"=>"0", "reinterpret_cast"=>"0""
DETAIL: Missing left parenthesis.
CONTEXT: COPY commits, line 1, column patch_keywords: ""do"=>"1", "if"=>"0", "asm"=>"41", "for"=>"5", "int"=>"13", "new"=>"0", "try"=>"0", "auto"=>"0", "bo..."
ERROR: syntax error at or near "l022_save"
LINE 1: l022_save, pl022_load, s);
^
invalid command \n
invalid command \N
invalid command \N
...
Now the three tables have been created, but there is no content in them.
3. Install hstore
After searching for "hstore"hstore type does not exist with hstore installed postgresql, I realized that the "hstore" should be installed, but not be manually created. So I typed this in psql command line:
postgres=# create EXTENSION hstore; And there were new error messages:
SET
SET
SET
SET
SET
SET
SET
ERROR: schema "export" already exists
SET
SET
SET
CREATE TABLE
ERROR: relation "cves" already exists
ERROR: relation "repositories" already exists
ERROR: missing data for column "hunk_count"
CONTEXT: COPY commits, line 23201: "11388700 178 \N other_commit 1d6198c3b01619151f3227c6461b3d53eeb711e5\N blueswir1#c046a42c-6fe2-441..."
ERROR: syntax error at or near "l022_save"
LINE 1: l022_save, pl022_load, s);
^
invalid command \n
invalid command \N
invalid command \N
...
And still, there is no content in those three tables.
4. Generate and view tables
After looking into the source code of the dump file, and trying to fix the "hunk_count" problem but end up with failure. It occurs to me that the above error messages just caused by one paticular row of code. So I had deleted the row and the old error messages were gone but there were new error messages caused by another row. Evetually I have deleted 10 rows in total, comparing to the total row number: 351409, those deleted parts are negligible. And three tables weren't empty anymore, as it's shown in pgAdmin 4.
However, the pgADmin only demonstrated the structure of those tables, I still didn't know how to view the content in them. By refering to 2 Ways to View the Structure of a Table in PostgreSQL, I typed
SELECT
*
FROM
export.repositories/ export.cves/ export.commits
WHERE
TRUE
to generate and view corresponding tables in pgAdmin 4. For example, final cve table:
5. In the end
Looking back at these steps, these are all easy steps, but for a guy who was not familiar with the tools or operations, it could cost several days to search and type, step by step for one simple purpose. I wish this post could be useful to someone like me.
However, I am not so familiar with psql commands or anything about postgreSQL, as a matter of fact, I had never used them before. So I'm wondering if someone could point out some mistakes I may have made in those attempts, or provide some suggestions for my dilemma.
First , ensure your dump format.
Try to read header (first 5 chars) of dump file.
If it is signed as PGDMP then it is binary/custom dump else it is sql (human readable format).
- use pg_restore for binary dump import.
$ pg_restore -U postgres -d <dbname> file.dump
- use psql to import plain text sql dump.
$ psql -U postgres -d <dbname> < file.dump
Solved, as I've demonstrated above.

Error while loading parquet format file into Amazon Redshift using copy command and manifest file

I'm trying to load parquet file using manifest file and getting below error.
query: 124138ailed due to an internal error. File 'https://s3.amazonaws.com/sbredshift-east/data/000002_0 has an invalid version number: )
Here is my copy command
copy testtable from 's3://sbredshift-east/manifest/supplier.manifest'
IAM_ROLE 'arn:aws:iam::123456789:role/MyRedshiftRole123'
FORMAT AS PARQUET
manifest;
here is my manifest file
**{
"entries":[
{
"url":"s3://sbredshift-east/data/000002_0",
"mandatory":true,
"meta":{
"content_length":1000
}
}
]
}**
I'm able to load the same file using copy command by specifying the file name.
copy testtable from 's3://sbredshift-east/data/000002_0' IAM_ROLE 'arn:aws:iam::123456789:role/MyRedshiftRole123' FORMAT AS PARQUET;
INFO: Load into table 'supplier' completed, 800000 record(s) loaded successfully.
COPY
What could be wrong in my copy statement?
This error happens when the content_length value is wrong. You have to specify the correct content_length. You could check it executing an s3 ls command.
aws s3 ls s3://sbredshift-east/data/
2019-12-27 11:15:19 539 sbredshift-east/data/000002_0
The 539 (file size) should be the same than the content_lenght value in your manifest file.
I don't know why they are using this meta value when you don't need it in the direct copy command.
¯\_(ツ)_/¯
The only way I've gotten parquet copy to work with manifest file is to add the meta key with the content_length.
From what I can gather in my error logs, the COPY command for parquet (w/ manifest) might first be reading the files using Redshift Spectrum as an external table. If that's the case, this hidden step does require the content_step which contradicts their initial statement about COPY commands.
https://docs.amazonaws.cn/en_us/redshift/latest/dg/loading-data-files-using-manifest.html

For some reason, a warning is issued when calling the procedure SYSPROC.ADMIN_CMD ('EXPORT to ...')

I have the following problem:
I am using the following command:
EXPORT TO "D:\ExportFiles\ACTIVATE_DICT.csv" OF DEL MODIFIED BY TIMESTAMPFORMAT="YYYY/MM/DD HH:MM:SS" STRIPLZEROS MESSAGES "D:\ExportFiles\FMessage.txt" SELECT * FROM DB2INST4.ACTIVATE_DICT;
In the Command Editor of the program, the Control Center successfully exported data from the ACTIVATE_DICT table to a CSV file ACTIVATE_DICT.csv.
But for a number of reasons, I need you to execute this command in the IBM Data Studio or DataGrip program, and there it cannot be executed in this form.
Therefore, I read the following manual enter link description here
and based on it wrote the following command:
CALL SYSPROC.ADMIN_CMD('EXPORT to /lotus/ExportFiles/ACTIVATE_DICT.csv OF DEL MODIFIED BY TIMESTAMPFORMAT="YYYY/MM/DD HH:MM:SS" STRIPLZEROS MESSAGES /lotus/ExportFiles/FMessage.txt SELECT * FROM DB2INST4.ACTIVATE_DICT');
Here is the message on the result of the command:
[2018-10-11 15:15:23] [ ][3107] There is at least one warning
message in the message file.. SQLCODE=3107, SQLSTATE= ,
DRIVER=4.23.42 [2018-10-11 15:15:23] 1 row retrieved starting from 1
in 75 ms (execution: 29 ms, fetching: 46 ms)
And in the / lotus / ExportFiles / directory there is no ACTIVATE_DICT.csv file and there is no FMessage.txt file in the / lotus / ExportFiles / directory.
Question: How then to correctly execute this command ??? Maybe I'm doing something wrong?
sqlcode 3107 is a warning message:
SQL3107W At least one warning message was encountered during LOAD processing.
Explanation
You can load data into a database from a file, tape, or named pipe using the LOAD command. You can specify that any warnings or errors from the LOAD processing be printed to a message file. If no message file is specified, the warnings or errors are printed to standard out (unless the database manager instance is configured as a partitioned-database environment.)
It is to tell you to read message log in the message file you specified. In your case: /lotus/ExportFiles/FMessage.txt
Please read into the file to see what error is logged and if you need help understand what is logged, please post the content of the file.
This message is returned when at least one warning was received during processing. If a message file is being used, the warnings and errors will be printed there.
This warning does not affect processing.
User response
Review the message file warning.
EXPORT command using the ADMIN_CMD procedure
See use of the 'MESSAGES ON SERVER' clause, and how to get these messages using the result set returned by this routine in this case.

error while insert column in postgresql using shell script?

Below is my shell script, I am trying to insert columns in posrgresql using shell script.
But getting below error.
Script:
i='dcm_account494401_click_2017050511_20170505_093843_556195422.csv.gz'
load_date='2017-05-12'
load_status='Fail'
message="INFO: Load into table 'stg_ft_raw_activity' completed, 362554 record(s) loaded successfully.
INFO: Load into table 'stg_ft_raw_activity' completed, 1 record(s) were loaded with replacements made for ACCEPTINVCHARS. Check 'stl_replacements' system table for details.
"
psql "host=$HOST port=$DBPORT dbname=$DBNAME user=$DBUSER password=$DBPASS" -F --no-align <<EOF
truncate table stg.notification_table;
\set fname $i
\set load_date $load_date
\set load_status $load_status
\set message $message
insert into stg.notification_table values (:'fname', :'load_date', :'load_status',:"message");
EOF
error:
Expanded display is used automatically.
TRUNCATE TABLE and COMMIT TRANSACTION
ERROR: syntax error at or near "INFO"
LINE 1: INFO: Load into table 'stg_ft_raw_activity' completed, 1 re...
^
message col is string value and also contains spl chars. is that a reason?
Please help to resolve.
Thansk,
There is no need to use these variables with \set.
Here is an example:
message="INFO: Load into table 'stg_ft_raw_activity' completed, 362554 record(s) loaded successfully.
INFO: Load into table 'stg_ft_raw_activity' completed, 1 record(s) were loaded with replacements made for ACCEPTINVCHARS. Check 'stl_replacements' system table for details.
"
psql <<EOF
INSERT INTO message_table VALUES (\$\$$message\$\$);
EOF
This makes use of “dollar quoting” for string literals, with the $ signs that are no shell variable reference escaped with backslashes.

How to log an error into a Table from SQLPLus

I am very new to Oracle so please base with me if this is covered else where.
I have a MS SQL box running Jobs calling batch files running scripts in SQLPLUS to ETL to an Oracle 10G database.
I have an intermittent issue with a script that is causing the ETL to fail which at the minute without error logging is something of an unknown. The current solution highlights the load failure based on rowcounts for before and after the script has finsihed.
I'd like to be able to insert any errors encoutered whilst running the offending script into an error log table on the same database receiving the data loads.
There's nothing too technical about the script, at a high level is performs the following steps all via SQL code and no procedural calls.
Updates a table with Date and current row counts
Pulls data from a remote source into a staging table
Merges the Staging table into an intermediate staging table
Performs some transformational actions
Merges the intermediate staging table into the final Fact table
Updates a table with new row counts
Please advise whether it is possible to pass error messages, codes, line number etc etc via SQLPLUS into a Database table? and if so the easiest method to achieve this.
A first few lines of the script are shown below to give a flavour
/*set echo off*/
set heading off
set feedback off
set sqlblanklines on
/* ID 1 BATCH START TIME */
INSERT INTO CITSDMI.CITSD_TIMETABLE_ORDERLINE TGT
(TGT.BATCH_START_TIME)
(SELECT SYSDATE FROM DUAL);
COMMIT;
insert into CITSDMI.CITSD_TIMETABLE_ALL_LOADS
(LOAD_NAME, LOAD_CRITICALITY,LOAD_TYPE,BATCH_START_TIME)
values
('ORDERLINE','HIGH','SMART',(SELECT SYSDATE FROM DUAL));
commit;
/* Clear the Staging Tables */
TRUNCATE TABLE STAGE_SMART_ORDERLINE;
Commit;
TRUNCATE TABLE TRANSF_FACT_ORDERLINE;
Commit;
and so it goes on with the rest of the steps.
Any assistant will be greatly appreciated.
Whilst not fully understanding your requirement, a couple of pointers.
The WHENEVER command will help you control what sqlplus should do when an error occurs, e.g.
WHENEVER SQLERROR EXIT FAILURE ROLLBACK
WHENEVER OSERROR EXIT FAILURE ROLLBACK
INSERT ...
INSERT ...
This will cause sqlplus to exit with error status 1 if any of the following statements fail.
You can also have WHENEVER SQLERROR CONTINUE ...
Since the WHENEVER ... EXIT FAILURE/SUCCESS controls the exit status, the calling script/program will know if it worked failed.
Logging
use SPOOL to spool the out to a file.
Logging to table.
Best way is to wrap your statements into PLSQL anonymous blocks and use exception hanlders to log errors.
So, putting the above together, using a UNIX shell as the invoker:
sqlplus -S /nolog <<EOF
WHENEVER SQLERROR EXIT FAILURE ROLLBACK
CONNECT ${USRPWD}
SPOOL ${SPLFILE}
BEGIN
INSERT INTO the_table ( c1, c1 ) VALUES ( '${V1}', '${V2}' );
EXCEPTION
WHEN OTHERS THEN
INSERT INTO the_error_tab ( name, errno, errm ) VALUES ( 'the_script', SQLCODE, SQLERRM );
COMMIT;
END;
/
SPOOL OFF
QUIT
EOF
if [ ${?} -eq 0 ]
then
echo "Success!"
else
echo "Oh dear!! See ${SPLFILE}"
fi