What could cause Firebird to silently turn calculated fields into "normal" fields? - firebird

I'm using Firebird 2.5.8 to store information for a software I designed.
A customer contacted me today to inform me of multiple errors that I couldn't understand, and I used the "IBExpert" tool to inspect its database.
To my surprise, all the calculated fields had been transformed into "standard" fields. This is clearly visible in the "DDL" tab of the database tool, which displays tables definition as SQL code.
For instance, the following table definition:
CREATE TABLE TVERSIONS (
...
PARENTPATH COMPUTED BY (((SELECT TFILES.FILEPATH FROM TFILES WHERE ID = TVERSIONS.FILEID))),
....
ISCOMPLETE COMPUTED BY ((((SELECT TBACKUPVERSIONS.ISCOMPLETE FROM TBACKUPVERSIONS WHERE ID = TVERSIONS.CVERSION)))),
CDATE COMPUTED BY (((SELECT TBACKUPVERSIONS.SERVERSTARTDATE FROM TBACKUPVERSIONS WHERE ID = TVERSIONS.CVERSION))),
DDATE COMPUTED BY (((SELECT TBACKUPVERSIONS.SERVERSTARTDATE FROM TBACKUPVERSIONS WHERE ID = TVERSIONS.DVERSION))),
...
);
has been "changed" in the client database into this:
CREATE TABLE TVERSIONS (
...
PARENTPATH VARCHAR(512) CHARACTER SET UTF8 COLLATE UNICODE,
...
ISCOMPLETE SMALLINT,
CDATE TIMESTAMP,
DDATE TIMESTAMP,
...
);
How can such a thing be possible?
I've been using Firebird for more than 10 years, and I've never seen such a behavior until now. Is it possible that it could be a corruption of RDB$FIELDS.RDB$COMPUTED_SOURCE fields?
What would you advise?

To summarize the discussion on firebird-support (and comments above):
The likely cause of this happening is that the database was backed up and restored using gbak, and the restore did not complete successfully. If this happens, gbak will have ended in an error, and the database is in single shutdown state (which means only SYSDBA or the database owner is allowed to create one connection). If the database is not currently in single shutdown mode, someone used gfix to bring the database online again in normal state.
When a database is restored using gbak, calculated fields are initially created as normal fields (though their values are not part of the backup). After data is restored successfully, those fields are altered to be calculated fields. If there are any errors before or during redefinition of the calculated fields, the restore will fail, and the database will be in single shutdown state, and the calculated fields will still be "normal" fields.
I recommend doing a structural comparison of the database to check if calculated fields are the only problem, or if other things (e.g. constraints) are missing. A simple way to do this is to export the DDL of the database and a "known-good" database, for example using ISQL (command line option -extract), and comparing them with a diff tool.
Then either fix the existing database by executing the necessary DDL to restore calculated fields (and other things), or create a new empty database, and move the data from the old to the new (using a datapump tool).
Also check if any data is missing. By default, gbak restores the data in a single transaction, so in that case either all data is present or all data is missing. However, gbak also has a "transaction-per-table" mode (-ONE_AT_A_TIME or -O), which could mean some tables have data, and others have no data.

Related

How to enhance length of Property of an attribute of class in eco Model designer

Iam using mdriven build 7.0.0.11347 for DDD project and have model designed in .ecomdl file.
In this file i have a class Job with WorkDone as one of a property. Backedup SQL table has WorkDone varchar(255) field. Now i wanted to increase length of this field and When i changed the WorkDone property length from 255 to 2000 then it modified the code file but when application runs EvolveSchema then evolving process doesn't recognize this change which leads to no scripts being generated. In the end database doesn't get this updated.
Can you please help me how to get this change persist to database. I thought to increase manually to SQL table but then if database gets change in case of new envrionment QA production then it has to be done every time, which id don't want to do.
In MDriven we dont evolve attribute changes - we only write a warning (255->2000 this change will not be evolved)
You should take steps to alter the column in the database yourself.
We should fix in the future but currently this is a limitation
To expand on my comment, VARCHAR can only be from 0-255 chars
Using TEXT will allow for non-binary (character) strings and BLOBs will allow for binary (byte) strings
Your mileage may vary with this as to what you can do with them, as I am using MySQL knowledge and knowledgebases (since you don't specify your SQL type)
See below for exaplanations of the types;
char / varchar
blobs / text

Firebird error: Attempted update of read-only database

I am using firebird database version 2.0. When I try to update a row, I get an error message: Attempted update of read-only database.
http://www.firebirdfaq.org/faq359/ suggests I may query a blob field that uses a different character set than the connection character set.
I do query a blob field and when the blob field has a value then the updating causes the error. If there is no value in the blob then the updating is just fine.
I use IBConsole to open the firebird database and check the database metadata, I find the metadata says "Default character set NONE".
To fix the problem I firstly need to know what are the character sets used in my database.
So my questions are:
what is the character set being used for my database (Connection character set)
the data type of the blob field is MEMOBLOB, and MEMOBLOB is created as Create domain MEMOBLOB as blob sub_type TEXT segment size 80; So what is the character set use for the MEMOBLOB?
No, that is not about queries or BLOBs.
Firebird databases have a number of modes, one of them is "read-only". In this mode no any change to database is allowed.
You can use gfix utility to change this database mode. You can also use corresponding menu in IBExpert and other development tools that use Firebird Services API
The very link you posted - http://www.firebirdfaq.org/faq359/ - says that:
It does not mean that the database file is read-only, but it (database) contains a read-only mark
gfix -mode read_only /path/to/database.fdb
gfix -mode read_write /path/to/database.fdb
See also https://www.firebirdsql.org/manual/gfix-dbmode.html
See also https://www.ibexpert.net/ibe/pmwiki.php?n=Doc.DatabaseProperties
Of tangential questions:
character set being used for my database
According to your text, it is NONE.
To be exact, database does not use some charset. It is every textual column (char/varchar/blob sub_type text) that do. But usually developer does not bother with specifying individual per-column charsets, so they inherit that default one.
Read also: https://www.firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref25-ddl-tbl.html#fblangref25-ddl-tbl-character
metadata says "Default character set NONE"
This is as close to "database charset" as can be.
Granted, that is only default one and you could override it when creating your columns, but i do not think you did. So probably all your textual columns have charset "NONE".
That is rather dangerous setting, meaning all texts in such columns are stored as raw bytes dump, and hoping application would correctly guess how to convert bytes to letters and letters to bytes.
Read more: https://www.firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref25-datatypes-chartypes.html
Flame Robin seems not to show charset by default, but maybe in DDL section it would.
http://www.flamerobin.org/images/screenshots/0.6.0/gtk2/property_page.png
IBExpert does: https://www.ibexpert.net/ibe/uploads/Doc/dmiles783.gif
(Connection character set)
....is not "character set used by database", it is character set used by, well, connection your application (such as IBConsole or FlameRobin or IBExpert) makes to the database.
You have to set it in every application's connection properties. Easiest option would be UTF-8, but when you have NONE-charset columns it might fail...
For example in FR: http://www.flamerobin.org/images/screenshots/0.7.1/winxp/databaseinfo.png
You can use monitoring tables to query for charset id of your CURRENT_CONNECTION, see more at https://www.firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref-appx05-monattach.html
After I add transaction parameters for the relevant tables. It solves the problem.
The parameter value I added are isc_tpb_lock_write and isc_tpb_shared.
Thank you Arioch 'The.

Unable to create db2 table with DATE data type

I am using DB2 9.7 (LUW) in a windows server, in which multiple DBs are available in a single DB instance. I just found that in one of these DBs, I am unable to add a column with DATE data type, during table creation or altering. The column been added is getting changed to timestamp instead.
Any help on this will be welcome.
Check out your Oracle Compatibility setting
Depending on that setting a date is interpreted as Timestamp(0) like in your example.
Because these settings take effect if the database has been created after setting the DB2_COMPATIBILITY_VECTOR registry variable your database can show a different behaviour.

How to populated the table via Pentaho Data Integration's table_output step?

I am performing an ETL job via Pentaho 7.1.
The job is to populate a table 'PRO_T_TICKETS' in PostgreSQL 9.2 via the Pentaho Jobs and transformations?
I have mapped the table fields with respect to the stream fields
Mapped Fields
My Table PRO_T_TICKETS contains the Schema (Column Names) in UPPERCASE.
Is this the reason I can't populate the table PRO_T_TICKETS with my ETL Job?
I duplicated the step TABLE_OUTPUT to PRO_T_TICKETS and changed the Target table field to 'PRO_T_TICKETS2'. Pentaho created a new table with lowercase schema and populated the data in it.
But I want this data to be uploaded in the table PRO_T_TICKETS only and with the UPPERCASE schema if possible.
I am attaching the whole job here and the error thrown by Pentaho. Pentaho Error I have also tried my query by adding double quotes to the column names as you can see in the error. But it didn't help.
What do you think I should do?
When you create (or modify) the connection, select Advanced on the left panel and click on the Force to upper case or Force to lower case or, even better, Preserve case of reserved words.
To know which option to choose, copy the 4th line of your error log, the line starting with INSERT INTO "public"."PRO_T_TICKETS("OID"... in your SQL-developer tool and change the connection advanced parameters until it works.
Also, at debug time, don't use batch updates, don't use lazy conversion on previous steps, and try with one (1) field rather than all (25).
Just as a complement: it worked for me following the tips from AlainD and using specific configurations that I'd like to share with you. I have a transformation streaming data from MySQL to PostgreSQL using a Table Input and Output. In both of DBs I have uppercase objects.
I did the following steps to work in the right way:
In the table input (MySQL) the objects are uppercase too, but I typed in lowercase and it worked and I didn't set any special option in the DB Connection.
In the table output (PostgreSQL) I typed everything in uppercase (schema, table name and columns) and I also set "specify the database fields" (clicking on "Get fields").
In the target DB Connection (PostgreSQL) I put the options (in "Advanced" section): "Quote all in database" and "Preserve case of reserved words".
PS: Ah, the last option is because I've found out that there was one more problem with my fields: there was a column called "Admin" (yes guys, they created a camelcase column using a reserved word!) and for that reason I must to put "Preserve case of reserved words" and type it as "Admin" (without quotes and in camelcase) in the Table Output.

db2 SQLCODE=-243, SQLSTATE=36001 ERROR

I am using the DB2Driver in my code like
Class.forName("com.ibm.db2.jcc.DB2Driver");
and I am getting the result set in my java code which is scroll sensitive. my sql query look like this select distinct day , month , year from XXX . here table XXX is read only for the user which I am using ... so it is giving the following error
com.ibm.db2.jcc.a.SqlException: DB2 SQL Error: SQLCODE=-243, SQLSTATE=36001, SQLERRMC=SQL_CURSH200C3, DRIVER=3.51.90 .. I know this is the problem of read only .. but when i try to execute the same query in db2 control center it is working
please help me out in this
PubLib is your friend :-)
SQL0243NSENSITIVE cursor <cursor-name> cannot be defined for the specified SELECT statement.
Explanation:
Cursor <cursor-name> is defined as SENSITIVE, but the content of the SELECT statement requires DB2 to build a temporary result table of the cursor, and DB2 cannot guarantee that changes made outside this cursor will be visible. This situation occurs when the content of the query makes the result table read-only. For example, if the query includes a join, the result table is read-only. In these cases, the cursor must be defined as INSENSITIVE or ASENSITIVE.
The statement cannot be processed.
User response:
Either change the content of the query to yield a result table that is not read-only, or change the type of the cursor to INSENSITIVE or ASENSITIVE.
If you can't change the cursor type, look in to the use of materialised queriey tables. These are like views but also provide temporary backing storage for the data so that it's not forced read-only by the query type.
Whether that will help in situations where you've forced the user to be read only, I'm not entirely sure but you may be able to have different permission on the materialised data and real data (unfortunately, I haven't done a lot of work with these, certainly none where permissions were locked down to read-only level).