I am using firebird database version 2.0. When I try to update a row, I get an error message: Attempted update of read-only database.
http://www.firebirdfaq.org/faq359/ suggests I may query a blob field that uses a different character set than the connection character set.
I do query a blob field and when the blob field has a value then the updating causes the error. If there is no value in the blob then the updating is just fine.
I use IBConsole to open the firebird database and check the database metadata, I find the metadata says "Default character set NONE".
To fix the problem I firstly need to know what are the character sets used in my database.
So my questions are:
what is the character set being used for my database (Connection character set)
the data type of the blob field is MEMOBLOB, and MEMOBLOB is created as Create domain MEMOBLOB as blob sub_type TEXT segment size 80; So what is the character set use for the MEMOBLOB?
No, that is not about queries or BLOBs.
Firebird databases have a number of modes, one of them is "read-only". In this mode no any change to database is allowed.
You can use gfix utility to change this database mode. You can also use corresponding menu in IBExpert and other development tools that use Firebird Services API
The very link you posted - http://www.firebirdfaq.org/faq359/ - says that:
It does not mean that the database file is read-only, but it (database) contains a read-only mark
gfix -mode read_only /path/to/database.fdb
gfix -mode read_write /path/to/database.fdb
See also https://www.firebirdsql.org/manual/gfix-dbmode.html
See also https://www.ibexpert.net/ibe/pmwiki.php?n=Doc.DatabaseProperties
Of tangential questions:
character set being used for my database
According to your text, it is NONE.
To be exact, database does not use some charset. It is every textual column (char/varchar/blob sub_type text) that do. But usually developer does not bother with specifying individual per-column charsets, so they inherit that default one.
Read also: https://www.firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref25-ddl-tbl.html#fblangref25-ddl-tbl-character
metadata says "Default character set NONE"
This is as close to "database charset" as can be.
Granted, that is only default one and you could override it when creating your columns, but i do not think you did. So probably all your textual columns have charset "NONE".
That is rather dangerous setting, meaning all texts in such columns are stored as raw bytes dump, and hoping application would correctly guess how to convert bytes to letters and letters to bytes.
Read more: https://www.firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref25-datatypes-chartypes.html
Flame Robin seems not to show charset by default, but maybe in DDL section it would.
http://www.flamerobin.org/images/screenshots/0.6.0/gtk2/property_page.png
IBExpert does: https://www.ibexpert.net/ibe/uploads/Doc/dmiles783.gif
(Connection character set)
....is not "character set used by database", it is character set used by, well, connection your application (such as IBConsole or FlameRobin or IBExpert) makes to the database.
You have to set it in every application's connection properties. Easiest option would be UTF-8, but when you have NONE-charset columns it might fail...
For example in FR: http://www.flamerobin.org/images/screenshots/0.7.1/winxp/databaseinfo.png
You can use monitoring tables to query for charset id of your CURRENT_CONNECTION, see more at https://www.firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref-appx05-monattach.html
After I add transaction parameters for the relevant tables. It solves the problem.
The parameter value I added are isc_tpb_lock_write and isc_tpb_shared.
Thank you Arioch 'The.
Related
Iam using mdriven build 7.0.0.11347 for DDD project and have model designed in .ecomdl file.
In this file i have a class Job with WorkDone as one of a property. Backedup SQL table has WorkDone varchar(255) field. Now i wanted to increase length of this field and When i changed the WorkDone property length from 255 to 2000 then it modified the code file but when application runs EvolveSchema then evolving process doesn't recognize this change which leads to no scripts being generated. In the end database doesn't get this updated.
Can you please help me how to get this change persist to database. I thought to increase manually to SQL table but then if database gets change in case of new envrionment QA production then it has to be done every time, which id don't want to do.
In MDriven we dont evolve attribute changes - we only write a warning (255->2000 this change will not be evolved)
You should take steps to alter the column in the database yourself.
We should fix in the future but currently this is a limitation
To expand on my comment, VARCHAR can only be from 0-255 chars
Using TEXT will allow for non-binary (character) strings and BLOBs will allow for binary (byte) strings
Your mileage may vary with this as to what you can do with them, as I am using MySQL knowledge and knowledgebases (since you don't specify your SQL type)
See below for exaplanations of the types;
char / varchar
blobs / text
I am performing an ETL job via Pentaho 7.1.
The job is to populate a table 'PRO_T_TICKETS' in PostgreSQL 9.2 via the Pentaho Jobs and transformations?
I have mapped the table fields with respect to the stream fields
Mapped Fields
My Table PRO_T_TICKETS contains the Schema (Column Names) in UPPERCASE.
Is this the reason I can't populate the table PRO_T_TICKETS with my ETL Job?
I duplicated the step TABLE_OUTPUT to PRO_T_TICKETS and changed the Target table field to 'PRO_T_TICKETS2'. Pentaho created a new table with lowercase schema and populated the data in it.
But I want this data to be uploaded in the table PRO_T_TICKETS only and with the UPPERCASE schema if possible.
I am attaching the whole job here and the error thrown by Pentaho. Pentaho Error I have also tried my query by adding double quotes to the column names as you can see in the error. But it didn't help.
What do you think I should do?
When you create (or modify) the connection, select Advanced on the left panel and click on the Force to upper case or Force to lower case or, even better, Preserve case of reserved words.
To know which option to choose, copy the 4th line of your error log, the line starting with INSERT INTO "public"."PRO_T_TICKETS("OID"... in your SQL-developer tool and change the connection advanced parameters until it works.
Also, at debug time, don't use batch updates, don't use lazy conversion on previous steps, and try with one (1) field rather than all (25).
Just as a complement: it worked for me following the tips from AlainD and using specific configurations that I'd like to share with you. I have a transformation streaming data from MySQL to PostgreSQL using a Table Input and Output. In both of DBs I have uppercase objects.
I did the following steps to work in the right way:
In the table input (MySQL) the objects are uppercase too, but I typed in lowercase and it worked and I didn't set any special option in the DB Connection.
In the table output (PostgreSQL) I typed everything in uppercase (schema, table name and columns) and I also set "specify the database fields" (clicking on "Get fields").
In the target DB Connection (PostgreSQL) I put the options (in "Advanced" section): "Quote all in database" and "Preserve case of reserved words".
PS: Ah, the last option is because I've found out that there was one more problem with my fields: there was a column called "Admin" (yes guys, they created a camelcase column using a reserved word!) and for that reason I must to put "Preserve case of reserved words" and type it as "Admin" (without quotes and in camelcase) in the Table Output.
I have been trying to get around this for several day's now with no luck. I loaded Libre Office to see how that would handle it, and its native support for PostgeSQL works wonderfully and I can see the true data structure. Which is how I found out I was dealing with more than one table. What I am seeing in MS Access is the two names concatenated together. The concatenation takes them over the 64 character limit that appears to be built into the ODBC driver. I have seen many references to modifying namedatalen on the server side, but my problem is on the ODBC side. Most of the tables are under the 64 character limit even with the concatenation and work fine. As such I know everything else is working. The specific error I am getting is
'your_extra_long_schema_name_your_table_name_that_you_want_to_get_data_from'
is not a valid name. Make sure it does not include invalid characters
or punctuation and that it is not too long.
Object names in an Access database are limited to 64 characters (ref: here). When creating an ODBC linked table in the Access UI the default behaviour is to concatenate the schema name and table name with an underscore and use that as the linked table name so, for example, the remote table table1 in the schema public would produce a linked table in Access named public_table1. If such a name exceeds 64 characters then Access will throw an error.
However, we can use VBA to create the table link with a shorter name, like so:
Option Compare Database
Option Explicit
Sub so38999346()
DoCmd.TransferDatabase acLink, "ODBC Database", "ODBC;DSN=PostgreSQL35W", acTable, _
"public.hodor_hodor_hodor_hodor_hodor_hodor_hodor_hodor_hodor_hodor", _
"hodor_linked_table"
End Sub
(Tested with Access 2010.)
I am producing a comma-separated file in S3 that needs to be copied to a staging table in a redshift database using the postgres COPY command.
It has one boolean field. With every sensible way I can think of to represent the boolean value in the file, redshift copy complains, usually with "Unknown boolean format".
I'm going to give up and change the staging table field to a smallint so that I can proceed with the copy and translate the value on the load from staging to the final redshift table, but I'm curious if anyone knows the correct incantation.
A zero or one works just fine for us.
Check your loads carefully, it may well be another issue that's 'pushing' invalid data into your boolean column.
For instance, we had all kinds of crazy characters embedded in our data that would cause errors like that. I eventually settled on using the US character for the record separator.
Check to make sure you're excluding the headers during the COPY command.
I ran into the same problem, but adding the ignoreheader 1 option (ignores 1 header line during import) solved the issue.
I have an ssis package where I am using an OLEDB source linking to SQL Server 2005 table. All columns except a date column are NVARCHAR(255). I am using an Excel destination and using a SQL statement to create the sheet in the Excel workbook, the SQL is in the excel connection manager (effectively a create table statement that creates a sheet) and is derived from the mapping of the columns from the DB.
No matter what I have done I keep getting this unicode --> non-unicode conversion error between my source and destination. Tried conversion to string[DT_STR] between S > D, removed it, changed SQL Table VARCHAR to NVARCHAR and still get this flippin error.
Because I am creating the sheet in Excel with a SQL statement I do not see any way to actually pre-define what the data types of the columns will be in the Excel sheet. I imagine it would be a default meta data but I do not know.
So between my SQL table destination and the creation of my Excel sheet with this SSIS sql statement how can I stop this error coming up?
My error is:
Error at Data Flow Task [OLE DB Source [1]]: Column "MyColumn" cannot convert between unicode and non-unicode string data types.
And for all nvarchar columns.
Appreciate any help
Thanks
Andrew
The below Steps worked for me:
right click on source task.
click on "Show Advanced editor".
Go to "Input and Output Properties" tab.
select the output column for which you are getting the error.
Its data type will be "String[DT_STR]".
Change that data type to "Unicode String[DT_WSTR]".
save and close.
Add Data Conversion transformations to convert string columns from non-Unicode (DT_STR) to Unicode (DT_WSTR) strings.
You need to do this for all the string columns...
The missing piece here is Data Conversion object. It should be in between OLE DB Source and Destination object.
First, add a data conversion block into your data flow diagram.
Open the data conversion block and tick the column for which the error is showing. Below change its data type to unicode string(DT_WSTR) or whatever datatype is expected and save.
Go to the destination block. Go to mapping in it and map the newly created element to its corresponding address and save.
Right click your project in the solution explorer.select properties. Select configuration properties and select debugging in it. In this, set the Run64BitRunTime option to false (as excel does not handle the 64 bit application very well).
Instead of adding an earlier suggested Data Conversion you can cast the nvarchar column to a varchar column. This prevents you from having an unnecessary step and has a higher performance then the alternative.
In the select of your SQL statement replace date with CAST(date AS varchar([size])). For some reason this does not yet change the output data type. To do this do the following:
Right click your OLE DB Source step and open the advanced editor.
Go to Input and Output Properties
Select Output Columns
Select your column
Under Data Type Properties change DataType to string [DT_STR]
Change Length to the length you specified in your CAST statement
After doing this your source data will be output as a varchar and your error will disappear.
Source
I have been having the same issue and tried everything written here but it was still giving me the same error.
Turned out to be NULL value in the column which I was trying to convert.
Removing the NULL value solved my issue.
Cheers,
Ahmed
No-one seems to mention this but, converting varchar to nvarchar in the source query also solves the issue.
On the above example I kept losing the values, I think that delaying the Validation will allow the new data types to be saved as part of the meta data.
On the connection Manager for 'Excel Connection Manager' set the Delay Validation to False from the Properties.
Then on the data flow Destination task for Excel set the ValidationExternalMetaData to False, again from the properties.
This will now allow you to right click on the Excel Destination Task and go to Advanced Editor for Excel Destination --> far right tab - Input and Output Properties. In the External Columns folder section you will be able to now change the Data Types and Length values of the problematic columns and this can now be saved.
Good Luck!
I experienced this condition when I had installed Oracle version 12 client 32 bit client connected to an Oracle 12 Server running on windows.
Although both of Oracle-source and SqlServer-destination are NOT Unicode, I kept getting this message, as if the oracle columns were Unicode.
I solved the problem inserting a data conversion box, and
selecting type DT-STR (not unicode) for varchar2 fields and DT-WSTR (unicode) for numeric fields, then I've dropped the 'COPY OF' from the output field name.
Note that I kept getting the error because I had connected the source box arrow with the conversion box BEFORE setting the convertion types. So I had to switch source box and this cleaned all the errors in the destination box.
When creating table in SQL Server make your table columns NVARCHAR instead of VARCHAR.
I think people are missing this. In my case I had 100 character columns to convert between Oracle and MS Sql. All this stuff about Data Conversion and Advanced Editor is incredibly tedious if you have a 100 separate character columns to assign. Plus SSIS being SSIS, it will sometimes reset all your 100 advanced editor changes even if you set VALIDATEEXTERNALMETADATA to false, incredibly obnoxious. I wouldn't mind doing the Data Conversion if there was some value to it but 20 years ago ETL tools used to take oracle character to ms sql characters without fussing. What Bakalolo and Zafer say is the answer if you have a lot of character columns and you can live with nvarchar, just declare all your output ms sql columns (nvarchar) and your data task will automatically assign your oracle fields into ms sql fields with no manual overrides. I have also found that the new Oracle Source (2021) doesn't complain about a unicode conversion to varchar in ms sql. A colleague just told me that the ssis wizard (it may be only in vs 2019+) to assign oracle character to ms sql varchar will do the assignments automatically with no override, but I haven't tried that personally.
2022 update - I think this is just vs 2019 created packages and later. An ado.net task reading a varchar ms sql table going to oledb (and ado.net I think) ms sql varchar will throw the unicode error. If you switch the input task to oledb reading ms sql varchar table you won't have to do the advanced editor overrides for the varchar fields. If you don't want to do advanced editor overrides (who does?) try different tasks and more oledb tasks.
I just encounter same issue, I solve it in my SQL request : using convert directly
CONVERT(NVARCHAR(50),'') AS MyVarName
I need to put an empty (or fix size string) into excel file. Converting force type of MyVarName from DT-STR to DT-WSTR (unicode)
I know this is a very old post but I ran into the same issue and found that I had to manually select the conversion component output alias as the mapping in the excel destination component. Since the names of the OLE DB Source match the excel column names it was mapping it to the OLE DB and not to the Output Alias. Such as SourceID column from the OLE DB component being named Copy of SourceID after conversion. I don't see the original question saying they specifically selected the new alias name just that they mapped to DB columns. #Serge Voloshenko post comes the closest but also does not mention to make sure the mapping happens. To a new SSIS user this might be overlooked.