Recently there has been some debate regarding the flexibility of column types in SQLite. Hence the question, How flexible are SQLite column types?
One argument was that types are restricted to the the main five, namely, TEXT, NUMERIC, INTEGER, REAL and BLOB, and additionally the named column types in the official documentation i.e. :-
INT, TINYINT, SMALLINT, MEDIUMINT, BIGINT, UNSIGNED BIG INT, INT2, INT8,
CHARACTER(20), VARCHAR(255), VARYING CHARACTER(255), NCHAR(55), NATIVE
CHARACTER(70), NVARCHAR(100), CLOB, no datatype specified (BLOB), DOUBLE,
DOUBLE PRECISION, FLOAT, DECIMAL(10,5), BOOLEAN, DATE & DATETIME.
3.1.1. Affinity Name Examples
Another argument was that the list was a list of examples and that column types are more flexible with the 5 rules (as below) being applied virtually universally.
3.1. Determination Of Column Affinity
The affinity of a column is determined by the declared type of the
column, according to the following rules in the order shown:
1) If the declared type contains the string "INT" then it is assigned INTEGER affinity.
2) If the declared type of the column contains any of the strings "CHAR", "CLOB", or "TEXT" then that column has TEXT affinity. Notice
that the type VARCHAR contains the string "CHAR" and is thus assigned
TEXT affinity.
3) If the declared type for a column contains the string "BLOB" or if no type is specified then the column has affinity BLOB.
4) If the declared type for a column contains any of the strings "REAL", "FLOA", or "DOUB" then the column has REAL affinity.
5) Otherwise, the affinity is NUMERIC.
Note that the order of the rules for determining column affinity is
important. A column whose declared type is "CHARINT" will match both
rules 1 and 2 but the first rule takes precedence and so the column
affinity will be INTEGER.
3.1. Determination Of Column Affinity
So what are the in and outs/rights and wrongs of SQLite Column Types?
SQLite column types are flexible (dynamic), primarily, it appears to cater for the adoption/adaptation of rigid column types used by other database Management Systems.
Note! this Asnwer is NOT recommending use of weird and wonderful column types.
1) You can actually use virtually any name for a column type, there are however some limitations.
2) Column type is the 2nd value in the column definition e.g. CREATE TABLE table (columnname columntype .....,....), although it may be omitted intentionally or perhaps inadvertently Note see 5a)
3) The first limitation is that mycolumnINTEGER PRIMARY KEY or mycolumnINTEGER PRIMARY KEY AUTOINCREMENT is a special column type. The column is an alias for the rowid which is a unique numeric identifier (AUTOINCREMENT imposes a rule that the rowid must be greater than the last used rowid for the table e.g. if a row uses id (9223372036854775807), then any subsequent attempts to add a row will result in an SQLITE FULL error. ). SQLite Autoincrement
4) Other limitations are that the column type mustn't confuse the SQLite parser. For example a column type of PRIMARY, TABLE, INDEX will result in an SQLite exception (syntax error (code 1)) e.g. when a column type of INDEX is used then:-
android.database.sqlite.SQLiteException: near "INDEX": syntax error (code 1):
occurs.
5) A column type is not mandatory, for example CREATE TABLE mytable (...,PRIMARY_COL,.... in which case a PRAGMA TABLE_INFO(tablename) will show no type e.g. (3rd Line).
08-08 07:56:23.391 13097-13097/? D/TBL_INFO: Col=cid Value=8
08-08 07:56:23.391 13097-13097/? D/ TBLINFO: Col=name Value=PRIMARY_COL
08-08 07:56:23.391 13097-13097/? D/ TBLINFO: Col=type Value=
08-08 07:56:23.391 13097-13097/? D/ TBLINFO: Col=notnull Value=1
08-08 07:56:23.391 13097-13097/? D/ TBLINFO: Col=dflt_value Value=null
08-08 07:56:23.391 13097-13097/? D/ TBLINFO: Col=pk Value=0
5a) In some cases the SQLite Parser will skip to valid KEYWORDS e.g. CREATE TABLE mytable (mycolumn NOT NULL,... results in NOT NULL being used to indicate a NOT NULL column and the type being taken as no type (the table_info above was actually from such a usage).
6) A type is not limited to a single word e.g. VARYING CHARACTER(255) or THE BIG BAD WOLF can be specified as a type as can be seen from this table_info extract :-
08-08 08:23:26.423 4799-4799/? D/ TBLINFO: Col=type Value=THE BIG BAD WOLF
The reason to use non-standard column types in SQLite!
In short there is no reason, as stated at first, the flexibility of column types appears to be primarily to cater for the easy adaptation of SQL from other Database Management Systems.
Column types themselves have little effect as data will be stored according to the what SQLite determines as the storage class to be used. With the exception of rowid (see 3) above) any column can hold values of any type.
With the exception of data stored as a Blob, which must be retrieved using the cursor.getBlob and that cursor.getBlob cannot be used for data not stored as a BLOB (getBlob doesn't fail with data stored as TEXT), You can very much retrieve data (all be it not necessarily useful) using any of the cursor.get???? methods.
Here's some examples:-
For a column where the data long myINT = 556677888; is added (via ContentValues e.g. cv1.put(columnanme,myINT));
Then :-
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: Column=INTEGER_COL<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS INT >>556677888<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS LONG >>556677888<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS STRING >>556677888<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS DOUBLE >>5.56677888E8<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS FLOAT >>5.566779E8<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS SHORT >>15104<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: Unable to handle with getBlob.
getShort does not return to the stored value, getBlob cannot get the stored value.
For Double myREAL = 213456789.4528791134567890109643534276; :-
08-08 09:19:03.658 13575-13575/mjt.soqanda D/ColTypes: Column=REAL_COL<<
08-08 09:19:03.658 13575-13575/mjt.soqanda D/ColTypes: VALUE AS INT >>213456789<<
08-08 09:19:03.658 13575-13575/mjt.soqanda D/ColTypes: VALUE AS LONG >>213456789<<
08-08 09:19:03.658 13575-13575/mjt.soqanda D/ColTypes: VALUE AS STRING >>2.13457e+08<<
08-08 09:19:03.658 13575-13575/mjt.soqanda D/ColTypes: VALUE AS DOUBLE >>2.134567894528791E8<<
08-08 09:19:03.658 13575-13575/mjt.soqanda D/ColTypes: VALUE AS FLOAT >>2.1345678E8<<
08-08 09:19:03.658 13575-13575/mjt.soqanda D/ColTypes: VALUE AS SHORT >>6037<<
08-08 09:19:03.658 13575-13575/mjt.soqanda D/ColTypes: Unable to handle with getBlob.
For String myTEXT = "The Lazy Quick Brown Fox Jumped Over the Fence or something like that.";
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: Column=TEXT_COL<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS INT >>0<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS LONG >>0<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS STRING >>The Lazy Quick Brown Fox Jumped Over the Fence or something like that.<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS DOUBLE >>0.0<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS FLOAT >>0.0<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS SHORT >>0<<
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ColTypes: VALUE AS BLOB >>[B#2f9e811e<<
And here's a pretty ridiculous example with a column type of my_char_is_not_a_char_but_an_int as per PRAGMA TABLE_INFO :-
08-08 09:19:03.657 13575-13575/mjt.soqanda D/TBL_INFO: Col=cid Value=7
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ TBLINFO: Col=name Value=my_char_is_not_a_char_but_an_int_COL
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ TBLINFO: Col=type Value=my_char_is_not_a_char_but_an_int
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ TBLINFO: Col=notnull Value=0
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ TBLINFO: Col=dflt_value Value=null
08-08 09:19:03.657 13575-13575/mjt.soqanda D/ TBLINFO: Col=pk Value=0
Results (stored as per 'Double' above) are:-
08-08 09:19:03.659 13575-13575/mjt.soqanda D/ColTypes: Column=my_char_is_not_a_char_but_an_int_COL<<
08-08 09:19:03.659 13575-13575/mjt.soqanda D/ColTypes: VALUE AS INT >>213456789<<
08-08 09:19:03.659 13575-13575/mjt.soqanda D/ColTypes: VALUE AS LONG >>213456789<<
08-08 09:19:03.659 13575-13575/mjt.soqanda D/ColTypes: VALUE AS STRING >>2.13457e+08<<
08-08 09:19:03.659 13575-13575/mjt.soqanda D/ColTypes: VALUE AS DOUBLE >>2.134567894528791E8<<
08-08 09:19:03.659 13575-13575/mjt.soqanda D/ColTypes: VALUE AS FLOAT >>2.1345678E8<<
08-08 09:19:03.659 13575-13575/mjt.soqanda D/ColTypes: VALUE AS SHORT >>6037<<
08-08 09:19:03.659 13575-13575/mjt.soqanda D/ColTypes: Unable to handle with getBlob.
The above was based upon the following:-
Datatypes In SQLite Version 3
SQLite Autoincrement
PRAGMA Statements
Code was tested/run on a GenyMotion emulated device running API22 compiled with a min version of 14 and target of 26.
Related
I'm trying to convert a column to double or float, however the column has 0 values, so I'm getting errors when I try to use that column after applying the cast.
df = (df.withColumn('received_sp_click_l1wk' ,df['received_sp_click_l1wk'].cast("double")))
Doesn't return any error, however applying any function to the casted column returns errors :
df.head(7)
TypeError: field received_sp_click_l1wk: FloatType can not accept object 0 in type <class 'int'>
Query:
SELECT CAST('123.1234' AS FLOAT) AS 'Float value', CAST(CAST('123.1234' AS FLOAT) AS NVARCHAR(MAX)) AS 'Text value'
returns:
Float value Text value
123,1234 123.123
Why there is a difference in number of decimal places?
https://learn.microsoft.com/en-us/sql/t-sql/functions/cast-and-convert-transact-sql
When expression is float or real, style can be one of the values shown in the following table. Other values are processed as 0.
0 (default) A maximum of 6 digits. Use in scientific notation, when appropriate.
1 Always 8 digits. Always use in scientific notation.
2 Always 16 digits. Always use in scientific notation.
3 Always 17 digits. Use for lossless conversion. With this style, every distinct float or real value is guaranteed to convert to a distinct character string. Applies to: Azure SQL Database, and starting in SQL Server 2016.
126, 128, 129 Included for legacy reasons and might be deprecated in a future release.
the below code will give you the required result
SELECT CAST('123.1234' AS FLOAT) AS 'Float value', convert(nvarchar(max),CAST('123.1234' AS FLOAT) ,128 ) AS 'Text value'
If you really want to get text back out of a number with scale you need to use numeric or decimal types and include the 'scale' second value of how much scale you want to weigh. Float I believe only will do an estimation and for more precise estimation you need to use it.
Declare #Text varchar(16) = '123.1234'
SELECT
CAST(#Text AS FLOAT) AS 'Float value'
, CAST(CAST(#Text AS Float) AS NVARCHAR(MAX)) AS 'float'
, CAST(CAST(#Text AS Float(4)) AS NVARCHAR(MAX)) AS 'float with precision'
, CAST(CAST(#Text AS numeric(8,4)) AS NVARCHAR(MAX)) AS 'numeric with precision'
, CAST(CAST(#Text AS decimal(8,4)) AS NVARCHAR(MAX)) AS 'decimal with precision'
More number data types: https://academy.vertabelo.com/blog/understanding-numerical-data-types-sql/
My query is something like this. I try to get a status for a list of ids.
select order_number, order_status_name
from data.order_fact s
join data.order_status_dim l
on s.order_status_key = l.order_status_key
where
order_number in (1512011196169,1512011760019,1512011898493,1512011972111)
I get an error though that says:
ERROR: operator does not exist: character varying = bigint
LINE 6: order_number in (1512011196169,1512011760019,1512011898493,1...
^
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
Do you have any clue on how I should reform the ids to get it work?
Thanks a lot!
Your order_number is a varchar, you can't compare that to a number (123 is a number in SQL, '123' is a string constant)
You need to use string literals:
order_number in ('1512011196169','1512011760019','1512011898493','1512011972111')
More details in the manual:
http://www.postgresql.org/docs/current/static/sql-syntax-lexical.html#SQL-SYNTAX-CONSTANTS
If you can't change the type of numbers within in, you could use cast:
select * from numbers_as_string
where cast(my_numbers_as_string as int) in (1,2,3)
This happen too when you are using native query in sprinboot and you are passing a parameter as string but that field is a (integer or long) in your model/entity, also when you are comparing a string with a integer due that param is used like string without casting.
so you should cast it as integer in the native query like this
x\:\:integer
for example:
#Query(value="
......
.....
inner join tablex t on t.x\\:\\:integer = pv.id \n"+
....
....
")
List<Object> getMyQuery(#Param("x") String x)
The following SQL SELECT query returns an integer:
SELECT (((get_byte(dalsk.data, 0)::bit(8)) ||
(get_byte(dalsk.data, 1)::bit(8)) ||
(get_byte(dalsk.data, 2)::bit(8)) ||
get_byte(dalsk.data, 3)::bit(8)) :: bit(32)) :: integer --AS rezult_float
FROM (SELECT substring(data from 2 for 5) AS data FROM raw_data WHERE
raw_data_id = 33)
AS dalsk;
Whenever I try to cast the result from integer to real, I get the following error:
Cannot cast type bit to real.
Is there a way to convert the value to float?
The IEEE754 doesn't cover endianess so that's not enough info to convert the binary to float. If you want to convert it you will have to create a function or a cast operator.
I want to create a function that converts a string of characters to bigint. If the conversion is not possible the function should return null. I want the function to work for normal representation (example '10000') and mantissa-exponent representation ('1e1+10') Here is what I have written so far:
ALTER FUNCTION [dbo].[udf_get_bigint]
(
#character varchar(100)
)
RETURNS bigint
AS
BEGIN
if ISNUMERIC(#character)=0 return null
if LEN(ltrim(rtrim(#character)))>25 return null
declare #nr numeric(25,4)
if charindex('e',lower(#character))>0
begin
declare #real real
**set #nr=CONVERT(real,#character)**
if #nr not between convert(numeric(25),-9223372036854775808) and
convert(numeric(25),9223372036854775807)
return null
set #real = convert(real, #nr)
return convert(bigint,convert(numeric(25),#real))
end
else
set #nr=CONVERT(numeric(25,4),#character)
if #nr between convert(numeric(25),-9223372036854775808) and
convert(numeric(25),9223372036854775807) return convert(bigint,#nr)
return null
END
Now the only problem appears when I need to deal with overflows for mantissa exponent representation. The bolded conversion falls in case of overflow; but what I want it to do is to return null.
How can I put some preconditions on that conversion so that it does not fall anymore.
call example : select dbo.udf_get_bigint('3e0210')
output: Arithmetic overflow error converting expression to data type real.
Use float instead of real. It may contradict the variable name, but it makes that part of the script work
declare #real float
This code will verify
select CONVERT(float,'3e0210')