I need to convert a given IPV6 address to integer. I do that by converting each part of the IP to integer and add those. But Azure Synapse analytics (or SQL Server) BIGINT does not have the capacity to hold the result. As part of the conversion I use power and multiplication that is failing. For example the below multiplication/computation is throwing error
set #Ip = power(65535,7) * 65536
I get the error: Arithmetic overflow error converting expression to data type bigint
Is there any way to achieve the above numerical computation in native SQL query?
Any help or pointers/references to convert IPV6 to integer is much appreciated,
Thanks.
Related
I have a table with money type column in my PostgreSQL database. I want to connect the data to Google Data Studio, but it doesn't support money type data.
is there a way to convert money type to double or bigint in a query like this or any other equivalent query?
SELECT todouble(maintenance.cost) FROM maintenance
Thanks!
The most natural choice is numeric:
select 123.45::money::numeric;
numeric
---------
123.45
(1 row)
You can also use integer or bigint but you have to take care of handling the fractional part then. Do not try real or double precision, per the documentation::
Floating point numbers should not be used to handle money due to the potential for rounding errors.
you can use ::numeric::float8 this to convert it. so it become
SELECT maintenance.cost::numeric::float8 FROM maintenance
you can see it in the documentation below:
https://www.postgresql.org/docs/current/datatype-money.html
I am working with a Java application which uses JPA and a Postgres database, and I am trying to create a flexible prepared statement which can handle a variable number of input parameters. An example query would best explain this:
SELECT *
FROM my_table
WHERE
(string_col = :param1 OR :param1 IS NULL) AND
(double_col = :param2 OR :param2 IS NULL);
The idea behind this "trick" is that if a user specifies only one parameter, say :param1, we can just bind null to :param2, and the WHERE clause would then behave as if only the first parameter were even being checked. This approach lets us handle, in theory, any number of input parameters using a single prepared statement, instead of needing to maintain many different statements.
I have gotten a simple POC working locally using pure JDBC prepared statements. However, doing so required casting the parameter before comparing it to NULL, e.g.
WHERE (double_col = ? OR ?::numeric IS NULL)
^^ does not work without cast
However, my actual application is using JPA, and I keep getting the following persistent error:
Caused by: org.postgresql.util.PSQLException: ERROR: operator does not exist: double precision = bytea
Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.
The problem does not occur with string/text columns, but only with columns which are double precision in my Postgres table. I have tried all combinations of casting, and nothing works:
(double_col = :param2 OR CAST(:param2 AS double precision) IS NULL);
(CAST(double_col AS double precision) = :param2 OR :param2 IS NULL);
(CAST(double_col AS double precision) = :param2 OR CAST(:param2 AS double precision) IS NULL);
The error seems to be saying that JDBC is sending Postgres a bytea type for the double columns, and then Postgres is rolling over because it can't find a way to cast byte to double precision.
The Java code looks something like:
Query query = entityManager.createNativeQuery(sqlString, MyEntity.class);
query.setParameter("param1", "some value");
// bind other parameters here
List<MyEntity> = query.getResultList();
For reference, here are the versions of everything I am using:
Hibernate version | 4.3.7.Final
Spring data JPA vesion | 1.7.1.RELEASE
Postgres driver version | 42.2.2
Postgres database version | 9.6.10
Java version | 1.8.0_171
Not having received any feedback in the form of answers or even a comment, I was getting ready to give up, when I stumbled onto this excellent blog post:
How to bind custom Hibernate parameter types to JPA queries
The post gives two options for controlling the types which JPA passes through the driver to Postgres (or whatever the underlying database actually is). I went with the approach using TypedParameterValue. Here is what my code looks like continuing with the example given above:
Query query = entityManager.createNativeQuery(sqlString, MyEntity.class);
query.setParameter("param1", new TypedParameterValue(StringType.INSTANCE, null));
query.setParameter("param2", new TypedParameterValue(DoubleType.INSTANCE, null));
List<MyEntity> = query.getResultList();
Of course, it is trivial to be passing null for every parameter in the query, but I am doing this mainly to show the syntax for the text and double columns. In practice, we would expect at least a few of the parameters to be non null, but the above syntax handles all values, null or otherwise.
If you want to keep using plain queries with automatic parameter binding, you could try the following.
WHERE (? IS NULL OR (CAST(CAST(? AS TEXT) AS DOUBLE PRECISION) = double_col
This seems to satisfy the PostgreSQL driver's type checks as well as yielding the correct results. I haven't done much testing, but the performance hit seems minimal because the CASTs happen on a constant value rather than rows from the database.
Bigint in postgresql is 8 byte integer. which is has half the range as uint64 (as one bit is used to sign the integer)
I need to do a lot of aggregation on the column and I am under the impression that aggregation on NUMERIC type is slow in comparison to integer types.
How should I optimize my storage in this case?
Unless you have a concrete reason, just use NUMERIC. It is slower, quite a lot slower, but that might not matter as much as you think.
You don't really have any alternative, as PostgreSQL doesn't support unsigned 64-bit integers at the SQL level. You could add a new datatype as an extension module, but it'd be a lot of work.
You could shove the unsigned 64-bit int bitwise into a 64-bit signed int, so values above maxuint64/2 are negative. But that'll be totally broken for aggregation, and would generally be horribly ugly.
sum() will return numeric if the input is bigint so it will not overflow
select sum(a)
from (values (9223372036854775807::bigint), (9223372036854775807)) s(a)
;
sum
----------------------
18446744073709551614
http://www.postgresql.org/docs/current/static/functions-aggregate.html
There is also an extension to provide an additional uint64 datatype in postgresql. See Github
It is by Peter Eisentraut
I came across this post (What is the difference between tinyint, smallint, mediumint, bigint and int in MySQL?) and realized that PostgreSQL does not support unsigned integer.
Can anyone help to explain why is it so?
Most of the time, I use unsigned integer as auto incremented primary key in MySQL. In such design, how can I overcome this when I port my database from MySQL to PostgreSQL?
Thanks.
It's not in the SQL standard, so the general urge to implement it is lower.
Having too many different integer types makes the type resolution system more fragile, so there is some resistance to adding more types into the mix.
That said, there is no reason why it couldn't be done. It's just a lot of work.
It is already answered why postgresql lacks unsigned types. However I would suggest to use domains for unsigned types.
http://www.postgresql.org/docs/9.4/static/sql-createdomain.html
CREATE DOMAIN name [ AS ] data_type
[ COLLATE collation ]
[ DEFAULT expression ]
[ constraint [ ... ] ]
where constraint is:
[ CONSTRAINT constraint_name ]
{ NOT NULL | NULL | CHECK (expression) }
Domain is like a type but with an additional constraint.
For an concrete example you could use
CREATE DOMAIN uint2 AS int4
CHECK(VALUE >= 0 AND VALUE < 65536);
Here is what psql gives when I try to abuse the type.
DS1=# select (346346 :: uint2);
ERROR: value for domain uint2 violates check constraint "uint2_check"
You can use a CHECK constraint, e.g.:
CREATE TABLE products (
id integer,
name text,
price numeric CHECK (price > 0)
);
Also, PostgreSQL has serial, smallserial and bigserial types for auto-increment.
The talk about DOMAINS is interesting but not relevant to the only possible origin of that question. The desire for unsigned ints is to double the range of ints with the same number of bits, it's an efficiency argument, not the desire to exclude negative numbers, everybody knows how to add a check constraint.
When asked by someone about it, Tome Lane stated:
Basically, there is zero chance this will happen unless you can find
a way of fitting them into the numeric promotion hierarchy that doesn't
break a lot of existing applications. We have looked at this more than
once, if memory serves, and failed to come up with a workable design
that didn't seem to violate the POLA.
What is the "POLA"? Google gave me 10 results that are meaningless. Not sure if it's politically incorrect thought and therefore censored. Why would this search term not yield any result? Whatever.
You can implement unsigned ints as extension types without too much trouble. If you do it with C-functions, then there will be about no performance penalties at all. You won't need to extend the parser to deal with literals because PgSQL has such an easy way to interpret strings as literals, just write '4294966272'::uint4 as your literals. Casts shouldn't be a huge deal either. You don't even need to do range exceptions, you can just treat the semantics of '4294966273'::uint4::int as -1024. Or you can throw an error.
If I wanted this, I would have done it. But since I'm using Java on the other side of SQL, to me it is of little value since Java doesn't have those unsigned integers either. So I gain nothing. I'm already annoyed if I get a BigInteger from a bigint column, when it should fit into long.
Another thing, if I did have the need to store 32 bit or 64 bit types, I can use PostgreSQL int4 or int8 respectively, just remembering that the natural order or arithmetic won't work reliably. But storing and retrieving is unaffected by that.
Here is how I can implement a simple unsigned int8:
First I will use
CREATE TYPE name (
INPUT = uint8_in,
OUTPUT = uint8_out
[, RECEIVE = uint8_receive ]
[, SEND = uint8_send ]
[, ANALYZE = uint8_analyze ]
, INTERNALLENGTH = 8
, PASSEDBYVALUE ]
, ALIGNMENT = 8
, STORAGE = plain
, CATEGORY = N
, PREFERRED = false
, DEFAULT = null
)
the minimal 2 functions uint8_in and uint8_out I must first define.
CREATE FUNCTION uint8_in(cstring)
RETURNS uint8
AS 'uint8_funcs'
LANGUAGE C IMMUTABLE STRICT;
CREATE FUNCTION uint64_out(complex)
RETURNS cstring
AS 'uint8_funcs'
LANGUAGE C IMMUTABLE STRICT;
need to implement this in C uint8_funcs.c. So I go use the complex example from here and make it simple:
PG_FUNCTION_INFO_V1(complex_in);
Datum complex_in(PG_FUNCTION_ARGS) {
char *str = PG_GETARG_CSTRING(0);
uint64_t result;
if(sscanf(str, "%llx" , &result) != 1)
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg("invalid input syntax for uint8: \"%s\"", str)));
return (Datum)SET_8_BYTES(result);
}
ah well, or you can just find it done already.
According to the latest documentation, the signed integer is supported but no unsigned integer in the table. However, the serial type is kind of similar to unsigned except it starts from 1 not from zero. But the upper limit is the same as signed. So the system truly does not have unsigned support. As pointed out by Peter, the door is open to implement the unsigned version. The code may have to be updated a lot, just too much work from my experience working with C programming.
https://www.postgresql.org/docs/10/datatype-numeric.html
integer 4 bytes typical choice for integer -2147483648 to +2147483647
serial 4 bytes autoincrementing integer 1 to 2147483647
Postgres does have an unsigned integer type that is unbeknownst to many: OID.
The oid type is currently implemented as an unsigned four-byte integer. […]
The oid type itself has few operations beyond comparison. It can be
cast to integer, however, and then manipulated using the standard
integer operators. (Beware of possible signed-versus-unsigned confusion
if you do this.)
It is not a numeric type though, and trying to do any arithmetic (or even bitwise operations) with it is going to fail. Also, it's just 4 bytes (INTEGER), there is no corresponding 8 byte (BIGINT) unsigned type.
So it's not really a good idea to use this yourself, and I agree with all the other answers that in a Postgresql database design you should always use an INTEGER or BIGINT column for your serial primary key - having it start in the negative (MINVALUE) or allowing it to wrap around (CYCLE) if you want to exhaust the full domain.
However, it is quite useful for input/output conversion, like your migration from another DBMS. Inserting the value 2147483648 into an integer column will lead to an "ERROR: integer out of range", while using the expression 2147483648::OID works just fine.
Similarly, when selecting an integer column as text with mycolumn::TEXT, you will get negative values at some point, but with mycolumn::OID::TEXT you will always get a natural number.
See an example at dbfiddle.uk.
I am putting few insert queries in a stored procedure. The insert queries are independently working fine without any issues like :
Implicit conversion from datatype 'TEXT' to 'VARCHAR' is not allowed.
But when the sp is run, it gives the above error for 3 queries.
Checked all the columns, non of them are TEXT type.
Has anyone has faced this issue, any clue would help.
It appears that the problem is not with the stored procedure at all. The error happens when the input exceeds 8000 characters. SQL Server 2000 doesn't have VARCHAR(MAX), the maximum length for VARCHAR is 8000. So, if you try to pass a longer string to your sp, it need to do a conversion to TEXT, but it can't be an implicit conversion, so you need a parameter of type TEXT. Of course, you would need to change your sp, and there are many operations that can't be done on a column of this datatype, so you may be unable to actually do what you want.