When I'm running query below I get message:
[Amazon](500310) Invalid operation: WITH RECURSIVE is not supported;
Can someone explain me why recursive function doesn't work? (I'm working on amazon redshift)
WITH RECURSIVE r AS (
SELECT
1 AS i,
1 AS factorial
UNION
SELECT
i+1 AS i,
factorial * (i+1) as factorial
FROM r
WHERE i < 10
)
SELECT * FROM r;
The official Amazon Redshift documentation: Unsupported PostgreSQL Features:
These PostgreSQL features are not supported in Amazon Redshift.
... ...
- Recursive common table expressions
...
WITH RECURSIVE is now supported in Redshift starting from April 29th, 2021:
https://aws.amazon.com/about-aws/whats-new/2021/04/amazon-redshift-announces-support-for-heirarchical-data-queries-with-recursive-cte/
https://docs.aws.amazon.com/redshift/latest/dg/r_WITH_clause.html#r_WITH_clause-recursive-cte
Related
does PostgreSQL support function in with Clause?
oracle has the functionality as below
WITH
function add_fnc(p_id number) return number
is
begin
return p_id + 1;
end;
select add_fnc(id1) from test_tbl;
There is no such thing as CTE functions in PostgreSQL (as cofemann noted), but it seems what you are looking for are temporary functions. Take a look How to create a temporary function in PostgreSQL?
We are in the process of migrating a MySQL 5.7 database to PostgreSQL 9.6.
A real issue is the lack of bit_count function in PostgreSQL. This function is also not available in the upcoming version 10.
Current MySQL code snippet (simplified):
-- mysql specific, tested with 5.7.19
select code,phash,bit_count(phash ^ -9187530158960050433) as hd
from documents
where phash is not null and bit_count(phash ^ -9187530158960050433) < 7
order by hd;
We tried a naive solution (converting the BIGINT to a String and counting the "1"'s), but it performs terribly compared to MySQL.
Java uses a trick from Hacker's Delight, but AFAIK this is not possible with PostgreSQL, because the >>> operator is (also) not available.
Question: Is a there solution/workaround available comparable with MySQL performance wise?
UPDATE 1
Best solution i could find is based on this SO answer:
First create bit_count function:
CREATE OR REPLACE FUNCTION bit_count(value bigint)
RETURNS numeric
AS $$ SELECT SUM((value >> bit) & 1) FROM generate_series(0, 63) bit $$
LANGUAGE SQL IMMUTABLE STRICT;
Now we can use almost the same SQL as with MySQL:
-- postgresql specific, tested with 9.6.5
select code,phash,bit_count(phash # -9187530158960050433) as hd
from documents
where phash is not null and bit_count(phash # -9187530158960050433) < 7
order by hd;
UPDATE 2
Based on #a_horse_with_no_name comment, i tried this function:
-- fastest implementation so far. 10 - 11 x faster than the naive solution (see UPDATE 1)
CREATE OR REPLACE FUNCTION bit_count(value bigint)
RETURNS integer
AS $$ SELECT length(replace(value::bit(64)::text,'0','')); $$
LANGUAGE SQL IMMUTABLE STRICT;
However, this is still 5 - 6 times slower than MySQL (tested wit exact the same data set of 200k phash values on the same hardware).
Function bit_count is available since PostgreSQL version 14, see
Bit String Functions and Operators.
Example:
select bit_count(B'1101');
Result is 3.
Note that the function is defined for types bit and bit varying. So if you want to use it with integer values, you need to cast.
Example:
select cast (cast (1101 as text) as bit varying);
Result is B'1101'.
Combining both examples:
select bit_count(cast (cast (1101 as text) as bit varying));
Question: Is a there solution/workaround available comparable with
MySQL performance wise?
To get a comparable speed, a compiled C function should be used.
If you can compile C code, see for instance
https://github.com/dverite/postgresql-functions/tree/master/hamming_weight
The code itself is very simple.
The result seems 10 times faster than the bit_count function based on counting the 0 characters in the bit(64) string as text.
Example:
plpgsql function:
test=> select sum(bit_count(x)) from generate_series(1,1000000) x;
sum
---------
9884999
(1 row)
Time: 2442,340 ms
C function:
test=> select sum(hamming_weight(x::int8)) from generate_series(1,1000000) x;
sum
---------
9884999
(1 row)
Time: 239,749 ms
If you are trying to compute the hamming distance of perceptual hashes or similar LSH bit strings, then this question may be a closely related to this answer
If you are looking specifically for a pre-built way to do hamming distance queries on a PostgreSQL database, then this may be the cure: an extension for hamming distance search
i tried to run this query in DB2 ( which includes regex ). I am getting the following error. Can someone help?
Here is the query:
SELECT COUNT(*) FROM TABLE WHERE REGEXP_LIKE(TRIM(FIELD), '[^[:digit:]]')
Support for BOOLEAN data type is new in Db2 11.1.1.1 (i.e. the first Mod Pack + Fix pack for Db2 11.1). If you are only on Db2 11.1.0.0, then you will need to explicitly test the result of your regex function.
SELECT COUNT(*) FROM TABLE
WHERE REGEXP_LIKE(TRIM(FIELD), '[^[:digit:]]') = 1;
I have the following issue:
Right now I have a table with the subnet/mask information (example 192.168.1.0 / 255.255.255.0 ) .. but I need to obtain the MAX and MIN IP from this subnet:
192.168.1.0 / 192.168.1.255
I've found this answer:
how to query for min or max inet/cidr with postgres
But it seems that:
network_smaller(inet, inet) and network_larger(inet, inet)
Doesn't exists. Even googling that I can't find any answer for those functions.
Thanks!
Edit:
Version info:
PostgreSQL 9.2.15 on x86_64-redhat-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4), 64-bit
I don't think that question is relavent to your needs anyway. The min and max defined there are similar to the SQL min() and max() functions for finding the smallest / largest in a table, not the smallest / largest in a subnet.
I'm not generally a fan of relying on undocumented features. They may be safe but may isn't a word I generally like.
There's a page of documented network functions here:
https://www.postgresql.org/docs/current/static/functions-net.html
The two you would need would be:
Min would be network(inet)
Max would be broadcast(inet)
That's because the network name is always the "first" ip in the range and the broadcast address is always the "last" ip in the range.
Don't google, just try:
select network_smaller('192.168.0.9'::inet, '192.168.0.11'::inet);
network_smaller
-----------------
192.168.0.9
(1 row)
Postgres has more than 2,600 internal functions. Most of them are useful for creating operator classes of various types. Not all of them are described in the documentation, but they are all generally available.
You can find them using pgAdmin III in pg_catalog. You only need to set the option: File -> Options -> UI Miscellaneous -> Show System Objects in treeview.
The aggregate functions min(inet) and max(inet) has been introduced in Postgres 9.5:
with test(ip) as (
values
('192.168.0.123'::inet),
('192.168.0.12'),
('192.168.0.1'),
('192.168.0.125')
)
select max(ip), min(ip)
from test;
max | min
---------------+-------------
192.168.0.125 | 192.168.0.1
(1 row)
See how the aggregate min(inet) is defined (it can be found in pg_catalog):
CREATE AGGREGATE min(inet) (
SFUNC=network_smaller,
STYPE=inet,
SORTOP="<"
);
The question How to query for min or max inet/cidr with postgres concerned Postgres 9.4. In my answer I suggested to use the functions network_smaller(inet, inet) and network_larger(inet, inet). I'm sure they were added for creating aggregate functions min(inet) and max(inet) but for some reasons (maybe oversight) the aggregates appeared only in Postgres 9.5.
In Postgres 9.2 you can create your own functions as substitutes, e.g.
create or replace function inet_larger(inet, inet)
returns inet language sql as $$
select case when network_gt($1, $2) then $1 else $2 end
$$;
create or replace function inet_smaller(inet, inet)
returns inet language sql as $$
select case when network_lt($1, $2) then $1 else $2 end
$$;
I'm looking for a DB2 function to calculate hashes on large CLOB values in order to quickly track changes. Other engines have functions such as CHECKSUM,CRC32 or MD5. The function in LUW is GET_HASH_VALUE but is not available in zOS.
Constraints: No access to UDFs or Stored Procedures.
Here is a quick and dirty code fragment that computes a CRC32, it only works to about 100 characters.
WITH crc(t,c,j) AS (
SELECT 'Hello World!',4294967295,0 FROM SYSIBM.SYSDUMMY1
UNION ALL
SELECT SUBSTR(t,2),bitxor(c,ASCII(t)),8 FROM crc WHERE t>'' AND j=0
UNION ALL
SELECT t,BITXOR(c/2,BITAND(3988292384,-BITAND(c,1))),j-1 FROM crc WHERE j>0
)
SELECT RIGHT(HEX(BITNOT(c)),8) FROM CRC WHERE t='' AND j=0
Result checked against http://www.lammertbies.nl/comm/info/crc-calculation.html :
1
--------
1C291CA3
Source: http://www.hackersdelight.org/hdcodetxt/crc.c.txt
The answer depends on which version of DB2 you have. If you are on DB2 9.7 or higher, have a look here: https://www.ibm.com/support/knowledgecenter/SSEPGG_9.7.0/com.ibm.db2.luw.sql.rtn.doc/doc/r0055167.html