In PostgreSQL, I can get the upper and lower boundary of a CIDR-range, like below.
But how can I get the CIDR from two IP addresses (by SQL) ?
e.g.
input "192.168.0.0";"192.168.255.255"
output "192.168.0.0/16"
SELECT
network
,network::cidr
-- http://technobytz.com/ip-address-data-types-postgresql.html
--,netmask(network::cidr) AS nm
--,~netmask(network::cidr) AS nnm
,host(network::cidr) AS lower
,host(broadcast(network::cidr)) AS upper -- broadcast: last address in the range
,family(network::cidr) as fam -- IPv4, IPv6
,masklen(network::cidr) as masklen
FROM
(
SELECT CAST('192.168.1.1/32' AS varchar(100)) as network
UNION SELECT CAST('192.168.0.0/16' AS varchar(100)) as network
--UNION SELECT CAST('192.168.0.1/16' AS varchar(100)) as network
) AS tempT
I think you are looking for inet_merge:
test=> SELECT inet_merge('192.168.0.0', '192.168.128.255');
┌────────────────┐
│ inet_merge │
├────────────────┤
│ 192.168.0.0/16 │
└────────────────┘
(1 row)
Related
As a follow up to this question How to concatenate strings of a string field in a PostgreSQL 'group by' query?
I am looking for a way to concatenate the strings of a field within a WITH RECURSIVE query (and NOT using GORUP BY). So for example, I have a table:
ID COMPANY_ID EMPLOYEE
1 1 Anna
2 1 Bill
3 2 Carol
4 2 Dave
5 3 Tom
and I wanted to group by company_id, ordered by the count of EMPLOYEE, to get something like:
COMPANY_ID EMPLOYEE
3 Tom
1 Anna, Bill
2 Carol, Dave
It's simple with GROUP BY:
SELECT company_id, string_agg(employee, ', ' ORDER BY employee) AS employees
FROM tbl
GROUP BY company_id
ORDER BY count(*), company_id;
Sorting in a subquery is typically faster:
SELECT company_id, string_agg(employee, ', ') AS employees
FROM (SELECT company_id, employee FROM tbl ORDER BY 1, 2) t
GROUP BY company_id
ORDER BY count(*), company_id;
As academic proof of concept: an rCTE solution without using any aggregate or window functions:
WITH RECURSIVE rcte AS (
(
SELECT DISTINCT ON (1)
company_id, employee, ARRAY[employee] AS employees
FROM tbl
ORDER BY 1, 2
)
UNION ALL
SELECT r.company_id, e.employee, r.employees || e.employee
FROM rcte r
CROSS JOIN LATERAL (
SELECT t.employee
FROM tbl t
WHERE t.company_id = r.company_id
AND t.employee > r.employee
ORDER BY t.employee
LIMIT 1
) e
)
SELECT company_id, array_to_string(employees, ', ') AS employees
FROM (
SELECT DISTINCT ON (1)
company_id, cardinality(employees) AS emp_ct, employees
FROM rcte
ORDER BY 1, 2 DESC
) sub
ORDER BY emp_ct, company_id;
db<>fiddle here
Related:
Select first row in each GROUP BY group?
Optimize GROUP BY query to retrieve latest row per user
Concatenate multiple result rows of one column into one, group by another column
No group by here:
select * from tarded;
┌────┬────────────┬──────────┐
│ id │ company_id │ employee │
├────┼────────────┼──────────┤
│ 1 │ 1 │ Anna │
│ 2 │ 1 │ Bill │
│ 3 │ 2 │ Carol │
│ 4 │ 2 │ Dave │
│ 5 │ 3 │ Tom │
└────┴────────────┴──────────┘
(5 rows)
with recursive firsts as (
select id, company_id,
first_value(id) over w as first_id,
row_number() over w as rn,
count(1) over (partition by company_id) as ncompany,
employee
from tarded
window w as (partition by company_id
order by id)
), names as (
select company_id, id, employee, rn, ncompany
from firsts
where id = first_id
union all
select p.company_id, c.id, concat(p.employee, ', ', c.employee), c.rn, p.ncompany
from names p
join firsts c
on c.company_id = p.company_id
and c.rn = p.rn + 1
)
select company_id, employee
from names
where rn = ncompany
order by ncompany, company_id;
┌────────────┬─────────────┐
│ company_id │ employee │
├────────────┼─────────────┤
│ 3 │ Tom │
│ 1 │ Anna, Bill │
│ 2 │ Carol, Dave │
└────────────┴─────────────┘
(3 rows)
I have a table with a column that needs to be split and inserted into a new table. Column's name is location and has data that could look like Detroit, MI, USA;Chicago, IL, USA or as simple as USA.
Ultimately, I want to insert the data into a new dimension table that looks like:
City | State | Country|
Detroit MI USA
Chicago IL USA
NULL NULL USA
I came across the string_to_array function and am able to split the larger example (Detroit, MI, USA; Chicago, IL, USA) into 2 strings of Detroit, MI, USA and Chicago, IL, USA.
Now I'm stumped on how to split those strings again and then insert them. Since there are two strings separated by a comma, does using string_to_array again work? It doesn't seem to work in Sqlfiddle.
Note: I'm using Sqlfiddle right now since I don't have access to my Redshift table at the moment.
This is for Redshift, which unfortunately is still using PostGresql 8.0.2 and thus does not have the unnest function
postgres=# select v[1] as city, v[1] as state, v[2] as country
from (select string_to_array(unnest(string_to_array(
'Detroit, MI, USA;Chicago, IL, USA',';')),',')) s(v);
┌─────────┬─────────┬─────────┐
│ city │ state │ country │
╞═════════╪═════════╪═════════╡
│ Detroit │ Detroit │ MI │
│ Chicago │ Chicago │ IL │
└─────────┴─────────┴─────────┘
(2 rows)
Tested on Postgres, not sure if it will work on Redshift too
Next query should to work on every Postgres
select v[1] as city, v[1] as state, v[2] as country
from (select string_to_array(v, ',') v
from unnest(string_to_array(
'Detroit, MI, USA;Chicago, IL, USA',';')) g(v)) s;
It use old PostgreSQL trick - using derived table.
SELECT v[1], v[2] FROM (SELECT string_to_array('1,2',',')) g(v)
Unnest function:
CREATE OR REPLACE FUNCTION _unnest(anyarray)
RETURNS SETOF anyelement AS '
BEGIN
FOR i IN array_lower($1,1) .. array_upper($1,1) LOOP
RETURN NEXT $1[i];
END LOOP;
RETURN;
END;
' LANGUAGE plpgsql;
following is my sample table and rows
create table com (company text,val int);
insert into com values ('com1',1),('com1',2),('com1',3),('com1',4),('com1',5);
insert into com values ('com2',11),('com2',22),('com2',33),('com2',44),('com2',55);
insert into com values ('com3',111),('com3',222),('com3',333),('com3',444),('com3',555);
I want to get the top 3 value of each company, expected output is :
company val
---------------
com1 5
com1 4
com1 3
com2 55
com2 44
com2 33
com3 555
com3 444
com3 333
Try This:
SELECT company, val FROM
(
SELECT *, ROW_NUMBER() OVER (PARTITION BY
company order by val DESC) AS Row_ID FROM com
) AS A
WHERE Row_ID < 4 ORDER BY company
--Quick Demo Here...
Since v9.3 you can do a lateral join
select distinct com_outer.company, com_top.val from com com_outer
join lateral (
select * from com com_inner
where com_inner.company = com_outer.company
order by com_inner.val desc
limit 3
) com_top on true
order by com_outer.company;
It might be faster but, of course, you should test performance specifically on your data and use case.
You can try arrays, which are available since Postgres v9.0.
WITH com_ordered AS (SELECT * FROM com ORDER BY company,val DESC)
SELECT company,unnest((array_agg(val))[0:3])
FROM com_ordered GROUP BY company;
I have a table "test_networks" that is a list of networks with a description about what each network is and where it is located.
CREATE TABLE test_networks
(
id serial PRIMARY KEY,
address cidr,
description text
);
The field "address" will be any of the following:
10.0.0.0/8
10.1.0.0/16
10.1.1.0/24
10.2.0.0/16
10.3.0.0/16
10.3.1.0/24
10.3.2.0/24
10.3.3.0/24
10.15.1.0/24
10.15.2.0/24
10.15.3.0/24
I also have a table "test_systems" which contains a list of systems and their properties (I have a few more properties, but those are irrelevant):
CREATE TABLE test_systems
(
id serial PRIMARY KEY,
address inet,
owner text
);
Lets assume I have systems with the following addresses:
10.1.1.1
10.2.0.1
I want to create a report of all systems and their closest networks description (or empty description if no network is found). As you can see, 10.1.1.1 matches multiple networks, so I only want to list the most specific one (i.e. the one with the highest masklen()) for each system. Example output would be:
hostaddr | netaddr | description
----------+-------------+----------------
10.1.1.1 | 10.1.1.0/24 | third network
10.2.0.1 | 10.2.0.0/16 | 4th network
I tried using this query:
SELECT s.address AS hostaddr, n.address AS netaddr, n.description AS description
FROM test_systems s
LEFT JOIN test_networks n
ON s.address << n.address;
However, this will give me a list of all system + network pairs, e.g.:
hostaddr | netaddr | description
----------+-------------+----------------
10.1.1.1 | 10.0.0.0/8 | first network
10.1.1.1 | 10.1.0.0/16 | second network
10.1.1.1 | 10.1.1.0/24 | third network
10.2.0.1 | 10.0.0.0/8 | first network
10.2.0.1 | 10.2.0.0/16 | 4th network
Does anyone know how I can query for only the most specific network for each system?
You're looking for the "top n in group" query, where n = 1 in this case. You can do this using the row_number() window function:
SELECT x.hostaddr, x.netaddr, x.description FROM (
SELECT
s.address AS hostaddr,
n.address AS netaddr,
n.description AS description,
row_number() OVER (
PARTITION BY s.address
ORDER BY masklen(n.address) DESC
) AS row
FROM test_systems s
LEFT JOIN test_networks n
ON s.address << n.address
) x
WHERE x.row = 1;
SELECT distinct on (s.address)
s.address AS hostaddr,
n.address AS netaddr,
n.description AS description
FROM
test_systems s
LEFT JOIN
test_networks n ON s.address << n.address
order by s.address, masklen(n.address) desc
The utf encoded string contains Zwnj(zero width non joiner) at the end and is stored in database.
Is it possible to remove that character during select statement. I tried trim() but doesn't work.
CREATE TABLE test (x text);
INSERT INTO test VALUES (E'abc');
INSERT INTO test VALUES (E'foo\u200C'); -- U+200C = ZERO WIDTH NON-JOINER
SELECT x, octet_length(x) FROM test;
x │ octet_length
─────┼──────────────
abc │ 3
foo │ 6
(2 rows)
CREATE TABLE test2 AS SELECT replace(x, E'\u200C', '') AS x FROM test;
SELECT x, octet_length(x) FROM test2;
x │ octet_length
─────┼──────────────
abc │ 3
foo │ 3
(2 rows)
You need to use replace(your_column, 'Zwnj','') instead of trim()