Please help me to build a proper PostgreSQL query with filled gaps in data according time interval.
I have data set:
[
{
"date": "2022-10-09T10:00:00+00:00",
"name": "entity 1",
"value": 10
},
{
"date": "2022-10-09T10:00:01+00:00",
"name": "entity 1",
"value": 11
},
{
"date": "2022-10-09T10:00:02+00:00",
"name": "entity 1",
"value": 10.5
},
{
"date": "2022-10-09T10:00:05+00:00",
"name": "entity 1",
"value": 11.5
}
]
DB:
╔════════════════════════════╦══════════╦═══════╗
║ date ║ name ║ value ║
╠════════════════════════════╬══════════╬═══════╣
║ 2022-10-09T10:00:00+00:00 ║ entity 1 ║ 10 ║
║ 2022-10-09T10:00:01+00:00 ║ entity 1 ║ 11 ║
║ 2022-10-09T10:00:02+00:00 ║ entity 1 ║ 10.5 ║
║ 2022-10-09T10:00:05+00:00 ║ entity 1 ║ 11.5 ║
╚════════════════════════════╩══════════╩═══════╝
The time interval as you might see is 1 second.
Expected result:
[
{
"date": "2022-10-09T10:00:00+00:00",
"name": "entity 1",
"value": 10
},
{
"date": "2022-10-09T10:00:01+00:00",
"name": "entity 1",
"value": 11
},
{
"date": "2022-10-09T10:00:02+00:00",
"name": "entity 1",
"value": 10.5
},
{
"date": "2022-10-09T10:00:03+00:00",
"name": "entity 1",
"value": null
},
{
"date": "2022-10-09T10:00:04+00:00",
"name": "entity 1",
"value": null
},
{
"date": "2022-10-09T10:00:05+00:00",
"name": "entity 1",
"value": 11.5
}
]
So I need to build a query with filled gaps for absent rows according the time interval, so we need to display rows with times 2022-10-09T10:00:03+00:00 and 2022-10-09T10:00:04+00:00.
I would be very appreciated for some example of the needed query or any kind of advice!
This might help:
WITH RECURSIVE date_series (d) AS (
SELECT min(date) FROM tbl
UNION ALL
SELECT d + '1 second'::interval FROM date_series
WHERE d < (SELECT max(date) FROM tbl))
SELECT
date_series.d,
name,
value
FROM
date_series
LEFT JOIN tbl ON date_series.d = tbl.date
ORDER BY
date_series.d;
Replace tbl with your table name.
Related
So this is my jsonb array of objects. Column is called bids in my db.
bids column
[
{
"id": "1",
"size": "5.5Y",
"price": 180
},
{
"id": "f0d1d36a-f6af-409e-968e-54c1dc104566",
"size": "6.5Y",
"price": 22
}
]
I want to update price property by the ID of an element for ex. "f0d1d36a-f6af-409e-968e-54c1dc104566", so the price would change from 22 to 150 IN ROW WHICH CONTAINS ELEMENT WITH DESIRED ID IN THE COLUMN.
How can I do that?
create table json_update (id integer, json_fld jsonb);
insert into json_update values (1, '[
{
"id": "1",
"size": "5.5Y",
"price": 180
},
{
"id": "f0d1d36a-f6af-409e-968e-54c1dc104566",
"size": "6.5Y",
"price": 22
}
]'
)
;
UPDATE
json_update
SET
json_fld = jsonb_set(json_fld, ARRAY[(idx)::text, 'price'::text], '150'::jsonb)
FROM (
SELECT
(row_number() OVER (ORDER BY t.a ->> 'id') - 1) AS idx,
t.a
FROM (
SELECT
jsonb_array_elements(json_fld)
FROM
json_update) AS t (a)) AS i
WHERE
i.a ->> 'id' = 'f0d1d36a-f6af-409e-968e-54c1dc104566';
select * from json_update ;
id | json_fld
----+---------------------------------------------------------------------------------------------------------------------------
1 | [{"id": "1", "size": "5.5Y", "price": 180}, {"id": "f0d1d36a-f6af-409e-968e-54c1dc104566", "size": "6.5Y", "price": 150}]
a have a simple table of purchases consisting of id and jsonb column, like this:
CREATE TABLE purchase (id SERIAL, products JSONB)
Then and index:
CREATE INDEX idx_purchase_products ON purchase USING GIN (products);
The sample data are like this:
INSERT INTO purchase VALUES (
1, jsonb('[
{
"country": 1,
"type": 1,
"size": 10,
"color": 3
}
]')
),
(
2, jsonb('[
{
"country": 1,
"type": 1,
"size": 10,
"color": 3
},
{
"country": 1,
"type": 2,
"size": 12,
"color": 4
},
{
"country": 2,
"type": 1,
"size": 12,
"color": 3
}
]')
),
(
3, jsonb('[
{
"country": 1,
"type": 1,
"size": 10,
"color": 3
}
]')
),
(
4, jsonb('[
{
"country": 1,
"type": 1,
"size": 10,
"color": 3
},
{
"country": 1,
"type": 2,
"size": 12,
"color": 4
},
{
"country": 2,
"type": 1,
"size": 12,
"color": 3
}
]')
);
And some scenarios of searching:
SELECT *
FROM purchase
WHERE products #> '[{"country": 1}]'
SELECT *
FROM purchase
WHERE products #> '[{"country": 1, "type": 1}]'
SELECT *
FROM purchase
WHERE products #> '[{"size": 12}]'
SELECT *
FROM purchase
WHERE products #> '[{"size": 12, "color": 4}]'
It is expected, that the customer could search for combinations:
country,
country + type
country + type + size
country + type + size + color
country + size
size + color
type + color
etc.
And there is a big chance, the list of 4 field (country, type, size, color) will grow in future to 7-10.
And of course we want also search combinations like this:
.. WHERE products #> '[{"country": 1}]' OR products #> '[{"color": 4}]' OR products #> '[{"type": 1, "size": 10}]'
Estimated size of the table purchase is 9-12 millions rows (depending on season).
Any idea how to implement the indexes to get the query result as fast as possible?
I am trying to get the output from DB using an inner join with 3 tables
say Table A and B.
Output Struct
type C struct {
A A `json:"A"`
B B `json:"B"`
SecID int64 `json:"section_id"`
SecName string `json:"section_name"`
}
type A struct {
AID int64 `json:"aid"`
Name string `json:"name"`
Des string `json:"des"`
Price string `json:"price"`
}
type B struct {
BID int64 `json:"bid"`
Answer string `json::answer"`
Score int16 `json:"score"`
}
DB query
var cs []C
rows, err := db.Query(sqlStatement, RequestBody.tID)
for rows.Next() {
var c C
err = rows.Scan(&c.A.ID, &c.A.Name, &c.A.Des, &c.A.Price, &c.A.Price, &c.B.ID, &c.B.Answer, &c.B.Score, &c.SecID, &c.SecName)
cs = append(cs, c)
The above code result in the following output:
[
{
"a": {
"aid": 1,
"name": "XXXXXX",
"description": "addd kdjd a jdljljlad",
"price": "10",
},
"section_id": 1,
"section_name": "personal details",
"b": {
"bid": 1,
"answer": "adfdf d fd d f",
"score": 0
}
},
{
"a": {
"aid": 1,
"name": "XXXXXX",
"description": "addd kdjd a jdljljlad",
"price": "10",
},
"section_id": 1,
"section_name": "personal details",
"b": {
"bid": 2,
"answer": "adfdf d fd d f",
"score": 10
}
}
]
But I am trying to merge field "b" in one single field with the list of dictionaries and writing "a" field only once as the values are repeated.
[
{
"a": {
"aid": 1,
"name": "XXXXXX",
"description": "addd kdjd a jdljljlad",
"price": "10",
},
"b": [
{
"section_id": 1,
"section_name": "personal details",
"bid": 1,
"answer": "adfdf d fd d f",
"score": 0
},
{
"section_id": 1,
"section_name": "personal details",
"bid": 2,
"answer": "adfdf d fd d f",
"score": 10
}
]
}
]
Tried changing the struct but doesn't seem to work.
DB details:
Table A (AID, Name, Des, Place)
Table B (BID, Answer, Score)
Query:
select * from A a
inner join temp_table tt on tt.aid = a.aid
inner join B b on b.bid = tt.bid
where a.aid=1;
How to merge below 2 records into single row using Azure data flows
Account Number
Phone type
Home Number
Mobile Number
1234
Home
989 324 1234
1234
Mobile
9893238893
Required OutPut
Account Number
Home Number
Mobile Number
1234
989 324 1234
989 323 8893
Assuming NULL values will be holded in the empty columns, you can simply use a max function in an aggregate activity to achieve the expected behaviour.
Test case I used, an inline dataset:
SELECT 1234 AS accountNumber, 'Home' AS phoneType, '989 324 1234' AS homeNumber, null AS mobileNumber
UNION ALL
SELECT 1234 AS accountNumber, 'Mobile' AS phoneType, null AS homeNumber, '9893238893' AS mobileNumber
The aggregate stage:
Result in data preview:
Complete json for the dataflow:
{
"name": "dataflow1",
"properties": {
"type": "MappingDataFlow",
"typeProperties": {
"sources": [
{
"linkedService": {
"referenceName": "azuresqluat1",
"type": "LinkedServiceReference"
},
"name": "source1"
}
],
"sinks": [],
"transformations": [
{
"name": "Aggregate1"
}
],
"script": "source(output(\n\t\taccountNumber as integer,\n\t\tphoneType as string,\n\t\thomeNumber as string,\n\t\tmobileNumber as string\n\t),\n\tallowSchemaDrift: true,\n\tvalidateSchema: false,\n\tformat: 'query',\n\tstore: 'sqlserver',\n\tquery: 'SELECT 1234 AS accountNumber, \\'Home\\' AS phoneType, \\'989 324 1234\\' AS homeNumber, \\'\\' AS mobileNumber\\nUNION ALL\\nSELECT 1234 AS accountNumber, \\'Mobile\\' AS phoneType, \\'\\' AS homeNumber, \\'9893238893\\' AS mobileNumber',\n\tisolationLevel: 'READ_UNCOMMITTED') ~> source1\nsource1 aggregate(groupBy(accountNumber),\n\thomeNumber = max(homeNumber),\n\t\tmobileNumber = max(mobileNumber)) ~> Aggregate1"
}
}
}
I have the the table orders which contains a column called stores with a jsonb with many elements like
id status stores
1 in progress [{"id": 1, "lat": 19.41, "lng": -99.18, "name": "Domino´s pizza condesa", "products": "3 tacos de cabeza"},
{"id": 2, "lat": 19.03, "lng": -99.9, "name": "Papa guapa roma", "products": "una papa"}]
1 done [{"id": 3, "lat": 19.44, "lng": -99.28, "name": "ABC", "products": "3 tacos de cabeza"},
{"id": 4, "lat": 19.23, "lng": -99.29, "name": "Papa guapa roma", "products": "una papa"}]
I want to query the table orders and select only the element from the json that matches {"lat:19.41", "lng":-99.18}
So I get something like
id status store_filtered
1 in progress [{"id": 1, "lat": 19.41, "lng": -99.18, "name": "Domino´s pizza condesa", "products": "3 tacos de cabeza"}]
I currently have
SELECT id, status, stores->0 AS stores_filtered FROM orders WHERE stores #> '[{"lat":19.41, "lng": -99.18}]'
But that 0 is what I need to make dynamic, to give me the element that matches not the first.
with r as (
select status, unnest(stores) as stores
from orders
)
select * from r where stores->lat='19.41' and stores->lng='-99.18'
Asssuming that you're using PostgreSQL 9.5 (or newer), you can use the jsonb_array_elements(json) (or jsonb_array_elements(json)) functions to "unnest" the array, and then choose.
For instance, you can try:
WITH orders(id, status, stores) AS
(
VALUES
(1, 'in progress',
'[{"id": 1, "lat": 19.41, "lng": -99.18, "name": "Domino´s pizza condesa", "products": "3 tacos de cabeza"},
{"id": 2, "lat": 19.03, "lng": -99.9, "name": "Papa guapa roma", "products": "una papa"}]'::jsonb),
(1, 'done',
'[{"id": 3, "lat": 19.44, "lng": -99.28, "name": "ABC", "products": "3 tacos de cabeza"},
{"id": 4, "lat": 19.23, "lng": -99.29, "name": "Papa guapa roma", "products": "una papa"}]')
)
, expanded_stores AS
(
SELECT
id, status, jsonb_array_elements(stores) AS single_store
FROM
orders
)
SELECT
id, status, single_store AS filtered_store
FROM
expanded_stores
WHERE
single_store #> '{"lat":19.41, "lng": -99.18}'
Or, assuming that you already have declared your orders table, you could just use:
SELECT
id, status, single_store AS filtered_store
FROM
(
SELECT
id, status, jsonb_array_elements(stores) AS single_store
FROM
orders
) AS expanded_stores
WHERE
single_store #> '{"lat":19.41, "lng": -99.18}' ;
You can check version 9.5 at http://rextester.com/MQE36487.
NOTE: Although this does work, it looks a bit "unnatural" for a SQL database. It is normally easier to normalize the information, and reserve JSON just where a lot of schema flexibility is really needed.