I'm working on a migration project (MongoDB to Snowflake) and trying to convert one of the mongo queries to Snowflake, we have a use case to fetch records if all the elements from an array matched based on the given parameters.
Mongo DB Function: $all
The $all operator selects the documents where
the value of a field is an array that contains all the specified
elements.
Mongo Query:
db.collection('collection_name').find({
'code': { '$in': [ 'C0001' ] },
'months': { '$all': [ 6, 7, 8, 9 ] } --> 6,7,8,9 given parameters
});
Table Structure in snowflake:
column name datatype
code varchar(50)
id int
months ARRAY
weeks ARRAY
Could you provide some suggestions on how to write this query in Snowflake?
Any recommendations would be helpful.
You can use ARRAY_SIZE and ARRAY_INTERSECTION to test it:
Here is sample table:
create or replace table test ( code varchar, months array );
insert into test select 'C0002', array_construct( 1, 2, 5,8,6,3)
union all select 'C0001', array_construct( 1,2,3 )
union all select 'C0002', array_construct( 2, 12, 3,7,9)
union all select 'C0001', array_construct( 7,8,9,3, 2) ;
Here is query to test:
select * from test where code in ('C0001','C0002')
and ARRAY_SIZE( ARRAY_INTERSECTION( months, array_construct( 3, 2 ))) = 2;
So I find the intersection of two arrays, and check the number of items. If you want to look for 3 items, you should set 3 (and it goes like this):
select * from test where code in ('C0001','C0002')
and ARRAY_SIZE( ARRAY_INTERSECTION( months, array_construct( 3, 2, 7 ))) = 3;
Related
Currently, I have 1 table consisting of id and otherIds
I want to calculate the sum of otherIds present in database.
id: 1, otherIds: {1,2,3,4,5}
id: 2, otherIds: {3,4,5}
id: 3, otherIds: {9,2,1}
id: 4, otherIds: {}
Desired result: 11 (5 + 3 + 3 + 0)
SELECT
sum(jsonb_array_elements("table"."otherIds")) as "sumLength"
FROM
"Table"
LIMIT 1
[42883] ERROR: function jsonb_array_elements(uuid[]) does not exist
I don't see how JSONB is relevant here. If otherIds is an array of UUID values then wouldn't you just need
SELECT
SUM(ARRAY_LENGTH("table"."otherIds")) as "sumLength"
FROM
"Table"
LIMIT 1
You can get the number of elements in an array with the cardinality() function. Just sum the results over all rows.
I'd like to remark that a table design that includes an array of UUIDs is not pretty and will probable gibe you performance and data integrity problems some day.
I made a similar post before, but deleted it as it had contextual errors.
One of the tables in my database includes a JSONB column which includes an array of JSON objects. It's not dissimilar to this example of a session table which I've mocked up below.
id
user_id
snapshot
inserted_at
1
37
{cart: [{product_id: 1, price_in_cents: 3000, name: "product A"}, {product_id: 2, price_in_cents: 2500, name: "product B"}]}
2022-01-01 20:00:00.000000
2
24
{cart: [{product_id: 1, price_in_cents: 3000, name: "product A"}, {product_id: 3, price_in_cents: 5500, name: "product C"}]}
2022-01-02 20:00:00.000000
3
88
{cart: [{product_id: 4, price_in_cents: 1500, name: "product D"}, {product_id: 2, price_in_cents: 2500, name: "product B"}]}
2022-01-03 20:00:00.000000
The query I've worked with to retrieve records from this table is as follows.
SELECT sessions.*
FROM sessions
INNER JOIN LATERAL (
SELECT *
FROM jsonb_to_recordset(sessions.snapshot->'cart')
AS product(
"product_id" integer,
"name" varchar,
"price_in_cents" integer
)
) AS cart ON true;
I've been trying to update the query above to retrieve only the records in the sessions table for which ALL of the products in the cart have a price_in_cents value of greater than 2000.
To this point, I've not had any success on forming this query but I'd be grateful if anyone here can point me in the right direction.
You can use a JSON path expression:
select *
from sessions
...
where not sessions.snapshot ## '$.cart[*].price_in_cents <= 2000'
There is no JSON path expression that would check that all array elements are greater 2000. So this returns those rows where no element is smaller than 2000 - because that can be expressed with a JSON path expression.
Here is one possible solution based on the idea of your original query.
Each element of the cart JSON array object is joined to its sessions parent row. You 're left adding the WHERE clause conditions now that the wanted JSON array elements are exposed.
SELECT *
FROM (
SELECT
sess.id,
sess.user_id,
sess.inserted_at,
cart_items.cart_name,
cart_items.cart_product_id,
cart_items.cart_price_in_cents
FROM sessions sess,
LATERAL (SELECT (snapshot -> 'cart') snapshot_cart FROM sessions WHERE id = sess.id) snap_arr,
LATERAL (SELECT
(value::jsonb ->> 'name')::text cart_name,
(value::jsonb -> 'product_id')::int cart_product_id,
(value::jsonb -> 'price_in_cents')::int cart_price_in_cents
FROM JSONB_ARRAY_ELEMENTS(snap_arr.snapshot_cart)) cart_items
) session_snapshot_cart_product;
Explanation :
From the sessions table, the cart array is exctracted and joined per sessions row
The necessary items of the cart JSON array is then unnested by the second join using the JSONB_ARRAY_ELEMENTS(jsonb) function
The following worked well for me and allowed me the flexibility to use different comparison operators other than just ones such as == or <=.
In one of the scenarios I needed to construct, I also needed to have my WHERE in the subquery also compare against an array of values using the IN comparison operator, which was not viable using some of the other solutions that were looked at.
Leaving this here in case others run into the same issue as I did, or if others find better solutions or want to propose suggestions to build upon this one.
SELECT *
FROM sessions
WHERE NOT EXISTS (
SELECT sessions.*
FROM sessions
INNER JOIN LATERAL (
SELECT *
FROM jsonb_to_recordset(sessions.snapshot->'cart')
AS product(
"product_id" integer,
"name" varchar,
"price_in_cents" integer
)
) AS cart ON true
WHERE name ILIKE "Product%";
)
I have a doubt with modification of jsonb data type in postgres
Basic setup:-
array=> ["1", "2", "3"]
and now I have a postgresql database with an id column and a jsonb datatype column named lets just say cards.
id cards
-----+---------
1 {"1": 3, "4": 2}
thats the data in the table named test
Question:
How do I convert the cards of id->1 FROM {"1": 3, "4": 2} TO {"1": 4, "4":2, "2": 1, "3": 1}
How I expect the changes to occur:
From the array, increment by 1 all elements present inside the array that exist in the cards jsonb as a key thus changing {"1": 3} to {"1": 4} and insert the values that don't exist as a key in the cards jsonb with a value of 1 thus changing {"1":4, "4":2} to {"1":4, "4":2, "2":1, "3":1}
purely through postgres.
Partial Solution
I asked a senior for support regarding my question and I was told this:-
Roughly (names may differ): object keys to explode cards, array_elements to explode the array, left join them, do the calculation, re-aggregate the object. There may be a more direct way to do this but the above brute-force approach will work.
So I tried to follow through it using these two functions json_each_text(), json_array_elements_text() but ended up stuck halfway into this as well as I was unable to understand what they meant by left joining two columns:-
SELECT jsonb_each_text(tester_cards) AS each_text, jsonb_array_elements_text('[["1", 1], ["2", 1], ["3", 1]]') AS array_elements FROM tester WHERE id=1;
TLDR;
Update statement that checks whether a range of keys from an array exist or not in the jsonb data and automatically increments by 1 or inserts respectively the keys into the jsonb with a value of 1
Now it might look like I'm asking to be spoonfed but I really haven't managed to find anyway to solve it so any assistance would be highly appreciated 🙇
The key insight is that with jsonb_each and jsonb_object_agg you can round-trip a JSON object in a subquery:
SELECT id, (
SELECT jsonb_object_agg(key, value)
FROM jsonb_each(cards)
) AS result
FROM test;
(online demo)
Now you can JOIN these key-value pairs against the jsonb_array_elements of your array input. Your colleague was close, but not quite right: it requires a full outer join, not just a left (or right) join to get all the desired object keys for your output, unless one of your inputs is a subset of the other.
SELECT id, (
SELECT jsonb_object_agg(COALESCE(obj_key, arr_value), …)
FROM jsonb_array_elements_text('["1", "2", "3"]') AS arr(arr_value)
FULL OUTER JOIN jsonb_each(cards) AS obj(obj_key, obj_value) ON obj_key = arr_value
) AS result
FROM test;
(online demo)
Now what's left is only the actual calculation and the conversion to an UPDATE statement:
UPDATE test
SET cards = (
SELECT jsonb_object_agg(
COALESCE(key, arr_value),
COALESCE(obj_value::int, 0) + (arr_value IS NOT NULL)::int
)
FROM jsonb_array_elements_text('["1", "2", "3"]') AS arr(arr_value)
FULL OUTER JOIN jsonb_each_text(cards) AS obj(key, obj_value) ON key = arr_value
);
(online demo)
# things :string is an Array
scope :things, ->(q) { where('ARRAY[?]::varchar[] IN things', Array.wrap(q)) }
scope :things, ->(q) { where('things && ARRAY[?]::varchar[]', Array.wrap(q)) }
scope :things, ->(q) { where('ARRAY[?]::varchar[] <# things', Array.wrap(q)) }
I've tried a few versions, but I can't seem to find the proper incantation. I'm looking to find any row that has any of the things in the array... is there any overlap?
[1, 2, 3] & [1, 8] = t
[1, 2, 3] & [8, 9] = f
I'm trying to mimic ActiveRecord's default where behavior. If I give it an array, it'll get all the matching rows. Is this possible with postgres arrays? Is it even efficient?
One way of doing this is by converting the arrays to a set of rows. Once you have the arrays as set of rows, you can do an intersection between them and check if the result is empty set.
For example:
CREATE TABLE my_test_table(id BIGINT, test_array BIGINT[]);
INSERT INTO my_test_table(id, test_array)
VALUES
(1, array[1,2,3]),
(2, ARRAY[1,5,8]);
SELECT * FROM my_test_table
WHERE array_length((SELECT array
(
SELECT UNNEST(test_array)
INTERSECT
SELECT UNNEST(array[3,15,2])
)), 1) > 0;
The result of the SELECT statement above is:
1 | {1,2,3}
This allows for more complex matching of elements of 2 arrays. For example, if you would like to select the arrays that have at least 2 common elements, you could just change the WHERE part to
WHERE array_length((SELECT array
(
SELECT UNNEST(test_array)
INTERSECT
SELECT UNNEST(array[3,15,2])
)), 1) > 1;
(Sorry for my bad English)
I have an array of users ids as below:
[5, 9, 3, 22, 16]
Obviously the values are dynamic.
Now, I need to SELECT all users but users with above ids come first.
What I've tried so far?
This query gives me exact answer :
SELECT * FROM users WHERE id IN (5, 9, 3, 22, 16)
UNION ALL
SELECT * FROM users WHERE id NOT IN (5, 9, 3, 22, 16);
But is there any better way?
P.S:
I'm using PostgreSQL 10.
Try this:
select *
from users
order by id not in (5, 9, 3, 22, 16), id
As stated in the documentation, an expression used in the ORDER BY clause
can be an arbitrary expression formed from input-column values.
In particular, you can use a Boolean expression, as values of this type are sortable. Note that false < true.
You can use CASE statements on your ORDER BY, I'll give you an example:
select *
from users
order by
CASE WHEN id=5 THEN 1
WHEN id=9 THEN 2
WHEN id=3 THEN 3
WHEN id=22 THEN 4
WHEN id=16 THEN 5
END, id
With CASE you can tell postgres the "priority" or value you want for each id if you know them beforehand. After that I added "id" so the rest of the rows gets ordered properly.