I have a Postgres jsonb column. It has multiple top-level attributes, including sub-hashes and sub-arrays. An example:
{
first: { a: 'a', b: 'b' },
second: ['a', 'b'],
third: { something: 'else' },
...
}
I want to merge multiple attributes into this value, ideally in a single operation. For example, I'd like to be able to merge new keys into first (leaving everything else untouched), and new values into second, resulting in e.g.
{
first: { a: 'a', b: 'b', c: 'c' },
second: ['a', 'b', 'c'],
third: { something: 'else' },
...
}
I've tried using both || operator and jsonb_set but have yet to get this working in a single operation.
For context, I'm doing this in Ruby on Rails using ActiveRecord, but am assuming I'll need to perform this action using raw SQL with the ActiveRecord::Base.connection.execute command. Also, in practice, the values are more complex – 'second' might actually be an array of hashes, but I don't see that impacting how the operation would work.
If necessary, I could perform this in multiple operations (one for 'first', one for 'second' etc), but a single operation would be preferred.
Any help is very much appreciated.
Related
Related to this Is there a way to return all non-null values even if one is null in PostgreSQL? - the solution of which allowed me to return null values, however it returns it into the same key instead of the one assigned.
For this example in particular so I'd like to insert it as A=valueOfA, B=valueOfB instead of A=valueOfA,valueOfB.
select concat_ws(",", A, B, C) into D;
// if C is null, it will return A=valueOfA,valueOfB
Thanks! :)
It might be easier to simply generate a JSON value:
jsonb_build_object('a', a, 'b', b, 'c', c)
This would e.g. include "a": null if column a was null. If you want to remove those, use jsonb_strip_nulls:
jsonb_strip_nulls(jsonb_build_object('a', a, 'b', b, 'c', c))
I have the following schema:
{
a: string;
b: date;
c: number;
}
My query is
find({
a: 'some value',
b: {
$gte: new Date('some date')
}
})
.sort({
c: -1
});
I have an index that is:
{ a: 1, b: 1, c: 1 }
But it's not using this index.
I have several other indexes, and when analyzing my explain(), it shows it's employing multiple other indexes to accomplish my query.
I believe since my "b" query is a date range, that's not considered an equality condition, so maybe that index won't work?
Should I have two indexes:
{ a: 1, c: 1} and separately { b: 1 }
Dates tend to be much more selective than other fields, so when you have an index that looks like {dateField: 1, otherField: 1}, the selectivity of the dateField means that otherField will be useless unless you have multiple items that share the same date.
Depending on what your data distribution actually looks like, you might consider {otherField: 1, dateField: 1} (which means that mongo can go through in sorted order to check whether the docs match your date query). In general, putting your sort field before any fields used in a range query is a good idea.
Mlab's indexing docs are the best resource I've seen on index usage, and they recommend:
A good rule of thumb for queries with sort is to order the indexed fields in this order:
First, the field(s) on which you will query for exact values
Second, one small $in array
Third, the field(s) on which you will sort in the same order and specification as the sort itself (sorting on multiple fields)
Finally, the field(s) on which you will query for a range of values in the order of most selective to least selective (see range operators below)
I have below json string in my table column which is of type jsonb,
{
"abc": 1,
"def": 2
}
i want to remove the "abc" key from it and insert "mno" with some default value. i followed the below approcach for it.
UPDATE books SET books_desc = books_desc - 'abc';
UPDATE books SET books_desc = jsonb_set(books_desc, '{mno}', '5');
and it works.
Now i have another table with json as below,
{
"a": {
"abc": 1,
"def": 2
},
"b": {
"abc": 1,
"def": 2
}
}
Even in this json, i want to do the same thing. take out "abc" and introduce "mno" with some default value. Please help me to achieve this.
The keys "a" and "b" are dynamic and can change. But the values for "a" and "b" will always have same keys but values may change.
I need a generic logic.
Requirement 2:
abc:true should get converted to xyz:1.
abc:false should get converted to xyz:0.
demo:db<>fiddle
Because of a possible variety of your JSON keys it might be complicated to generate a common query. This is because you need to give the path within the json_set() function. But without actual values it would be hard.
A simple work-around is using the regexp_replace() function on the text representation of the JSON string to replace the relevant objects.
UPDATE my_table
SET my_data =
regexp_replace(my_data::text, '"abc"\s*:\s*\d+', '"mno":5', 'g')::jsonb
For added requirement 2:
I wrote the below query based on already given solution:
UPDATE books
SET book_info =
regexp_replace(book_info::text, '"abc"\s*:\s*true', '"xyz":1', 'g')::jsonb;
UPDATE books
SET book_info =
regexp_replace(book_info::text, '"abc"\s*:\s*false', '"xyz":0', 'g')::jsonb;
I have a column (let's call it jsn) in my database with json object (actually stored as plain text for reasons). This json object looks like this:
{"a":
{"b":[{"num":123, ...},
{"num":456, ...},
...,
{"num":789, ...}],
...
},
...
}
I'm interested in the biggest "num" inside that list of objects "b" inside the object "a".
If the list if of known length I can do it like this:
SELECT
GREATEST((jsn::json->'a'->'b'->>0)::int,
(jsn::json->'a'->'b'->>1)::int,
... ,
(jsn::json->'a'->'b'->>N)::int))
FROM table
Note that I'm new to PostgreSQL (and database querying in general!) so that may be a rubbish way to do it. In any case it works. What I can't figure out is how to make this work when the list, 'b', is of arbitrary and unknown length.
In case it is relevant, I am using PostgreSQL 10 hosted on AWS RDS and running queries using pgAdmin 4.
You need to unnest the array then you can apply a max() on the result:
select max((n.x -> 'num')::int)
from the_table t
cross join jsonb_array_elements(t.jsn::jsonb -> 'a' -> 'b') as n(x);
you probably want to add a group by, so that you can distinguish rom which row the max value came from. Assuming your table has a column id that is unique:
select id, max((n.x -> 'num')::int)
from the_table t
cross join jsonb_array_elements(t.jsn::jsonb -> 'a' -> 'b') as n(x)
group by id;
Let's say I have some records as below:
{
id: 1,
age: 22,
name: 'A',
class: 'Y'
},
{
id: 2,
age: 25,
name: 'B',
class: 'D'
},
{
id: 3,
age: 30,
name: 'C',
class: 'Y'
},
{
id: 4,
age: 40,
name: 'D',
class: 'B'
}
Now I need to get the last (closest) record which has an age less than 28. For this, I can use the following code:
const firstUsersYoungerThan28 = await Users.find({
class: 'C',
age: {
$lt: 28
}
})
.sort({
age: -1
})
.limit(1)
.lean();
const firstUserYoungerThan28 = firstUsersYoungerThan28[0];
Let's say the collection have millions of records. My question is, is this the most efficient way? Is there a better way to do this?
My biggest concern is, does my app load the records to the memory in order to sort them in the first place?
See the documentation about cursor.sort():
Limit Results
You can use sort() in conjunction with limit() to return the first (in
terms of the sort order) k documents, where k is the specified limit.
If MongoDB cannot obtain the sort order via an index scan, then
MongoDB uses a top-k sort algorithm. This algorithm buffers the first
k results (or last, depending on the sort order) seen so far by the
underlying index or collection access. If at any point the memory
footprint of these k results exceeds 32 megabytes, the query will
fail.
Make sure that you have an index on age. Then when MongoDB does the sorting, it will only keep the first k (in your case, 1) results in memory.
MongoDB can handle millions of documents for such basic queries, don't worry. Just make sure that you do have the proper indexes specified.
You should create an index with the following properties:
The index filters everyone with an age below 28
The index sorts by currentMeterReading.
This type of index is a partial index. Refer to the documentation about partial indexes and using indexes for sorting for more information.
The following index should do the trick:
db.users.createIndex(
{ age: -1, currentMeterReading: -1},
{ partialFilterExpression: { age: {$lt: 28} }
)
Without the index, a full scan would probably be in order which is very bad for performance.
With the index, only the columns you specified will be stored in memory (when possible) and searching them would be much faster.
Note that MongoDB may load the values of the index lazily instead of on index creation in some or all cases. This is an implementation choice.
As far as I know, there's no way to create an index with only the last record in it.
If you want to understand how databases work you have to understand how indexes work, specifically B-Trees. B-Trees are very common constructs of all databases.
Indexes do have their disadvantages so don't create one for each query. Always measure before creating an index since it might not be necessary.