PostgreSQL: Return filtered rows along with total number of rows matching a condition - postgresql

Say I have a table, posts, that I'm able to query and filter on a client app. Each post has a type associated with it, and I'd like to be able to see both the filtered posts on the client and the total number of rows that match the filters by type on a dashboard. Obviously, I'd like to do this in a single query. It's also important to note that I'm paginating the data so I can't just use filter(...).length in some backend logic, as there might be 100000 posts but only 10 returned to the client.
Here's my query that correctly filters the data:
knex('posts').select('id', 'created_at', 'content', 'type').modify((builder) => (
filterPosts(builder, filters)
)).paginate({currentPage: offset, perPage: limit})
I'm wondering if there's some way to count the number of posts (by type) that match the filters, and return those counts in my existing query.
E.g. my results currently look like this:
[
{
id: 123,
created_at: "Jan 1, 2022",
content: "Lorem ipsum",
type: "Type 1"
},
{
id: 456,
created_at: "Feb 1, 2022",
content: "Ipsum dolor",
type: "Type 2"
}
...
]
and I'd like something like this:
[
{
id: 123,
created_at: "Jan 1, 2022",
content: "Lorem ipsum",
type: "Type 1"
countType1: 3, // Total rows where type = "Type 1" that match the filters
countType2: 6 // Total rows where type = "Type 2" that match the filters
},
{
id: 456,
created_at: "Feb 1, 2022",
content: "Ipsum dolor",
type: "Type 2",
countType1: 3,
countType2: 6
}
...
]
I've tried using a window function but so far can only get the number of posts of the current row's type, not all types
knex('posts').select(
'id',
'created_at',
'content',
'type',
'count(*) over (partition by posts.type)' // If I could add a WHERE clause here I'd be golden
).modify((builder) => (
filterPosts(builder, filters)
)).paginate({currentPage: offset, perPage: limit})
The above gives:
[
{
id: 123,
created_at: "Jan 1, 2022",
content: "Lorem ipsum",
type: "Type 1"
count: 3,
},
{
id: 456,
created_at: "Feb 1, 2022",
content: "Ipsum dolor",
type: "Type 2",
count: 6
}
...
]
Which isn't optimal since it's possible that 10 posts of only Type 2 are returned to the client due to the pagination, making the client think there are 0 posts of Type 1.
Open to suggestions on how to improve this, any help is greatly appreciated!
This post seems to be on the right track but I can't figure out how to get it working for my scenario

I was able to solve this by building off that post I linked:
knex
.with(
'posts',
knex
.from(
knex('posts').select(
{ id: 'posts.id' },
{ created_at: 'posts.created_at' },
{ content: 'posts.content' },
{ type: 'posts.type' },
)
)
.modify((builder) => filterPosts(builder, filters)) // Contains joins and where clauses
)
.from('posts')
.rightJoin(
knex.raw(
"(select count(1) filter(where posts.type = 'Type 1'), count(1) filter(where posts.type = 'Type 2') from posts) c(type_1_count, type_2_count) on true"
)
)
.select(
'posts.*',
'type_1_count',
'type_2_count'
);

Related

Filtering items within object that is multiple level down with MongoQuery

Please consider below model.
interface Model {
title: string
content: string
comments: {
content: string
likes?: number
subComments: {
parentCommentId: string
content: string
likes?: number
}[]
}[]
}
const post: Model = {
title: 'test',
content: 'content',
comments: [
{
content: 'level2',
likes: 3,
subComments: [
{
parentCommentId: '1',
content: 'level3',
likes: 3,
},
],
},
{
content: 'level2',
likes: 3,
subComments: [
{
parentCommentId: '1',
content: 'level3',
likes: 5,
},
{
parentCommentId: '1',
content: 'level3',
likes: 5,
},
],
},
],
}
Let's say we have a post that has comment that has subComment.
The level of subComment is at level 3 but the depth is fixed.
Is there a way to filter subComments that has optional key "likes" and whose value is greater than 3 with MongoClient?
I don't know exactly what output do you want but you can try something like this example using only projection into find query:
Here there are two $filter and one map.
The $map get each object into comments array and:
Set the content as it is: content: "$$comment.content"
Set the subcomment as an array filtered only by these one objects where likes is greater than 3.
*I've not added likes because it seems no congruent but can be added using likes: "$$comment.likes"
So this produce an array with only obbjects where there are more than 3 likes.
That's mean it can produce an empty subComments array. So the result of the $map is used into another $filter to get only object where subComments is not empty (i.e. there is at least one comment with more than 3 likes).
db.collection.find({},
{
title: 1,
content: 1,
comments: {
$filter: {
input: {
$map: {
input: "$comments",
as: "comment",
in: {
content: "$$comment.content",
subComments: {
$filter: {
input: "$$comment.subComments",
as: "subComment",
cond: {
$gt: [
"$$subComment.likes",
3
]
}
}
}
}
}
},
cond: {
$ne: [
"$$this.subComments",
[]
]
}
}
}
})
Example here.
Which is also the same to use $project in an aggregate query: example

How can I sort data by the value of a flutter map? [duplicate]

This question already has answers here:
How to sort a Map by its value in dart?
(3 answers)
Closed 2 months ago.
i'm trying to sort the values i get from my realtime database converted to Map. This is what I get:
{Table 2:
{
order 1:
{
WATER: {price: 1.50, quantity: 2},
date: 2022-12-14 17:34:51.428972}
},
Table 1:
{
order 2:
{
date: 2022-12-14 17:35:04.761532,
COCA COLA: {price: 2, quantity: 1}
},
order 1:
{
DAISY: {price: 7, quantity: 1},
date: 2022-12-14 17:22:12.678864,
4 CHEESES: {price: 7, quantity: 1},
WATER: {price: 1.50, quantity: 2}
}
}
}
What I want is to compare all orders by date so as to get them chronologically, resulting in something like this:
{Table 1:
{
order 1:
{
DAISY: {price: 7, quantity: 1},
dates: 2022-12-14 17:22:12.678864,
4 CHEESES: {price: 7, quantity: 1},
WATER: {price: 1.50, quantity: 2}
},
Table 2:
{
order 1:
{
WATER: {price: 1.50, quantity: 2},
date: 2022-12-14 17:34:51.428972}
},
Table 1:
{
order 2:
{
dates: 2022-12-14 17:35:04.761532,
COCA COLA: {price: 2, quantity: 1}
},
},
}
you could easily sort any list based on date using this way
newData.sort((a, b) {
return (a['date'] as DateTime).compareTo(b['date'] as DateTime);
});
but your data is map not list and map does not have a sort or order at all ,
you should change your data structure first into array then sort it
somethings like this should work
final tables = <String, Map<String, Map>>{}; // ...your data here
final newData = tables.entries
.map(
(table) => table.value.entries.map(
(order) => {"table": table.key, "order": order.key, ...order.value},
),
)
.expand(
(element) => element,
)
.toList();
newData.sort((a, b) {
return (a['date'] as DateTime).compareTo(b['date'] as DateTime);
});

MongoDB Aggregate by date on many fields

I have the following schema (abbreviated!)
{
date: {
_id: false,
day: { type: Number },
month: { type: Number },
year: { type: Number },
},
flightTimes: {
_id: false,
totalTime: Number,
...
},
loggedTimes: {
_id: false,
P1: {
type: Number,
},
P2: {
type: Number,
},
...
},
customFields: [
{
_id: false,
value: String,
type: {
type: String,
enum: ["text", "number", "time", "yesno", "paragraph"],
required: true,
},
},
],
}
I am trying to produce the following output using aggregation, but struggling.
{
total: [10,20,30],
P1: [45, 55, 12],
P2: [0, 12, 15],
}
Where the 1st item in the total array is the sum of any documents where Month = 1 (January), the second is the sum for February etc.
Any suggestions on how to proceed? For bonus points: the array should have a 0 if no documents match the current month. For more bonus points: how could I create an additional array for the sum of values in each custom field where type = time.
Ultimately looking to produce a graph with Charts.JS which shows the sum per month for each category of time.

mongoDB Query not returning anything

db.studentSeminar.insert({
seminar: {
seminarID: "sem004",
seminarDescription: "In-memory Database.",
seminarDate: "2-May-2020",
creditPoint: 3,
seminarEnrolment: [
{ studentID: "std004", enrolmentDate: "20-April-2020" },
{ studentID: "std002", enrolmentDate: "10-April-2020" },
],
},
student: [
{
studentID: "std004",
studentName: "Klein Acevedo",
address: "Block 23, Toa Payoh",
telephone: [{ handphone1: "93589248", handphone2: "82354723" }],
enrolTo: [{ seminarID: "sem004", markReceived: 64 }],
},
{
studentID: "std002",
studentName: "Selma Hobbs",
address: "87 Pearl Hill",
telephone: [{ handphone: "91647249", residentphone: "62691355" }],
enrolTo: [{ seminarID: "sem004", markReceived: 89 }],
},
],
});
With a simple collection of mongoDB above, i'm trying to do a simple find query below
db.studentSeminar.find({"seminarDescription": "In-memory Database."})
However, it's not returning anything, or showing up any error message
Can anyone help me? I'm still new to MongoDB
Thank you in advance
You have to use the dot . operator, since seminarDescription is a nested field of a document.
The below will work for you:
db.studentSeminar.find({
"seminar.seminarDescription": "In-memory Database."
})

MongoDB linked relation

I have two tables users (fields: name, age) and links (fields: title, url, userId(ref to users)). There is a one-to-many relation between users to links. Now I want to run a query like : select * from links where user has age more than 26.
I have been trying with query like :
Links
.find({title:'Chemistry'})
.populate({
path: 'userId',
select: 'name.first age -_id',
match: {age: {$gte: 26}}
})
.exec( function(err,data) {
if (err) console.log( err );
console.log( 'data', data);
res.jsonp(data);
});
This gives me the following result :
[
{
_id: "54037d00e74f00f917e709ff",
title: "Chemistry",
url: "http://chemistry.com",
comment: "Great chemo slides",
favourites: "507",
userId: {
name: {
first: "Bob"
},
age: 31
},
tags: [
"tutorials",
"chemo"
]
},
{
_id: "540457f2557cded82b331ee2",
title: "Chemistry",
url: "http://chemistry.com",
comment: "Great chemo slides",
favourites: "507",
userId: null,
tags: [
"tutorials",
"chemo"
]
}
]
Notice that the userId is null in the second object. I am not sure my query is right, but I think it should be a simple enough query for mongodb to support it. I probably don't know how to do it, do you have any idea about this ? Thanks.