flask_mongo_engine how to paginate order by create_time desc? - mongodb

class Session(db.DynamicDocument):
title = db.StringField(required=False)
created = db.FloatField(required=True)
source = db.StringField()
#app.route('/page/<int:page>')
def sessions(page):
return jsonify(Session.objects.paginate(page=page, per_page=5).items)
The response is list of dict which is ordered ASC and first page contains the oldest documents by default.
So how to get the desc order result?

Just add order_by() before paginate().
keys may be prefixed with + or - to determine the ordering direction
Just an example:
class Session(db.DynamicDocument):
create_time = db.DateTimeField(required=True)
# clean before test
for i in Session.objects.all():
i.delete()
for i in range(20):
create_time = datetime.now()
create_time = create_time.replace(year=2000 + i)
ses = Session(create_time=create_time)
ses.save()
print('2000 -> 2009')
items = Session.objects.order_by('create_time').paginate(page=1, per_page=10).items
for i in items:
print(i.create_time)
print('2019 -> 2010')
items = Session.objects.order_by('-create_time').paginate(page=1, per_page=10).items
for i in items:
print(i.create_time)
Hope this helps.

Related

Group By with Entity Framework

enter image description hereI have a code. And there you need to make a grouping by name.
//<date,<partid,amount>>
Dictionary<string, Dictionary<int, double>> emSpending = new Dictionary<string, Dictionary<int, double>>();
foreach (Orders order in db.Orders.ToList())
{
foreach (OrderItems orderitem in order.OrderItems.ToList())
{
if (!emSpending.ContainsKey(order.Date.ToString("yyyy-MM"))) emSpending.Add(order.Date.ToString("yyyy-MM"), new Dictionary<int, double>());
if (!emSpending[order.Date.ToString("yyyy-MM")].ContainsKey(Convert.ToInt32(orderitem.PartID))) emSpending[order.Date.ToString("yyyy-MM")].Add(Convert.ToInt32(orderitem.PartID), 0);
emSpending[order.Date.ToString("yyyy-MM")][Convert.ToInt32(orderitem.PartID)] += Convert.ToDouble(orderitem.Amount);
}
}
DataGridViewColumn col1 = new DataGridViewColumn();
col1.CellTemplate = new DataGridViewTextBoxCell();
col1.Name = "Department";
col1.AutoSizeMode = DataGridViewAutoSizeColumnMode.Fill;
col1.HeaderText = "Department";
dgvEMSpending.Columns.Add(col1);
foreach (string date in emSpending.Keys)
{
DataGridViewColumn col = new DataGridViewColumn();
col.Name = date;
col.HeaderText = date;
col.AutoSizeMode = DataGridViewAutoSizeColumnMode.Fill;
col.CellTemplate = new DataGridViewTextBoxCell();
dgvEMSpending.Columns.Add(col);
}
List<string> allKey = emSpending.Keys.ToList();
foreach (string date in allKey)
if (date == "Department") continue;
else
{
dgvEMSpending.Rows.Add();
foreach (int partid in emSpending[date].Keys)
{
dgvEMSpending.Rows[dgvEMSpending.Rows.Count - 1].Cells[0].Value = db.Parts.Where(x => x.ID == partid).SingleOrDefault().Name.GroupBy(Name);
for (int i = 1; i < dgvEMSpending.Columns.Count; i++)
{
if (!emSpending.ContainsKey(dgvEMSpending.Columns[i].Name)) emSpending.Add(dgvEMSpending.Columns[i].Name, new Dictionary<int, double>());
if (!emSpending[dgvEMSpending.Columns[i].Name].ContainsKey(partid)) emSpending[dgvEMSpending.Columns[i].Name].Add(partid, 0);
double val = emSpending[dgvEMSpending.Columns[i].Name][partid];
dgvEMSpending.Rows[dgvEMSpending.RowCount - 1].Cells[i].Value = val;
}
}
}
I tried to use group by myself, but something doesn't work. It just outputs the same names, and I want to group them so that there is a grouping. Pls helped to me.
Ok, a few issues to help you out first. This code:
foreach (Orders order in db.Orders.ToList())
{
foreach (OrderItems orderitem in order.OrderItems.ToList())
{
if (!emSpending.ContainsKey(order.Date.ToString("yyyy-MM"))) emSpending.Add(order.Date.ToString("yyyy-MM"), new Dictionary<int, double>());
if (!emSpending[order.Date.ToString("yyyy-MM")].ContainsKey(Convert.ToInt32(orderitem.PartID))) emSpending[order.Date.ToString("yyyy-MM")].Add(Convert.ToInt32(orderitem.PartID), 0);
emSpending[order.Date.ToString("yyyy-MM")][Convert.ToInt32(orderitem.PartID)] += Convert.ToDouble(orderitem.Amount);
}
}
Right off the bat this is going to trip lazy loading on OrderItems. If you have 10 orders 1-10 you're going to be running 11 queries against the database:
SELECT * FROM Orders;
SELECT * FROM OrderItems WHERE OrderId = 1;
SELECT * FROM OrderItems WHERE OrderId = 2;
// ...
SELECT * FROM OrderItems WHERE OrderId = 10;
Now if you have 100 orders or 1000 orders, you should see the problem. At a minimum ensure that if you are touching a collection or reference on entities you are loading, eager load it with Include:
foreach (Orders order in db.Orders.Include(x => x.OrderItems).ToList())
This will run a single query that fetches the Orders and their OrderItems. However, if you have a LOT of rows this is going to take a while and consume a LOT of memory.
The next tip is "only load what you need". You need 1 field from Order and 2 fields from OrderItem. So why load everything from both tables??
var orderItemDetails = db.Orders
.SelectMany(o => o.OrderItems.Select(oi => new { o.Date, oi.PartId, oi.Amount })
.ToList();
This would give us just the Order date, and each Part ID and Amount. Now that this data is in memory we can group it to populate your desired dictionary structure without having to iterate over it row by row.
var emSpending = orderItemDetails.GroupBy(x => x.Date.ToString("yyyy-MM"))
.ToDictionary(g => g.Key,
g => g.GroupBy(y => y.PartId)
.ToDictionary(g2 => g2.Key, g2 => g2.Sum(z => z.Amount)));
Depending on the Types in your entities you may need to insert casts. This first groups the outer dictionary of the yyyy-MM of the order dates, then it groups the remaining data for each date by part ID, and sums the Amount.
Now relating to your question, from your code example I'm guessing the problem area you are facing is this line:
dgvEMSpending.Rows[dgvEMSpending.Rows.Count - 1].Cells[0].Value = db.Parts
.Where(x => x.ID == partid)
.SingleOrDefault().Name.GroupBy(Name);
Now the question would be to explain what exactly you are expecting from this? You are fetching a single Part by ID. How would you expect this to be "grouped"?
If you want to display the Part name instead of the PartId then I believe you would just want to Select the Part Name:
dgvEMSpending.Rows[dgvEMSpending.Rows.Count - 1].Cells[0].Value = db.Parts
.Where(x => x.ID == partid)
.Select(x => x.Name)
.SingleOrDefault();
We can go one better to fetch the Part names for each used product in one hit using our loaded order details:
var partIds = orderItemDetails
.Select(x=> x.PartId)
.Distinct()
.ToList();
var partDetails = db.Parts
.Where(x => partIds.Contains(x.ID))
.ToDictionary(x => x.ID, x => x.Name);
This fetches us a dictionary set indexed by ID for the part names, it would be done outside of the loop after we had loaded the orderItemDetails. Now we don't have to go to the DB with every row:
dgvEMSpending.Rows[dgvEMSpending.Rows.Count - 1].Cells[0].Value = partDetails[partId];

Flask Translate Choice in Form To ForeignKey Pointing to Another Table

This part of the app works, buts it's ugly and not sustainable. Need a more evolved solution.
PROBLEM I AM TRYING TO SOLVE:
This part of the application enables users to access a form to enter purchases they've made and store them in a Postgres DB. I am using Flask SQLAlchemy ORM.
Within my purchase table exists a field store_id, that has a ForeignKey relationship to my store table. I don't want my user to select a store ID # in the form, so I am using a SelectField to enable them to choose the store name instead. However, I can't seem to find a sustainable way to translate the store name back to its associated ID. Right now I am using the ugly IF statements seen below.
What is a better way to map/translate store name to ID which is already housed in the "store" table in my DB?
MY MODEL:
class Purchase(db.Model):
id = db.Column(db.Integer, primary_key=True)
item = db.Column(db.String(80))
quantity = db.Column(db.Integer)
unit_cost = db.Column(db.Integer)
total_cost= db.Column(db.Integer)
date = db.Column(db.DateTime)
store_id = db.Column(db.Integer, db.ForeignKey('store.id'))
MY FORM:
class CreatePurchase(FlaskForm):
item = StringField('Item', validators=[DataRequired()])
quantity = IntegerField('Quantity', validators=[DataRequired()])
unit_cost = IntegerField('Unit Cost', validators=[DataRequired()])
total_cost = IntegerField('Total Cost', validators=[DataRequired()])
store_id = SelectField("Store Selector", choices=[('0','Select Store'),('1','Furgesons'), ('2','Ocean State'), ('3','Chewy'), ('4','Amazon'), ('5', 'Rumford')])
date = DateField('Purchase Date', validators=[DataRequired()])
submit = SubmitField('Enter')
MY ROUTE:
#main.route('/enter_purchases', methods=['GET', 'POST'])
def enter_purchase():
form = CreatePurchase()
x = str(form.store_id) # this is a store name from our form
p = 0
if "Ocean State" in x:
p = 2
elif "Amazon" in x:
p = 4
elif "Furgesons" in x:
p = 1
elif "Chewy" in x:
p = 3
elif "Rumford" in x:
p = 5
if form.validate_on_submit():
purchase = Purchase(item=form.item.data, quantity=form.quantity.data, unit_cost=form.unit_cost.data,
total_cost=form.total_cost.data, date=form.date.data,store_id=p)
db.session.add(purchase)
db.session.commit()
flash('New purchase added successfully')
return redirect(url_for('main.success'))
return render_template('enter_purchases.html', form=form)
You have a store table, with a numeric id (as the PK) and a name attribute:
class Store(db.Model):
store_id = ..
store_name = ..
You populate your form with all of the unique values from the store_name. This needs to be a dynamically generated form, so instead of using a form that is statically created do something like:
def CreatePurchase()
class TempForm(FlaskForm):
item = StringField('Item', validators=[DataRequired()])
quantity = IntegerField('Quantity', validators=[DataRequired()])
unit_cost = IntegerField('Unit Cost', validators=[DataRequired()])
total_cost = IntegerField('Total Cost', validators=[DataRequired()])
date = DateField('Purchase Date', validators=[DataRequired()])
submit = SubmitField('Enter')
choices = ## some SQLalchemy code to get all unique store_names from the table
TempForm.store_id = SelectField("Store Selector", choices=choices)
return TempForm()
Then form.store_id will provide the right id, but display the string name of the store in the form.
The key in this setup is making sure you use the right SQLalchemy code to populate the SelectField dynamically. Basically for each store in the table you can just iterate through something like this:
choices = list()
for store in Store.query.all(): # <- assumes store_id and store_names are unique
choices.append((store.store_id, store.store_name))

pg-promise update where in custom array

How can the following postgresql query be written using the npm pg-promise package?
update schedule
set student_id = 'a1ef71bc6d02124977d4'
where teacher_id = '6b33092f503a3ddcc34' and (start_day_of_week, start_time) in (VALUES ('M', (cast('17:00:00' as time))), ('T', (cast('19:00:00' as time))));
I didn't see anything in the formatter namespace that can help accomplish this. https://vitaly-t.github.io/pg-promise/formatting.html
I cannot inject the 'cast' piece into the '17:00:00' value without it being considered part of the time string itself.
The first piece of the query is easy. It's the part after VALUES that i can't figure out.
First piece:
var query = `update schedule
set student_id = $1
where teacher_id = $2 and (start_day_of_week, start_time) in (VALUES $3)`;
var inserts = [studentId, teacherId, values];
I'm using this messiness right now for $3 (not working yet), but it completely bypasses all escaping/security built into pg-promise:
const buildPreparedParams = function(arr, colNames){
let newArr = [];
let rowNumber = 0
arr.forEach((row) => {
const rowVal = (rowNumber > 0 ? ', ' : '') +
`('${row.startDayOfWeek}', (cast('${row.startTime}' as time)))`;
newArr.push(rowVal);
});
return newArr;
};
The structure I am trying to convert into this sql query is:
[{
"startTime":"17:00:00",
"startDayOfWeek":"U"
},
{
"startTime":"16:00:00",
"startDayOfWeek":"T"
}]
Use CSV Filter for the last part: IN (VALUES $3:csv).
And to make each item in the array format itself correctly, apply Custom Type Formatting:
const data = [{
startTime: '17:00:00',
startDayOfWeek: 'U'
},
{
startTime: '16:00:00',
startDayOfWeek: 'T'
}];
const values = data.map(d => ({
toPostgres: () => pgp.as.format('(${startDayOfWeek},(cast(${startTime} as time))', d),
rawType: true
}));
Now passing in values for $3:csv will format your values correctly:
('U',(cast('17:00:00' as time)),('T',(cast('16:00:00' as time))

Map to avoid Duplicate records

I have a Holiday Custom object,it contains Unique Dates based on
country based How to retrieve the value in Map
I have tried:
List<Holiday__c> holidays = new List<Holiday__c>([Select id, Date__c,Country__c from Holiday__c]);
Map<date, Holiday__c> mapOfHolidays = new Map<date, Holiday__c>();
for (Holiday__c holiday: holidays) mapOfHolidays.put(holiday.Date__c , holiday);
Suppose date for example 3/8/17 is for country france and Italy,If i put Key Value as Date__c,Map does not contain Key as duplicates,But for each country unique Dates are there
So what i have to use Key and Value in Map,I want to retrieve the date using Map,please anyone Help for me the answer
trigger ignoreweekends on order (before insert,before update) {
Set<id> accountIds = new Set<Id>();
for (order ord : Trigger.new) {
accountids.add(ord.AccountId);
}
map<id, account> mapOfAccounts = new map<id, account>([select id,JDE_Date_Delivery_Offset__c from account where id IN: accountIds]);
List<Holiday__c> holidays = new List<Holiday__c>([Select id, Date__c from Holiday__c]);
Map<date, Holiday__c> mapOfHolidays = new Map<date, Holiday__c>();
for (Holiday__c holiday: holidays) mapOfHolidays.put(holiday.Date__c , holiday);
for (order so : Trigger.new) {
Account acc = mapOfAccounts.get(so.AccountId);
for(integer i=0;i<=acc.JDE_Date_Delivery_Offset__c;i++) {
Datetime dt = DateTime.newInstance(Date.today() +acc.JDE_Date_Delivery_Offset__c.longvalue(), Time.newInstance(0, 0, 0, 0));
String dayOfWeek=dt.format('EEEE');
if(dayOfWeek == 'saturday' || dayOfWeek == 'sunday'|| mapOfHolidays.containskey(mapOfHolidays.Date__c))
acc.JDE_Date_Delivery_Offset__c = acc.JDE_Date_Delivery_Offset__c+ 1;
so.JDE_Synchronization_Date__c = Date.today().addDays(acc.JDE_Date_Delivery_Offset__c.intValue());
}
}
}
If I understood correctly, for one unique date you have multiple countries.
For this you can create a map of Date as key and List as value instead of creating a Map of Date and Holiday_c.
For example :
Map<Date, List<Holiday__c>> mapofHolidays = new HashMap<Date, List<Holiday__c>>();
After that you can retrieve all countries for a unique date and do the required processing.

How to search by multi Category IDs with OR Boolean opeeation at Lucence

How to search by multi Category IDs with OR Boolean opeeation at Lucence
We have CategoryID as field at Lucene index, when we are using single value to search in CategoryID field, it is working fine but if we are passing multi values the search doesn't return a values
Example:
Category IDs = 10, 20, 30
Search Data:
Category ID = 10
Results:
It is working fine
Search Data:
Category ID = 10, 20
Results:
It is not working
// search in categories
if (checkedCategories != null && checkedCategories.Count > 0)
{
foreach (int categoryID in checkedCategories)
{
Query queryEntity = new TermQuery(new Term("CategoryID", categoryID.ToString()));
booleanQuery.Add(queryEntity, Occur.MUST);
}
}
I found the answer, it is into the following great article
http://searchhub.org/2011/12/28/why-not-and-or-and-not/
What I did to solve my problem is:
Search by Lucene_KsuDmsDesc = A
Search by Lucene_EmployeeNO = B
Search by Lucene_CreatedBy = C
Search by Lucene_CreatedDateTime = D
Search by Lucene_CategoryID = E
The formula is (A and B and C and D) and (E1 or E2 or .... En), where and = MUST and or = SHOULD