public long DebitLedgerBalance(string drLedgerId, string transactionId, decimal amount, string userID)
{
var filter = Builders<Transaction>.Filter.Eq(x => x._id, transactionId);
UpdateDefinition<Transaction> update;
update = Builders<Transaction>.Update.Inc(x => x.debitLedgerBalance, amount * -1);
return db.Transaction.UpdateOne(x => x._id == transactionId, update).ModifiedCount;
}
i'm using mongodb driver(v2.11.3) in asp.net core(v3.1) application.
In the code i'm checking equality of transactionId.I also want to check drLederId.
in short i need to check drledgerId and transactionId both.because I want to find a transaction from the database whose _id is transactionId and a ledgerId is drledgerId.and if that found then update balance with use Inc.
so how can i check both fields at a time?
Related
The application is syncing data records between devices via an online Mongo DB collection. Multiple devices can send batches of new or modified records to the server Mongo collection at any time. Devices get all record updates for them that they don't already have, by requesting records added or modified since their last get request.
Approach 1 - was to add a Date object field (called stored1) to the records before saving to MongoDb. When a device requests records , mongoDb paging is used to skip entries up to the current page, and then limit to 1000. Now that the data set is large, each page request is taking a long time, and mongo hit a memory error.
https://docs.mongodb.com/manual/reference/limits/#operations
Setting allowDiskUse(true) as shown in the posted code in my current configuration isn't fixing the memory error for some reason. If that can be fixed, it still wouldn't be a long term solution as the query times with the paging are already too long.
Approach 2:
What is the best way for pagination on mongodb using java
https://arpitbhayani.me/blogs/benchmark-and-compare-pagination-approach-in-mongodb
The 2nd approach considered is to change from Mongo paging skipping returned records, to just asking for stored time > largest stored time last received, until the number of records in a return is less than the limit. This requires the stored timestamp to be unique between all records matching the query, or it could miss records or get duplicate records etc.
In the example code, using the stored2 field, there's still a chance of duplicate timestamps, even if the probability is low.
Mongo has a BSON timestamp that guarantees unique values per collection, but I don't see a way to use it with document save(), or query on it in Spring Boot. It would need to be set on each record newly inserted, or replaced, or updated.
https://docs.mongodb.com/manual/reference/bson-types/#timestamps
Any suggestions on how to do this?
#Getter
#Setter
public abstract class DataModel {
private Map<String, Object> data;
#Id // maps this field name to the database _id field, automatically indexed
private String uid;
/** Time this entry is written to the db (new or modified), to support querying for changes since last query */
private Date stored1; //APPROCAH 1
private long stored2; //APPROACH 2
}
/** SpringBoot+MongoDb database interface implementation */
#Component
#Scope("prototype")
public class SpringDb implements DbInterface {
#Autowired
public MongoTemplate db; // the database
#Override
public boolean set(Collection<?> newRecords, Collection<?> updatedRecords) {
// get current time for this set
Date date = new Date();
int randomOffset = ThreadLocalRandom.current().nextInt(0, 500000);
long startingNanoSeconds = Instant.now().getEpochSecond() * 1000000000L + instant.getNano() + randomOffset;
int ns = 0;
if (updatedRecords != null && updatedRecords.size() > 0) {
for (Object entry : updatedRecords) {
entry.setStored1(date); //APPROACH 1
entry.setStored2(startingNs + ns++); //APPROCH 2
db.save(entry, repoName);
}
}
// for new documents only
if (newRecords != null && newRecords.size() > 0) {
for (DataModel entry : newRecords) {
entry.setStored1(date); //APPROACH 1
entry.setStored2(startingNs + ns++); // APPROACH 2
}
//multi record insert
db.insert(newRecords, repoName);
}
return true;
}
#Override
public List<DataModel> get(Map<String, String> params, int maxResults, int page, String sortParameter) {
// generate query
Query query = buildQuery(params);
//APPROACH 1
// do a paged query
Pageable pageable = PageRequest.of(page, maxResults, Direction.ASC, sortParameter);
List<T> queryResults = db.find(query.allowDiskUse(true).with(pageable), DataModel.class, repoName); //allowDiskUse(true) not working, still get memory error
// count total results
Page<T> pageQuery = PageableExecutionUtils.getPage(queryResults, pageable,
() -> db.count(Query.of(query).limit(-1).skip(-1), clazz, getRepoName(clazz)));
// return the query results
queryResults = pageQuery.getContent();
//APPROACH 2
List<T> queryResults = db.find(query.allowDiskUse(true), DataModel.class, repoName);
return queryResults;
}
#Override
public boolean update(Map<String, String> params, Map<String, Object> data) {
// generate query
Query query = buildQuery(params);
//This applies the same changes to every entry
Update update = new Update();
for (Map.Entry<String, Object> entry : data.entrySet()) {
update.set(entry.getKey(), entry.getValue());
}
db.updateMulti(query, update, DataModel.class, repoName);
return true;
}
private Query buildQuery(Map<String, String> params) {
//...
}
}
The solution I ended up using was to define, and index on, another field called storedId, which is a string concatenation of the modified record storedTime, and the _id. This guarantees all these storedId record fields are unique, because _id is unique.
Here's an example to show how indexing and querying on the concatenated storedTime+_id field works, while indexing and querying on the separate storedTime and _id fields fails:
public abstract class DataModel {
private Map<String, Object> data;
#Indexed
private String _id; // Unique id
#Indexed
private String storedTime; // Time this entry is written to the db (new or modified)
#Indexed
String storedId; // String concatenation of storedTime and _id field
}
//Querying on separate fields and indexes:
{
//storedTime, _id
"time1", "id2"
"time1", "id3"
"time1", "id4"
"time2", "id1"
"time2", "id5"
}
get (storedTime>"time0", _id>"id0", limit=2) // returns _id's 2,3 (next query needs to check for more at storedTime="time1" but skip _id’s <="id3")
get (storedTime>="time1", _id>"id3", limit=2) // returns _id's 4,5
//FAILS because this second query MISSES _id 1 (Note any existing _id record can be modified at any time, so the _id fields are not in storedTime order)
//Querying on the combined field and index:
{
//storedId
"time1-id2"
"time1-id3"
"time1-id4"
"time2-id1"
"time2-id5"
}
get (storedId>"time0", limit=2) // returns _id's 2,3 (next query for values greater than the greatest last value returned)
get (storedId>"time1-id3", limit=2) // returns _id's 4,1 (next query for values greater than the greatest last value returned)
get (storedId>"time2-id1", limit=2) //: returns _id 5
//WORKS, this doesn't miss or duplicate any records
I'm using entity framework to connect to database from my application. I have table in SQL, named Orders. It contains such fields as: TransactionId, ParticipantId and is linked to Transactions table which has one to many connection to Participants table. I need to get data from it using List of classes with such properties: TransactionId, ParticipantId, OrganizationId. Linq must meet such conditions: (orders.TransactionId == TransactionId && orders.ParticipantId == ParticipantId && orders.Transaction.Participants.Any(x=> x.Id == OrganizationId)). This should be done by one query, not by multiple, so, please don't recommend foreach or smth like that.
Like #NetMage said, generally we need examples. Assuming that you've got a dbcontext set up, the ask is pretty simple:
public static void GetData(int transactionId, int participantId, int organizationId)
{
using (var db = new MyDbContext())
{
var query =
(
from t in db.Transactions
from o in db.Orders
.Where(w => w.TransactionId == t.TransactionId)
from p in db.Participants
.Where(w => w.TransactionId == t.TransactionId)
where t.TransactionId = transactionId &&
o.ParticipantId = participantId
select new { Order = o, Transaction = t, Participant = p}
);
}
}
Again since we don't have a lot of information here it's hard to do more. You should be able to take it from there. I know I didn't use the organizationId filter, but since I don't know the target shape of the data I'm not sure what the best path would be
I would like to know what happens when I use IQueryable with and without AsQueryable(). Here is an example:
public partial class Book
{
.......
public Nullable<System.DateTime> CheckoutDate{get; set;}
}
I need to filter the data from SQL server before it is returned to an application server. I need to return books checked out more recently than entered date. Which one should I use?
A.
IQueryable<Book> books = db.Books;
books = books.Where(b => b.CheckoutDate >= date);
B.
IQueryable<Book> books = db.Books.ToList().AsQueryable();
books = books.Where(b => b.CheckoutDate >= date);
Basically I would like to know what is the difference between the above two options. Do they work on the similar grounds? Do they return same values?
With B option, you're basically retrieving every book from database and filtering data in memory.
A option is more performance, as it filters data at the database and return only the rows that match your query.
I have an entity query that when I run increments the value of my field. I am using entity framework and sql server 2012. Here is my query;
public void GetLastAccountNumber(ProductLine productLine, Action completed)
{
EntityQuery query = WASMDomainContext.GetContactCustomerAccountsQuery()
.Where(cca => cca.ProductLineId == productLine.Id)
.OrderBy(cca => cca.AccountNumber);
WASMDomainContext.Load(query, loadOp =>
{
Exception error = null;
ContactCustomerAccount lastAccount = null;
if (loadOp.HasError)
error = loadOp.Error;
else
lastAccount = loadOp.Entities.LastOrDefault();
// Invoke completion callback
completed(lastAccount, error);
}, null);
}
Query should return the last account number which is an integer field for now. However it returns an incremented value. For example in my table I have an accountnumber 0 the query returns an entity with account number as 1. My account number field is not auto increament and I find this very strange. And each time the above is called the AccountNumber field value increases by one but the database value will remain 0. I just want the query to return what is in my database. Any idea why this could be happening? Any help will be appreciated. Thank you all.
I have an ASP.NET MVC application that displays data in a table format. I want to give my users the ability to search the table, so I take a text string and pass it into my service layer to construct a query using Linq to Entities.
I want to search a number of columns using the string. Some of the columns are integers (order ids), but the user doesn't care about integers and strings. They want to type '1200' and get any order with '1200' in the order number, or '1200' in the address.
The problem is that I can't find a way to construct a Linq-to-Entities query that results in SQL that looks like this:
select orderid, address from orders where orderid like '%1200%' or address like '%1200%'
Database context:
public DbSet<Person> Persons { get; set; }
public DbSet<Worker> Workers { get; set; }
public DbSet<WorkerSignin> WorkerSignins { get; set; }
The Persons and Workers tables are in a 1 to 0..1 relationship. If a worker record exists, a person record must also exist. They share the same ID. A worker record doesn't have to exist, however.
The Workers and WorkerSignins tables are related, but it's not enforced because of a client requirement. The Worker has an id-card with a barcode number on it (dwccardnum), but there may be discrepancies between cards issued and records in the DB, so I record all cards scanned in WorkerSignins, regardless of whether there is a matching record in the Workers table.
Here is the code I am working with:
allWSI = signinRepo.GetAllQ()
.Where(jj => jj.dateforsignin == date)
.Select(a => a);
if (!string.IsNullOrEmpty(search))
{
allWSI = allWSI
.Join(workerRepo.GetAllQ(), s => s.dwccardnum, w => w.dwccardnum, (s, w) => new { s, w })
.DefaultIfEmpty()
.Join(personRepo.GetAllQ(), oj => oj.w.ID, p => p.ID, (oj, p) => new { oj, p }).DefaultIfEmpty()
.DefaultIfEmpty()
.Where(jj => Convert.ToString(jj.oj.w.dwccardnum).Contains(search) ||
jj.p.firstname1.Contains(search) ||
jj.p.firstname2.Contains(search) ||
jj.p.lastname1.Contains(search) ||
jj.p.lastname2.Contains(search))
.Select(a => a.oj.s);
}
The GetAllQ() methods return an IQueryable() object.
The problem is on this line:
.Where(jj => Convert.ToString(jj.oj.w.dwccardnum).Contains(search) ||
I get this error:
LINQ to Entities does not recognize the method 'System.String ToString(Int32)' method, and this method cannot be translated into a store expression."
If I take out the convert, and try this:
.Where(jj => jj.oj.w.dwccardnum.Contains(search) ||
I get this error:
'int' does not contain a definition for 'Contains' and the best extension method overload 'System.Linq.ParallelEnumerable.Contains(System.Linq.ParallelQuery, TSource)' has some invalid arguments
So the question is...
How do I construct a Where clause to generate a like '%string%' and execute it against a integer column using Linq to Entities? (e.g. without using LINQ to SQL)
One option is to replace ...
jj => Convert.ToString(jj.oj.w.dwccardnum).Contains(search)
... by:
jj => SqlFunctions.StringConvert((decimal)jj.oj.w.dwccardnum).Contains(search)
SqlFunctions is a static class in namespace System.Data.Objects.SqlClient and I believe it only works with SQL Server. The weird cast to decimal is necessary because StringConvert doesn't have an overload for an int and without the cast the compiler complains that it cannot select the right overload unambiguously. (It has one for decimal? and one for double?.) But I just tested that the code above works indeed (with SQL Server and assuming dwccardnum is an int).
Try this
if (!string.IsNullOrEmpty(search))
{
int cardnum;
bool searchIsInt = int.TryParse(search, out cardnum);
allWSI = allWSI
.Join(workerRepo.GetAllQ(), s => s.dwccardnum, w => w.dwccardnum, (s, w) => new { s, w })
.DefaultIfEmpty()
.Join(personRepo.GetAllQ(), oj => oj.w.ID, p => p.ID, (oj, p) => new { oj, p }).DefaultIfEmpty()
.DefaultIfEmpty()
.Where(jj => (searchIsInt ? jj.oj.w.dwccardnum == cardnum : true) ||
jj.p.firstname1.Contains(search) ||
jj.p.firstname2.Contains(search) ||
jj.p.lastname1.Contains(search) ||
jj.p.lastname2.Contains(search))
.Select(a => a.oj.s);
}
Basically, you're first checking to see if the search is an int and then use it in your linq if it is.