In a Mongo DB, I have the below entries:
> db.DomainObjects.find()
// [...]
{ "_id" : ObjectId("6..3"), "lev" : 7, "date" : ISODate("2021-02-01T00:00:00Z"), "val" : 29.1 }
{ "_id" : ObjectId("6..4"), "lev" : 7, "date" : ISODate("2021-02-02T00:00:00Z"), "val" : 20.3 }
{ "_id" : ObjectId("6..5"), "lev" : 7, "date" : ISODate("2021-02-05T00:00:00Z"), "val" : 15.8 }
// [...]
In a ASP.NET Core Service class, I have a reference to the collection using the Mongo DB driver with the below version installed:
Here is how the reference is created:
public DomainService(IDomainDatabaseSettings settings)
{
var client = new MongoClient(settings.ConnectionString);
var database = client.GetDatabase(settings.DatabaseName);
_repo = database.GetCollection<Domain>(settings.DomainCollectionName);
}
Then I'd like to get the entries by DateTime range (including the limits) using the following method:
public List<Domain> Get(int lev, DateTime dateFrom, DateTime dateTo) {
Expression<Func<Domain, bool>> expr = entry => (entry.lev == lev) &&
(entry.date.CompareTo(dateFrom) >= 0) &&
(entry.date.CompareTo(dateTo) <= 0);
return _repo.Find(expr).ToList();
I use the following GET request:
https://localhost:45645/api/DomainObjects/7/?dateFrom=2021-02-01&dateTo=2021-02-02
However, I only get the entries starting and including the lower limit, the entry on the upper limit is excluded.
For completeness, the controller is this:
[HttpGet("{lev:int}")]
public ActionResult<List<Domain>> Get(int lev, [FromQuery] DateTime? dateFrom = null, [FromQuery] DateTime? dateTo = null) {
if (!dateFrom.HasValue) {
dateFrom = DateTime.MinValue;
return _domainService.Get(lev, (DateTime) dateFrom, (DateTime) dateTo);
}
if (!dateTo.HasValue) {
dateTo = DateTime.Today;
return _domainService.Get(lev, (DateTime) dateFrom, (DateTime) dateTo);
}
return _domainService.Get(lev, (DateTime) dateFrom, (DateTime) dateTo);
}
If the lower limit dateFrom is omitted in the request, it should start with the earliest entry, if the limit dateTo is omitted, it should return all values from dateFrom up until and including today.
Related
Here is the group operation that I want to create depending on a nomenclature object.
private static GroupOperation createStatsGroupOperationFromNomenclature(Nomenclature nomenclature) {
Fields groupFields = Fields.fields("departmentCode");
nomenclature.getDepartmentCode().ifPresent(code -> groupFields.and("subDepartmentCode"));
nomenclature.getSubDepartmentCode().ifPresent(code -> groupFields.and("categoryCode"));
nomenclature.getCategoryCode().ifPresent(code -> groupFields.and("subCategoryCode"));
return group(groupFields)
.count().as("nbProducts")
.sum("$proposedMatchesAmount").as("nbProposedMatches")
.sum("$reviewedMatchesAmount").as("nbReviewedMatches");
}
With the previous function if I provide a departmentCode and a subDepartmentCode inside the nomenclature parameter, here is the mongo query that is executed :
{
_id: {
"departmentCode": "$departmentCode",
"subDepartmentCode": "$subDepartmentCode"
},
"nbProduct": {
$sum: 1
},
"proposedMatchesAmount": {
$sum: "$proposedMatchesAmount"
},
"reviewedMatchesAmount": {
$sum: "$reviewedMatchesAmount"
}
}
The result of this query are parsed in the following object :
#Builder
#Value
public class ProductsStatsDocument {
#Id
Nomenclature nomenclature;
Integer nbProducts;
Integer nbProposedMatches;
Integer nbReviewedMatches;
}
Problems append when I provide only a departmentCode inside the nomenclature parameter. Then the builded group operation has the following mongo query language equivalent:
{
_id: "$departmentCode",
"nbProduct": {
$sum: 1
},
"proposedMatchesAmount": {
$sum: "$proposedMatchesAmount"
},
"reviewedMatchesAmount": {
$sum: "$reviewedMatchesAmount"
}
}
And the result of this query couldn't be parsed to the previous ProductsStatsDocument because the result _id field id now a String and not a Nomenclature object.
Is it possible to force the group method to use an object as result _id field even with only one field ? Or is there an other way to build such a mongo group operation ?
=================================================================
Found the "why" of this issue. Here is a piece of code from spring data that transform the GroupOperation into a bson object :
} else if (this.idFields.exposesSingleNonSyntheticFieldOnly()) {
FieldReference reference = context.getReference((Field)this.idFields.iterator().next());
operationObject.put("_id", reference.toString());
} else {
And here is the exposesSingleNonSyntheticFieldOnly method :
boolean exposesSingleNonSyntheticFieldOnly() {
return this.originalFields.size() == 1;
}
As you can see, as soon as there is only one field to group on, it's used as _id result value.
So finally the solution that seems to works for now is to create a custom AggregationOperation that manage the document transformation _id part :
public class ProductsStatsGroupOperation implements AggregationOperation {
private static GroupOperation getBaseGroupOperation() {
return group()
.count().as("nbProducts")
.sum("$proposedMatchesAmount").as("nbProposedMatches")
.sum("$reviewedMatchesAmount").as("nbReviewedMatches");
}
private final Nomenclature nomenclature;
public ProductsStatsGroupOperation(Nomenclature nomenclature) {
this.nomenclature = nomenclature;
}
#Override
public Document toDocument(AggregationOperationContext context) {
Document groupOperation = getBaseGroupOperation().toDocument(context);
Document operationId = new Document();
for (Field field : getFieldsToGroupOn()) {
FieldReference reference = context.getReference(field);
operationId.put(field.getName(), reference.toString());
}
((Document)groupOperation.get("$group")).put("_id", operationId);
return groupOperation;
}
private Fields getFieldsToGroupOn() {
Fields groupFields = Fields.fields("departmentCode");
if (nomenclature.getDepartmentCode().isPresent()) {
groupFields = groupFields.and("subDepartmentCode");
}
if (nomenclature.getSubDepartmentCode().isPresent()) {
groupFields = groupFields.and("categoryCode");
}
if (nomenclature.getCategoryCode().isPresent()) {
groupFields = groupFields.and("subCategoryCode");
}
return groupFields;
}
}
There is a bad thing about this solution: the overrided method toDocument seems to be deprecated.
SQL query=>
where and ((subtype="dailyMessage" and registDate="today") or(subtype !="dailyMessage"))
i want to retrive this query by using mongodb.
i have this collection. (today is '2017-08-16T15:48:19.947Z')
{
"_id" : ObjectId("597768443b1fd6308c0350c0"),
"type" : "message",
"subtype" : "dailyMessage",
"message" : test1",
"registDate" : ISODate("2017-08-16T15:48:19.947Z")
},
{
"_id" : ObjectId("597768443b1fd6308c0350c1"),
"type" : "message",
"subtype" : "dailyPush",
"message" : test2",
"registDate" : ISODate("2017-07-25T15:48:19.947Z")
},
{
"_id" : ObjectId("597768443b1fd6308c0350c2"),
"type" : "message",
"subtype" : "dailyPush",
"message" : test3",
"registDate" : ISODate("2017-07-24T15:48:19.947Z")
}
here is my code(spring boot mongodb , using java 8)
List<String> listOfmessage = new ArrayList<String>();
listOfSubtype.add("dailyMessage");
criteria = Criteria.where("test").is("james");
criteria.andOperator(Criteria.where("registDate").gte(LocalDateTime.of(LocalDate.now(), LocalTime.MIDNIGHT)).lte(LocalDateTime.now()) , Criteria.where("subtype").in(listOfmessage) .orOperator(Criteria.where("subtype").nin(listOfmessage)));
my problem is that how can i insert orOperator on the andOperator?
As mentioned in the comment, the and outside the whole condition doesn't make sense. The below code is written for the following condition.
(condition1 AND condition2) OR condition3
Code:-
You may need to alter the below code to get the MongoOperations object based on your Spring configuration.
"register" is the collection name. You can change it accordingly as per your collection name.
public Boolean getRegisterData() {
MongoOperations mongoOperations = getMongoConnection();
List<String> listOfSubType = new ArrayList<String>();
listOfSubType.add("dailyMessage");
List<String> listOfMessage = new ArrayList<String>();
listOfMessage.add("test1");
Criteria criteriaSubTypeAndDate = new Criteria();
criteriaSubTypeAndDate.andOperator(Criteria.where("registDate")
.gte(LocalDateTime.of(LocalDate.now(), LocalTime.MIDNIGHT)).lte(LocalDateTime.now()),
Criteria.where("subtype").in(listOfSubType));
Criteria criteriaSubType = Criteria.where("subtype").ne("dailyMessage");
Criteria criteriaFull = new Criteria();
criteriaFull.orOperator(criteriaSubTypeAndDate, criteriaSubType);
Query query = new Query();
query.addCriteria(criteriaFull);
System.out.println(query);
mongoOperations.executeQuery(query, "register", new RegistryDocumentCallbackHandler());
return true;
}
Result set document processing class:-
processDocument() method will get executed for each document in the result set.
public class RegistryDocumentCallbackHandler implements DocumentCallbackHandler {
#Override
public void processDocument(DBObject dbObject) throws MongoException, DataAccessException {
System.out.println("Registry collections data ===>" + dbObject.toString());
}
}
I am trying to Update the Mongo DB COllection with Yesterday Date in this format (YYYY-MM-DD)
I am trying this way
var currentDate = new Date();
currentDate.setDate(currentDate.getDate()-1);
var month = (currentDate.getMonth() < 9 ? "0"+ (currentDate.getMonth()+1) : (currentDate.getMonth()+1));
var day = (currentDate.getDate() < 10 ? "0" + currentDate.getDate() : currentDate.getDate());
var date = currentDate.getTime();
var patt = date;
db.LoginCount.update({yesterday : {$exists : false}}, {$set: {yesterday : new Date(date)}})
The Output i am geting is
db.LoginCount.find().pretty()
{
"yesterday" : ISODate("2013-05-01T03:25:28.640Z")
}
PLease tell me how to update COllection so that it inserts Date in this format YYYY-MM-DD
Anyway i solved this way
m = function() { emit(this.cust_id, 1); }
r = function (k, vals) { var sum = 0; for (var i in vals) { sum += vals[i]; } return sum; }
q = function() {
var currentDate = new Date();
currentDate.setDate(currentDate.getDate()-1);
var month = (currentDate.getMonth() < 9 ? "0"+ (currentDate.getMonth()+1) : (currentDate.getMonth()+1));
var day = (currentDate.getDate() < 10 ? "0" + currentDate.getDate() : currentDate.getDate());
var date = currentDate.getTime();
var patt = date;
var query = {"created_at":"2013-04-30 11:19:52.587"};
return query;
}
res = db.logins.mapReduce(m, r, { query : q(), out : "LoginCount" });
var yesterday = new Date(Date.now() - 86400000);
db.LoginCount.update({yesterday : {$exists : false}}, {$set: {yesterday : yesterdayday}})
Your $set is using a Date typed value for the yesterday field value. That's why your find query returns an ISODate. See http://docs.mongodb.org/manual/core/document/.
If you really need yesterday to be a string, you will need use:
{$set: {yesterday:currentDate.toISOString().substring(0,10)}}
If you want to store the date as a Date typed field with a normalized time component (e.g. 00:00:00) use something like this:
var d = new Date(Date)
{$set {yesterday:new Date(Date.UTC(currentDate.getUTCFullYear(), currentDate.getUTCMonth(), currentDate.getUTCDate(), 0,0,0))}}
I ran into a situation with StreamInsight where I have 1 input source with different types of events that need to be treated differently, but are eventually matched with other events from the same source.
I created a (MUCH) simpler scenario below where an input source generates random numbers (for simplicity sake, 1s and 0s). If the number is even, we want to save it until further notice (unknown duration). If the number is odd, we want to match it with n-1 from the even stream and then remove n-1 from the even stream. If there is no match, the odd number is simply processed through as is with no further calculation. I have everything working as expected up to the point of removing a matching n-1 from the even stream. A match is made, and the match gets pushed to the output adapter, but remains available for another join to be made to the given event. What I have gathered of several days worth of experimentation and research, is somehow I need to clip the even stream event duration (ClipEventDuration), presumably as part of the filter in GenerateEvenJoinQuery, however, everything I have
tried has produced no change, or undesired results. I have also tried changing the evenStream to an Interval shape with even less luck. Any help or suggestions would be greatly appreciated.
For example, given a simplified list:
[ 1, 0, 1, 1, 0, 0, 1, 1, 1 ]
I would expect the output to look like:
[ 1, 100, 1, 100, 100, 1 ]
I would also be accepting of as the real scenario I'm working with, the first output isn't actually possible:
Note the 2nd and 3rd 0s are joined to a single 1.
[ 1, 100, 1, 100, 1, 1 ]
...
CepStream<int> inputStream = CepStream<int>.Create(
app
, "ATGInputStream"
, typeof(InputAdapterFactory)
, new InputConfig()
, EventShape.Point);
var everythingFilter = from e in inputStream select e;
Query everythingQuery = GenerateEverythingQuery(app, inputStream);
var everythingStream = everythingQuery.ToStream<int>("everythingStream");
Query oddQuery = GenerateOddQuery(app, everythingStream);
var oddAts = new AdvanceTimeSettings(new AdvanceTimeGenerationSettings(1, TimeSpan.FromTicks(-1), false), null, AdvanceTimePolicy.Drop);
var oddStream = oddQuery.ToStream<int>("oddStream", oddAts);
// only inject a cti in to even when we need it
var ats = new AdvanceTimeSettings(null, new AdvanceTimeImportSettings("oddStream"), AdvanceTimePolicy.Adjust);
Query evenQuery = GenerateEvenQuery(app, everythingStream);
var evenStream = evenQuery.ToStream<int>("evenStream", ats);
Query joinQuery = GenerateOddEvenJoinQuery(app, evenStream, oddStream);
var joinStream = joinQuery.ToStream<int>("joinStream");
...
private Query GenerateOddEvenJoinQuery(Application app, CepStream<int> evenStream, CepStream<int> oddStream) {
// (o * e) + 100 is an easy way to tell we had a match
var filter = (from o in oddStream
from e in evenStream
where e == (o - 1)
select (o * e) + 100);
// LEFT ANTI SEMI JOIN
var filter2 = from o in oddStream
where (from e in evenStream where e == o - 1 select e).IsEmpty()
select o;
var joinFilter = filter.Union(filter2);
return joinFilter.ToQuery(
app
, "Number Join Query"
, "Joins number streams."
, EventShape.Point
, StreamEventOrder.FullyOrdered);
}
private Query GenerateEvenQuery(Application app, CepStream<int> stream) {
var evenFilter = (from e in stream where e % 2 == 0 select e).AlterEventDuration(e => TimeSpan.MaxValue);
return evenFilter.ToQuery(
app
, "EvenQuery"
, ""
, EventShape.Edge
, StreamEventOrder.FullyOrdered);
}
private Query GenerateOddQuery(Application app, CepStream<int> stream) {
var filter = (from f in stream where (f % 2) == 1 select f);
return filter.ToQuery(
app
, "OddQuery"
, "Queries for odd numbers in stream."
, EventShape.Point
, StreamEventOrder.FullyOrdered);
}
private Query GenerateEverythingQuery(Application app, CepStream<int> stream) {
var everythingFilter = from e in stream select e;
return everythingFilter.ToQuery(
app
, "EverythingQuery"
, "Queries everything from the input stream."
, EventShape.Point
, StreamEventOrder.FullyOrdered);
}
SOLUTION:
While I was hoping for something a little more elaborate and potentially faster, the delayed processing may help with performance.
public Program() {
public Program() {
...
var stream = CepStream<RandomNumber>.Create(
app
, "StaticInputStream"
, typeof(StaticInputAdapterFactory)
, new InputConfig()
, EventShape.Point);
var processedStream = stream.Scan(new StreamMatcher());
Query consoleQuery = GenerateConsoleOutputQuery(app, processedStream);
...
}
private Query GenerateConsoleOutputQuery(Application app, CepStream<int> stream) {
var filter = from f in stream select f;
return filter.ToQuery(
app
, "Console Output Query"
, "Queries for messages to output to the console."
, typeof(OutputAdapterFactory)
, new OutputConfig()
, EventShape.Point
, StreamEventOrder.FullyOrdered);
}
public class StreamMatcher : CepPointStreamOperator<RandomNumber, int> {
private List<int> unmatched = new List<int>();
public override bool IsEmpty {
get { return false; }
}
public override IEnumerable<int> ProcessEvent(PointEvent<RandomNumber> inputEvent) {
if(inputEvent.Payload.value % 2 == 0) {
unmatched.Add(inputEvent.Payload.value);
} else {
var result = inputEvent.Payload.value;
int match = -1;
try {
match = (from f in unmatched where f == result - 1 select f).Take(1).Single();
unmatched.Remove(match);
} catch { }
if(match > -1) {
result += match + 100;
}
yield return result;
}
}
}
}
public class RandomNumber {
public int value { get; set; }
public DateTime timeStamp { get; set; }
}
You might consider using a UDSO ( User defined stream operator), where you can keep the state of your zeros.
void Main()
{
var randomNumbers = new []
{
new RandomNumber(){ Value = 1, TimeStamp = DateTime.Parse("2012-01-01 10:01:00 AM") },
new RandomNumber(){ Value = 0, TimeStamp = DateTime.Parse("2012-01-01 10:02:00 AM") },
new RandomNumber(){ Value = 0, TimeStamp = DateTime.Parse("2012-01-01 10:02:00 AM") },
new RandomNumber(){ Value = 1, TimeStamp = DateTime.Parse("2012-01-01 10:03:00 AM") },
new RandomNumber(){ Value = 1, TimeStamp = DateTime.Parse("2012-01-01 10:04:00 AM") },
new RandomNumber(){ Value = 0, TimeStamp = DateTime.Parse("2012-01-01 10:05:00 AM") },
};
var stream = randomNumbers.ToPointStream(Application,
e=> PointEvent.CreateInsert(e.TimeStamp ,e),
AdvanceTimeSettings.IncreasingStartTime) ;
var query = stream.Scan(new MyCalculation());
query.Dump();
}
public class MyCalculation : CepPointStreamOperator<RandomNumber,string>
{
private Queue<int> _queue = new Queue<int>() ;
public override bool IsEmpty
{
get { return false; }
}
public override IEnumerable<string> ProcessEvent(PointEvent<RandomNumber> inputEvent)
{
if (inputEvent.Payload.Value % 2 == 0)
{
_queue.Enqueue(inputEvent.Payload.Value);
}
else
{
var result= inputEvent.Payload.Value.ToString() ;
var list = _queue.ToArray();
for (int i = 0; i < list.Count(); i++)
{
result += list[i];
}
yield return result ;
}
}
}
public class RandomNumber
{
public int Value {get;set;}
public DateTime TimeStamp {get;set;}
}
I have this structure:
public class User
{
public ObjectId Id { get; set; }
public Location Location { get; set; }
public DateTime LastAround {get;set;}
}
public class Location
{
public double Latitude { get; set; }
public double Longitude { get; set; }
}
I've tried a few things but I want to update User's Location and when they were last around.
Tried this:
userHelper.Collection.Update(
Query.EQ("_id", userId),
Update.SetWrapped<Location>("Location", new Location { Latitude = latitude, Longitude = longitude }).Set("LastAround", DateTime.UtcNow));
and this:
userHelper.Collection.Update(
Query.EQ("_id", userId),
Update.Set("Location.Latitude", latitude)
.Set("Location.Longitude", longitude)
.Set("LastAround", DateTime.UtcNow));
Nothing worked...how can I do this?
Update 4/17:
userHelper.Collection.Update(
Query.EQ("_id", new ObjectId(userId)),
Update
.SetWrapped<Location>("Location", new Location { Longitude = longitude, Latitude = latitude })
.Set("LastAround", DateTime.UtcNow)
);
The lng and lat value orders seem to be very important when doing queries on them. I was doing a geonear query and getting an strange out of bounds error. If you update in the wrong order it will put lats first and then you get the error.
Both of your original Update statements should work. I wrote a small sample program to demonstrate.
After executing this Insert statement:
var userId = ObjectId.GenerateNewId();
var user = new User
{
Id = userId,
Location = new Location { Latitude = 1.0, Longitude = 2.0 },
LastAround = new DateTime(2012, 4, 14, 0, 0, 0, DateTimeKind.Utc)
};
collection.Insert(user);
The document looks like this in the mongo shell:
> db.test.find()
{ "_id" : ObjectId("4f8c5d33e447ad34b8c7ac84"), "Location" : { "Latitude" : 1, "Longitude" : 2 }, "LastAround" : ISODate("2012-04-14T00:00:00Z") }
>
After executing the first form of the Update statement:
collection.Update(
Query.EQ("_id", userId),
Update
.SetWrapped<Location>("Location", new Location { Latitude = 3.0, Longitude = 4.0 })
.Set("LastAround", new DateTime(2012, 4, 15, 0, 0, 0, DateTimeKind.Utc)));
the document looks like this:
> db.test.find()
{ "_id" : ObjectId("4f8c5d33e447ad34b8c7ac84"), "Location" : { "Latitude" : 3, "Longitude" : 4 }, "LastAround" : ISODate("2012-04-15T00:00:00Z") }
>
And after executing the second form of the Update statement:
collection.Update(
Query.EQ("_id", userId),
Update
.Set("Location.Latitude", 5.0)
.Set("Location.Longitude", 6.0)
.Set("LastAround", new DateTime(2012, 4, 16, 0, 0, 0, DateTimeKind.Utc)));
the document looks like this:
> db.test.find()
{ "_id" : ObjectId("4f8c5d33e447ad34b8c7ac84"), "Location" : { "Latitude" : 5, "Longitude" : 6 }, "LastAround" : ISODate("2012-04-16T00:00:00Z") }
>
So the two forms of the Update statement are working.
The full program is here:
http://www.pastie.org/3799469
Both of your alternatives look ok.
Are you sure that your userId variable had the correct value? It could be that the Update is not finding a matching document to update.
What I ended up doing:
var userHelper = new MongoHelper<User>();
ObjectId id = new ObjectId(userId);
var user = userHelper.Collection.FindAll().Where(u => u.Id == id).Single();
user.LastAround = DateTime.UtcNow;
user.Location = new Location { Longitude = longitude, Latitude = latitude };
userHelper.Collection.Save(user);
which works. Not sure why the other way wouldn't. I suppose this is more "SQL" like although might not have the best performance. :(