How to query/update sub document in MongoDB using C# driver - mongodb

public class DayData
{
public string _id
{get;set;}
public string Data
{get;set;}
public HourData HR1
{get;set;}
public HourData HR2
{get;set;}
...
public HourData HR24
{get;set;}
}
public class HourData
{
public long _id
{get;set;}
public int Count
{get;set;}
public string Data
{get;set;}
}
// Sample Data
{
"_id": "2012_11_10",
"Data": "some data",
"HR1":
{
"_id": 1
"Count": 100,
"Data": "Hour 1 Data"
},
"HR2":
{
"_id": 2
"Count": 200,
"Data": "Hour 2 Data"
},
...
"HR24":
{
"_id": 24
"Count": 2400,
"Data": "Hour 24 Data"
}
}
I have following questions (by using C# official driver):
How to retrieve single HourData document from DayData collection (using single database query)? e.g. I need to retrieve HourData document of HR1 for DayData (where _id="2012_11_10"). Please refer to code snippet i tried as Edit-1.
How to update/upsert HourData document to increment its Count (using single database operation, like: collection.update(Query, Update))? e.g. I need to increment Count of HR1 for DayData (where _id="2012_11_10").
How to retrieve Sum of all Count values of HR1, HR2,...,HR24 for DayData (where _id="2012_11_10") (using some aggregate function).
What is the best way to convert the HourData Counts of all hours to an array (for any DayData). e.g. for a DayData with _id="2012_11_10", i need:
int []Counts = [100,200,300, ... , 2400]
Edit-1
With this code I intend to get HourData of HR1 where its _id=1 and DayData with _id="2012_11_10", but it does not return anything.
MongoCollection<HourData> dayInfo = mdb.GetCollection<HourData>("HourData");
var query = Query.And(Query.EQ("_id", "2012_11_10"), Query.EQ("HR1._id", 1));
MongoCursor<HourData> hri = dayInfo.Find(query);'

1)
QueryComplete = Query.EQ(_id, "2012_11_10");
DayData myData = db.GetCollection<DayData>("DayDataCollection").FindOne(query);
// As HourData is the class member you can retrieve it from the instance of the DayData:
HourData myHour = myData.HR1;
2)
QueryComplete = Query.EQ(_id, "2012_11_10");
UpdateBuilder update = Update.Inc("HR1.Count", 1);
db.GetCollection<DayData>("DayDataCollection").Update(query, update, SafeMode.True)
;
3)
In your case you just retrieve the DayData instance and then sum all needed values explicitly:
QueryComplete = Query.EQ(_id, "2012_11_10");
DayData myData = db.GetCollection<DayData>("DayDataCollection").FindOne(query);
// As HourData is the class member you can retrieve it from the instance of the DayData:
int sum = myData.HR1.Count + myData.HR2.Count + ... + myData.HR24.Count;
But it's not elegant. If you want the elegant solution you need to transform your fields into the array like:
DayData
{
HR:
[{
Count: 1,
Data: "Hour 1 data"
},
{
},
...
]
}
and work as with the array. Let me know if it's possible to transform it into an array.
4)
In your case, again, there is no any elegant solution. What you can do just go through your fields and create an array:
int[] Counts = new int[24];
Counts[0] = myData.HR1.Count;
...
Or you can create enumerator right in the class but I think it's overkill in your case.

Related

How to make a Spring Data MongoDB GroupOperation on a single field but with an object as a result id?

Here is the group operation that I want to create depending on a nomenclature object.
private static GroupOperation createStatsGroupOperationFromNomenclature(Nomenclature nomenclature) {
Fields groupFields = Fields.fields("departmentCode");
nomenclature.getDepartmentCode().ifPresent(code -> groupFields.and("subDepartmentCode"));
nomenclature.getSubDepartmentCode().ifPresent(code -> groupFields.and("categoryCode"));
nomenclature.getCategoryCode().ifPresent(code -> groupFields.and("subCategoryCode"));
return group(groupFields)
.count().as("nbProducts")
.sum("$proposedMatchesAmount").as("nbProposedMatches")
.sum("$reviewedMatchesAmount").as("nbReviewedMatches");
}
With the previous function if I provide a departmentCode and a subDepartmentCode inside the nomenclature parameter, here is the mongo query that is executed :
{
_id: {
"departmentCode": "$departmentCode",
"subDepartmentCode": "$subDepartmentCode"
},
"nbProduct": {
$sum: 1
},
"proposedMatchesAmount": {
$sum: "$proposedMatchesAmount"
},
"reviewedMatchesAmount": {
$sum: "$reviewedMatchesAmount"
}
}
The result of this query are parsed in the following object :
#Builder
#Value
public class ProductsStatsDocument {
#Id
Nomenclature nomenclature;
Integer nbProducts;
Integer nbProposedMatches;
Integer nbReviewedMatches;
}
Problems append when I provide only a departmentCode inside the nomenclature parameter. Then the builded group operation has the following mongo query language equivalent:
{
_id: "$departmentCode",
"nbProduct": {
$sum: 1
},
"proposedMatchesAmount": {
$sum: "$proposedMatchesAmount"
},
"reviewedMatchesAmount": {
$sum: "$reviewedMatchesAmount"
}
}
And the result of this query couldn't be parsed to the previous ProductsStatsDocument because the result _id field id now a String and not a Nomenclature object.
Is it possible to force the group method to use an object as result _id field even with only one field ? Or is there an other way to build such a mongo group operation ?
=================================================================
Found the "why" of this issue. Here is a piece of code from spring data that transform the GroupOperation into a bson object :
} else if (this.idFields.exposesSingleNonSyntheticFieldOnly()) {
FieldReference reference = context.getReference((Field)this.idFields.iterator().next());
operationObject.put("_id", reference.toString());
} else {
And here is the exposesSingleNonSyntheticFieldOnly method :
boolean exposesSingleNonSyntheticFieldOnly() {
return this.originalFields.size() == 1;
}
As you can see, as soon as there is only one field to group on, it's used as _id result value.
So finally the solution that seems to works for now is to create a custom AggregationOperation that manage the document transformation _id part :
public class ProductsStatsGroupOperation implements AggregationOperation {
private static GroupOperation getBaseGroupOperation() {
return group()
.count().as("nbProducts")
.sum("$proposedMatchesAmount").as("nbProposedMatches")
.sum("$reviewedMatchesAmount").as("nbReviewedMatches");
}
private final Nomenclature nomenclature;
public ProductsStatsGroupOperation(Nomenclature nomenclature) {
this.nomenclature = nomenclature;
}
#Override
public Document toDocument(AggregationOperationContext context) {
Document groupOperation = getBaseGroupOperation().toDocument(context);
Document operationId = new Document();
for (Field field : getFieldsToGroupOn()) {
FieldReference reference = context.getReference(field);
operationId.put(field.getName(), reference.toString());
}
((Document)groupOperation.get("$group")).put("_id", operationId);
return groupOperation;
}
private Fields getFieldsToGroupOn() {
Fields groupFields = Fields.fields("departmentCode");
if (nomenclature.getDepartmentCode().isPresent()) {
groupFields = groupFields.and("subDepartmentCode");
}
if (nomenclature.getSubDepartmentCode().isPresent()) {
groupFields = groupFields.and("categoryCode");
}
if (nomenclature.getCategoryCode().isPresent()) {
groupFields = groupFields.and("subCategoryCode");
}
return groupFields;
}
}
There is a bad thing about this solution: the overrided method toDocument seems to be deprecated.

Get total count of days between two dates Linq C#

I want to get total number of days between two dates using Linq C# with Entity Framework 6. My issue is I need a where clause condition and finding it difficult to subtract the dates. I am getting errors when I run my code below. Any help to achieve this will be appreciated.
var finalexams = report.Select(x =>
new ReportVieModel
{
EnglishDates = x.admin.mastertablebles.Where(d => d.EndDate == null).Select(s => DbFunctions.DiffDays(s.beginDate, DateTime.Today)),
}).Distinct();
What type of data does EnglishDate represent?
The only condition I see is that the final date can be null, I suppose this will return more than one value and you will need a list.
I don't understand if you want a list with the start date and the number of days because you could do this according to the model:
Model:
public class ReportViewModel
{
public DateTime? Date { get; set; }
public int? NumberDays { get; set; }
}
LINQ:
List<ReportViewModel> list = (from p in x.admin.mastertablebles
where p.EndDate == null
select new ReportViewModel() { Date = p.beginDate, NumberDays = DbFunctions.DiffDays(p.beginDate, DateTime.Now) })
.Distinct().ToList();
Or you want a list that only contains the number of days:
LINQ:
var numberDays = (from p in x.admin.mastertablebles
where p.EndDate == null
select DbFunctions.DiffDays(p.beginDate, DateTime.Now))
.Distinct().ToList();

Spring Data with MongoDB - Find by nullable fields

I have a Mongo collection with documents like this:
a: { product: 1, country: 2, stock: 1}
b: { product: 1, country: 3, stock: 3}
c: { product: 2, country: 1, stock: 1}
Sometimes I want to get the stock of a product in all countries (so I retrieve the product stock in all countries and then I add them) and other times I want the stock in an specific country.
Is it possible to make a single method like:
findByProductAndCountry(Integer product, Integer country)
that works like this:
findByProductAndCountry(1, 2) //returns document a
findByProductAndCountry(1, null) //returns documents a and b
Thanks in advance!
Answering to your question: No. It is not possible to write such a query in mongodb so you can not achieve that with a single spring data mongodb method.
What I suggest is to write a default method in the repository interface for that. This way you can have it with the rest of your query methods:
public interface ProductRepository extends MongoRepository<Product, String> {
List<Product> findByProduct(int product);
List<Product> findByProductAndCountry(int product, int country);
default List<Product> findByProductAndNullableCountry(Integer product, Integer country) {
if (country != null) {
return findByProductAndCountry(product, country);
} else {
return findByProduct(product);
}
}
}

Cleanest way to implement multiple parameters filters in a REST API

I am currently implementing a RESTFUL API that provides endpoints to interface with a database .
I want to implement filtering in my API , but I need to provide an endpoint that can provide a way to apply filtering on a table using all the table's columns.
I've found some patterns such as :
GET /api/ressource?param1=value1,param2=value2...paramN=valueN
param1,param2...param N being my table columns and the values.
I've also found another pattern that consists of send a JSON object that represents the query .
To filter on a field, simply add that field and its value to the query :
GET /app/items
{
"items": [
{
"param1": "value1",
"param2": "value",
"param N": "value N"
}
]
}
I'm looking for the best practice to achieve this .
I'm using EF Core with ASP.NET Core for implementing this.
Firstly be cautious about filtering on everything/anything. Base the available filters on what users will need and expand from that depending on demand. Less code to write, less complexity, fewer indexes needed on the DB side, better performance.
That said, the approach I use for pages that have a significant number of filters is to use an enumeration server side where my criteria fields are passed back their enumeration value (number) to provide on the request. So a filter field would comprise of a name, default or applicable values, and an enumeration value to use when passing an entered or selected value back to the search. The requesting code creates a JSON object with the applied filters and Base64's it to send in the request:
I.e.
{
p1: "Jake",
p2: "8"
}
The query string looks like:
.../api/customer/search?filters=XHgde0023GRw....
On the server side I extract the Base64 then parse it as a Dictionary<string,string> to feed to the filter parsing. For example given that the criteria was for searching for a child using name and age:
// this is the search filter keys, these (int) values are passed to the search client for each filter field.
public enum FilterKeys
{
None = 0,
Name,
Age,
ParentName
}
public JsonResult Search(string filters)
{
string filterJson = Encoding.UTF8.GetString(Convert.FromBase64String(filters));
var filterData = JsonConvert.DeserializeObject<Dictionary<string, string>>(filterJson);
using (var context = new TestDbContext())
{
var query = context.Children.AsQueryable();
foreach (var filter in filterData)
query = filterChildren(query, filter.Key, filter.Value);
var results = query.ToList(); //example fetch.
// TODO: Get the results, package up view models, and return...
}
}
private IQueryable<Child> filterChildren(IQueryable<Child> query, string key, string value)
{
var filterKey = parseFilterKey(key);
if (filterKey == FilterKeys.None)
return query;
switch (filterKey)
{
case FilterKeys.Name:
query = query.Where(x => x.Name == value);
break;
case FilterKeys.Age:
DateTime birthDateStart = DateTime.Today.AddYears((int.Parse(value) + 1) * -1);
DateTime birthDateEnd = birthDateStart.AddYears(1);
query = query.Where(x => x.BirthDate <= birthDateEnd && x.BirthDate >= birthDateStart);
break;
}
return query;
}
private FilterKeys parseFilterKey(string key)
{
FilterKeys filterKey = FilterKeys.None;
Enum.TryParse(key.Substring(1), out filterKey);
return filterKey;
}
You can use strings and constants to avoid the enum parsing, however I find enums are readable and keep the sent payload a little more compact. The above is a simplified example and obviously needs error checking. The implementation code for complex filter conditions such as the age to birth date above would better be suited as a separate method, but it should give you some ideas. You can search for children by name, and/or age, and/or parent's name for example.
I have invented and found it useful to combine a few filters into one type for example CommonFilters and make this type parseable from string:
[TypeConverter(typeof(CommonFiltersTypeConverter))]
public class CommonFilters
{
public PageOptions PageOptions { get; set; }
public Range<decimal> Amount { get; set; }
//... other filters
[JsonIgnore]
public bool HasAny => Amount.HasValue || PageOptions!=null;
public static bool TryParse(string str, out CommonFilters result)
{
result = new CommonFilters();
if (string.IsNullOrEmpty(str))
return false;
var parts = str.Split(new[] { ' ', ';' }, StringSplitOptions.RemoveEmptyEntries);
foreach (var part in parts)
{
if (part.StartsWith("amount:") && Range<decimal>.TryParse(part.Substring(7), out Range<decimal> amount))
{
result.Amount = amount;
continue;
}
if (part.StartsWith("page-options:") && PageOptions.TryParse(part.Substring(13), out PageOptions pageOptions))
{
result.PageOptions = pageOptions;
continue;
}
//etc.
}
return result.HasAny;
}
public static implicit operator CommonFilters(string str)
{
if (TryParse(str, out CommonFilters res))
return res;
return null;
}
}
public class CommonFiltersTypeConverter : TypeConverter
{
public override bool CanConvertFrom(ITypeDescriptorContext context, Type sourceType)
{
if (sourceType == typeof(string))
{
return true;
}
return base.CanConvertFrom(context, sourceType);
}
public override object ConvertFrom(ITypeDescriptorContext context,
CultureInfo culture, object value)
{
if (value is string str)
{
if (CommonFilters.TryParse(str, out CommonFilters obj))
{
return obj;
}
}
return base.ConvertFrom(context, culture, value);
}
}
the request looks like this:
public class GetOrdersRequest
{
[DefaultValue("page-options:50;amount:0.001-1000;min-qty:10")]
public CommonFilters Filters { get; set; }
//...other stuff
}
In this way you reduce the number of input request parameters, especially when some queries don't care about all filters
If you use swagger map this type as string:
c.MapTypeAsString<CommonFilters>();
public static void MapTypeAsString<T>(this SwaggerGenOptions swaggerGenOptions)
{
swaggerGenOptions.MapType(typeof(T), () => new OpenApiSchema(){Type = "string"});
}

Equivalent for SetUnion in C# driver

I have a field in mongodb document which contains an array of numbers. I want to update this field with newly received numbers. But i need to add the numbers only if the number is not present in the array. In MongoDB we can use $setUnion but i am not sure on C# driver side. Anyone please suggest the solution.
$setUnion is Used to produce aggregation output.
You need to use AddToSetEach from c# driver
Please find full code snippet with check after insert
public static void Main()
{
var client = new MongoClient("mongodb://localhost:27017");
var database = client.GetDatabase("test");
var collection = database.GetCollection<KalaimaniData>("kalaimani");
// create array to inser
var arrayToInsert = new[] { 1, 4, 5, 6 };
var arrayToMerge = new[] { 2, 3, 4, 5 };
var arrayExpected = new[] { 1, 4, 5, 6, 2, 3 };
var doc = new KalaimaniData { Numbers = arrayToInsert };
collection.InsertOne(doc);
var filter = Builders<KalaimaniData>.Filter.Eq(x => x.Id, doc.Id);
var updateDef = new UpdateDefinitionBuilder<KalaimaniData>().AddToSetEach(x => x.Numbers, arrayToMerge);
collection.UpdateOne(filter, updateDef);
// retrive and compare
var changed = collection.Find(filter).First();
var matched = 0;
foreach (var element in arrayExpected)
{
if (changed.Numbers.Contains(element))
{
matched++;
}
}
if (changed.Numbers.Length == matched)
{
Console.WriteLine("OK");
}
else
{
Console.WriteLine("NOK");
}
Console.ReadLine();
}
/// <summary>TODO The kalaimani data.</summary>
class KalaimaniData
{
/// <summary>Gets or sets the id.</summary>
public ObjectId Id { get; set; }
/// <summary>Gets or sets the numbers.</summary>
public int[] Numbers { get; set; }
}