IMongoDatabase.ListCollections return a cursor over BsonDocument.
Why doesn't it return a cursor over IMongoCollection<T> instead?
I was trying to write a generic GetCollection method to retrieve the collection given just the document type, something like this:
private IMongoCollection<T> GetCollection<T>()
{
var client = new MongoClient("connectionString");
var db = client.GetDatabase("dbName");
var coll = db.ListCollectionsAsync().Result.ToListAsync().Result
// Find collection of document of type T
// Collection is a BsonDocument instead
.Find(collection => typeof(collection) == typeof(T));
return coll;
}
The driver doesn't know what kind of document is in a collection, which is why it takes a type parameter T. MongoDB itself isn't aware of how the documents in the database map to the types in your application.
It is not possible to take a connection to a "generic" MongoDB deployment and simply discover the collections and types in them. This is code you would need to write, and probably won't work out well as it'll be something akin to trial and error.
If you are simply trying to create a factory type, you will need to build the backing list of collections before calling GetCollection<T>.
You could try using the type name as the collection name. This would make the collection name repeatable (unless the type name is changed). But I've never tested it and it might have some idiosyncrasies in the real world.
public class MyDatabase
{
private readonly IMongoClient _client;
public MyDatabase(string connectionString)
{
_client = new MongoClient(connectionString);
}
public IMongoCollection<T> GetCollection<T>()
{
var db = _client.GetDatabase("dbName");
return db.GetCollection<T>(typeof(T).Name);
}
}
If you prefer collection names to be pluralized, something like Humanizer can help there.
I generally prefer to create a type that has the collections as fields on the class. For example:
public class MyDatabase
{
private readonly IMongoClient _client;
public IMongoCollection<Foo> Foos { get; }
public IMongoCollection<Bar> Bars { get; }
public MyDatabase(string connectionString)
{
_client = new MongoClient(connectionString);
var db = _client.GetDatabase("myDb");
Foos = db.GetCollection<Foo>("foos");
Bars = db.GetCollection<Bar>("bars");
}
}
MongoDB collections have a flexible scheme that allows you to insert any document with any structure into the collection. For example, We can insert the following 3 objects into the test collection and it's valid:
> db.test.insertMany([{one: 1}, {two:2}, {three: 3}])
{
"acknowledged" : true,
"insertedIds" : [
ObjectId("5c87c954ed372bf469367e57"),
ObjectId("5c87c954ed372bf469367e58"),
ObjectId("5c87c954ed372bf469367e59")
]
}
> db.test.find().pretty()
{ "_id" : ObjectId("5c87c954ed372bf469367e57"), "one" : 1 }
{ "_id" : ObjectId("5c87c954ed372bf469367e58"), "two" : 2 }
{ "_id" : ObjectId("5c87c954ed372bf469367e59"), "three" : 3 }
Thus you can't map a MongoDB collection to a .NET Type as the collection has no knowledge of the type.
Related
I seem to be unable to store a simple object to cosmos db?
this is the database model.
public class HbModel
{
public Guid id { get; set; }
public string FormName { get; set; }
public Dictionary<string, object> Form { get; set; }
}
and this is how I store the data into the database
private static void SeedData(HbModelContext dbContext)
{
var cosmosClient = dbContext.Database.GetCosmosClient();
cosmosClient.ClientOptions.AllowBulkExecution = true;
if (dbContext.Set<HbModel>().FirstOrDefault() == null)
{
// No items could be picked hence try seeding.
var container = cosmosClient.GetContainer("hb", "hb_forms");
HbModel first = new HbModel()
{
Id = Guid.NewGuid(),//Guid.Parse(x["guid"] as string),
FormName = "asda",//x["name"] as string,
Form = new Dictionary<string, object>() //
}
string partitionKey = await GetPartitionKey(container.Database, container.Id);
var response = await container.CreateItemAsync(first, new PartitionKey(partitionKey));
}
else
{
Console.WriteLine("Already have data");
}
}
private static async Task<string> GetPartitionKey(Database database, string containerName)
{
var query = new QueryDefinition("select * from c where c.id = #id")
.WithParameter("#id", containerName);
using var iterator = database.GetContainerQueryIterator<ContainerProperties>(query);
while (iterator.HasMoreResults)
{
foreach (var container in await iterator.ReadNextAsync())
{
return container.PartitionKeyPath;
}
}
return null;
}
but when creating the item I get this error message
A host error has occurred during startup operation '3b06df1f-000c-4223-a374-ca1dc48d59d1'.
[2022-07-11T15:02:12.071Z] Microsoft.Azure.Cosmos.Client: Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 24bac0ba-f1f7-411f-bc57-3f91110c4528; Reason: ();.
Value cannot be null. (Parameter 'provider')
no idea why it fails?
the data should not be formatted incorreclty?
It also fails in case there is data in the dictionary.
What is going wrong?
There are several things wrong with the attached code.
You are enabling Bulk but you are not following the Bulk pattern
cosmosClient.ClientOptions.AllowBulkExecution = true is being set, but you are not parallelizing work. If you are going to use Bulk, make sure you are following the documentation and creating lists of concurrent Tasks. Reference: https://learn.microsoft.com/azure/cosmos-db/sql/tutorial-sql-api-dotnet-bulk-import#step-6-populate-a-list-of-concurrent-tasks. Otherwise don't use Bulk.
You are blocking threads.
The call to container.CreateItemAsync(first, new PartitionKey("/__partitionKey")).Result; is a blocking call, this can lead you to deadlocks. When using async operations (such as CreateItemAsync) please use the async/await pattern. Reference: https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/blob/master/AsyncGuidance.md#avoid-using-taskresult-and-taskwait
The PartitionKey parameter should be the value not the definition.
On the call container.CreateItemAsync(first, new PartitionKey("/__partitionKey")) the Partition Key (second parameter) should be the value. Assuming your container has a Partition Key Definition of /__partitionKey then your documents should have a __partitionKey property and you should pass the Value in this parameter of such property in the current document. Reference: https://learn.microsoft.com/azure/cosmos-db/sql/troubleshoot-bad-request#wrong-partition-key-value
Optionally, if your documents do not contain such a value, just remove the parameter from the call:
container.CreateItemAsync(first)
Be advised though that this solution will not scale, you need to design your database with Partitioning in mind: https://learn.microsoft.com/azure/cosmos-db/partitioning-overview#choose-partitionkey
Missing id
The model has Id but Cosmos DB requires id, make sure the content of the document contains id when serialized.
I'm trying to build a dynamic filter, where the user can search for columns in tables. To that I'm trying to use System.Linq.Dynamic.Core
var data = Db.MyTable1.Select($"new ({string.Join(", ", queryParams.Columns)})", "T", StringComparison.OrdinalIgnoreCase);
But now I also would like to get the table from the parameter string, something like
var query = Db.Get(queryParams.Table);
query = query.Select...
Working with EntityFrameworkCore 3.1, is there a way to achieve this?
You can override the Query method of DbContext.
First,create a custom Query method:
public static partial class CustomExtensions
{
public static IQueryable Query(this DbContext context, string entityName) =>
context.Query(context.Model.FindEntityType(entityName).ClrType);
static readonly MethodInfo SetMethod = typeof(DbContext).GetMethod(nameof(DbContext.Set));
public static IQueryable Query(this DbContext context, Type entityType) =>
(IQueryable)SetMethod.MakeGenericMethod(entityType).Invoke(context, null);
}
Then, you need to store the table names and types of all the tables involved in the Dictionary.
Dictionary<string, Type> TableTypeDictionary = new Dictionary<string, Type>()
{
{ "Teachers", typeof(Teachers) },//store the tables name and type.
{ "Students", typeof(Students) },
{ "Product", typeof(Product) }
//...
};
Last, use db to call Query method :
var query = Db.Query(TableTypeDictionary[queryParams.Table]).Select($"new ({string.Join(",", queryParams.Columns)})", "T", StringComparison.OrdinalIgnoreCase);
You can also bring "Namespace.MyTable" in the Query method. At this time, you need to rewrite the Query method to another writing method, please refer to this.
I have a stored procedure that returns me a list of IDs. (I then use this list of IDs as keys for objects.)
I am migrating this from .NET to .NET Core. In normal .NET I could use an extension library to get the numbers out like this:
var getOrderDetailIdsStoredProc = new GetOrderDetailIdsStoredProc()
{
NumberOfOrderDetailIdsNeeded = numberOfOrderDetailIdsNeeded
};
var orderDetailIds = contextProvider.Context.Database
.ExecuteStoredProcedure<long>(getOrderDetailIdsStoredProc);
But that library (EntityFrameworkExtras) is not working with EF Core (I found a version for EF Core, but it doesn't work.)
So I have been looking for other solutions:
DbContext.Database.ExecuteSqlCommand: Cannot return records, only output variables
DbSet.FromSQL: Can only be run on a DbSet<T> (basically it needs an entity type)
Right now, all I can think of is to make an entity called Number:
public class Number
{
public long Value;
}
public DbSet<Number> Numbers;
And then do something like this:
Numbers.FromSql("exec GenerateOrderDetailSequencedIds #numberNeeded", numberNeeded)
Aside from the fact that this is very ugly (making an entity out of a native type), I have no table to hook it up to, so I worry it will not work.
Is there any way in EF Core to run a stored procedure and get back a list of numbers?
NOTE: This worked, but was not compatable with BreezeJs (it could not deal with a DbQuery). See my other answer for what I ended up doing.
OrderDetailIdHolder.cs
public class OrderDetailIdHolder
{
public long NewId { get; set; }
}
MyEntitiesContext
public DbQuery<OrderDetailIdHolder> OrderDetailIdHolders { get; set; }
internal List<long> GetOrderDetailIds(int numberOfIdsNeeded)
{
var result = OrderDetailIdHolders.FromSql($"exec Sales.GenerateOrderDetailIds {numberOfIdsNeeded}").ToList();
return result.Select(x=>x.NewId).ToList();
}
This a bit extra complexity for just a list of longs. But it works.
It is important to note that the property (NewId in this case) must match what is returned from the sproc. Also, the type is not a DbSet. It is a DbQuery.
It is also important to note that this is only for EF Core 2.2. EF Core 3 has a different way to do this (Keyless Entity Types)
This is what I ended up using:
public static List<T> SqlQueryList<T>(this DatabaseFacade database, string query, params SqlParameter[] sqlParameters)
{
// TODO: Add a using statement here so we don't leak the connection's resources.
var conn = database.GetDbConnection();
conn.Open();
var command = conn.CreateCommand();
command.CommandText = query;
command.Parameters.AddRange(sqlParameters);
var reader = command.ExecuteReader();
List<T> result = new List<T>();
while (reader.Read())
{
T typedRow;
var row = reader.GetValue(0);
if (typeof(T).IsValueType)
{
typedRow = (T) row;
}
else
{
typedRow = (T)Convert.ChangeType(result, typeof(T));
}
result.Add(typedRow);
}
return result;
}
Called like this:
var numberOfOrderDetailIdsNeededParam = new SqlParameter
{
ParameterName = "#numberOfOrderDetailIdsNeeded",
SqlDbType = SqlDbType.Int,
Direction = ParameterDirection.Input
};
numberOfOrderDetailIdsNeededParam.Value = numberOfOrderDetailIdsNeeded;
var result = contextProvider.Context.Database.SqlQueryList<long>($"exec Sales.GenerateOrderDetailIds #numberOfOrderDetailIdsNeeded", numberOfOrderDetailIdsNeededParam);
I did it this way because it was compatible with BreezeJs for .NET Core. Note that I only really tested this with Value types.
I'm trying to complete a general repository for all of the entities in my application. I Have a BaseEntity with property Id, CreatorId and LastModifiedUserId. Now I'd like to Update a record in a collection, without having to modify the field CreatorId, so I have (from the client) an Entity valorized with some fields updated that I want to update.
Hi have 2 ways:
UpdateOneAsync
ReplaceOneAsync
The repo is created like this:
public class BaseRepository<T> : IRepository<T> where T : BaseEntity
{
public async Task<T> Replace/Update(T entity){...}
}
So it's very hard to use Update(1), since I should retrieve with reflection all the fields of T and exclude the ones that I don't want to update.
With Replace(2) I cannot find a way to specify which fields i should exclude when replacing an object with another. Projectionproperty in FindOneAndReplaceOptions<T>() just excludes the fields on the document that is returned after the update.
Am I missing a way in the replace method to exclude the fields or should I try to retrieve the fields with reflection and use a Update?
I don't know if this solution is ok for you .. what i do is :
Declare in Base Repo a method like
public virtual bool Update(TEntity entity, string key)
{
var result = _collection.ReplaceOne(x => x.Id.Equals(key), entity, new UpdateOptions
{
IsUpsert = false
});
return result.IsAcknowledged;
}
then in your controller when you want to update your entities is there where you set the prop you want to change .. like:
[HttpPut]
[ProducesResponseType(typeof(OrderDTO), 200)]
[ProducesResponseType(400)]
public async Task<ActionResult<bool>> Put([FromBody] OrderDTO value)
{
try
{
if (!ModelState.IsValid) return BadRequest(ModelState);
var orderOnDb = await _orderService.FindAsync(xx => xx.Id == value.Id);
if (orderOnDb == null) return BadRequest(Constants.Error.NOT_FOUND_ON_MONGO);
// SET PROPERTY TO UPDATE (MANUALLY)
orderOnDb.LastUpdateDate = DateTime.Now;
orderOnDb.PaymentMethod = value.PaymentMethod;
orderOnDb.StateHistory = value.StateHistory;
//Save on db
var res = await _orderRepo.UpdateAsync(orderOnDb, orderOnDb.Id);
return res;
}
catch (Exception ex)
{
_logger.LogCritical(ex, ex.Message);
throw ex;
}
}
Hope it helps you!!!
I created a web api project using .net core and entity framework.
This uses a stored procedure which returns back most of the properties of a database table defined by entity framework.
The entity framwrok does not bring back all the columns of the table. And I get an error when I call the api complaining it cannot find the missing columns when I execute the stored procedure using ,
_context.Set<TableFromSql>().FromSql("execute dbo.spr_GetValue").ToList();
I created another model class which defines the properties brought back from the SP( called NewClass).
_context.Set<NewClass>().FromSql("execute dbo.spr_GetValue").ToList();
This works, but just wanted to check if there is a convention that the SP should only return the model classes from the database.
The SQL query must return data for all properties of the entity or query type
For this limitation, it is caused when mapping the sql query result to Model. It loop through the properties in model and try to retrive the values from query result. If the model properties are not exist in query result, it will throw error.
If you want to return required columns instead of all columns, one options is to define the returned model by Query.
For your demo code, you may define this in OnModelCreating.
builder.Query<TableFromSql>();
Note, for this way, you need to make sure all properties in TableFromSql exist in execute dbo.spr_GetValue.
For another way, you may implement your own FromSql which will add condition to check whether the properties are exist in query result.
public static class DbContextExtensions
{
public static List<T> RawSqlQuery<T>(this DbContext context,string query)
{
using (var command = context.Database.GetDbConnection().CreateCommand())
{
command.CommandText = query;
command.CommandType = CommandType.Text;
context.Database.OpenConnection();
using (var result = command.ExecuteReader())
{
var entities = new List<T>();
return DataReaderMapToList<T>(result);
}
}
}
public static List<T> DataReaderMapToList<T>(IDataReader dr)
{
List<T> list = new List<T>();
T obj = default(T);
while (dr.Read())
{
obj = Activator.CreateInstance<T>();
foreach (PropertyInfo prop in obj.GetType().GetProperties())
{
if (ColumnExists(dr, prop.Name))
{
if (!object.Equals(dr[prop.Name], DBNull.Value))
{
prop.SetValue(obj, dr[prop.Name], null);
}
}
}
list.Add(obj);
}
return list;
}
public static bool ColumnExists(IDataReader reader, string columnName)
{
return reader.GetSchemaTable()
.Rows
.OfType<DataRow>()
.Any(row => row["ColumnName"].ToString() == columnName);
}
}
Use above code like :
var result = _context.RawSqlQuery<ToDoItemVM>("execute [dbo].[get_TodoItem]");