There appears to be mismatch betweeen the SQL generated by NHibernate and SQL expected by SQL2008 in the following case:
public void PersistPerson()
{
var sessionFactory = CreateSessionFactory();
using (var session = sessionFactory.OpenSession())
{
using(var transaction = session.BeginTransaction())
{
session.Save(new Person {FirstName = "Foo", LastName = "Bar"});
session.Save(new Person {FirstName = "Foo", LastName = "Dah"});
session.Save(new Person {FirstName = "Foo", LastName = "Wah"});
transaction.Commit();
}
}
using (var session = sessionFactory.OpenSession())
{
using(var transaction = session.BeginTransaction())
{
var queryable = from p in session.Query<Person>() select p;
var lastNames = new[]{"B", "D"};
var result = queryable.Where(r => lastNames.Contains(r.LastName.Substring(0, 1))).ToList();
transaction.Commit();
Assert.That(result[0].LastName, Is.EqualTo("Bar"));
}
}
}
The resulting sql query generated by NHibernate for
var result = queryable.Where(r => lastNames.Contains(r.LastName.Substring(0, 1))).ToList();
is:
select person0_.Id as Id0_,
person0_.FirstName as FirstName0_,
person0_.LastName as LastName0_ from [Person] person0_ where upper(substring(person0_.LastName,
0 /* #p0 */,
1 /* #p1 */)) in ('B' /* #p2 */)
From the MSDN documentation for T-SQL SUBSTRING
http://msdn.microsoft.com/en-us/library/ms187748.aspx
SUBSTRING (value_expression ,start_expression ,length_expression )
although the documentation says otherwise, from the comments posted start_expression appears to be 1 - based (not 0 indexed)
For example:
SQL: SELECT x = SUBSTRING('abcdef', 0, 3);
RESULT: x = 'ab'
and NOT x = 'abc'
Any thoughts on how I can get around this ?
I think it's a bug. just change your codes to r.LastName.Substring(1, 1) and it works (resulting sql will be substring(1,1)).
Related
I have 20 Dbsets in my context which I want to get the row count for each dbset to make sure all dbset row count is 0. To get the count for one dbset, this is my code:
var person = context.Persons.Count();
Is there a way to loop through the context, get the count for each dbset dynamically?
There is solution. Usage is simple:
var tablesinfo = ctx.GetTablesInfo();
if (tablesinfo != null)
{
var withRecords = tablesinfo
.IgnoreQueryFilters()
.Where(ti => ti.RecordCount > 0)
.ToArray();
}
Extension returns IQueryable<TableInfo> and you can reuse this query later. Probably you will need to filter out Views, but I think you can handle that. Note that IgnoreQueryFilters can be important if you have Global Query Filters defined.
What extension do:
It scans Model for entity types registered for particular DbContext and generates big Concat of Count queries. Here we have to do that via grouping by constant value.
Schematically it will generate the following LINQ query:
var tablesinfo =
ctx.Set<Entity1>.GroupBy(e => 1).Select(g => new TableInfo { TableName = "Entity1", RecordCount = g.Count()})
.Concat(ctx.Set<Entity2>.GroupBy(e => 1).Select(g => new TableInfo { TableName = "Entity2", RecordCount = g.Count()}))
.Concat(ctx.Set<Entity3>.GroupBy(e => 1).Select(g => new TableInfo { TableName = "Entity3", RecordCount = g.Count()}))
...
Which wll be converted to the following SQL:
SELECT "Entity1" AS TableName, COUNT(*) AS RecordCount FROM Entity1
UNION ALL
SELECT "Entity2" AS TableName, COUNT(*) AS RecordCount FROM Entity2
UNION ALL
SELECT "Entity3" AS TableName, COUNT(*) AS RecordCount FROM Entity3
...
Implementation:
public static class QueryableExtensions
{
public class TableInfo
{
public string TableName { get; set; } = null!;
public int RecordCount { get; set; }
}
public static IQueryable<TableInfo> GetTablesInfo(this DbContext ctx)
{
Expression query = null;
IQueryProvider provider = null;
var ctxConst = Expression.Constant(ctx);
var groupingKey = Expression.Constant(1);
// gathering information for MemberInit creation
var newExpression = Expression.New(typeof(TableInfo).GetConstructor(Type.EmptyTypes));
var tableNameProperty = typeof(TableInfo).GetProperty(nameof(TableInfo.TableName));
var recordCountProperty = typeof(TableInfo).GetProperty(nameof(TableInfo.RecordCount));
foreach (var entityType in ctx.Model.GetEntityTypes())
{
var entityParam = Expression.Parameter(entityType.ClrType, "e");
var tableName = entityType.GetTableName();
// ctx.Set<entityType>()
var setQuery = Expression.Call(ctxConst, nameof(DbContext.Set), new[] {entityType.ClrType});
// here we initialize IQueryProvider, which is needed for creating final query
provider ??= ((IQueryable) Expression.Lambda(setQuery).Compile().DynamicInvoke()).Provider;
// grouping paraneter has generic type, we have to specify it
var groupingParameter = Expression.Parameter(typeof(IGrouping<,>).MakeGenericType(typeof(int), entityParam.Type), "g");
// g => new TableInfo { TableName = "tableName", RecordCount = g.Count() }
var selector = Expression.MemberInit(newExpression,
Expression.Bind(tableNameProperty, Expression.Constant(tableName)),
Expression.Bind(recordCountProperty,
Expression.Call(typeof(Enumerable), nameof(Enumerable.Count), new[] {entityParam.Type}, groupingParameter)));
// ctx.Set<entityType>.GroupBy(e => 1)
var groupByCall = Expression.Call(typeof(Queryable), nameof(Queryable.GroupBy), new[]
{
entityParam.Type,
typeof(int)
},
setQuery,
Expression.Lambda(groupingKey, entityParam)
);
// ctx.Set<entityType>.GroupBy(e => 1).Select(g => new TableInfo { TableName = "tableName", RecordCount = g.Count()}))
groupByCall = Expression.Call(typeof(Queryable), nameof(Queryable.Select),
new[] {groupingParameter.Type, typeof(TableInfo)},
groupByCall,
Expression.Lambda(selector, groupingParameter));
// generate Concat if needed
if (query != null)
query = Expression.Call(typeof(Queryable), nameof(Queryable.Concat), new[] {typeof(TableInfo)}, query,
groupByCall);
else
query = groupByCall;
}
// unusual situation, but Model can have no registered entities
if (query == null)
return null;
return provider.CreateQuery<TableInfo>(query);
}
}
I have this sql query which perform an foreing key validation, and validated whether both entries uphold a certain standard.
The query should return something (since the data I pass in should fail), as long there is data in the database.
But what I've noticed is that, when I in the same transaction insert data and then perform the validation in the same transaction the validation query returns nothing, when it in actuality should return numerous rows.
if I do the same step as above, meaning I already have committed data in the database, and insert the same data, my validation script returns the excepted rows?
is there any limitations in regards to using transcations?
in here I insert data
private IEnumerable<long> PersistEntityRegistrations(IEnumerable<EntityRegistration> payload, string entityName, NpgsqlTransaction transaction)
{
IEnumerable<string> customAttributeNames = GetEntitySpecificAttributes(entityName);
IEnumerable<string> allAttributeNames =
EntityRegistration.standardAttributeNames.Union(customAttributeNames);
var insertNewRegistrationSql =
#$"INSERT INTO ""{entityName}_registration"" ({string.Join(",", allAttributeNames)})
SELECT * FROM unnest({string.Join(",", allAttributeNames.Select(x => "#" + x))})
RETURNING row_id;";
NpgsqlCommand insertRegistrations = new NpgsqlCommand(insertNewRegistrationSql, transaction.Connection, transaction);
DateTime now = DateTime.UtcNow;
NpgsqlParameter entity_id = new NpgsqlParameter("entity_id", payload.Where(x => x.EntityId.HasValue).Select(x => x.EntityId ?? 0).ToArray());
insertRegistrations.Parameters.Add(entity_id);
NpgsqlParameter entryName = new NpgsqlParameter("entry_name", payload.Select(x => x.EntryName).ToArray());
insertRegistrations.Parameters.Add(entryName);
NpgsqlParameter registrationBy = new NpgsqlParameter("registration_by", payload.Select(x => x.RegistrationBy).ToArray());
insertRegistrations.Parameters.Add(registrationBy);
NpgsqlRange<DateTime>[] valid = payload.Select(x => x.Valid).ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter("valid", valid));
NpgsqlRange<DateTime>[] registration = payload.Select(x => new NpgsqlRange<DateTime>(now, lowerBoundIsInclusive: true, DateTime.MaxValue, upperBoundIsInclusive: true)).ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter("registration", registration));
Entity entity = schemaService.ReadSchema().Entities.SingleOrDefault(x => x.InternalName == entityName);
Relation relation = schemaService.ReadSchema().Relations.SingleOrDefault(x => x.InternalName == entityName);
ICollection<Attribute> entityAttributeDefinitions = entity != null ? entity.Attributes : relation.Attributes ;
foreach (Attribute attributeDefinition in entityAttributeDefinitions)
{
IEnumerable<object> attributeValues = payload.Where(x => x.Attributes.ContainsKey(attributeDefinition.InternalName))
.Select(x => x.Attributes[attributeDefinition.InternalName]);
switch (attributeDefinition.Type)
{
case DataType.Dropdown:
case DataType.WeekDay:
case DataType.Text:
var castedStringValues = attributeValues.Cast<string?>().ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter(attributeDefinition.InternalName, castedStringValues));
break;
case DataType.Number:
var castedNumberValues = attributeValues.Cast<decimal>().ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter(attributeDefinition.InternalName, castedNumberValues));
break;
case DataType.Lookup:
var castedLookupValues = attributeValues.Cast<decimal>().ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter(attributeDefinition.InternalName, castedLookupValues));
break;
case DataType.DateTime:
var castedDateTimeValues = attributeValues.Cast<DateTime?>().ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter(attributeDefinition.InternalName, castedDateTimeValues));
break;
case DataType.Boolean:
// Npgsql is currently not able to handle bool?[], hence this exact type cannot handle null values.
// This is currently support in v. 5 but since EFCore has a hard dependency on v.4
// which forces us to use bool[].
var castedBooleanValues = attributeValues.Cast<bool>().ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter(attributeDefinition.InternalName, castedBooleanValues));
break;
}
}
insertRegistrations.Prepare();
insertRegistrations.CommandTimeout = 0;
var insertedRowIds = new List<long>();
using (NpgsqlDataReader dataReader = insertRegistrations.ExecuteReader())
{
while (dataReader.Read())
{
insertedRowIds.Add((long)dataReader[0]);
}
}
return insertedRowIds;
}
Here I do the validation
private IEnumerable<(string, long, NpgsqlRange<DateTime>, bool, NpgsqlRange<DateTime>, long, string, bool)> DetectLookupLifespanViolations(string entityName, IEnumerable<long> entity_ids,
NpgsqlTransaction transaction)
{
var sql = File.ReadAllText(Path.Combine("SQLQueries", "IntegrityCheck.sql"));
var integrityCheckCommand = new NpgsqlCommand(sql, transaction.Connection, transaction);
integrityCheckCommand.Parameters.Add(new NpgsqlParameter<long[]>("#entity_ids", entity_ids.ToArray()));
integrityCheckCommand.Parameters.Add(new NpgsqlParameter<string>("table_name", entityName));
var integrity_breached =
new List<(string, long, NpgsqlRange<DateTime>, bool, NpgsqlRange<DateTime>, long, string, bool)>();
var has_values = new NpgsqlCommand($#"select count(*) from {entityName}_registration", transaction.Connection, transaction);
var result = has_values.ExecuteScalar();
Console.WriteLine(result);
using NpgsqlDataReader dataReader = integrityCheckCommand.ExecuteReader();
while (dataReader.Read())
{
bool hasDateTimeIntegrity = (bool)(dataReader.GetValue(3) ?? true);
if (!hasDateTimeIntegrity)
{
integrity_breached.Add((dataReader.GetFieldValue<string>(0),
dataReader.GetFieldValue<long>(1),
dataReader.GetFieldValue<NpgsqlRange<DateTime>>(2),
dataReader.GetFieldValue<bool>(3),
dataReader.GetFieldValue<NpgsqlRange<DateTime>>(4),
dataReader.GetFieldValue<long>(5),
dataReader.GetFieldValue<string>(6),
dataReader.GetFieldValue<bool>(7)
));
}
}
return integrity_breached;
}
The both called within a method were
public bool InsertEntityRegistrations(string entityName, IEnumerable<EntityRegistration> payload)
{
using var connection = new NpgsqlConnection(configuration.GetConnectionString("PostgreSQL"));
connection.Open();
using var transaction = connection.BeginTransaction();
....
IEnumerable<long> insertedRowIds = PersistEntityRegistrations(payload, entityName, transaction);
....
var registrationsCausingIntegrityIssues = DetectLookupLifespanViolations(entityName, allEntityIds, transaction);
...
}
}
The UpdateRange method of Entity Framework Core is used here to update multiple records but it is not working.
My code is:
var dept1 = new Department()
{
Id = 8,
Name = "New Designing"
};
var dept2 = new Department()
{
Id = 9,
Name = "New Research"
};
var dept3 = new Department()
{
Id = 102,
Name = "New HR"
};
List<Department> modifiedDept = new List<Department>() { dept1, dept2, dept3 };
using (var context = new CompanyContext())
{
context.UpdateRange(modifiedDept);
await context.SaveChangesAsync();
}
And the error I get is:
Microsoft.EntityFrameworkCore.DbUpdateConcurrencyException: 'Database operation expected to affect 1 row(s) but actually affected 0 row(s). Data may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions.'
What should be done in this case?
You are supposed to get data from database and modify data. Not creating new class.
using (var context = new JobContext())
{
var depts = context.Department.Where(x => x.Id > 1).AsQueryable();
depts.Where(x => x.Id == 2).FirstOrDefault().Name = "New Designing";
depts.Where(x => x.Id == 3).FirstOrDefault().Name = "New Research";
depts.Where(x => x.Id == 4).FirstOrDefault().Name = "New HR";
context.UpdateRange(depts);
context.SaveChanges();
}
Before
After
Below find a method that does not work. We fail on the line query.Select(...
Below that find a method with hard coded object property names which does work. But, this method is obviously not dynamic, nor flexible. There may be many properties of a Customer I may wish to search on.
The error string is at bottom. I get it that somehow the LINQ to Entity is unable to deal with conversion of GetValue to some sort of TSQL. Would anyone know of how I might code this up?
public List<Customer> GetForQuery(params Tuple<string, string>[] keyValuePairs) {
using (var db = new DBEntities()) {
var availableProperties = typeof(Customer).GetTypeInfo().DeclaredProperties.ToList();
var query = db.Customers.Select(c => c);
foreach (Tuple<string, string> pair in keyValuePairs) {
PropertyInfo pi = availableProperties.First(p => p.Name.Equals(pair.Item1));
if (pi == null)
continue;
query = query.Where(u => pi.GetValue(u, null).ToString().StartsWith(pair.Item2));
}
var results = query.Select(c => c).ToList();
return results;
}
}
How I might call the above:
CustomerController custController = new CustomerController();
List<Customer> results = custController.GetForQuery(Tuple.Create<string, string>("FName", "Bob" ));
The working fixed method:
public List<Customer> GetForQuery(string firstName = "", string lastName = "", string phoneNumber = "") {
using (var db = new DBEntities()) {
var query = db.Customers.Select(c => c);
if (firstName.HasContent())
query = query.Where(u => u.FName.StartsWith(firstName));
if (lastName.HasContent())
query = query.Where(u => u.LName.StartsWith(lastName));
if (phoneNumber.HasContent())
query = query.Where(u => u.EveningPhone.StartsWith(phoneNumber));
var results = query.Select(c => c).ToList();
return results;
}
}
ERROR:
LINQ to Entities does not recognize the method 'System.Object GetValue(System.Object, System.Object[])' method, and this method cannot be translated into a store expression.
I'm trying to create WHERE clause with OR conditions using DbExtensions.
I'm trying to generate SQL statement which looks like
SELECT ID, NAME
FROM EMPLOYEE
WHERE ID = 100 OR NAME = 'TEST'
My C# code is
var sql = SQL.SELECT("ID, FIRSTNAME")
.FROM("EMPLOYEE")
.WHERE("ID = {0}", 10)
.WHERE("NAME = {0}", "TEST");
How do I get the OR seperator using the above mentioned DbExtensions library?
I have found a definition for logical OR operator here
public SqlBuilder _OR<T>(IEnumerable<T> items, string itemFormat, Func<T, object[]> parametersFactory) {
return _ForEach(items, "({0})", itemFormat, " OR ", parametersFactory);
}
And some code examples here
public SqlBuilder Or() {
int[][] parameters = { new[] { 1, 2 }, new[] { 3, 4} };
return SQL
.SELECT("p.ProductID, p.ProductName")
.FROM("Products p")
.WHERE()
._OR(parameters, "(p.CategoryID = {0} AND p.SupplierID = {1})", p => new object[] { p[0], p[1] })
.ORDER_BY("p.ProductName, p.ProductID DESC");
}
I think (by analogy with example) in your case code should be something like this (but I can't test it for sure):
var params = new string[] { "TEST" };
var sql = SQL.SELECT("ID, FIRSTNAME")
.FROM("EMPLOYEE")
.WHERE("ID = {0}", 10)
._OR(params, "NAME = {0}", p => new object[] { p })
Hope this helps :)
By the way... have you tried this way?
var sql = SQL.SELECT("ID, FIRSTNAME")
.FROM("EMPLOYEE")
.WHERE(string.Format("ID = {0} OR NAME = '{1}'", 10, "TEST"))
It's simpler than you think:
var sql = SQL
.SELECT("ID, FIRSTNAME")
.FROM("EMPLOYEE")
.WHERE("(ID = {0} OR NAME = {1})", 10, "TEST");
One can use:
.AppendClause("OR", ",", "NAME = {0}",new object[]{"TEST"});