I have set a case-insensitive ValueComparer<string> for the primary key, but when calling Find() it returns null if the entity isn't being tracked.
The configuration of the property and its value comparer:
modelBuilder.Entity<ProductEntity>().HasKey(e => e.Name);
modelBuilder.Entity<ProductEntity>(e =>
{
e.Property(p => p.Name).Metadata.SetValueComparer(new ValueComparer<string>(
(s1, s2) => string.Equals(s1, s2, StringComparison.OrdinalIgnoreCase),
s => s.ToUpper().GetHashCode(),
s => s
));
});
Code example to highlight the problem:
var builder = new DbContextOptionsBuilder<ShoppingListContext>();
builder.UseSqlite(ShoppingListContextFactory.SqliteConn.Value);
var contextFirst = new ShoppingListContext(builder.Options);
contextFirst.Database.EnsureCreated();
contextFirst.Products.Add(new ProductEntity { Name = "Apple" });
contextFirst.SaveChanges();
var contextSecond = new ShoppingListContext(builder.Options);
Console.WriteLine(contextFirst.Products.Find("apple") is null);
Console.WriteLine(contextSecond.Products.Find("apple") is null);
Outputs:
False
True
I would suspect this is by design. The ValueComparer is used internally within EF for change tracking, essentially when working with entities that are already loaded and tracked where certain data types or rules around what constitutes a change would not be otherwise detected. Where a Database is normally case-insensitive, EF would normally detect changing "Atlanta" to "atlanta" as a valid change, where you might not want to actually trigger an UPDATE in that case since it wouldn't matter as far as the database is concerned. The side effect is that fetching data from the change tracking cache may use the configured value comparer, that doesn't automatically flow through to generated SQL commands. If your database collation is case-sensitive then your querying against string values will need to accommodate for that.
Forcing case insensitivity against a database that is case sensitive needs to be done carefully since indexing on the database may or may not match what you are configuring your Linq Queries to use which can have significant performance implications.
Related
There are a few things going on with Prisma update that I just don't get.
Why is the update (using the ORM way) not performed ?
Why the value of data.address seems to affect the outcome of the update ?
Do I have to provide all of the fields of the entity when updating ? Or in this case, could I just put what I want changed inside of data ?
I am using #prisma/client#3.15.2
Here is what I am currently working with:
const { valid: validFor, expire, state, address, ...safeProperties } = data;
const addressAsUnsigned = address >>> 0; // address is an ip address represented as an integer. It needs to be treated as unsigned
const extendBy = newValidFor - validFor;
const extended = add(expire, { seconds: extendBy });
const payload: Prisma.DataTableUpdateArgs = {
where: { address: addressAsUnsigned },
data: {
...safeProperties,
address: addressAsUnsigned,
expire: extended,
valid: authenticated,
state: {},
},
}
Logger.debug(payload);
// contains the changes I expect
const result = await db.dataTable.update(payload);
Logger.debug(result);
// result contains the same values as before the update.
// And indeed, when I check the database, nothing changed.
// Something like this does what I want, so there is really nothing complicated going on...
await db.$executeRaw`
UPDATE data_table SET
expire = ${extended},
valid = ${authenticated}
WHERE address = ${addressAsUnsigned}
`;
Hopefully, I have not missed something too obvious.
In my experience,
Why is the update (using the ORM way) not performed ?
You might be updating the wrong thing. Is your address an #unique field in your prisma.schema?
Why the value of data.address seems to affect the outcome of the update ?
Prisma might have messed some things up with wrong data. If your data is not unique, you might be updating the first row with that address. If you want to update multiple fields with same address, use updateMany
Do I have to provide all of the fields of the entity when updating ? Or in this case, could I just put what I want changed inside of data ?
No, you only need to put in the data that you need. In your "where" field, add the unique address, and in your data, only the fields that you are changing. In your case, expired and valid. If you want to skip updating some values, use "expired: undefined" and so on
Since you are using typescript, I would advise you to put your object directly inside the prisma update to get the correct types. (At least to fix this problem)
prisma.dataTable.update({where: {...}})
This way you will get the correct types. There is also a command to list all available args inside (control + space on mac)
Note that using the spread operator (...) will remove the listed types, so use it last.
Some other things: double check if your prisma import is correct. Is your data correct, is your returned data correct? Did you refresh your database on update? It might be updated but you just need to refresh for new changes.
I'm loading some data from a CSV file that has some extra notes fields that I don't want in the DB. Is there an option to just ignore extra fields when storing to the DB?
I think mongoose did this by default - which does have a downside that stuff goes missing without warning if your schema is wrong but... thats what i want in this case.
Otherwise what is a way to reflect and get the schema so I can remove extra fields from the data manually?
I'm getting this error on .create
Unknown arg `notes` in data.notes for type WalletCreateInput.
Did you mean `name`?
Available args:
...
It's not allowed to add extra fields while interacting with Prisma Query.
The current behaviour is intentional and it throws the error as an extra validation layer to make sure you're passing the right data.
There is a feature request that discusses allowing extra fields while interacting with Query.
As of now, destructuring the fields which are necessary and passing only the required fields is the only option.
Late to the party, but there is a way around this.
If you use the "fieldReference" preview feature:
generator client {
provider = "prisma-client-js"
previewFeatures = ["fieldReference"]
}
You can then create the following to strip out any extra keys.
function stripPrisma<T extends {}>(input: {fields:{}},data: T) : T {
let validKeys = Object.keys(input.fields);
let dataCopy: any = {...data};
for(let key of Object.keys(data)) {
if(!(validKeys.includes(key))) {
delete dataCopy[key];
}
}
return dataCopy as T;
}
And use it like this
data = stripPrisma(prisma.myTable, data);
prisma.myTable.create({data:data});
It is not perfect, since it will only be able to use "checked input", meaning you can only use the foreign key in your input and not the foreign object.
I know tis question has been asked several times on SO but I haven't found a solution that works.
It's clear that Cosmos is capable of case insensitive searches since June 2020 but I haven't managed to find a solution that works with EF.
For example...
await objDbContext.Tags.SingleOrDefaultAsync(t => t.Value.Equals("value", StringComparison.InvariantCultureIgnoreCase))
... throws the following exception:
System.InvalidOperationException: 'The LINQ expression 'DbSet()
.Where(t => t.Value.Equals(
value: "Value",
comparisonType: InvariantCultureIgnoreCase))' could not be translated. Additional information: Translation of the 'string.Equals' overload with a 'StringComparison' parameter is not supported. See https://go.microsoft.com/fwlink/?linkid=2129535 for more information. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.'
I've also tried .ToLower and string.Equals with similar results.
Is there a right way to do it? If not, is there an alternative? I'm happy to store the value in lowercase along side the actual value, to be used for searching only, but it must be updated automatically.
EF Core 5 has now implemented the transactions. Once you have updated you can change this line
await objDbContext.Tags.SingleOrDefaultAsync(t => t.Value.Equals("value", StringComparison.InvariantCultureIgnoreCase));
to
await objDbContext.Tags.SingleOrDefaultAsync(t => t.Value.ToLower() == "value");
I can see from the following example how to get the table name of an OSpace type:
https://lowrymedia.com/2014/06/10/ef6-1-mapping-between-types-tables-including-derived-types/
But how do I go about getting the SSpace column name from an OSpace property name (i.e. CLR type property)?
By browsing the MetadataProperties from the corresponding CSpace property, I can see there is a "Configuration" entry containing the column name if changed using the Fluid API or ColumnAttribute, but the value of the entry is an internal class on EF's part. Is it at all possible?
I have browsed a few answers regarding this topic, but none of them take into account the Fluid API configuration.
P.S. the specific property I'm looking for is scalar, if that can simplify things...
Column Name
To get the column name, you have to first get the EdmProperty associated with that column in the “structural space” (SSpace). I provide code to do that below. Once you have the EdmProperty, the name of the column is simply EdmProperty.Name:
string GetColumnName(DbContext context, PropertyInfo property) {
return GetStructuralSpaceEdmProperty(context, property).Name;
}
Structural Space Property
This is based on an article. That article gives you enough information to map all the way to the structural EntityType. I added a bit at the end to do the actual property mapping to get the EdmProperty representing the column. As the article states, these APIs require ≥EntityFramework-6.1.
EdmProperty GetStructuralSpaceEdmProperty(DbContext context, PropertyInfo property) {
IObjectContextAdapter adapter = context;
var metadata = adapter.ObjectContext.MetadataWorkspace;
// First, you map the Object Space to the Conceptual Space.
var objectItemCollection = (ObjectItemCollection)metadata.GetItemCollection(DataSpace.OSpace);
var objectEntityType = metadata.GetItems<EntityType>(DataSpace.OSpace)
.Single(oet => objectItemCollection.GetClrType(oet) == property.DeclaringType);
// Note: we are assuming that CSpace and OSpace name their properties the
// same instead of trying to use EF’s own OSSpace mappings here.
var conceptualEntityType = metadata.GetItems<EntityType>(DataSpace.CSpace)
.Single(cet => objectEntityType.Name == cet.Name);
var conceptualEdmProperty = conceptualEntityType.Properties
.Single(ep => ep.Name == property.Name);
// Then you map the conceptual space onto the structural space.
var entitySet = metadata.GetItems<EntityContainer>(DataSpace.CSpace)
.Single().EntitySets
.Single(es => es.ElementType.Name == conceptualEntityType.Name);
var entityMapping = metadata.GetItems<EntityContainerMapping>(DataSpace.CSSpace)
.Single().EntitySetMappings
.Single(esm => esm.EntitySet == entitySet);
// The entity may be split to different tables or fragments.
var fragments = entityMapping.EntityTypeMappings
.SelectMany(etm => etm.Fragments);
var propertyMappings = fragments.SelectMany(f => f.PropertyMappings);
// Normal properties will be “ScalarPropertyMapping”.
// Depending on what information you are seeking or your
// model, you may be interested in other PropertyMapping.
var structuralSpaceProperty = propertyMappings
.OfType<ScalarPropertyMapping>()
.Single(pm => pm.Property == conceptualEdmProperty).Column;
return structuralSpaceProperty;
}
Note that once you have EdmProperty in structural space, there are a bunch of other useful properties you can read from it. For example, for SQL Server, EdmProperty.IsUnicode will be true for NVARCHAR/NCHAR and false for VARCHAR/CHAR types whereas this property is not set to a useful value in the conceptual space.
Random Information
Spaces in EF
Alex D. James’s blog post “Tip 10 — How to understand Entity Framework jargon” explains some of the terms of the API which do not make sense on their own. DataSpace.OSpace stands for “Object Space”, meaning the .net POD classes. DataSpace.SSpace stands for “Structural Space”, probably named after “structured” in the term “SQL” and thus meaning it most directly describes the backend database. DataSpace.CSpace stands for “Conceptual Space” which seems intended to be a neutral space which both the “Object Space” and “Structural Space” can map into. DataSpace.OCSpace stands for the mapping from the object space onto the conceptual space. We bypass this mapping because we assume that property names in the object space are the same as in our .net types. DataSpace.CSSpace stands for the mapping of conceptual space onto structural space. We use this mapping because the model may be configured to use a different column name via the fluent API or ColumnAttribute.
API Confusion
The metadata API of EF seems to assume that the consumer of the API has an understanding of the internals of EF to an extent. It is not made in a very type safe way which helps consumers. For example, the fact that we had to use Enumerable.OfType<TResult> to get to ScalarPropertyMapping means that one has to know to expect the collection to have ScalarPropertyMapping instances in it. Likewise, the MetadataWorkspace.GetItems<T>() method requires us to know that the sorts of items one would find in the metadata include EntityType. Thus, a deep understanding of the internals of EF or complete examples are necessary to write code that consumes the mapping portion of these APIs.
I have a really long Excel file wich I read using EPPlus. For each line I test if it meets certain criteria and if so I add the line (an object representing the line) to a collection. When the file is read, I store those objects to the database. Would it be possible to do both things at the same time? My idea is to have a collection of objects that somehow would be consumed by thread that would save the objects to the DB. At the same time the excel reader method would populate the collection... Could this be done using Rx or is there a better method?
Thanks.
An alternate answer - based on comments to my first.
Create a function returning an IEnumberable<Records> from EPPlus/Xls - use yield return
then convert the seqence to an observable on the threadpool and you've got the Rx way of having a producer/consumer and BlockingCollection.
function IEnumberable<Records> epplusRecords()
{
while (...)
yield return nextRecord;
}
var myRecords = epplusRecords
.ToObservable(Scheduler.ThreadPool)
.Where(rec => meetsCritera(rec))
.Select(rec => newShape(rec))
.Do(newRec => writeToDb(newRec))
.ToArray();
Your case seems to be of pulling data (IEnumerable) and not pushing data (IObservable/Rx).
Hence I would suggest LINQ to objects is something that can be used to model the solution.
Something like shown in below code.
publis static IEnumerable<Records> ReadRecords(string excelFile)
{
//Read from excel file and yield values
}
//use linq operators to do filtering
var filtered = ReadRecords("fileName").Where(r => /*ur condition*/)
foreach(var r in filtered)
WriteToDb(r);
NOTE: In using IEnumerable you don't create intermediate collections in this case and the whole process looks like a pipeline.
It doesn't seem like a good fit, as there's no inherent concurrency, timing, or eventing in the use case.
That said, it may be a use case for plinq. If EEPlus supports concurrent reads. Something like
epplusRecords
.AsParallel()
.Where(rec => meetsCritera(rec))
.Select(rec => newShape(rec))
.Do(newRec => writeToDb(newRec))
.ToArray();