Entity Core Insert Adding Duplicate Rows - entity-framework

I'm trying to inserts an object and it's coming from a JSON file. The object has relationships which I've configured in EF Core and show in the database properly.
Unfortunately some of the data has duplicate entries in it but the Save action only happens at the end - so instead of getting a single child I get two child object when only one should exist.
JArray obj2 = (JArray)result.SelectToken("resistances");
List<CardResistance> pCardResistances = new List<CardResistance>();
foreach (var result2 in obj2)
{
//Search for Resistance Value object and create if not exists
Resistance ResistanceObj = ctx.Resistances.SingleOrDefault(m => m.ResistanceValue.Equals((string)result2["value"]))
?? new Resistance
{
ResistanceValue = (string)result2["value"],
LastUpdateDate = DateTime.Now
};
pCardResistances.Add(
new PokemonCardResistance
{
PokemonCard = PokemonCardObj,
Resistance = ResistanceObj
}
);
}
CardObj.CardResistances = pCardResistances;
...
ctx.AddOrUpdate and ctx.SaveChanges occur later in the code.
If you have JSON with the:
"resistances": [
{
"type": "Value1",
"value": "-30"
},
{
"type": "Value1",
"value": "-30"
}
]
In it what's happening is both rows get inserted for the entity (caused as the Save happens right at the end of all of the object data being populated. How can I save just the child object after each loop so it won't insert this data twice?

Your problem is to do with how you are checking for existence:
List<CardResistance> pCardResistances = new List<CardResistance>();
foreach (var result2 in obj2)
{
//Search for Resistance Value object and create if not exists.
//This will do a DB hit each time - if it doesn't exist and you have two
//items of the same value to add, then you will get that it does not exist
//each time.
Resistance ResistanceObj = ctx.Resistances.SingleOrDefault(m =>
m.ResistanceValue.Equals((string)result2["value"]))
?? new Resistance
{
ResistanceValue = (string)result2["value"],
LastUpdateDate = DateTime.Now
};
//One way to fix this would be to check that there is not already an item in this
//list that has this value set. And only either add/overwrite if that is the case.
pCardResistances.Add(
new PokemonCardResistance
{
PokemonCard = PokemonCardObj,
Resistance = ResistanceObj
}
);
}
CardObj.CardResistances = pCardResistances;
The other thing to note is more of a question as to what this should do exactly. Getting two updates in the same JSON for the same entity like this is a bit of an odd approach - is this valid?
If it is valid, then which one should 'win' (first or last in the json) and you may then be better placed altering the collection that you loop around (obj2 in your code - less said about your naming conventions the better...).

Related

DynamoDB - How to upsert nested objects with updateItem

Hi I am newbie to dynamoDB. Below is the schema of the dynamo table
{
"user_id":1, // partition key
"dob":"1991-09-12", // sort key
"movies_watched":{
"1":{
"movie_name":"twilight",
"movie_released_year":"1990",
"movie_genre":"action"
},
"2":{
"movie_name":"harry potter",
"movie_released_year":"1996",
"movie_genre":"action"
},
"3":{
"movie_name":"lalaland",
"movie_released_year":"1998",
"movie_genre":"action"
},
"4":{
"movie_name":"serendipity",
"movie_released_year":"1999",
"movie_genre":"action"
}
}
..... 6 more attributes
}
I want to insert a new item if the item(that user id with dob) did not exist, otherwise add the movies to existing movies_watched map by checking if the movie is not already available the movies_watched map .
Currently, I am trying to use update(params) method.
Below is my approach:
function getInsertQuery (item) {
const exp = {
UpdateExpression: 'set',
ExpressionAttributeNames: {},
ExpressionAttributeValues: {}
}
Object.entries(item).forEach(([key, item]) => {
if (key !== 'user_id' && key !== 'dob' && key !== 'movies_watched') {
exp.UpdateExpression += ` #${key} = :${key},`
exp.ExpressionAttributeNames[`#${key}`] = key
exp.ExpressionAttributeValues[`:${key}`] = item
}
})
let i = 0
Object.entries(item. movies_watched).forEach(([key, item]) => {
exp.UpdateExpression += ` movies_watched.#uniqueID${i} = :uniqueID${i},`
exp.ExpressionAttributeNames[`#uniqueID${i}`] = key
exp.ExpressionAttributeValues[`:uniqueID${i}`] = item
i++
})
exp.UpdateExpression = exp.UpdateExpression.slice(0, -1)
return exp
}
The above method just creates update expression with expression names and values for all top level attributes as well as nested attributes (with document path).
It works well if the item is already available by updating movies_watched map. But throws exception if the item is not available and while inserting. Below is exception:
The document path provided in the update expression is invalid for update
However, I am still not sure how to check for duplicate movies in movies_watched map
Could someone guide me in right direction, any help is highly appreciated!
Thanks in advance
There is no way to do this, given your model, without reading an item from DDB before an update (at that point the process is trivial). If you don't want to impose this additional read capacity on your table for update, then you would need to re-design your data model:
You can change movies_watched to be a Set and hold references to movies. Caveat is that Set can contain only Numbers or Strings, thus you would have movie id or name or keep the data but as JSON Strings in your Set and then parse it back into JSON on read. With SET you can perform ADD operation on the movies_watched attribute. https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.UpdateExpressions.html#Expressions.UpdateExpressions.ADD
You can go with single table design approach and have these movies watched as separate items with (PK:userId and SK:movie_id). To get a user you would perform a query and specify only PK=userId -> you will get a collection where one item is your user record and others are movies_watched. If you are new to DynamoDB and are learning the ropes, then I would suggest go with this approach. https://www.alexdebrie.com/posts/dynamodb-single-table/

EF6 can I update model/table after lambda querying?

I am lambda querying models (I make projection with other classes-GameBankVM, GameCouponBankVM) and at the end, I would like to loop throuh query result and update the model field. But I am getting The entity or complex type 'EPINMiddleWareAPI.Models.GameBankVM' cannot be constructed in a LINQ to Entities query.
Here is my sample code:
var gameBankResult = await (context.GameBanks.Where(g => g.productCode == initiate.productCode)
.Take(initiate.quantity)
.Select(g => new GameBankVM
{
quantity = g.quantity,
currency = g.currency,
initiationResultCode = g.initiationResultCode,
productCode = g.productCode,
productDescription = g.productDescription,
referenceId = g.referenceId,
responseDateTime = g.responseDateTime,
unitPrice = g.unitPrice,
totalPrice = g.totalPrice,
coupons = g.coupons.Select(c => new GameCouponBankVM
{
Pin = c.Pin,
Serial = c.Serial,
expiryDate = c.expiryDate
}).ToList()
})).ToListAsync();
if (gameBankResult.Count() != 0)
{
foreach (var item in gameBankResult)
{
item.referenceId = initiate.referenceId;
context.Entry(item).State = System.Data.Entity.EntityState.Modified;
}
await context.SaveChangesAsync();
return Ok(gameBankResult);
}
How can I update referenceId on my GameBank model/table?
In this scenario, your data won't be updated because your query is returning a List of GameBankVM and not a List of GameBank, now technically speaking, you are breaking SRP, you should either update your data or query your data not both in the same method, you may want to refactor your method like this :
1.- Create a private method for data update, in this case, you query directly GameBank iterate thru list entries, make your changes and save them to the database, this same method can return List of GameBank to avoid another database roundtrip.
2.- In the controller after you call your new method, you can run the transformation query to convert List of GameBank to List of GameBankVM and return it to the view.
There are many other ways to do this, I'm just recommending this as a less impact way to make your controller work. But if you are willing to make things better, you can create a business layer where you resolve all your business rules, or you can use patterns like CQS or CQRS.

Bulk Operations - Adding multiple rows to sheet

I'm attempting to write data from a data table to a sheet via the smartsheet API (using c# SDK). I have looked at the documentation and I see that it supports bulk operations but I'm struggling with finding an example for that functionality.
I've attempted to do a work around and just loop through each record from my source and post that data.
//Get column properties (column Id ) for existing smartsheet and add them to List for AddRows parameter
//Compare to existing Column names in Data table for capture of related column id
var columnArray = getSheet.Columns;
foreach (var column in columnArray)
{
foreach (DataColumn columnPdiExtract in pdiExtractDataTable.Columns)
{
//Console.WriteLine(columnPdiExtract.ColumnName);
if(column.Title == columnPdiExtract.ColumnName)
{
long columnIdValue = column.Id ?? 0;
//addColumnArrayIdList.Add(columnIdValue);
addColumnArrayIdList.Add(new KeyValuePair<string, long>(column.Title,columnIdValue));
}
}
}
foreach(var columnTitleIdPair in addColumnArrayIdList)
{
Console.WriteLine(columnTitleIdPair.Key);
var results = from row in pdiExtractDataTable.AsEnumerable() select row.Field<Double?>(columnTitleIdPair.Key);
foreach (var record in results)
{
Cell[] cells = new Cell[]
{
new Cell
{
ColumnId = columnTitleIdPair.Value,
Value = record
}
};
cellRecords = cells.ToList();
cellRecordsInsert.Add(cellRecords);
}
Row rows = new Row
{
ToTop = true,
Cells = cellRecords
};
IList<Row> newRows = smartsheet.SheetResources.RowResources.AddRows(sheetId, new Row[] { rows });
}
I expected to generate a value for each cell, append that to the list and then post it through the Row Object. However, my loop is appending the column values as such: A1: 1, B2: 2, C3: 3 instead of A1: 1, B1: 2, C3: 3
The preference would be to use bulk operations, but without an example I'm a bit at a loss. However, the loop isn't working out either so if anyone has any suggestions I would be very grateful!
Thank you,
Channing
Have you seen the Smartsheet C# sample read / write sheet? That may be a useful reference. It contains an example use of bulk operations that updates multiple rows with a single call.
Taylor,
Thank you for your help. You lead me in the right direction and I figured my way through a solution.
I grouped by my column value list and built records for the final bulk operation. I used a For loop but the elements in each grouping of columns is cleaned and assigned a 0 prior to this method so that they retain the same count of values per grouping.
// Pair column and cell values for row building - match
// Data source column title names with Smartsheet column title names
List<Cell> pairedColumnCells = new List<Cell>();
//Accumulate cells
List<Cell> cellsToImport = new List<Cell>();
//Accumulate rows for additions here
List<Row> rowsToInsert = new List<Row>();
var groupByCells = PairDataSourceAndSmartsheetColumnToGenerateCells(
sheet,
dataSourceDataTable).GroupBy(
c => c.ColumnId,
c => c.Value,
(key, g) => new {
ColumnId = key, Value = g.ToList<object>()
});
var countGroupOfCells = groupByCells.FirstOrDefault().Value.Count();
for (int i = 0; i <= countGroupOfCells - 1; i++)
{
foreach (var groupOfCells in groupByCells)
{
var cellListEelement = groupOfCells.Value.ElementAt(i);
var cellToAdd = new Cell
{
ColumnId = groupOfCells.ColumnId,
Value = cellListEelement
};
cellsToImport.Add(cellToAdd);
}
Row rows = new Row
{
ToTop = true,
Cells = cellsToImport
};
rowsToInsert.Add(rows);
cellsToImport = new List<Cell>();
}
return rowsToInsert;

Linq to Entity - Explanation/Advice on behaviour Where() vs Where().ToList() in foreach loop

So the scenario in a nutshell: I have an entity (EF4) that has a foreign key to the primary key in the same entity (hierarchical structure) e.g.
MyEntityId (Primary Key)
ParentMyEntityId (Foreign key to Primary Key)
If I have a MyEntityList List<MyEntity> where the EntityState for both is Unchanged:
entity 1 - {MyEntityId = 10, ParentMyEntityId = null}
entity 2 - {MyEntityId = 11, ParentMyEntityId = 10}
and then I do this:
//Initially has 2 items in 'in' clause - iterates once and then exits because the EntityState of the second item has changed to Modified
foreach(MyEntity m in MyEntityList.Where(e => e.EntityState == System.Data.EntityState.Unchanged))
{
db.DeleteObject(m);
}
The first MyEntity is deleted, but the second changes to "Modified" and the foreach doesn't run a second time - I'm guessing due to the foreign key constraint.
However if I do:
//Iterates twice, even though the EntityState of the second item has changed to Modified
foreach(MyEntity m in MyEntityList.Where(e => e.EntityState == System.Data.EntityState.Unchanged).ToList())
{
db.DeleteObject(m);
}
Both entities are deleted (which is the desired effect).
Whilst I have a solution, I'm interested in why this happens, I was always under the impression that the iterator "set" that was defined at the start of the foreach loop remained the same, or threw a runtime error if you tried to modify it.
Should I not be using Where? Is there a better way to do this?
The iterator set is defined before the loop runs because you execute ToList (if you didn't do this it wouldn't be well defined).
So the iteration source is constant. But not the object you iterate over. The object references you get are constant but not the objects pointed to by them.
The version without ToList is equivalent to:
foreach(MyEntity m in MyEntityList) //always 2 items
{
//loop body always called two times
if (e.EntityState == System.Data.EntityState.Unchanged) //2 times
db.DeleteObject(m); //1 time
}
The ToList-version is equivalent to:
var copy = MyEntityList.Where(e => e.EntityState == System.Data.EntityState.Unchanged).ToList(); //always 2 items
foreach(MyEntity m in copy)
{
//loop body always called two times
db.DeleteObject(m); //2 times
}

Join 2 different entities from 2 different models into a single Linq to Entities query

I have a default Entity Framework model that holds all of my default tables for my product, and that all customers share in common. However, on some customers, I have some custom tables that exist for only that customer, but they relate to the default product's tables. I have a second Entity Framework model to hold these custom tables.
My question is how can I make a Linq to Entities query using Join so I can relate the entities from my default model to the tables on my custom model? I don't mind not having the Navigation properties from the custom entity to the entities on the default model; I just need a way to query both models in a single query.
Below is the code:
using (ProductEntities oProductDB = new ProductEntities())
{
using (ProductEntitiesCustom oProductCustomDB = new ProductEntitiesCustom())
{
var oConsulta = oProductCustomDB.CTBLCustoms
.Where(CTBLCustoms => CTBLCustoms.IDWOHD >= 12)
.Join(oProductDB.TBLResources,
CTBLCustoms => new
{
CTBLCustoms.IDResource
},
TBLResources => new
{
TBLResources.IDResource
},
(CTBLCustoms, TBLResources) => new
{
IDCustom = CTBLCustoms.IDCustom,
Descricao = CTBLCustoms.Descricao,
IDWOHD = CTBLCustoms.IDWOHD,
IDResource = CTBLCustoms.IDResource,
ResourceCode = TBLResources.Code
});
gvwDados.DataSource = oConsulta;
}
}
I get a The specified LINQ expression contains references to queries that are associated with different contexts error.
EDIT
Could I merge the 2 ObjectContext into a third, and then run the Linq query?
Tks
EDIT
Below is the code that worked, using the AsEnumerable() proposed solution:
using (ProductEntities oProductDB = new ProductEntities())
{
using (ProductEntitiesCustom oProductCustomDB = new ProductEntitiesCustom())
{
var oConsulta = (oProductCustomDB.CTBLCustoms.AsEnumerable()
.Where(CTBLCustoms => CTBLCustoms.IDWOHD >= 12)
.Join(oProductDB.TBLResources,
CTBLCustoms => new
{
CTBLCustoms.IDResource
},
TBLResources => new
{
TBLResources.IDResource
},
(CTBLCustoms, TBLResources) => new
{
IDCustom = CTBLCustoms.IDCustom,
Descricao = CTBLCustoms.Descricao,
IDWOHD = CTBLCustoms.IDWOHD,
IDResource = CTBLCustoms.IDResource,
ResourceCode = TBLResources.Code
})).ToList();
gvwDados.DataSource = oConsulta;
}
}
I added the AsEnumerable() as suggested, but I had to add the ToList() at the end so I could databind it to the DataGridView.
You can't do this in L2E. You could bring this into object space with AsEnumerable(), and it would work, but possibly be inefficient.
Merging the ObjectContexts is possible, and would work, but would need to be done manually.