Entity Framework Stored Procedure Call Significantly Slower - entity-framework

Any reason why I'm getting significantly lower performance using Entity Framework? Both versions of the code below are accomplishing the exact same goal.
The Excel document has 1700 records that are being inserted into a SQL Azure database.
Using Entity Framework Execution Time: 4:55
foreach (IXLRow row in wb.Worksheet(1).RowsUsed())
{
dbContext.InsertQuestion(row.Cell("B").Value.ToString().Trim(), row.Cell("A").Value.ToString().Trim(), row.Cell("C").Value.ToString().Trim(), row.Cell("D").Value.ToString().Trim(), null);
}
Non-Entity Framework Execution Time: 1:54
foreach (IXLRow row in wb.Worksheet(1).RowsUsed())
{
using (SqlCommand cmd = new SqlCommand("InsertQuestion", conn))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add(new SqlParameter("#CategoryName", row.Cell("B").Value.ToString().Trim()));
cmd.Parameters.Add(new SqlParameter("#TypeName", row.Cell("A").Value.ToString().Trim()));
cmd.Parameters.Add(new SqlParameter("#Text", row.Cell("C").Value.ToString().Trim()));
cmd.Parameters.Add(new SqlParameter("#Answer", row.Cell("D").Value.ToString().Trim()));
cmd.ExecuteNonQuery();
}
}

First, create a data type on your database (like the database has VARCHAR and INT types you can create your own TYPE.)
CREATE TYPE SurveyAnswerType AS TABLE
(
CategoryName VARCHAR(255),
TypeName VARCHAR(255),
[Text] VARCHAR(255),
Answer VARCHAR(255)
)
GO
Second, use the new type as a parameter to a stored proc
CREATE PROCEDURE InsertSurveyAnswers
#Answers SurveyAnswerType READONLY
AS
INSERT INTO Answers
(CategoryName, TypeName, [Text], Answer)
SELECT CategoryName, TypeName, [Text], Answer
FROM #Answers
GO
Third, call the stored proc from C# code by loading a table that looks like your new type:
DataTable tAnswers = new DataTable();
tAnswers.Columns.Add("CategoryName", typeof(String));
tAnswers.Columns.Add("TypeName", typeof(String));
tAnswers.Columns.Add("Text", typeof(String));
tAnswers.Columns.Add("Answer", typeof(String));
var rows = wb.Worksheet(1).RowsUsed();
var totalRows = rows.Count();
int index = 0;
int bufferSize;
using (SqlCommand cmd = new SqlCommand("InsertAnswers", conn))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add(new SqlParameter("#SurveyAnswerType",
SqlDbType.Structured));
while(index < totalRows)
{
int bufferSize = Math.Min(200, totalRows - index);
foreach (index = index; index < bufferSize; index++)
{
var newRow = tAnswerstAnswers.NewRow();
newRow[0] = rows[index].Cell("A").Value.ToString().Trim();
newRow[1] = row[index].Cell("B").Value.ToString().Trim();
newRow[2] = row[index].Cell("C").Value.ToString().Trim();
newRow[3] = row[index].Cell("D").Value.ToString().Trim();
tAnswerstAnswers.AddRow(newRow);
}
cmd.Parameters["#SurveyAnswerType"].Value = tAnswers;
cmd.ExecuteNonQuery();
}
}
Things to note:
The SqlParameter used SqlDbType.Structured.
The connection is only opened and closed / destroyed once, not once for
each row.
You should play with the buffer size to see what is the best size.
You will have to come up with your own entity framework version of
this - but I suspect that there is overhead with each call that you
will minimize with this pattern.

Related

Linq query in stored procedure result set Ado.Net

I have a stored procedure with multiple joins, pulling all the data into a resultset I.e in dataset, now I want to write a linq query over it. How can I do that?
I am expecting:
IEnumerable<SomeType> result;
[where I need to know how the Properties of SomeType are defined.]
This is what I have tried but it does not look efficient.
SqlCommand cmd = new SqlCommand("Select top 10 * from trade");
cmd.Connection = con;
if (con.State != ConnectionState.Open)
{
con.Open();
}
SqlDataReader dr = cmd.ExecuteReader();
DataTable dt = new DataTable();
dt.Load(dr);
var result = dt.AsEnumerable();
string valresukir = string.Empty;
var sortResult = result.OrderBy(x => Convert.ToInt32(x["trade_num"]) > 12);
string valuedata = string.Empty;
foreach (var i in sortResult)
{
valuedata += i["trade_num"].ToString();
}
to can write linq query on data table like
var data= from dataRow in dt.AsEnumerable()
where dataRow.Field<int>("trade_num") > 12
select dataRow
if trade_num is integer . Take it as a example and add your conditions accordingly.
Hope it will help you.

EntityFrameWork and TableValued Parameter

I'm trying to call a stored procedure from EntityFramework which uses Table-value parameter.
But when I try to do function import I keep getting a warning message saying -
The function 'InsertPerson' has a parameter 'InsertPerson_TVP' at
parameter index 0 that has a data type 'table type' which is currently
not supported for the target .NET Framework version. The function was
excluded.
I did a initial search here and found few posts which says It's possible in EntityFrameWork with some work arounds and few saying it's not supported in current versions.
Does any one know a better approach or solution for this problem?
I ended up doing this, Please note we are working on EF DataContext(not ObjectContext)
Executing a Stored procedure with output parameter
using (DataContext context = new DataContext())
{
////Create table value parameter
DataTable dt = new DataTable();
dt.Columns.Add("Displayname");
dt.Columns.Add("FirstName");
dt.Columns.Add("LastName");
dt.Columns.Add("TimeStamp");
DataRow dr = dt.NewRow();
dr["Displayname"] = "DisplayName";
dr["FirstName"] = "FirstName";
dr["LastName"] ="LastName";
dr["TimeStamp"] = "TimeStamp";
dt.Rows.Add(dr);
////Use DbType.Structured for TVP
var userdetails = new SqlParameter("UserDetails", SqlDbType.Structured);
userdetails.Value = dt;
userdetails.TypeName = "UserType";
////Parameter for SP output
var result = new SqlParameter("ResultList", SqlDbType.NVarChar, 4000);
result.Direction = ParameterDirection.Output;
context.Database.ExecuteSqlCommand("EXEC UserImport #UserDetails, #ResultList OUTPUT", userdetails, result);
return result == null ? string.Empty : result.Value.ToString();
}
My Table-Value-Parameter (UDT Table) script looks like this:
CREATE TYPE [dbo].[UserType] AS TABLE (
[DisplayName] NVARCHAR (256) NULL,
[FirstName] NVARCHAR (256) NULL,
[LastName] NVARCHAR (256) NULL,
[TimeStamp] DATETIME NULL
)
And my store procedure begins like
CREATE PROCEDURE UserImport
-- Add the parameters for the stored procedure here
#UserDetails UserType Readonly,
#ResultList NVARCHAR(MAX) output
AS
For Stored procedure without output parameter we don't need any ouput parameter added/passed to SP.
Hope it helps some one.
Perhaps we could also consider the SqlQuery method:
[Invoke]
public SomeResultType GetResult(List<int> someIdList)
{
var idTbl = new DataTable();
idTbl.Columns.Add("Some_ID");
someIdList.ForEach(id => idTbl.Rows.Add(id));
var idParam = new SqlParamter("SomeParamName", SqlDbType.Structured);
idParam.TypeName = "SomeTBVType";
idParam.Value = idTbl;
// Return type will be IEnumerable<T>
var result = DbContext.Database.SqlQuery<SomeResultType>("EXEC SomeSPName, #SomeParamName", idParam);
// We can enumerate the result...
var enu = result.GetEnumerator();
if (!enu.MoveNext()) return null;
return enu.Current;
}
var detailTbl = new DataTable();
detailTbl.Columns.Add("DetailID");
detailTbl.Columns.Add("Qty");
txnDetails.ForEach(detail => detailTbl.Rows.Add(detail.DetailID, detail.Qty));
var userIdParam = new System.Data.SqlClient.SqlParameter("#UserID", SqlDbType.Int);
userIdParam.Value = 1;
var detailParam = new System.Data.SqlClient.SqlParameter("#Details", SqlDbType.Structured);
detailParam.TypeName = "DetailUpdate";
detailParam.Value = detailTbl;
var txnTypeParam = new System.Data.SqlClient.SqlParameter("#TransactionType", SqlDbType.VarChar);
txnTypeParam.Value = txnType;
var result = await db.Database.ExecuteSqlCommandAsync("MySP #UserID, #Details, #TransactionType", userIdParam, detailParam, txnTypeParam);
if(result >= 0)
return StatusCode(HttpStatusCode.OK);
else
return StatusCode(HttpStatusCode.InternalServerError);

get primary key of last inserted record with JPA

I've been using JPA to insert entities into a database but I've run up against a problem where I need to do an insert and get the primary key of the record last inserted.
Using PostgreSQL I would use an INSERT RETURNING statement which would return the record id, but with an entity manager doing all this, the only way I know is to use SELECT CURRVAL.
So the problem becomes, I have several data sources sending data into a message driven bean (usually 10-100 messages at once from each source) via OpenMQ and inside this MDB I persists this to PostgreSQL via the entity manager. It's at this point I think there will be a "race condition like" effect of having so many inserts that I won't necessarily get the last record id using SELECT CURRVAL.
My MDB persists 3 entity beans via an entity manager like below.
Any help on how to better do this much appreciated.
public void onMessage(Message msg) {
Integer agPK = 0;
Integer scanPK = 0;
Integer lookPK = 0;
Iterator iter = null;
List<Ag> agKeys = null;
List<Scan> scanKeys = null;
try {
iag = (IAgBean) (new InitialContext()).lookup(
"java:comp/env/ejb/AgBean");
TextMessage tmsg = (TextMessage) msg;
// insert this into table only if doesn't exists
Ag ag = new Ag(msg.getStringProperty("name"));
agKeys = (List) (iag.getPKs(ag));
iter = agKeys.iterator();
if (iter.hasNext()) {
agPK = ((Ag) iter.next()).getId();
}
else {
// no PK found so not in dbase, insert new
iag.addAg(ag);
agKeys = (List) (iag.getPKs(ag));
iter = agKeys.iterator();
if (iter.hasNext()) {
agPK = ((Ag) iter.next()).getId();
}
}
// insert this into table always
iscan = (IScanBean) (new InitialContext()).lookup(
"java:comp/env/ejb/ScanBean");
Scan scan = new Scan();
scan.setName(msg.getStringProperty("name"));
scan.setCode(msg.getIntProperty("code"));
iscan.addScan(scan);
scanKeys = (List) iscan.getPKs(scan);
iter = scanKeys.iterator();
if (iter.hasNext()) {
scanPK = ((Scan) iter.next()).getId();
}
// insert into this table the two primary keys above
ilook = (ILookBean) (new InitialContext()).lookup(
"java:comp/env/ejb/LookBean");
Look look = new Look();
if (agPK.intValue() != 0 && scanPK.intValue() != 0) {
look.setAgId(agPK);
look.setScanId(scanPK);
ilook.addLook(look);
}
// ...
The JPA spec requires that after persist, the entity be populated with a valid ID if an ID generation strategy is being used. You don't have to do anything.

Returning a DataTable using Entity Framework ExecuteStoreQuery

I am working with a system that has many stored procedures that need to be displayed. Creating entities for each of my objects is not practical.
Is it possible and how would I return a DataTable using ExecuteStoreQuery ?
public ObjectResult<DataTable> MethodName(string fileSetName) {
using (var dataContext = new DataContext(_connectionString))
{
var returnDataTable = ((IObjectContextAdapter)dataContext).ObjectContext.ExecuteStoreQuery<DataTable>("SP_NAME","SP_PARAM");
return returnDataTable;
}
Yes it's possible, but it should be used for just dynamic result-set or raw SQL.
public DataTable ExecuteStoreQuery(string commandText, params Object[] parameters)
{
DataTable retVal = new DataTable();
retVal = context.ExecuteStoreQuery<DataTable>(commandText, parameters).FirstOrDefault();
return retVal;
}
Edit: It's better to use classical ADO.NET to get the data model rather than using Entity Framework because most probably you cannot use DataTable even if you can run the method: context.ExecuteStoreQuery<DataTable>(commandText, parameters).FirstOrDefault();
ADO.NET Example:
public DataSet GetResultReport(int questionId)
{
DataSet retVal = new DataSet();
EntityConnection entityConn = (EntityConnection)context.Connection;
SqlConnection sqlConn = (SqlConnection)entityConn.StoreConnection;
SqlCommand cmdReport = new SqlCommand([YourSpName], sqlConn);
SqlDataAdapter daReport = new SqlDataAdapter(cmdReport);
using (cmdReport)
{
SqlParameter questionIdPrm = new SqlParameter("QuestionId", questionId);
cmdReport.CommandType = CommandType.StoredProcedure;
cmdReport.Parameters.Add(questionIdPrm);
daReport.Fill(retVal);
}
return retVal;
}
No, I don't think that'll work - Entity Framework is geared towards returning entities and isn't meant to return DataTable objects.
If you need DataTable objects, use straight ADO.NET instead.
This method uses the connection string from the entity framework to establish an ADO.NET connection, to a MySQL database in this example.
using MySql.Data.MySqlClient;
public DataSet GetReportSummary( int RecordID )
{
var context = new catalogEntities();
DataSet ds = new DataSet();
using ( MySqlConnection connection = new MySqlConnection( context.Database.Connection.ConnectionString ) )
{
using ( MySqlCommand cmd = new MySqlCommand( "ReportSummary", connection ) )
{
MySqlDataAdapter adapter = new MySqlDataAdapter( cmd );
adapter.SelectCommand.CommandType = CommandType.StoredProcedure;
adapter.SelectCommand.Parameters.Add( new MySqlParameter( "#ID", RecordID ) );
adapter.Fill( ds );
}
}
return ds;
}
Yes it can easily be done like this:
var table = new DataTable();
using (var ctx = new SomeContext())
{
var cmd = ctx.Database.Connection.CreateCommand();
cmd.CommandText = "Select Col1, Col2 from SomeTable";
cmd.Connection.Open();
table.Load(cmd.ExecuteReader());
}
By the rule, you shouldn't use a DataSet inside a EF application. But, if you really need to (for instance, to feed a report), that solution should work (it's EF 6 code):
DataSet GetDataSet(string sql, CommandType commandType, Dictionary<string, Object> parameters)
{
// creates resulting dataset
var result = new DataSet();
// creates a data access context (DbContext descendant)
using (var context = new MyDbContext())
{
// creates a Command
var cmd = context.Database.Connection.CreateCommand();
cmd.CommandType = commandType;
cmd.CommandText = sql;
// adds all parameters
foreach (var pr in parameters)
{
var p = cmd.CreateParameter();
p.ParameterName = pr.Key;
p.Value = pr.Value;
cmd.Parameters.Add(p);
}
try
{
// executes
context.Database.Connection.Open();
var reader = cmd.ExecuteReader();
// loop through all resultsets (considering that it's possible to have more than one)
do
{
// loads the DataTable (schema will be fetch automatically)
var tb = new DataTable();
tb.Load(reader);
result.Tables.Add(tb);
} while (!reader.IsClosed);
}
finally
{
// closes the connection
context.Database.Connection.Close();
}
}
// returns the DataSet
return result;
}
In my Entity Framework based solution I need to replace one of my Linq queries with sql - for efficiency reasons.
Also I want my results in a DataTable from one stored procedure so that I could create a table value parameter to pass into a second stored procedure. So:
I'm using sql
I don't want a DataSet
Iterating an IEnumerable probably isn't going to cut it - for efficiency reasons
Also, I am using EF6, so I would prefer DbContext.SqlQuery over ObjectContext.ExecuteStoreQuery as the original poster requested.
However, I found that this just didn't work:
_Context.Database.SqlQuery<DataTable>(sql, parameters).FirstOrDefault();
This is my solution. It returns a DataTable that is fetched using an ADO.NET SqlDataReader - which I believe is faster than a SqlDataAdapter on read-only data. It doesn't strictly answer the question because it uses ADO.Net, but it shows how to do that after getting a hold of the connection from the DbContext
protected DataTable GetDataTable(string sql, params object[] parameters)
{
//didn't work - table had no columns or rows
//return Context.Database.SqlQuery<DataTable>(sql, parameters).FirstOrDefault();
DataTable result = new DataTable();
SqlConnection conn = Context.Database.Connection as SqlConnection;
if(conn == null)
{
throw new InvalidCastException("SqlConnection is invalid for this database");
}
using (SqlCommand cmd = new SqlCommand(sql, conn))
{
cmd.Parameters.AddRange(parameters);
conn.Open();
using (SqlDataReader reader = cmd.ExecuteReader())
{
result.Load(reader);
}
return result;
}
}
The easiest way to return a DataTable using the EntityFramework is to do the following:
MetaTable metaTable = Global.DefaultModel.GetTable("Your EntitySetName");
For example:
MetaTable metaTable = Global.DefaultModel.GetTable("Employees");
Maybe your stored procedure could return a complex type?
http://blogs.msdn.com/b/somasegar/archive/2010/01/11/entity-framework-in-net-4.aspx

Postgres function(via npgsql) call to ExecuteNonQuery returns -1 as result for rows affected

I have a simple function that just inserts the parameter values provided to it into columns in a table.
When I run the function via the ExecuteNonQuery() method on the command object I always get -1, even if the insert took place.
If i run the same query as a Text command it gives be the correct result of 1.
I'm new to postgresql/npgsql. Is there trick to making the function feed back the number of rows affected? Something like "set nocount off" in SQL Server?
EDIT:
The code I am using: (with npgsql 2.0.11)
var connStr = #"Server=127.0.0.1;Port=5432;User Id=postgres;Password=***;Database=Test;";
using (var conn = new NpgsqlConnection(connStr)) {
conn.Open();
using (var cmd = conn.CreateCommand())
{
cmd.CommandText = "insert_something";
cmd.CommandType = CommandType.StoredProcedure;
NpgsqlCommandBuilder.DeriveParameters(cmd);
cmd.Parameters["_id"].Value = 1;
cmd.Parameters["_val"].Value = 2;
var rowsAffected = cmd.ExecuteNonQuery();
Console.WriteLine(rowsAffected);
}
}
I haven't used Npgsql, but the documentation has an example with
the following code:
NpgsqlCommand command = new NpgsqlCommand("insert into table1 values(1, 1)", conn);
Int32 rowsaffected;
try
{
rowsaffected = command.ExecuteNonQuery();
}
If you are talking about some PostgreSQL function like this:
CREATE FUNCTION myinsert(integer) RETURNS void
LANGUAGE 'sql' AS 'INSERT INTO mytable VALUES ($1)';
and you are doing something like SELECT myinsert(1);, you can't get the number of affected rows, because you are running a SELECT and what the function does internally is opaque to you.
For ExecuteNonQuery() to get the number of rows affected, your postgresql function should return just that. An example would be:
CREATE OR REPLACE FUNCTION insert_something(_id int, _val int)
RETURNS int as $$
DECLARE count int;
BEGIN
INSERT INTO some_table (id, value) VALUES(_id, _val);
GET DIAGNOSTICS count = ROW_COUNT;
RETURN count;
END;
$$ language plpgsql;
Now if you call ExecuteNonQuery() from your code, you should get the inserted rows count.
Here's how I have implemented and its working for me without any issues:
Add an OUT Parameter to your function.
CREATE OR REPLACE FUNCTION schema.UpdateSomeValue(
IN par_updateId text,
OUT par_returnvalue INT4)
BEGIN
UPDATE schema.TableToUpdate
SET status = 1
WHERE ID = par_updateId;
GET DIAGNOSTICS par_returnvalue = ROW_COUNT;
END;
Now on the C# side
private int UpdateSomeValue()
{
var connStr = #"Server=127.0.0.1;Port=5432;UserId=postgres;Password=***;Database=Test;";
int result = -1;
using (var conn = new NpgsqlConnection(connStr)) {
conn.Open();
using (var cmd = conn.CreateCommand())
{
cmd.CommandText = "shema.UpdateSomeValue";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("par_updateId",NpgsqlTypes.NpgsqlDbType.INT, 1);
NpgsqlParameter outParm = new NpgsqlParameter("par_returnvalue",
NpgsqlTypes.NpgsqlDbType.Integer)
{
Direction = ParameterDirection.Output
};
cmd.ExecuteNonQuery();
result = Convert.ToInt32(outParm.Value);
}
}
return result;
}
PS : I tried the just ExecuteNonQuery() approach but it was always returning -1 for me (as if no rows has been affected NpgsqlDocumentation) Using above way I was able to capture the Number of rows affected from the PostgreSQL function and with OUT parameter it was easy to catch the affected rows on the client side.