I need to know the correct way to handle SQL Injection when using the FromSQL command.
At runtime, I need to dynamically create a where statement. So I am using FromSql to create the SQL command. Now I know that using use string interpolation is the way to go. However, I need to step through a list of "Where Parameters" to generate the command. Simple enough to do;
foreach (var x in wp)
{
if (!string.IsNullOrEmpty(results))
results = $"{results} and {x.Field} = {x.Value}";
if (string.IsNullOrEmpty(results))
results = $"where {x.Field} = {x.Value}";
}
Problem is that this return a simple string and would not be string interpolation. How can I do this correctly?
Entityframework will parameterize your queries if you put it in the following format:
db.something.FromSql("SELECT * FROM yourTable WHERE AuthorId = {0}", id)
Is x.Field a form field that has a fixed number of possibilities? i.e. title, firstname etc. If so then something like the following:
var sqlstring = new StringBuilder();
var sqlp = new List<SqlParameter>();
var i = 0;
foreach (var x in wp)
{
var param = "#param" + i.ToString();
if (i!=0)
{
sqlstring.Append($" AND {x.Field} = " + param);
sqlp.Add(new SqlParameter(param, x.Value));
}
if (i==0)
{
sqlstring.Append($"WHERE {x.Field} = " + " #param" + i.ToString());
sqlp.Add(new SqlParameter(param, x.Value));
}
i++;
}
You'd then need to do something like this:
db.something.FromSql(sqlstring.ToString(), sqlp.ToArray())
Might be a better/cleaner way but that should work.
My solution to this problem is a VS extension, QueryFirst. QueryFirst generates a C# wrapper for sql that lives in a .sql file. As such, parameters are the only way to get data into your query and SQL injection is near impossible. There are numerous other advantages: you edit your sql in a real environment, it's constantly validated against your db, and using your query in your code is very simple.
Related
I have some word templates(maybe thousands). Each template has merge fields which will be filled from database. I don`t like writing separate code for every template and then build the application and deploy it whenever a template is changed or a field on the template is added!
Instead, I'm trying to define all merge fields in a separate xml file and for each field I want to write the "query" which will be called when needed. EX:
mergefield1 will call query "Case.Parties.FirstOrDefault.NameEn"
mergefield2 will call query "Case.CaseNumber"
mergefield3 will call query "Case.Documents.FirstOrDefault.DocumentContent.DocumentType"
Etc,
So, for a particular template I scan its merge fields, and for each merge field I take it`s "query definition" and make that request to database using EntityFramework and LINQ. Ex. it works for these queries: "TimeSlots.FirstOrDefault.StartDateTime" or
"Case.CaseNumber"
This will be an engine which will generate word documents and fill it with merge fields from xml. In addition, it will work for any new template or new merge field.
Now, I have worked a version using reflection.
public string GetColumnValueByObjectByName(Expression<Func<TEntity, bool>> filter = null, string objectName = "", string dllName = "", string objectID = "", string propertyName = "")
{
string objectDllName = objectName + ", " + dllName;
Type type = Type.GetType(objectDllName);
Guid oID = new Guid(objectID);
dynamic Entity = context.Set(type).Find(oID); // get Object by Type and ObjectID
string value = ""; //the value which will be filled with data from database
IEnumerable<string> linqMethods = typeof(System.Linq.Enumerable).GetMethods(BindingFlags.Static | BindingFlags.Public).Select(s => s.Name).ToList(); //get all linq methods and save them as list of strings
if (propertyName.Contains('.'))
{
string[] properies = propertyName.Split('.');
dynamic object1 = Entity;
IEnumerable<dynamic> Child = new List<dynamic>();
for (int i = 0; i < properies.Length; i++)
{
if (i < properies.Length - 1 && linqMethods.Contains(properies[i + 1]))
{
Child = type.GetProperty(properies[i]).GetValue(object1, null);
}
else if (linqMethods.Contains(properies[i]))
{
object1 = Child.Cast<object>().FirstOrDefault(); //for now works only with FirstOrDefault - Later it will be changed to work with ToList or other linq methods
type = object1.GetType();
}
else
{
if (linqMethods.Contains(properies[i]))
{
object1 = type.GetProperty(properies[i + 1]).GetValue(object1, null);
}
else
{
object1 = type.GetProperty(properies[i]).GetValue(object1, null);
}
type = object1.GetType();
}
}
value = object1.ToString(); //.StartDateTime.ToString();
}
return value;
}
I`m not sure if this is the best approach. Does anyone have a better suggestion, or maybe someone has already done something like this?
To shorten it: The idea is to make generic linq queries to database from a string like: "Case.Parties.FirstOrDefault.NameEn".
Your approach is very good. I have no doubt that it already works.
Another approach is using Expression Tree like #Egorikas have suggested.
Disclaimer: I'm the owner of the project Eval-Expression.NET
In short, this library allows you to evaluate almost any C# code at runtime (What you exactly want to do).
I would suggest you use my library instead. To keep the code:
More readable
Easier to support
Add some flexibility
Example
public string GetColumnValueByObjectByName(Expression<Func<TEntity, bool>> filter = null, string objectName = "", string dllName = "", string objectID = "", string propertyName = "")
{
string objectDllName = objectName + ", " + dllName;
Type type = Type.GetType(objectDllName);
Guid oID = new Guid(objectID);
object Entity = context.Set(type).Find(oID); // get Object by Type and ObjectID
var value = Eval.Execute("x." + propertyName, new { x = entity });
return value.ToString();
}
The library also allow you to use dynamic string with IQueryable
Wiki: LINQ-Dynamic
I am using Npgsql for postgresql in C++/CLI. So, the problem is, I have a db on my computer, and I am trying to select some of data from it's "movies" table. I already entered some data inside it, so I know that it has some data. But when I try to select some of them, answer to my query is empty. My code is like below:
public: string* SelectData(string* torrent)
{
conn->Open();
String ^ query = "SELECT title, director, actors, genre FROM movies";
Npgsql::NpgsqlCommand ^ command = gcnew NpgsqlCommand(query, conn);
try{
Npgsql::NpgsqlDataReader ^ dr = command->ExecuteReader();
for (int i = 0; i < N_TORRENT; i++)
{
if(dr->Read())
{
string std1 = toStandardString((String^)dr[0]);
string std2 = toStandardString((String^)dr[1]);
string std3 = toStandardString((String^)dr[2]);
string std4 = toStandardString((String^)dr[3]);
torrent[i] = std1 + " " + std2 + " " + std3 + " " + std4;
}
}
return torrent;
}
finally{
conn->Close();
}
}
(For the ones who will look for this question's answer)
Problem solved when I changed my query and look for the "title" column that are not empty. But this is ridiculus, so I beleive the problem was about pgAdmin. Because my insert query was not working either, but I added "rowseffected" variable and it shows the effected row's number and looks like it is working. So the problem is probably about the pgAdmin.
What is the best way to deal with batch updates using (Entity Framework) EF5?
I have 2 particular cases I'm interested in:
Updating a field (e.g. UpdateDate) for a list (List) of between 100 and 100.000 Id's, which the primary key. Calling each update separately seem to be to much overhead and takes a long time.
Inserting many, also between the 100 and 100.000, of the same objects (e.g. Users) in a single go.
Any good advice?
There are two open source projects allowing this: EntityFramework.Extended and Entity Framework Extensions. You can also check discussion about bulk updates on EF's codeplex site.
Inserting 100k records through EF is in the first place wrong application architecture. You should choose different lightweight technology for data imports. Even EF's internal operation with such big record set will cost you a lot of processing time. There is currently no solution for batch inserts for EF but there is broad discussion about this feature on EF's code plex site.
I see the following options:
1 . The simplest way - create your SQL request by hands and execute through ObjectContext.ExecuteStoreCommand
context.ExecuteStoreCommand("UPDATE TABLE SET FIELD1 = {0} WHERE FIELD2 = {1}", value1, value2);
2 . Use EntityFramework.Extended
context.Tasks.Update(
t => t.StatusId == 1,
t => new Task {StatusId = 2});
3 . Make your own extension for EF. There is an article Bulk Delete where this goal was achieved by inheriting ObjectContext class. It's worth to take a look. Bulk insert/update can be implemented in the same way.
You may not want to hear it, but your best option is to not use EF for bulk operations. For updating a field across a table of records, use an Update statement in the database (possibly called through a stored proc mapped to an EF Function). You can also use the Context.ExecuteStoreQuery method to issue an Update statement to the database.
For massive inserts, your best bet is to use Bulk Copy or SSIS. EF will require a separate hit to the database for each row being inserted.
Bulk inserts should be done using the SqlBulkCopy class. Please see pre-existing StackOverflow Q&A on integrating the two: SqlBulkCopy and Entity Framework
SqlBulkCopy is a lot more user-friendly than bcp (Bulk Copy command-line utility) or even OPEN ROWSET.
Here's what I've done successfully:
private void BulkUpdate()
{
var oc = ((IObjectContextAdapter)_dbContext).ObjectContext;
var updateQuery = myIQueryable.ToString(); // This MUST be above the call to get the parameters.
var updateParams = GetSqlParametersForIQueryable(updateQuery).ToArray();
var updateSql = $#"UPDATE dbo.myTable
SET col1 = x.alias2
FROM dbo.myTable
JOIN ({updateQuery}) x(alias1, alias2) ON x.alias1 = dbo.myTable.Id";
oc.ExecuteStoreCommand(updateSql, updateParams);
}
private void BulkInsert()
{
var oc = ((IObjectContextAdapter)_dbContext).ObjectContext;
var insertQuery = myIQueryable.ToString(); // This MUST be above the call to get the parameters.
var insertParams = GetSqlParametersForIQueryable(insertQuery).ToArray();
var insertSql = $#"INSERT INTO dbo.myTable (col1, col2)
SELECT x.alias1, x.alias2
FROM ({insertQuery}) x(alias1, alias2)";
oc.ExecuteStoreCommand(insertSql, insertParams.ToArray());
}
private static IEnumerable<SqlParameter> GetSqlParametersForIQueryable<T>(IQueryable<T> queryable)
{
var objectQuery = GetObjectQueryFromIQueryable(queryable);
return objectQuery.Parameters.Select(x => new SqlParameter(x.Name, x.Value));
}
private static ObjectQuery<T> GetObjectQueryFromIQueryable<T>(IQueryable<T> queryable)
{
var dbQuery = (DbQuery<T>)queryable;
var iqProp = dbQuery.GetType().GetProperty("InternalQuery", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
var iq = iqProp.GetValue(dbQuery, null);
var oqProp = iq.GetType().GetProperty("ObjectQuery", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
return (ObjectQuery<T>)oqProp.GetValue(iq, null);
}
public static bool BulkDelete(string tableName, string columnName, List<object> val)
{
bool ret = true;
var max = 2000;
var pages = Math.Ceiling((double)val.Count / max);
for (int i = 0; i < pages; i++)
{
var count = max;
if (i == pages - 1) { count = val.Count % max; }
var args = val.GetRange(i * max, count);
var cond = string.Join("", args.Select((t, index) => $",#p{index}")).Substring(1);
var sql = $"DELETE FROM {tableName} WHERE {columnName} IN ({cond}) ";
ret &= Db.ExecuteSqlCommand(sql, args.ToArray()) > 0;
}
return ret;
}
I agree with the accepted answer that ef is probably the wrong technology for bulk inserts.
However, I think it's worth having a look at EntityFramework.BulkInsert.
i have some methods like:
public static string ToOtherFormat (this string inp)
{
// some code to change inp
return inp;
}
and in my select i want to have code like this:
var DetailMembers = db.TB_Members
.Where(x=> x.FName == obj.ToOtherFormat())
.Select( x=> new { name = (x.FName.ToOtherFormat() + " " + x.LName) , x.ActCode });
i try and just have error. is it possible?
thanks!
i receive this error in simple convert to integer
LINQ to Entities does not recognize the method 'Int32 ToInt32(System.String)' method, and this method cannot be translated into a store expression.
with this code
.Where(x => x.MemberID == Convert.ToInt32(Hmemid.Hash_two_Decrypt())
Looks like you are querying against the database. Your current query will get translated into SQL query and since SQL doesn't recognize your function that is why you get error.
You may get the data from the tables using a query without that function and then later do the formatting on the result set.
i found it on use .AsEnumerable() method like:
var DetailMembers = db.TB_Members.AsEnumerable()
.Where(x=> x.FName == obj.ToOtherFormat())
.Select( x=> new { name = (x.FName.ToOtherFormat() + " " + x.LName) , x.ActCode });
Is it possible to implement batching of multiple stored procedure calls (doing updates/deletes) in ADO.NET without resorting to DataAdapters?
You could try using System.Data.SqlClient.SqlCommandSet. It's actually internal, but Ayende made a wrapper to make it public.
Code is currently hosted in sourceforge.
You're SQL text can contain multiple commands. If you return multiple result sets, then you can use a DataReader and use the NextResult function. What I often do is store the SQL to execute as an Embedded Resource, then load that text. If it contains parameters, then set the parameters just like you would normally.
For example, I have a file:
UPDATE dbo.QuotePricedLineItem
SET fkQuoteVendorLineSet = NULL
FROM dbo.QuotePricedLineItem qpli
INNER JOIN dbo.QuoteLineItem qli ON qpli.Id = qli.Id
WHERE qli.fkQuote = #quoteId AND qpli.fkQuoteVendorLineSet = #ciscoConfigId
DELETE CiscoQuoteLineItem
FROM CiscoQuoteLineItem cqli
INNER JOIN QuoteLineItem qli ON cqli.Id = qli.Id
WHERE qli.fkQuote = #quoteId AND cqli.fkCiscoQuoteVendorLineSet = #ciscoConfigId
that I execute as such:
using (SqlConnection conn = DbUtils.CreateConnection() as SqlConnection)
{
conn.Open();
SqlCommand cmd = conn.CreateCommand();
cmd.CommandText = MvcApplication.GetResource("SQL.DemoteCiscoQuoteLineItems.sql");
cmd.Parameters.AddWithValue("#quoteId", q.Id);
cmd.Parameters.AddWithValue("#ciscoConfigId", configSetId);
cmd.ExecuteNonQuery();
}
Note that MvcApplication.GetResource is not a built in function - it's one you have to write... here's mine:
public static string GetResource(string p)
{
Stream s = Assembly.GetExecutingAssembly().GetManifestResourceStream("CortexQuoting.Res." + p);
if (s == null) return null;
StreamReader sw = new StreamReader(s);
string ss = sw.ReadToEnd();
sw.Close();
return ss;
}