How do I view the SQL generated by the Entity Framework? - entity-framework

How do I view the SQL generated by entity framework ?
(In my particular case I'm using the mysql provider - if it matters)

For those using Entity Framework 6 and up, if you want to view the output SQL in Visual Studio (like I did) you have to use the new logging/interception functionality.
Adding the following line will spit out the generated SQL (along with additional execution-related details) in the Visual Studio output panel:
using (MyDatabaseEntities context = new MyDatabaseEntities())
{
context.Database.Log = s => System.Diagnostics.Debug.WriteLine(s);
// query the database using EF here.
}
More information about logging in EF6 in this nifty blog series: http://blog.oneunicorn.com/2013/05/08/ef6-sql-logging-part-1-simple-logging/
Note: Make sure you are running your project in DEBUG mode.

You can do the following:
IQueryable query = from x in appEntities
where x.id == 32
select x;
var sql = ((System.Data.Objects.ObjectQuery)query).ToTraceString();
or in EF6:
var sql = ((System.Data.Entity.Core.Objects.ObjectQuery)query)
.ToTraceString();
or in EF6.3+:
var sql = ((dynamic)flooringStoresProducts).Sql;
That will give you the SQL that was generated.

Starting with EF6.1 you can use Interceptors to register a database logger.
See chapters "Interceptors" and "Logging Database Operations" to a File here
<configuration>
<entityFramework>
<interceptors>
<interceptor type="System.Data.Entity.Infrastructure.Interception.DatabaseLogger, EntityFramework">
<parameters>
<parameter value="C:\Temp\LogOutput.txt"/>
<parameter value="true" type="System.Boolean"/>
</parameters>
</interceptor>
</interceptors>
</entityFramework>
</configuration>

If you are using a DbContext, you can do the following to get the SQL:
var result = from i in myContext.appEntities
select new Model
{
field = i.stuff,
};
var sql = result.ToString();

EF Core 5.0+
This loooong-awaited feature is available in EF Core 5.0! This is from the weekly status updates:
var query = context.Set<Customer>().Where(c => c.City == city);
Console.WriteLine(query.ToQueryString())
results in this output when using the SQL Server database provider:
DECLARE p0 nvarchar(4000) = N'London';
SELECT [c].[CustomerID], [c].[Address], [c].[City], [c].[CompanyName],
[c].[ContactName], [c].[ContactTitle], [c].[Country], [c].[Fax], [c].[Phone],
[c].[PostalCode], [c].[Region]
FROM [Customers] AS [c]
WHERE [c].[City] = #__city_0
Notice that declarations for parameters of the correct type are also
included in the output. This allows copy/pasting to SQL Server
Management Studio, or similar tools, such that the query can be
executed for debugging/analysis.
woohoo!!!
(Note: You will need using Microsoft.EntityFrameworkCore;)

Applicable for EF 6.0 and above:
For those of you wanting to know more about the logging functionality and adding to the some of the answers already given.
Any command sent from the EF to the database can now be logged. To view the generated queries from EF 6.x, use the DBContext.Database.Log property
What Gets Logged
- SQL for all different kinds of commands. For example:
- Queries, including normal LINQ queries, eSQL queries, and raw queries from methods such as SqlQuery.
- Inserts, updates, and deletes generated as part of SaveChanges
- Relationship loading queries such as those generated by lazy loading
- Parameters
- Whether or not the command is being executed asynchronously
- A timestamp indicating when the command started executing
- Whether or not the command completed successfully, failed by throwing an exception, or, for async, was canceled
- Some indication of the result value
- The approximate amount of time it took to execute the command. Note that this is the time from sending the command to getting the result object back. It does not include time to read the results.
Example:
using (var context = new BlogContext())
{
context.Database.Log = Console.Write;
var blog = context.Blogs.First(b => b.Title == "One Unicorn");
blog.Posts.First().Title = "Green Eggs and Ham";
blog.Posts.Add(new Post { Title = "I do not like them!" });
context.SaveChangesAsync().Wait();
}
Output:
SELECT TOP (1)
[Extent1].[Id] AS [Id],
[Extent1].[Title] AS [Title]
FROM [dbo].[Blogs] AS [Extent1]
WHERE (N'One Unicorn' = [Extent1].[Title]) AND ([Extent1].[Title] IS NOT NULL)
-- Executing at 10/8/2013 10:55:41 AM -07:00
-- Completed in 4 ms with result: SqlDataReader
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Title] AS [Title],
[Extent1].[BlogId] AS [BlogId]
FROM [dbo].[Posts] AS [Extent1]
WHERE [Extent1].[BlogId] = #EntityKeyValue1
-- EntityKeyValue1: '1' (Type = Int32)
-- Executing at 10/8/2013 10:55:41 AM -07:00
-- Completed in 2 ms with result: SqlDataReader
UPDATE [dbo].[Posts]
SET [Title] = #0
WHERE ([Id] = #1)
-- #0: 'Green Eggs and Ham' (Type = String, Size = -1)
-- #1: '1' (Type = Int32)
-- Executing asynchronously at 10/8/2013 10:55:41 AM -07:00
-- Completed in 12 ms with result: 1
INSERT [dbo].[Posts]([Title], [BlogId])
VALUES (#0, #1)
SELECT [Id]
FROM [dbo].[Posts]
WHERE ##ROWCOUNT > 0 AND [Id] = scope_identity()
-- #0: 'I do not like them!' (Type = String, Size = -1)
-- #1: '1' (Type = Int32)
-- Executing asynchronously at 10/8/2013 10:55:41 AM -07:00
-- Completed in 2 ms with result: SqlDataReader
To log to an external file:
using (var context = new BlogContext())
{
using (var sqlLogFile = new StreamWriter("C:\\temp\\LogFile.txt"))
{
context.Database.Log = sqlLogFile.Write;
var blog = context.Blogs.First(b => b.Title == "One Unicorn");
blog.Posts.First().Title = "Green Eggs and Ham";
context.SaveChanges();
}
}
More info here: Logging and Intercepting Database Operations

You can do the following in EF 4.1:
var result = from x in appEntities
where x.id = 32
select x;
System.Diagnostics.Trace.WriteLine(result .ToString());
That will give you the SQL that was generated.

My answer addresses EF core. I reference this github issue, and the docs on configuring DbContext:
Simple
Override the OnConfiguring method of your DbContext class (YourCustomDbContext) as shown here to use a ConsoleLoggerProvider; your queries should log to the console:
public class YourCustomDbContext : DbContext
{
#region DefineLoggerFactory
public static readonly LoggerFactory MyLoggerFactory
= new LoggerFactory(new[] {new ConsoleLoggerProvider((_, __) => true, true)});
#endregion
#region RegisterLoggerFactory
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
=> optionsBuilder
.UseLoggerFactory(MyLoggerFactory); // Warning: Do not create a new ILoggerFactory instance each time
#endregion
}
Complex
This Complex case avoids overriding the DbContext OnConfiguring method. , which is discouraged in the docs: "This approach does not lend itself to testing, unless the tests target the full database."
This Complex case uses:
The IServiceCollection in Startup class ConfigureServices method
(instead of overriding the OnConfiguring method; the benefit is a looser coupling between the DbContext and the ILoggerProvider you want to use)
An implementation of ILoggerProvider (instead of using the ConsoleLoggerProvider implementation shown above; benefit is our implementation shows how we would log to File (I don't see a File Logging Provider shipped with EF Core))
Like this:
public class Startup
public void ConfigureServices(IServiceCollection services)
{
...
var lf = new LoggerFactory();
lf.AddProvider(new MyLoggerProvider());
services.AddDbContext<YOUR_DB_CONTEXT>(optionsBuilder => optionsBuilder
.UseSqlServer(connection_string)
//Using the LoggerFactory
.UseLoggerFactory(lf));
...
}
}
Here's the implementation of a MyLoggerProvider (and its MyLogger which appends its logs to a File you can configure; your EF Core queries will appear in the file.)
public class MyLoggerProvider : ILoggerProvider
{
public ILogger CreateLogger(string categoryName)
{
return new MyLogger();
}
public void Dispose()
{ }
private class MyLogger : ILogger
{
public bool IsEnabled(LogLevel logLevel)
{
return true;
}
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception exception, Func<TState, Exception, string> formatter)
{
File.AppendAllText(#"C:\temp\log.txt", formatter(state, exception));
Console.WriteLine(formatter(state, exception));
}
public IDisposable BeginScope<TState>(TState state)
{
return null;
}
}
}

To have the query always handy, without changing code
add this to your DbContext and check it on the output window in visual studio.
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
Database.Log = (query)=> Debug.Write(query);
}
Similar to #Matt Nibecker answer, but with this you do not have to add it in your current code, every time you need the query.

There are two ways:
To view the SQL that will be generated, simply call ToTraceString(). You can add it into your watch window and set a breakpoint to see what the query would be at any given point for any LINQ query.
You can attach a tracer to your SQL server of choice, which will show you the final query in all its gory detail. In the case of MySQL, the easiest way to trace the queries is simply to tail the query log with tail -f. You can learn more about MySQL's logging facilities in the official documentation. For SQL Server, the easiest way is to use the included SQL Server profiler.

I am doing integration test, and needed this to debug the generated SQL statement in Entity Framework Core 2.1, so I use DebugLoggerProvider or ConsoleLoggerProvider like so:
[Fact]
public async Task MyAwesomeTest
{
//setup log to debug sql queries
var loggerFactory = new LoggerFactory();
loggerFactory.AddProvider(new DebugLoggerProvider());
loggerFactory.AddProvider(new ConsoleLoggerProvider(new ConsoleLoggerSettings()));
var builder = new DbContextOptionsBuilder<DbContext>();
builder
.UseSqlServer("my connection string") //"Server=.;Initial Catalog=TestDb;Integrated Security=True"
.UseLoggerFactory(loggerFactory);
var dbContext = new DbContext(builder.Options);
........
Here is a sample output from Visual Studio console:

SQL Management Studio => Tools => SQL Server profiler
File => New Trace...
Use the Template => Blank
Event selection => T-SQL
Lefthandside check for: SP.StmtComplete
Column filters can be used to select a specific ApplicationName or DatabaseName
Start that profile running then trigger the query.
Click here for Source information

Use Logging with Entity Framework Core 3.x
Entity Framework Core emits SQL via the logging system. There are only a couple of small tricks. You must specify an ILoggerFactory and you must specify a filter. Here is an example from this article
Create the factory:
var loggerFactory = LoggerFactory.Create(builder =>
{
builder
.AddConsole((options) => { })
.AddFilter((category, level) =>
category == DbLoggerCategory.Database.Command.Name
&& level == LogLevel.Information);
});
Tell the DbContext to use the factory in the OnConfiguring method:
optionsBuilder.UseLoggerFactory(_loggerFactory);
From here, you can get a lot more sophisticated and hook into the Log method to extract details about the executed SQL. See the article for a full discussion.
public class EntityFrameworkSqlLogger : ILogger
{
#region Fields
Action<EntityFrameworkSqlLogMessage> _logMessage;
#endregion
#region Constructor
public EntityFrameworkSqlLogger(Action<EntityFrameworkSqlLogMessage> logMessage)
{
_logMessage = logMessage;
}
#endregion
#region Implementation
public IDisposable BeginScope<TState>(TState state)
{
return default;
}
public bool IsEnabled(LogLevel logLevel)
{
return true;
}
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception exception, Func<TState, Exception, string> formatter)
{
if (eventId.Id != 20101)
{
//Filter messages that aren't relevant.
//There may be other types of messages that are relevant for other database platforms...
return;
}
if (state is IReadOnlyList<KeyValuePair<string, object>> keyValuePairList)
{
var entityFrameworkSqlLogMessage = new EntityFrameworkSqlLogMessage
(
eventId,
(string)keyValuePairList.FirstOrDefault(k => k.Key == "commandText").Value,
(string)keyValuePairList.FirstOrDefault(k => k.Key == "parameters").Value,
(CommandType)keyValuePairList.FirstOrDefault(k => k.Key == "commandType").Value,
(int)keyValuePairList.FirstOrDefault(k => k.Key == "commandTimeout").Value,
(string)keyValuePairList.FirstOrDefault(k => k.Key == "elapsed").Value
);
_logMessage(entityFrameworkSqlLogMessage);
}
}
#endregion
}

IQueryable query = from x in appEntities
where x.id = 32
select x;
var queryString = query.ToString();
Will return the sql query. Working using datacontext of EntityFramework 6

Necromancing.
This page is the first search result when searching for a solution for any .NET Framework, so here as a public service, how it's done in EntityFrameworkCore (for .NET Core 1 & 2):
var someQuery = (
from projects in _context.projects
join issues in _context.issues on projects.Id equals issues.ProjectId into tmpMapp
from issues in tmpMapp.DefaultIfEmpty()
select issues
) //.ToList()
;
// string sql = someQuery.ToString();
// string sql = Microsoft.EntityFrameworkCore.IQueryableExtensions.ToSql(someQuery);
// string sql = Microsoft.EntityFrameworkCore.IQueryableExtensions1.ToSql(someQuery);
// using Microsoft.EntityFrameworkCore;
string sql = someQuery.ToSql();
System.Console.WriteLine(sql);
And then these extension methods (IQueryableExtensions1 for .NET Core 1.0, IQueryableExtensions for .NET Core 2.0) :
using System;
using System.Linq;
using System.Reflection;
using Microsoft.EntityFrameworkCore.Internal;
using Microsoft.EntityFrameworkCore.Query;
using Microsoft.EntityFrameworkCore.Query.Internal;
using Microsoft.EntityFrameworkCore.Storage;
using Remotion.Linq.Parsing.Structure;
namespace Microsoft.EntityFrameworkCore
{
// https://stackoverflow.com/questions/1412863/how-do-i-view-the-sql-generated-by-the-entity-framework
// http://rion.io/2016/10/19/accessing-entity-framework-core-queries-behind-the-scenes-in-asp-net-core/
public static class IQueryableExtensions
{
private static readonly TypeInfo QueryCompilerTypeInfo = typeof(QueryCompiler).GetTypeInfo();
private static readonly FieldInfo QueryCompilerField = typeof(EntityQueryProvider).GetTypeInfo().DeclaredFields
.First(x => x.Name == "_queryCompiler");
private static readonly PropertyInfo NodeTypeProviderField =
QueryCompilerTypeInfo.DeclaredProperties.Single(x => x.Name == "NodeTypeProvider");
private static readonly MethodInfo CreateQueryParserMethod =
QueryCompilerTypeInfo.DeclaredMethods.First(x => x.Name == "CreateQueryParser");
private static readonly FieldInfo DataBaseField =
QueryCompilerTypeInfo.DeclaredFields.Single(x => x.Name == "_database");
private static readonly PropertyInfo DatabaseDependenciesField =
typeof(Database).GetTypeInfo().DeclaredProperties.Single(x => x.Name == "Dependencies");
public static string ToSql<TEntity>(this IQueryable<TEntity> query) where TEntity : class
{
if (!(query is EntityQueryable<TEntity>) && !(query is InternalDbSet<TEntity>))
{
throw new ArgumentException("Invalid query");
}
var queryCompiler = (QueryCompiler) QueryCompilerField.GetValue(query.Provider);
var nodeTypeProvider = (INodeTypeProvider) NodeTypeProviderField.GetValue(queryCompiler);
var parser = (IQueryParser) CreateQueryParserMethod.Invoke(queryCompiler, new object[] {nodeTypeProvider});
var queryModel = parser.GetParsedQuery(query.Expression);
var database = DataBaseField.GetValue(queryCompiler);
var databaseDependencies = (DatabaseDependencies) DatabaseDependenciesField.GetValue(database);
var queryCompilationContext = databaseDependencies.QueryCompilationContextFactory.Create(false);
var modelVisitor = (RelationalQueryModelVisitor) queryCompilationContext.CreateQueryModelVisitor();
modelVisitor.CreateQueryExecutor<TEntity>(queryModel);
var sql = modelVisitor.Queries.First().ToString();
return sql;
}
}
public class IQueryableExtensions1
{
private static readonly TypeInfo QueryCompilerTypeInfo = typeof(QueryCompiler).GetTypeInfo();
private static readonly FieldInfo QueryCompilerField = typeof(EntityQueryProvider).GetTypeInfo()
.DeclaredFields
.First(x => x.Name == "_queryCompiler");
private static readonly PropertyInfo NodeTypeProviderField =
QueryCompilerTypeInfo.DeclaredProperties.Single(x => x.Name == "NodeTypeProvider");
private static readonly MethodInfo CreateQueryParserMethod =
QueryCompilerTypeInfo.DeclaredMethods.First(x => x.Name == "CreateQueryParser");
private static readonly FieldInfo DataBaseField =
QueryCompilerTypeInfo.DeclaredFields.Single(x => x.Name == "_database");
private static readonly FieldInfo QueryCompilationContextFactoryField = typeof(Database).GetTypeInfo()
.DeclaredFields.Single(x => x.Name == "_queryCompilationContextFactory");
public static string ToSql<TEntity>(IQueryable<TEntity> query) where TEntity : class
{
if (!(query is EntityQueryable<TEntity>) && !(query is InternalDbSet<TEntity>))
{
throw new ArgumentException("Invalid query");
}
var queryCompiler = (IQueryCompiler) QueryCompilerField.GetValue(query.Provider);
var nodeTypeProvider = (INodeTypeProvider) NodeTypeProviderField.GetValue(queryCompiler);
var parser =
(IQueryParser) CreateQueryParserMethod.Invoke(queryCompiler, new object[] {nodeTypeProvider});
var queryModel = parser.GetParsedQuery(query.Expression);
var database = DataBaseField.GetValue(queryCompiler);
var queryCompilationContextFactory =
(IQueryCompilationContextFactory) QueryCompilationContextFactoryField.GetValue(database);
var queryCompilationContext = queryCompilationContextFactory.Create(false);
var modelVisitor = (RelationalQueryModelVisitor) queryCompilationContext.CreateQueryModelVisitor();
modelVisitor.CreateQueryExecutor<TEntity>(queryModel);
var sql = modelVisitor.Queries.First().ToString();
return sql;
}
}
}

Well, I am using Express profiler for that purpose at the moment, the drawback is that it only works for MS SQL Server. You can find this tool here: https://expressprofiler.codeplex.com/

While there are good answers here, none solved my problem completely (I wished to get the entire SQL statement, including Parameters, from the DbContext from any IQueryable. The following code does just that. It is a combination of code snippets from Google. I have only tested it with EF6+.
Just an aside, this task took me way longer than I thought it would. Abstraction in Entity Framework is a bit much, IMHO.
First the using. You will need an explicit reference to 'System.Data.Entity.dll'.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Data.SqlClient;
using System.Data.Common;
using System.Data.Entity.Core.Objects;
using System.Data.Entity;
using System.Data;
using System.Data.Entity.Infrastructure;
using System.Reflection;
The following class converts an IQueryable into a DataTable. Modify as your need may be:
public class EntityFrameworkCommand
{
DbContext Context;
string SQL;
ObjectParameter[] Parameters;
public EntityFrameworkCommand Initialize<T>(DbContext context, IQueryable<T> query)
{
Context = context;
var dbQuery = query as DbQuery<T>;
// get the IInternalQuery internal variable from the DbQuery object
var iqProp = dbQuery.GetType().GetProperty("InternalQuery", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
var iq = iqProp.GetValue(dbQuery, null);
// get the ObjectQuery internal variable from the IInternalQuery object
var oqProp = iq.GetType().GetProperty("ObjectQuery", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
var objectQuery = oqProp.GetValue(iq, null) as ObjectQuery<T>;
SQL = objectQuery.ToTraceString();
Parameters = objectQuery.Parameters.ToArray();
return this;
}
public DataTable GetData()
{
DataTable dt = new DataTable();
var connection = Context.Database.Connection;
var state = connection.State;
if (!(state == ConnectionState.Open))
connection.Open();
using (var cmd = connection.CreateCommand())
{
cmd.CommandText = SQL;
foreach (var p in Parameters)
{
var param = cmd.CreateParameter();
param.Name = "#" + p.Name;
param.Value = p.Value;
cmd.Parameters.Add(param);
}
using (var da = DbProviderFactories.GetFactory(connection).CreateDataAdapter())
{
da.SelectCommand = cmd;
da.Fill(dt);
}
}
if (!(state == ConnectionState.Open))
connection.Close();
return dt;
}
}
To use, simply call it as below:
var context = new MyContext();
var data = ....//Query, return type can be anonymous
.AsQueryable();
var dt = new EntityFrameworkCommand()
.Initialize(context, data)
.GetData();

For me, using EF6 and Visual Studio 2015 I entered query in the immediate window and it gave me the generated SQL Statement

In my case for EF 6+, instead of using this in the Immediate Window to find the query string:
var sql = ((System.Data.Entity.Core.Objects.ObjectQuery)query).ToTraceString();
I ended up having to use this to get the generated SQL command:
var sql = ((System.Data.Entity.Infrastructure.DbQuery<<>f__AnonymousType3<string,string,string,short,string>>)query).ToString();
Of course your anonymous type signature might be different.
HTH.

I've just done this:
IQueryable<Product> query = EntitySet.Where(p => p.Id == id);
Debug.WriteLine(query);
And the result shown in the Output:
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Code] AS [Code],
[Extent1].[Name] AS [Name],
[Extent2].[Id] AS [Id1],
[Extent2].[FileName] AS [FileName],
FROM [dbo].[Products] AS [Extent1]
INNER JOIN [dbo].[PersistedFiles] AS [Extent2] ON [Extent1].[PersistedFileId] = [Extent2].[Id]
WHERE [Extent1].[Id] = #p__linq__0

Entity Framework 4 Solution
Most of the answers here were EF6-specific. Here's one for those of you still using EF4.
This method replaces the #p__linq__0/etc. parameters with their actual values, so you can just copy and paste the output into SSMS and run it or debug it.
/// <summary>
/// Temporary debug function that spits out the actual SQL query LINQ is generating (with parameters)
/// </summary>
/// <param name="q">IQueryable object</param>
private string Debug_GetSQLFromIQueryable<T>(IQueryable<T> q)
{
System.Data.Objects.ObjectQuery oq = (System.Data.Objects.ObjectQuery)q;
var result = oq.ToTraceString();
List<string> paramNames = new List<string>();
List<string> paramVals = new List<string>();
foreach (var parameter in oq.Parameters)
{
paramNames.Add(parameter.Name);
paramVals.Add(parameter.Value == null ? "NULL" : ("'" + parameter.Value.ToString() + "'"));
}
//replace params in reverse order, otherwise #p__linq__1 incorrectly replaces #p__linq__10 for instance
for (var i = paramNames.Count - 1; i >= 0; i--)
{
result = result.Replace("#" + paramNames[i], paramVals[i]);
}
return result;
}

Starting from Entity Framework Core 5.0+ one can simply override the OnConfiguring method in the DbContext once for logging. This works also for Single() or Any() queries.
For logging to debug window:
public class ExampleDbContext : DbContext
{
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
// using System.Diagnostics;
optionsBuilder.LogTo(message => Debug.WriteLine(message));
}
}
For logging to console:
public class ExampleDbContext : DbContext
{
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.LogTo(Console.WriteLine);
}
}
See here for more details including log levels and filtering: https://learn.microsoft.com/en-us/ef/core/logging-events-diagnostics/simple-logging

If you want to have parameter values (not only #p_linq_0 but also their values) too, you can use IDbCommandInterceptor and add some logging to ReaderExecuted method.

Related

Use EF Core to get a list of Longs

I have a stored procedure that returns me a list of IDs. (I then use this list of IDs as keys for objects.)
I am migrating this from .NET to .NET Core. In normal .NET I could use an extension library to get the numbers out like this:
var getOrderDetailIdsStoredProc = new GetOrderDetailIdsStoredProc()
{
NumberOfOrderDetailIdsNeeded = numberOfOrderDetailIdsNeeded
};
var orderDetailIds = contextProvider.Context.Database
.ExecuteStoredProcedure<long>(getOrderDetailIdsStoredProc);
But that library (EntityFrameworkExtras) is not working with EF Core (I found a version for EF Core, but it doesn't work.)
So I have been looking for other solutions:
DbContext.Database.ExecuteSqlCommand: Cannot return records, only output variables
DbSet.FromSQL: Can only be run on a DbSet<T> (basically it needs an entity type)
Right now, all I can think of is to make an entity called Number:
public class Number
{
public long Value;
}
public DbSet<Number> Numbers;
And then do something like this:
Numbers.FromSql("exec GenerateOrderDetailSequencedIds #numberNeeded", numberNeeded)
Aside from the fact that this is very ugly (making an entity out of a native type), I have no table to hook it up to, so I worry it will not work.
Is there any way in EF Core to run a stored procedure and get back a list of numbers?
NOTE: This worked, but was not compatable with BreezeJs (it could not deal with a DbQuery). See my other answer for what I ended up doing.
OrderDetailIdHolder.cs
public class OrderDetailIdHolder
{
public long NewId { get; set; }
}
MyEntitiesContext
public DbQuery<OrderDetailIdHolder> OrderDetailIdHolders { get; set; }
internal List<long> GetOrderDetailIds(int numberOfIdsNeeded)
{
var result = OrderDetailIdHolders.FromSql($"exec Sales.GenerateOrderDetailIds {numberOfIdsNeeded}").ToList();
return result.Select(x=>x.NewId).ToList();
}
This a bit extra complexity for just a list of longs. But it works.
It is important to note that the property (NewId in this case) must match what is returned from the sproc. Also, the type is not a DbSet. It is a DbQuery.
It is also important to note that this is only for EF Core 2.2. EF Core 3 has a different way to do this (Keyless Entity Types)
This is what I ended up using:
public static List<T> SqlQueryList<T>(this DatabaseFacade database, string query, params SqlParameter[] sqlParameters)
{
// TODO: Add a using statement here so we don't leak the connection's resources.
var conn = database.GetDbConnection();
conn.Open();
var command = conn.CreateCommand();
command.CommandText = query;
command.Parameters.AddRange(sqlParameters);
var reader = command.ExecuteReader();
List<T> result = new List<T>();
while (reader.Read())
{
T typedRow;
var row = reader.GetValue(0);
if (typeof(T).IsValueType)
{
typedRow = (T) row;
}
else
{
typedRow = (T)Convert.ChangeType(result, typeof(T));
}
result.Add(typedRow);
}
return result;
}
Called like this:
var numberOfOrderDetailIdsNeededParam = new SqlParameter
{
ParameterName = "#numberOfOrderDetailIdsNeeded",
SqlDbType = SqlDbType.Int,
Direction = ParameterDirection.Input
};
numberOfOrderDetailIdsNeededParam.Value = numberOfOrderDetailIdsNeeded;
var result = contextProvider.Context.Database.SqlQueryList<long>($"exec Sales.GenerateOrderDetailIds #numberOfOrderDetailIdsNeeded", numberOfOrderDetailIdsNeededParam);
I did it this way because it was compatible with BreezeJs for .NET Core. Note that I only really tested this with Value types.

Can I see the SQL in progress when an EntitySqlException is thrown, or can I override an EntityCommandCompilationException?

I've gone down a bit of a rabbit hole trying to execute a window function in Entity Framework. (The project in question is built on EF but would gain a lot from calculating PERCENTILE_DISC on the SQL Server.)
I'm creating a new Convention and adding it to the object model to translate calls to a certain method into SQL that executes the PERCENT_DISC window function.
The created EdmFunction has a CommandText of:
PERCENTILE_DISC (0.8) WITHIN GROUP (ORDER BY o.ExpectedPayment) OVER (PARTITION BY o.OrderType)
But when I do this to execute the function:
var medians = context.Set<UserLocation>().Select(x => CustomFunction.Percentile()).ToList();
This throws an EntityCommandCompilationException with the following message:
System.Data.Entity.Core.EntityCommandCompilationException:
'An error occurred while preparing definition of the function 'ConsoleApp5.Percentile'. See the inner exception for details.'
Inner Exception:
EntitySqlException: The query syntax is not valid. Near identifier 'WITHIN', line 1, column 35.
Yet this direct query gets the expected results:
var p80s = context.Database.SqlQuery<decimal>("SELECT DISTINCT PERCENTILE_DISC (0.8) WITHIN GROUP (ORDER BY o.ExpectedPayment) OVER (PARTITION BY o.OrderType) from Orders o").ToList();
I suspect that this is due to the EF parser not being built to handle window functions. So, I'd like to be able to override that EntityCommandCompilationException and have EF try to execute the query anyhow. Failing that, I'd like to at least see the SQL generated so far to see if there's a different problem causing genuinely invalid SQL. How can I accomplish either of these?
I work in one project that uses one interceptor to register slow queries.
Below, I put the code of the interceptor:
using System;
using System.Data.Common;
using System.Data.Entity.Infrastructure.Interception;
using System.Data.SqlClient;
using System.Diagnostics;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
namespace Nash.Data.Interceptors.SlowQuery
{
public sealed class SqlQueryCommandInterceptor : IDbCommandInterceptor
{
private readonly ISlowQueryLogger _slowQueryLogger;
public SqlQueryCommandInterceptor(ISlowQueryLogger slowQueryLogger)
{
_slowQueryLogger = slowQueryLogger;
}
public void NonQueryExecuting(DbCommand command, DbCommandInterceptionContext<int> interceptionContext)
=> interceptionContext.UserState = Stopwatch.StartNew();
public void NonQueryExecuted(DbCommand command, DbCommandInterceptionContext<int> interceptionContext)
=> LogSlowQuery(interceptionContext, command);
public void ReaderExecuting(DbCommand command, DbCommandInterceptionContext<DbDataReader> interceptionContext)
=> interceptionContext.UserState = Stopwatch.StartNew();
public void ReaderExecuted(DbCommand command, DbCommandInterceptionContext<DbDataReader> interceptionContext)
=> LogSlowQuery(interceptionContext, command);
public void ScalarExecuting(DbCommand command, DbCommandInterceptionContext<object> interceptionContext)
=> interceptionContext.UserState = Stopwatch.StartNew();
public void ScalarExecuted(DbCommand command, DbCommandInterceptionContext<object> interceptionContext)
=> LogSlowQuery(interceptionContext, command);
private void LogSlowQuery<T>(DbCommandInterceptionContext<T> interceptionContext, DbCommand dbCommand)
{
var debugText = GetDbCommandDebugText(dbCommand);
var userState = (Stopwatch)interceptionContext.UserState;
userState.Stop();
var elapsed = userState.Elapsed;
if (elapsed > TimeSpan.FromSeconds(2.6))
_slowQueryLogger.LogSlowQuery(debugText, elapsed);
}
private static string GetDbCommandDebugText(DbCommand dbCommand)
{
var debugText = dbCommand.CommandText;
if (dbCommand is SqlCommand && debugText.Contains("#"))
{
var matches = Regex.Matches(debugText, #"(\#[\w\.]+)").Cast<Match>().ToArray();
var paramDict = dbCommand.Parameters.Cast<SqlParameter>()
.Select(x => new
{
ParameterName = x.ParameterName.StartsWith("#") ? x.ParameterName : "#" + x.ParameterName,
Value = x.Value,
})
.ToDictionary(x => x.ParameterName, x => x.Value);
var buffer = new StringBuilder();
var i = 0;
foreach (var m in matches)
{
if (m.Index > i)
{
buffer.Append(debugText.Substring(i, m.Index - i));
i = m.Index;
}
var paramName = m.Groups[1].Value;
if (paramDict.TryGetValue(paramName, out var paramVal))
if (paramVal == null || DBNull.Value.Equals(paramVal))
buffer.Append($"NULL");
else
buffer.Append($"'{paramVal}'");
else
buffer.Append(paramName);
i += m.Length;
}
if (i < debugText.Length)
buffer.Append(debugText.Substring(i, debugText.Length - i));
debugText = buffer.ToString();
}
return debugText;
}
}
}
To use this interceptor you can code something like this:
using System.Data.Entity.Infrastructure.Interception;
DbInterception.Add(NinjectWebCommon.Kernel.Get<SqlQueryCommandInterceptor>());
In my case, I put this code into my web app startup: Global.asax.cs. And I use Ninject to get one instance of the interceptor.
My code can do more then you need... so you maybe needs to read to a better comprehension and adapt it to your needs.
I hope it can help you.

Execute SP using Entity Framework and .net Core

I created a web api project using .net core and entity framework.
This uses a stored procedure which returns back most of the properties of a database table defined by entity framework.
The entity framwrok does not bring back all the columns of the table. And I get an error when I call the api complaining it cannot find the missing columns when I execute the stored procedure using ,
_context.Set<TableFromSql>().FromSql("execute dbo.spr_GetValue").ToList();
I created another model class which defines the properties brought back from the SP( called NewClass).
_context.Set<NewClass>().FromSql("execute dbo.spr_GetValue").ToList();
This works, but just wanted to check if there is a convention that the SP should only return the model classes from the database.
The SQL query must return data for all properties of the entity or query type
For this limitation, it is caused when mapping the sql query result to Model. It loop through the properties in model and try to retrive the values from query result. If the model properties are not exist in query result, it will throw error.
If you want to return required columns instead of all columns, one options is to define the returned model by Query.
For your demo code, you may define this in OnModelCreating.
builder.Query<TableFromSql>();
Note, for this way, you need to make sure all properties in TableFromSql exist in execute dbo.spr_GetValue.
For another way, you may implement your own FromSql which will add condition to check whether the properties are exist in query result.
public static class DbContextExtensions
{
public static List<T> RawSqlQuery<T>(this DbContext context,string query)
{
using (var command = context.Database.GetDbConnection().CreateCommand())
{
command.CommandText = query;
command.CommandType = CommandType.Text;
context.Database.OpenConnection();
using (var result = command.ExecuteReader())
{
var entities = new List<T>();
return DataReaderMapToList<T>(result);
}
}
}
public static List<T> DataReaderMapToList<T>(IDataReader dr)
{
List<T> list = new List<T>();
T obj = default(T);
while (dr.Read())
{
obj = Activator.CreateInstance<T>();
foreach (PropertyInfo prop in obj.GetType().GetProperties())
{
if (ColumnExists(dr, prop.Name))
{
if (!object.Equals(dr[prop.Name], DBNull.Value))
{
prop.SetValue(obj, dr[prop.Name], null);
}
}
}
list.Add(obj);
}
return list;
}
public static bool ColumnExists(IDataReader reader, string columnName)
{
return reader.GetSchemaTable()
.Rows
.OfType<DataRow>()
.Any(row => row["ColumnName"].ToString() == columnName);
}
}
Use above code like :
var result = _context.RawSqlQuery<ToDoItemVM>("execute [dbo].[get_TodoItem]");

ASP.NET Core Entity Framework SQL Query SELECT

I am one of the many struggling to "upgrade" from ASP.NET to ASP.NET Core.
In the ASP.NET project, I made database calls from my DAL like so:
var result = context.Database.SqlQuery<Object_VM>("EXEC [sp_Object_GetByKey] #Key",
new SqlParameter("#Key", Key))
.FirstOrDefault();
return result;
My viewmodel has additional fields that my object does not, such as aggregates of related tables. It seems unnecessary and counter intuitive to include such fields in a database / table structure. My stored procedure calculates all those things and returns the fields as should be displayed, but not stored.
I see that ASP.NET Core has removed this functionality. I am trying to continue to use stored procedures and load view models (and thus not have the entity in the database). I see options like the following, but as a result I get "2", the number of rows being returned (or another mysterious result?).
using(context)
{
string cmd = "EXEC [sp_Object_getAll]";
var result = context.Database.ExecuteSQLCommand(cmd);
}
But that won't work because context.Database.ExecuteSQLCommand is only for altering the database, not "selecting".
I've also seen the following as a solution, but the code will not compile for me, as "set" is really set<TEntity>, and there isn't a database entity for this viewmodel.
var result = context.Set().FromSql("EXEC [sp_Object_getAll]");
Any assistance much appreciated.
Solution:
(per Tseng's advice)
On the GitHub Entity Framework Issues page, there is a discussion about this problem. One user recommends creating your own class to handle this sort of requests, and another adds an additional method that makes it run smoother. I changed the methods slights to accept slightly different params.
Here is my adaptation (very little difference), for others that are also looking for a solution:
Method in DAL
public JsonResult GetObjectByID(int ID)
{
SqlParameter[] parms = new SqlParameter[] { new SqlParameter("#ID", ID) };
var result = RDFacadeExtensions.GetModelFromQuery<Object_List_VM>(context, "EXEC [sp_Object_GetList] #ID", parms);
return new JsonResult(result.ToList(), setting);
}
Additional Class
public static class RDFacadeExtensions
{
public static RelationalDataReader ExecuteSqlQuery(
this DatabaseFacade databaseFacade,
string sql,
SqlParameter[] parameters)
{
var concurrencyDetector = databaseFacade.GetService<IConcurrencyDetector>();
using (concurrencyDetector.EnterCriticalSection())
{
var rawSqlCommand = databaseFacade
.GetService<IRawSqlCommandBuilder>()
.Build(sql, parameters);
return rawSqlCommand
.RelationalCommand
.ExecuteReader(
databaseFacade.GetService<IRelationalConnection>(),
parameterValues: rawSqlCommand.ParameterValues);
}
}
public static IEnumerable<T> GetModelFromQuery<T>(
DbContext context,
string sql,
SqlParameter[] parameters)
where T : new()
{
DatabaseFacade databaseFacade = new DatabaseFacade(context);
using (DbDataReader dr = databaseFacade.ExecuteSqlQuery(sql, parameters).DbDataReader)
{
List<T> lst = new List<T>();
PropertyInfo[] props = typeof(T).GetProperties();
while (dr.Read())
{
T t = new T();
IEnumerable<string> actualNames = dr.GetColumnSchema().Select(o => o.ColumnName);
for (int i = 0; i < props.Length; ++i)
{
PropertyInfo pi = props[i];
if (!pi.CanWrite) continue;
System.ComponentModel.DataAnnotations.Schema.ColumnAttribute ca = pi.GetCustomAttribute(typeof(System.ComponentModel.DataAnnotations.Schema.ColumnAttribute)) as System.ComponentModel.DataAnnotations.Schema.ColumnAttribute;
string name = ca?.Name ?? pi.Name;
if (pi == null) continue;
if (!actualNames.Contains(name)) { continue; }
object value = dr[name];
Type pt = pi.DeclaringType;
bool nullable = pt.GetTypeInfo().IsGenericType && pt.GetGenericTypeDefinition() == typeof(Nullable<>);
if (value == DBNull.Value) { value = null; }
if (value == null && pt.GetTypeInfo().IsValueType && !nullable)
{ value = Activator.CreateInstance(pt); }
pi.SetValue(t, value);
}//for i
lst.Add(t);
}//while
return lst;
}//using dr
}

EF 4.1 Code First Multiple Result Sets

I need to execute a Raw SQL query that returns multiple result sets. Is that possible in EF CF ?
Thanks!
Description
Yes you can! ;) You can perform a raw sql query using DbCommand in Entity Framework Code First too.
You can perform queries with multiple result sets and jump to the next result set using NextResult() method of the SqlDataReader class.
Sample
namespace MyNamespace
{
public class MyDbContext : DbContext
{
}
public class Test
{
public void Test()
{
MyDbContext context = new MyDbContext();
DbCommand db = context.Database.Connection.CreateCommand();
db.CommandText = "SELECT propertie1 FROM Table1; SELECT propertie1 from Table2";
try
{
DbDataReader reader = db.ExecuteReader();
while (reader.Read())
{
// read your data from result set 1
string value = Convert.ToString(reader["propertie1"]);
}
reader.NextResult();
while (reader.Read())
{
// read your data from result set 2
string value = Convert.ToString(reader["propertie1"]);
}
}
catch
{
// Exception handling
}
finally
{
if (db.Connection.State != ConnectionState.Closed)
db.Connection.Close();
}
}
}
}
More Information
Using DbContext in EF 4.1 Part 10: Raw SQL Queries