We are using ASP.NET Zero and are running into issues with parallel processing from an AppService. We know requests must be transactional, but unfortunately we need to break out to slow running APIs for numerous calls, so we have to do parallel processing.
As expected, we are running into a DbContext contingency issue on the second database call we make:
System.InvalidOperationException: A second operation started on this context
before a previous operation completed. This is usually caused by different
threads using the same instance of DbContext, however instance members are
not guaranteed to be thread safe. This could also be caused by a nested query
being evaluated on the client, if this is the case rewrite the query avoiding
nested invocations.
We read that a new UOW is required, so we tried using both the method attribute and the explicit UowManager, but neither of the two worked.
We also tried creating instances of the referenced AppServices using the IocResolver, but we are still not able to get a unique DbContext per thread (please see below).
public List<InvoiceDto> CreateInvoices(List<InvoiceTemplateLineItemDto> templateLineItems)
{
List<InvoiceDto> invoices = new InvoiceDto[templateLineItems.Count].ToList();
ConcurrentQueue<Exception> exceptions = new ConcurrentQueue<Exception>();
Parallel.ForEach(templateLineItems, async (templateLineItem) =>
{
try
{
XAppService xAppService = _iocResolver.Resolve<XAppService>();
InvoiceDto invoice = await xAppService
.CreateInvoiceInvoiceItem();
invoices.Insert(templateLineItems.IndexOf(templateLineItem), invoice);
}
catch (Exception e)
{
exceptions.Enqueue(e);
}
});
if (exceptions.Count > 0) throw new AggregateException(exceptions);
return invoices;
}
How can we ensure that a new DbContext is availble per thread?
I was able to replicate and resolve the problem with a generic version of ABP. I'm still experiencing the problem in my original solution, which is far more complex. I'll have to do some more digging to determine why it is failing there.
For others that come across this problem, which is exactly the same issue as reference here, you can simply disable the UnitOfWork through an attribute as illustrated in the code below.
public class InvoiceAppService : ApplicationService
{
private readonly InvoiceItemAppService _invoiceItemAppService;
public InvoiceAppService(InvoiceItemAppService invoiceItemAppService)
{
_invoiceItemAppService = invoiceItemAppService;
}
// Just add this attribute
[UnitOfWork(IsDisabled = true)]
public InvoiceDto GetInvoice(List<int> invoiceItemIds)
{
_invoiceItemAppService.Initialize();
ConcurrentQueue<InvoiceItemDto> invoiceItems =
new ConcurrentQueue<InvoiceItemDto>();
ConcurrentQueue<Exception> exceptions = new ConcurrentQueue<Exception>();
Parallel.ForEach(invoiceItemIds, (invoiceItemId) =>
{
try
{
InvoiceItemDto invoiceItemDto =
_invoiceItemAppService.CreateAsync(invoiceItemId).Result;
invoiceItems.Enqueue(invoiceItemDto);
}
catch (Exception e)
{
exceptions.Enqueue(e);
}
});
if (exceptions.Count > 0) {
AggregateException ex = new AggregateException(exceptions);
Logger.Error("Unable to get invoice", ex);
throw ex;
}
return new InvoiceDto {
Date = DateTime.Now,
InvoiceItems = invoiceItems.ToArray()
};
}
}
public class InvoiceItemAppService : ApplicationService
{
private readonly IRepository<InvoiceItem> _invoiceItemRepository;
private readonly IRepository<Token> _tokenRepository;
private readonly IRepository<Credential> _credentialRepository;
private Token _token;
private Credential _credential;
public InvoiceItemAppService(IRepository<InvoiceItem> invoiceItemRepository,
IRepository<Token> tokenRepository,
IRepository<Credential> credentialRepository)
{
_invoiceItemRepository = invoiceItemRepository;
_tokenRepository = tokenRepository;
_credentialRepository = credentialRepository;
}
public void Initialize()
{
_token = _tokenRepository.FirstOrDefault(x => x.Id == 1);
_credential = _credentialRepository.FirstOrDefault(x => x.Id == 1);
}
// Create an invoice item using info from an external API and some db records
public async Task<InvoiceItemDto> CreateAsync(int id)
{
// Get db record
InvoiceItem invoiceItem = await _invoiceItemRepository.GetAsync(id);
// Get price
decimal price = await GetPriceAsync(invoiceItem.Description);
return new InvoiceItemDto {
Id = id,
Description = invoiceItem.Description,
Amount = price
};
}
private async Task<decimal> GetPriceAsync(string description)
{
// Simulate a slow API call to get price using description
// We use the token and credentials here in the real deal
await Task.Delay(5000);
return 100.00M;
}
}
I have a stored procedure that returns me a list of IDs. (I then use this list of IDs as keys for objects.)
I am migrating this from .NET to .NET Core. In normal .NET I could use an extension library to get the numbers out like this:
var getOrderDetailIdsStoredProc = new GetOrderDetailIdsStoredProc()
{
NumberOfOrderDetailIdsNeeded = numberOfOrderDetailIdsNeeded
};
var orderDetailIds = contextProvider.Context.Database
.ExecuteStoredProcedure<long>(getOrderDetailIdsStoredProc);
But that library (EntityFrameworkExtras) is not working with EF Core (I found a version for EF Core, but it doesn't work.)
So I have been looking for other solutions:
DbContext.Database.ExecuteSqlCommand: Cannot return records, only output variables
DbSet.FromSQL: Can only be run on a DbSet<T> (basically it needs an entity type)
Right now, all I can think of is to make an entity called Number:
public class Number
{
public long Value;
}
public DbSet<Number> Numbers;
And then do something like this:
Numbers.FromSql("exec GenerateOrderDetailSequencedIds #numberNeeded", numberNeeded)
Aside from the fact that this is very ugly (making an entity out of a native type), I have no table to hook it up to, so I worry it will not work.
Is there any way in EF Core to run a stored procedure and get back a list of numbers?
NOTE: This worked, but was not compatable with BreezeJs (it could not deal with a DbQuery). See my other answer for what I ended up doing.
OrderDetailIdHolder.cs
public class OrderDetailIdHolder
{
public long NewId { get; set; }
}
MyEntitiesContext
public DbQuery<OrderDetailIdHolder> OrderDetailIdHolders { get; set; }
internal List<long> GetOrderDetailIds(int numberOfIdsNeeded)
{
var result = OrderDetailIdHolders.FromSql($"exec Sales.GenerateOrderDetailIds {numberOfIdsNeeded}").ToList();
return result.Select(x=>x.NewId).ToList();
}
This a bit extra complexity for just a list of longs. But it works.
It is important to note that the property (NewId in this case) must match what is returned from the sproc. Also, the type is not a DbSet. It is a DbQuery.
It is also important to note that this is only for EF Core 2.2. EF Core 3 has a different way to do this (Keyless Entity Types)
This is what I ended up using:
public static List<T> SqlQueryList<T>(this DatabaseFacade database, string query, params SqlParameter[] sqlParameters)
{
// TODO: Add a using statement here so we don't leak the connection's resources.
var conn = database.GetDbConnection();
conn.Open();
var command = conn.CreateCommand();
command.CommandText = query;
command.Parameters.AddRange(sqlParameters);
var reader = command.ExecuteReader();
List<T> result = new List<T>();
while (reader.Read())
{
T typedRow;
var row = reader.GetValue(0);
if (typeof(T).IsValueType)
{
typedRow = (T) row;
}
else
{
typedRow = (T)Convert.ChangeType(result, typeof(T));
}
result.Add(typedRow);
}
return result;
}
Called like this:
var numberOfOrderDetailIdsNeededParam = new SqlParameter
{
ParameterName = "#numberOfOrderDetailIdsNeeded",
SqlDbType = SqlDbType.Int,
Direction = ParameterDirection.Input
};
numberOfOrderDetailIdsNeededParam.Value = numberOfOrderDetailIdsNeeded;
var result = contextProvider.Context.Database.SqlQueryList<long>($"exec Sales.GenerateOrderDetailIds #numberOfOrderDetailIdsNeeded", numberOfOrderDetailIdsNeededParam);
I did it this way because it was compatible with BreezeJs for .NET Core. Note that I only really tested this with Value types.
I created a web api project using .net core and entity framework.
This uses a stored procedure which returns back most of the properties of a database table defined by entity framework.
The entity framwrok does not bring back all the columns of the table. And I get an error when I call the api complaining it cannot find the missing columns when I execute the stored procedure using ,
_context.Set<TableFromSql>().FromSql("execute dbo.spr_GetValue").ToList();
I created another model class which defines the properties brought back from the SP( called NewClass).
_context.Set<NewClass>().FromSql("execute dbo.spr_GetValue").ToList();
This works, but just wanted to check if there is a convention that the SP should only return the model classes from the database.
The SQL query must return data for all properties of the entity or query type
For this limitation, it is caused when mapping the sql query result to Model. It loop through the properties in model and try to retrive the values from query result. If the model properties are not exist in query result, it will throw error.
If you want to return required columns instead of all columns, one options is to define the returned model by Query.
For your demo code, you may define this in OnModelCreating.
builder.Query<TableFromSql>();
Note, for this way, you need to make sure all properties in TableFromSql exist in execute dbo.spr_GetValue.
For another way, you may implement your own FromSql which will add condition to check whether the properties are exist in query result.
public static class DbContextExtensions
{
public static List<T> RawSqlQuery<T>(this DbContext context,string query)
{
using (var command = context.Database.GetDbConnection().CreateCommand())
{
command.CommandText = query;
command.CommandType = CommandType.Text;
context.Database.OpenConnection();
using (var result = command.ExecuteReader())
{
var entities = new List<T>();
return DataReaderMapToList<T>(result);
}
}
}
public static List<T> DataReaderMapToList<T>(IDataReader dr)
{
List<T> list = new List<T>();
T obj = default(T);
while (dr.Read())
{
obj = Activator.CreateInstance<T>();
foreach (PropertyInfo prop in obj.GetType().GetProperties())
{
if (ColumnExists(dr, prop.Name))
{
if (!object.Equals(dr[prop.Name], DBNull.Value))
{
prop.SetValue(obj, dr[prop.Name], null);
}
}
}
list.Add(obj);
}
return list;
}
public static bool ColumnExists(IDataReader reader, string columnName)
{
return reader.GetSchemaTable()
.Rows
.OfType<DataRow>()
.Any(row => row["ColumnName"].ToString() == columnName);
}
}
Use above code like :
var result = _context.RawSqlQuery<ToDoItemVM>("execute [dbo].[get_TodoItem]");
How do I view the SQL generated by entity framework ?
(In my particular case I'm using the mysql provider - if it matters)
For those using Entity Framework 6 and up, if you want to view the output SQL in Visual Studio (like I did) you have to use the new logging/interception functionality.
Adding the following line will spit out the generated SQL (along with additional execution-related details) in the Visual Studio output panel:
using (MyDatabaseEntities context = new MyDatabaseEntities())
{
context.Database.Log = s => System.Diagnostics.Debug.WriteLine(s);
// query the database using EF here.
}
More information about logging in EF6 in this nifty blog series: http://blog.oneunicorn.com/2013/05/08/ef6-sql-logging-part-1-simple-logging/
Note: Make sure you are running your project in DEBUG mode.
You can do the following:
IQueryable query = from x in appEntities
where x.id == 32
select x;
var sql = ((System.Data.Objects.ObjectQuery)query).ToTraceString();
or in EF6:
var sql = ((System.Data.Entity.Core.Objects.ObjectQuery)query)
.ToTraceString();
or in EF6.3+:
var sql = ((dynamic)flooringStoresProducts).Sql;
That will give you the SQL that was generated.
Starting with EF6.1 you can use Interceptors to register a database logger.
See chapters "Interceptors" and "Logging Database Operations" to a File here
<configuration>
<entityFramework>
<interceptors>
<interceptor type="System.Data.Entity.Infrastructure.Interception.DatabaseLogger, EntityFramework">
<parameters>
<parameter value="C:\Temp\LogOutput.txt"/>
<parameter value="true" type="System.Boolean"/>
</parameters>
</interceptor>
</interceptors>
</entityFramework>
</configuration>
If you are using a DbContext, you can do the following to get the SQL:
var result = from i in myContext.appEntities
select new Model
{
field = i.stuff,
};
var sql = result.ToString();
EF Core 5.0+
This loooong-awaited feature is available in EF Core 5.0! This is from the weekly status updates:
var query = context.Set<Customer>().Where(c => c.City == city);
Console.WriteLine(query.ToQueryString())
results in this output when using the SQL Server database provider:
DECLARE p0 nvarchar(4000) = N'London';
SELECT [c].[CustomerID], [c].[Address], [c].[City], [c].[CompanyName],
[c].[ContactName], [c].[ContactTitle], [c].[Country], [c].[Fax], [c].[Phone],
[c].[PostalCode], [c].[Region]
FROM [Customers] AS [c]
WHERE [c].[City] = #__city_0
Notice that declarations for parameters of the correct type are also
included in the output. This allows copy/pasting to SQL Server
Management Studio, or similar tools, such that the query can be
executed for debugging/analysis.
woohoo!!!
(Note: You will need using Microsoft.EntityFrameworkCore;)
Applicable for EF 6.0 and above:
For those of you wanting to know more about the logging functionality and adding to the some of the answers already given.
Any command sent from the EF to the database can now be logged. To view the generated queries from EF 6.x, use the DBContext.Database.Log property
What Gets Logged
- SQL for all different kinds of commands. For example:
- Queries, including normal LINQ queries, eSQL queries, and raw queries from methods such as SqlQuery.
- Inserts, updates, and deletes generated as part of SaveChanges
- Relationship loading queries such as those generated by lazy loading
- Parameters
- Whether or not the command is being executed asynchronously
- A timestamp indicating when the command started executing
- Whether or not the command completed successfully, failed by throwing an exception, or, for async, was canceled
- Some indication of the result value
- The approximate amount of time it took to execute the command. Note that this is the time from sending the command to getting the result object back. It does not include time to read the results.
Example:
using (var context = new BlogContext())
{
context.Database.Log = Console.Write;
var blog = context.Blogs.First(b => b.Title == "One Unicorn");
blog.Posts.First().Title = "Green Eggs and Ham";
blog.Posts.Add(new Post { Title = "I do not like them!" });
context.SaveChangesAsync().Wait();
}
Output:
SELECT TOP (1)
[Extent1].[Id] AS [Id],
[Extent1].[Title] AS [Title]
FROM [dbo].[Blogs] AS [Extent1]
WHERE (N'One Unicorn' = [Extent1].[Title]) AND ([Extent1].[Title] IS NOT NULL)
-- Executing at 10/8/2013 10:55:41 AM -07:00
-- Completed in 4 ms with result: SqlDataReader
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Title] AS [Title],
[Extent1].[BlogId] AS [BlogId]
FROM [dbo].[Posts] AS [Extent1]
WHERE [Extent1].[BlogId] = #EntityKeyValue1
-- EntityKeyValue1: '1' (Type = Int32)
-- Executing at 10/8/2013 10:55:41 AM -07:00
-- Completed in 2 ms with result: SqlDataReader
UPDATE [dbo].[Posts]
SET [Title] = #0
WHERE ([Id] = #1)
-- #0: 'Green Eggs and Ham' (Type = String, Size = -1)
-- #1: '1' (Type = Int32)
-- Executing asynchronously at 10/8/2013 10:55:41 AM -07:00
-- Completed in 12 ms with result: 1
INSERT [dbo].[Posts]([Title], [BlogId])
VALUES (#0, #1)
SELECT [Id]
FROM [dbo].[Posts]
WHERE ##ROWCOUNT > 0 AND [Id] = scope_identity()
-- #0: 'I do not like them!' (Type = String, Size = -1)
-- #1: '1' (Type = Int32)
-- Executing asynchronously at 10/8/2013 10:55:41 AM -07:00
-- Completed in 2 ms with result: SqlDataReader
To log to an external file:
using (var context = new BlogContext())
{
using (var sqlLogFile = new StreamWriter("C:\\temp\\LogFile.txt"))
{
context.Database.Log = sqlLogFile.Write;
var blog = context.Blogs.First(b => b.Title == "One Unicorn");
blog.Posts.First().Title = "Green Eggs and Ham";
context.SaveChanges();
}
}
More info here: Logging and Intercepting Database Operations
You can do the following in EF 4.1:
var result = from x in appEntities
where x.id = 32
select x;
System.Diagnostics.Trace.WriteLine(result .ToString());
That will give you the SQL that was generated.
My answer addresses EF core. I reference this github issue, and the docs on configuring DbContext:
Simple
Override the OnConfiguring method of your DbContext class (YourCustomDbContext) as shown here to use a ConsoleLoggerProvider; your queries should log to the console:
public class YourCustomDbContext : DbContext
{
#region DefineLoggerFactory
public static readonly LoggerFactory MyLoggerFactory
= new LoggerFactory(new[] {new ConsoleLoggerProvider((_, __) => true, true)});
#endregion
#region RegisterLoggerFactory
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
=> optionsBuilder
.UseLoggerFactory(MyLoggerFactory); // Warning: Do not create a new ILoggerFactory instance each time
#endregion
}
Complex
This Complex case avoids overriding the DbContext OnConfiguring method. , which is discouraged in the docs: "This approach does not lend itself to testing, unless the tests target the full database."
This Complex case uses:
The IServiceCollection in Startup class ConfigureServices method
(instead of overriding the OnConfiguring method; the benefit is a looser coupling between the DbContext and the ILoggerProvider you want to use)
An implementation of ILoggerProvider (instead of using the ConsoleLoggerProvider implementation shown above; benefit is our implementation shows how we would log to File (I don't see a File Logging Provider shipped with EF Core))
Like this:
public class Startup
public void ConfigureServices(IServiceCollection services)
{
...
var lf = new LoggerFactory();
lf.AddProvider(new MyLoggerProvider());
services.AddDbContext<YOUR_DB_CONTEXT>(optionsBuilder => optionsBuilder
.UseSqlServer(connection_string)
//Using the LoggerFactory
.UseLoggerFactory(lf));
...
}
}
Here's the implementation of a MyLoggerProvider (and its MyLogger which appends its logs to a File you can configure; your EF Core queries will appear in the file.)
public class MyLoggerProvider : ILoggerProvider
{
public ILogger CreateLogger(string categoryName)
{
return new MyLogger();
}
public void Dispose()
{ }
private class MyLogger : ILogger
{
public bool IsEnabled(LogLevel logLevel)
{
return true;
}
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception exception, Func<TState, Exception, string> formatter)
{
File.AppendAllText(#"C:\temp\log.txt", formatter(state, exception));
Console.WriteLine(formatter(state, exception));
}
public IDisposable BeginScope<TState>(TState state)
{
return null;
}
}
}
To have the query always handy, without changing code
add this to your DbContext and check it on the output window in visual studio.
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
Database.Log = (query)=> Debug.Write(query);
}
Similar to #Matt Nibecker answer, but with this you do not have to add it in your current code, every time you need the query.
There are two ways:
To view the SQL that will be generated, simply call ToTraceString(). You can add it into your watch window and set a breakpoint to see what the query would be at any given point for any LINQ query.
You can attach a tracer to your SQL server of choice, which will show you the final query in all its gory detail. In the case of MySQL, the easiest way to trace the queries is simply to tail the query log with tail -f. You can learn more about MySQL's logging facilities in the official documentation. For SQL Server, the easiest way is to use the included SQL Server profiler.
I am doing integration test, and needed this to debug the generated SQL statement in Entity Framework Core 2.1, so I use DebugLoggerProvider or ConsoleLoggerProvider like so:
[Fact]
public async Task MyAwesomeTest
{
//setup log to debug sql queries
var loggerFactory = new LoggerFactory();
loggerFactory.AddProvider(new DebugLoggerProvider());
loggerFactory.AddProvider(new ConsoleLoggerProvider(new ConsoleLoggerSettings()));
var builder = new DbContextOptionsBuilder<DbContext>();
builder
.UseSqlServer("my connection string") //"Server=.;Initial Catalog=TestDb;Integrated Security=True"
.UseLoggerFactory(loggerFactory);
var dbContext = new DbContext(builder.Options);
........
Here is a sample output from Visual Studio console:
SQL Management Studio => Tools => SQL Server profiler
File => New Trace...
Use the Template => Blank
Event selection => T-SQL
Lefthandside check for: SP.StmtComplete
Column filters can be used to select a specific ApplicationName or DatabaseName
Start that profile running then trigger the query.
Click here for Source information
Use Logging with Entity Framework Core 3.x
Entity Framework Core emits SQL via the logging system. There are only a couple of small tricks. You must specify an ILoggerFactory and you must specify a filter. Here is an example from this article
Create the factory:
var loggerFactory = LoggerFactory.Create(builder =>
{
builder
.AddConsole((options) => { })
.AddFilter((category, level) =>
category == DbLoggerCategory.Database.Command.Name
&& level == LogLevel.Information);
});
Tell the DbContext to use the factory in the OnConfiguring method:
optionsBuilder.UseLoggerFactory(_loggerFactory);
From here, you can get a lot more sophisticated and hook into the Log method to extract details about the executed SQL. See the article for a full discussion.
public class EntityFrameworkSqlLogger : ILogger
{
#region Fields
Action<EntityFrameworkSqlLogMessage> _logMessage;
#endregion
#region Constructor
public EntityFrameworkSqlLogger(Action<EntityFrameworkSqlLogMessage> logMessage)
{
_logMessage = logMessage;
}
#endregion
#region Implementation
public IDisposable BeginScope<TState>(TState state)
{
return default;
}
public bool IsEnabled(LogLevel logLevel)
{
return true;
}
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception exception, Func<TState, Exception, string> formatter)
{
if (eventId.Id != 20101)
{
//Filter messages that aren't relevant.
//There may be other types of messages that are relevant for other database platforms...
return;
}
if (state is IReadOnlyList<KeyValuePair<string, object>> keyValuePairList)
{
var entityFrameworkSqlLogMessage = new EntityFrameworkSqlLogMessage
(
eventId,
(string)keyValuePairList.FirstOrDefault(k => k.Key == "commandText").Value,
(string)keyValuePairList.FirstOrDefault(k => k.Key == "parameters").Value,
(CommandType)keyValuePairList.FirstOrDefault(k => k.Key == "commandType").Value,
(int)keyValuePairList.FirstOrDefault(k => k.Key == "commandTimeout").Value,
(string)keyValuePairList.FirstOrDefault(k => k.Key == "elapsed").Value
);
_logMessage(entityFrameworkSqlLogMessage);
}
}
#endregion
}
IQueryable query = from x in appEntities
where x.id = 32
select x;
var queryString = query.ToString();
Will return the sql query. Working using datacontext of EntityFramework 6
Necromancing.
This page is the first search result when searching for a solution for any .NET Framework, so here as a public service, how it's done in EntityFrameworkCore (for .NET Core 1 & 2):
var someQuery = (
from projects in _context.projects
join issues in _context.issues on projects.Id equals issues.ProjectId into tmpMapp
from issues in tmpMapp.DefaultIfEmpty()
select issues
) //.ToList()
;
// string sql = someQuery.ToString();
// string sql = Microsoft.EntityFrameworkCore.IQueryableExtensions.ToSql(someQuery);
// string sql = Microsoft.EntityFrameworkCore.IQueryableExtensions1.ToSql(someQuery);
// using Microsoft.EntityFrameworkCore;
string sql = someQuery.ToSql();
System.Console.WriteLine(sql);
And then these extension methods (IQueryableExtensions1 for .NET Core 1.0, IQueryableExtensions for .NET Core 2.0) :
using System;
using System.Linq;
using System.Reflection;
using Microsoft.EntityFrameworkCore.Internal;
using Microsoft.EntityFrameworkCore.Query;
using Microsoft.EntityFrameworkCore.Query.Internal;
using Microsoft.EntityFrameworkCore.Storage;
using Remotion.Linq.Parsing.Structure;
namespace Microsoft.EntityFrameworkCore
{
// https://stackoverflow.com/questions/1412863/how-do-i-view-the-sql-generated-by-the-entity-framework
// http://rion.io/2016/10/19/accessing-entity-framework-core-queries-behind-the-scenes-in-asp-net-core/
public static class IQueryableExtensions
{
private static readonly TypeInfo QueryCompilerTypeInfo = typeof(QueryCompiler).GetTypeInfo();
private static readonly FieldInfo QueryCompilerField = typeof(EntityQueryProvider).GetTypeInfo().DeclaredFields
.First(x => x.Name == "_queryCompiler");
private static readonly PropertyInfo NodeTypeProviderField =
QueryCompilerTypeInfo.DeclaredProperties.Single(x => x.Name == "NodeTypeProvider");
private static readonly MethodInfo CreateQueryParserMethod =
QueryCompilerTypeInfo.DeclaredMethods.First(x => x.Name == "CreateQueryParser");
private static readonly FieldInfo DataBaseField =
QueryCompilerTypeInfo.DeclaredFields.Single(x => x.Name == "_database");
private static readonly PropertyInfo DatabaseDependenciesField =
typeof(Database).GetTypeInfo().DeclaredProperties.Single(x => x.Name == "Dependencies");
public static string ToSql<TEntity>(this IQueryable<TEntity> query) where TEntity : class
{
if (!(query is EntityQueryable<TEntity>) && !(query is InternalDbSet<TEntity>))
{
throw new ArgumentException("Invalid query");
}
var queryCompiler = (QueryCompiler) QueryCompilerField.GetValue(query.Provider);
var nodeTypeProvider = (INodeTypeProvider) NodeTypeProviderField.GetValue(queryCompiler);
var parser = (IQueryParser) CreateQueryParserMethod.Invoke(queryCompiler, new object[] {nodeTypeProvider});
var queryModel = parser.GetParsedQuery(query.Expression);
var database = DataBaseField.GetValue(queryCompiler);
var databaseDependencies = (DatabaseDependencies) DatabaseDependenciesField.GetValue(database);
var queryCompilationContext = databaseDependencies.QueryCompilationContextFactory.Create(false);
var modelVisitor = (RelationalQueryModelVisitor) queryCompilationContext.CreateQueryModelVisitor();
modelVisitor.CreateQueryExecutor<TEntity>(queryModel);
var sql = modelVisitor.Queries.First().ToString();
return sql;
}
}
public class IQueryableExtensions1
{
private static readonly TypeInfo QueryCompilerTypeInfo = typeof(QueryCompiler).GetTypeInfo();
private static readonly FieldInfo QueryCompilerField = typeof(EntityQueryProvider).GetTypeInfo()
.DeclaredFields
.First(x => x.Name == "_queryCompiler");
private static readonly PropertyInfo NodeTypeProviderField =
QueryCompilerTypeInfo.DeclaredProperties.Single(x => x.Name == "NodeTypeProvider");
private static readonly MethodInfo CreateQueryParserMethod =
QueryCompilerTypeInfo.DeclaredMethods.First(x => x.Name == "CreateQueryParser");
private static readonly FieldInfo DataBaseField =
QueryCompilerTypeInfo.DeclaredFields.Single(x => x.Name == "_database");
private static readonly FieldInfo QueryCompilationContextFactoryField = typeof(Database).GetTypeInfo()
.DeclaredFields.Single(x => x.Name == "_queryCompilationContextFactory");
public static string ToSql<TEntity>(IQueryable<TEntity> query) where TEntity : class
{
if (!(query is EntityQueryable<TEntity>) && !(query is InternalDbSet<TEntity>))
{
throw new ArgumentException("Invalid query");
}
var queryCompiler = (IQueryCompiler) QueryCompilerField.GetValue(query.Provider);
var nodeTypeProvider = (INodeTypeProvider) NodeTypeProviderField.GetValue(queryCompiler);
var parser =
(IQueryParser) CreateQueryParserMethod.Invoke(queryCompiler, new object[] {nodeTypeProvider});
var queryModel = parser.GetParsedQuery(query.Expression);
var database = DataBaseField.GetValue(queryCompiler);
var queryCompilationContextFactory =
(IQueryCompilationContextFactory) QueryCompilationContextFactoryField.GetValue(database);
var queryCompilationContext = queryCompilationContextFactory.Create(false);
var modelVisitor = (RelationalQueryModelVisitor) queryCompilationContext.CreateQueryModelVisitor();
modelVisitor.CreateQueryExecutor<TEntity>(queryModel);
var sql = modelVisitor.Queries.First().ToString();
return sql;
}
}
}
Well, I am using Express profiler for that purpose at the moment, the drawback is that it only works for MS SQL Server. You can find this tool here: https://expressprofiler.codeplex.com/
While there are good answers here, none solved my problem completely (I wished to get the entire SQL statement, including Parameters, from the DbContext from any IQueryable. The following code does just that. It is a combination of code snippets from Google. I have only tested it with EF6+.
Just an aside, this task took me way longer than I thought it would. Abstraction in Entity Framework is a bit much, IMHO.
First the using. You will need an explicit reference to 'System.Data.Entity.dll'.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Data.SqlClient;
using System.Data.Common;
using System.Data.Entity.Core.Objects;
using System.Data.Entity;
using System.Data;
using System.Data.Entity.Infrastructure;
using System.Reflection;
The following class converts an IQueryable into a DataTable. Modify as your need may be:
public class EntityFrameworkCommand
{
DbContext Context;
string SQL;
ObjectParameter[] Parameters;
public EntityFrameworkCommand Initialize<T>(DbContext context, IQueryable<T> query)
{
Context = context;
var dbQuery = query as DbQuery<T>;
// get the IInternalQuery internal variable from the DbQuery object
var iqProp = dbQuery.GetType().GetProperty("InternalQuery", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
var iq = iqProp.GetValue(dbQuery, null);
// get the ObjectQuery internal variable from the IInternalQuery object
var oqProp = iq.GetType().GetProperty("ObjectQuery", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
var objectQuery = oqProp.GetValue(iq, null) as ObjectQuery<T>;
SQL = objectQuery.ToTraceString();
Parameters = objectQuery.Parameters.ToArray();
return this;
}
public DataTable GetData()
{
DataTable dt = new DataTable();
var connection = Context.Database.Connection;
var state = connection.State;
if (!(state == ConnectionState.Open))
connection.Open();
using (var cmd = connection.CreateCommand())
{
cmd.CommandText = SQL;
foreach (var p in Parameters)
{
var param = cmd.CreateParameter();
param.Name = "#" + p.Name;
param.Value = p.Value;
cmd.Parameters.Add(param);
}
using (var da = DbProviderFactories.GetFactory(connection).CreateDataAdapter())
{
da.SelectCommand = cmd;
da.Fill(dt);
}
}
if (!(state == ConnectionState.Open))
connection.Close();
return dt;
}
}
To use, simply call it as below:
var context = new MyContext();
var data = ....//Query, return type can be anonymous
.AsQueryable();
var dt = new EntityFrameworkCommand()
.Initialize(context, data)
.GetData();
For me, using EF6 and Visual Studio 2015 I entered query in the immediate window and it gave me the generated SQL Statement
In my case for EF 6+, instead of using this in the Immediate Window to find the query string:
var sql = ((System.Data.Entity.Core.Objects.ObjectQuery)query).ToTraceString();
I ended up having to use this to get the generated SQL command:
var sql = ((System.Data.Entity.Infrastructure.DbQuery<<>f__AnonymousType3<string,string,string,short,string>>)query).ToString();
Of course your anonymous type signature might be different.
HTH.
I've just done this:
IQueryable<Product> query = EntitySet.Where(p => p.Id == id);
Debug.WriteLine(query);
And the result shown in the Output:
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Code] AS [Code],
[Extent1].[Name] AS [Name],
[Extent2].[Id] AS [Id1],
[Extent2].[FileName] AS [FileName],
FROM [dbo].[Products] AS [Extent1]
INNER JOIN [dbo].[PersistedFiles] AS [Extent2] ON [Extent1].[PersistedFileId] = [Extent2].[Id]
WHERE [Extent1].[Id] = #p__linq__0
Entity Framework 4 Solution
Most of the answers here were EF6-specific. Here's one for those of you still using EF4.
This method replaces the #p__linq__0/etc. parameters with their actual values, so you can just copy and paste the output into SSMS and run it or debug it.
/// <summary>
/// Temporary debug function that spits out the actual SQL query LINQ is generating (with parameters)
/// </summary>
/// <param name="q">IQueryable object</param>
private string Debug_GetSQLFromIQueryable<T>(IQueryable<T> q)
{
System.Data.Objects.ObjectQuery oq = (System.Data.Objects.ObjectQuery)q;
var result = oq.ToTraceString();
List<string> paramNames = new List<string>();
List<string> paramVals = new List<string>();
foreach (var parameter in oq.Parameters)
{
paramNames.Add(parameter.Name);
paramVals.Add(parameter.Value == null ? "NULL" : ("'" + parameter.Value.ToString() + "'"));
}
//replace params in reverse order, otherwise #p__linq__1 incorrectly replaces #p__linq__10 for instance
for (var i = paramNames.Count - 1; i >= 0; i--)
{
result = result.Replace("#" + paramNames[i], paramVals[i]);
}
return result;
}
Starting from Entity Framework Core 5.0+ one can simply override the OnConfiguring method in the DbContext once for logging. This works also for Single() or Any() queries.
For logging to debug window:
public class ExampleDbContext : DbContext
{
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
// using System.Diagnostics;
optionsBuilder.LogTo(message => Debug.WriteLine(message));
}
}
For logging to console:
public class ExampleDbContext : DbContext
{
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.LogTo(Console.WriteLine);
}
}
See here for more details including log levels and filtering: https://learn.microsoft.com/en-us/ef/core/logging-events-diagnostics/simple-logging
If you want to have parameter values (not only #p_linq_0 but also their values) too, you can use IDbCommandInterceptor and add some logging to ReaderExecuted method.