I know this has been covered to some extend in other chains, but even after interpolating it to my needs, I'm still having issues. As many of you know documentation is virtually non-existent. Therefore, any help is greatly appreciated.
I'm trying to create a generic method which merely generates a DB script from a DLL containing EF entities. Then, I pass a DLL path and a name of the connection string defined in App.config. The first time through, the script is generated fine. I run it in SQLServer Studio and it generates all the tables, etc. However, when I run it the 2nd time (DB exists), it fails on the ScriptUpdate method with the following error:
Failed to set database initializer of type 'Disabled' for DbContext type 'AdminovateLibrary.Repository.EntityDbContext, Adminovate.AdminovateLibrary-Project' specified in the application configuration. See inner exception for details.
Inner exception is: "Could not load file or assembly 'MyLibrary' or one of its dependencies. The system cannot find the file specified.":"MyLibrary".
I assume that if the DLL contains exactly the same code as in the 1st run, the method should product an empty string. However, even if I run it with a modified DLL, it gives me the same error.
My code is as follows:
public string GenerateUpdateSchemaScript( string sourceDllFilePath, string targetConnectionName ) {
var dbMigrationsConfiguration = CreateConfiguration( sourceDllFilePath, targetConnectionName );
var dbMigrator = new DbMigrator( dbMigrationsConfiguration );
Database.SetInitializer( new CreateDatabaseIfNotExists<DbContext>() );
var scriptor = new MigratorScriptingDecorator( dbMigrator );
var script = scriptor.ScriptUpdate( null, null );
return RemoveCreateMigrationHistoryTable( dbMigrationsConfiguration, script );
}
private static DbMigrationsConfiguration CreateConfiguration2( string sourceDllFilePath, string targetConnectionName ) {
var assembly = Assembly.LoadFrom( sourceDllFilePath );
var configType = assembly.GetTypes().Single( type => typeof( DbMigrationsConfiguration ).IsAssignableFrom( type ) );
var configuration = ( DbMigrationsConfiguration )assembly.CreateInstance( configType.FullName );
if( configuration != null ) {
configuration.ContextType = assembly.GetTypes().Single( type => type.BaseType == typeof( DbContext ) );
configuration.MigrationsAssembly = assembly;
configuration.TargetDatabase = new DbConnectionInfo( targetConnectionName );
configuration.AutomaticMigrationsEnabled = true;
configuration.AutomaticMigrationDataLossAllowed = true;
}
return configuration;
}
Related
I have a console program that moves Data between two different servers (DatabaseA and DatabaseB).
Database B is a Postgres-Server.
It calls a lot of stored procedures and other raw queries.
I use ExecuteSqlRaw a lot.
I also use NpsqlBulk.EfCore.
The program uses the same context instance for DatabaseB during the whole run it takes to finish.
Somehow i get locks on some of my tables on DatabaseB that never get released.
This happens always on my table mytable_fromdatabase_import.
The code run on that is the following:
protected override void AddIdsNew()
{
var toAdd = IdsNotInDatabaseB();
var newObjectsToAdd = GetByIds(toAdd).Select(Converter.ConvertAToB);
DatabaseBContext.Database.ExecuteSqlRaw("truncate mytable_fromdatabase_import; ");
var uploader = new NpgsqlBulkUploader(DatabaseBContext);
uploader.Insert(newObjectsToAdd); // inserts data into mytable_fromdatabase_import
DatabaseBContext.Database.ExecuteSqlRaw("call insert_myTable_from_importTable();");
}
After i run it the whole table is not accessable annymore and when i query the locks on the server i can see there is a process holding it.
How can i make sure this process always closes and releases its locks on tables?
I thought ef-core would do that automaticaly.
-----------Edit-----------
I just wanted to add that this is not a temporary problem during the run of the console. When i run this code and it is finished my table is still locked and nothing can access it. My understanding was that the ef-core context would release everything after it is disposed (if by error or by being finished)
The problem had nothing to do with ef core but with a wrong configured backupscript. The program is running now with no changes to it and it works fine
For concrete task you need right tools. Probably you have locks when retrieve Ids and also when trying to do not load already imported records. These steps are slow!
I would suggest to use linq2db (disclaimer, I'm co-author of this library)
Create two projects with models from different databases:
Source.Model.csproj - install linq2db.SQLServer
Destination.Model.csproj - install linq2db.PostgreSQL
Follow instructions in T4 templates how to generate model from two databases. It is easy and you can ask questions on linq2db`s github site.
I'll post helper class which I've used for transferring tables on my previous project. It additionally uses library CodeJam for mapping, but in your project, for sure, you can use Automapper.
public class DataImporter
{
private readonly DataConnection _source;
private readonly DataConnection _destination;
public DataImporter(DataConnection source, DataConnection destination)
{
_source = source;
_destination = destination;
}
private long ImportDataPrepared<TSource, TDest>(IOrderedQueryable<TSource> source, Expression<Func<TSource, TDest>> projection) where TDest : class
{
var destination = _destination.GetTable<TDest>();
var tableName = destination.TableName;
var sourceCount = source.Count();
if (sourceCount == 0)
return 0;
var currentCount = destination.Count();
if (currentCount > sourceCount)
throw new Exception($"'{tableName}' what happened here?.");
if (currentCount >= sourceCount)
return 0;
IQueryable<TSource> sourceQuery = source;
if (currentCount > 0)
sourceQuery = sourceQuery.Skip(currentCount);
var projected = sourceQuery.Select(projection);
var copied =
_destination.BulkCopy(
new BulkCopyOptions
{
BulkCopyType = BulkCopyType.MultipleRows,
RowsCopiedCallback = (obj) => RowsCopiedCallback(obj, currentCount, sourceCount, tableName)
}, projected);
return copied.RowsCopied;
}
private void RowsCopiedCallback(BulkCopyRowsCopied obj, int currentRows, int totalRows, string tableName)
{
var percent = (currentRows + obj.RowsCopied) / (double)totalRows * 100;
Console.WriteLine($"Copied {percent:N2}% \tto {tableName}");
}
public class ImporterHelper<TSource>
{
private readonly DataImporter _improrter;
private readonly IOrderedQueryable<TSource> _sourceQuery;
public ImporterHelper(DataImporter improrter, IOrderedQueryable<TSource> sourceQuery)
{
_improrter = improrter;
_sourceQuery = sourceQuery;
}
public long To<TDest>() where TDest : class
{
var mapperBuilder = new MapperBuilder<TSource, TDest>();
return _improrter.ImportDataPrepared(_sourceQuery, mapperBuilder.GetMapper().GetMapperExpressionEx());
}
public long To<TDest>(Expression<Func<TSource, TDest>> projection) where TDest : class
{
return _improrter.ImportDataPrepared(_sourceQuery, projection);
}
}
public ImporterHelper<TSource> ImprortData<TSource>(IOrderedQueryable<TSource> source)
{
return new ImporterHelper<TSource>(this, source);
}
}
So begin transferring. Note that I have used OrderBy/ThenBy to specify Id order to do not import already transferred records - important order fields should be Unique Key combination. So this sample is reentrant and can be re-run again when connection is lost.
var sourceBuilder = new LinqToDbConnectionOptionsBuilder();
sourceBuilder.UseSqlServer(SourceConnectionString);
var destinationBuilder = new LinqToDbConnectionOptionsBuilder();
destinationBuilder.UsePostgreSQL(DestinationConnectionString);
using (var source = new DataConnection(sourceBuilder.Build()))
using (var destination = new DataConnection(destinationBuilder.Build()))
{
var dataImporter = new DataImporter(source, destination);
dataImporter.ImprortData(source.GetTable<Source.Model.FirstTable>()
.OrderBy(e => e.Id1)
.ThenBy(e => e.Id2))
.To<Dest.Model.FirstTable>();
dataImporter.ImprortData(source.GetTable<Source.Model.SecondTable>().OrderBy(e => e.Id))
.To<Dest.Model.SecondTable>();
}
For sure boring part with OrderBy can be generated automatically, but this will explode this already not a short answer.
Also play with BulkCopyOptions. Native Npgsql COPY may fail and Multi-Line variant should be used.
I'm trying to use repository pattern for my vsto project.
How do I use a repository pattern to execute a stored procedure? I'm using Entity Framework. Any link for code sample would really useful
To your generic repository add
public IEnumerable<T> ExecWithStoreProcedure(string query, params object[] parameters)
{
return _context.Database.SqlQuery<T>(query, parameters);
}
And then you can call it with any unitofwork/repository like
IEnumerable<Products> products =
_unitOfWork.ProductRepository.ExecWithStoreProcedure(
"spGetProducts #bigCategoryId",
new SqlParameter("bigCategoryId", SqlDbType.BigInt) { Value = categoryId }
);
A non generic solution in your repository would be:
private int ExecWithStoreProcedure(string query, params object[] parameters)
{
return _context.Database.ExecuteSqlCommand("EXEC " + query, parameters);
}
And then a few typical examples of use:
var param = new SqlParameter("SomethingToCheck", SqlDbType.NVarChar) { Value = shortCode };
var result = ExecWithStoreProcedure("mySchema.myStoredProc #SomethingToCheck", param);
with multiple parameters:
var param1 = new SqlParameter("SomeCode", SqlDbType.VarChar) { Value = shortCode };
var param2 = new SqlParameter("User", SqlDbType.VarChar) { Value = userName };
var result = ExecWithStoreProcedure("mySchema.myStoredProc #SomeCode, #User", param1, param2 );
this link guided me. [Link]
But when you execute stored procedure you have to put "exec" informant of SP name
Eg: if sp is "sp_aa"
string should be "exec sp_aa"
trying to execute the stored proc in EF using the following code:
var params = new object[] {new SqlParameter("#FirstName", "Bob")};
return this._repositoryContext.ObjectContext.ExecuteStoreQuery<ResultType>("GetByName", params);
but keep getting this error:
Procedure or function 'GetByName' expects parameter '#FirstName',
which was not supplied.
and from sql profiler:
exec sp_executesql N'GetByName',N'#FirstName nvarchar(100),#FirstName=N'Bob'
what is wrong wit the above ExecuteStoreQuery code?
Ignoring the fact that params is a reserved word...
Think your query needs to be:
var params = new object[] {new SqlParameter("#FirstName", "Bob")};
return this._repositoryContext.ObjectContext.ExecuteStoreQuery<ResultType>("exec GetByName #FirstName", params);
Should also say that if that proc is a standard part of your database and data model then you should import it into your EDM so it's available directly on your context.
Use the ExecuteFunction instead of ExecuteStoreQuery which is more suitable for the "ad-hoc" queries.
var parameters = new ObjectParameter[] {new ObjectParameter("FirstName", "Bob")};
return this._repositoryContext.ObjectContext.ExecuteFunction<ResultType>("GetByName", parameters);
The stored procedures can also be mapped as function in the context and thus can be used as typed method. Take a look at Using stored procedures with Entity Framework.
This is what I did to use a SP in EF, if you have multiple parameters:-
public virtual ObjectResult<GetEpisodeCountByPracticeId_Result> GetEpisodeCountByPracticeId(Nullable<int> practiceId, Nullable<System.DateTime> dat1)
{
SqlParameter practiceIdParameter = practiceId.HasValue ?
new SqlParameter() { ParameterName = "practiceId", Value = practiceId, SqlDbType = SqlDbType.Int } :
new SqlParameter() { ParameterName = "practiceId", SqlDbType = SqlDbType.Int };
SqlParameter dat1Parameter = dat1.HasValue ?
new SqlParameter() { ParameterName = "dat1", Value = dat1, SqlDbType = SqlDbType.DateTime }:
new SqlParameter() { ParameterName = "dat1", SqlDbType = SqlDbType.DateTime };
return ((IObjectContextAdapter)this).ObjectContext.ExecuteStoreQuery<GetEpisodeCountByPracticeId_Result>("exec GetEpisodeCountByPracticeId #practiceId, #dat1", practiceIdParameter, dat1Parameter);
}
If you dont add the parameters (e.g. #practiceId) in the commandText property then you get the error you received
I can't find good documentation on these operators. Can someone provide some examples of use and explain what they do?
Entity SQL's CREATEREF reference: http://msdn.microsoft.com/en-us/library/bb386880(v=VS.90)
It's used to "Fabricates references to an entity in an entityset". You can also find references of REF and DEREF from the link.
For VS 2010, the reference is: http://msdn.microsoft.com/en-us/library/bb386880(v=VS.100)
Sample from MSDN:
In the example below, Orders and BadOrders are both entitysets of type
Order, and Id is assumed to be the single key property of Order. The
example illustrates how we may produce a reference to an entity in
BadOrders. Note that the reference may be dangling. That is, the
reference may not actually identify a specific entity. In those cases,
a DEREF operation on that reference returns a null.
select CreateRef(LOB.BadOrders, row(o.Id))
from LOB.Orders as o
Sample code of using entity framework SQL:
using (EntityConnection conn =
new EntityConnection("name=AdventureWorksEntities"))
{
conn.Open();
// Create a query that takes two parameters.
string esqlQuery =
#"SELECT VALUE Contact FROM AdventureWorksEntities.Contact
AS Contact WHERE Contact.LastName = #ln AND
Contact.FirstName = #fn";
try
{
using (EntityCommand cmd = new EntityCommand(esqlQuery, conn))
{
// Create two parameters and add them to
// the EntityCommand's Parameters collection
EntityParameter param1 = new EntityParameter();
param1.ParameterName = "ln";
param1.Value = "Adams";
EntityParameter param2 = new EntityParameter();
param2.ParameterName = "fn";
param2.Value = "Frances";
cmd.Parameters.Add(param1);
cmd.Parameters.Add(param2);
using (DbDataReader rdr = cmd.ExecuteReader(CommandBehavior.SequentialAccess))
{
// Iterate through the collection of Contact items.
while (rdr.Read())
{
Console.WriteLine(rdr["FirstName"]);
Console.WriteLine(rdr["LastName"]);
}
}
}
}
catch (EntityException ex)
{
Console.WriteLine(ex.ToString());
}
conn.Close();
}
I need to change the storage schema of the entities on runtime.
I've followed a wonderful post, available here:
http://blogs.microsoft.co.il/blogs/idof/archive/2008/08/22/change-entity-framework-storage-db-schema-in-runtime.aspx?CommentPosted=true#commentmessage
This works perfectly, but only for queries, not for modifications.
Any idea why?
Well, I was looking for this piece of code all around the Internet. In the end I had to do it myself. It's based on Brandon Haynes adapter, but this function is all you need to change the schema on runtime - and you don't need to replace the autogenerated context constructors.
public static EntityConnection Create(
string schema, string connString, string model)
{
XmlReader[] conceptualReader = new XmlReader[]
{
XmlReader.Create(
Assembly
.GetExecutingAssembly()
.GetManifestResourceStream(model + ".csdl")
)
};
XmlReader[] mappingReader = new XmlReader[]
{
XmlReader.Create(
Assembly
.GetExecutingAssembly()
.GetManifestResourceStream(model + ".msl")
)
};
var storageReader = XmlReader.Create(
Assembly
.GetExecutingAssembly()
.GetManifestResourceStream(model + ".ssdl")
);
XNamespace storageNS = "http://schemas.microsoft.com/ado/2009/02/edm/ssdl";
var storageXml = XElement.Load(storageReader);
foreach (var entitySet in storageXml.Descendants(storageNS + "EntitySet"))
{
var schemaAttribute = entitySet.Attributes("Schema").FirstOrDefault();
if (schemaAttribute != null)
{
schemaAttribute.SetValue(schema);
}
}
storageXml.CreateReader();
StoreItemCollection storageCollection =
new StoreItemCollection(
new XmlReader[] { storageXml.CreateReader() }
);
EdmItemCollection conceptualCollection = new EdmItemCollection(conceptualReader);
StorageMappingItemCollection mappingCollection =
new StorageMappingItemCollection(
conceptualCollection, storageCollection, mappingReader
);
var workspace = new MetadataWorkspace();
workspace.RegisterItemCollection(conceptualCollection);
workspace.RegisterItemCollection(storageCollection);
workspace.RegisterItemCollection(mappingCollection);
var connectionData = new EntityConnectionStringBuilder(connString);
var connection = DbProviderFactories
.GetFactory(connectionData.Provider)
.CreateConnection();
connection.ConnectionString = connectionData.ProviderConnectionString;
return new EntityConnection(workspace, connection);
}
The resulting EntityConnection should be passed as a parameter when instantiating the context. You can modify it, so all ssdl models are modified by this function, not only the specified one.
I've managed to resolve this issue by using a brilliant library, located in CodePlex (courtesy of Brandon Haynes), named "Entity Framework Runtime Model Adapter", available here:
http://efmodeladapter.codeplex.com/
I've tweaked it a bit, to fit our needs and without the need of replacing the designer code at all.
So, I'm good.
Thanks anyways, and especially to Brandon, amazing job!
I need import data from postgres database. It by default use schema "public". So I use Entity Framework CTP 4 "Code first". It by default use schema "dbo". To change it in runtime I use:
public class PublicSchemaContext : DbContext
{
protected override void OnModelCreating(System.Data.Entity.ModelConfiguration.ModelBuilder builder)
{
builder.Entity<series_categories>().MapSingleType().ToTable("[public].[series_categories]");
}
public DbSet<series_categories> series_categories { get; set; }
}
It work for select, insert, update and delete data. So next test in pass:
[Test]
public void AccessToPublicSchema()
{
// Select
var db = new PublicSchemaContext();
var list = db.series_categories.ToList();
Assert.Greater(list.Count, 0);
Assert.IsNotNull(list.First().series_category);
// Delete
foreach (var item in db.series_categories.Where(c => c.series_category == "Test"))
db.series_categories.Remove(item);
db.SaveChanges();
// Insert
db.series_categories.Add(new series_categories { series_category = "Test", series_metacategory_id = 1 });
db.SaveChanges();
// Update
var test = db.series_categories.Single(c => c.series_category == "Test");
test.series_category = "Test2";
db.SaveChanges();
// Delete
foreach (var item in db.series_categories.Where(c => c.series_category == "Test2"))
db.series_categories.Remove(item);
db.SaveChanges();
}
Not an answer per se but a followup on Jan Matousek's Create[EntityConnection] method showing how to use from a DbContext. Note DB is the DbContext type passed to the generic repository.
public TxRepository(bool pUseTracking, string pServer, string pDatabase, string pSchema="dbo")
{
// make our own EF database connection string using server and database names
string lConnectionString = BuildEFConnectionString(pServer, pDatabase);
// do nothing special for dbo as that is the default
if (pSchema == "dbo")
{
// supply dbcontext with our connection string
mDbContext = Activator.CreateInstance(typeof(DB), lConnectionString) as DB;
}
else // change the schema in the edmx file before we use it!
{
// Create an EntityConnection and use that to create an ObjectContext,
// then that to create a DbContext with a different default schema from that specified for the edmx file.
// This allows us to have parallel tables in the database that we can make available using either schema or synonym renames.
var lEntityConnection = CreateEntityConnection(pSchema, lConnectionString, "TxData");
// create regular ObjectContext
ObjectContext lObjectContext = new ObjectContext(lEntityConnection);
// create a DbContext from an existing ObjectContext
mDbContext = Activator.CreateInstance(typeof(DB), lObjectContext, true) as DB;
}
// finish EF setup
SetupAndOpen(pUseTracking);
}
I was able to convert the solution from Jan Matousek to work in vb.net 2013 with entity framework 6. I will also try to explain how to use the code in vb.net.
We have a JD Edwards Database which uses different Schema's for each environment (TESTDTA, CRPDTA, PRODDTA). This makes switching between environments cumbersome as you have to manually modify the .edmx file if you want to change environments.
First step is to create a partial class that allows you to pass a value to the constructor of your entities, by default it uses the values from your config file.
Partial Public Class JDE_Entities
Public Sub New(ByVal myObjectContext As ObjectContext)
MyBase.New(myObjectContext, True)
End Sub
End Class
Next create the function that will modify your store schema .ssdl file in memory.
Public Function CreateObjectContext(ByVal schema As String, ByVal connString As String, ByVal model As String) As ObjectContext
Dim myEntityConnection As EntityConnection = Nothing
Try
Dim conceptualReader As XmlReader = XmlReader.Create(Me.GetType().Assembly.GetManifestResourceStream(model + ".csdl"))
Dim mappingReader As XmlReader = XmlReader.Create(Me.GetType().Assembly.GetManifestResourceStream(model + ".msl"))
Dim storageReader As XmlReader = XmlReader.Create(Me.GetType().Assembly.GetManifestResourceStream(model + ".ssdl"))
Dim storageNS As XNamespace = "http://schemas.microsoft.com/ado/2009/11/edm/ssdl"
Dim storageXml = XDocument.Load(storageReader)
Dim conceptualXml = XDocument.Load(conceptualReader)
Dim mappingXml = XDocument.Load(mappingReader)
For Each myItem As XElement In storageXml.Descendants(storageNS + "EntitySet")
Dim schemaAttribute = myItem.Attributes("Schema").FirstOrDefault
If schemaAttribute IsNot Nothing Then
schemaAttribute.SetValue(schema)
End If
Next
storageXml.Save("storage.ssdl")
conceptualXml.Save("storage.csdl")
mappingXml.Save("storage.msl")
Dim storageCollection As StoreItemCollection = New StoreItemCollection("storage.ssdl")
Dim conceptualCollection As EdmItemCollection = New EdmItemCollection("storage.csdl")
Dim mappingCollection As StorageMappingItemCollection = New StorageMappingItemCollection(conceptualCollection, storageCollection, "storage.msl")
Dim workspace = New MetadataWorkspace()
workspace.RegisterItemCollection(conceptualCollection)
workspace.RegisterItemCollection(storageCollection)
workspace.RegisterItemCollection(mappingCollection)
Dim connectionData = New EntityConnectionStringBuilder(connString)
Dim connection = DbProviderFactories.GetFactory(connectionData.Provider).CreateConnection()
connection.ConnectionString = connectionData.ProviderConnectionString
myEntityConnection = New EntityConnection(workspace, connection)
Return New ObjectContext(myEntityConnection)
Catch ex As Exception
End Try
End Function
Make sure that the storageNS namespace hardcoded value matches the one used in your code, you can view this by debugging the code and examining the storageXML variable to see what was actually used.
Now you can pass a new schema name, and different database connection info at runtime when you create your entities. No more manual .edmx changes required.
Using Context As New JDE_Entities(CreateObjectContext("NewSchemaNameHere", ConnectionString_EntityFramework("ServerName", "DatabaseName", "UserName", "Password"), "JDE_Model"))
Dim myWO = From a In Context.F4801 Where a.WADOCO = 400100
If myWO IsNot Nothing Then
For Each r In myWO
Me.Label1.Text = r.WADL01
Next
End If
End Using
These were the .net libraries used:
Imports System.Data.Entity.Core.EntityClient
Imports System.Xml
Imports System.Data.Common
Imports System.Data.Entity.Core.Metadata.Edm
Imports System.Reflection
Imports System.Data.Entity.Core.Mapping
Imports System.Data.Entity.Core.Objects
Imports System.Data.Linq
Imports System.Xml.Linq
Hope that helps anyone out there with the same issues.
I had a lot of problems getting this to work when using EF6 with an OData Data Service, so I had to find an alternate solution. In my case, I didn't really need to do it on the fly. I could get away with changing the schema when deploying to some test environments, and in the installer.
Use Mono.Cecil to rewrite the embedded .ssdl resources straight in the DLLs. This works just fine in my case.
Here is a simplified example of how you can do this:
var filename = "/path/to/some.dll"
var replacement = "Schema=\"new_schema\"";
var module = ModuleDefinition.ReadModule(filename);
var ssdlResources = module.Resources.Where(x => x.Name.EndsWith(".ssdl"));
foreach (var resource in ssdlResources)
{
var item = (EmbeddedResource)resource;
string rewritten;
using (var reader = new StreamReader(item.GetResourceStream()))
{
var text = reader.ReadToEnd();
rewritten = Regex.Replace(text, "Schema=\"old_schema\"", replacement);
}
var bytes = Encoding.UTF8.GetBytes(rewritten);
var newResource = new EmbeddedResource(item.Name, item.Attributes, bytes);
module.Resources.Remove(item);
module.Resources.Add(newResource);
}