I am trying to Seed (using an external recource, which is CSV file) the Azure SQL database associated with an Azure Website.
I am able to Seed the database in development environment using EF Migration Seed method and a CSV file as defined as follows in the Migration.cs file. Note: the CSV file in the project in Visual Studio are set to the Build Action to Embedded Resource.
public void SeedData(WebApp.Models.ApplicationDbContext Context)
{
Assembly assembly = Assembly.GetExecutingAssembly();
string resourceName = "WebApp.SeedData.Name.csv";
using (Stream stream = assembly.GetManifestResourceStream(resourceName))
{
using (StreamReader reader = new StreamReader(stream, Encoding.UTF8))
{
CsvReader csvReader = new CsvReader(reader);
csvReader.Configuration.WillThrowOnMissingField = false;
var records = csvReader.GetRecords<Products>().ToArray();
foreach (Product record in records)
{
Context.Products.AddOrUpdate(record);
}
}
}
Context.SaveChanges();
}
When I deploy to Azure from VS2013 and select Execute Code First Migration, the database does not get seeded.
UPDATE
This is now working. After I did a clean build, then built the project, and then deployed the the web site with selecting Execute Code First Migration.
Related
I'm trying to use Liquibase version 3.6 to maintain database version changes. I'm able to execute database version changes when i need to execute single file changes.
I have using below code to execute version changes successfully, but my intention is to dynamically execute the change logs from a folder and not single file. I'm using only Java for all configuration of Liquibase
#Startup
#Singleton
#TransactionManagement(TransactionManagementType.BEAN)
public class InitializerBean {
#Resource(mappedName = "java:/M2M_RESOURCES")
private DataSource ds;
#PostConstruct
protected void bootstrap() {
ResourceAccessor resourceAccessor = new ClassLoaderResourceAccessor(getClass().getClassLoader());
try (Connection connection = ds.getConnection()) {
JdbcConnection jdbcConnection = new JdbcConnection(connection);
Database db = DatabaseFactory.getInstance().findCorrectDatabaseImplementation(jdbcConnection);
Liquibase liquiBase = new Liquibase("data/liquibase/", resourceAccessor, db);
liquiBase.update("Development");
} catch (SQLException | LiquibaseException e) {
}
}
}
When the first parameter of Liquibase class is single file , liquibase is able to execute changes but when I intent to execute all file of single folder is not able to track and execute changes.
I'm using JDK 1.8 and file in data/liquibase/ is dbChangelog.sql and dbChangelog_2.sq. This all code is deployed on Wildfly 10 as part of ear archive
I would like to create a database with entity Framework core that would be automatically added to my azure elactic pool.
I do that with a DatabaseFacadeExtension that execute SQL command after the db creation like suggested here:
Azure SQL Server Elastic Pool - automatically add database to pool
public static async Task<bool> EnsureCreatedAsync(this DatabaseFacade databaseFacade, string elasticPoolName, CancellationToken cancellationToken = default)
{
if (!await databaseFacade.EnsureCreatedAsync(cancellationToken)) return false;
// the database has been created.
var dbName = databaseFacade.GetDbConnection().Database;
try
{
cancellationToken.ThrowIfCancellationRequested();
if (!string.IsNullOrEmpty(elasticPoolName))
{
await databaseFacade.ExecuteSqlCommandAsync(new RawSqlString(
$"ALTER DATABASE {dbName} MODIFY ( SERVICE_OBJECTIVE = ELASTIC_POOL (name = [{elasticPoolName}] ));"),
cancellationToken);
}
return true;
}
catch
{
await databaseFacade.EnsureDeletedAsync(cancellationToken);
throw;
}
}
It's works but I will prefer an atomic operation where the database would be created directly in the Azure Elastic Pool.
I had a very similar issue. Fortunately, I took cues from the previous answer and I improvised on it to arrive at a solution.
I had a common database to manage the application and whenever a new client onboards, I need to create a new database. So, I had to maintain multiple database contexts in my .NET core application. Also, I had migrations for the clientContext ready in my codebase, which just needed
client_db.Database.MigrateAsync();
to create the database. But, I couldn't create it directly under elastic pool as Azure doesn't have any default settings which support that. So, MigrateAsync always created the database outside the pool.
So, I created the database within the pool using T-SQL command from my common database context, followed by MigrateAsync() to migrate all the required schema.
var commandText = "CREATE DATABASE client1 ( SERVICE_OBJECTIVE = ELASTIC_POOL ( name = demoPool ) );";
db.Database.ExecuteSqlCommand(commandText);
clientContext client_db = new clientContext(approved_corporate.Id, _configuration);
client_db.Database.MigrateAsync();
Also I had a custom Constructor in my clientContext to support this:
public clientContext(int client_id, IConfiguration configuration = null)
{
string client_code = "client" + client_id.ToString();
connection_string = configuration["ConnectionStrings:Client"].ToString().Replace("client_code", client_code);
}
Azure Elastic Pool supports you creates a new database in an existing pool or as a single database. You must be connected to the master database to create a new database.
For more details, please see: Transact-SQL: Manage pooled databases.
Example T-SQL Code:
Creating a Database in an Elastic Pool:
CREATE DATABASE db1 ( SERVICE_OBJECTIVE = ELASTIC_POOL ( name = S3M100 ) ) ;
Please see: Azure SQL Database single database/elastic pool
You can replace the T-SQL statement and try again.
Hope this helps
I have successfully configured spring boot with a new project to work
with flyway
Migrated with the Postgres database from the version 0001.0 to 0008.0
I have made manually alter the script in local but
flyway migration getting failed.
Sample Error message:
org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'flywayInitializer' defined in class path
resource
[org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]:
Invocation of init method failed; nested exception is
org.flywaydb.core.api.FlywayException: Validate failed: Migration
checksum mismatch for migration version 0006.0
How to alter the database tables without affecting flyway script from the flyway_schema_history?
For example, I need to change the table name using alter command but executing the flyway migration script without failed.
Any suggestions, Kindly appreciated.
Note:- I don't want to remove the script entries from the table flyway_schema_history.
There are a few ways to do this:-
1) Create a new script file with incremented version. Put your DDL commands to alter the table in this file. Then run migration.
2) If you don't want to delete the entry from the schema_version table, you can change the checksum value in that table. To calculate checksum, use the following method copied from org.flywaydb.core.internal.resolver.sql.SqlMigrationResolver. You can pass null for resource parameter:-
/**
* Calculates the checksum of this string.
*
* #param str The string to calculate the checksum for.
* #return The crc-32 checksum of the bytes.
*/
/* private -> for testing */
static int calculateChecksum(Resource resource, String str) {
final CRC32 crc32 = new CRC32();
BufferedReader bufferedReader = new BufferedReader(new StringReader(str));
try {
String line;
while ((line = bufferedReader.readLine()) != null) {
crc32.update(line.getBytes("UTF-8"));
}
} catch (IOException e) {
String message = "Unable to calculate checksum";
if (resource != null) {
message += " for " + resource.getLocation() + " (" + resource.getLocationOnDisk() + ")";
}
throw new FlywayException(message, e);
}
return (int) crc32.getValue();
}
3) If you are using Flyway Pro version 5+, you can rollback the migration https://flywaydb.org/getstarted/undo.
The answers here are outdated but can still help you.
It sounds like you might be in one of two situations:
You want to re-run a versioned migration. This isn't really how flyway works, as Kartik has suggested, create a new versioned migration to alter the table.
A migration file has been modified and you want to leave it that way and run new ones (eg 0009.0). In this situation you can try:
Run repair. Which will recalculate the checksums (among other things).
Turn off the validateOnMigrate option which will not fail a migration if there are modified migration files.
To solve this error locally without dropping your whole db:
Fix the migration error which caused the root problem
Disconnect your db server
Open the table "flyway_schema_history" which would be created automatically
Delete the rows with the versions that are causing the mismatch problem
Open the tables that have columns depending on the conflict migrations and drop those columns (if needed)
Run again your db server with the new migrations
Code related info:
Refer to Entity Framework 6.13 by NuGet under Visual Studio 2013
Use database-first mode, that is, create .edmx file with Visual Studio template ADO.NET Entity Data Model -> EF Designer from database
My DbContext code:
public partial class MySampleEntities : DbContext
{
public MySampleEntities ()
{
}
public MySampleEntities (string nameOrConnectionString)
: base(nameOrConnectionString)
{
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
throw new UnintentionalCodeFirstException();
}
...
}
One method to modify model with below sample code:
using (var dbContext = GetDbContext())
{
...
dbContext.Entry(blobItem).State = EntityState.Modified;
await dbContext.SaveChangesAsync();
}
Unit Test Cases info:
Use NUnit and Moq
Some Unit Test code:
var mockContext = new Mock<MySampleEntities>();
Test passed with initial database schema, after some database table column names changed, and execute Visual Studio "Update Model from Database..." command on .edmx diagram, update all corresponding model properties, build code, and finally failed to run unit test on code dbContext.Entry(blobItem).State = EntityState.Modified with this exception:
The model backing the 'MySampleEntities' context has changed since the database was created. Consider using Code First Migrations to update the database (http://go.microsoft.com/fwlink/?LinkId=238269).
Solutions tried:
Use Database.SetInitializer(null) in constructors of MySampleEntities
Failed to execute NuGet Package Manager commands like Enable-Migrations, Update-Database etc with below exception, seems it is for code-first mode:
System.Data.Entity.Infrastructure.UnintentionalCodeFirstException: The context is being used in Code First mode with code that was generated from an EDMX file for either Database First or Model First development. This will not work correctly.
My questions:
How to resolve such issue to make unit test case run successfully?
I have a Web API project accessing a database file in its App_Data folder. The database is created using EF Migrations.
I have some integration tests to confirm data structures and queries. They just create a Web API controller and use it to perform queries. My Test Project has an App.config file, which currently has an absolute path to the *.mdf database file (in the web api project).
This is a problem because a) I am doing my tests on the application database, and b) I have an absolute path in my App.config file.
I would like to create a test database in the test project.
Is there some way I can get Migrations to create another database (with a different Seed method) in the Test Project?
In the Web Api project's Web.config connection string we use |DataDirectory| to specify the location of the database. Is there an equivalent in a test project's App.config?
Failing all that, is there some way I can point the test project's connection string to the application db without using an absolute path?
My solution is to have a static DbManager class that does a Setup(), which I will run from the [ClassInitialize] method of the database testing classes. Setup()will:
Change the connection string according to the current AppDomain.CurrentDomain.BaseDirectory (so no longer an absolute path in App.config).
Use other AppSettings from App.config to determine which database to use, which test data set to use, and whether or not to drop and recreate the database.
Some code:
public static class DbManager
{
private static string ErrorMsg { get; set; }
private static ApplicationDbContext _db;
private static readonly Random Rdm = new Random();
public static void SetupDb()
{
Configuration config = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None);
bool refreshDb = Convert.ToBoolean(ConfigurationManager.AppSettings["RefreshDbInTestClassInitialization"]);
string dataSet = ConfigurationManager.AppSettings["DataSetToAddForEachTestClass"];
string dbName = ConfigurationManager.AppSettings["testDatabaseName"];
var connectionStr = config.ConnectionStrings
.ConnectionStrings["DefaultConnection"].ConnectionString
.Replace("{dbPath}", Path.Combine(AppDomain.CurrentDomain.BaseDirectory, (dbName + ".mdf")))
.Replace("{dbName}", dbName);
config.AppSettings.SectionInformation.ForceSave = true;
config.ConnectionStrings.ConnectionStrings["DefaultConnection"].ConnectionString = connectionStr;
config.Save(ConfigurationSaveMode.Modified);
ConfigurationManager.RefreshSection("connectionStrings");
if (refreshDb)
{
Database.Delete(connectionStr);
AddDataSet(dataSet, true);
}
}
// AddDataSet() creates a DbContext object and generates test data.
}
So the connection string in App.config looks like this:
connectionString="Data Source=(LocalDb)\v11.0;AttachDbFilename={dbPath};Initial Catalog={dbName};Integrated Security=True"
Then App.config AppSettings section has these extra key value pairs:
<add key ="RefreshDbInTestClassInitialization" value ="true"/>
<add key="DataSetToAddForEachTestClass" value="Large"/> <!--Possible values: Large, Small-->
<add key="testDatabaseName" value="testDb"/>