I have searched everywhere and most articles I found suggest that to seed data, I have to use the Program Class which would run my seed data code whenever I run the project.
The issue I'm having is I am setting up a CI\CD pipeline in which I need to run a seed data command before I run the project. Other posts have suggested that I have to build my own dotnet cli package to seed data.
Is there a better solution, I would like a solution similar to laravel's database seeding command php artisan db:seed
The better way I've found to seed data separately from running the application is by using the Main method args:
public static void Main(string[] args) {
var host = BuildWebHost(args);
if(args.Length > 0){
bool seed = args.Any(arg => arg == "/seed");
bool start = args.Any(arg => arg == "/start");
if(seed) InitializeDatabase(host);
if(start) host.Run();
}else{
InitializeDatabase(host);
host.Run();
}
}
So to seed separately from running the run dotnet run /seed and to run do dotnet run /start. If you do only dotnet run then it will seed and run.
So if you're having trouble separating seeding of your app from runnning then this is the best solution I've come up with.
Related
For migration management, I've decided to use Prisma Migrate instead of node-pg-migrate (PostgreSQL). I followed the instructions in this link and everything works fine. However, the main challenge is that my integration tests fail when trying to run migrations on the test database (not the develop database). How can I override the configurations of my test database?
In node-pg-migrate I could simply provide configurations before running the integration test:
import migrationRunner from 'node-pg-migrate';
// --- Integration test setup:
beforeAll(async () => {
await migrationRunner({
migrationsTable: dbConfig['migrations-table'],
dir: dbConfig['migrations-dir'],
schema: dbConfig.schema,
databaseUrl: databaseURL,
// --- other configs
});
}, config.get('test').timeout);
You can override the entire connection string directly in your test setup as follows. Prisma reads the DB string from environment variables so overriding in the following manner will work fine.
I'm trying to run migration on Azure DevOps Release Pipeline. Because I want to run my DB scripts automatically on every release. My release pipeline does not have source code, I just have compiled DLLs.
When I execute this command on my local machine, it runs successfully. How can I convert this command so I can use it with DLLs.
dotnet ef database update --project MyEntityFrameworkProject --context MyDbContext --startup-project MyStartupProject
Another approach is to generate migration script (a regular sql script) during build pipeline and make this script a part of your artifact. To do so run following command:
dotnet ef migrations script --output $(build.artifactstagingdirectory)\sql\migrations.sql -i
Note -i flag which makes this script runnable multiple times on the same database
Once you have this script as a part of your artifact you can run it on database in your release pipeline by using Azure SQL Database Deployment built in task.
Check this link for more info
EDIT: As #PartickBorkowicz pointed out there are some issues related to the fact that database is not available in a regular way from Build/Release Agent perspective. Here are some additional tips how to live without a database and connection string stored anywhere in your code.
1. Build pipeline
If you do nothing, an Build Agent will require database connection in order to run dotnet ef migrations script script. But there's one trick you can do which will allow you to work without database and connection string: IDesignTimeDbContextFactory
It's enough that you create class like this:
public class YourDesignTimeContextFactory : IDesignTimeDbContextFactory<YourDbContext>
{
public YourDbContext CreateDbContext(string[] args)
{
var databaseConnectionString = "Data Source=(LocalDB)\\MSSQLLocalDB;Initial Catalog=LocalDB;Integrated Security=True;Pooling=False";
var builder = new DbContextOptionsBuilder<YourDbContext>();
builder.UseSqlServer(databaseConnectionString);
return new YourDbContext(builder.Options);
}
}
Once it is present in your project (you don't need to register it anyhow) your Build Agent will be able to generate sql script with migrations logic without access to actual database
2. Release pipeline
Now, you're having sql script generated and being part of artifact from a build pipeline. Now, release pipeline is the time when you want this script to be run on actual database - you'll need a connection string to this database somehow. To do so in secure manner you should not store it anywhere in the code. A good way to do it is to keep password in Azure Key Vault. There's built in task in Azure Release pipelines called Azure Key Vault. This will fetch your secrets which you can use in next step: Azure SQL Database Deployment. You just need to setup options:
AuthenticationType: Connection String
ConnectionString: `$(SqlServer--ConnectionString)` - this value depends on your secret name in Key Vault
Deploy type: SQL Script File
SQL Script: $(System.DefaultWorkingDirectory)/YourProject/drop/migrations/script.sql - this depends how you setup your artifact in build pipeline.
This way you're able to generate migrations without access to database and run migrations sql without storing your connection string anywhere in the code.
If you don't want to include your source code with the artifacts you can use the following script:
set rootDir=$(System.DefaultWorkingDirectory)\WebApp\drop\WebApp.Web
set efPath=C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.entityframeworkcore.tools\2.1.1\tools\netcoreapp2.0\any\ef.dll
dotnet exec --depsfile "%rootDir%\WebApp.deps.json" --additionalprobingpath %USERPROFILE%\.nuget\packages --additionalprobingpath "C:\Program Files\dotnet\sdk\NuGetFallbackFolder" --runtimeconfig "%rootDir%\WebApp.runtimeconfig.json" "%efpath%" database update --verbose --prefix-output --assembly "%rootDir%\AssemblyContainingDbContext.dll" --startup-assembly "%rootDir%\AssemblyContainingStartup.dll" --working-dir "%rootDir%"
It turns out you can get away with the undocumented dotnet exec command like the following example (assuming the web application is called WebApp):
Note that the Working Directory (hidden under Advanced) of this run command line task must be set to where the artifacts are (rootDir above).
Another option is to install Build & Release Tools extension and use the "Deploy Entity Framework Core Migrations from a DLL" task.
You can read more info here, here and here.
Im trying to use Entity Framework Core with a Code First approach.
So far I have developed some models and my context. I have then added the EntityFrameworkCore.Tools to work with migrations. I have defined my ConnectionString property in the appsettings like follows:
"Data": {
"ConnectionString": "Data Source=localhost;Initial Catalog=TestDb;User Id=user;Password=pass; Pooling=true; Max Pool Size=200; Min Pool Size=5"
}
(The user and password are correct btw)
In my DbContext then I have done the following:
public MyContext()
{
Database.EnsureCreated();
}
// ...
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
var connString = Startup.Configuration["Data:ConnectionString"];
optionsBuilder.UseSqlServer(connString);
base.OnConfiguring(optionsBuilder);
}
When I run the Add-Migration InitialDatabase it works fine, it creates my migration with its tables. And then when I launch the App, because of the Database.EnsureCreated() in MyContext it creates the database for me and it works fine.
The problem is I would like to control my migrations. So I would like to not rely on the EnsureCreated. The thing is when I execute Update-Database the command apparentely works (it doesnt show any error) but it does not create my database. I think I am missing a way to tell Update-Database where is my connection string. I know you can pass a parameter to it but I've seen people using it without parameters. How can I make this work?
Thanks!
You might need to set the ASPNETCORE_ENVIRONMENT variable first using the VS Package Manager and this command:
$env:ASPNETCORE_ENVIRONMENT='Debug'
then run:
Update-Database
If have configured the connection string in appsettings.debug.json, then it should create and initialize the database using your migration.
I have a solution containing two projects. one asp.net core and and class lib dotnet.core. I am using the class lib only to run migrations . I would like to be able to switch the connection string from dev to prod when i invoke "dotnet ef database update". Is there a clean way to do that ?
You can tell entity framework exactly what db context to update by passing the "-c" arg if your using the dotnet cli or the --context command in the package manager console.
dotnet ef database update -c {DbContextName}
or
Update-Database --context {DbContextName}
https://learn.microsoft.com/en-us/aspnet/core/data/ef-mvc/migrations
If you want to change the connection string based on the environment at run time you can use the
dotnet run -e|--environment {EnvironmentName}
Then inside your configure services method inside the Starup.cs method do something like
public IHostingEnvironment hostingEnv;
public void ConfigureServices(IServiceCollection services)
{
if (hostingEnv.IsEnvironment("{EnvironmentName}"))
{
services.AddDbContext<DbContextName>(options =>
options.UseSqlServer(Configuration.GetConnectionString("DbContextNameFromAppSettings.JsonFile")));
}
else
{
services.AddDbContext<DbContextName>(options =>
options.UseSqlServer(Configuration.GetConnectionString("OtherDbContextNameFromAppSettings.JsonFile")));
}
}
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/environments
As you want to change connection string based on Environment type, you may use --environment option:
-e|--environment <environment> The environment to use. If omitted, "Development" is used.
to specify is it Dev or Prod and then override connection string based on this.
I have EF migrations working nicely, but I also want to generate the sql script for the seed data from my DbMigrationsConfiguration class.
The seed data runs ok when I do Update-Database, but when I do UpdateDatabase -Script I do not get the sql for the seed inserts. I tried -Verbose on a normal Update-Database but I do not see the seed statements output there either.
Is this possible?
No it is not possible. Configuration class is not part of migration itself - it is infrastructure executing the migration. You have single configuration class for all your migrations and its Seed method is executed after every migration run - you can even use context for seeding data and because of that this method is executed after the migration is completed = it cannot be part of migration. Only content of the migration class is scripted.
Whether you are using EF or EF Core, a solution/workaround is to have SSMS generate the seed script for you:
Start with a clean database generated by your DB initializer and seed method. Make sure the data you want scripted is in there.
Using SSMS, right-click the database, go to Tasks > "Generate Scripts...", and follow the wizard. Under Advanced options, be sure to select "Data only" for "Types of data to script".
From the generated script, copy required seed statements over to your target script.
I know it's bit of an old thread but, here is an answer that could help someone else looking for an answer.
You can use the Migrate.exe supplied by Entity Framework. This will allow you to run the Seed method on the database context.
If you need to run a specific Seed method you can place that in a separate migration config file like this:
Enable-Migrations -MigrationsDirectory "Migrations\ContextA" -ContextTypeName MyProject.Models.ContextA
Command:
Migrate.exe MyAssembly CustomConfig /startupConfigurationFile=”..\web.config”
Look for it in the NuGet packages directory: "..\packages\EntityFramework.6.1.3\tools"
You can specify migration configuration as an argument to it. The CustomConfig should contain your code based Seed method. So, This way you do not require SQL scripts to be generated from the migration.
More info here:
http://www.eidias.com/blog/2014/10/13/initialcreate-migration-and-why-is-it-important
http://www.gitshah.com/2014/06/how-to-run-entity-framework-migrations.html
Using this solution, you do not need to generate an SQL script and can run multiple Seeds for different environments.