I'm sure everyone has to deal with these situations, we check in our solution to source control and each dev machine will have its own resources for debugging, building and testing..
The most common being:
Web server (IIS)
Database (SQL)
The web server is easy to handle, each dev machine will have its own proj.user file to specify different debug information.
But connection strings for the app are stored in the web.config (which is under source control), ideally we don't want the web.config to be 'aware', so having to do config sections where we delegate them to other config files (not under sc) wouldn't be the best solution..
asp.net (.net?) already supports a model to have web.config inheritance, which would be an ideal scenario.. however this only works for directories.
It would be great if we could have
web.config <-- under version control
web.machine.config <-- not under version control
Of course I'm open for better suggestions of how people solve this problem.
Like.. maybe having:
web.base.config <-- under version control
web.machine.config <-- not under version control
And having a build script that creates a web.config by merging them?
Thanks in advance,
Stephen.
edit
Looks like the next vs may have a way to handle this:
http://blogs.msdn.com/webdevtools/archive/2009/05/04/web-deployment-web-config-transformation.aspx
edit edit
Possibly do'able with xml mass update today:
http://blogs.microsoft.co.il/blogs/dorony/archive/2008/01/18/easy-configuration-deployment-with-msbuild-and-the-xmlmassupdate-task.aspx
edit edit edit
Well its certainly possible to do with a simple xslt build task and a small transform that copied everything and intercepts certain properties.. just tried a proof of concept and this will save us lots of frustration, but the transformation file may be more than people are willing to accept.
Basically we store a Web.base.config in version control, and run it through the transform to generate the Web.config on a build event.
Seems like vs2010 will really help in terms of having a much more friendly version of this.
One approach that I sometimes use is to break out environment-specific section into separate config file, that are usually excluded from deployment (except for the first time or if their structure change):
Example for connection strings:
In web.config:
<connectionStrings configSource="connections.config"></connectionStrings>
The connections.config file (that is typically not included in the deployment; so it is unchanged):
<?xml version="1.0"?>
<connectionStrings>
<add name="connectionName" connectionString="[connection string goes here]"/>
</connectionStrings>
Like that we have created an "incapsulation" of the information, and can easily deal with issues like source control, deployment and such of that information.
Whilst there are certainly plenty of solutions, none of them really give you a huge amount of control over the generated configuration, one solution that I noted in my edit where you get a huge amount of control but with the overhead of having to write an xslt file, was using an xslt build task to use the template web.config/app.config from source control (which I personally name web.base.config/app.base.config), and use an xslt file to transform the version controlled config file at build time, and generate a web.config/app.config.
Here is an example of an xslt build task (although you may want to write it to your own coding standards), and an example of a mundane xslt transform that will change the value of a connection string and copy everything else in the config:
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:msxsl="urn:schemas-microsoft-com:xslt" exclude-result-prefixes="msxsl">
<xsl:output method="xml" indent="yes"/>
<!-- Copy all. -->
<xsl:template match="#* | node()">
<xsl:copy>
<xsl:apply-templates select="#* | node()"/>
</xsl:copy>
</xsl:template>
<!-- Swap connection string. -->
<xsl:template match="/configuration/connectionStrings/add[#name='my_connection_string_name']">
<xsl:copy>
<xsl:apply-templates select="#* | node()"/>
<xsl:attribute name="connectionString">my replacement connection string value</xsl:attribute>
</xsl:copy>
</xsl:template>
</xsl:stylesheet>
This is a mediocre example, but you can imagine you can completely transform entire sections where you previously would struggle with an inheritance based scenario.
VS 2010 will provide you with a lot of control to manage web.config files for various configurations... Please check out.
The configuration of an application can be split into two categories.
Application specific
Deployment specific
Application specific configuration includes things like caching implementation, business rules implementation, and will apply to every deployment of the application. This should go into the web.config file that is part of the application directory structure and is checked into source control.
Deployment specific configuration includes things like connection strings, timeout periods, etc, and may differ from one deployment to another. This should be entered as part of the configuration of the IIS instance that is involved in the deployment and preserved by whatever backup strategy is in place for the machine in question.
As far as I can tell, this is exactly what the hierarchical nature of the web.config files was designed to handle.
The advantages of such an arrangement are...
No need to worry about which developer's version of the settings end up in source control, because none of them do.
Every deployment uses the same binary, so deployment issues are more likely to involve the deployment configuration.
Subsequent deployments should need no deployment specific configuration changes, because they are already in place.
We don't store environment settings in the web.config.
They're stored in a database.
This enables us to do xcopy deploys, and to store the web.config file in our version control system.
Access to the database is via one registry key.
What you're describing sounds a lot like using a Machine.config file to store connection strings. I haven't seen this mentioned yet, so have you tried it? It looks like you can use a global Web.config that sits beside your Machine.config as well.
A few links:
ASP.NET Configuration File Hierarchy and Inheritance
Difference between Web.config and Machine.config
Related
Is there a method/process that can take a web config file, i.e. WebConfigDev, and a second web config file, i.e. WebConfigQA, and generate a web config transform file for WebConfigDev to WebConfigQA?
I'm looking to have the files generated automatically because I currently have separate config files for each environment that I manually rename when deploying to a new environment. Each config file is fairly extensive and would require a fair amount of time to rework by hand and I'm hoping there is a more efficient way to do this.
Unfortunately there is not an existing utility within Visual Studio 2010 to do this.
I'd like to run a local NuGet Gallery to serve dependencies to my build system.
I notice in the web.config it asks for Azure details, but the code seems to suggest you can choose 'FileSystem' as a storage backend.
My questions are:
If I choose 'FileSystem' how do I configure the target folder?
Can I instead point the storage engine at an in-house instance of SQL Server?
I'm trying to avoid using a file system because that's what we are using now with NuGet Server and it's very slow. A couple of the devs like to pack and push every single successful build, so scalability is important.
I hope any answers here will help others, too. For background, here is a great link of setting up your own NuGet Gallery. Sadly, the author has omitted all details pertaining to the actual package storage: https://github.com/NuGet/NuGetGallery/wiki/Hosting-the-NuGet-Gallery-Locally-in-IIS
To configure File System Package Store:
<appSettings>
<add key="Gallery:PackageStoreType" value="FileSystem" />
<add key="Gallery:FileStorageDirectory" value="C:\Path\To\Packages" />
</appSettings>
To point to a different SQL Server:
<connectionStrings>
<add name="NuGetGallery" connectionString="Data Source=SQLSERVERNAME;Initial Catalog=NuGetGallery;Integrated Security=SSPI" providerName="System.Data.SqlClient" />
</connectionStrings>
EDIT: Support SQL Server as Package Store
If you want to store your packages as BLOBs in SQL Server, you'll have to make a couple of changes to the code.
First, create a class named SqlServerFileStorageService and implement IFileStorageService. This interface has several methods. The important ones are GetFile() and SaveFile(). Combining folderName and fileName will create a unique key you can use in your database table.
You can use the same connection string NuGetGallery or add a new one for your data access.
You then add an item to the enum PackageStoreType called SqlServer.
In ContainerBinding.cs add a case for PackageStoreType.SqlServer to bind to your SqlServerFileStorageService.
Now the NuGet Gallery should create a SqlServerFileStorageService and all gets and saves will use your class to store the blob in SQL Server.
BTW: I'm basing this on a cursory look at the code. There may be an extra step or two, but these look like the main areas you'll need to focus on.
Hope that helps.
To deploy to different azure environments I modify the csdef as part of the compilation step to change the host headers. Doing so requires building the cspkg once for each environment instead of being able to reuse the cspkg and specify different configs for deployment.
I would like to instead modify the csdef file of a cspkg after it has been created, without recompiling. Is that possible, and if so how?
I've done something similar to what you're after to differentiate between test and live environments. First of all you need to create a new .csdef file that you want to use for your alternate settings. This needs to be the complete file as we're just going to swap it out with the original one. Now we need to add this to the cloud project. Right click on the cloud project and select unload project. Right click on it again and select Edit [Name of project]. There's a section that looks a bit like this:
<ItemGroup>
<ServiceConfiguration Include="ServiceConfiguration.Test.cscfg" />
<ServiceDefinition Include="ServiceDefinition.csdef" />
<ServiceConfiguration Include="ServiceConfiguration.cscfg" />
</ItemGroup>
Add a new ServiceDefinition item that points to your newly created file. Now find the following line:
<Import Project="$(CloudExtensionsDir)Microsoft.WindowsAzure.targets" />
Then add this code block, editing the TargeProfile check to be the build configuration you're wanting to use for your alternate and ensuring that it points to your new .csdef file
<Target Name="AfterResolveServiceModel">
<!-- This should be run after it has figured out which definition file to use
but before it's done anything with it. This is all a bit hard coded, but
basically it should remove everything from the SourceServiceDefinition
item and replace it with the one we want if this is a build for test-->
<ItemGroup>
<!-- This is an interesting way of saying remove everything that is in me from me-->
<SourceServiceDefinition Remove="#(SourceServiceDefinition)" />
<TargetServiceDefinition Remove="#(TargetServiceDefinition)" />
</ItemGroup>
<ItemGroup Condition="'$(TargetProfile)' == 'Test'">
<SourceServiceDefinition Include="ServiceDefinition.Test.csdef" />
</ItemGroup>
<ItemGroup Condition="'$(TargetProfile)' != 'Test'">
<SourceServiceDefinition Include="ServiceDefinition.csdef" />
</ItemGroup>
<ItemGroup>
<TargetServiceDefinition Include="#(SourceServiceDefinition->'%(RecursiveDirectory)%(Filename).build%(Extension)')" />
</ItemGroup>
<Message Text="Source Service Definition Changed To Be: #(SourceServiceDefinition)" />
</Target>
To go back to normal, right click on the project and select Reload Project. Now when you build your project, depending on which configuration you use, it will use different .csdef files. It's worth noting that the settings editor in is not aware of your second .csdef file so if you add any new settings through the GUI you will need to add them manually to this alternate version.
If you would want to just have a different CSDEF then you can do it easily by using CSPACK command prompt directly as below:
Open command windows and locate the folder where you have your CSDEF/CSCFG and CSX folder related to your Windows Azure Project
Create multiple CSDEF depend on your minor changes
Be sure to have Windows Azure SDK in path to launch CS* commands
USE CSPACK command and pass parameters to use different CSDEF and Output CSPKG file something similar to as below:
cspack <ProjectName>\ServiceDefinitionOne.csdef /out:ProjectNameSame.csx /out:ProjectOne.cspkg /_AddMoreParams
cspack <ProjectName>\ServiceDefinitionTwo.csdef /out:ProjectNameSame.csx /out:ProjectTwo.cspkg /_AddMoreParams
More about CSPACK: http://msdn.microsoft.com/en-us/library/windowsazure/gg432988.aspx
As far as I know, you can't easily modify the .cspkg after it is created. I guess you probably technically could as the .cspkg is a zip file that follows a certain structure.
The question I'd ask is why? If it is to modify settings like VM role size (since that's defined in the .csdef file), then I think you have a couple of alternative approaches:
Create a seperate Windows Azure deployment project (.csproj) for each variation. Yes, I realize this can be a pain, but it does allow the Visual Studio tooling to work well. The minor pain may be worth it to have the easier to use tool support.
Run a configuration file transformation as part of the build process. Similiar to a web.config transform.
Personally, I go with the different .csproj approach. Mostly because I'm not a config file transformation ninja . . . yet. ;) This was the path of least resistance and it worked pretty well so far.
I've chosen a solution to our config file process and need to justify it. I'd appreciate insight and criticism. Thanks!
Problem:
Our web applications have a single application.cfm file containing variables and conditionals
<cfif url = "*dev*" or "jargon123">
this is a dev enviroment, do devy things
</cfif>
So a dev new to the application will deploy a local instance. Fire it up and start poking around. Problem is that the config file contains production values. It starts hitting production data and sending production emails. Also, since the url they are hitting is http://App_name:PORT or http://localhost - the dev conditionals are never set. So there is more production stuff happening in dev.
What other co-workers want:
A Switch statement. The app.cfm will lead with an environment variable set to "development", then declares general variables. It will then go into a switch statement and declare environment specific variables. I disagree with this method as some of our config files are 100 - 250 lines. That can be a massive Switch statement I don't want to muck around in.
My chosen solution:
App.cfm has been deleted and removed from version control. We now have multiple Applicaiton.Enviroment.cfm files, i.e. Applicaiton.Prod.cfm, Application.Dev.cfm, Applicaiton.MyName.cfm etc. This file contains all of the environment specific data. I moved Production specific settings out of conditionals and into App.Prod.cfm. Deployments to new environments are now 1. Duplicate App.Dev.cfm as App.Me.cfm and commit. 2. Update all variables to my personal data (email, login, etc) 3. Duplicate App.me.cfm as App.cfm and use for config file.
I won't go into why I'm not doing the other solutions but here is my reason for my solutions:
Forces the deployment engineer into selecting the right config file for the environment. The app won't work without an app.cfm
Limits potential of user error. Scenario would be a user copies data into a new environment mode and accidentally copies production content.
It's cleaner and easier to work with - config value's are completely compartmentalized from each other.
I've found a lot of articles on working with environment specific config files but not why they are better. That's the motivation behind this post.
I would also delete the production config and provide only development versions of the config file. Reasons:
a config file could contain security relevant data
many developers are just lazzy, if the application runs, the don't care about the config
if the developer do not use the currently provided mechanisms (the dev url), who could you be sure they set the environment variable?
using the live config during testing could result in active debug options on the production later (forgotten to remove from configuration)
You (development) need to be able to switch between different configurations for different versions of your software at any time. If each setup has its own configuration file, this is a lot easier than if they all share the same file.
If you have all configuration in a single file, you have to read the whole big file, deciding which parts to ignore. This is messier than just reading the whole file.
(I assume that you can have multiple versions of the software installed concurrently in different locations on the same machine. If you can't, you have a bigger problem. But even so, having separate configuration files is beneficial.)
Those are strong 'pros' for separate configuration files - they outweigh the minor 'con': you have to identify where the configuration file is by some mechanism or another. It might be via an environment variable or via a command-line option, with a suitable default if neither is specified. Command-line should override environment.
Is it common for a developer to keep their NAnt global configuration file (NAnt.exe.config) in version control?
And should or shouldn't the the rest of the files in the NAnt installation be added to the ignore file of the version control system?
One use of version control is as a backup. If the only copy of NAnt.exe.config is on a hard disk that dies, it will take some effort to reconstruct it (along with everything else that disappeared and wasn't backed up).
From the corporate perspective, having all of the work in progress backed up is a method for preserving assets. The corporate owner of the source code asset is assured that the asset will not be destroyed.
When there is another backup strategy, then sometimes the rule of thumb is not to put anything into version control that should not be shared with other developers. Such as customized data relevant only to one user and/or machine, or confidential information.
I keep a copy of the NAnt code for the version I'm using. This includes the .config file. This is so my build system is safe from "it disappeared from the internet" events (unlikely, but still).
Beyond that I see no reason to keep it around on your code repository, unless for some reason you've modified it somehow. Most everything in NAnt can be overridden in build files, like the target framework and so on.