Split configuration in typo3conf into managed, manual and server specific parts - typo3

In TYPO3, Version 6.1.5, I'd like to split my configuration in typo3conf in three parts:
One part generated by the install-tool, under version control.
One manually managed part, under version control.
One manually managed part, containing server specific stuff, not under version control. Contains e.g. database credentials.
I've tried to do it like this:
is done using LocalConfiguration.php,
is done using AdditionalConfiguration.php
is done by and additional file, included by the AdditionalConfiguration.php.
This does not work, as it seams that the files are evaluated in this order:
LocalConfiguration.php
AddtitionalConfiguration.php
LocalConfiguration.php
So the changes from my server specific file (and AdditionalConfiguration.php) are simple overwritten by the stuff from LocalConfiguration.php.
Any idea how to get around something like this?

First: don't apply any manual changes in your: LocalConfiguration.php file as it will be removed after each operation in the Install Tool, Extension Manager etc.
For adding custom configuration, you should use AddtitionalConfiguration.php file which isn't changed (as probably you know while you are using it). In this additional conf you need to use 'old' syntax for overwriting values, ie:
<?php
$GLOBALS['TYPO3_CONF_VARS']['DB']['database']='some_db_loc';
$GLOBALS['TYPO3_CONF_VARS']['DB']['host']='localhost';
$GLOBALS['TYPO3_CONF_VARS']['DB']['username']='jost';
$GLOBALS['TYPO3_CONF_VARS']['DB']['password']='yourpass';
finally at the end of the additional conf use common include() for including next PHP file in which you can overwrite these values again:
#include('jost_localconf_dont_commit.php');
At least in TYPO3 ver. 6.1.1 this scenario works like a charm.
EDIT: also take a look at Viktor's answer according to accessing the properties in additional config.
BTW: I'm not really sure why you need to commit AdditionalConfiguration.php , IMHO it should be ignored in the git, and on every environment it should have this file filled with local data typical for this env. Main (production) instance should keep whole its config in LocalConfiguration.php

Just one things to add to biesior's answer:
For security reasons, it is even better not to have the DB credentials in AdditionalConfiguration.php. It's better to include a PHP file with the credentials from a path that is outside the document root of the host. Therefore if PHP doesn't work properly, the file cannot be downloaded and the DB credentials are not exposed.

Related

environmental variable substitution when including another configuration file in Play 2.2

Is it possible to use environmental variable substitution when including another configuration file?
I would like to have something like that:
include "${HOME}/.foo/credentials.conf"
Configuration documentation mentions locating resources and include substitution but not together.
This works:
include "/home/me/.foo/credentials.conf"
and my HOME is correctly set.
But all attempts to make include "${HOME}/.foo/credentials.conf" so far failed
Background:
I deliberately want to keep credentials and other sensitive data out of our code base but have them available for local dev environments for testing. I am aware of more sophisticated solutions using external storage like hinted here Playframework 2 - Storing your credentials and we use something similar for live and preview environments but these are not suitable for local dev setup.
An alternative is to include credentials file to code base after all but use git ignore to prevent pushing it, but it is fragile solution and risk is someone will eventually push it and compromise credentials.
TBH I'm not even able to include file with absolute path /home/me... anyway approach which will work for you is just using alternative conf file as described in the same doc:
In file /home/me/.foo/credentials.conf you need to include application.conf - Play will fallback it to the file in classpath (this which is under VCS):
include "application.conf"
myCredentials.user="Espinosa"
myCredentials.password="fooBar123"
then run/start your app with this config file locally:
play -Dconfig.file=${HOME}/.foo/credentials.conf ~run
and that's it.
Note: of course it's easier to setup this addition in your IDE (i.e. IntelliJ: Run > Edit configurations) or write a shell script containing this command

Play Framework - How to maintain configuration files for different environments?

For my Play 2.2/Scala application (built with SBT), I would like to deploy different configuration files depending on the environment I'm deploying to (e.g. to couple a deployment with a particular database server). How does one create different variants of the application's configuration file (conf/application.conf) for different deployment targets? Hopefully variants can be generated from a base version?
What I'm used to from .NET is to have a base configuration file (Web.config), which undergoes a certain transformation depending on the profile one is deploying (e.g. Production). Does one use a similar technique in the Play/Scala world?
Alternative configuration files are covered in Play's documentation quite well in section Specifying alternative configuration file.
In short - in application.conf you place default configuration of your app, and additionally you need to create additional files for you environment(s) ie. life.conf, dev.conf etc. In these files you first need to include application.conf (which will read whole default configuration) and next just overwrite only parts which have to be changed - ie. DB credentials, it could be dev.conf:
include "application.conf"
db.default.driver=org.h2.Driver
db.default.url="jdbc:h2:mem:alternative-database-for-dev-testing"
db.default.user=developer
db.default.password="developerpass"
So finally you start your application (after dist) as
./start -Dconfig.resource=dev.conf
or with the Play console
play -Dconfig.resource=dev.conf run
Several tips:
It's good idea to do not place your 'life' DB credentials in default application.conf file, if some dev will forget to include his dev.conf he won't damage the production DB, instead you should put it in prod.conf.
Also these additional configs shouldn't be placed in any VCS (ie. git) repository - creating them directly on target machine (and ignoring in repository) give you sure, that people who shouldn't know the life database credentials won't see it.
It's also possible to use remote alternative config file, which can be useful ie. when you deploying several instances of the same app ie. on several hosts in the cloud.
Each dev can has own config file ie dev_aknuds1.conf, dev_biesior.conf etc, so you can ignore them with one pattern dev_*.conf in repo.
Finally you can just create a shell script (unix) or bat file (Windows) to start using choosen config file like start_dev.sh, run_dev.sh etc. so you won't need to write -Dconfig.resource=... each time

Named Config files - Hot or Not?

I've chosen a solution to our config file process and need to justify it. I'd appreciate insight and criticism. Thanks!
Problem:
Our web applications have a single application.cfm file containing variables and conditionals
<cfif url = "*dev*" or "jargon123">
this is a dev enviroment, do devy things
</cfif>
So a dev new to the application will deploy a local instance. Fire it up and start poking around. Problem is that the config file contains production values. It starts hitting production data and sending production emails. Also, since the url they are hitting is http://App_name:PORT or http://localhost - the dev conditionals are never set. So there is more production stuff happening in dev.
What other co-workers want:
A Switch statement. The app.cfm will lead with an environment variable set to "development", then declares general variables. It will then go into a switch statement and declare environment specific variables. I disagree with this method as some of our config files are 100 - 250 lines. That can be a massive Switch statement I don't want to muck around in.
My chosen solution:
App.cfm has been deleted and removed from version control. We now have multiple Applicaiton.Enviroment.cfm files, i.e. Applicaiton.Prod.cfm, Application.Dev.cfm, Applicaiton.MyName.cfm etc. This file contains all of the environment specific data. I moved Production specific settings out of conditionals and into App.Prod.cfm. Deployments to new environments are now 1. Duplicate App.Dev.cfm as App.Me.cfm and commit. 2. Update all variables to my personal data (email, login, etc) 3. Duplicate App.me.cfm as App.cfm and use for config file.
I won't go into why I'm not doing the other solutions but here is my reason for my solutions:
Forces the deployment engineer into selecting the right config file for the environment. The app won't work without an app.cfm
Limits potential of user error. Scenario would be a user copies data into a new environment mode and accidentally copies production content.
It's cleaner and easier to work with - config value's are completely compartmentalized from each other.
I've found a lot of articles on working with environment specific config files but not why they are better. That's the motivation behind this post.
I would also delete the production config and provide only development versions of the config file. Reasons:
a config file could contain security relevant data
many developers are just lazzy, if the application runs, the don't care about the config
if the developer do not use the currently provided mechanisms (the dev url), who could you be sure they set the environment variable?
using the live config during testing could result in active debug options on the production later (forgotten to remove from configuration)
You (development) need to be able to switch between different configurations for different versions of your software at any time. If each setup has its own configuration file, this is a lot easier than if they all share the same file.
If you have all configuration in a single file, you have to read the whole big file, deciding which parts to ignore. This is messier than just reading the whole file.
(I assume that you can have multiple versions of the software installed concurrently in different locations on the same machine. If you can't, you have a bigger problem. But even so, having separate configuration files is beneficial.)
Those are strong 'pros' for separate configuration files - they outweigh the minor 'con': you have to identify where the configuration file is by some mechanism or another. It might be via an environment variable or via a command-line option, with a suitable default if neither is specified. Command-line should override environment.

How should I handle Sphinx configuration in version control?

I have a problem with my development workflow and Sphinx. I want to keep configuration file for Sphinx in version control so it's easier to manage. This means it's easier to link the file to code updates, etc ... However, the configuration file is stored in /usr/local/etc.
There are two solutions I can think of. Store the file in the repository and move it to the correct folder on deployment or recompile Sphinx to look for the file in my repository. I had a suggestion from someone to use a symlink, but that still requires a change on deployment.
Is there an elegant solution in Sphinx I'm missing?
perhaps have the /usr/local/etc/sphinx.conf file be a script that pulls the actual sphinx config from the file in your repo.
http://sphinxsearch.com/docs/current.html#rel098 scroll down to general and you'll see:
"added scripting (shebang syntax) support to config files (example: #!/usr/bin/php in the first line)"

Best practices for using Hg with Grails?

What should I check in/not check in? Since many of the files are sometimes auto-generated I'm not entirely sure how to handle this using version control...does it have something to do with tags?
For instance in ANT, I know not to check-in my target/bin directories...but Grails adds another level of confusion to this...since some of code is generated and some of it is not.
(It may become clearer as I go...but it seems to be that there needs to be some way of being able to tell what was just generated and what was modified by a developer so that it needs to be placed in version control)
Here's the .hgignore directory I've got on my most recent grails project (with a couple of project specific things taken out):
syntax: glob
out
target
logs
*.iml
.idea
*.iws
*.war
workspace.xml
lib/build
lib/compile
lib/runtime
lib/test
~$*
stacktrace.log
*.tmproj
*.launch
reports/
*.orig
*.zip
.DS_Store
*/.DS_Store
web-app/WEB-INF/classes
cobertura.ser
The generated code in Grails should be placed under version control. It's not secondary executable code that is generated by the build process like class files, but instead is code that is part of your source. The generated code is intended to be just a starting point for your application and will most likely be modified at some point anyway.
Also check this:
http://www.grails.org/Checking+Projects+into+SVN
and this:
https://stackoverflow.com/questions/4201578