I'd like to be able to modify some build tasks in response to command line parameters. How do I (can I?) access command line parameters from the Build.scala file?
I don't think it's possible and you should rather resort to using input tasks or environment properties with -D.
Related
I am very new to spark. Please help me with below query:
Suppose I have different file path in dev and prod and i do not want to hardcode the same in my spark scala code rather I want to use variable in my code so my doubt is that where I define these variables for dev and prod and how to access them in my code.
Here are a few options:
Pass variables as command line parameters to your Spark application
Use the property/config file to keep those variables
Create environmental variables for these variables
For implementation, follow these articles.
Command Line Argument in Scala
Passing command line arguments to spark-shell
Passing command line arguments to spark-submit
Does anybody know how I can run gauge with a custom argument to command line and handle it in python plugin?
For example:
When I launch gauge, I want also pass additional argument/flag to command line (> gauge run specs 17) and then handle it and use in my python code before/or when suites starts
Solution was found.
I just added session enviroment variable.
https://github.com/getgauge/docs.gauge.org/blob/master/configuration.rst
You can add a property to env/default/default.properties and use that property as an argument.
E.g. gauge run --env ci specs
Reference: https://docs.gauge.org/latest/configuration.html#id1
Currently, I use export JAVA_OPTS ... on the command line, but there seem to be other possibilities, using the build.sbt or an external property file.
I have found several relevant github issues here, here and here but the many options are confusing. Is there a recommended approach?
The approach you take to setting JVM options depends mainly on your use case:
Inject options every time
If you want to be able to specify the options every time you run your service, the two mechanisms are environment variables, and command line parameters. Which you use is mostly a matter of taste or convenience (but command line parameters will override environment variable settings).
Environment variables
You can inject values using the JAVA_OPTS environment variable. This is specified as a sequence of parameters passed directly to the java binary, with each parameter separated by whitespace.
Command line parameters
You can inject values by adding command line parameters in either of two formats:
-Dkey=val
Passes a Java environment property into the java binary.
-J-X
Passes any flag -X to the java binary, stripping the leading -J.
Inject options from a file which can be modified
If you want to end up with a file on the filesystem which can be modified after install time, you will want to use sbt-native-packager's ability to read from a .ini file to initialise a default value for Java options. The details of this can be seen at http://www.scala-sbt.org/sbt-native-packager/archetypes/cheatsheet.html#file-application-ini-or-etc-default
Following the instructions, and depending on the archetype you are using, you will end up with a file at either /etc/default, application.ini, or another custom name, which will be read by the startup script to add settings.
Each line of this file are treated as if they were extra startup parameters, so the same rules as mentioned earlier are still enforced; e.g. -X flags need to be written as if they were -J-X.
Inject options & code which never need to be changed
You can hardcode changes directly into the shell script which is run to start your binary, by using the SBT setting bashScriptExtraDefines, and following the details at http://www.scala-sbt.org/sbt-native-packager/archetypes/cheatsheet.html#extra-defines
This is the most flexible option in terms of what is possible (you can write any valid bash code, and this is added to the start script). But it is also less flexible in that it is not modifiable afterwards; any optional calculations have to be described in terms of the bash scripting language.
I'm trying to pass a standard teamcity build parameter vcsroot.url as the parameter of a rake task, using Teamcity's built in Rake build step. However, the build parameter doesn't seem to be evaluated.
In the "Rake Tasks" box, I've got:
setup_github_pages["%vcsroot.url%"]
When I run this build I get the following error:
[Execute setup_github_pages] NoMethodError: undefined method `[]' for nil:NilClass
Yet on the build results parameter tab, I see the correct value for the vcsroot.url parameter.
Are there rules about which build step fields do / don't have parameter substitution performed? Or is there an escape sequence required (I've scoured the teamcity docs in vain...)
Try adding a custom environment variable to expose the configuration variable you're trying to access:
Reference Teamcity and Rake: Where are the tc system properties?
For example, you want to pass system.CUSTOM property defined in the agent.conf file. Click the Add new variable link, specify CUSTOM as a name and %system.CUSTOM% as a value. Now in the rakefile you can access it as ENV['CUSTOM'].
I've been able to access vcsroot.url directly from within the rake task using this approach.
I'm trying to set up a new build configuration in TeamCity using the Powershell runner. However, I can't seem to find a way to access the TeamCity System Properties in the build script. I've seen hints that it is possible, but cannot find documentation on how to do it.
I have tried accessing the system properties using Powershell variable syntax, $variable. I have also printed out all variables in memory and see no teamcity variables to use.
Is this possible with the Powershell runner, and if so what is the syntax necessary to get it working?
TeamCity will set up environment variables, such as build.number (you can see a list of these within TeamCity).
In Powershell you can access environment variables using the env "provider", e.g.
$env:PATH
TeamCity variables are accessible by replacing the . with a _, so the build.number variable can be accessed as
$env:build_number
As it says in the TeamCity documentation, the system parameters are passed to the build script runner, but not all build script runners know what to do with them. In the case of the Powershell script runner, when using a script file, they don't propagate down to your scripts.
It's occurred to me to write a psake-optimized build runner that does, but in the meantime you can do one of the following:
explicitly map any of the TeamCity build properties to script parameters using the parameter expansion that's available within the Script Source box. eg .\build.ps1 -someParam:%build.name%
use environment parameters, which can be accessed explicitly within PowerShell using $env:NAME_IN_TEAMCITY syntax, eg $env:TEAMCITY_VERSION, or looped over and pushed into variable scope
access the build properties file that TeamCity makes available during the build. The file is available at $env:TEAMCITY_BUILD_PROPERTIES_FILE, and if you load the XML version it's fairly easy to loop through and push them all into scope (though you do get everything as a string of course). I posted a gist on how to do this (https://gist.github.com/piers7/6432985). Or, if using Psake, modify the script above to return you a hashtable which you can pass directly to Psake's -properties argument.
It is posible. Here is example how to pass system properties into PSake script:
& .\psake.ps1 -parameters #{build_number=%build.number%; personal_build=%build.is.personal%}
If you don't use Psake, you can define your variables like this:
$build_number = %build.number%
The %build.number% part will be replaced with TeamCity provided data. I think, it works only in Source code script input mode.
I created a meta-runner that will pass through System parameters to parameters declared in the Powershell script. It's not perfect (if you put '# in your source it will break) but it works for what I needed, you can find it here: https://gist.github.com/anonymous/ef60ada3f48f0fb25093