Using ElasticSearch, one can place scripts of various languages in ElasticSearch's /config/scripts directory, and they will be automatically loaded for use in Update requests and other types of operations. In my production environment, I was able to accomplish this and run a successful Update using the script.
So far, however, I've been unsuccessful in getting this feature to work when running a node in local mode for integration tests. I assumed that, since one can configure the ElasticSearch node, with an elasticsearch.yml on the classpath, one should also be able to add a scripts directory and place her desired script there, causing it to be loaded into the local node. That doesn't seem to be the case since when I try to execute an Update utilizing that script it cannot be found.
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: Unable to find on disk script scripts.my_script
at org.elasticsearch.script.ScriptService.compile(ScriptService.java:269)
at org.elasticsearch.script.ScriptService.executable(ScriptService.java:417)
at org.elasticsearch.action.update.UpdateHelper.prepare(UpdateHelper.java:194)
... 6 more
Does anyone know the proper way to do automatic script loading into a local ElasticSearch node for testing?
I am using the basic ElasticSearch client included in "org.elasticsearch:elasticsearch:1.5.2".
After perusing the source code, I discovered that the reason my script was not being picked up by Elasticsearch's directory watcher was because it was watching user.dir, the default configuration directory. The scripts/ subdirectory would have had to have been under there for the node to pick up my script and load it into the ScriptService for it to be used during updates.
The configuration directory can be overridden in your elasticsearch.yml with the key path.conf. Setting that to somewhere in your project would allow you to load scripts during testing and add those scripts to version control as well. Make sure that under that directory is a scripts/ directory; that is where your scripts will be loaded from.
Related
Is it possible include arbitrary files (in this case a .csv) from a TwinCAT project direct to the Boot directory of a PLC?
By using PATH_BOOTPATH in the file open/read FBs it is possible to load files from this directory in a convenient manner regardless of whether using a CE or Windows deployment, However deployment of files to this location seems to be the sticking point.
I know that a copy of the project code is included within the CurrentConfig<Project>.tpzip file, but this file is not easily accessible from code, or updateable.
I've found the 'Additional Files' section within the system configuration, but it makes little sense.
Adding a file from inside the project as a 'Relative' path doesn't seem to do anything
Adding a file from inside the project as an external path includes the file (via symbolic links?) in the 'CurrentConfig.tszip' file, which has the same issues as the .tpzip
Adding an external file as an external path again includes the file inside of the .tszip.
I'm willing to accept that this might not be possible, but it just feels odd that the PATH_BOOTPRJ and PATH_BOOTPATH roots are there and not accessing useful paths.
Deployment
To quote Beckhoff:
Deployment is used to set up commands that are to be executed during the installation and startup of an application.
The event types are essentially at what stage of the deployment process the command is performed, where the command can either be copying a file or execution of a script/program.
Haven't performed extensive testing but between absolute/relative pathing and execution this should solve nearly all issues with deployment configuration.
I am running automated tests of our application on different versions of an OS build (Windows 7, Windows 10, etc...). My testing suite requires that I copy files to the Slave computers when there are changes in the tests (external to the build application). The test files are not in the Jenkins work space as they do not change frequently and therefore do not need to be copied to the Slave with each execution.
I am looking to be able to update the files on the Slaves, but not under the work space directory, so the Copy-To-Slave plugin will not work from my understanding.
I am looking to have batch files, testing resource files, DB generation scripts and others copied to the Slave computer by a Jenkins job. This job may monitor GIT, but not everything being copied is from GIT.
In essence, execute the following but to the Slave computer
xcopy C:\Testing*.* C:\Resources\Testing /s/v/e
The reason for this is our testing scripts look for certain files to execute (DB scripts for building the database for the current platform/DB Engine) and as these do not change too frequently, we only need to copy the files when they are changed, and leave the files in place for subsequent test runs. There is a large amount of files and GBs of data that does not need to be copied with each test run. There are also multiple executions of the application with the same testing files where the application has different configurations, but should produce the same results, so the test files do not need to be copied with each of these executions.
I found a configuration on the Copy-To-Slave plugin to add additional directories as destinations, that are relative to the file system root directory (C:\ in my case) which will solve my problem.
For my Play 2.2/Scala application (built with SBT), I would like to deploy different configuration files depending on the environment I'm deploying to (e.g. to couple a deployment with a particular database server). How does one create different variants of the application's configuration file (conf/application.conf) for different deployment targets? Hopefully variants can be generated from a base version?
What I'm used to from .NET is to have a base configuration file (Web.config), which undergoes a certain transformation depending on the profile one is deploying (e.g. Production). Does one use a similar technique in the Play/Scala world?
Alternative configuration files are covered in Play's documentation quite well in section Specifying alternative configuration file.
In short - in application.conf you place default configuration of your app, and additionally you need to create additional files for you environment(s) ie. life.conf, dev.conf etc. In these files you first need to include application.conf (which will read whole default configuration) and next just overwrite only parts which have to be changed - ie. DB credentials, it could be dev.conf:
include "application.conf"
db.default.driver=org.h2.Driver
db.default.url="jdbc:h2:mem:alternative-database-for-dev-testing"
db.default.user=developer
db.default.password="developerpass"
So finally you start your application (after dist) as
./start -Dconfig.resource=dev.conf
or with the Play console
play -Dconfig.resource=dev.conf run
Several tips:
It's good idea to do not place your 'life' DB credentials in default application.conf file, if some dev will forget to include his dev.conf he won't damage the production DB, instead you should put it in prod.conf.
Also these additional configs shouldn't be placed in any VCS (ie. git) repository - creating them directly on target machine (and ignoring in repository) give you sure, that people who shouldn't know the life database credentials won't see it.
It's also possible to use remote alternative config file, which can be useful ie. when you deploying several instances of the same app ie. on several hosts in the cloud.
Each dev can has own config file ie dev_aknuds1.conf, dev_biesior.conf etc, so you can ignore them with one pattern dev_*.conf in repo.
Finally you can just create a shell script (unix) or bat file (Windows) to start using choosen config file like start_dev.sh, run_dev.sh etc. so you won't need to write -Dconfig.resource=... each time
If I compile a script for distribution as a standalone exe, is there any way I can store settings within the exe itself, to save having to write to an external file? The main incentive for this is to save having to develop an installation process. I only need to store a few bytes.
Also, can resources such as images be compiled into the exe?
Using alternate data streams opens up a can of worms so i wouldn't go that way. Writing back config data into the exe itself won't work as the file is locked for write access during execution.
What i usually do is to store config data under %A_AppData%\%A_ScriptName%\%A_ScriptName%.ini
When the script starts i use IniRead which also provides a default value if the key isn't found - which is the case the script is executing for the first time.
The complementing IniWrite's in a OnExit subroutine/function will create the ini file if necessary.
This way no installation is needed and the config is stored in the proper, familiar place.
The autohotkey forum has dealt with this question before.
In that case, the user didn't want extra files -- period.
The method was to use the file system to save alternate data.
Unfortunately I can't find the post.
A simpler method is to use fileinstall command.
When the script is compiled, the external file is stored within the exe.
When the script executes the same command as an exe, the file is copied to the same
directory as the running script. It is a simple yet effective 'install'.
With a little testing for the config file, the fileinstall command can be skipped.
Skipping the fileinstall could allow changes to be made to the configuration after 'installation'
I have not tried saving settings within the compiled exe file, but I have included resources. I'm not sure which version of AHK you're using or how you are compiling, but I can right-click my scripts to compile. There's an option to compile with options, where you can include resources in your compiled exe.Compile with options
I store all SSIS packages in Subversion repository, their configuration files as well. Configuration file almost always stored in the same folder where package is.
Problem is - SSIS seems to always store path to configuration file (the one saved in the package itself) as an absolute path.
When someone else checks out folder with the package in the location different from where I had on my development PC the configuration file is not detected (because my absolute path is stored and it doesn't exist on the other developer PC). So another developer has to remove this configuration and add it again from where it is now on his local hard drive. Then changed package is saved which will cause new version to be committed. When I get that version from SVN it will no longer match local path on my PC.
On a related note: another developer may want to change values in configuration file as well. If I later get the latest version of everything from SVN package will no longer work on my PC.
How do you work around these inconveniences?
Another solution is to save your configuration in a database with an environment variable as the first configuration to tell it what database to look in, that's what we do. We have scripts to populate ssisconfig for each server in our source control, but the package uses the actual table data for the database in the environment variable we are using.
Anyone who has heard my SQL Saturday presentations knows I don't much care for XML and this is one of the reasons. A trick to using XML configuration with varying locations is to use an environment variable (indirect configuration) to direct SSIS where it can look for that resource. The big, big downside to this approach is you'd generally need to create an environment variable for each set of configuration files or have a massive, honking .dtsconfig file which becomes painful for versioning.
The option I prefer if XML configuration is a must is that the "variableness" is removed. Developers and admins get together and everyone agrees "there will be a folder everywhere SSIS is done to hold configuration files and that location is X" and then it's just a matter of solving for X. At a previous job, we used D:\ssisdata\configs
#HLGEM's approach of a table for configurations is hands down my favorite approach to SSIS configuration (until you get to 2012 and their project deployment model where configuration is an entirely different animal)
I add a folder called "config" under my projects folder, add it to source control and mantain the config file in this folder. You can also add it to the SSIS project if you like.
I think its a good solution because everybody can have this folder and dowload the config file.
When the package is deployed it will read the config file from where you inform in the deployment manifest so this solution wont impact your development