My objective is to understand how to include sensitive constants/secrets (eg. api token) and non-sensitive constants (eg. api route) in my code.
Example for a sensitive constant:
val apiToken = "Hushhhh!"
Example for a non-sensitive constant:
val happyUsersEndpoint = "https://happy.foo.io/v1/users"
My questions are:
In which part of my folder structure should I store non-sensitive constants?
Is there a safe way to store sensitive constants in my code? If not, what is the right way to reference them from outside within my code?
To my understanding, there are ways to do that in Java-ish code, but I am mainly looking for ways that fit well into Scala code (if possible).
It should not be in the source code and in the VCS repo.
It should be a part of the deployment/operation process.
You can use some sort of a config or retrieve it from env variables. Which is better depends on your stack. Is it a docker? Is it a Play or something from typelevel stack?
For each situation, there are would be a different appropriate method. For example:
For Play you can use just play config files in HOCON format.
In typelevel you have pureconfig.
For docker installation, it is better to use env.
If you have a simple zero dependency console app - I suggest adding command line args with needed properties.
For a simple app without play, cats and docker - just pick a lightweight simple library.
Related
I want to build up a tests library and keep it separated from the libraries under development. My first thought is to go for a structure like the following:
PensLib
--Variants
----BallPoint
----FountainPen
----Tests
------TB_BallPoint
HammocksLib
--Variants
----SingleHammock
----DoubleHammock
----Tests
------TB_DoubleHammock
--Systems
----IndoorWalls
----OutdoorWallAndTree
----CoconutPalms
----Tests
------TB_IndoorWalls
Tests
--PensLib
----Variants
------Test_BallPoint // extends PensLib.Variants.Tests.TB_BallPoint
--HammocksLib
----Variants
------Test_DoubleHammock // extends HammocksLib.Variants.Tests.TB_DoubleHammock
----Systems
------Test_IndoorWalls // extends HammocksLib.Systems.Tests.TB_IndoorWalls
For now let's assume that the way I structure my libraries make sense (which most likely doesn't). I will soon ask more questions on good practices in setting up the testing environment in Dymola and with the Testing Library.
My question is about the correct way to handle relative and absolute paths within models, if possible at all.
The model PensLib.Variants.Tests.TB_BallPoint is used for developing the variant BallPoint
The model Tests.PensLib.Variants.Tests_BallPoint is used for automated testing
I want the model Test_BallPoint to extend the model TB_BallPoint, but I cannot link them. I guess the absolute path PensLib.Variants.Tests.TB_BallPoint is treated as a relative one, since PensLib is found "on the way out" of the Tests library, and from there it goes looking for the rest of the path. Is there perhaps a way to control the path, kind of ..\..\..\PensLib\Variants\Tests\TB_BallPoint?
As you already noted such a setup makes troubles. There are ways around that, namely global name lookup and imports, which I explain briefly further below.
Both solutions are nice when you have such a case in a few situations. But if you have to use it all the time, you make your setup unnecessarily complicated.
Hence, I suggest to make yourself the live easier and change your package structure:
Either create a dedicated test library for every library, maybe PensLib_Tests and HammocksLib_Tests
Or rename the packages in the Tests library and don't use the exact library names
Global name lookup
You can use absolute class paths. They are denoted with a leading ., so this should work:
extends .PensLib.Variants.Tests.TB_BallPoint;
See Modelica Specification chapter 5: Scoping, Name Lookup, and Flattening for details, especially 5.3.3 Global Name Lookup
Importing
You can simply import the library. Lookup of imports is always performed globally.
import PensLib;
extends PensLib.Variants.Tests.TB_BallPoint;
I have a really complicated system which use multiple languages and frameworks (Java Python Scala Bash). In each module I need to retrieve configuration values which are similar and change frequently. Currently I'm maintaining multiple conf files which holds lots of duplicates.
I wonder if there is out of the box RestAPI which can retrieve variables by demand from remote location.
All I manage to find by now are ways to load the entire file from remote source which is half a solution from me:
YAML.parse(open('https://link_to_file/file.yaml'))
My goal, which I fail to find a lead to it, is to make a direct call.
MyRemoteAPI.get("level1.level2.x")
P.S
YAML is not mandatory solution for me, I'm Open for suggestions.
I don't know about an out-of-the-box API, but it's fairly trivial to build. Make a service that will read the YAML file and traverse to the appropriate key. e.g. using a dynamic language like Ruby (+Rails), you could do something like
def value
config = YAML.load_file '/local/path/to/config.yaml'
render plain: config.dig(params[:key].split('.'))
end
dig essentially traverses a structure and safely returns nil if a key isn't found, so this returns the value at the "leaf" of the requested path.
You might also want to cache the structure in memory to prevent constantly reading from the file, e.g. could do something like ##config ||= YAML.parse(open('https://link_to_file/file.yaml')) or config = Rails.cache.fetch('config', expire_in: 1.hour) { ... }. And/or cache the API's HTTP response.
I'm newbie to Scala, and I have years of experience programming in Java.
Usually there are two patterns passing some config:
Using a global object sounds like "ConfigManager". And every time I
needs a config I get directly from it.
Passing the config through parameter. The config param may exists in
many layers in the program.
I choose one pattern depends on how the config will be used when I'm writing Java.
But in Scala, many people talks about eliminating side effects. This makes me wonder if I should use the second patterns at any costs.
Which pattern is better in Scala?
Global objects are bad: https://softwareengineering.stackexchange.com/questions/148108/why-is-global-state-so-evil
Make each component take it's configuration (individual pieces) as constructor parameters (possibly with some defaults). That prevents the creation of invalid components or components that have not been configured.
You can collect the initial processing of configuration values in a single class to centralize configuration code and to fail-fast when things are missing. But don't make your components (classes needing the configuration) depend on a global object or take in an entire configuration as a parameter. Just what they need as constructor params.
Example:
// centralize the parsing of configuration
case class AppConfig (config: Config) {
val timeInterval = config.getInt("type_interval")
val someOtherSetting = config.getString("some_other_setting")
}
...
// don't depend on global objects
class SomeComponent (timeInterval: Int) {
...
}
object SomeApplication extends App {
val config = AppConfig(ConfigFactory.load())
val component = new SomeComponent(config.timeInterval)
}
Use global object (this object stores only read-only immutable data, so no issues) which loads configuration object and config variables at once. This has many benefits over loading the configuration deep inside the code.
object ConfigParams {
val config = ConfigFactory.load()
val timeInterval = config.getInt("time_interval")
....
}
Benefits:
Prevents runtime errors (Fail fast approach).
If you have miss spelt any property name your app fails during startup as you are trying to fetch the data eagerly. If this were to be deep inside the codebase then it would be hard to know and it fails when the control of the program goes to that line. So, it cannot be easily detected unless rigorous testing is done.
Central place for all configuration logic and configuration transformations if any.
This serves as a central place for all config logic. easy to change and maintain.
Transformations can be done without need for refactoring the code.
Maintainable and readable.
Easy refactoring.
Functional programming point of view
Yes, loading the config file eagerly is great idea from Fail fast point of view but its not a good functional programming practice.
But important thing is you are not mixing the side effect with any other logic and keeping it separate during the loading of the app. So, as you are isolating the side effect and side effecting at the starting of your project, this would not be a program.
Once the side effecting is done and app has started. Your pure code base will not effected from this and remains pure and clean. So, though it is side effecting, it is isolated and does not effect your codebase. Benefits you again from this are worth experiencing, So go ahead.
I want to call the API of uima-text-segmenter https://code.google.com/p/uima-text-segmenter/source/browse/trunk/INSTALL?r=22 to run an example.
But I don`t know how to call the API...
the readme said,
With the DocumentAnalyzer, run the following descriptor
`desc/textSegmenter/wst-snowball-C99-JTextTilingAAE.xml` by taking the
uima-examples data as input.
Could anyone give me some code which could be run directly in main func for example?
Thanks a lot!
Long answer:
The link describes how you would set up the application from within the Eclipse UIMA environment. This sort of set-up is typically targeted at subject matter specialists with little or no coding experience. It allows them to work (relatively fast) with UIMA in a declarative way: all data structures and analysis engines (computing blocks within UIMA) are declared in xml (with a GUI on top of it), after which the framework takes care of the rest. In this scenario, you would typically run a UIMA pipeline using a run configuration from within Eclipse (or the included UIMA pipeline runner application). Luckily, UIMA allows you to do exactly the same from code, but I would recommend using UIMAFit (http://uima.apache.org/d/uimafit-current/tools.uimafit.book.html#d5e137) for this purpose instead of of UIMA, as it bundles lots of useful things and coding shortcuts.
Short answer:
Using UIMAFit, you can call Factory methods that create CollectionReader (read input), AnalysisEngine (process input) and Consumer objects (write/do other stuff) from (third-party provided) XML files. Use these methods to construct your pipeline and the SimplePipeline class to run it. To extract the data you need, you would manipulate the CAS object (containing your data) in a Consumer object, possibly with a callback. You could also do this in a Analysis Engine object. I recommend using DKPro's FeaturePathFactory (https://code.google.com/p/dkpro-core-asl/source/browse/de.tudarmstadt.ukp.dkpro.core-asl/trunk/de.tudarmstadt.ukp.dkpro.core.api.featurepath-asl/src/main/java/de/tudarmstadt/ukp/dkpro/core/api/featurepath/FeaturePathFactory.java?spec=svn1811&r=1811) to quickly access the feature you are after.
Code examples:
http://uima.apache.org/d/uimafit-current/tools.uimafit.book.html#d5e137 contains examples, but they all go in the opposite direction (class objects are used in the factory methods, instead of XML files - XML is generated from these classes). Take a look at the UIMAFit API to find the method you need, AnalysisEngineDescription from XML for example: http://uima.apache.org/d/uimafit-current/api/org/apache/uima/fit/factory/AnalysisEngineFactory.html#createEngineDescriptionFromPath-java.lang.String-java.lang.Object...-
I've done some searching on this, but I cannot find info. I'm building an application inside sinatra, and using the coffeescript templating engine. By default the compiled code is wrapped as such:
(function() {
// code
}).call(this);
I'd like to remove that using the --bare flag, so different files can access classes and so forth that I'm defining. I realize that having it more contained helps against variable conflicts and so forth, but I'm working on two main pieces here. One is the business logic, and arrangement of data in class structures. The other is the view functionality using raphaeljs. I would prefer to keep these two pieces in separate files. Since the two files wrapped as such cannot access the data, it obviously won't work. However, if you can think of a better solution than using the --bare option, I'm all ears.
Bare compilation is simply a bad practice. Each file should export to the global scope only the public objects that matter to the rest of your app.
# foo.coffee
class Foo
constructor: (#abc) ->
privateVar = 123
window.Foo = Foo # export
Foo is now globally available. Now if that pattern isn't practical, maybe you should rethink your structure a bit. If you have to export too may things, you nest and namespace things better, so that more data can be exposed through fewer global variables.
I support Alex's answer, but if you absolutely must do this, I believe my answer to the same question for Rails 3.1 is applicable here as well: Put the line
Tilt::CoffeeScriptTemplate.default_bare = true
somewhere in your application.