How do I get an unwrapped key in Typesafe Config? - scala

Test case:
import org.specs2.mutable._
class HelloWorldSpec extends Specification {
"Typesafe Config" should "allow me to see my escaped key" in {
val entries = ConfigFactory.parseString("""
"quoted.key.1" = 5
"quoted.key.2" = 6""").entrySet
entries.head.getKey === "quoted.key.1"
}
}
This test fails because the key is actually "quoted.key.1", not quoted.key.1. Is there a suggested way to unwrap this or do I have to manually look for surrounding quotes and remove them every time?

Read about "Paths, keys, and Config vs. ConfigObject" in the API docs here: http://typesafehub.github.io/config/latest/api/com/typesafe/config/Config.html
and in the README here: https://github.com/typesafehub/config#understanding-config-and-configobject
(Suggestions for improving those docs are welcome.)
The keys in the entry set (and the Config) are path expressions. These are strings that require parsing. There is a parse method in ConfigUtil, see http://typesafehub.github.io/config/latest/api/com/typesafe/config/ConfigUtil.html#splitPath%28java.lang.String%29
It won't work to just remove quotes, the parsing is somewhat more complex than that. Fortunately you can just use the ConfigUtil.splitPath method.
So the two ways to iterate keys at the root level are something like, first with Config:
Config config = ... ;
for (Map.Entry<String, ConfigValue> entry: config.entrySet()) {
String[] keys = ConfigUtil.splitPath(entry.getKey());
System.out.println("Root key = " + keys[0]);
}
Then with ConfigObject:
Config config = ... ;
for (Map.Entry<String, ConfigValue> entry: config.root().entrySet()) {
System.out.println("Root key = " + entry.getKey());
}
I didn't try compiling the above examples so forgive any silly syntax errors.
If your configuration contains only one level (no nested objects) the above two ways of iterating are the same; but if you have nested values they are not the same because iterating Config will give you all leaf values while iterating ConfigObject (config.root()) will give you all immediate children of the root, even if those immediate children are themselves objects.
Say you have:
foo {
bar {
baz = 10
}
}
If you iterate that as a Config you will get one entry which will have the path foo.bar.baz as key and the value 10. If you iterate it as a ConfigObject then you will have one entry which will have the key foo and the value will be an object which will in turn contain the key bar. When iterating this as a Config you could splitPath the foo.bar.baz and you would get an array of three strings, foo, bar, and baz.
To convert a Config to a ConfigObject use the root() method and to convert ConfigObject to Config use the toConfig() method. So config.root().toConfig() == config.
Also, the above config file could be equivalently written as:
foo.bar.baz = 10
But would be different if written as:
"foo.bar.baz" = 10
Because in the first case you have nested objects, and in the second case you have a single object with periods in the key name. This is due to the quotes.
If you write "foo.bar.baz" with quotes, then when iterating Config the path you got back would be quoted and splitPath() would return an array of one element foo.bar.baz. When iterating ConfigObject you would have a single object which would contain an entry with foo.bar.baz as key and 10 as value. Keys containing ., or other special characters, have to be quoted so they are interpreted as single keys rather than as paths.
To make your test case pass you could do this using splitPath:
import org.specs2.mutable._
class HelloWorldSpec extends Specification {
"Typesafe Config" should "allow me to see my escaped key" in {
val entries = ConfigFactory.parseString("""
"quoted.key.1" = 5
"quoted.key.2" = 6""").entrySet
// the ordering of entrySet is not guaranteed so this may
// still fail because it gets quoted.key.2 instead
ConfigUtil.splitPath(entries.head.getKey).head === "quoted.key.1"
}
}
You could also do this, using ConfigObject:
import org.specs2.mutable._
class HelloWorldSpec extends Specification {
"Typesafe Config" should "allow me to see my escaped key" in {
val entries = ConfigFactory.parseString("""
"quoted.key.1" = 5
"quoted.key.2" = 6""").root.entrySet // note ".root." here
// the ordering of entrySet is not guaranteed so this may
// still fail because it gets quoted.key.2 instead
// no need to splitPath because ConfigObject has keys not paths
entries.head.getKey === "quoted.key.1"
}
}

in java, first unwarp the whole ConfigObject to a hashmap, then you can use quoted.key.1 (without surrounding quotes) as key to get correct value.

Related

How to use if else in ARM template azure

I have created sample function here in c# which set the location value based on the parameter. I want to write below expression by using arm template style format.
public static Main(string name)
{
string location = string.Empty;
if(name == "uksouth")
{
location = "UKS";
}else if(name == "ukwest")
{
location = "UKE";
}else if(name == "IndiaWest")
{
location = "INDW";
}
else {
location = "INDS";
}
}
I have written this for one match condition, but i want to return value based on the user resource group.
"value": "[if(equals(resourceGroup().location,'uksouth'), 'UKS', 'EUS')]"
Unfortunately, ARM templates don't provide the equivalent of a "switch" mechanism, which is what might make this easier. However, you can nest multiple if statements. The syntax is a bit clunky, but this should be the equivalent of the code you've written:
"value": "[if(equals(resourceGroup().location,'uksouth'), 'UKS', [if(equals(resourceGroup().location,'ukwest'), 'UKE', [if(equals(resourceGroup().location,'IndiaWest'), 'INDW', 'INDS')])])]"
Here's the same code with a little formatting applied to make it more obvious what's happening here:
"value": "
[if(equals(resourceGroup().location,'uksouth'),
'UKS',
[if(equals(resourceGroup().location,'ukwest'),
'UKE',
[if(equals(resourceGroup().location,'IndiaWest'),
'INDW',
'INDS')])])]
"
You might also consider the approach described in this answer for a bit of a cleaner solution.
Try defining a variable that is an object used like a hashtable. Retrieve different properties from the object by key-name, accessing the properties as key-value pairs. I use something very similar to lookup values inside my ARM templates.
"variables" {
"locationShorten": {
"uksouth": "UKS",
"ukwest": "UKE",
"IndiaWest": "INDW",
"IndiaSouth": "INDS"
},
"locationShort": "[variables('locationShorten')[resourceGroup().location]]"}
Microsoft defines an object's properties as key-value pairs. "Each property in an object consists of key and value. The key and value are enclosed in double quotes and separated by a colon (:)."
Source: https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/data-types#objects
As for the documentation on using [] to access object properties, I can no longer find it for JSON but it is there for BICEP. "You can also use the [] syntax to access a property."
Source: https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/data-types#objects

Do two copies of a case class use twice the memory when only one property is changed, or does Scala reuse immutable values on copy to save memory?

In the following code I instantiate Name once and memory must be allocated to store two strings. I then make a copy of that first object, but only changing one property:
case class Name(first: String, last: String)
val original = Name("Freddie", "Mercury")
val copy = original.copy(first = "Robert")
My question - since last is an immutable val, does Scala point copy.last to the same string in memory as original.last in order to save space?
Name would look something like this in Java (output after compiling and decompiling):
public Name copy(final String first, final String last) {
return new Name(first, last);
}
public String copy$default$1() {
return this.first();
}
public String copy$default$2() {
return this.last();
}
public String first() {
return this.first;
}
public String last() {
return this.last;
}
You can see that when you copy a Name object, the default second argument is last()/last, meaning that the last field of the newly created Name is, as you guessed, pointing to the same object as the original Name's last field.
I suppose you could say this is to save the JVM space, but it would also just not make any sense to copy all the fields of a Product instance. copy is not deepCopy, and if you wanted to copy the original object's fields, where would you stop? Do you also copy the fields of those fields, and so on? And how exactly would you copy a field of type Foo without the compiler generating code that does some nasty reflection? It just makes more sense to reuse the fields.

Accessing Constants which rely on a list buffer to be populated Scala

Encountering a problem whereby I am specifying Private Constants at the start of a scala step definiton file which relies on a List Buffer element to be populated, however when compiling I get a 'IndexOutOfBoundsException' because the list is empty initially and only gets populated later in a for loop.
For Example I have the following 2 constants:
private val ConstantVal1= globalExampleList(2)
private val ConstantVal2= globalExampleList(3)
globalExampleList is populated further down in the file using a for loop:
for (i <- 1 to numberOfW) {
globalExampleList += x.xy }
This List Buffer adds as many values as required to a global mutable ListBuffer.
Is there a better way to declare these constants? I've tried to declare them after the for loop but then other methods are not able to access these. I have around 4 different methods within the same file which use these values and instead of accessing it via index each time i thought it would be better to declare them as a constant to keep it neat and efficient for whenever they require changing.
Thanks
You can create list buffer of necessary size with default value and populate it later:
val globalExampleList: ListBuffer[Int] = ListBuffer.fill(numberOfW)(0)
for (i <- 0 until numberOfW) {
globalExampleList(i) = x.xy
}
But ConstantVal1, ConstantVal2 will still have original default value. So you can make them vars and re-assign them after you populate the buffer.
Your code seems to have a lot of mutations and side effects.
You have 2 ways to go.
First you can use lazy modifier
private lazy val ConstantVal1= globalExampleList(2)
private lazy val ConstantVal2= globalExampleList(3)
Or you can write the two lines after the for loop.
val globalExampleList = XXXX
for (i <- 1 to numberOfW) { globalExampleList += x.xy }
private val ConstantVal1= globalExampleList(2)
private val ConstantVal2= globalExampleList(3)

How to generate an unique ID for an class instance in Scala?

I have a class that needs to write to a file to interface with some legacy C++ application.
Since it will be instantiated several times in a concurrent manner,
it is a good idea to give the file an unique name.
I could use System.currentTimemili or hashcode, but there exists the possibility of collisions.
Another solution is to put a var field inside a companion object.
As an example, the code below shows one such class with the last solution, but I am not sure it is the best way to do it (at least it seems thread-safe):
case class Test(id:Int, data: Seq[Double]) {
//several methods writing files...
}
object Test {
var counter = 0
def new_Test(data: Seq[Double]) = {
counter += 1
new Test(counter, data)
}
}
Did you try this :
def uuid = java.util.UUID.randomUUID.toString
See UUID javadoc, and also How unique is UUID? for a discussion of uniqueness guarantee.
it is a good idea to give the file an unique name
Since all you want is a file, not id, the best solution is to create a file with unique name, not a class with unique id.
You could use File.createTempFile:
val uniqFile = File.createTempFile("myFile", ".txt", "/home/user/my_dir")
Vladimir Matveev mentioned that there is a better solution in Java 7 and later - Paths.createTempFile:
val uniqPath = Paths.createTempFile(Paths.get("/home/user/my_dir"), "myFile", ".txt"),

How to use Typesafe's Config in Scala with encrypted passwords

I would like to use Typesafe's Config in my project but I don't want any passwords in clear text in any file on the file system of any integration or production server. Also, I do not want to use environment variables to store clear text passwords.
Ideally, I would like a solution similar to the Jasypt EncryptablePropertyPlaceholderConfigurer available for Spring that would allow me to designate some property values as being encrypted and have the config system automatically decrypt them before handing the value down to the application. I'd like to use the JCE keystore to store the key and pass it into my app, but I'm also open to other tools that use a database to store keys.
Has anyone managed to get the Typesafe Config project to work this way?
Update: sourcedelica was completely correct to criticize the solution that relied on passing the key as an environment variable. I changed my question to request a solution that uses a more secure way of handling keys.
You could try pimping the typesafe Config class like so:
object ConfigPimping{
implicit class RichConfig(conf:Config){
def getPasswordString(path:String, encryptKey:String):String = {
val encrypted = conf.getString(path)
val decrypted = ... //do decripy logic of your choice here
decrypted
}
}
}
object ConfigTest{
import ConfigPimping._
def main(args: Array[String]) {
val conf = ConfigFactory.load()
val myPass = conf.getPasswordString("myPass", "myDecryptKey")
}
}
Then, as long as the RichConfig is always imported and available, you can get access to your custom decrpyt logic for passwords via the getPasswordString function.
If you are happy to pass the encryption key as an environment variable then instead you could pass all of the sensitive properties as environment variables and not worry about using encryption directly with the Typesafe config library.
For example:
my.sensitive = ${FOO_ENV}
You said that you do not want to use environment variables to store clear text passwords, but if you are storing your encryption key in an environment variable it is the equivalent.
Alternatively you could use a system property instead of an environment variable. For example, when starting your app, use -Dmy.sensitive=xxx.
If you do end up getting encrypted values into your configuration then you could use a wrapper class to that would do the decryption. I use a wrapper class to add methods like optString to Config. You could add a method like decryptString.
For a discussion on securing keys to be used in production see my question: Securing passwords in production environment.
I chose the path that cmbaxter suggested. I'm putting the example code here because comments don't seem to support code.
I added some special syntax, to the config file, so if I want to put an encrypted password in my config file I do it like this:
my-app-config{
db-username="foo"
db-password="ENC(9yYqENpuCkkL6gpoVh7a11l1IFgZ0LovX2MBF9jn3+VD0divs8TLRA==)"
}
Note the "ENC()" wrapper around the encrypted password.
Then I made a config factory that returns a DycryptingConfig object instead of the typesafe config:
import rzrelyea.config.crypto.DecryptingConfig;
import rzrelyea.config.crypto.KeyProvider;
public class ConfigFactory{
public static final Config makeDecryptingConfig(com.typesafe.config.Config config, KeyProvider keyProvider){
return new DecryptingConfig(config, keyProvider);
}
}
And here's the code for the DecryptingConfig:
import java.security.Key;
import static rzrelyea.config.Validators.require;
public class DecryptingConfig extends rzrelyae.config.Config {
private final com.typesafe.config.Config config;
private final Decryptor decryptor;
public DecryptingConfig(com.typesafe.config.Config config, KeyProvider keyProvider){
super(config);
require(keyProvider, "You must initialize DecryptingConfig with a non-null keyProvider");
this.config = config;
final Key key = keyProvider.getKey();
require(key, "KeyProvider must provide a non-null key");
decryptor = new Decryptor(config.getString("crypto-algorithm"), key, config.getString("encoding-charset"));
}
#Override
public String getString(String s) {
final String raw = config.getString(s);
if (EncryptedPropertyUtil.isEncryptedValue(raw)){
return decryptor.decrypt(EncryptedPropertyUtil.getInnerEncryptedValue(raw));
}
return raw;
}
Obviously, you'd need to implement your own rzrelyea.config.Config object, your own EncryptedPropertyUtil, your own Decryptor, and your own KeyProvider. My implementation of rzrelya.config.Config takes a typesafe config object as a constructor parameter and forwards all calls to it. LOTS of boiler plate code in it! But I thought it was better to forward calls to an interface rather than to extend com.typesafe.config.impl.SimpleConfig. You know, prefer composition to inheritance and code to interfaces, not implementations. You may choose a different route.
At the risk of telling you something you already know...
Never store a password -- store and compare against a hash instead
Use Bcrypt for password hashes -- it's slow which is good for guarding against a brute-force attack
Use a salt -- to guard against a rainbow table style attack
Use SSL (https) -- to prevent passwords from being seen in the clear
Here is an example that uses the Mindrot jBCrypt library:
def PasswordHash( name:String, pwd:String, version:Int = 1 ) : String = {
if( version == 2 && false )
{
// ANY CHANGES SHOULD BE MADE AS A NEW VERSION AND ADDED HERE
""
}
else
{
import org.mindrot.jbcrypt.BCrypt // jbcrypt-0.3m.jar
// Salt will be incorporated in the password hash
val salt = BCrypt.gensalt(12) // Default is 10, or 2**10 rounds. More rounds is slower.
BCrypt.hashpw( (name + pwd), salt )
}
}
def VerifyPassword( name:String, pwd:String, hash:String, version:Int = 1 ) : Boolean = {
if( version == 1 )
{
import org.mindrot.jbcrypt.BCrypt // jbcrypt-0.3m.jar
BCrypt.checkpw( (name + pwd), hash )
}
else
false
}
> PasswordHash( "johnny", "mypassword" )
res4: String = $2a$12$dHIlTL14.t37Egf7DqG4qePE446GzzhIUAVuewMfkhfK0xxw3NW6i
> VerifyPassword( "johnny", "mypassword", "$2a$12$dHIlTL14.t37Egf7DqG4qePE446GzzhIUAVuewMfkhfK0xxw3NW6i" )
res5: Boolean = true
> VerifyPassword( "johnny", "mommiespassword", "$2a$12$dHIlTL14.t37Egf7DqG4qePE446GzzhIUAVuewMfkhfK0xxw3NW6i" )
res6: Boolean = false
For what you are trying to do, I presume you would store the "name", "password hash", and "hash version" in your configuration.