I have the following params in hiera:
base::users:
john#example.com:
ensure: present
user: john
sudo: true
type: ssh-rsa
key: AAAAB3NzaC1yc2EAAAABJ
in puppet i'm getting the following hash:
{john#example.com => {ensure => present, user => john, sudo => true, type => ssh-rsa, key => AAAAB3NzaC1yc2EAAAABJ}}
Then i'm calling create resources to create appropriate authorized_keys file:
create_resources('ssh_authorized_key', $users)
but it doesn't work because i have added new parameter 'sudo' and before calling create_resources I want to remove this key from hash and operate in another resource.
I've tried the next step to remove it:
$users_filtered = $users.each |$k, $v| { $v.delete['sudo'] }
i'm getting the next error:
Error while evaluating a Function Call, delete(): Wrong number of arguments given 1 for 2.
As I understand puppet tried to use 'delete' function from stdlib module.
But i have also tried:
$users_filtered = $users.each |$k, $v| { delete($users, $v['sudo'] }
But it doesn't work. Appreciate any help
Checking the documentation for the delete function from stdlib, we see that the two arguments in your case needs to be the hash to remove the key from and the key to remove from the hash.
https://github.com/puppetlabs/puppetlabs-stdlib#delete
$users_filtered = $users.each |$k, $v| { $v.delete['sudo'] }
The problem with this line is that you are treating delete as a hash with a key sudo. delete is a function and not a hash. $v is your hash values in the each lambda iterator here. You can fix this with
$users_filtered = $users.each |$k, $v| { $v.delete('sudo') }
to treat delete as a function. Also, if you want to pass $users_filtered into a create_resources function, then it needs to be a nested hash with each key as the title. Therefore, your lambda needs to be returning the nested hash, which means you need to use map instead to return a nested hash.
$users_filtered = $users.map |$k, $v| { $v.delete('sudo') }
https://docs.puppet.com/puppet/4.10/function.html#map
Then we have the other attempt:
$users_filtered = $users.each |$k, $v| { delete($users, $v['sudo'] }
which also needs to be returning a hash and needs to have a key as the second argument. You are giving $v['sudo'] as the second argument, which is instead the value of the sudo key in that hash. We fix this in a similar way via:
$users_filtered = $users.map |$k, $v| { delete($v, 'sudo'}
Note that the two versions of the solution are syntactically different but produce the same result and are both acceptable in modern Puppet DSL function invocations.
It is also worth noting that you can eliminate the need for the iterator entirely by using delete on the entire hash from your example.
$users_filtered = delete($users, 'sudo')
Since Puppet 4.0.0, the minus (-) operator deletes values from arrays and deletes keys from a hash:
['a', 'b', 'c', 'b'] - 'b'
# would return ['a', 'c']
{'a'=>1,'b'=>2,'c'=>3} - ['b','c'])
# would return {'a' => '1'}
https://github.com/puppetlabs/puppetlabs-stdlib#delete
Related
I am trying to call the hint method on a MongoDB::Cursor object. However, it throwing an exception when it's trying to execute the query. See the code sample below:
sub some_method_which_returns_cursor {
my $cursor = $collection->find($filter);
if ($hint) {
$cursor->hint({‘some_index’ => 1}); #failing here.
}
if ($sort) {
$cursor->sort($sort);
}
return $cursor;
}
Any thoughts as to what's going on and how I can fix this?
Harish asked me via email and I'll repeat my answer here for posterity:
The hint method takes a string when given an index name, or an array reference when given keys/order pairs:
$cursor->hint("some_index"); # by name
$cursor->hint([field1 => 1, field2 => -1]); # by keys
It also takes a hash reference, but don't use that because modern Perls randomize key order when serializing, so your hint may not match an index.
I need to be able to pluck specific values from data that I receive from different 3rd parties. The data can structured differently depending on the 3rd party. For example:
my $first =
{
email => "joe\#example.com",
firstname => "Joe",
lastname => "Regular",
};
my $second =
{
user => {
e-mail => "joe\#example.com",
firstName => "Joe",
lastName => "Regular",
}
};
I know what the data structure will be for each 3rd party, so I can define that as config. What I want to end up with is
my $email = _magic($first_config,$first);
my $other_email = _magic($second_config,$second);
Any ideas much appreciated.
Build a look-up table. And you can use a dispatch table, hash with values being code references, so that when a party-identification is used as the key the code for that party executes
my %get_value = ( first => \&fetch_first, second => \&fetch_second );
my $party = 'first'; # input via command-line options, STDIN ...
my $email = $get_value{$party}->();
where \&fetch_first is a reference to the subroutine fetch_first. You can also enter it directly, first => sub { ... }, suitable for simple code. See perlreftut, perlref, and perlsub.
There are many ways to carry data in your program, and so to implement the lookup itself.
Here is an illustration, built in steps. It uses the (confirmed) fact that the data is in valid Perl data structures, and for simplicity it specifies the data right in each sub.
sub fetch_first {
my $data = {
email => '...',
firstName => '...',
};
return $data->{email};
}
This only delivers the email address, but we can do better.
Once you dereference a code reference you can also pass arguments
my $first_name = $get_value{$party}->('firstName');
where the subs are now written to use this input to return the required field
sub fetch_first {
my ($query) = #_;
my $data = {
email => '...',
firstName => '...',
};
return $data->{$query};
}
A big weakness of the above is that the calling code must use valid names of keys, so it needs to know the details of implementation of what it is using.
This can be improved, for example by choosing an interface for the call which is then translated in the subs into key names (or via yet another look-up structure). Then you make calls such as
my $email = $get_value{$party}->('email'); # or: 'first', 'last'
and somewhere you have association first => 'firstName' (etc) which subs can look up.
The flexibility is greatly helped by data being set up in a consistent way. The whole thing can also be quite maintainable if the code is organized thoughtfully.
If this grows more complex the solution is to write a class. Then you can build a very nice system.
I want to build a struct in Perl that has a hash and a variable,then create a hash that each field will contain the struct I created.
like this:
use Class::Struct;
struct exmpl => {hash=>'%' , val => '$'};
my %hash_of_structs;
$hash_of_structs { "one" } = exmpl -> new ();
Now hash_of_structs has a field with "one" key that contains the struct exmpl.
My question is how do I push new values into the hash that is inside the struct?
I figured how to work with the value in the struct:
$hash_of_structs { "one" } -> val ("1");
printf ( "The value is: %d\n",$hash_of_structs { "one" }-> val );
But it's not working the same way with the hash in the struct. I tried:
$hash_of_structs { "one" } => hash{"uno"}("1");
Thanks :)
Use the following syntax. If a hash reference is passed, the old content is forgotten, if you supply two arguments, a key - value pair is added.
$hash_of_structs{one}->hash({'A', 'a', 'B', 'b'});
$hash_of_structs{one}->hash('key', 'value');
How to use dot in field name ?
I see error in example:
db.test2.insert({ "a.a" : "b" })
can't have . in field names [a.a]
You can replace dot symbols of your field name to Unicode equivalent of \uff0E
db.test.insert({"field\uff0ename": "test"})
db.test.find({"field\uff0ename": "test"}).forEach(printjson)
{ "_id" : ObjectId("5193c053e1cc0fd8a5ea413d"), "field．name" : "test" }
See more:
http://docs.mongodb.org/manual/faq/developers/#faq-dollar-sign-escaping
http://docs.mongodb.org/manual/core/document/#dot-notation
Actualy you may use dots in queries. See: http://www.mongodb.org/display/DOCS/Dot+Notation+%28Reaching+into+Objects%29
Because of this special dot symbol mean you cannot use it in field names. Like you cannot use dot symbol in identifiers in most of programming languages.
You may write query db.test2.find({ "a.a" : "b" }) but if you want to be able to write such a query you need to insert your object like so: db.test2.insert({"a": {"a": "b"}}). This will create document with the field named "a" with the value of embeded document containing the field named "a" (again) with the value "b".
You can also write a SONManipulator using the pymongo library that transforms the data going to and back out of mongodb. There are downsides; there is a performance hit (impact depends on your use case) and you have to transform your keys when you do searches using find.
Here's code with an example of how to use it in the comment for the KeyTransform class:
from pymongo.son_manipulator import SONManipulator
class KeyTransform(SONManipulator):
"""Transforms keys going to database and restores them coming out.
This allows keys with dots in them to be used (but does break searching on
them unless the find command also uses the transform).
Example & test:
# To allow `.` (dots) in keys
import pymongo
client = pymongo.MongoClient("mongodb://localhost")
db = client['delete_me']
db.add_son_manipulator(KeyTransform(".", "_dot_"))
db['mycol'].remove()
db['mycol'].update({'_id': 1}, {'127.0.0.1': 'localhost'}, upsert=True,
manipulate=True)
print db['mycol'].find().next()
print db['mycol'].find({'127_dot_0_dot_0_dot_1': 'localhost'}).next()
Note: transformation could be easily extended to be more complex.
"""
def __init__(self, replace, replacement):
self.replace = replace
self.replacement = replacement
def transform_key(self, key):
"""Transform key for saving to database."""
return key.replace(self.replace, self.replacement)
def revert_key(self, key):
"""Restore transformed key returning from database."""
return key.replace(self.replacement, self.replace)
def transform_incoming(self, son, collection):
"""Recursively replace all keys that need transforming."""
for (key, value) in son.items():
if self.replace in key:
if isinstance(value, dict):
son[self.transform_key(key)] = self.transform_incoming(
son.pop(key), collection)
else:
son[self.transform_key(key)] = son.pop(key)
elif isinstance(value, dict): # recurse into sub-docs
son[key] = self.transform_incoming(value, collection)
return son
def transform_outgoing(self, son, collection):
"""Recursively restore all transformed keys."""
for (key, value) in son.items():
if self.replacement in key:
if isinstance(value, dict):
son[self.revert_key(key)] = self.transform_outgoing(
son.pop(key), collection)
else:
son[self.revert_key(key)] = son.pop(key)
elif isinstance(value, dict): # recurse into sub-docs
son[key] = self.transform_outgoing(value, collection)
return son
def remove_dots(data):
for key in data.keys():
if type(data[key]) is dict: data[key] = remove_dots(data[key])
if '.' in key:
data[key.replace('.', '\uff0E')] = data[key]
del data[key]
return data
this recursive method replaces all dot characters from keys of a dict with \uff0E
as suggested by Fisk
I replaced the key value using myString.replace(".","\u2024") before inserting it into the JsonObject.
Initially I used a simple recursion to replace all "." characters with its unicode equivalent but figured it out that even the dots in the values was getting replaced. So I thought that we should replace the dots only from keys and made the changes accordingly in case "if isinstance(input, dict)".
I thought it should be a sufficient condition to do the magic but I forgot that dict value can also be a dict or a list and then I finally added that check that if value of a dict was not string then, go inside recursively and was finally able to come up with this solution which eventually did the trick.
def remove_dots(data):
if isinstance(data, dict):
return {remove_dots(key): value if isinstance(value, str) else remove_dots(value) for key,value in data.iteritems()}
elif isinstance(data, list):
return [remove_dots(element) for element in data]
elif isinstance(data, str):
return data.replace('.','\u002e')
else:
return data
I've only really come across this problem when trying to serialize Dictionaries and such where the offending dot can appear as a key name.
Edited to show the references.
The quick and dirty C# approach:
using MongoDB.Bson;
using Newtonsoft.Json.Linq;
using System.Text.RegularExpressions;
public static T Sanitize<T>(T obj)
{
var str = JObject.FromObject(obj).ToJson();
var parsed = Regex.Replace(str, #"\.(?=[^""]*"":)", "_"); //i.e. replace dot with underscore when found as a json property name { "property.name": "don't.care.what.the.value.is" }
return JObject.Parse(parsed).ToObject<T>();
}
I've got an array of hashes. I want the a list of the values in a key of those hashes based on the uniqueness of another key.
my #obs = ({
value => 'three',
id => 3
},{
value => 'one-2',
id => 1
},{
value => 'one',
id => 1
});
# This works, prints "one\nthree"
say for values %{{ map { $_->{id} => $_->{value} } #obs }};
Can I avoid the reference + dereference bit around the map? At first I tried just calling values directly on the return from map but Perl won't have it:
Type of arg 1 to values must be hash (not map iterator) at script\workbench.pl line 55, near "#obs ;"
The problem is that values really, really wants a hash to operate on. That's because it's special: it clears the place holder used by each. It needs an actual object to clear that on.
You can go one of two ways here. First, if you don't like the ref/deref, you could just pull the creation of the temporary hash out of the one-liner (please pick a better name than %h for your actual code):
my %h = map { $_->{id} => $_->{value} } #obs;
say for values %h;
If you don't want %h to hang around, just drop it into a temporary block:
...code code code...
{
my %h = map { $_->{id} => $_->{value} } #obs;
say for values %h;
}
...code code code...
Another approach could be to emulate what your temporary hash creation and values is doing:
my %seen;
for ( reverse #obs ) { say $_->{value} unless $seen{$_->{id}}++ }
What really matters is what you're going to be doing with this data. If you just need the values of your inverted hash just once, then your one-liner may be the best solution. If you need this data (id & value) later on, then create the actual hash and use that -- don't do this transform more than once just so that you can keep them as one-liners.
Without further context, it's hard to give advice on which approach to take.
If values were to work on a list, it would take every second element of that list. So
say for values %{{ map { $_->{id} => $_->{value} } #obs }};
would be
say for every_odd map { $_->{id} => $_->{value} } #obs;
Now, it's entirely possible to write such a function, but it's simply not needed in this case. One can simply do
say for map { $_->{value} } #obs;
And that simplifies to
say $_->{value} for #obs;
One catch: By not using a hash, you don't eliminate duplicates.