I have this recursive copy action called through
class first_class{
file{'some name':
ensure => 'directory',
path => '/path/to/here',
owner => 'some_owner',
group => 'some_group',
recurse => remote,
source => 'puppet:///modules/scripts'
}
}
class second_class{
file{'tmp':
ensure => 'present',
path => '/path/to/here/tmp.sh',
owner => 'some_owner',
group => 'some_group',
mode => '0755',
notify => Exec['some_process']
}
}
The files are recursively copied but the content is not. So it seems that the file is recreated by the second_class, however in my main manifest file I have
node default {
Class { 'my_module::first_class':} -> Class { 'my_module::second_class':}
Is there a way to tackle it ?
The files are recursively copied but the content is not. So it seems that the file is recreated by the second_class
Actually, no, that's not quite what is happening. Your second_class contains an explicit declaration of File['tmp']. Each resource can be declared only once, and explicit File declarations take precedence over implicit ones generated for the contents of recursively-managed directories. This is a feature. Thus, the file in question is not being recreated by the explicit declaration; rather, it is being managed only according to that declaration.
Because File['tmp'] uses ensure => present, it accepts any form of file (directory, symlink, regular file, etc.), and if there is no such file at all then it will create an empty ordinary file. That's what you observe. It has nothing to do with order of resource application.
Is there a way to tackle it ?
Yes. If you want the file to be managed via File['some name'] then do not declare an explicit resource for it. If you must declare it explicitly, e.g. so as to set its notify property, then make that declaration reflect the complete desired target-machine state.
Overall, I suspect you might benefit from some refactoring, as the situation has some code smell. Be aware also that recursive file management has always been ... quirky ... at best. It has its uses, but often you are better off with something else.
Related
I have a manifest file to add registry keys and values based on facts (works fine).
registry_key { 'HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate':
ensure => present,
}
registry_value { 'HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\TestKey':
ensure => present,
type => dword,
data => $test_key_value,
}
I want to add a second file to remove these if required but when i do i get an error
"Duplicate declaration: Registry_key[HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate] is already declared"
Not sure how to get around this - if at all? Any advice appreciated. Obviously a puppet novice...
Thanks
If you want to solve this problem, you would probably use custom or external facts, something like this:
$ensure = $facts['include_windows_update_keys']
registry_key { 'HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate':
ensure => $ensure,
}
registry_value { 'HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\TestKey':
ensure => $ensure,
type => dword,
data => $test_key_value,
}
As you have discovered, declaring the same resource more than once but with different attributes is not allowed in Puppet.
There is more on custom facts here in the docs.
In most situations Alex's suggestion is the way I'd proceed. Usually the best way, to default it in common.yaml and override based on node name or another level in hiera. Depending on your use case, a less straight-forward way is to wrap those blocks in a conditional (if/unless/else) where it's present/absent depending on a boolean set in hiera. Something along the lines of unless $exclude_from_testkey or a case statement. Let me know if you're new to hiera and/or parameterization.
You have resource declarations for specifying that a particular registry key and value should be ensured present on the target node. If you also have declarations specifying that one or both should be absent, and Puppet evaluates both sets, then what are you actually telling Puppet to do? It cannot comply with both sets of declarations.
Puppet takes an extremely cautious approach to situations like this, which makes sense given its role in managing infrastructure. In the event that the same resource is declared more than once for the same target, Puppet aborts. This produces practical difficulties from time to time, but I am confident that it has protected many, many systems from misconfiguration.
The solution is to ensure that your manifest set declares only one set of those declarations for each node. You could do that by having only one set, and twiddling their $ensure parameters dynamically, as #AlexHarvey suggests. You could also do it by putting the two sets of declarations in different blocks, and selecting between them with conditional statements. Or you could put them in altogether different classes, and be certain to include just one of them for each node.
But I have to differ with Alex here on the specifics. I would not typically use a custom fact here, because that gives control over which option is exercised to the client. Generally speaking, I want the master to dictate questions of how various nodes are configured. For this purpose, it is a pretty common idiom to use a class parameter to control whether the resources are ensured present or absent:
class mymodule::windows_update(
Enum['absent','present'] $ensure = $present,
$test_key_value
) {
registry_key { 'HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate':
ensure => $ensure,
}
registry_value { 'HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\TestKey':
ensure => $ensure,
type => dword,
data => $test_key_value,
}
}
I'm new to scala/java and I'm trying to understand this code below which returns a list of files in a directory. Function source
It takes an argument called dir - is the type of dir a File or File object?
It returns an array of type file.
It calls the method listFiles on dir.
What does the last line do?
def getRecursiveListOfFiles(dir: File): Array[File] = {
val these = dir.listFiles
these ++ these.filter(_.isDirectory).flatMap(getRecursiveListOfFiles)
}
This code does a breadth-first search, using recursion.
A File can either be a file, or a directory.
The code dir.listFiles list all files in directory. Remember that this will be a list of files and directories!
We can then break down the last line into 3 things. These could easily be separate lines.
these.filter(_.isDirectory) will return a list of directories that need searching in. It filters out the files.
flatMap(getRecursiveListOfFiles) takes this list of directories, and calls getRecursiveListOfFiles for every single directory. It then adds flattens these results into one list.
++ adds together two arrays. We add these to the result of the recursive call.
flatMap is key here. Read up about it, and how it differs from the map function to fully understand what's going on.
In short:
these ++ these.filter(_.isDirectory).flatMap(getRecursiveListOfFiles)
is fundamentally:
val allSubDirectories:Array[Files] = these.filter(_.isDirectory)
allSubDirectories.flatMap(getRecursiveListOfFiles)
//i.e. for each sub-directory, again find all files in sub-directory
these ++ (files of all sub-directories)
//ultimately add files of sub-directory to the actual list
Another alternative way to understand would be:
def getAllFiles(dir: File): List[File] = {
val these = dir.listFiles.toList
these ::: these.filter(_.isDirectory).map(x => getAllFiles(x)).flatten
}
Fundamentally same control flow, that for each sub-directory, you get a list of all the files and then to the same list, you add files of sub-directory.
I'm going to try to walk you through how I would think of this as someone trying to read a piece of unknown code.
It should be pretty clear by context that File can either be a regular file or directory. It doesn't contain the file contents, but represents any entry in the file system, and you can open the file with other library commands. For the purposes of this function, File is just something that happens to contain more Files in the case that it's a directory, and these can be accessed via the method listFiles: List[Files]. Presumably, it also provides other info, so that the original caller of getRecursiveListOfFiles could do something with the resulting list.
Also by context, these is pretty clearly the entries in the current directory.
The last line is the most subtle. But to break it down, it augments these with the Files found in those entries in these which happen to be directories.
To explain this step, the signature of flatMap on a List[File] can be thought of as flatMap[B](f: File => List[B]): List[B], where B is a type variable. In this case, because the same function getRecursiveListOfFiles, which is of type File => List[File], is being passed recursively, B is just File, so we can think of this particular call as flatMap(f: File => List[File]): List[File].
Roughly speaking, flatMap applies a function f to each item in a container, where f is required to return the same type of container. The "flat" part is simply the fact that these individual containers get combined, instead of being nested, which is what map would do. This is what allows the function to recursively add all the files found in its subdirectories. Pretty slick.
What's the distinction you're drawing? The type of dir is File.
An array of Files, yes.
It's probably implicitly converting these to a scala List or Buffer, which is the confusing part (e.g. it may import scala.collection.JavaConversions._ - it's better to use JavaConverters which makes the conversions explicit calls to .asScala or asJava). You can find the definitions of ++, filter and flatMap in the scaladoc, which will hopefully be enough to understand what's happening.
(note: my 4. is coming out as a 3. because of markdown :()
I'm switching from using DBIx::Class::Schema::Loader in dynamic mode to static.
But there's a problem, my result classes are mixed up with non result classes. Here's the specifics.
lib/BackPAN/Index.pm # main API
lib/BackPAN/Index/Dist.pm # result class
lib/BackPAN/Index/File.pm # result class
lib/BackPAN/Index/Release.pm # result class
lib/BackPAN/Index/Schema.pm # subclass of DBIC::Schema::Loader
lib/BackPAN/Index/Role/... # various roles
When I switch to static schema generation it gets tripped up by the role.
DBIx::Class::Schema::load_namespaces(): Attempt to load_namespaces()
class BackPAN::Index::Role::HasCache failed - are you sure this is a
real Result Class?
I'm stuck with this class layout. BackPAN::Index::Dist, File and Release are all publicly documented. Many methods are expected to return them as the result of queries.
I need a way to use DBIx::Class::Schema::Loader in static mode while BackPAN::Index::Dist, File and Release are used as result classes.
I've been trying to make DBIx::Class::Schema::Loader spell out the generated result classes rather than relying on load_namespaces to search the subdirectory.
Ideally, I'd like the generated result classes to be in their own subdirectory with Dist, File and Release as subclasses for easier customization. However, queries against the schema must return the customized subclasses.
Normally you have a Result and a ResultSet namespace in which the corresponding classes reside:
BackPAN::Index::Schema::Result::Dist
BackPAN::Index::Schema::ResultSet::Dist
The DBIx::Class::Schema#load_namespaces docs show an example of setting them to non-default values.
You can also use DBIx::Class::Schema#load_classes and specify each class:
BackPAN::Index::Schema->load_classes({
BackPAN::Index => [qw( Dist File Release )],
});
Normally it's not a problem to move Result and ResultSet classes into different namespaces because they are always accessed through an instance of the Schema which loads them.
I suggest trying to move them and see if it really breaks something before going with load_classes.
I realized the important part of Schema::Loader is making the result classes. The schema is simple and I can make it by hand. Unfortunately there's no way to tell Schema::Loader not to generate the schema. I've hacked around it by telling it to make a dummy and just delete the file.
DBIx::Class::Schema::Loader::make_schema_at(
'BackPAN::Index::SchemaThrowaway',
{
result_namespace => '+BackPAN::Index',
use_namespaces => 1,
dump_directory => 'lib',
},
);
# Throw the generated schema away.
unlink "lib/BackPAN/Index/SchemaThrowaway.pm";
Then I write the schema class by hand.
package BackPAN::Index::Schema;
use strict;
use warnings;
use base 'DBIx::Class::Schema';
__PACKAGE__->load_classes({
"BackPAN::Index" => [qw(Dist File Release)],
});
Its a hack, but it works. Still looking for a better solution.
First off, I'm not really sure how much information is necessary to include because I'm having a really hard time tracing the origin of this problem.
I have a Moose role with a subroutine that (along with a few other things) tries to set the attributes for a class like this:
$genre = Movie::Genre->new({
genreName => 'Drama',
genreID => '1'
});
The problem is, it doesn't. The dump of $genre immediately after, indicates that it's still empty:
$genre: bless( {}, 'Movie::Genre' )
Stranger still, when I execute THE EXACT SAME LINE in my test file, it works as expected with this dump:
$genre: bless( {
genreID => '1',
genreName => 'Drama'
}, 'Movie::Genre' )
I'm struggling to find what makes these two lines of code different, causing one to work and one to fail.
Any ideas as to what conditions would cause the first example to fail and allow the second to succeed? I'd be happy to provide more context if necessary. Thanks!
That line simply passes those parameters to the Movie::Genre constructor. It's up to that constructor to decide what to do with them.
It sounds like that call (in the role) is getting executed before the Movie::Genre class has acquired attributes named genreName and genreID. By default, Moose constructors ignore any parameters they don't recognize, so this doesn't generate a warning.
Your test file must be making that call after the attributes have been added to Movie::Genre.
We'd have to see more of the code to figure out exactly why this is happening.
This may turn out to be an embarrassingly stupid question, but better than potentially creating embarrassingly stupid code. :-) This is an OO design question, really.
Let's say I have an object class 'Foos' that represents a set of dynamic configuration elements, which are obtained by querying a command on disk, 'mycrazyfoos -getconfig'. Let's say that there are two categories of behavior that I want 'Foos' objects to have:
Existing ones: one is, query ones that exist in the command output I just mentioned (/usr/bin/mycrazyfoos -getconfig`. Make modifications to existing ones via shelling out commands.
Create new ones that don't exist; new 'crazyfoos', using a complex set of /usr/bin/mycrazyfoos commands and parameters. Here I'm not really just querying, but actually running a bunch of system() commands. Affecting changes.
Here's my class structure:
Foos.pm
package Foos, which has a new($hashref->{name => 'myfooname',) constructor that takes a 'crazyfoo NAME' and then queries the existence of that NAME to see if it already exists (by shelling out and running the mycrazyfoos command above). If that crazyfoo already exists, return a Foos::Existing object. Any changes to this object requires shelling out, running commands and getting confirmation that everything ran okay.
If this is the way to go, then the new() constructor needs to have a test to see which subclass constructor to use (if that even makes sense in this context). Here are the subclasses:
Foos/Existing.pm
As mentioned above, this is for when a Foos object already exists.
Foos/Pending.pm
This is an object that will be created if, in the above, the 'crazyfoo NAME' doesn't actually exist. In this case, the new() constructor above will be checked for additional parameters, and it will go ahead and, when called using ->create() shell out using system() and create a new object... possibly returning an 'Existing' one...
OR
As I type this out, I am realizing it is perhaps it's better to have a single:
(an alternative arrangement)
Foos class, that has a
->new() that takes just a name
->create() that takes additional creation parameters
->delete(), ->change() and other params that affect ones that exist; that will have to just be checked dynamically.
So here we are, two main directions to go with this. I'm curious which would be the more intelligent way to go.
In general it's a mistake (design-wise, not syntax-wise) for the new method to return anything but a new object. If you want to sometimes return an existing object, call that method something else, e.g. new_from_cache().
I also find it odd that you're splitting up this functionality (constructing a new object, and returning an existing one) not just into separate namespaces, but also different objects. So in general, you're closer with your second approach, but you can still have the main constructor (new) handle a variety of arguments:
package Foos;
use strict;
use warnings;
sub new
{
my ($class, %args) = #_;
if ($args{name})
{
# handle the name => value option
}
if ($args{some_other_option})
{
# ...
}
my $this = {
# fill in any fields you need...
};
return bless $this, $class;
}
sub new_from_cache
{
my ($class, %args) = #_;
# check if the object already exists...
# if not, create a new object
return $class->new(%args);
}
Note: I don't want to complicate things while you're still learning, but you may also want to look at Moose, which takes care of a lot of the gory details of construction for you, and the definition of attributes and their accessors.
It is generally speaking a bad idea for a superclass to know about its subclasses, a principle which extends to construction.[1] If you need to decide at runtime what kind of object to create (and you do), create a fourth class to have just that job. This is one kind of "factory".
Having said that in answer to your nominal question, your problem as described does not seem to call for subclassing. In particular, you apparently are going to be treating the different classes of Foos differently depending on which concrete class they belong to. All you're really asking for is a unified way to instantiate two separate classes of objects.
So how's this suggestion[3]: Make Foos::Exists and Foos::Pending two separate and unrelated classes and provide (in Foos) a method that returns the appropriate one. Don't call it new; you're not making a new Foos.
If you want to unify the interfaces so that clients don't have to know which kind they're talking about, then we can talk subclassing (or better yet, delegation to a lazily-created and -updated Foos::Handle).
[1]: Explaining why this is true is a subject hefty enough for a book[2], but the short answer is that it creates a dependency cycle between the subclass (which depends on its superclass by definition) and the superclass (which is being made to depend on its subclass by a poor design decision).
[2]: Lakos, John. (1996). Large-scale C++ Software Design. Addison-Wesley.
[3]: Not a recommendation, since I can't get a good enough handle on your requirements to be sure I'm not shooting fish in a dark ocean.
It is also a factory pattern (bad in Perl) if the object's constructor will return an instance blessed into more than one package.
I would create something like this. If the names exists than is_created is set to 1, otherwise it is set to 0.. I would merge the ::Pending, and ::Existing together, and if the object isn't created just put that into the default for the _object, the check happens lazily. Also, Foo->delete() and Foo->change() will defer to the instance in _object.
package Foo;
use Moose;
has 'name' => ( is => 'ro', isa => 'Str', required => 1 );
has 'is_created' => (
is => 'ro'
, isa => 'Bool'
, init_arg => undef
, default => sub {
stuff_if_exists ? 1 : 0
}
);
has '_object' => (
isa => 'Object'
, is => 'ro'
, lazy => 1
, init_arg => undef
, default => sub {
my $self = shift;
$self->is_created
? Foo->new
: Bar->new
}
, handles => [qw/delete change/]
);
Interesting answers! I am digesting it as I try out different things in code.
Well, I have another variation of the same question -- the same question, mind you, just a different problem to the same class:subclass creation issue!
This time:
This code is an interface to a command line that has a number of different complex options. I told you about /usr/bin/mycrazyfoos before, right? Well, what if I told you that that binary changes based on versions, and sometimes it completely changes its underlying options. And that this class we're writing, it has to be able to account for all of these things. The goal (or perhaps idea) is to do: (perhaps called FROM the Foos class we were discussing above):
Foos::Commandline, which has as subclasses different versions of the underlying '/usr/bin/mycrazyfoos' command.
Example:
my $fcommandobj = new Foos::Commandline;
my #raw_output_list = $fcommandobj->getlist();
my $result_dance = $fcommandobj->dance();
where 'getlist' and 'dance' are version-dependent. I thought about doing this:
package Foos::Commandline;
new (
#Figure out some clever way to decide what version user has
# (automagically)
# And call appropriate subclass? Wait, you all are telling me this is bad OO:
# if v1.0.1 (new Foos::Commandline::v1.0.1.....
# else if v1.2 (new Foos::Commandline::v1.2....
#etc
}
then
package Foos::Commandline::v1.0.1;
sub getlist ( eval... system ("/usr/bin/mycrazyfoos", "-getlistbaby"
# etc etc
and (different .pm files, in subdir of Foos/Commandline)
package Foos::Commandline::v1.2;
sub getlist ( eval... system ("/usr/bin/mycrazyfoos", "-getlistohyeahrightheh"
#etc
Make sense? I expressed in code what I'd like to do, but it just doesn't feel right, particularly in light of what was discussed in the above responses. What DOES feel right is that there should be a generic interface / superclass to Commandline... and that different versions should be able to override it. Right? Would appreciate a suggestion or two on that. Gracias.