PUPPET Invalid byte Sequence in UTF-8 - encoding

I've got a big problem with my puppet infrastructure.
I have a file which i want to copy to an other node.
This File is coded in us-ascii.
IMPORTANT: I am not allowed to change something of this file.
Do someone have the same problem or an solution?
My puppet class:
class addfile::copy{
file {'/tmp/FILE.def':
ensure => file,
owner => 'root',
group => 'root',
mode => '0755',
source => 'puppet:///modules/addfile/FILE.def',
}
notify {'File COPY':}
}
My FILE.def contains all characters and all special characters.
My error message:
Error: invalid byte sequence in UTF-8
Error: /Stage[main]/Addfile::Copy/File[/tmp/FILE.def]/content: change from {md5}XXIDXXX to {md5}XXIDXXX failed: invalid byte sequence in UTF-8

This is a known problem with Puppet. Apparently it is something to do with the way diffs are displayed during puppet apply when the show_diff option is on.
https://tickets.puppetlabs.com/browse/PUP-1441
The ticket implies the problem is fixed in recent versions.
As a workaround, try running puppet agent with --no-show_diff:
puppet agent -t --no-show_diff

Related

Passing parameters to puppet manifest via command line

I have been searching for an answer to this question with no luck, but is there a way to pass parameters into puppet manifests when running the 'apply' command, in a similar way to the way you pass parameters when running a UNIX script on the command line?
The suggestions I see mention either keeping variables at the top of the manifest for use later, or to store them in a hiera file. But neither really answer the question I am posing?
Any guidance on how to do this would be greatly appreciated?
Edit:
An example of what I have been doing is:
$doc_root = "/var/www/example"
exec { 'apt-get update':
command => '/usr/bin/apt-get update'
}
package { 'apache2':
ensure => "installed",
require => Exec['apt-get update']
}
file { $doc_root:
ensure => "directory",
owner => "www-data",
group => "www-data",
mode => 644
}
file { "$doc_root/index.html":
ensure => "present",
source => "puppet:///modules/main/index.html",
require => File[$doc_root]
}
As you can see the variable is hardcoded at the top, whereas whilst I am trying to use the variable in the same way, I need to be able to pass the value in when running the apply command.
Using lookup functions in conjunction with hiera.yaml files doesn't fulfil my requirements for the same reason.
The only thing I can think may be a work around is to create a UNIX script that accepts parameters, saves those values in a yaml file, and then have the script execute the .pp file.
But I'm hoping that puppet has a way to do this directly.
The common procedure for passing variables into a classless manifest for use with the puppet apply subcommand would be to assign the value to a Facter fact from the CLI, and then resolve its value inside the manifest. You would begin with removing the hardcoded variable doc_root from the head of the manifest. Then, you would modify the variable into a fact like:
file { $facts['doc_root']:
...
file { "${facts['doc_root']}/index.html":
...
require => File["${facts['doc_root']}"] <-- interpolation required due to Puppet DSL inability to resolve hash value as first class expression
You would then pass the Facter value from the puppet apply subcommand like:
FACTER_doc_root=/var/www/example puppet apply manifest.pp
Note this also causes FACTER_doc_root to be temporarily set as an environment variable as a side effect.

How to resolve a "Metadata error: chr must be valid" error on Linux?

I am relatively new to the world of coding, so I am having trouble resolving an issue when running TranslocWrapper.pl tutorial_metadata.txt preprocess/ results/ --threads 2. I am trying to run the HTGTS Pipeline according to this GitHub project. This is the full error:
. Library Genome Chr Start End Strand
1 RAG1A_SRep2 hg19 chr11 36594878 36595030 -
Metadata error: chr must be valid at /home/micah/transloc_pipeline/bin/TranslocWrapper.pl line 285.
main::check_validity_of_metadata('HASH(0x2903ac8)') called at /home/micah/transloc_pipeline/bin/TranslocWrapper.pl line 248
main::read_in_meta_file() called at /home/micah/transloc_pipeline/bin/TranslocWrapper.pl line 90
I have already double-checked the successful installation of the Software Dependencies, so everything should be all good, but I am having trouble interpreting the "Metadata error: chr must be valid at ..." line. If it helps, these are the specific lines that are being called in the error:
TranslocWrapper.pl line 285:
croak "Metadata error: chr must be valid" unless grep { $_ eq $expt->{chr} } #chrlist;
TranslocWrapper.pl line 248:
check_validity_of_metadata($expt);
TranslocWrapper.pl line 90:
read_in_meta_file;
Thanks in advance for the help!
So the error is saying that one of the sequence characters in the metadata file is not present in the sequence's assembly file.
Given that this is the provided example you should assume that the data is correct and your invocation is faulty.
Have you done the TranslocPreprocess.pl preprocessing steps?
If you have try looking at the first line of the metadata file, identify the assembly entry. Ensure that the assembly file exists and that it contains the required sequence.
One common problem with this kind of code is the case of the filenames. The examples are designed to be run in Linux where filename case matters. Windows likes to pretend that case doesn't matter, this can cause problems. If you are running this code from Microsoft Windows or extracted any of the archives from within Windows this is a likely cause of the error.

gcloud crashed (UnicodeEncodeError): 'ascii' codec can't encode character u'\xe7' in position 13: ordinal not in range(128)

Welcome to the Google Cloud SDK! Run "gcloud -h" to get the list of
available commands.
C:\Program Files (x86)\Google\Cloud SDK>gcloud init Welcome! This
command will take you through the configuration of gcloud.
Your current configuration has been set to: [default]
You can skip diagnostics next time by using the following flag:
gcloud init --skip-diagnostics
Network diagnostic detects and fixes local network connection issues.
Checking network connection...done. Reachability Check passed. Network
diagnostic (1/1 checks) passed.
ERROR: gcloud crashed (UnicodeEncodeError): 'ascii' codec can't encode
character u'\xe7' in position 13: ordinal not in range(128)
If you would like to report this issue, please run the following
command: gcloud feedback
To check gcloud for common problems, please run the following command:
gcloud info --run-diagnostics
I don't know what to do... I just want to use the sdk but I can't init it...
Have a look at the file which gave error , edit the .py file to add following lines
import sys
reload(sys)
sys.setdefaultencoding('utf8')
Try edit the code in google-cloud-sdk/lib/third_party/socks/__init__.py(google-cloud-sdk is the archive you downloaded to install google cloud sdk) in line 262:
req = req + struct.pack(">H", destport)
to
if isinstance(req, unicode):
req = req.encode('UTF-8')
req = req + struct.pack(">H", destport)
reference: https://c11e.wodemo.com/gcloud-crashed-unicodedecodeerror
For me the fix was removing an accent ('é') from a folder in the path of my project. Hope it can help someone since I didn't find this solution after googling it for hours.
I kept getting this similar error every time I ran a gcloud command after a crash:
ERROR: gcloud crashed (UnicodeDecodeError): 'utf8' codec can't decode byte 0xa4 in position 1: invalid start byte
The solution was to delete this file:
~/.config/gcloud/gce
Don't ask me why that works or what that files does, I don't know (if you do, please let me know), but it gets recreated on the next command run and it fixed my issue.
In my case, it was a special character in the folders of the current directory. After changing the current directory, it worked!
As you’ve noticed that the error is due to a non-ASCII character in the username. As a workaround, you can set the CLOUDSDK_CONFIG environment variable to a path that contains only ASCII characters.

Reading SNMP values from a device with multiple view-based ACM contexts, using Net::SNMP

I am trying to use Perl and Net::SNMP to query a device that has multiple configured views/contexts (a Cisco ACE 4710, for example). The equivalent snmpwalk command is:
snmpwalk -c public#CONTEXT_NAME -v 2c 1.2.3.4 '.1.3.6.1.4.1.9.9.480.1.1.2'
I can enumerate the various contexts/views with SNMP-VIEW-BASED-ACM-MIB, e.g.:
my $vacmContextName = '.1.3.6.1.6.3.16.1.1.1.1';
my ($session, $error) = Net::SNMP->session(-hostname => '1.2.3.4', -community => 'public');
my %contexts = %{ $snmp->get_entries(-columns => [ $vacmContextName ]) };
...but am having trouble now reading any further OIDs from each specific context, e.g.:
my $ciscoL4L7ResourceLimitTable = '.1.3.6.1.4.1.9.9.480.1.1.2';
my ($session, $error) = Net::SNMP->session(-hostname => '1.2.3.4', -community => 'public#CONTEXT_NAME');
my %stats = %{ $snmp->get_entries(-columns => [ $ciscoL4L7ResourceLimitTable ]) }
If I run with -debug => DEBUG_ALL, I see the data being returned in the packet inspection (with recognisable data from that context), but then I get a lot of the following errors:
error: [131] Net::SNMP::Security::Community::process_incoming_msg(): The community name "public#CONTEXT_NAME" was expected, but "public" was found
error: [218] Net::SNMP::MessageProcessing::prepare_data_elements(): The community name "public#CONTEXT_NAME" was expected, but "public" was found
error: [398] Net::SNMP::Dispatcher::_transport_response_received(): The community name "public#CONTEXT_NAME" was expected, but "public" was found
error: [1234] Net::SNMP::__ANON__(): The community name "public#CONTEXT_NAME" was expected, but "public" was found
error: [2363] Net::SNMP::__ANON__(): The community name "public#CONTEXT_NAME" was expected, but "public" was found
...and the resulting contents of %stats is undef.
If I try with -community => 'public', it works, but I only get values from the default context (which doesn't contain everything I need).
If I try with -contextname => 'CONTEXT_NAME', I just get:
error: [2423] Net::SNMP::_context_name(): The contextName argument is only supported in SNMPv3
Is it not possible to do what I need with Net::SNMP?
This is perl, v5.10.1 (*) built for i386-linux-thread-multi
CPAN_FILE D/DT/DTOWN/Net-SNMP-v6.0.1.tar.gz
INST_VERSION v6.0.1
From an email conversation I just had with the package author, Word-of-$DEITY is:
Cisco's "context implementation" is vendor specific and is not standards based. The use of "contexts" is officially defined in the SNMPv3 RFCs (specifications). The Net::SNMP modules tries to follow the RFCs in its implementation. I would have to go back to the RFCs determine if it is legal to respond to an SNMP v1/2c message with a "community" string that is not the same as the one with which the request was made. This is the problem that you are seeing.
You are most likely not going to get Cisco to change their implementation even if it is violating an RFC, so your only recourse is to comment out the code in the Net/SNMP/Security/Community.pm module that is returning an error on the send/receive of differing community strings. I typically do not add modification to the module to work around vendor specific problems.
Rather than comment out the relevant section of the Community.pm (which is used by far more on my server than just this script), I have instead elected to shim/monkey-patch the method in question in my own code, for example:
no warnings 'redefine';
use Net::SNMP::Security::Community;
*Net::SNMP::Security::Community::process_incoming_msg = sub { return 1; };
This way, the failing method is bypassed in favour of just accepting any returned community string.
Filthy dirty on all counts, but acceptable within my controlled environment. Anyone coming across this in the future should make sure that version != 6.0.1 hasn't changed the way this module works, or what the existing subroutine does before attempting the same.

Asterisk: callfiles and Realtime

Since I'm using Realtime it seems that callfiles don't work properly. As a callfile is executed the phone rings as expected. But asterisk (v 1.6) hang up immediately as the call is answered.
My callfile:
Channel: SIP/1
Callerid: <123>
Context: test
Extension: 100
My Realtime-Extensions:
cont|ext|pr|App
----+---+-+---------
test|100|1|Answer
----+---+-+---------
test|100|2|SayNumber(123)
Error message on the cli:
Channel 'SIP/1-0000001' sent into invalid extension 's' in context 'default', but no invalid handler
It's wired that as I change the extension in the table above from "100" to "s" everything works fine.
Does anybody have a hint?
Update:
Unfortunately also the mv command does not solve the problem. (I also added a further line to my callfile Priority: 1.)
Here are the files:
extconfig.conf
sipusers => mysql,general,sip_users
sippeers => mysql,general,sip_users
extensions => mysql,general,extensions
sip.conf
[general]
language=en
bindport=5060
context=default
canreinvite=no
tos=throughput
nat=yes
Well, I'm not familiar with Realtime, but a posting of the generated sip.conf and extensions.conf would be helpful (at least the relevant sections).
My first instinct though is that you're using cp to copy the call files for asterisk, which is not an atomic operation (files are copied line by line), which can lead to asterisk executing a not-quite-complete call file.
Use the mv command, which is an atomic operation, and will ensure that asterisk is operating on a 100% complete call file.
The reason I suspect this is the issue is that your call file is correct, but if it starts executing with only 2 lines, by default any incoming call will go to the 's' extension of the default context, and if it reads to the third line, it will go to the 's' extension of your test context.
Weird error to be sure.