I have a string where special characters like ! or " or & or # or #, ... can appear. How can I convert in the string
str = " Hello "XYZ" this 'is' a test & so *n #."
automatically every special characters with their html entities, so that I get this:
str = " Hello " ;XYZ" ; this ' ;is' ; a test & ; so on #"
I tried
$str=HTML::Entities::encode_entities($str);
but it does a partial work the # is not transformed in @ ;
SOLUTION:
1) with your help (Quentin and vol7ron) I came up with this solution(1)
$HTML::Entities::char2entity{'#'} = '#';
$HTML::Entities::char2entity{'!'} = '!';
$HTML::Entities::char2entity{'#'} = '#';
$HTML::Entities::char2entity{'%'} = '%';
$HTML::Entities::char2entity{'.'} = '.';
$HTML::Entities::char2entity{'*'} = '*';
$str=HTML::Entities::encode_entities($str, q{#"%'.&#*$^!});
2) and I found a shorter(better) solution(2) found it here:
$str=HTML::Entities::encode_entities($str, '\W');
the '\W' does the job
#von7ron with solution(1) you will need to specify the characters you want to translate as Quentin mentioned earlier even if they are on the translation table.
# isn't transformed because it isn't considered to be a "special character". It can be represented in ASCII and has no significant meaning in HTML.
You can expand the range of characters that are converted with the second argument to the function you are using, as described in the documentation.
You can manually add a character to the translation table (char2entity hash).
$HTML::Entities::char2entity{'#'} = '#';
my $str = q{ Hello "XYZ" this 'is' a test & so on #};
my $encoded = HTML::Entities::encode_entities( $str, q{<>&"'#} );
The above adds #, which will be translated to #.
You then need to specify the characters you want to translate, if you don't it uses <>&", so I added both # and '. Notice, I didn't have to add the ' to the translation table, because it's already there by default.
You don't need to add ASCII characters (0-255) to the char2entity hash, since the module will do it automatically.
Note: Setting the char2entity for #, was done as an example. The module automatically sets numerical entities for ASCII characters (0-255) that weren't found. You'd have to use it for unicode characters, though.
Cheap, dirty, and ugly, but works:
my %translations;
$translations{'"'} = '" ;';
$translations{'\''} = '' ;';
etc...
sub transform()
{
my $str = shift;
foreach my $character (keys(%translations))
{
$str =~ s/$character/$translations{$character}/g;
}
return $str;
}
Related
I have a little Perl script which includes a substring search as follows.
#!/usr/bin/perl
use strict;
use warnings;
my $line = "this && is || a test if && ||";
my $nb_if = findSymbols($line, "if ");
my $nb_and = findSymbols($line, "&&");
my $nb_or = findSymbols($line, "||");
print "\nThe result for this func is $nb_if=if , $nb_and=and, $nb_or=or\n";
sub findSymbols {
my $n = () = ($_[0] =~ m/$_[1]/g);
return $n;
}
It should return:
The result for this func is 1=if , 2=and, 2=or
but, instead it returns:
The result for this func is 1=if , 2=and, 30=or
I don't understand what's wrong with my code.
Use quotemeta to escape the special meaning of the regular expression containing || (and any other characters which you pass to the function):
sub findSymbols {
my $pat = quotemeta $_[1];
my $n = () = ($_[0] =~ m/$pat/g);
return $n;
}
The pipe character (|) has a special meaning in regular expressions. It means "or" (matching either the thing on its left or the thing on its right). So having a regex that consists of just two pipes is interpreted as meaning "match an empty string or an empty string or an empty string" - and that matches everywhere in your string (30 times!)
So you need to stop the pipe being interpreted as a special character and let it just represent an actual pipe character. Here are three ways to do that:
Escape the pipes with backslashes when you're creating the string that you pass to findSymbols().
# Note: I've also changed "..." to '...'
# to avoid having to double-escape
my $nb_or = findSymbols($line, '\|\|');
Use quotemeta() to automatically escape problematic characters in any string passed to findSymbols().
my $escaped_regex = quotemeta($_[0]);
my $n = () = ($_[0] =~ m/$escaped_regex/g);
Use \Q...\E to automatically escape any problematic characters used in your regex.
# Note: In this case, the \E isn't actually needed
# as it's at the end of the regex.
my $n = () = ($_[0] =~ m/\Q$_[0]\E/g);
For more detailed information on using regular expressions in Perl, see perlretut and perlre.
| is the alternation operator in the regular expression used by m//. You need to escape each | with a backslash to match literal |s.
my $nb_or = findSymbols($line, "\\|\\|"); # or '\|\|`
(but using quotemeta as suggested by #toolic is a much better idea, as it frees your caller from having to worry about details that should be part of the abstraction provided by findSymbols.)
Been scratching my head on this one...
I'd like to remove .com and capitalize S and T from: "sometext.com"
So output would be Some Text
Thank you in advance
For most of this you can use the replace() member of the String object.
The syntax is:
$string = $string.replace('what you want replaced', 'what you will replace it with')
Replace can be used to erase things by using blank quotes '' for the second argument. That's how you can get rid of .com
$string = $string.replace('.com','')
It can also be used to insert things. You can insert a space between some and text like this:
$string = $string.replace('et', 'e t')
Note that using replace does NOT change the original variable. The command below will print "that" to your screen, but the value of $string will still be "this"
$string = 'this'
$string.replace('this', 'that')
You have to set the variable to the new value with =
$string = "this"
$string = $string.replace("this", "that")
This command will change the value of $string to that.
The tricky part here comes in changing the first t to capital T without changing the last t. With strings, replace() replaces every instance of the text.
$string = "text"
$string = $string.replace('t', 'T')
This will set $string to TexT. To get around this, you can use Regex. Regex is a complex topic. Here just know that Regex objects look like strings, but their replace method works a little differently. You can add a number as a third argument to specify how many items to replace
$string = "aaaaaa"
[Regex]$reggie = 'a'
$string = $reggie.replace($string,'a',3)
This code sets $string to AAAaaa.
So here's the final code to change sometext.com to Some Text.
$string = 'sometext.com'
#Use replace() to remove text.
$string = $string.Replace('.com','')
#Use replace() to change text
$string = $string.Replace('s','S')
#Use replace() to insert text.
$string = $string.Replace('et', 'e t')
#Use a Regex object to replace the first instance of a string.
[regex]$pattern = 't'
$string = $pattern.Replace($string, 'T', 1)
What you're trying to achieve isn't well-defined, but here's a concise PowerShell Core solution:
PsCore> 'sometext.com' -replace '\.com$' -replace '^s|t(?!$)', { $_.Value.ToUpper() }
SomeText
-replace '\.com$' removes a literal trailing .com from your input string.
-replace '^s|t(?!$), { ... } matches an s char. at the start (^), and a t that is not (!) at the end ($); (?!...) is a so-called negative look-ahead assertion that looks ahead in the input string without including what it finds in the overall match.
Script block { $_.Value.ToUpper() } is called for each match, and converts the match to uppercase.
-replace (a.k.a -ireplace) is case-INsensitive by default; use -creplace for case-SENSITIVE replacements.
For more information about PowerShell's -replace operator see this answer.
Passing a script block ({ ... }) to dynamically determine the replacement string isn't supported in Windows PowerShell, so a Windows PowerShell solution requires direct use of the .NET [regex] class:
WinPs> [regex]::Replace('sometext.com' -replace '\.com$', '^s|t(?!$)', { param($m) $m.Value.ToUpper() })
SomeText
Trying to extract the alphanumeric characters from this string:
A_phase_I-II,_open-req_project_id_PX15RAD001
The problem is: the term PX15RAD001 can occur anywhere in the string.
Trying to extract the alpha-numeric part using the below expression. But this returns the entire string. I thought Alum was a valid keyword for alpha-numerics. Is that not the case?
(my $string = $line ) =~ s/\P{Alnum}//g;
print $string;
How can I extract the alphanumeric part of the afore mentioned string?
Thanks in advance.
-simak
At the end as per your input:
> echo "A_phase_I-II,_open-req_project_id_PX15RAD001"|perl -lne 'print $1 if(/id_([A-Z0-9]*)/)'
PX15RAD001
In the middle:
> echo "A_phase_I-II,_open-req_id_PX15RAD001_project" | perl -lne 'print $1 if(/id_([A-Z0-9]*)/)'
PX15RAD001
or in your terms:
$line=~m/id_([A-Z0-9]*)/g;
print $1;
Here are some testcases, produced with the comments of #Vijay s Answer:
my #line = (
'A_phase_I-II,_open-req_project_id_PX15RAD001',
'_PX15RAD001_A_phase_I-II,_open-req_project_id',
'A_pha3333se_I-II,_ope_PX15RAD001_n-req_project',
'A_phase_I-II,_PX15RAD001_open-req_projec123123123t_id',
'A_phase_I-II_PX15RAD001_roject_id'
);
foreach my $string ( #line ) {
$string =~ m{_([^_]{10})_?}g;
print $1 . "\n" if $1;
}
These kinds of questions are hard to answer because there is not enough information. What information we have is:
You say your target string is "alphanumeric", but the entire input string is alphanumeric, except for some punctuation, so that really doesn't tell us anything.
You say it is 12 characters long, but the sample you show is 10 characters long.
You seem to think that "alphanumeric" does not include underscore.
So, the reliable information I can sense from you is:
Target string is always delimited by underscore _
Target string is 10-12 characters, all alphanumeric except underscore.
The "reliable" solution based on this rather skimpy information is:
my $str = "A_phase_I-II,_open-req_project_id_PX15RAD001";
for my $field (split /_/, $str) {
if (length($field) <= 12 and
length($field) >= 10 and # field is 10-12 characters
$field !~ /\W/) { # and contains no non-alphanumerics
# do something
}
}
By splitting on underscore, we can easily isolate each field in the string and perform simpler tests on it, such as the ones above.
In Perl, how can I convert string containing utf-8 characters to HTML where such characters will be converted into &...; ?
First, split on an empty pattern to get a list of single characters. Then, map each character to itself, if it is ASCII, or its code, if it is not:
use Encode qw( decode_utf8 );
my $utf8_string = "\xE2\x80\x9C\x68\x6F\x6D\x65\xE2\x80\x9D";
my $unicode_string = decode_utf8($utf8_string);
my $html = join q(),
map { ord > 127 ? "&#" . ord . ";"
: $_
} split //, $unicode_string;
Just replace every symbol that is not printable and not low ASCII (that is, anything outside \x20 - \x7F region) with simple calculation of its ord + necessary HTML entity formatting. Perl regexp have /e flag to indicate that replacement should be treated as code.
use utf8;
my $str = "testТест"; # This is correct UTF-8 string right in the code
$str =~ s/([^[\x20-\x7F])/"&#" . ord($1) . ";"/eg;
print $str;
# testТест
Part 3 (Part 2 is here) (Part 1 is here)
Here is the perl Mod I'm using: Unicode::String
How I'm calling it:
print "Euro: ";
print unicode_encode("€")."\n";
print "Pound: ";
print unicode_encode("£")."\n";
would like it to return this format:
€ # Euro
£ # Pound
The function is below:
sub unicode_encode {
shift() if ref( $_[0] );
my $toencode = shift();
return undef unless defined($toencode);
print "Passed: ".$toencode."\n";
Unicode::String->stringify_as("utf8");
my $unicode_str = Unicode::String->new();
my $text_str = "";
my $pack_str = "";
# encode Perl UTF-8 string into latin1 Unicode::String
# - currently only Basic Latin and Latin 1 Supplement
# are supported here due to issues with Unicode::String .
$unicode_str->latin1($toencode);
print "Latin 1: ".$unicode_str."\n";
# Convert to hex format ("U+XXXX U+XXXX ")
$text_str = $unicode_str->hex;
# Now, the interesting part.
# We must search for the (now hex-encoded)
# Unicode escape sequence.
my $pattern =
'U\+005[C|c] U\+0058 U\+00([0-9A-Fa-f])([0-9A-Fa-f]) U\+00([0-9A-Fa-f])([0-9A-Fa-f]) U\+00([0-9A-Fa-f])([0-9A-Fa-f]) U\+00([0-9A-Fa-f])([0-9A-Fa-f])';
# Replace escapes with entities (beginning of string)
$_ = $text_str;
if (/^$pattern/) {
$pack_str = pack "H8", "$1$2$3$4$5$6$7$8";
$text_str =~ s/^$pattern/\&#x$pack_str/;
}
# Replace escapes with entities (middle of string)
$_ = $text_str;
while (/ $pattern/) {
$pack_str = pack "H8", "$1$2$3$4$5$6$7$8";
$text_str =~ s/ $pattern/\;\&#x$pack_str/;
$_ = $text_str;
}
# Replace "U+" with "&#x" (beginning of string)
$text_str =~ s/^U\+/&#x/;
# Replace " U+" with ";&#x" (middle of string)
$text_str =~ s/ U\+/;&#x/g;
# Append ";" to end of string to close last entity.
# This last ";" at the end of the string isn't necessary in most parsers.
# However, it is included anyways to ensure full compatibility.
if ( $text_str ne "" ) {
$text_str .= ';';
}
return $text_str;
}
I need to get the same output but need to Support Latin-9 characters as well, but the Unicode::String is limited to latin1. any thoughts on how I can get around this?
I have a couple of other questions and think I have a somewhat understanding of Unicode and Encodings but having time issues as well.
Thanks to anyone who helps me out!
As you have been told already, Unicode::String is not an appropriate choice of module. Perl ships with a module called 'Encode' which can do everything you need.
If you have a character string in Perl like this:
my $euro = "\x{20ac}";
You can convert it to a string of bytes in Latin-9 like this:
my $bytes = encode("iso8859-15", $euro);
The $bytes variable will now contain \xA4.
Or you can have Perl automatically convert it out output to a filehandle like this:
binmode(STDOUT, ":encoding(iso8859-15)");
You can refer to the documentation for the Encode module. And also, PerlIO describes the encoding layer.
I know you are determined to ignore this final piece of advice but I'll offer it one last time. Latin-9 is a legacy encoding. Perl can quite happily read Latin-9 data and convert it to UTF-8 on the fly (using binmode). You should not be writing more software that generates Latin-9 data you should be migrating away from it.