This works:
say "\c[COMBINING BREVE, COMBINING DOT ABOVE]" # OUTPUT: «̆̇»
However, this does not:
say "\c[0306, 0307]"; # OUTPUT: «IJij»
It's treating it as two different characters. Is there a way to make it work directly by using the numbers, other than use uniname to convert it to names?
The \c[…] escape is for declaring a character by its name or an alias.
0306 is not a name, it is the ordinal/codepoint of a character.
The \x[…] escape is for declaring a character by its hexadecimal ordinal.
say "\x[0306, 0307]"; # OUTPUT: «̆̇»
(Hint: There is an x in a hexadecimal literal 0x0306)
\c uses decimal numbers:
say "\c[774, 775]"
where 774 is the decimal equivalent of 0306, works perfectly.
Related
I have a column that should contain phone numbers but it contains whatever the user wanted. I need to create an update to remove all the characters after an invalid character.
To do this I am using a regex as PATINDEX('%[^0-9+-/()" "]%', [MobilNr]) and it seemed to work until I had some numbers as +1235, 36446 and to my surprise the result is 0 instead of 6. Also if the number contains . it returns 0.
Does PATINDEX ignores dot(".") and comma(",")? Are there other characters that PATINDEX will ignore?
It's not that PATINDEX ignores the comma and the dot, it's your pattern that created this problem.
With PATINDEX, the hyphen char (-) has a special meaning - it's in fact an operator that denotes an inclusive range - like 0-9 denotes all digits between 0 and 9 - so when you do +-/ it means all the chars between + and / (inclusive, of course). The comma and dot chars are within this range, that's why you get this result.
Fixing the pattern is easy: either use | as a logical or, or simply move the hyphen to the end of the pattern:
SELECT PATINDEX('%[^0-9/()" "+-]%', '+1235, 36446') -- Result: 6
Under what circumstances must one use quotes in a YAML file, specifically when using docker-compose.
For instance,
service:
image: "my-registry/repo:tag1"
environment:
ENV1: abc
ENV2: "abc"
ENV3: "a b c"
If spaces are required, for example, must one use quotes around the environment variable, as depicted in ENV3?
After some googling I've found a blog post
that touches this problem as I understood it.
I'll cite the most important part here:
plain scalars:
- a string
- a string with a \ backslash that doesn't need to be escaped
- can also use " quotes ' and $ a % lot /&?+ of other {} [] stuff
single quoted:
- '& starts with a special character, needs quotes'
- 'this \ backslash also does not need to be escaped'
- 'just like the " double quote'
- 'to express one single quote, use '' two of them'
double quoted:
- "here we can use predefined escape sequences like \t \n \b"
- "or generic escape sequences \x0b \u0041 \U00000041"
- "the double quote \" needs to be escaped"
- "just like the \\ backslash"
- "the single quote ' and other characters must not be escaped"
literal block scalar: |
a multiline text
line 2
line 3
folded block scalar: >
a long line split into
several short
lines for readability
Also I have not seen such docker-compose syntax to set env variables. Documentation suggests using simple values like
environment:
- ENV1=abc
- "ENV2=abc"
Where quotes " or ' are optional in this particular example according to what I've said earlier.
To see how to include spaces in env variables you can check out this so answer
Whether or not you need quotes, depends on the parser. Docker-compose AFAIK is still relying on the PyYAML module and that implements most of YAML 1.1 and has a few quirks of its own.
In general you only need to quote what could otherwise be misinterpreted or clash with some YAML construct that is not a scalar string. You also need (double) quotes for things that cannot be represented in plain scalars, single quoted scalars or block style literal or folded scalars.
Misinterpretation
You need to quote strings that look like some of the other data structures:
booleans: "True", "False", but PyYAML also assumes alternatives words like "Yes", "No", "On", "Off" represent boolean values ( and the all lowercase, all uppercase versions should be considered as well). Please note that the YAML 1.2 standard removed references to these alternatives.
integers: this includes string consisting of numbers only. But also hex (0x123) and octal number (0123). The octals in YAML 1.2 are written as 0o123, but PyYAML doesn't support this, however it is best to quote both.
A special integer that PyYAML still supports but again not in the YAML 1.2 specification are sexagesimals: base 60 number separated by colon (:), time indications, but also MAC addresses can be interpreted as such if the values between/after the colons are in the range 00-59
floats: strings like 1E3 (with optional sign ans mantissa) should be quoted. Of course 3.14 needs to be quoted as well if it is a string. And sexagesimal floats (with a mantissa after the number after the final colon) should be quoted as well.
timestamps: 2001-12-15T02:59:43.1Z but also iso-8601 like strings should be quoted to prevent them from being interpreted as timestamps
The null value is written as the empty string, as ~ or Null (in all casing types), so any strings matching those need to be quoted.
Quoting in the above can be done with either single or double quotes, or block style literal or folded scalars can be used. Please note that for the block-style you should use |- resp. >- in order not to introduce a trailing newline that is not in the original string.
Clashes
YAML assigns special meaning to certain characters or character combinations. Some of these only have special meaning at the beginning of a string, others only within a string.
characters fromt the set !&*?{[ normally indicate special YAML constructs. Some of these might be disambiguated depending on the following character, but I would not rely on that.
whitespace followed by # indicates an end of line comment
wherever a key is possible (and within block mode that is in many places) the combination of colon + space (:) indicates a value will be following. If that combination is part of your scalar string, you have to quote.
As with the misinterpretation you can use single or double quoting or block-style literal or folding scalars. There can be no end-of-line comments beyond the first line of a block-style scalar.
PyYAML can additionally get confused by any colon + space within a plain scalar (even when this is in a value) so always quote those.
Representing special characters
You can insert special characters or unicode code-points in a YAML file, but if you want these to be clearly visible in all cases, you might want to use escape sequences. In that case you have to use double quotes, this is the only mode that
allows backslash escapes. And e.g. \u2029. A full list of such escapes can be taken from the standard, but note that PyYAML doesn't implement e.g \/ (or at least did not when I forked that library).
One trick to find out what to quote or not is to use the library used to dump the strings that you have. My ruamel.yaml and PyYAML used by docker-compose, when potentially dumping a plain scalar, both try to read back (yes, by parsing the result) the plain scalar representation of a string and if that results in something different than a string, it is clear quotes need to be applied. You can do so too: when in doubt write a small program dumping the list of strings that you have using PyYAML's safe_dump() and apply quotes anywhere that PyYAML does.
I would like to replace characters with certain Unicode values in a variable with dash. I have two ideas which might work, but I do not know how to check for the value of character:
1/ processing variable as string, checking every characters value and placing these characters in a new variable (replacing those characters which are invalid)
2/ use these magic :-)
$variable = s/[$char_range]/-/g;
char_range should be similar to [0-9] or [A-Z], but it should be values for utf-8 characters. I need range from 0x00 to 0x7F to be exact.
The following expression should replace anything that is not ASCII with a hyphen, which is (I think) what you want to do:
s/[\N{U+0080}-\N{U+FFFF}]/-/g
There's no such thing as UTF-8 characters. There are only characters that you encode into UTF-8. Even then, you don't want to make ranges outside of the magical ones that Perl knows about. You're likely to get more than you expect.
To get the ordinal value for a character, use ord:
use utf8;
my $code_number = ord '😸'; # U+1F638
say sprintf "%#x", $code_number;
However, I don't think that's what you need. It sounds like you want to replace characters in the ASCII range with a -. You can specify ranges of code numbers:
s/[\000-\177]/-/g; # in octal
s/[\x00-\x7f]/-/g; # in hexadecimal
You can specify wide character ordinal values in braces:
s/[\x80-\x{10ffff}]/-/g; # wide characters, replace non-ASCII in this case
When the characters have a common property, you can use that:
s/\p{ASCII}/-/g;
However, if you are replacing things character for character, you might want a transliteration:
$string =~ tr/\000-\177/-/;
Does Io have built in methods that mirror the ord() and chr() functions in other languages (namely being able to take an integer and return the ASCII character associated with it, or take a string character and return the ASCII number for that character)?
Is there a print/write function that allows for formatting of the output? I'm wanting to create ANSI colored output to the command line, and need the means to print an escape character (ASCII character 27) to do that.
For chr() see asCharacter in the Number object.
For ord() either asBinarySignedInteger or asBinaryUnsignedInteger from the Seqence object seems to fit the bill.
# ord
"#" asBinarySignedInteger println # => 64
# chr
64 asCharacter println # => "#"
I'm attempting to decode text which is prefixing certain 'special characters' with \x. I've worked out the following mappings by hand:
\x28 (
\x29 )
\x3a :
e.g. 12\x3a39\x3a03 AM
Does anyone recognise what this encoding is?
It's ASCII. All occurrences of the four characters \xST are converted to 1 character, whose ASCII code is ST (in hexadecimal), where S and T are any of 0123456789abcdefABCDEF.
The '\xAB' notation is used in C, C++, Perl, and other languages taking a cue from C, as a way of expressing hexadecimal character codes in the middle of a string.
The notation '\007' means use octal for the character code, when there are digits after the backslash.
In C99 and later, you can also use \uabcd and \U00abcdef to encode Unicode characters in hexadecimal (with 4 and 8 hex digits required; the first two hex digits in \U must be 0 to be valid, and often the third digit will be 0 too — 1 is the only other valid value).
Note that in C, octal escapes are limited to a maximum of 3 digits but hexadecimal escapes are not limited to 2 or 3 digits; the hexadecimal escape ends at the first character that's not a hexadecimal digit. In the question, the sequence is "12\x3a39\x3a03". That is a string containing 4 characters: 1, 2, \x3a39 and \x3a03. The actual value used for the 4-digit hex characters is implementation-defined. To achieve the desired result (using \x3A to represent a colon :), the code would have to use string concatenation:
"12\x3a" "39\x3a" "03"
This now contains 8 characters: 1, 2, :, 3, 9, :, 0, 3.
I use CyberChef for this sort of thing.
If you drop it in the input field and drag Magic from the Favourites list into the recipe it'll tell you the conversion and that you could've used the From_Hex recipe with a \x delimiter.
I'm guessing that what you are dealing with is a unicode string that has been encoded differently than the output stream it was sent to. ie. a utf-16 string output to a latin-1 device. In that situation, certain characters will be outputted as escape values to avoid sending control characters or wrong characters to the output device. This happens in python at least.