If I write "\char"0254" in the source file XeLaTeX produces a space instead of reverse c. Reverse c stands for the sound "aw" as in "claw". I am running XeLaTeX in Windows 10. It is reputed to print Unicode characters so I would have thought it would print this one. Is there another way to do this?
Related
I have a file with this unicode character ỗ
File saved in notepad as UTF-8
I tried this line
C:\blah>perl -wln -e "/\x{1ed7}/ and print;" blah.txt
But it's not picking it up. If the file has a letter like 'a'(unicode hex 61), then \x{61} picks it up. But for a 4 digit unicode character, I have an issue picking up the character.
You had the right idea with using /\x{1ed7}/. The problem is that your regex wants to match characters but you're giving it bytes. You'll need to tell Perl to decode the bytes from UTF-8 when it reads them and then encode them to UTF-8 when it writes them:
perl -CiO -ne "/\x{1ed7}/ and print" blah.txt
The -C option controls how Unicode semantics are applied to input and output filehandles. So for example -CO (capital 'o' for 'output') is equivalent to adding this before the start of your script:
binmode(STDOUT, ":utf8")
Similarly, -CI is equivalent to:
binmode(STDIN, ":utf8")
But in your case, you're not using STDIN. Instead, the -n is wrapping a loop around your code that opens each file listed on the command-line. So you can instead use -Ci to add the ':utf8' I/O layer to each file Perl opens for input. You can combine the -Ci and the -CO as: -CiO
Your script works fine. The problem is the unicode you're using for searching. Since your file is utf-8 then your unique search parameters need to be E1, BB, or 97. Check the below file encoding and how that changes the search criteria.
UTF-8 Encoding: 0xE1 0xBB 0x97
UTF-16 Encoding: 0x1ED7
UTF-32 Encoding: 0x00001ED7
Resource https://www.compart.com/en/unicode/U+1ED7
I would like to start by saying that I am not familiar with Perl. That being said, I came across this piece of code and I could not figure out what the \x was for in the code below. In addition, I was unsure why nothing was displayed when I ran the following:
perl -e 'print "\x7c\x8e\x04\x08"'
It's not about print: it's about string representation, in which codes represent characters from your character set. For more information you should read Quote and Quote-like Operators and Effects of Character Semantics
In your case the character code is in hex. You should look in your character set table, and you may need to convert to decimal first.
You said "I was unsure why nothing was displayed when I ran the following:"
perl -e 'print "\x7c\x8e\x04\x08"'
That command outputs 4 characters to STDOUT. Each of the characters is specified in hexadecimal. The "\x7c" part will output the vertical bar character |. The other three characters are control characters, so probably wouldn't produce any visible output. If you redirect output to a file, you will end up with a 4 byte file.
It's possible that you're not seeing the vertical bar character because it's being overwritten by your command prompt. Unlike the shell echo or Python's print, Perl's print function does not automatically append a newline to all output. If you want new lines, you can insert them in the string using \n.
\x signifies the start of a hexadecimal character notation.
I need to print label with special character like degree (°).
I'm using qz print applet on my website.
How i can say to applet that i'm going to print UtF-8 character?
Because it doesn't print correctly.
Thanks!
Well, you need to escape characters above ASCII by putting ^FH (Field Hexadecimal Indicator) before any ^FD field that might contain an UTF char and you also need to prefix the UTF Hex code with an underscore. Like happens in this other question: Unicode characters on ZPL printer
I want to read an input file line by line, but this file has unknown ending character.
Editor vim does not know it either, it represents this character as ^A and immediately starts with characters from new line. The same is for perl. It tried to load all lines in once, because it ignores these strange end of line character.
How can I set this character as end of line for perl? I don't want to use any special module for it (because of our strict system), I just want to define the character (maybe in hex code) of end of line.
The another option is to convert the file to another one, with good end of line character (replace them). Can I make it in some easy way (something like sed on input file)? But everything need to be done in perl.
It is possible?
Now, my reading part looks like:
open (IN, $in_file);
$event=<IN>; # read one line
The ^A character you mention is the "start of heading" character. You can set the special Perl variable $/ to this character. Although, if you want your code to be readable and editable by the guy who comes after you (and uses another editor), I would do something like this:
use English;
local $INPUT_RECORD_SEPARATOR = "\cA" # 'start of heading' character
while (<>)
{
chomp; # remove the unwanted 'start of heading' character
print $_ . "\n";
}
From Perldoc:
$INPUT_RECORD_SEPARATOR
$/
The input record separator, newline by default. This influences Perl's idea of what a "line" is.
More on special character escaping on PerlMonks.
Oh and if you want, you can enter the "start of heading" character in VI, in insert mode, by pressing CTRL+V, then CTRL+A.
edit: added local per Drt's suggestion
I am using Perl to read UTF-16LE files in Windows 7.
If I read in an ASCII file with following code then each "\r\n" in file will be converted into a "\n" in memory:
open CUR_FILE, "<", $asciiFile;
If I read in an UTF-16LE(windows 1200) file with following code, this inconsistency cause problems when I trying to regexp lines with line breaks.
open CUR_FILE, "<:encoding(UTF-16LE)", $utf16leFile;
Then "\r\n" will keep unchanged.
Update:
For each line of a UTF-16LE file:
line =~ /(.*)$/
Then the string matched in $1 will include a "\r" at the end...
What version of Perl are you using? UTF-16 and CRLF handling did not mix properly before 5.8.9 (Unicode changes in 5.8.9). I'm not sure about 5.10.0, but it works in 5.10.1 and 5.8.9. You might need to use "<:encoding(UTF-16LE):crlf" when opening the file.
That is windows performing that magic for you.... If you specify UTF this is the equivalent of opening the file in binary mode vs text.
Newer versions of Perl have the \R which is a generic newline (ie, will match both \r\n and \n) as well as \v which will match all the OS and Unicode notions of vertical whitespace (ie, \r \n \r\n nonbreaking space, etc)
Does you regex logic allow using \R instead of \n?