I got the string \x01\x01 from a tcp/ip socket, when I try to print it to console, no output is coming
void main() {
var out = "\x01\x01";
print("printing out as --> $out <--");
final runes = out.runes.toList();
print(runes);
}
It gives the output as
printing out as --> <--
[1, 1]
dart pad link: https://dartpad.dev/?id=854e4479bfec03d7e8fd40621c845567
I tried to use hex package and it gives Non-hex character detected error.
Questions.
How do I print these types of strings to the console?
If some conversion is needed, how do I know data belongs to these type ?
my socket client is like the following
socket.listen(
// handle data from the server
(Uint8List data) async {
var serverResponse = String.fromCharCodes(data);
print('Server: $serverResponse');
final runes = serverResponse.runes.toList();
print(runes);
},
EDIT
The socket server is the x0vnc server, on reading the input with wire shark I can see the server sent 01 01
To display a hexa, you have to escape the characters like this:
var out = '\\x01\\x01';
this will work.
I suspect you have misunderstood what the server is sending.
Given you've not stated the server language I'm going to guess that ` ab = b'\x01\x01' generates a array with two bytes both with the value 1.
If you treat this as an ASCII value then 1 is a non printable character.
As such you need to iterate over the array and convert each byte into a suitable visual format.
This might mean that when you see a 1 you print x01.
Edit:
actually dart will convert an int to a string for you:
void main() {
final bytes = <int>[1, 2, 3];
for (final byte in bytes) {
print(byte.toString());
}
}
I've been trying to code a UTF-16 string structure, and although the standard library provides a unicode module, it doesn't seem to provide a way to print out a slice of u16.
I've tried this:
const std = #import("std");
const unicode = std.unicode;
const stdout = std.io.getStdOut().outStream();
pub fn main() !void {
const unicode_str = unicode.utf8ToUtf16LeStringLiteral("😎 hello! 😎");
try stdout.print("{}\n", .{unicode_str});
}
This outputs:
[12:0]u16#202e9c
Is there a way to print a unicode string ([]u16) without converting it back into a non-unicode string ([]u8)?
Both []const u8 and []const u16 store encoded unicode codepoints. Unicode codepoints fit within the range 0..1,114,112 so an actual unicode string with one array index per codepoint would have to be []const u21. utf-8 and utf-16 both require encoding for codepoints that don't fit. Unless there is a compatability reason for utf-16 (like some windows functions), you should probably be using []const u8 unicode strings.
To print utf-16 to a utf-8 stream, you have to decode utf-16 and re-encode it into utf-8. There is currently no formatting specifier to do this automatically.
You can either convert the entire string at once, requiring allocation:
const utf8string = try std.unicode.utf16leToUtf8Alloc(alloc, utf16le);
Or, without allocation:
var writer = std.io.getStdOut().writer();
var it = std.unicode.Utf16LeIterator.init(utf16le);
while (try it.nextCodepoint()) |codepoint| {
var buf: [4]u8 = [_]u8{undefined} ** 4;
const len = try std.unicode.utf8Encode(codepoint, &buf);
try writer.writeAll(buf[0..len]);
}
Note that this will be very slow without using a buffered writer if you are writing somewhere that requires a syscall to write.
Here is my base64 encoded String :
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzZXNzaW9uSWQiOiJkMjNiN2ViMy03MDgyLTRkZDktOGQ0OC1lMjU2YTM3OTNiOTciLCJyZWZyZXNoVG9rZW4iOiJiN2M3MTc4Yi04OWRjLTQxMDctYjUzNC1hOGZiOTNhMzEwNzAiLCJuYW1lIjoiTGVuIiwiaWF0IjoxNTczMDI4MjU2fQ
Using https://jwt.io/ it decodes correctly
But When trying to use base64.decode('--Base64String--); in Flutter it gives me these errors
FormatException: Invalid character (at character 37)
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzZXNzaW9uSWQiOiIzYWNiNzBjZS0wYzYxLT...
When removing the string in front of the . (I only need the info that comes after the .)
I get this error
FormatException: Invalid length, must be multiple of four (at character 183)
...jLTQxMDctYjUzNC1hOGZiOTNhMzEwNzAiLCJuYW1lIjoiTGVuIiwiaWF0IjoxNTczMDI4MjU2fQ
Are there any other ways of decoding base64 encoded Strings for Dart
You can use base64.normalize first.
For example:
import 'dart:convert';
void main() {
final String b64 = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzZXNzaW9uSWQiOiJkMjNiN2ViMy03MDgyLTRkZDktOGQ0OC1lMjU2YTM3OTNiOTciLCJyZWZyZXNoVG9rZW4iOiJiN2M3MTc4Yi04OWRjLTQxMDctYjUzNC1hOGZiOTNhMzEwNzAiLCJuYW1lIjoiTGVuIiwiaWF0IjoxNTczMDI4MjU2fQ';
String foo = b64.split('.')[0];
List<int> res = base64.decode(base64.normalize(foo));
print(utf8.decode(res));
}
Result:
{"alg":"HS256","typ":"JWT"}
try it. This will help you
Center(child:
Image.memory(
base64Decode(image6464.substring(23).replaceAll("\n", ""))
)
),
To read a file contents of .txt file I am using
List<String> linesList = await file.readAsLines(encoding: latin1);
return linesList;
Files with Encodng UTF-8 are working perfectly with this above code.
But for Encoding UTF-16LE its returning a list with length double of the lines in the file but are all empty except first line. This first index contains ÿþ#
As package:utf is now abandoned (and therefore will never support null-safety), another way to read a UTF-16LE file as a String is to take advantage of Dart Strings using UTF-16 code units internally. You therefore can read the file, interpret the data as 16-bit (unsigned) integers, and then create a String using those as UTF-16 code units:
Basically:
var f = File("utf-16le.txt");
var bytes = f.readAsBytesSync();
// Note that this assumes that the system's native endianness is the same as
// the file's.
var utf16CodeUnits = bytes.buffer.asUint16List();
var s = String.fromCharCodes(utf16CodeUnits);
I also leave it as an exercise for the reader to deal with potential BOMs at the beginning of the file.
Also see https://github.com/dart-lang/convert/issues/30, which requests that the Dart SDK provide UTF-16 conversion functions.
So the first credit goes to #Richard_Heap , who has commented on the above question. He mentioned a dart package that encodes and decodes UTF formats. I have been able to decode the txt files as expected in my Flutter app using this package.
First I am identifying the utf format type by these functions available from the package mentioned by #Richard_Heap
List<int> bytes = await file.readAsBytes();
hasUtf16beBom(bytes)
hasUtf16Bom(bytes)}
hasUtf16leBom(bytes)
hasUtf32beBom(bytes)
hasUtf32Bom(bytes)
hasUtf32leBom(bytes)
There are different decoder & encoder functions in this package that can be used once the utf format is known using these above functions. Like I used
String decodedString = decodeUtf16le(bytes);
Check out the charset package. It is null-safe and supports both UTF-16BE and UTF-16LE according to documentation.
Usage:
import 'package:charset/charset.dart';
main() {
// default
print(utf16.decode([254, 255, 78, 10, 85, 132, 130, 229, 108, 52]));
print(utf16.encode("上善若水"));
// detect
print(hasUtf16Bom([0xFE, 0xFF, 0x6C, 0x34]));
// advance
Utf16Encoder encoder = utf16.encoder as Utf16Encoder;
print(encoder.encodeUtf16Be("上善若水", false));
print(encoder.encodeUtf16Le("上善若水", true));
}
I'm trying to draw QR barcodes in a PDF file using iTextSharp. If I'm using English text the barcodes are fine, they are decoded properly, but if I'm using Chinese text, the barcode is decoded as question marks. For example this character '测' (\u6D4B) is decoded as '?'. I tried all supported character sets, but none of them helped.
What combination of parameters should I use for the QR barcode in iTextSharp in order to encode correctly Chinese text?
iText and iTextSharp apparently don't natively support this but you can write some code to handle this on your own. The trick is to get the QR code parser to work with just an arbitrary byte array instead of a string. What's really nice is that the iTextSharp code is almost ready for this but doesn't expose the functionality. Unfortunately many of the required classes are sealed so you can't just subclass them, you'll have to recreate them. You can either download the entire source and add these changes or just create separate classes with the same names. (Please check over the license to make sure you are allowed to do this.) My changes below don't have any error correction so make sure you do that, too.
The first class that you'll need to recreate is iTextSharp.text.pdf.qrcode.BlockPair and the only change you'll need to make is to make the constructor public instead of internal. (You only need to do this if you are creating your own code and not modifying the existing code.)
The second class is iTextSharp.text.pdf.qrcode.Encoder. This is where we'll make the most changes. Add an overload to Append8BitBytes that looks like this:
static void Append8BitBytes(byte[] bytes, BitVector bits) {
for (int i = 0; i < bytes.Length; ++i) {
bits.AppendBits(bytes[i], 8);
}
}
The string version of this method converts text to a byte array and then uses the above so we're just cutting out the middle man. Next, add a new overload to the constructor that takes in a byte array instead of a string. We'll then just cut out the string detection part and force the system to byte-mode, otherwise the code below is pretty much the same.
public static void Encode(byte[] bytes, ErrorCorrectionLevel ecLevel, IDictionary<EncodeHintType, Object> hints, QRCode qrCode) {
String encoding = DEFAULT_BYTE_MODE_ENCODING;
// Step 1: Choose the mode (encoding).
Mode mode = Mode.BYTE;
// Step 2: Append "bytes" into "dataBits" in appropriate encoding.
BitVector dataBits = new BitVector();
Append8BitBytes(bytes, dataBits);
// Step 3: Initialize QR code that can contain "dataBits".
int numInputBytes = dataBits.SizeInBytes();
InitQRCode(numInputBytes, ecLevel, mode, qrCode);
// Step 4: Build another bit vector that contains header and data.
BitVector headerAndDataBits = new BitVector();
// Step 4.5: Append ECI message if applicable
if (mode == Mode.BYTE && !DEFAULT_BYTE_MODE_ENCODING.Equals(encoding)) {
CharacterSetECI eci = CharacterSetECI.GetCharacterSetECIByName(encoding);
if (eci != null) {
AppendECI(eci, headerAndDataBits);
}
}
AppendModeInfo(mode, headerAndDataBits);
int numLetters = dataBits.SizeInBytes();
AppendLengthInfo(numLetters, qrCode.GetVersion(), mode, headerAndDataBits);
headerAndDataBits.AppendBitVector(dataBits);
// Step 5: Terminate the bits properly.
TerminateBits(qrCode.GetNumDataBytes(), headerAndDataBits);
// Step 6: Interleave data bits with error correction code.
BitVector finalBits = new BitVector();
InterleaveWithECBytes(headerAndDataBits, qrCode.GetNumTotalBytes(), qrCode.GetNumDataBytes(),
qrCode.GetNumRSBlocks(), finalBits);
// Step 7: Choose the mask pattern and set to "qrCode".
ByteMatrix matrix = new ByteMatrix(qrCode.GetMatrixWidth(), qrCode.GetMatrixWidth());
qrCode.SetMaskPattern(ChooseMaskPattern(finalBits, qrCode.GetECLevel(), qrCode.GetVersion(),
matrix));
// Step 8. Build the matrix and set it to "qrCode".
MatrixUtil.BuildMatrix(finalBits, qrCode.GetECLevel(), qrCode.GetVersion(),
qrCode.GetMaskPattern(), matrix);
qrCode.SetMatrix(matrix);
// Step 9. Make sure we have a valid QR Code.
if (!qrCode.IsValid()) {
throw new WriterException("Invalid QR code: " + qrCode.ToString());
}
}
The third class is iTextSharp.text.pdf.qrcode.QRCodeWriter and once again we just need to add an overloaded Encode method supports a byte array and that calls are new constructor created above:
public ByteMatrix Encode(byte[] bytes, int width, int height, IDictionary<EncodeHintType, Object> hints) {
ErrorCorrectionLevel errorCorrectionLevel = ErrorCorrectionLevel.L;
if (hints != null && hints.ContainsKey(EncodeHintType.ERROR_CORRECTION))
errorCorrectionLevel = (ErrorCorrectionLevel)hints[EncodeHintType.ERROR_CORRECTION];
QRCode code = new QRCode();
Encoder.Encode(bytes, errorCorrectionLevel, hints, code);
return RenderResult(code, width, height);
}
The last class is iTextSharp.text.pdf.BarcodeQRCode which we once again add our new constructor overload:
public BarcodeQRCode(byte[] bytes, int width, int height, IDictionary<EncodeHintType, Object> hints) {
newCode.QRCodeWriter qc = new newCode.QRCodeWriter();
bm = qc.Encode(bytes, width, height, hints);
}
The last trick is to make sure when calling this that you include the byte order mark (BOM) so that decoders know to decode this properly, in this case UTF-8.
//Create an encoder that supports outputting a BOM
System.Text.Encoding enc = new System.Text.UTF8Encoding(true, true);
//Get the BOM
byte[] bom = enc.GetPreamble();
//Get the raw bytes for the string
byte[] bytes = enc.GetBytes("测");
//Combine the byte arrays
byte[] final = new byte[bom.Length + bytes.Length];
System.Buffer.BlockCopy(bom, 0, final, 0, bom.Length);
System.Buffer.BlockCopy(bytes, 0, final, bom.Length, bytes.Length);
//Create are barcode using our new constructor
var q = new BarcodeQRCode(final, 100, 100, null);
//Add it to the document
doc.Add(q.GetImage());
Looks like you may be out of luck. I tried too and got the same results as you did. Then looked at the Java API:
"*CHARACTER_SET the values are strings and can be Cp437, Shift_JIS and
ISO-8859-1 to ISO-8859-16. The default value is ISO-8859-1.*"
Lastly, looked at the iTextSharp BarcodeQRCode class source code to confirm that only those characters sets are supported. I'm by no means an authority on Unicode or encoding, but according to ISO/IEC 8859, the character sets above won't work for Chinese.
Essentially the same trick that Chris has done in his answer could be implemented by specifying UTF-8 charset in barcode hints.
var hints = new Dictionary<EncodeHintType, Object>() {{EncodeHintType.CHARACTER_SET, "UTF-8"}};
var q = new BarcodeQRCode("\u6D4B", 100, 100, hints);
If you want to be more safe, you can start your string with BOM character '\uFEFF', like Chris suggested, so it would be "\uFEFF\u6D4B".
UTF-8 is unfortunately not supported by QR codes specification, and there are a lot of discussions on this subject, but the fact is that most QR code readers will correctly read the code created by this method.