TCP socket - sending control characters - sockets

I am trying to connect by a TCP/IP protocol to a program that is not made in Delphi, and I have no documentation on how to do it.
Using a service made in Delphi, I have monitored communication between two clients of the program, and write sendt text to a file, so I can play it back in my Delphi program.
I managed to send several things, but the problem is when I have to give this text:
SOH NUL ÜÜ NUL ENQ NUL Data ENQ NUL NUL NUL SOH NUL NUL NUL NUL 557
I see that there are special characters in string parts, and am having trouble sending these data.
I have tried to send it as a string and put the ASCII values but special characters are not working.
Here is part of my code:
TCPClient := TIdTCPClient.Create(nil);
TCPClient.Host := edtIP.Text;
Memo1.Lines.Clear;
TCPClient.Port := PORT;
TCPClient.ConnectTimeout := 20000;
TCPCliente.Connect;
Memo1.Lines.Add('Conectado con el Sevidor');
TCPClient.Socket.ReadTimeout := 20000;
edtTextoEnvio.Text := edtVersion.Text+' '+edtIP.Text+' '+
edtAlias.Text+' '+edtFuncion.Text+' 'edtPassword.Text;
TCPClient.Socket.WriteLnRFC(edtTextoEnvio.Text, IndyTextEncoding_8Bit);
Memo1.Lines.Add('Mensaje Enviado: '+edtTextoEnvio.Text);
TextoRecibido := TCPCliente.Socket.ReadLn(IndyTextEncoding_8Bit);
Memo1.Lines.Add('Mensaje Recibido: '+TextoRecibido);
TCPCliente.Socket.WriteLnRFC(NUNProc, IndyTextEncoding_8Bit);
Memo1.Lines.Add('Mensaje Enviado: '+NUNProc);
envioMensaje:= Char(1)+ Char(0)+'ÜÜ'+ Char(0)+ Char(5)+Char(0)+'Data'+Char()5)+Char(0)+Char(0)+Char(0)+Char(1)+CHar(0)+Char(0)+Char(0)+Char(0)+'557';
TCPCliente.Socket.WriteLnRFC(envioMensaje, IndyTextEncoding_8Bit); //here is the problem
TextoRecibido := TCPCliente.Socket.ReadLn(IndyTextEncoding_8Bit);
Memo1.Lines.Add('Mensaje Recibido: '+TextoRecibido);


Some of the string values that you want to send have non-ASCII characters in them, but sockets have no concept of text, only bytes. You have to encode the strings into bytes using an appropriate charset, then you can send the bytes as needed. Indy has functionality built-in to handle that for you (such as the TIdIOHandler.DefStringEncoding property, and the AByteEncoding parameter of string-based TIdIOHandler.Write...() and TIdIOHandler.Read...() methods). You are trying to take that into account, but you are using the wrong Indy encoding to do it. IndyTextEncoding_8bit is not really a charset, it is primarily intended as a workaround for dealing with 8bit binary data in text-based protocols, and should not be used in general code. What you need to do instead is identify the actual charset that the other party is expecting you to use. Different charsets encode the same non-ASCII characters in different ways (if at all), so you and the other party must agree on the same charset to encode and decode the strings, otherwise you are going to lose data.
That being said, there are different ways in Indy to accomplish what you are attempting. Given the data you have shown (it would have been better if you had shown the data actually captured by your monitoring service, or better from a packet sniffer like Wireshark), at a minimum I would suggest something more like the following:
TCPClient := TIdTCPClient.Create(nil);
TCPClient.Host := edtIP.Text;
TCPClient.Port := PORT;
TCPClient.ConnectTimeout := 20000;
TCPClient.ReadTimeout := 20000;
Memo1.Lines.Clear;
TCPClient.Connect;
TCPClient.IOHandler.DefStringEncoding := CharsetToEncoding('ISO-8859-1'); // <-- this is just a guess!
Memo1.Lines.Add('Conectado con el Sevidor');
edtTextoEnvio.Text := edtVersion.Text + ' ' + edtIP.Text + ' ' + edtAlias.Text + ' ' + edtFuncion.Text + ' ' + edtPassword.Text;
TCPClient.IOHandler.WriteLn(edtTextoEnvio.Text);
Memo1.Lines.Add('Mensaje Enviado: ' + edtTextoEnvio.Text);
TextoRecibido := TCPClient.IOHandler.ReadLn;
Memo1.Lines.Add('Mensaje Recibido: ' + TextoRecibido);
TCPClient.IOHandler.WriteLn(NUNProc);
Memo1.Lines.Add('Mensaje Enviado: ' + NUNProc);
TCPClient.IOHandler.WriteBufferOpen;
try
TCPClient.IOHandler.Write(Byte(1));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write('ÜÜ');
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write(Byte(5));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write('Data');
TCPClient.IOHandler.Write(Byte(5));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write(Byte(1));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write('557');
TCPClient.IOHandler.WriteLn;
TCPClient.IOHandler.WriteBufferClose;
except
TCPClient.IOHandler.WriteBufferCancel;
raise;
end;
TextoRecibido := TCPClient.IOHandler.ReadLn;
Memo1.Lines.Add('Mensaje Recibido: ' + TextoRecibido);
That being said, all of those nul bytes are more likely to be part of multi-byte binary integers, which you can send using TIdIOHandler.Write(Int16), TIdIOHandler.Write(Int32), etc:
TCPClient.IOHandler.WriteBufferOpen;
try
{
TCPClient.IOHandler.Write(Byte(1));
TCPClient.IOHandler.Write(Byte(0));
}
TCPClient.IOHandler.Write(Int16(1), False);
TCPClient.IOHandler.Write('ÜÜ');
TCPClient.IOHandler.Write(Byte(0));
{
TCPClient.IOHandler.Write(Byte(5));
TCPClient.IOHandler.Write(Byte(0));
}
TCPClient.IOHandler.Write(Int16(5), False);
TCPClient.IOHandler.Write('Data');
{
TCPClient.IOHandler.Write(Byte(5));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write(Byte(0));
}
TCPClient.IOHandler.Write(Int32(5), False);
{
TCPClient.IOHandler.Write(Byte(1));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write(Byte(0));
}
TCPClient.IOHandler.Write(Int32(1), False);
TCPClient.IOHandler.Write(Byte(0));
TCPClient.IOHandler.Write('557');
TCPClient.IOHandler.WriteLn;
TCPClient.IOHandler.WriteBufferClose;
except
TCPClient.IOHandler.WriteBufferCancel;
raise;
end;
And the ÜÜ is also very suspicious. It is most likely NOT actually a ÜÜ string, but more likely just bytes $DC $DC, 2 8-bit 220 values, a 16-bit 56540 value, etc.
Without clear documentation about the actual communication protocol, you are just making guesses, and you are likely to guess wrong at times. So I strongly suggest you contact the author of the other client app and ask for documentation. Or search online and see if a 3rd party has already written such documentation for that app. Otherwise you are just coding blind, and that is not the good option.

Related

Convert Hexadecimal to Base64 in Authotkey

I just found this code written in python to convert Hexadecimal to Base64:
import codecs
hex = "10000000000002ae"
b64 = codecs.encode(codecs.decode(hex, 'hex'), 'base64').decode()
So, Is it possible to find same working one but in Autohotkey?

There many implementations, some very fast and some more simple. You can take a look at libcrypt.ahk for many encoding and encryption algorithms including Hexadecimal and Base64.
If you do use it, you'll want to look at LC_Hex2Bin() and LC_Base64_Encode(). Examples are available too. You will likely want this libcrypt.ahk file in particular.
Example
#Include libcrypt.ahk
Hex := "48656c6c6f20576f726c642100"
len := LC_Hex2Bin(Bin, Hex)
LC_Base64_Encode(base64, Bin, len)
MsgBox % base64
Or as single function
#Include libcrypt.ahk
MsgBox % hexstring2base64("48656c6c6f20576f726c642100")
hexstring2base64(hex_string) {
len := LC_Hex2Bin(Bin, hex_string)
LC_Base64_Encode(base64, Bin, len)
return base64
}

Syntax for Returning One Character of String by Index

I am attempting to compare one character of a string to see if it is my delimiter character. However, when I execute the following code the value that gets placed in the variable valstring is a number that represents the byte that was converted to a string and not a character itself. For Example the value may be the string '58'.
Through my testing in CoDeSys using the debugging features I know that the string sReadLine contains a valid string of characters. I'm just not sure of the syntax to single only one of them out; the sReadLine[valPos + i] part is what I don't understand.
sReadLine : STRING;
valstring : STRING;
i : INT;
valPos : INT;
FOR i := 0 TO 20 DO
IF BYTE_TO_STRING(sReadLine[valPos + i]) = '"' THEN
EXIT;
END_IF
valstring := CONCAT(STR1 := valstring, STR2 := BYTE_TO_STRING(sReadLine[valPos + i]));
END_FOR

I think you have multiple choises.
1) Use built-in string functions instead. You can use MID function get get part of a string. So in your case something like "get one character from valPos + 1 from sReadLine.
FOR i := 0 TO 20 DO
IF MID(sReadLine, 1, valPos + i) = '"' THEN
EXIT;
END_IF
valstring := CONCAT(STR1 := valstring, STR2 := MID(sReadLine, 1, valPos + i));
END_FOR
2) Convert the ASCII byte to string. In TwinCAT systems, there is a function F_ToCHR. It takes a ASCII byte in and returns the character as string. I can't find something like that for Codesys, but i'm sure there would be a solution in some library. So please note that this won't work in Codesys without modifications:
FOR i := 0 TO 20 DO
IF F_ToCHR(sReadLine[valPos + i]) = '"' THEN
EXIT;
END_IF
valstring := CONCAT(STR1 := valstring, STR2 := F_ToCHR(sReadLine[valPos + i]));
END_FOR
3) The OSCAT library seems to have a CHR_TO_STRING function. You could use this instead of F_ToCHR in step 2.
4) You can use pointers to copy the ASCII byte to a string array (MemCpy) and add a string end character. This needs some knowledge of pointers etc. See Codesys forum for some example.
5) You can write a helper function similar to step 2 youself. Check the example from Codesys forums. That example doesn't include all characters so it needs to be updated. It's not quite elegant.

When you convert a byte to a string, what is beeing converted is the digital representation of the byte.
This means you are interpreting that byte as an ascii character (The ascii decimal value of : is 58).
So if you want to Concat chars instead of their ascii decimal representation, you need another function:
valstring := CONCAT(STR1 := valstring, STR2 := F_ToCHR(sReadLine[valPos + i]));
EDIT:
As Quirzo, I couldn't find a similar F_ToCHR function for Codesys, but you could easily build one yourself.
For example:
Declaration Part:
FUNCTION F_ASCII_TO_STRING : STRING
VAR_INPUT
input : BYTE;
END_VAR
VAR
ascii : ARRAY[0..255] OF STRING(1):=
[
33(' '),'!','"','#',
'$$' ,'%' ,'&' ,'´',
'(' ,')' ,'*' ,'+' ,
',' ,'-' ,'.' ,'/' ,
'0' ,'1' ,'2' ,'3' ,
'4' ,'5' ,'6' ,'7' ,
'8' ,'9' ,':' ,';' ,
'<' ,'=' ,'>' ,'?' ,
'#' ,'A' ,'B' ,'C' ,
'D' ,'E' ,'F' ,'G' ,
'H' ,'I' ,'J' ,'K' ,
'L' ,'M' ,'N' ,'O' ,
'P' ,'Q' ,'R' ,'S' ,
'T' ,'U' ,'V' ,'W' ,
'X' ,'Y' ,'Z' ,'[' ,
'\' ,']' ,'^' ,'_' ,
'`' ,'a' ,'b' ,'c' ,
'd' ,'e' ,'f' ,'g' ,
'h' ,'i' ,'j' ,'k' ,
'l' ,'m' ,'n' ,'o' ,
'p' ,'q' ,'r' ,'s' ,
't' ,'u' ,'v' ,'w' ,
'x' ,'y' ,'z' ,'{' ,
'|' ,'}' ,'~'
];
END_VAR
Implementation part:
F_ASCII_TO_STRING := ascii[input];

As Sergey said, this might not be an optimal solution to your problem. It seems like you want to extract the longest substring not containing any character " from initial input sReadLine to valstring, starting from position valPos.
In your implementation, for each valid input character, CONCAT() needs to search for the end of valstring, before appending only 1 character to it.
You should rather decompose your problem and use two standard functions to be optimal:
FIND() --> to get the position of the next character " (or to know if there is none),
MID() --> to create a string from initial position up to before the first character " (or the end of the input string).
That way, there remains only 2 loops; each one is hidden in these functions.

Inno Setup: recording/recover file path in UTF8

We are using Inno Setup (unicode version) to create resource package (or "samples") for our product. The program part of our product knows the location of the samples by a file that is written by samples installer. At current, it is implemented in plain way:
procedure CurStepChanged(CurStep: TSetupStep);
begin
if ( CurStep = ssPostInstall) then
begin
ForceDirectories(ExpandConstant('{userappdata}\MyCompany\MyApp'))
SaveStringToFile(ExpandConstant('{userappdata}\MyCompany\MyApp\SamplePath.txt'), ExpandConstant('{app}'), False);
end;
end;
This plain way has a fatal issue: the installer is run in Chinese language Windows, and the whole stuff works in GBK encoding, but our product is built up on UTF8 base.
After some search, I got some solution by calling Windows WideCharToMultiByte inside Pascal code. However this won't work, as it requires UTF16 as input, but what I have is GBK.
In addition, the Inno Setup also won't work with existing UTF8 file name in my SamplePath.txt. If I manually edit the SamplePath.txt file to fill UTF8-encoded Chinese letters, and initialize the app builtin with following code, it displays messy characters in dir selection page:
[Setup]
DefaultDirName={code:GetPreviousSampleDir}
[code]
function GetPreviousSampleDir(Param: String): String;
var
tmp: AnsiString;
begin
if FileExists( ExpandConstant('{userappdata}\MyCompany\MyApp\SamplePath.txt') ) then
begin
LoadStringFromFile(ExpandConstant('{userappdata}\MyCompany\MyApp\SamplePath.txt'), tmp)
Result := tmp
end
else
begin
Result := 'D:\MyApp_samples'
end;
end;
So is there any way to load/store a file name with i18n characters in UTF8?

To load a string from UTF-8 file, use LoadStringFromFileInCP from
Inno Setup - Convert array of string to Unicode and back to ANSI
const
CP_UTF8 = 65001;
{ ... }
var
FileName: string;
S: string;
begin
FileName := 'test.txt';
if not LoadStringFromFileInCP(FileName, S, CP_UTF8) then
begin
Log('Error reading the file');
end
else
begin
Log('Read: ' + S);
end;
end;
To save UTF-8 file without BOM:
either use SaveStringsToFileInCP from the same question
or see Create a UTF8 file without BOM with Inno Setup (Unicode version)

Matching Unicode punctuation using LPeg

I am trying to create an LPeg pattern that would match any Unicode punctuation inside UTF-8 encoded input. I came up with the following marriage of Selene Unicode and LPeg:
local unicode = require("unicode")
local lpeg = require("lpeg")
local punctuation = lpeg.Cmt(lpeg.Cs(any * any^-3), function(s,i,a)
local match = unicode.utf8.match(a, "^%p")
if match == nil
return false
else
return i+#match
end
end)
This appears to work, but it will miss punctuation characters that are a combination of several Unicode codepoints (if such characters exist), as I am reading only 4 bytes ahead, it probably kills the performance of the parser, and it is undefined what the library match function will do, when I feed it a string that contains a runt UTF-8 character (although it appears to work now).
I would like to know whether this is a correct approach or if there is a better way to achieve what I am trying to achieve.

The correct way to match UTF-8 characters is shown in an example in the LPeg homepage. The first byte of a UTF-8 character determines how many more bytes are a part of it:
local cont = lpeg.R("\128\191") -- continuation byte
local utf8 = lpeg.R("\0\127")
+ lpeg.R("\194\223") * cont
+ lpeg.R("\224\239") * cont * cont
+ lpeg.R("\240\244") * cont * cont * cont
Building on this utf8 pattern we can use lpeg.Cmt and the Selene Unicode match function kind of like you proposed:
local punctuation = lpeg.Cmt(lpeg.C(utf8), function (s, i, c)
if unicode.utf8.match(c, "%p") then
return i
end
end)
Note that we return i, this is in accordance with what Cmt expects:
The given function gets as arguments the entire subject, the current position (after the match of patt), plus any capture values produced by patt. The first value returned by function defines how the match happens. If the call returns a number, the match succeeds and the returned number becomes the new current position.
This means we should return the same number the function receives, that is the position immediately after the UTF-8 character.

I need to concatenate several strings

I am new to stuctured text and I would like to know how to concatenate several string. The cleanest way possible. I this instance I only need to change one variable when creating the string. I have another where I need to do 2. That number will probably grow. The purpose of this is so I can send XML message to an HTTP server. This is for logging data.
In this instance it is the reader variable which is a word.
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Header>
<Action s:mustUnderstand="1" xmlns="http://schemas.microsoft.com/ws/2005/05/addressing/none">http://tempuri.org/IService/receiveHeartbeat</Action>
</s:Header>
<s:Body>
<receiveHeartbeat xmlns="http://tempuri.org/">
<reader>**Word Variable**</reader>
</receiveHeartbeat>
</s:Body>
</s:Envelope>

You can chain CONCAT functions like so:
concat3: STRING := CONCAT(CONCAT(str1, str2), str3);
However, beware that by default STRING is only 80 characters (bytes) long. You can specify the size using parenthesis:
concat3: STRING(255) := CONCAT(CONCAT(str1, str2), str3);
But again, the standard CONCAT function only accepts and returns strings of up to 255 in length!
If you need strings longer than 255, then check Working with Strings More Than 255 Characters in the codesys documentation

If you are using Wago then you should have access to their CONCAT functions...CONCAT3(),CONCAT4()...CONCAT9(). This is much cleaner than nesting a lot of the standard CONCAT funct

I take it that you need to do this in JavaScript.
var replaceDue = "<s:Envelope xmlns:s=\"http://schemas.xmlsoap.org/soap/envelope/\">\
<s:Header>\
<Action s:mustUnderstand=\"1\"xmlns=\"http://schemas.microsoft.com/ws/2005/05/addressing/none\">http://tempuri.org/IService/receiveHeartbeat</Action>\
</s:Header>\
<s:Body>\
<receiveHeartbeat xmlns=\"http://tempuri.org/\">\
<reader>**Word Variable**</reader>\
</receiveHeartbeat>\
</s:Body>\
</s:Envelope>";
var wordVariable = "value to replace";
var replaceDone = replaceDue.replace("**Word Variable**", wordVariable);

I think I have found a solution. I don't like it though. It's not very clean.
Reader_ID: STRING := '0';
msg: STRING(500);
Msg1: STRING(250) := '<s:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"><s:Header>';
Msg2: STRING(250) := '<Action s:mustUnderstand="1" xmlns="http://schemas.microsoft.com/ws/2005/05/addressing/none">http://tempuri.org/IService/receiveHeartbeat</Action>';
Msg3: STRING := '</s:Header><s:Body><receiveHeartbeat xmlns="http://tempuri.org/"><reader>';
MsgAfter: STRING := '</reader></receiveHeartbeat></s:Body></s:Envelope>';
msg := CONCAT(Msg1,Msg2);
msg := CONCAT(msg,Msg3);
msg := CONCAT(msg,Reader_ID);
msg := CONCAT(msg,MsgAfter);
It seems that string sizes are limited to 500 characters. Since the point of this is to create an XML message to send via HTTP. What happens when my messages inevitably get larger than 500 character. I am using the WagoLibHttp_02 library for http.