Ok, so just a question popped up in my mind while writing some code.
Does the CPU compute this
gap = (gap*3) + 1;
with the same efficiency as the below expression without the brackets?
gap = gap*3 + 1;
Update to be more clarifying:
Often while writing arithmetic expressions, we tend to add brackets to make the code easier to read, even if the brackets do not change what the expression evaluates to. So the question is, does adding such brackets have any affect on the performance?
As mentioned above in the comments:
file test1.c:
int main(void)
{
int gap = 1;
gap = (gap*3) + 1;
return gap;
}
file test2.c:
int main(void)
{
int gap = 1;
gap = gap*3 + 1;
return gap;
}
Using gcc and its -S option ("Compile only; do not assemble or link"): gcc -S test1.c && gcc -S test2.c. Comparing those two files:
$ diff test1.s test2.s
1c1
< .file "test1.c"
---
> .file "test2.c"
(i.e. only the file names differ in the first line)
When you still don't believe me (for this particular example), then you can further compile & assemble: gcc test1.c -o test1 && gcc test2.c -o test2. Comparing those binary files (or rather ELF executables) will give you one byte difference, i.e. the file name again:
$ hexdump -C test1 > hex1.txt && hexdump -C test2 > hex2.txt
$ diff hex1.txt hex2.txt
387c387
< 00001fc0 79 5f 65 6e 74 72 79 00 74 65 73 74 31 2e 63 00 |y_entry.test1.c.|
---
> 00001fc0 79 5f 65 6e 74 72 79 00 74 65 73 74 32 2e 63 00 |y_entry.test2.c.|
Related
I want to get a byte offset of a string pattern from a binary file on embedded linux platform.
If I can use "grep -b" option, It would be best way but It is not supported on my machine.
machine does not support
ADDR=`grep -oba <pattern string> <file path> | cut -d ":" -f1`
Here the manual of grep command on the machine.
root# grep --help
BusyBox v1.29.3 () multi-call binary.
Usage: grep \[-HhnlLoqvsriwFE\] \[-m N\] \[-A/B/C N\] PATTERN/-e PATTERN.../-f FILE \[FILE\]...
Search for PATTERN in FILEs (or stdin)
-H Add 'filename:' prefix
-h Do not add 'filename:' prefix
-n Add 'line_no:' prefix
-l Show only names of files that match
-L Show only names of files that don't match
-c Show only count of matching lines
-o Show only the matching part of line
-q Quiet. Return 0 if PATTERN is found, 1 otherwise
-v Select non-matching lines
-s Suppress open and read errors
-r Recurse
-i Ignore case
-w Match whole words only
-x Match whole lines only
-F PATTERN is a literal (not regexp)
-E PATTERN is an extended regexp
-m N Match up to N times per file
-A N Print N lines of trailing context
-B N Print N lines of leading context
-C N Same as '-A N -B N'
-e PTRN Pattern to match
-f FILE Read pattern from file
Since that option isn't available, I'm looking for an alternative.
the combination of hexdump and grep can be also useful
such as
ADDR=`hexdump <file path> -C | grep <pattern string> | cut -d' ' -f1`
But if pattren spans multiple lines, it will not be found.
Is there a way to find the byte offset of a specific pattern with a Linux command?
Set the pattern as the record separator in awk. The offset of the occurrence is the length of the first record. BusyBox awk treats RS as an extended regular expression, so add backslashes before any of .[]\*+?^$ in the pattern string.
<myfile.bin awk -v RS='pattern' '{print length($0); exit}'
If the pattern contains a null byte, you need a little extra work. Use tr to exchange null bytes with some byte value that doesn't appear in the pattern. For example, if the pattern's hex dump is 00002a61:
<myfile.bin tr '\0!' '!\0' | awk -v RS='!!-A' '{print length($0); exit}'
If the pattern is not found, this prints the length of the whole file. So if you aren't sure whether the pattern is present, you need again some extra work. Append some text that can't be part of a pattern match to the file, so that you know that if there's a match, it won't be at the very end of the file. Then, if the pattern is present, the file will contain at least two records. But if the pattern is not present, the file only contains the first record (without a record separator after it).
{ cat myfile.bin; echo garbage; } |
awk -v RS='pattern' '
NR==1 {n = length($0)}
NR==2 {print n; found = 1; exit}
END {exit !found}
'
Something like this?
hexdump -C "$file" |
awk -v pattern="$pattern" 'residue { matched = ($0 ~ "\\|" residue)
if (matched) print $1; residue = ""; if (matched) next }
$0 ~ pattern { print $1 }
{ for(i=length(pattern)-1; i>0; i--)
if ($0 ~ substr(pattern, 1, i) "\\|$") { residue=substr(pattern, i+1); break } }'
The offset is just the first field from the hexdump output; if you need the precise location of the match, this requires some additional massaging to figure out the offset to add to the address, or subtract if it was wrapped.
Briefly tested in a clean-slate Busybox Docker container where hexdump -C output looks like this:
/ # hexdump -C /etc/resolv.conf
00000000 23 20 44 4e 53 20 72 65 71 75 65 73 74 73 20 61 |# DNS requests a|
00000010 72 65 20 66 6f 72 77 61 72 64 65 64 20 74 6f 20 |re forwarded to |
00000020 74 68 65 20 68 6f 73 74 2e 20 44 48 43 50 20 44 |the host. DHCP D|
00000030 4e 53 20 6f 70 74 69 6f 6e 73 20 61 72 65 20 69 |NS options are i|
00000040 67 6e 6f 72 65 64 2e 0a 6e 61 6d 65 73 65 72 76 |gnored..nameserv|
00000050 65 72 20 31 39 32 2e 31 36 38 2e 36 35 2e 35 0a |er 192.168.65.5.|
00000060 20 | |
For the sake of readability, it would be nice to be able to have multi-line "script" (whatever the syntax of WinDBG is called) code.
For example, I want to search through a chunk of memory looking for things that look somewhat like a wxArrayString...
r $t1 = 0xe53801c
r $t2 = 0x2e800
.for (r $t3 = 0; #$t3 < 0x400; r $t3 = #$t3 + 1) {
.if (##c++(
((int)(((MyModule!wxArrayString*)(#$t1+#$t2+(#$t3*8)))->m_compareFunction) >= 0x10000) &&
((int)(((MyModule!wxArrayString*)(#$t1+#$t2+(#$t3*8)))->m_pItems) >= 0x10000) &&
(((MyModule!wxArrayString*)(#$t1+#$t2+(#$t3*8)))->m_nCount <= 5000) &&
(((MyModule!wxArrayString*)(#$t1+#$t2+(#$t3*8)))->m_nCount > 0) &&
(((MyModule!wxArrayString*)(#$t1+#$t2+(#$t3*8)))->m_nSize <= 10000) &&
(((MyModule!wxArrayString*)(#$t1+#$t2+(#$t3*8)))->m_nSize > 0)
)) {
dx ((MyModule!wxArrayString*)(#$t1+#$t2+(#$t3*8)))
}
}
Smushing that into a single line isn't too hard, but it would be nice if I didn't have to do it every time I wanted to test a change in the code.
Is there a way to do this (with line continuations or some other magic)?
If you have an editor that allows and supports files with different line endings (in one file!), it's feasible:
Insert Macintosh linebreaks (CR; 0x0D) where you want to see line breaks and Unix line breaks (LF; 0x0A) where you want WinDbg to end a statement.
$<, $><, $$<, $$><, $$ >a< (MSDN; not some injection) has some details:
The $$>< and $>< tokens execute the commands that are found in the script file literally, which means they open the script file, replace all carriage returns with semicolons, and execute the resulting text as a single command block.
Actually that statement is wrong. It replaces 0x0A by semicolons and that is LF (line feed) and not carriage return (CR).
Example:
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 2E 65 63 68 6F 20 22 57 6F 77 0D 2E 65 63 68 6F .echo "Wow..echo
00000010 20 74 68 65 72 65 20 69 73 20 61 20 6C 69 6E 65 there is a line
00000020 62 72 65 61 6B 22 0A 72 20 24 74 30 0A 6B 20 4C break".r $t0.k L
00000030 31
which looks like 4 lines but will be executed as 1 line:
2:004> $$>a< b:\newline.windbg
Wow .echo there is a linebreak
$t0=00000000
# ChildEBP RetAddr
00 0116f508 7726c088 ntdll!LdrpDoDebuggerBreak+0x2b
The WinDbg command parser, scripting parser and whatever parsers are implemented ... erm ... in a quite simplistic way, I assume. They didn't think much about escape characters etc. Note that they do not care about the " of .echo and would even replace even replace LF in the middle of a string. There are plenty of examples which are like that.
Maybe you want to look into Python scripting with pykd, implement a WinDbg extension or similar.
are those numbers/address 10000 for m_compareFunction , m_pItems an example and not actual value ??
m_compareFunction is a Function Pointer
0:000> ?? labels[0].m_compareFunction
<function> * 0x00000000`00000000
m_pItems is a wxString *
0:000> ?? labels[0].m_pItems
class wxString * 0x000001c8`2553f598
=00007ff7`cc3b8a60 npos : 0xffffffff`ffffffff
+0x000 m_impl : std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> >
+0x028 m_convertedToChar : wxString::ConvertedBuffer<char>
also adding $t1+$t2 could be eliminated
anyway put your script in a txtfile like myscript.wds and use
$$>a< myscript.wds
to execute it
you can write script as multiline
:\>cat d:\myscript.wds
r? $t0 = labels
?? #$t0
?? #$t0[0]
?? #$t0[1].m_pItems
?? #$t0[3].m_nCount
:\>
executed
0:000> $$>a< d:\myscript.wds
class wxArrayString [8] 0x00000035`350fe650
+0x000 m_compareFunction : (null)
+0x008 m_nSize : 0x20
+0x010 m_nCount : 0x13
+0x018 m_pItems : 0x000001c8`2553f598 wxString
+0x020 m_autoSort : 0
class wxArrayString
+0x000 m_compareFunction : (null)
+0x008 m_nSize : 0x20
+0x010 m_nCount : 0x13
+0x018 m_pItems : 0x000001c8`2553f598 wxString
+0x020 m_autoSort : 0
class wxString * 0x000001c8`26f40fa8
=00007ff7`cc3b8a60 npos : 0xffffffff`ffffffff
+0x000 m_impl : std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t> >
+0x028 m_convertedToChar : wxString::ConvertedBuffer<char>
unsigned int64 0
also if you use r? you can eliminate all the casting
0:000> r #$t0 = 0x00000035`350fe654
0:000> r? $t19 = (widgets!wxArrayString *)#$t0
0:000> ?? #$t19
class wxArrayString * 0x00000035`350fe654
+0x000 m_compareFunction : 0x00000020`00000000 int +2000000000
+0x008 m_nSize : 0x00000013`00000000
+0x010 m_nCount : 0x2553f598`00000000
+0x018 m_pItems : 0x350ff800`000001c8 wxString
+0x020 m_autoSort : 35
I want to dump (and later work with) the paths of the locally changed files in my SVN repository. Problem is, there are umlauts in some filenames (like ä, ö, ü).
When I open a powershell window in my lokal trunk folder, I can do svn status and get the result with correct umlauts ("ü" in this case):
PS C:\trunk> svn status -q
M Std\ClientComponents\Prüfung.xaml
M Std\ClientComponents\Prüfung.xaml.cs
M Std\ClientComponents\PrüfungViewModel.cs
When I do the same in my powershell script, the results are different.
Script "DumpChangedFiles.ps1":
foreach ( $filename in svn status -q )
{
Write-Host $filename
}
Results:
PS C:\trunk> .\DumpChangedFiles.ps1
M Std\ClientComponents\Pr³fung.xaml
M Std\ClientComponents\Pr³fung.xaml.cs
M Std\ClientComponents\Pr³fungViewModel.cs
Question: Why are the umlauts wrong? How do I get to the correct results?
Hex-Dump:
ef bb bf 4d 20 20 20 20 20 20 20 53 74 64 5c 43 6c 69 65 6e 74 43 6f 6d 70 6f 6e 65 6e 74 73 5c 50 72 c2 b3 66 75 6e 67 2e 78 61 6d 6c 0d 0a 4d 20 20 20 20 20 20 20 53 74 64 5c 43 6c 69 65 6e 74 43 6f 6d 70 6f 6e 65 6e 74 73 5c 50 72 c2 b3 66 75 6e 67 2e 78 61 6d 6c 2e 63 73 0d 0a 4d 20 20 20 20 20 20 20 53 74 64 5c 43 6c 69 65 6e 74 43 6f 6d 70 6f 6e 65 6e 74 73 5c 50 72 c2 b3 66 75 6e 67 56 69 65 77 4d 6f 64 65 6c 2e 63 73
Here's the output of the script DumpChangedFiles.ps1 compared to the output of your desired command:
PS C:\trunk> .\DumpChangedFiles.ps1
M Std\ClientComponents\Pr³fung.xaml
M Std\ClientComponents\Pr³fung.xaml.cs
M Std\ClientComponents\Pr³fungViewModel.cs
PS C:\trunk> $PSDefaultParameterValues['*:Encoding'] = 'utf8'; svn status -q
M Std\ClientComponents\Prüfung.xaml
M Std\ClientComponents\Prüfung.xaml.cs
M Std\ClientComponents\PrüfungViewModel.cs
Output of SVN--version is:
PS C:\trunk> svn --version
svn, version 1.14.0 (r1876290)
compiled May 24 2020, 17:07:49 on x86-microsoft-windows
Copyright (C) 2020 The Apache Software Foundation.
This software consists of contributions made by many people;
see the NOTICE file for more information.
Subversion is open source software, see http://subversion.apache.org/
The following repository access (RA) modules are available:
* ra_svn : Module for accessing a repository using the svn network protocol.
- with Cyrus SASL authentication
- handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
- handles 'file' scheme
* ra_serf : Module for accessing a repository via WebDAV protocol using serf.
- using serf 1.3.9 (compiled with 1.3.9)
- handles 'http' scheme
- handles 'https' scheme
The following authentication credential caches are available:
* Wincrypt cache in C:\Users\reichert\AppData\Roaming\Subversion
The problem comes from PowerShell ISE, the svn command in your script is executed through PowerShell ISE which encode its output with Windows-1252 (or your default windows locales).
You can go with the following to get a correct output (check your Windows locales) :
[Console]::OutputEncoding = [System.Text.Encoding]::GetEncoding(1252)
foreach ( $filename in svn status -q )
{
Write-Host $filename
}
It seems a previous unanswered question relates to the same problem with ISE :
Powershell ISE has different codepage from Powershell and I can not change it
I want to look inside my queues, the msm console snapin has this property dialog, but it is very difficult to read and the messages which are important to me are encoded and look like this:
3C 3F 78 6D 6C 20 76 65 <?xml ve
72 73 69 6F 6E 3D 22 31 rsion="1
2E 30 22 20 65 6E 63 6F .0" enco
64 69 6E 67 3D 22 75 74 ding="ut
66 2D 38 22 3F 3E 0D 0A f-8"?>..
3C 65 73 62 3A 6D 65 73 <esb:mes
73 61 67 65 73 20 78 6D sages xm
6C 6E 73 3A 65 73 62 3D lns:esb=
22 68 74 74 70 3A 2F 2F "http://
73 65 72 76 69 63 65 62 serviceb
75 73 2E 68 69 62 65 72 us.hiber
6E 61 74 69 6E 67 72 68 natingrh
...
Anyone knows of a tool that would allow me to see my messages in a bit developer friendly way? A tool for easier administering queues would come handy to (like selecting multiple messages and drag and drop them)
This is about the best tool I've found: http://www.cogin.com/msmq/QueueExplorer/QueueExplorer2.2.php
I found these two methods while searching for an answer to this question and they actually worked perfectly.
public System.Xml.XmlDocument ConvertToXMLDoc(System.Messaging.Message msg)
{
byte[] buffer = new byte[msg.BodyStream.Length];
msg.BodyStream.Read(buffer, 0, (int)msg.BodyStream.Length);
int envelopeStart = FindEnvolopeStart(buffer);
System.IO.MemoryStream stream = new System.IO.MemoryStream(buffer, envelopeStart, buffer.Length - envelopeStart);
System.ServiceModel.Channels.BinaryMessageEncodingBindingElement elm = new System.ServiceModel.Channels.BinaryMessageEncodingBindingElement();
System.ServiceModel.Channels.Message msg1 = elm.CreateMessageEncoderFactory().Encoder.ReadMessage(stream, Int32.MaxValue);
System.Xml.XmlDocument doc = new System.Xml.XmlDocument();
doc.Load(msg1.GetReaderAtBodyContents());
msg.BodyStream.Position = 0;
return doc;
}
private int FindEnvolopeStart(byte[] stream)
{
int i = 0;
byte prevByte = stream[i];
byte curByte = (byte)0;
for (i = 0; i < stream.Length; i++)
{
curByte = stream[i];
if (curByte == (byte)0x02 &&
prevByte == (byte)0x56)
break;
prevByte = curByte;
}
return i - 1;
}
Simply call the ConvertToXmlDoc function, providing the message from the message queue and you'll get an XmlDocument back. I am lazy, so I just drop the innerXml into a file so I can read it.
MessageQueue queue = new MessageQueue(queueName);
var msg = queue.Receive();
var doc = ConvertToXMLDoc(msg);
using (var sw = new StreamWriter(#"C:\message.txt")))
sw.Write(doc.InnerXml);
No application to buy and you get your data back in code so you can mess around with it.
PS: Credit where credit is due. The snippet came from http://social.msdn.microsoft.com/forums/en-US/wcf/thread/c03d80cd-492c-4ece-8890-6a35b12352e0 , which also links to a more detailed discussion of MSMQ's encoding format.
Try this:
string QueueName = #".\private$\publishingQueue";
//note, you cannot use method exists on remote queues
if (MessageQueue.Exists(QueueName))
{
var queue = new MessageQueue(queueInfo.QueueName)
{
MessageReadPropertyFilter = new MessagePropertyFilter
{
ArrivedTime = true,
Body = true
}
};
var messages = queue.GetAllMessages();
var m = messages[0];
m.Formatter = new System.Messaging.XmlMessageFormatter(new String[] {});
StreamReader sr = new StreamReader(m.BodyStream);
string ms = "";
string line;
while (sr.Peek() >= 0)
{
ms += sr.ReadLine();
}
//ms now contains the message
}
If you just have some hexadecimal data that can easily be converted to ASCII and back, then I suggest a text editor that lets you do just that. UltraEdit has a "view hex" function that works to convert both to and from hexadecimal view. You might also try Notepad++ but I don't know if it has that feature.
You can use Service Bus MQ Manager, its a free open-source tool I wrote for viewing messages in MSMQ, it supports coloring and formatting of XML and JSON messages.
http://blog.halan.se/page/Service-Bus-MQ-Manager.aspx
You could also check out MSMQ Studio from https://msmq-studio.com
I have a hex string (length 48 chars) that I want to convert to raw bytes with the pack function in order to put it in a Win32 vector of bytes.
How I can do this with Perl?
my $bytes = pack "H*", $hex;
See perlpacktut for more information.
The steps are:
Extract pairs of hexadecimal characters from the string.
Convert each pair to a decimal number.
Pack the number as a byte.
For example:
use strict;
use warnings;
my $string = 'AA55FF0102040810204080';
my #hex = ($string =~ /(..)/g);
my #dec = map { hex($_) } #hex;
my #bytes = map { pack('C', $_) } #dec;
Or, expressed more compactly:
use strict;
use warnings;
my $string = 'AA55FF0102040810204080';
my #bytes = map { pack('C', hex($_)) } ($string =~ /(..)/g);
I have the string:
"61 62 63 64 65 67 69 69 6a"
which I want to interpret as hex values, and display those as ASCII chars (those values should reproduce the character string "abcdefghij").
Typically, I try to write something quick like this:
$ echo "61 62 63 64 65 67 69 69 6a" | perl -ne 'print "$_"; print pack("H2 "x10, $_)."\n";'
61 62 63 64 65 67 69 69 6a
a
... and then I wonder, why do I get only one character back :)
First, let me note down that the string I have, can also be represented as the hex values of bytes that it takes up in memory:
$ echo -n "61 62 63 64 65 67 68 69 6a" | hexdump -C
00000000 36 31 20 36 32 20 36 33 20 36 34 20 36 35 20 36 |61 62 63 64 65 6|
00000010 37 20 36 38 20 36 39 20 36 61 |7 68 69 6a|
0000001a
_(NB: Essentially, I want to "convert" the above byte values in memory as input, to these below ones, if viewed by hexdump:
$ echo -n "abcdefghij" | hexdump -C
00000000 61 62 63 64 65 66 67 68 69 6a |abcdefghij|
0000000a
... which is how the original values for the input hex string were obtained.
)_
Well, this Pack/Unpack Tutorial (AKA How the System Stores Data) turns out is the most helpful for me, as it mentions:
The pack function accepts a template string and a list of values [...]
$rec = pack( "l i Z32 s2", time, $emp_id, $item, $quan, $urgent);
It returns a scalar containing the list of values stored according to the formats specified in the template [...]
$rec would contain the following (first line in decimal, second in hex, third as characters where applicable). Pipe characters indicate field boundaries.
Offset Contents (increasing addresses left to right)
0 160 44 19 62| 41 82 3 0| 98 111 120 101 115 32 111 102
A0 2C 13 3E| 29 52 03 00| 62 6f 78 65 73 20 6f 66
| b o x e s o f
That is, in my case, $_ is a single string variable -- whereas pack expects as input a list of several such 'single' variables (in addition to a formatting template string); and outputs again a 'single' variable (which could, however, be a sizeable chunk of memory!). In my case, if the output 'single' variable contains the ASCII code in each byte in memory, then I'm all set (I could simply print the output variable directly, then).
Thus, in order to get a list of variables from the $_ string, I can simply split it at the space sign - however, note:
$ echo "61 62 63 64 65 67 68 69 6a" | perl -ne 'print "$_"; print pack("H2", split(/ /, $_))."\n";'
61 62 63 64 65 67 68 69 6a
a
... that amount of elements to be packed must be specified (otherwise again we get only one character back); then, either of these alternatives work:
$ echo "61 62 63 64 65 67 68 69 6a" | perl -ne 'print "$_"; print pack("H2"x10, split(/ /, $_))."\n";'
61 62 63 64 65 67 68 69 6a
abcdeghij
$ echo "61 62 63 64 65 67 68 69 6a" | perl -ne 'print "$_"; print pack("(H2)*", split(/ /, $_))."\n";'
61 62 63 64 65 67 68 69 6a
abcdeghij