I'm attempting to write a single bpftrace script which grab the strings passing from a postfix process and a saslauthd for the authentication part. The goal is detect compromise account of my company. The strace command give me some good results:
strace -p PID -s 100 -e 'read'
read(7, "\0\20", 2) = 2
read(7, "xxxxxxxxxx", 10) = 10
read(7, "\0\t", 2) = 2
read(7, "YYYYYYYYY", 9) = 9
read(7, "\0\4", 2) = 2
read(7, "smtp", 4) = 4
I can recover login/password and detect if there is a bruteforce running.
but I try to have the same results with bpftrace with:
$ bpftrace -e 'kprobe:sys_read /comm=="saslauthd"/ {printf("%<%s>\n",str(arg1,arg2));}'
<>
<login>
<>
<>
<>
<smtp>
In this case, I can read some sys_read syscall strings but not all. I don't understand why my bpftrace doesn't have the same result.
I also think about the null character and that why i use str(arg1,arg2) to force the size of the array. I'v also tried to use tracepoint and this is the same result.
Maybe someone can help me to understand where is my error ? So any input will be appreciated
TL;DR. That's actually the expected behavior of str(buf, len). It retrieves the string pointed to by buf, with a limit to len characters including the NULL character. Thus, since in your case some strings start with a NULL character, str() will copy an empty string.
Sources. bpftrace translates str() into a call to the BPF_FUNC_probe_read_str BPF helper. In the kernel, that helper itself calls strncpy_from_unsafe.
I don't think bpftrace already has a function implementing what you're looking for. If you want your described semantics, you could ask for a copy() function in bpftrace. Though, looking at the commit that introduced str(), it shouldn't be too hard to write a patch for that. Don't hesitate to send a pull request!
Related
I basically want to check the incoming 'From' in the email received and then either
Keep it and make it deliver to the intended mailbox if the email matches a Specified MySQL/PostgreSQL
Database User (eg. select email from users where exists ('from email address') )
If the 'From' address is blank or it is not found in the database, the email should be discarded
Any way I can achieve this before the e-mail is delivered to the intended mailbox?
I am using Procmail + Virtualmin + Webmin + PostgreSQL
PS: I want to apply this filter not to the wole server but to some specified mailboxes/users (i'm assuming 1 user = 1 mailbox here)
Procmail can easily run an external command in a condition and react to its exit status. How exactly to make your particular SQL client set its exit code will depend on which one you are using; perhaps its man page will reveal an option to make it exit with an error when a query produces an empty result set, for example? Or else write a shell wrapper to look for empty output.
A complication is that Procmail (or rather, the companion utility formail) can easily extract a string from e.g. the From: header; but you want to reduce this to just the email terminus. This is a common enough task that it's easy to find a canned solution - generate a reply and then extract the To: address (sic!) from that.
FROM=`formail -rtzxTo:`
:0
* FROM ?? ^(one#example\.com|two#site\.example\.net|third#example\.org)$
{
:0
* ? yoursql --no-headers --fail-if-empty-result \
--batch --query databasename \
--eval "select yada yada where address = '$FROM'"
{ }
:0E
/dev/null
}
The first condition examines the variable and succeeds if it contains one of the addresses (my original answer simply had the regex . which matches if the string contains at least one character, any character; I'm not convinced this is actually necessary or useful; there should be no way for From: to be empty). If it is true, Procmail enters the braces; if not, they will be skipped.
The first recipe inside the braces runs an external command and examines its exit code. I'm imagining your SQL client is called yoursql and that it has options to turn off human-friendly formatting (table headers etc) and for running a query directly from the command line on a specific database. We use double quotes so that the shell will interpolate the variable FROM before running this command (maybe there is a safer way to pass string variables which might contain SQL injection attempts with something like --variable from="$FROM" and then use that variable in the query? See below.)
If there is no option to directly set the exit code, but you can make sure standard output is completely empty in the case of no result, piping the command to grep -q . will produce the correct exit code. In a more complex case, maybe write a simple Awk script to identify an empty result set and set its exit status accordingly.
Scraping together information from
https://www.postgresql.org/docs/current/app-psql.html,
How do you use script variables in psql?,
Making an empty output from psql,
and from your question, I end up with the following attempt to implement this in psql; but as I don't have a Postgres instance to test with, or any information about your database schema, this is still approximate at best.
* ? psql --no-align --tuples-only --quiet \
--dbname=databasename --username=something --no-password \
--variable=from="$FROM" \
--command="select email from users where email = :'from'" \
| grep -q .
(We still can't use single quotes around the SQL query, to completely protect it from the shell, because Postgres insists on single quotes around :'from', and the shell offers no facility for embedding literal single quotes inside single quotes.)
The surrounding Procmail code should be reasonably self-explanatory, but here goes anyway. In the first recipe inside the braces, if the condition is true, the empty braces in its action line are a no-op; the E flag on the next recipe is a condition which is true only if any of the conditions on the previous recipe failed. This is a common idiom to avoid having to use a lot of negations; perhaps look up "de Morgan's law". The net result is that we discard the message by delivering it to /dev/null if either condition in the first recipe failed; and otherwise, we simply pass it through, and Procmail will eventually deliver it to its default destination.
The recipe was refactored in response to updates to your question; perhaps now it would make more sense to just negate the exit code from psql with a ! in front:
FROM=`formail -rtzxTo:`
:0
* FROM ?? ^(one#example\.com|two#site\.example\.net|third#example\.org)$
* ! ? psql --no-align --tuples-only --quiet \
--dbname=databasename --username=something --no-password \
--variable=from="$FROM" \
--command="select email from users where email = :'from'" \
| grep -q .
/dev/null
Tangentially, perhaps notice how Procmail's syntax exploits the fact that a leading ? or a doubled ?? are not valid in regular expressions. So the parser can unambiguously tell that these conditions are not regular expressions; they compare a variable to the regex after ??, or examine the exit status of an external command, respectively. There are a few other special conditions like this in Procmail; arguably, all of them are rather obscure.
Newcomers to shell scripting should also notice that each shell command pipeline has two distinct results: whatever is being printed on standard output, and, completely separate from that, an exit code which reveals whether or not the command completed successfully. (Conventionally, a zero exit status signals success, and anything else is an error. For example, the exit status from grep is 0 if it finds at least one match, 1 if it doesn't, and usually some other nonzero exit code if you passed in an invalid regular expression, or you don't have permission to read the input file, etc.)
For further details, perhaps see also http://www.iki.fi/era/procmail/ which has an old "mini-FAQ" which covers several of the topics here, and a "quick reference" for looking up details of the syntax.
I'm not familiar with Virtualmin but https://docs.virtualmin.com/Webmin/PostgreSQL_Database_Server shows how to set up Postgres and as per https://docs.virtualmin.com/Webmin/Procmail_Mail_Filter I guess you will want to use the option to put this code in an include file.
-- Edit : Resolved. See answer.
Background:
I'm writing a shell that will perform some extra actions required on our system when someone resizes a database.
The shell is written in ksh (requirement), the OS is Solaris 5.10 .
The problem is with one of the checks, which verifies there's enough free space on the underlying OS.
Problem:
The check reads the df -k line for root, which is what I check in this step, and prints it to a file. I then "read" the contents into variables which I use in calculations.
Unfortunately, when I try to run an arithmetic operation on one of the variables, I get an error indicating it is null. And a debug output line I've placed after that line verifies that it is null... It lost it's value...
I've tried every method of doing this I could find online, they work when I run it manually, but not inside the shell file.
(* The file does have #!/usr/bin/ksh)
Code:
df -k | grep "rpool/ROOT" > dftest.out
RPOOL_NAME=""; declare -i TOTAL_SIZE=0; USED_SPACE=0; AVAILABLE_SPACE=0; AVAILABLE_PERCENT=0; RSIGN=""
read RPOOL_NAME TOTAL_SIZE USED_SPACE AVAILABLE_SPACE AVAILABLE_PERCENT RSIGN < dftest.out
\rm dftest.out
echo $RPOOL_NAME $TOTAL_SIZE $USED_SPACE $AVAILABLE_SPACE $AVAILABLE_PERCENT $RSIGN
((TOTAL_SIZE=$TOTAL_SIZE/1024))
This is the result:
DBResize.sh[11]: TOTAL_SIZE=/1024: syntax error
I'm pulling hairs at this point, any help would be appreciated.
The code you posted cannot produce the output you posted. Most obviously, the error is signalled at line 11 but you posted fewer than 11 lines of code. The previous lines may matter. Always post complete code when you ask for help.
More concretely, the declare command doesn't exist in ksh, it's a bash thing. You can achieve the same result with typeset (declare is a bash equivalent to typeset, but not all options are the same). Either you're executing this script with bash, or there's another error message about declare, or you've defined some additional commands including declare which may change the behavior of this code.
None of this should have an impact on the particular problem that you're posting about, however. The variables created by read remain assigned until the end of the subshell, i.e. until the code hits a ), the end of a pipe (left-hand side of the pipe only in ksh), etc.
About the use of declare or typeset, note that you're only declaring TOTAL_SIZE as an integer. For the other variables, you're just assigning a value which happens to consist exclusively of digits. It doesn't matter for the code you posted, but it's probably not what you meant.
One thing that may be happening is that grep matches nothing, and therefore read reads an empty line. You should check for errors. Use set -e in scripts to exit at the first error. (There are cases where set -e doesn't catch errors, but it's a good start.)
Another thing that may be happening is that df is splitting its output onto multiple lines because the first column containing the filesystem name is too large. To prevent this splitting, pass the option -P.
Using a temporary file is fragile: the code may be executed in a read-only directory, another process may want to access the same file at the same time... Here a temporary file is useless. Just pipe directly into read. In ksh (unlike most other sh variants including bash), the right-hand side of a pipe runs in the main shell, so assignments to variables in the right-hand side of a pipe remain available in the following commands.
It doesn't matter in this particular script, but you can use a variable without $ in an arithmetic expression. Using $ substitutes a string which can have confusing results, e.g. a='1+2'; $((a*3)) expands to 7. Not using $ uses the numerical value (in ksh, a='1+2'; $((a*3)) expands to 9; in some sh implementations you get an error because a's value is not numeric).
#!/usr/bin/ksh
set -e
typeset -i TOTAL_SIZE=0 USED_SPACE=0 AVAILABLE_SPACE=0 AVAILABLE_PERCENT=0
df -Pk | grep "rpool/ROOT" | read RPOOL_NAME TOTAL_SIZE USED_SPACE AVAILABLE_SPACE AVAILABLE_PERCENT RSIGN
echo $RPOOL_NAME $TOTAL_SIZE $USED_SPACE $AVAILABLE_SPACE $AVAILABLE_PERCENT $RSIGN
((TOTAL_SIZE=TOTAL_SIZE/1024))
Strange...when I get rid of your "declare" line, your original code seems to work perfectly well (at least with ksh on Linux)
The code :
#!/bin/ksh
df -k | grep "/home" > dftest.out
read RPOOL_NAME TOTAL_SIZE USED_SPACE AVAILABLE_SPACE AVAILABLE_PERCENT RSIGN < dftest.out
\rm dftest.out
echo $RPOOL_NAME $TOTAL_SIZE $USED_SPACE $AVAILABLE_SPACE $AVAILABLE_PERCENT $RSIGN
((TOTAL_SIZE=$TOTAL_SIZE/1024))
print $TOTAL_SIZE
The result :
32962416 5732492 25552588 19% /home
5598
Which are the value a simple df -k is returning. The variables seem to last.
For those interested, I have figured out that it is not possible to use "read" the way I was using it.
The variable values assigned by "read" simply "do not last".
To remedy this, I have applied the less than ideal solution of using the standard "while read" format, and inside the loop, echo selected variables into a variable file.
Once said file was created, I just "loaded" it.
(pseudo code:)
LOOP START
echo "VAR_A="$VAR_A"; VAR_B="$VAR_B";" > somefile.out
LOOP END
. somefile.out
I see this awk idiom all the time and I would like to know exactly what it means
while (getline <"FILE" > 0)
I understand what getline is doing, but I don't get redirection to 0.
According to The Awk Programming Language, you do this to avoid a infinite loop in the case of FILE being nonexistant.
Why does redirection get rid of that error? What exactly is being redirected? It can't be the return value of getline, otherwise, what expression is while evaluating? I'm missing something!
EDIT: Thanks for the comments. I got redirection operators confused with relational operators. Perhaps it would be clearer if it was written like this
while ((getline <"FILE") > 0)
Or even clearer
while (0 < (getline <"FILE"))
The > sign is testing the return value from getline (for greater than zero). It is not a redirection sign.
From https://www.gnu.org/software/gawk/manual/html_node/Getline.html :
The getline command returns one if it finds a record and zero if it
encounters the end of the file. If there is some error in getting a
record, such as a file that cannot be opened, then getline returns -1.
In this case, gawk sets the variable ERRNO to a string describing the
error that occurred.
The confusion may come from the fact that output redirection to a file is not used by getline. (It is used only to read data). So there should be no ambiguity regarding the > sign. (It cannot mean output redirection)
To print something to a file you can use output redirection from the print or printf commands.
GCC version 4.6
The Problem: To find a way to feed in parameters to the executable, say a.out, from the command line - more specifically feed in an array of double precision numbers.
Attempt: Using the READ(*,*) command, which is older in the standard:
Program test.f -
PROGRAM MAIN
REAL(8) :: A,B
READ(*,*) A,B
PRINT*, A+B, COMMAND_ARGUMENT_COUNT()
END PROGRAM MAIN
The execution -
$ gfortran test.f
$ ./a.out 3.D0 1.D0
This did not work. On a bit of soul-searching, found that
$./a.out
3.d0,1.d0
4.0000000000000000 0
does work, but the second line is an input prompt, and the objective of getting this done in one-line is not achieved. Also the COMMAND_ARGUMENT_COUNT() shows that the numbers fed into the input prompt don't really count as 'command line arguments', unlike PERL.
If you want to get the arguments fed to your program on the command line, use the (since Fortran 2003) standard intrinsic subroutine GET_COMMAND_ARGUMENT. Something like this might work
PROGRAM MAIN
REAL(8) :: A,B
integer :: num_args, ix
character(len=12), dimension(:), allocatable :: args
num_args = command_argument_count()
allocate(args(num_args)) ! I've omitted checking the return status of the allocation
do ix = 1, num_args
call get_command_argument(ix,args(ix))
! now parse the argument as you wish
end do
PRINT*, A+B, COMMAND_ARGUMENT_COUNT()
END PROGRAM MAIN
Note:
The second argument to the subroutine get_command_argument is a character variable which you'll have to parse to turn into a real (or whatever). Note also that I've allowed only 12 characters in each element of the args array, you may want to fiddle around with that.
As you've already figured out read isn't used for reading command line arguments in Fortran programs.
Since you want to read an array of real numbers, you might be better off using the approach you've already figured out, that is reading them from the terminal after the program has started, it's up to you.
The easiest way is to use a library. There is FLAP or f90getopt available. Both are open source and licensed under free licenses.
The latter is written by Mark Gates and me, just one module and can be learned in minutes but contains all what is needed to parse GNU- and POSIX-like command-line options. The first is more sophisticated and can be used even in closed-source projects. Check them out.
Furthermore libraries at https://fortranwiki.org/fortran/show/Command-line+arguments
What READ (*,*) does is that it reads from the standard input. For example, the characters entered using the keyboard.
As the question shows COMMAND_ARGUMENT_COUNT() can be used to get the number of the command line arguments.
The accepted answer by High Performance Mark show how to retrieve the individual command line arguments separated by blanks as individual character strings using GET_COMMAND_ARGUMENT(). One can also get the whole command line using GET_COMMAND(). One then has to somehow parse that character-based information into the data in your program.
I very simple cases you just need the program requires, for example, two numbers, so you read one number from arg 1 and another form arg 2. That is simple. Or you can read a triplet of numbers from a single argument if they are comma-separated like 1,2,3 using a simple read(arg,*) nums(1:3).
For general complicated command line parsing one uses libraries such as those mentioned in the answer by Hani. You have set them up so that the library knows the expected syntax of the command line arguments and the data it should fill with the values.
There is a middle ground, that is still relatively simple, but one already have multiple arguments, that correspond to Fortran variables in the program, that may or may not be present. In that case one can use the namelist for the syntax and for the parsing.
Here is an example, the man point is the namelist /cmd/ name, point, flag:
implicit none
real :: point(3)
logical :: flag
character(256) :: name
character(1024) :: command_line
call read_command_line
call parse_command_line
print *, point
print *, "'",trim(name),"'"
print *, flag
contains
subroutine read_command_line
integer :: exenamelength
integer :: io, io2
command_line = ""
call get_command(command = command_line,status = io)
if (io==0) then
call get_command_argument(0,length = exenamelength,status = io2)
if (io2==0) then
command_line = "&cmd "//adjustl(trim(command_line(exenamelength+1:)))//" /"
else
command_line = "&cmd "//adjustl(trim(command_line))//" /"
end if
else
write(*,*) io,"Error getting command line."
end if
end subroutine
subroutine parse_command_line
character(256) :: msg
namelist /cmd/ name, point, flag
integer :: io
if (len_trim(command_line)>0) then
msg = ''
read(command_line,nml = cmd,iostat = io,iomsg = msg)
if (io/=0) then
error stop "Error parsing the command line or cmd.conf " // msg
end if
end if
end subroutine
end
Usage in bash:
> ./command flag=T name=\"data.txt\" point=1.0,2.0,3.0
1.00000000 2.00000000 3.00000000
'data.txt'
T
or
> ./command flag=T name='"data.txt"' point=1.0,2.0,3.0
1.00000000 2.00000000 3.00000000
'data.txt'
T
Escaping the quotes for the string is unfortunately necessary, because bash eats the first quotes.
Background: I need to port a ksh script from SunOS 5.10 to RHEL 5.8. It makes a call to isql to retrieve some data and, quite contrary to the intended application of final endpoint client utilities such as isql, it parses its out to be used by a variable in the shell script. Please note that I just inherited this and by no means did design such a hack myself. I certainly never would be parsing isql out to assign value to a var in shell -- if the script needed that info, I would use Perl with some API like DBD::DBI that is designed to marshall data between the application and the data store. But I have what I have and must work within the parameters.
What is happening is that the following piped input does return data on SunOS but not in RHEL:
echo "SELECT some_field FROM some_table WHERE some_crtra = 'X' \ngo" | isql -U$USER -P$PASS -D$DB -S$SERVER
That output on Solaris being:
some_field
------
Y
(1 row affected)
From that point, the script uses awk to extract just the field value from the above stream but let's ignore that because that's not the problem.
Also please note that I am able to get the data executing the piped commands separately, i.e. by going manually into isql and running the SQL. So the SQL or the connection string are not the problem -- it is either how the piping streams data OR isql itself works differently on the different platforms.
Can anybody see why there is disparate response to the same input on the two systems? Any idea how I can change the piping to make it work?
Thanks
echo "SELECT some_field FROM some_table WHERE some_crtra = 'X' \ngo"
is non portable.
I would suggest instead:
printf "SELECT some_field FROM some_table WHERE some_crtra = 'X' \ngo\n"
From the ksh93 manual page:
When the first arg does not begin with a -, and none of the
arguments contain a \, then echo prints each of its arguments
separated by a space and terminated by a new-line. Otherwise,
the behavior of echo is system dependent and print or printf
described below should be used. See echo(1) for usage and
description.