Better method to append to output crontabs - sh

So i have simple shell commands to ping websites to retrieve data about said websites.
For example one of my pinging.sh looks like this:
ping -R -c 120 blar.org.cn >> pingdata.txt
ping -R -c 120 another.net >> pingdata.txt
But then my crontabs look like this:
7 * * * ./pinging.sh >> pingdata.log
The pingdata.log doesn't output. Is it best to do it through the crontab or through the script? I assumed the crontab would be better because it would cover the entire script rather than having to write it out multiple times.

You need to indicate the full path of your script in the cronjob, together with the binary running it.
For example:
7 * * * * /bin/sh /home/you/pinging.sh >> /home/you/pingdata.log
Note also you are just adding 4 parameters to the cronjob, whereas you need at least 5:
+---------------- minute (0 - 59)
| +------------- hour (0 - 23)
| | +---------- day of month (1 - 31)
| | | +------- month (1 - 12)
| | | | +---- day of week (0 - 6) (Sunday=0 or 7)
| | | | |
* * * * * command to be executed
You can test your cron syntax with Crontab guru (---> http://crontab.guru/).

First, the executable must be provided as full path in cron.
Example:
7 * * * * /bin/bash /path/to/pinging.sh
Second, create a wrapper script for pinging.sh >> pingdata.log and add that to crontab.
Third, your crontab entry is wrong. There must be 5 fields whereas your's have 4 (maybe that's a typo ?)

Related

How to use if condition in selenium IDE

3 login based questions in the website to be automated
I need correct code to automate these.
enter image description here
When using "if ..." Commands in test steps in Selenium IDE for Chrome, you need to remember the following:
Use "if ... end" to form a single branch of test steps.
Use "if ... else ... end" to form two branches of test steps.
Use "if ... else if ... else ... end" to form multiple branches of test steps.
Use JavaScript Boolean expression syntax to specify conditions to "if" and "else if" commands.
${variable} is allowed in condition expressions.
Here is a sample test called "Condition" on how to use variables:
1 execute script | return (new Date()).getDay(); | day
2 echo | ${day}
3 if | ${day} < 1
4 echo | Sunday
5 else if | ${day} > 5
6 echo | Saturday
7 else
8 echo Weekday
9 end
Here is the log output when you execute the above test.
1. executeScript on return (new Date()).getDay(); with value day OK
echo: 0
3. if on ${day} < 1 OK
echo: Sunday
5. elseIf on ${day} > 5 OK
7. else OK
9. end OK

Issue with aix sed command

I am trying to uncomment "/usr/lib/sa/sa1" or "/usr/lib/sa/sa2" entries from one of the configuration file.
Below is the regex:
"^[0-9].*/usr/lib/sa/sa(1|2)"
Eg:
#0 * * * * /usr/lib/sa/sa1 1200 3 &
#5 23 * * * /usr/lib/sa/sa2 -s 0:00 -e 23:01 -i 3600 -ubcwyaqvm &
Output should be:
0 * * * * /usr/lib/sa/sa1 1200 3 &
5 23 * * * /usr/lib/sa/sa2 -s 0:00 -e 23:01 -i 3600 -ubcwyaqvm &
Tried the below "sed" command but no luck:
sed -e '/^#.*\/usr\/lib\/sa\/sa\(1\|2\)/s/^#//g' adm
Can you please let me know where am I going wrong here.
Thanks in advance!
You can use
sed '/^#.*\/usr\/lib\/sa\/sa[12]/s/^#//'
Here, no grouping construct is used, no alternation operator is used either. It is possible as the 1 and 2 are single char alternatives, and bracket expressions are supported in all versions of sed.
Also, see this online demo.

replacing first pattern using sed

I have a cron job with below details
The timing section is unknown, it might be hourly or daily.
cat zyx.txt
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
0 * * * * root /usr/local/bin/log-cleaner
I have to update it on the fly using sed, so that it should run every minute .
Here is what I have tried
sed -in 's/\(.*\) \(.*\) \(.*\)/\* \* \* \* \*/1' zyx.txt
But it is removing the user and path section
cat zyx.txt
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
* * * * *
Any help / suggestion using sed is much appreciated

Scala : Cant run gcloud compute ssh

I am trying to run a hive query using gcloud compute ssh via scala
First, here is what i tried
scala> import sys.process._
scala> val results = Seq("hive", "-e", "show databases;").!!
asd
zxc
qwe
scala>
which is good. Now, i want to run the same hive command, but against a GCP cluster. I have gcloud setup on my VM and from the command line, i can easily do
$ gcloud compute ssh --zone myZone myNode --internal-ip -- 'hive -e "show databases;"'
Updating project ssh metadata...⠶Updated [https://www.googleapis.com/compute/v1/projects/myProject].
Updating project ssh metadata...done.
Waiting for SSH key to propagate.
Warning: Permanently added 'compute.2746937995265952194' (RSA) to the list of known hosts.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 19 100 19 0 0 2982 0 --:--:-- --:--:-- --:--:-- 3166
Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j2.properties Async: true
OK
asd
zxc
qwe
Now, I want to run the above using scala. Here is what i tried
scala> val results = Seq("gcloud", "compute", "ssh", "--zone", "myZone", "myNode", "--internal-ip", "--", "hive", "-e" ,"show databases").!!
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 19 100 19 0 0 3270 0 --:--:-- --:--:-- --:--:-- 3800
Pseudo-terminal will not be allocated because stdin is not a terminal.
Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j2.properties Async: true
NoViableAltException(-1#[846:1: ddlStatement : ( createDatabaseStatement | switchDatabaseStatement | dropDatabaseStatement | createTableStatement | dropTableStatement | truncateTableStatement | alterStatement | descStatement | showStatement | metastoreCheck | createViewStatement | createMaterializedViewStatement | dropViewStatement | dropMaterializedViewStatement | createFunctionStatement | createMacroStatement | createIndexStatement | dropIndexStatement | dropFunctionStatement | reloadFunctionStatement | dropMacroStatement | analyzeStatement | lockStatement | unlockStatement | lockDatabase | unlockDatabase | createRoleStatement | dropRoleStatement | ( grantPrivileges )=> grantPrivileges | ( revokePrivileges )=> revokePrivileges | showGrants | showRoleGrants | showRolePrincipals | showRoles | grantRole | revokeRole | setRole | showCurrentRole | abortTransactionStatement );])
at org.antlr.runtime.DFA.noViableAlt(DFA.java:158)
at org.antlr.runtime.DFA.predict(DFA.java:144)
at org.apache.hadoop.hive.ql.parse.HiveParser.ddlStatement(HiveParser.java:3757)
at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2382)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1333)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:208)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:787)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:244)
at org.apache.hadoop.util.RunJar.main(RunJar.java:158)
FAILED: ParseException line 1:4 cannot recognize input near 'show' '<EOF>' '<EOF>' in ddl statement
java.lang.RuntimeException: Nonzero exit value: 64
at scala.sys.package$.error(package.scala:27)
at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.slurp(ProcessBuilderImpl.scala:132)
at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.$bang$bang(ProcessBuilderImpl.scala:102)
... 50 elided
scala>
why am i getting this error ? I also tried
scala> val results = Seq("gcloud", "compute", "ssh", "--zone", "myZone", "myNode", "--internal-ip", "--", "hive", "-e" ,"show databases;").!!
but got the same error. Then i tried
scala> val results = Seq("gcloud", "compute", "ssh", "--zone", "myZone", "myNode", "--internal-ip", "--", "'hive -e \"show databases;\"'").!!
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 19 100 19 0 0 3245 0 --:--:-- --:--:-- --:--:-- 3800
Pseudo-terminal will not be allocated because stdin is not a terminal.
bash: hive -e "show databases;": command not found
java.lang.RuntimeException: Nonzero exit value: 127
at scala.sys.package$.error(package.scala:27)
at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.slurp(ProcessBuilderImpl.scala:132)
at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.$bang$bang(ProcessBuilderImpl.scala:102)
... 50 elided
How can I run the gcloud comput ssh properly using scala ?
You don't need the single quotes in your last example. You're trying to pass the string:
hive -e "show databases;"
For fun, I would use triple quotes in Scala:
"""hive -e "show databases;""""
to avoid backslash. Single quotes in your good command line are processed by bash.
This is what worked in bash:
$ gcloud compute ssh --zone myZone myNode --internal-ip -- 'hive -e "show databases;"'
scala.sys.process got some basic parsing at some point. There is a space in this file name that must be quoted. Amazingly, it seems to do shell-style quotes:
$ scala
Welcome to Scala 2.13.0 (OpenJDK 64-Bit Server VM, Java 11.0.3).
Type in expressions for evaluation. Or try :help.
scala> import scala.sys.process._
import scala.sys.process._
scala> "ls -l /tmp/skypeforlinux Crashes".!!
ls: cannot access '/tmp/skypeforlinux': No such file or directory
ls: cannot access 'Crashes': No such file or directory
java.lang.RuntimeException: Nonzero exit value: 2
at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.slurp(ProcessBuilderImpl.scala:155)
at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.$bang$bang(ProcessBuilderImpl.scala:112)
... 28 elided
scala> """ls -l "/tmp/skypeforlinux Crashes"""".!!
res1: String =
"total 0
"
scala> """ls -l '/tmp/skypeforlinux Crashes'""".!!
res2: String =
"total 0
"
scala> """ls -l /tmp/skypeforlin'ux Cr'ashes""".!!
res3: String =
"total 0
"
scala> """echo 'hive -e "show databases;"'""".!!
res4: String =
"hive -e "show databases;"
"
The double quotes around "my house" are part of the file name:
scala> """ls '/tmp/"my house"'""".!!
res5: String =
"/tmp/"my house"
"
I guess that code is where I learned how shell-style quotes work, though I never have a chance to use that knowledge. Except for this answer, so thanks for the opportunity.

crontab : yesterday's date not showing up

I need a cron job to work on a file named like this:
20160307_20160308_xxx_yyy.csv
(yesterday_today_xxx_yyy.csv)
And my cron job looks like this:
53 11 * * * /path/to/python /path/to/python/script /path/to/file/$(date -d "yesterday" +"\%Y\%m\%d")_$(date +"\%Y\%m\%d")_xxx_yyy.csv >> /path/to/logfile/cron.log 2>&1
Today's date is getting calculated properly but I am unable to get yesterday's date working. The error is:
IOError: [Errno 2] No such file or directory: 'tmp/_20160308_xxx_yyy.csv'
Please help!
I found the answer to my own question.
I needed to use this to get yesterday's date:
53 11 * * * /path/to/python /path/to/python/script /path/to/file/$(date -v-1d +"\%Y\%m\%d")_$(date +"\%Y\%m\%d")_xxx_yyy.csv >> /path/to/logfile/cron.log 2>&1
Hope it helps somebody!
This version worked for me. Maybe it can be helpful for someone:
53 11 * * * /path/to/python /path/to/python/script /path/to/file/$(date --date '-1 day' +"\%Y\%m\%d")_$(date +"\%Y\%m\%d")_xxx_yyy.csv >> /path/to/logfile/cron.log 2>&1