Working on a BSD system and having difficultly searching a variable to see if it contains any of multiple different substrings. This is what I have that works:
> echo $lenck if test "${lenck#*$pattern1}" != "$lenck";
> then
> if test "${lenck#*$pattern2}" != "$lenck";
> then
> echo "Invalid or corrupted IP or process"
> echo $lenck | tee -a invalid.txt
> else
> pat2out=$lenck
> if test "${pat2out#*$pattern3}" != "$pat2out";
> then
> echo "Invalid or corrupted IP or process"
> echo $pat2out | tee -a invalid.txt
> else
> pat3out=$pat2out
> if test "${pat3out#*$pattern4}" != "$pat3out";
> then
> echo "Invalid or corrupted IP or process"
> echo $pat3out | tee -a invalid.txt
> else
> echo $pat3out | tee -a processed.txt
> fi
> fi
> fi
>
> else
> echo "Invalid or corrupted IP or process"
> echo $lenck | tee -a invalid.txt fi
The problem is that I need to be able to search for many more than four patterns. Tried grep -e and other approaches but they aren't correctly filtering the strings to the correct file. Any help is appreciated.
It looks like all of your tests actually test against the same input string, so you could use a case statement:
echo "$lenck"
case "$lenck" in
*$pattern1*|*$pattern2*|*$pattern3*)
echo "Invalid or corrupted IP or process"
echo "$lenck" >> invalid.txt
;;
*)
echo "$lenck" >> processed.txt
;;
esac
Related
I have a Scala application that I want to run inside a Docker container. To build the docker image, I use sbt-native-packager.
The base image I am using is "openjdk:8-jre-alpine".
Tried "openjdk:8-jdk-alpine" - does not make any difference
Tried sbt-native-packager 1.3.20 - does not make any difference
project/plugins.sbt
resolvers += Resolver.typesafeRepo("releases")
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.3.17")
build.sbt
enablePlugins(JavaAppPackaging)
mainClass in Compile := Some("MyAppClass")
enablePlugins(DockerPlugin)
dockerBaseImage := "openjdk:8-jre-alpine" // startup fails with permission denied if using alpine :-(
Running a container with the resulting image leads to following error on startup:
docker run my-app:latest
docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: \"/opt/docker/bin/my-app\": permission denied": unknown.
ERRO[0000] error waiting for container: context canceled
The app starts up normally when using "openjdk:8-jre".
Update
Contents of /opt/docker/bin/my-app:
#!/usr/bin/env bash
### ------------------------------- ###
### Helper methods for BASH scripts ###
### ------------------------------- ###
die() {
echo "$#" 1>&2
exit 1
}
realpath () {
(
TARGET_FILE="$1"
CHECK_CYGWIN="$2"
cd "$(dirname "$TARGET_FILE")"
TARGET_FILE=$(basename "$TARGET_FILE")
COUNT=0
while [ -L "$TARGET_FILE" -a $COUNT -lt 100 ]
do
TARGET_FILE=$(readlink "$TARGET_FILE")
cd "$(dirname "$TARGET_FILE")"
TARGET_FILE=$(basename "$TARGET_FILE")
COUNT=$(($COUNT + 1))
done
if [ "$TARGET_FILE" == "." -o "$TARGET_FILE" == ".." ]; then
cd "$TARGET_FILE"
TARGET_FILEPATH=
else
TARGET_FILEPATH=/$TARGET_FILE
fi
# make sure we grab the actual windows path, instead of cygwin's path.
if [[ "x$CHECK_CYGWIN" == "x" ]]; then
echo "$(pwd -P)/$TARGET_FILE"
else
echo $(cygwinpath "$(pwd -P)/$TARGET_FILE")
fi
)
}
# TODO - Do we need to detect msys?
# Uses uname to detect if we're in the odd cygwin environment.
is_cygwin() {
local os=$(uname -s)
case "$os" in
CYGWIN*) return 0 ;;
*) return 1 ;;
esac
}
# This can fix cygwin style /cygdrive paths so we get the
# windows style paths.
cygwinpath() {
local file="$1"
if is_cygwin; then
echo $(cygpath -w $file)
else
echo $file
fi
}
# Make something URI friendly
make_url() {
url="$1"
local nospaces=${url// /%20}
if is_cygwin; then
echo "/${nospaces//\\//}"
else
echo "$nospaces"
fi
}
# This crazy function reads in a vanilla "linux" classpath string (only : are separators, and all /),
# and returns a classpath with windows style paths, and ; separators.
fixCygwinClasspath() {
OLDIFS=$IFS
IFS=":"
read -a classpath_members <<< "$1"
declare -a fixed_members
IFS=$OLDIFS
for i in "${!classpath_members[#]}"
do
fixed_members[i]=$(realpath "${classpath_members[i]}" "fix")
done
IFS=";"
echo "${fixed_members[*]}"
IFS=$OLDIFS
}
# Fix the classpath we use for cygwin.
fix_classpath() {
cp="$1"
if is_cygwin; then
echo "$(fixCygwinClasspath "$cp")"
else
echo "$cp"
fi
}
# Detect if we should use JAVA_HOME or just try PATH.
get_java_cmd() {
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
echo "$JAVA_HOME/bin/java"
else
echo "java"
fi
}
echoerr () {
echo 1>&2 "$#"
}
vlog () {
[[ $verbose || $debug ]] && echoerr "$#"
}
dlog () {
[[ $debug ]] && echoerr "$#"
}
execRunner () {
# print the arguments one to a line, quoting any containing spaces
[[ $verbose || $debug ]] && echo "# Executing command line:" && {
for arg; do
if printf "%s\n" "$arg" | grep -q ' '; then
printf "\"%s\"\n" "$arg"
else
printf "%s\n" "$arg"
fi
done
echo ""
}
# we use "exec" here for our pids to be accurate.
exec "$#"
}
addJava () {
dlog "[addJava] arg = '$1'"
java_args+=( "$1" )
}
addApp () {
dlog "[addApp] arg = '$1'"
app_commands+=( "$1" )
}
addResidual () {
dlog "[residual] arg = '$1'"
residual_args+=( "$1" )
}
addDebugger () {
addJava "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=$1"
}
require_arg () {
local type="$1"
local opt="$2"
local arg="$3"
if [[ -z "$arg" ]] || [[ "${arg:0:1}" == "-" ]]; then
die "$opt requires <$type> argument"
fi
}
is_function_defined() {
declare -f "$1" > /dev/null
}
# Attempt to detect if the script is running via a GUI or not
# TODO - Determine where/how we use this generically
detect_terminal_for_ui() {
[[ ! -t 0 ]] && [[ "${#residual_args}" == "0" ]] && {
echo "true"
}
# SPECIAL TEST FOR MAC
[[ "$(uname)" == "Darwin" ]] && [[ "$HOME" == "$PWD" ]] && [[ "${#residual_args}" == "0" ]] && {
echo "true"
}
}
# Processes incoming arguments and places them in appropriate global variables. called by the run method.
process_args () {
local no_more_snp_opts=0
while [[ $# -gt 0 ]]; do
case "$1" in
--) shift && no_more_snp_opts=1 && break ;;
-h|-help) usage; exit 1 ;;
-v|-verbose) verbose=1 && shift ;;
-d|-debug) debug=1 && shift ;;
-no-version-check) no_version_check=1 && shift ;;
-mem) echo "!! WARNING !! -mem option is ignored. Please use -J-Xmx and -J-Xms" && shift 2 ;;
-jvm-debug) require_arg port "$1" "$2" && addDebugger $2 && shift 2 ;;
-main) custom_mainclass="$2" && shift 2 ;;
-java-home) require_arg path "$1" "$2" && jre=`eval echo $2` && java_cmd="$jre/bin/java" && shift 2 ;;
-D*|-agentlib*|-XX*) addJava "$1" && shift ;;
-J*) addJava "${1:2}" && shift ;;
*) addResidual "$1" && shift ;;
esac
done
if [[ no_more_snp_opts ]]; then
while [[ $# -gt 0 ]]; do
addResidual "$1" && shift
done
fi
is_function_defined process_my_args && {
myargs=("${residual_args[#]}")
residual_args=()
process_my_args "${myargs[#]}"
}
}
# Actually runs the script.
run() {
# TODO - check for sane environment
# process the combined args, then reset "$#" to the residuals
process_args "$#"
set -- "${residual_args[#]}"
argumentCount=$#
#check for jline terminal fixes on cygwin
if is_cygwin; then
stty -icanon min 1 -echo > /dev/null 2>&1
addJava "-Djline.terminal=jline.UnixTerminal"
addJava "-Dsbt.cygwin=true"
fi
# check java version
if [[ ! $no_version_check ]]; then
java_version_check
fi
if [ -n "$custom_mainclass" ]; then
mainclass=("$custom_mainclass")
else
mainclass=("${app_mainclass[#]}")
fi
# Now we check to see if there are any java opts on the environment. These get listed first, with the script able to override them.
if [[ "$JAVA_OPTS" != "" ]]; then
java_opts="${JAVA_OPTS}"
fi
# run sbt
execRunner "$java_cmd" \
${java_opts[#]} \
"${java_args[#]}" \
-cp "$(fix_classpath "$app_classpath")" \
"${mainclass[#]}" \
"${app_commands[#]}" \
"${residual_args[#]}"
local exit_code=$?
if is_cygwin; then
stty icanon echo > /dev/null 2>&1
fi
exit $exit_code
}
# Loads a configuration file full of default command line options for this script.
loadConfigFile() {
cat "$1" | sed $'/^\#/d;s/\r$//'
}
# Now check to see if it's a good enough version
# TODO - Check to see if we have a configured default java version, otherwise use 1.6
java_version_check() {
readonly java_version=$("$java_cmd" -version 2>&1 | awk -F '"' '/version/ {print $2}')
if [[ "$java_version" == "" ]]; then
echo
echo No java installations was detected.
echo Please go to http://www.java.com/getjava/ and download
echo
exit 1
else
local major=$(echo "$java_version" | cut -d'.' -f1)
if [[ "$major" -eq "1" ]]; then
local major=$(echo "$java_version" | cut -d'.' -f2)
fi
if [[ "$major" -lt "6" ]]; then
echo
echo The java installation you have is not up to date
echo $app_name requires at least version 1.6+, you have
echo version $java_version
echo
echo Please go to http://www.java.com/getjava/ and download
echo a valid Java Runtime and install before running $app_name.
echo
exit 1
fi
fi
}
### ------------------------------- ###
### Start of customized settings ###
### ------------------------------- ###
usage() {
cat <<EOM
Usage: $script_name [options]
-h | -help print this message
-v | -verbose this runner is chattier
-d | -debug set sbt log level to debug
-no-version-check Don't run the java version check.
-main <classname> Define a custom main class
-jvm-debug <port> Turn on JVM debugging, open at the given port.
# java version (default: java from PATH, currently $(java -version 2>&1 | grep version))
-java-home <path> alternate JAVA_HOME
# jvm options and output control
JAVA_OPTS environment variable, if unset uses "$java_opts"
-Dkey=val pass -Dkey=val directly to the java runtime
-J-X pass option -X directly to the java runtime
(-J is stripped)
# special option
-- To stop parsing built-in commands from the rest of the command-line.
e.g.) enabling debug and sending -d as app argument
\$ ./start-script -d -- -d
In the case of duplicated or conflicting options, basically the order above
shows precedence: JAVA_OPTS lowest, command line options highest except "--".
Available main classes:
MyAppClass
EOM
}
### ------------------------------- ###
### Main script ###
### ------------------------------- ###
declare -a residual_args
declare -a java_args
declare -a app_commands
declare -r real_script_path="$(realpath "$0")"
declare -r app_home="$(realpath "$(dirname "$real_script_path")")"
# TODO - Check whether this is ok in cygwin...
declare -r lib_dir="$(realpath "${app_home}/../lib")"
declare -a app_mainclass=(MyAppClass)
declare -r script_conf_file="${app_home}/../conf/application.ini"
declare -r app_classpath="$lib_dir/my-app-0.1.0-SNAPSHOT.jar:$lib_dir/org.scala-lang.scala-library-2.12.8.jar:$lib_dir/com.thenewmotion.ocpp.ocpp-j-api_2.12-9.0.1.jar:$lib_dir/com.thenewmotion.ocpp.ocpp-messages_2.12-9.0.1.jar:$lib_dir/com.thenewmotion.enum-utils_2.12-0.2.1.jar:$lib_dir/com.thenewmotion.ocpp.ocpp-json_2.12-9.0.1.jar:$lib_dir/org.json4s.json4s-native_2.12-3.6.1.jar:$lib_dir/org.json4s.json4s-core_2.12-3.6.1.jar:$lib_dir/org.json4s.json4s-ast_2.12-3.6.1.jar:$lib_dir/org.json4s.json4s-scalap_2.12-3.6.1.jar:$lib_dir/com.thoughtworks.paranamer.paranamer-2.8.jar:$lib_dir/org.slf4j.slf4j-api-1.7.25.jar:$lib_dir/org.java-websocket.Java-WebSocket-1.3.9.jar:$lib_dir/org.apache.logging.log4j.log4j-api-2.11.2.jar:$lib_dir/org.apache.logging.log4j.log4j-core-2.11.2.jar:$lib_dir/org.apache.logging.log4j.log4j-slf4j-impl-2.11.2.jar:$lib_dir/com.typesafe.config-1.3.4.jar"
# java_cmd is overrode in process_args when -java-home is used
declare java_cmd=$(get_java_cmd)
# if configuration files exist, prepend their contents to $# so it can be processed by this runner
[[ -f "$script_conf_file" ]] && set -- $(loadConfigFile "$script_conf_file") "$#"
run "$#"
run the sbt task docker:stage. Then analyze the output created in the folder target/docker/stage.
In my case the Dockerfile contains the following:
FROM openjdk:11-jre-slim as stage0
WORKDIR /opt/docker
COPY opt /opt
USER root
RUN ["chmod", "-R", "u=rX,g=rX", "/opt/docker"]
RUN ["chmod", "u+x,g+x", "/opt/docker/bin/sample"]
FROM openjdk:11-jre-slim
LABEL MAINTAINER="your name"
USER root
RUN id -u demiourgos728 2> /dev/null || useradd --system --create-home --uid 1001 --gid 0 demiourgos728
WORKDIR /opt/docker
COPY --from=stage0 --chown=demiourgos728:root /opt/docker /opt/docker
EXPOSE 9000
USER 1001
ENTRYPOINT ["/opt/docker/bin/sample"]
CMD []
I had the problem that the PID file could not be created. I think in your case it will be something similar. There is no magic involved here.
The folder /opt/docker does not have write permissions by default. As the documentation states, you could add the following line to your build.sbt:
dockerAdditionalPermissions += (DockerChmodType.UserGroupWriteExecute, "/opt/docker")
which will add an additional line:
RUN ["chmod", "u=rwX,g=rwX", "/opt/docker"]
to the stage0 container. See nativer packager docs.
Alternatively, disable the PID file by passing a parameter to the JVM:
bashScriptExtraDefines ++= Seq( "addJava '-Dpidfile.path=/dev/null'" )
to your build.sbt. Play Production configuration Docs
I would like to list all files that bitbake will fetch when I bake an image.
Currently, I am able to get the SRC_URI of all files needed to bake a Yocto image by executing bitbake core-image-minimal -c fetchall and then parse log files.
Is there is a simpler way to get the same result without the need to download files ?
I am not sure bitbake supports such feature. Ideally, I am looking for a command that prints out the package name and lists all files with corresponding URL
> bitbake core-image-minimal -c fetchall --print-only
Generally bitbake doesn't provides such functionality.
But I was able to create a simple solution witch creating simple .bbclass file which is inherited in all recipes, by adding it into local.conf file, please see my steps in order to archive that:
Steps:
let's create a class print-src.bbclass file used to get and print SRC_URI variable (remember to store this class file in layer which is available in conf/bblayers.conf):
$ cat print-src.bbclass
python do_print_src () {
# should probably be indented
srcuri = d.getVar('SRC_URI', True).split()
bb.warn("SRC_URI look like: %s" % srcuri)
}
addtask do_print_src before do_fetch
Add INHERIT += "print-src" into Your conf/local.conf file
Edit: it is important to use bitbake --runonly option, that allows to run specific task of the taskgraph for the specified target (with --runonly option do_print_src needs to be used as print_src),
Edit: Please note that --runall=RUNALL and --runonly=RUNONLY was introduced with Yocto Sumo release 2.5,
$ bitbake core-image-minimal --runonly print_src
Loaded 1236 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
Build Configuration:
BB_VERSION = "1.37.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal-4.8"
TARGET_SYS = "i586-poky-linux"
MACHINE = "qemux86"
DISTRO = "poky"
DISTRO_VERSION = "2.5"
TUNE_FEATURES = "m32 i586"
TARGET_FPU = ""
meta
meta-poky
meta-yocto-bsp = "master:13cc30cd7de4841990b600e83e1249c81a5171dd"
Initialising tasks: 100% |##########################################################################################################################################################################| Time: 0:00:00
NOTE: Executing RunQueue Tasks
WARNING: ptest-runner-2.2+gitAUTOINC+49956f65bb-r0 do_print_src: SRC_URI look like: ['git://git.yoctoproject.org/ptest-runner2']
WARNING: grep-3.1-r0 do_print_src: SRC_URI look like: ['http://ftp.gnu.org/gnu/grep/grep-3.1.tar.xz', 'file://0001-Unset-need_charset_alias-when-building-for-musl.patch']
...
...
NOTE: Tasks Summary: Attempted 201 tasks of which 0 didn't need to be rerun and all succeeded.
Summary: There were 202 WARNING messages shown.
Please see sample warning output log line:
WARNING: ptest-runner-2.2+gitAUTOINC+49956f65bb-r0 do_print_src: SRC_URI look like: ['git://git.yoctoproject.org/ptest-runner2'].
I patched poky to create *.src files in downloads that contains the effective fetch URL of the package.
bitbake/lib/bb/fetch2/__init__.py | 19 ++++++++++++++-----
1 file changed, 14 insertions(+), 5 deletions(-)
diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index b853da30bd..03e84e0919 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
## -1257,16 +1257,16 ## class FetchData(object):
# Note: .done and .lock files should always be in DL_DIR whereas localpath may not be.
if self.localpath and self.localpath.startswith(dldir):
- basepath = self.localpath
+ self.basepath = self.localpath
elif self.localpath:
- basepath = dldir + os.sep + os.path.basename(self.localpath)
+ self.basepath = dldir + os.sep + os.path.basename(self.localpath)
elif self.basepath or self.basename:
- basepath = dldir + os.sep + (self.basepath or self.basename)
+ self.basepath = dldir + os.sep + (self.basepath or self.basename)
else:
bb.fatal("Can't determine lock path for url %s" % url)
- self.donestamp = basepath + '.done'
- self.lockfile = basepath + '.lock'
+ self.donestamp = self.basepath + '.done'
+ self.lockfile = self.basepath + '.lock'
def setup_revisions(self, d):
self.revisions = {}
## -1607,6 +1607,15 ## class Fetch(object):
m = ud.method
localpath = ""
+ p = "%s.src"%ud.basepath
+ d = os.path.dirname(p)
+ if d != '':
+ bb.utils.mkdirhier(d)
+ with open(p, 'wb') as f:
+ data = "%s" % ud.url
+ f.write(bytes(data, 'ASCII'))
+ return True
+
if ud.lockfile:
lf = bb.utils.lockfile(ud.lockfile)
Running bitbake core-image-minimal -c fetchall results:
$> find downloads/ -name '*.src' | head -n 5
downloads/lzop-1.03.tar.gz.src
downloads/libtheora-1.1.1.tar.bz2.src
downloads/mpfr-3.1.5.tar.xz.src
downloads/makedevs.c.src
downloads/expat-2.2.0.tar.bz2.src
This is not an optimal solution but I hope that such feature gets its way to mainline stream.
I needed something like this and got part way there before now.
I can generate a messy list of URIs used by executing the following command:
bitbake -g zlib && cat recipe-depends.dot | \
grep -v -e '-native' | grep -v digraph | \
grep -v -e '-image' | awk '{print $1}' | \
sort | uniq | xargs -I {} -t bitbake -e {} | grep SRC_URI=
This gives you all the URIs and files used in a recipe and some comments.
Not a perfect solution but I will see if I can improve on it.
A bit manual and straight forward way is to build your image from scratch, and copy all the fetch logs, and go thru it.
$ mkdir fetch_logs
$ find tmp/work/ -type f -name log.do_fetch* | cpio -pdm fetch_logs
Now grep "Fetch(ing/er)" in this fetch_logs and it will provide info on which URL your recipe used to fetch the source. This will be helpful to see the final URI used to fetch in PREMIRROR based builds.
NOTE: if you have sstate-cache enabled and functional, you may not see do_fetch at all.
I wrote scripts based on the solution in the question.
Basically I need to make a cscope project of all the sources, and hence I need to list all URI and clone all repos.
I wrote 2 scripts 1) List all SRC_URI and Branches
2) Install(clone) all the code in a single directory
listURI.sh
#!/bin/bash
TMP_FILE="___x__2242230p.txt"
SRCH_FILE="log.do_fetch"
TIME=$(date +%Y-%m-%d-%H-%M)
OUT_FILE="Project-$TIME.txt"
DEF_BRANCH="master"
BRANCH=""
SRC_URI=""
function usage {
echo "Usage : $0 < -f | -g > <Component-List-file>"
echo "Options :"
echo " -f : Run bitbake fetch and collect URI"
echo " -g : Get URI from Already fetched Project"
}
if [ $# -eq 0 ]
then
usage
exit
fi
if [ "$1" == "-f" ] || [ "$1" == "-g" ];then
if [ "$1" == "-f" ] && [ -z "$2" ];then
echo "Should pass Project-Name after -f Option"
exit
fi
else
echo "Unknown Option"
usage
exit
fi
POKYROOTDIR=`basename "$PWD"`
#echo $POKYROOTDIR
if [[ $POKYROOTDIR != "build" ]];then
echo "You are not on poky/build (Sourced Poky) Directory"
exit 0
fi
POKYROOTDIR=$PWD
if [ "$1" == "-f" ];then
echo "Running === bitbake -c fetchall $2 -f --no-setscene =="
bitbake -c fetchall $2 -f --no-setscene
fi
if [ "$1" == "-g" ];then
if [ ! -d tmp/work ];then
echo " Please run == bitbake -c fetchall <image> -f --no-setscene == before this"
usage
exit
fi
fi
echo "Looking for URIs --- It may take couple of minuites!"
rm -rf $OUT_FILE
cd tmp/work
find . -name $SRCH_FILE -exec grep -i 'DEBUG: For url git' {} \; -print > $POKYROOTDIR/$TMP_FILE
cd $POKYROOTDIR
while IFS= read -r line
do
#echo "$line"
BRANCH=$(echo $line | awk -F"branch=" '/branch=/{print $2}' | sed -e 's/;/ /' | awk '{print $1}')
SRC_URI=$(echo $line | cut -d';' -f1-1 | awk -F"url" '/url/{print $2}' | awk '{print $1}' | sed -e 's/git:/ssh:/')
if [ ! -z "$BRANCH" ] && [ ! -z "$SRC_URI" ];then
echo "$BRANCH $SRC_URI" >> $OUT_FILE
elif [ ! -z "$SRC_URI" ];then
echo "$DEF_BRANCH $SRC_URI" >> $OUT_FILE
fi
if [ ! -z "$BRANCH" ];then
echo "Found URI : BRANCH:$BRANCH URI:$SRC_URI"
elif [ ! -z "$SRC_URI" ];then
echo "Found URI : BRANCH:Default URI:$SRC_URI"
fi
#echo "$BRANCH $SRC_URI" >> $OUT_FILE
done < "$TMP_FILE"
#echo "=================== OUT PUT ==========================="
#cat $OUT_FILE
rm -rf $TMP_FILE
echo "----------------INFO-----------------"
echo "Project Creation: Success"
echo "Project Path : $POKYROOTDIR"
echo "Project file : $OUT_FILE"
echo "-------------------------------------"
installURI.sh
#!/bin/bash
function usage {
echo "Usage : $0 <List-file>"
}
if [ $# -eq 0 ]
then
usage
exit
fi
if [ -z "$1" ];then
usage
exit
fi
if [ ! -z "$(ls -A $PWD)" ]; then
echo "Directory ($PWD) must be Empty.... else it will mess up project creation"
exit
fi
if [ ! -f $1 ];then
echo "list file ($1) not found"
usage
exit
fi
filename=$1
while read -r line; do
name="$line"
echo "Fetching Projects"
git clone -b $line
done < "$filename"
echo "----------------INFO-----------------"
echo "Project Creation: Success"
echo "Project Path : $PWD"
echo "-------------------------------------"
I have a very large project where we have made changes to lots of files. Now we need to exactly know which line numbers we have changed/added so that we can run some other tool on that particular line only.
While searching for the answer i found this as an answer to some other question..
echo ${f}:
for n in $(git --no-pager blame --line-porcelain $1 |
awk '/author Not Committed Yet/{if (a && a !~ /author Not Committed Yet/) print a} {a=$0}' |
awk '{print $3}') ; do
if (( prev_line > -1 )) ; then
if (( "$n" > (prev_line + 1) )) ; then
if (( (prev_line - range_start) > 1 )) ; then
echo -n "$range_start-$prev_line,"
else
echo -n "$range_start,$prev_line,"
fi
range_start=$n
fi
else
range_start=$n
fi
prev_line=$n
done
if (( "$range_start" != "$prev_line" )) ; then
echo "$range_start-$prev_line"
else
echo "$range_start"
fi
And it ends up looking like this:
views.py:
403,404,533-538,546-548,550-552,554-559,565-567,580-582
It does great.. i need the exact same output using cvs.. Is there any way to get similar output..
I need to execute a script on a remote server over ssh, I can't locate the script as a file on the remote server nor create files during the script process.
The script checks for a non existent or zero byte file, and if exists, checks if is outdated.
I've followed a thread here on SO and tried this:
myvar=$(ssh user#server <<EOF
myfile=/mnt/file.csv
if [ -s $myfile ]; then
filedate=$(stat -c %x $myfile|grep '[0-9\-]*' --max-count=1 -o);
yesterday=$(date --date 'now -1 day' --iso-8601);
if [ $filedate < $yesterday ]; then
echo '1 '$yesterday;
else
echo '0 ok';
fi
else
echo $(date --iso-8601);
fi
EOF
)
sadly, the pipe appears to be truncating the string or something, because the script returns
stat: too few arguments
maybe just cannot use "myfile" var declaration. Any suggestions?
Thanks in advance.
---- Edit: Clarifying answer:
keber-laptop:~ keberflores$ echo $myvar
keber-laptop:~ keberflores$ myvar=$(ssh user#server <<EOF
> myfile=/mnt/file.csv
> if [ -s \$myfile ]; then
> filedate=\$(stat -c %x \$myfile|grep '[0-9\-]*' --max-count=1 -o);
> yesterday=\$(date --date 'now -1 day' --iso-8601);
> if [ \$filedate < \$yesterday ]; then
> echo '1 '\$yesterday;
> else
> echo '0 ok';
> fi
> else
> echo '1 '\$(date --iso-8601);
> fi
> EOF
> )
Pseudo-terminal will not be allocated because stdin is not a terminal.
user#server's password:
keber-laptop:~ keberflores$ echo $?
0
keber-laptop:~ keberflores$ echo $myvar
1 2011-10-22
---- Edit: calling inside perl:
my $myvar = qx'ssh user#server <<\'EOF\'
myfile=/mnt/file.csv
if [ -s $myfile ]; then
filedate=$(stat -c %x $myfile|grep \'[0-9\-]*\' --max-count=1 -o);
yesterday=$(date --date \'now -1 day\' --iso-8601);
if [ $filedate < $yesterday ]; then
echo \'1 \'$yesterday;
else
echo \'0 ok\';
fi
else
echo \'1 \'$(date --iso-8601);
fi
EOF
';
print $myvar;
You need more escaping to get this to work. bash is going to evaluate the variables, etc in the here-file locally, then the result of that will be evaluated on the remote server.
In particular, your $myfile is probably evaluating to an empty string when the here-file is being evaluated, causing the stat to not have a file argument.
I have a directory with full svn backups named like this:
name1.20100412.r9.bz2
name1.20100413.r10.bz2
name2.20100411.r101.bz2
name3.20100412.r102.bz2
...
I need to check if a backup file exists using name and revision number only. I tried test but it didn't work:
if [ -e name1.*.r9.bz2 ]; then echo exists; fi
[: too many arguments
How can I test if file exists?
You could do:
shopt -s nullglob
set -- name1.*.r9.bz2
if [ -n "$1" ]; then echo exists; fi
or
pattern="name1.*.r9.bz2"
if [ "$(echo $pattern)" != "$pattern" ]; then echo exists; fi
Edit: Added shopt -s nullglob.
Just check if the shell has expanded the pattern. This works in sh and bash:
pattern="name1.*.r9.bz2"
if [ "$(echo $pattern)" != "$pattern" ]; then
echo exists
fi
P.S. If the pattern itself can be a valid filename, you can add a -e check:
if [ -e "$pattern" -o "$(echo $pattern)" != "$pattern" ]; then
echo exists
fi
A slight variation on eugene y's answer:
test "$(echo name1.*.r9.bz2)" != "name1.*.r9.bz2" && echo exists
I would use:
if ls name1.*.r9.bz2 &> /dev/null ; then
echo exists
fi
Try something like
FILES=(`ls name1.*.r9.bz2`)
if [ ${#FILES[#]} -gt 0 ]; then
echo "Files!"
else
echo "No Files"
fi
Related post:
Check if a file exists with wildcard in shell script
It seems to have a good answer, using the return value of ls.