I have tried to submit the script below to HPC
#!/bin/bash
#PBS -N bwa_mem_tumor
#PBS -q batch
#PBS -l walltime=02:00:00
#PBS -l nodes=2:ppn=2
#PBS -j oe
sample=x
ref=absolute/path/GRCh38.p13.genome.fa
fwd=absolutepath/forward_read.fq.gz
rev=absolutepath/reverse_read.fq.gz
module load bio/samtools/1.9
bwa mem $ref $fwd $rev > $sample.tumor.sam && samtools view -S $sample.tumor.sam -b > $sample.tumor.bam && samtools sort $sample.tumor.bam > $sample.tumor.sorted.bam
However as an output I can get only the $sample.tumor.sam and log file says that
Lmod has detected the following error: The following module(s) are unknown:
"bio/samtools/1.9"
Please check the spelling or version number. Also try "module spider ..."
It is also possible your cache file is out-of-date; it may help to try:
$ module --ignore-cache load "bio/samtools/1.9"
Also make sure that all modulefiles written in TCL start with the string
#%Module
However when I input modeles avail it shows that bio/samtools/1.9 is on the list.
Also when i use the option module --ignore-cache load "bio/samtools/1.9"
the result is the same
If i try to continue working with the sam file and input manually the command line
samtools view -b RS0107.tumor.sam > RS0107.tumor.bam
it shows
[W::sam_read1] Parse error at line 200943
[main_samview] truncated file.
What's possibly wrong with the samtools module ir we with the script?
I once saw (can't remember where) a lint check in a file that looked something like:
"If this file is modified, I will complain if files /somehwere/a and /somehwere/b aren't modified in the same commit".
Do you know of any linter capable of handling something like this? Maybe not a linter but something that can be integrated to a github repo?
If you have a CI system set up for your repository, you can run a command like the following, which will exit nonzero if there's a problem commit, and zero if all commits are fine (with $BASE and $HEAD set appropriately):
git rev-list $BASE..$HEAD | \
xargs -L1 sh -c 'lines=$(git show --name-only $0 | grep -e somewhere/a -e somewhere/b | wc -l); \
test $lines -ne 1 || { echo "bad commit $0"; false; }'
Any suitable CI system can test this by running a shell script and failing if it exits nonzero.
I want to download a list of webpages from a file. How can I stop Wget appending the query strings on to the saved files?
wget http://www.example.com/index.html?querystring
I need this to be downloaded as index.html, not index.html?querystring
There is the -O option:
wget -O file.html http://www.example.com/index.html?querystring
so you can alter a little bit your script to pass to the -O argument the right file name.
I've finally resigned to using the -O and just wrapped it in a bash function to make it easier. I put this in my ~/.bashrc file:
wget-rmq ()
{
[ -z "$1" ] && echo 'error: wget-rmq requires a URL to retrieve as the first arg'
local output_filename="$(echo $1 | sed 's/?.*//g' | sed 's|https.*/||g')"
wget -O "${output_filename}" "${1}"
}
Then when I want to download a file:
wget-rmq http://www.example.com/index.html?querystring
The replacement regex is fairly simple. If any ?s appear in the URL before the query string begins then it will break. In practice that hasn't happened though since URL encoding requires ? to be in URLs as %3F, but I wanted to note the possibility.
I'd need to see what has been changed between two directories which contain different version of a software sourcecode. While I have found a way to get a unique .diff file, how can I obtain a different file for each changed file in the two directories? I'd need this, as the "main" is about 6 MB and wanted some more handy thing.
I came around this problem too, so I ended up with some lines of a shell script. It takes three arguments: Source and destination directory (as used for diff) and a target folder (should exist) for the output.
It's a bit hacky, but maybe it would be useful for someone. So use with care, especially if your paths have special characters.
#!/bin/sh
DIFFARGS="-wb"
LANG=C
TARGET=$3
SRC=`echo $1 | sed -e 's/\//\\\\\\//g'`
DST=`echo $2 | sed -e 's/\//\\\\\\//g'`
if [ ! -d "$TARGET" ]; then
echo "'$TARGET' is not a directory." >&2
exit 1
fi
diff -rqN $DIFFARGS "$1" "$2" | sed "s/Files $SRC\/\(.*\?\) and $DST\/\(.*\?\) differ/\1/" | \
while read file
do
if [ ! -d "$TARGET/`dirname \"$file\"`" ]; then
mkdir -p "$TARGET/`dirname \"$file\"`"
fi
diff $DIFFARGS -N "$1/$file" "$2/$file" > "$TARGET"/"$file.diff"
done
if you want to compare source code it is better to commit it to a source vesioning program as "svn".
after you have done so. do a diff of your uploaded code and pipe it to file.diff
svn diff --old svn:url1 --new svn:url2 > file.diff
A bash for loop will work for you. The following will diff two directories with C source code and produce a separate diff for each file.
for FILE in $(find <FIRST_DIR> -name '*.[ch]'); do DIFF=<DIFF_DIR>/$(echo $FILE | grep -o '[-_a-zA-Z0-9.]*$').diff; diff -u $FILE <SECOND_DIR>/$FILE > $DIFF; done
Use the correct patch level for the lines starting with +++
Problem: I can't find any way to reliably get the current playing file in an MPlayer playlist.
Here is how far I have gotten. This working ash script monitors a text file with the path to the current playlist. When I update the file, the script closes the old instance of MPlayer and opens a new one with the new playlist:
# POLL PLAYLIST FILE FOR CHANGES
CURRENTPLAYLISTPATH=/home/tc/currentplaylist
INFIFO=/tmp/mplayer-in
CURRENTPLAYLIST="NEVERMATCHAPLAYLIST"
FIRSTRUN=1
while [ 1 ];
do
# CHECK FOR NEW PLAYLIST
NEWPLAYLIST=$(head -n 1 $CURRENTPLAYLISTPATH)
if [[ "$NEWPLAYLIST" != "$CURRENTPLAYLIST" ]]; then
if [ "$FIRSTRUN" == 0 ]; then
echo "quit" > "$INFIFO"
fi
# CREATE NAMED PIPE, IF NEEDED
trap "rm -f $INFIFO" EXIT
if [ ! -p $INFIFO ]; then
mkfifo $INFIFO
fi
# START MPLAYER
mplayer -fixed-vo -nolirc -vc ffmpeg12vdpau,ffh264vdpau, -playlist $NEWPLAYLIST -loop 0 -geometry 1696x954 -slave -idle -input file=$INFIFO -quiet -msglevel all=0 -identify | tee -a /home/tc/mplayer.log &
CURRENTPLAYLIST=$NEWPLAYLIST
FIRSTRUN=0
fi
sleep 5;
done
My original plan was just to use the "-identify" flag and parse the log file. This actually works really well up until I need to truncate the log file to keep it from getting too large. As soon as my truncating script is run, MPlayer stops writing to the log file:
FILENAME=/home/tc/mplayer.log
MAXCOUNT=100
if [ -f "$FILENAME" ]; then
LINECOUNT=`wc -l "$FILENAME" | awk '{print $1}'`
if [ "$LINECOUNT" -gt "$MAXCOUNT" ]; then
REMOVECOUNT=`expr $LINECOUNT - $MAXCOUNT`
sed -i 1,"$REMOVECOUNT"d "$FILENAME"
fi
fi
I have searched and searched but have been unable to find any other way of getting the current playing file that works.
I have tried piping the output to another named pipe and then monitoring it, but only works for a few seconds, then MPlayer completely freezes.
I have also tried using bash (instead of ash) and piping the output to a function like the following, but get the same freezing problem:
function parseOutput()
{
while read LINE
do
echo "get_file_name" > /tmp/mplayer-in
if [[ "$LINE" == *ANS_FILENAME* ]]
then
echo ${LINE##ANS_FILENAME=} > "$CURRENTFILEPATH"
fi
sleep 1
done
}
# START MPLAYER
mplayer -fixed-vo -nolirc -vc ffmpeg12vdpau,ffh264vdpau, -playlist $NEWPLAYLIST -loop 0 -geometry 1696x954 -slave -idle -input file=/tmp/mplayer-in -quiet | parseOutput &
I suspect I am missing something very obvious here, so any help, ideas, points in the right direction would be greatly appreciated.
fodder
Alright then, so I'll post mine too.
Give this one a try (assuming there is only one instance running, like on fodder's machine):
basename "$(readlink /proc/$(pidof mplayer)/fd/* | grep -v '\(/dev/\|pipe:\|socket:\)')"
This is probably the safer way, since the file descriptors might not always be in the same order on all systems.
However, this can be shortened, with a little risk:
basename "$(readlink /proc/$(pidof mplayer)/fd/*)" | head -1
You might probably like to install this, too:
http://mplayer-tools.sourceforge.net/
Well, I gave up on getting the track from MPlayer itself.
My 'solution' is probably too hackish, but works for my needs since I know my machine will only ever have one instance of MPlayer running:
lsof -p $(pidof mplayer) | grep -o "/path/to/my/assets/.*"
If anyone has a better option I'm certainly still interested in doing this the right way, I just couldn't make any of the methods work.
fodder
You can use the run command.
Put this in ~/.mplayer/input.conf:
DEL run "echo ${filename} ${stream_pos} >> /home/knarf/out"
Now if you press the delete key while playing a file it will do what you expect i.e. append the current file playing and the position in the stream to the ~/out file. You can replace echo with your program.
See slave mod docs for more info (Ctrl-F somevar).
About getting properties from MPlayer
I have used a non-elegant solution, but it is working for me.
stdbuf -oL mplayer --slave --input=file=$FIFO awesome_awesome.mp3 |
{
while IFS= read -r line
do
if [[ "${line}" == ANS_* ]]; then
echo "${line#*=}" > ${line%=*} # echo property_value > property_name
fi
done
} &
mplayer_pid=&!
read filename < ./ANS_FILENAME
read timeLength < ./ANS_LENGTH
echo ($timeLength) $filename
and so on..
It is in another proccess, that's why I've used files to bring properties
'stdbuf' is for not to miss anything
I started putting together a bash library to handle tasks like this. Basically, you can accomplish this by dumping the mplayer output to a file. Then you grep that dump for "Playing " and take the last result with tail. This should give you the name of the file that's currently playing or that last finished playing.
Take a look at my bash code. You'll want to modify the playMediaFile function to your needs, but the getMediaFileName function should do exactly what you're asking. You'll find the code on my github.