Why is Ghostscript pswrite encoding my text in its output? Consider the following MWE:
%!PS-Adobe-3.0
%%Title: mwe.ps
%%Pages: 001
%%BoundingBox: 0 0 595 842
%%EndComments
%%Page: 1 1
%%PageBoundingBox: 0 0 595 842
0 0 1 setrgbcolor
0 0 595 842 rectfill
1 0 0 setrgbcolor
247 371 100 100 rectfill
/Times-Roman findfont
72 scalefont
setfont
newpath
247 300 moveto
(Chris) show
showpage
Saving this MWE to file and viewing in GSview will display a blue page with red square and my name underneath. Now run this file through Ghostscript 9.06 with the following command line:
"c:\Program Files\gs\gs9.06\bin\gswin64c.exe" ^
-dSAFER -dBATCH -dNOPAUSE ^
-sDEVICE=pswrite -sPAPERSIZE=a4 -r72 -sOutputFile=mwe_gs.ps mwe.ps
See Ghostscript output below. Can someone please explain what is happening here. Whilst the two rectfill commands are still apparent, my text (Chris) has been encoded and is no longer distinguishable.
Is there an alternative postscript device which would retain my text please?
<snip>
%%Page: 1 1
%%PageBoundingBox: 0 0 595 842
%%BeginPageSetup
GS_pswrite_2_0_1001 begin
595 842 /a4 setpagesize
/pagesave save store 197 dict begin
1 1 scale
%%EndPageSetup
gsave mark
255 0 r6
0 0 595 842 rf
255 0 r3
247 371 100 100 rf
Q q
0 0 595 0 0 842 ^ Y
255 0 r3
249 299 43 50 /5D
$C
,6CW56m1G"ZORNkWR*rB:!c2;9rlWTH="2^^[(q"h>cG<omZ2l^=qC[XbO:8_[?kji-8^"N#3q*
jhL~>
,
289 300 41 49 /0P
$C
4r?0p$m<EkK3,0>s8W-!s8W-!s8W,u]<1irI=*p=<t0>_#<)>Is8K6,aTi'$~>
,
325 300 30 33 /5I
$C
49S"pc4+Rhs8W-!s8W)oqdD:saRZq[4+k%):]~>
,
349 300 24 49 /0T
$C
4q%Ms%;PqCs8W-!s8W%1_qkn/K?*sYFSGd:5Q~>
,
377 299 23 34 /5M
$C
-TQR7$&O'!K+D:XribR9;$mr4#sqUi.T#,dX=Y&Llg+F`d^HC#%$"]~>
,
cleartomark end end pagesave restore
showpage
%%PageTrailer
%%Trailer
%%Pages: 1
%%EOF
NOTE: This might seem an odd activity but I'm exploring the idea of using Ghostscript to 'clean up' postscript output from Matlab application..
The 'text' has been converted to images, not vector paths. This is a serious limitation of the pswrite device, and one of the reasons it is deprecated, you should use the ps2write device instead. The only reason the pswrite device is still included at all is for epswrite which uses it (which is why the pswrite and epswrite output looks the same). At some point there will be an eps2write device and pswrite will be binned.
ps2write output is, by default, compressed. If you want uncompressed output, use the -dCompressPages=false switch on the command line.
If all you want is the location of the text you might consider the txtwrite device. The default implementation of this creates a plain text representation of the input, but you can have it output a faked up XML instead which includes things like the origin of the text.
Here is a simple example of the show operator being redefined to display position information about the show, along with performing the standard show operation. With ghostscript you can run multiple files, so the header file would be a prefix to the other file, which alters standard behavior.
The redefined show could have included font name and size. The data could have been written to a disk file, rather than dumped to the console. Any of other operator could have also been redefined, like rectfill, fill, stroke... Because the original operator is also called, you can convert a .ps to .pdf using a pdfwrite device, while at the same time obtaining position information.
gswin32c.exe -dBATCH -dNOPAUSE header.ps trash.ps
gswin32c.exe -sDEVICE=pdfwrite -dCompressPages=false -sOutputFile=test.pdf header.ps trash.ps
output
currentpoint x:247.0 y:300.0 pathbbox 249.015,298.992 400.066,349.184 text:Chris currentrgbcolor:1.0,0.0,0.0( )
currentpoint x:50.0 y:90.0 pathbbox 50.8682,89.2852 181.327,139.184 text:Fred currentrgbcolor:1.0,0.0,0.0( )
currentpoint x:150.0 y:200.0 pathbbox 150.867,184.298 304.154,247.673 text:Mary currentrgbcolor:1.0,0.0,0.0( )
currentpoint x:300.0 y:350.0 pathbbox 300.867,348.993 598.79,398.681 text:Mr. Green currentrgbcolor:0.0,1.0,0.0( )
currentpoint x:100.0 y:400.0 pathbbox 100.866,399.202 358.547,449.183 text:Mr. Blue currentrgbcolor:0.0,0.0,1.0( )
Header.ps
/mydict 5 dict def
mydict begin
/show
{
(currentpoint ) print
currentpoint exch 10 string cvs ( x:) print print 10 string cvs ( y:) print print
gsave dup false charpath flattenpath
( pathbbox ) print
pathbbox
4 -1 roll 10 string cvs print (,) print
3 -1 roll 10 string cvs print ( ) print
2 -1 roll 10 string cvs print (,) print
10 string cvs print ( ) print
grestore
( text:) 10 string cvs print
dup print ( ) print
( currentrgbcolor:) print
currentrgbcolor
3 -1 roll 10 string cvs print (,) print
2 -1 roll 10 string cvs print (,) print
10 string cvs print ( ) ==
systemdict /show get exec
} def
trash.ps
%!PS-Adobe-3.0
%%Title: mwe.ps
%%Pages: 001
%%BoundingBox: 0 0 595 842
%%EndComments
%%Page: 1 1
%%PageBoundingBox: 0 0 595 842
0 0 1 setrgbcolor
0 0 595 842 rectfill
1 0 0 setrgbcolor
247 371 100 100 rectfill
/Times-Roman findfont
72 scalefont
setfont
newpath
247 300 moveto (Chris) show
50 90 moveto (Fred) show
150 200 moveto (Mary) show
0 1 0 setrgbcolor
300 350 moveto (Mr. Green) show
0 0 1 setrgbcolor
100 400 moveto (Mr. Blue) show
showpage
The text has been converted to vector paths. 249 299 43 50 /5D begins the first letter "C", then 289 300 is the "h", 289 300 the "r"....
What pswrite has done is eliminate the need for a font, so while your original code used /Times-Roman, the distilled code doesn't need any font, but rather draws the text using vectors.
I'm not sure exactly what you are after, but you could try "ps2write" or "epswrite" as alternatives to "pswrite". pswrite is used to write to ps level 1 standard and ps2write will write ps level 2 output. Nobody requires ps level 1 anymore, so level 2 would be acceptable. The epswrite will write to encapsulated postscript (eps).
Related
I have a csv file with experiment results that goes like this:
64 4 8 1 1 2 1 ttt 62391 4055430 333 0.0001 10 161 108 288 0
64 4 8 1 1 2 1 ttt 60966 3962810 322 0.0001 10 164 112 295 0
64 4 8 1 1 2 1 ttt 61530 3999475 325 0.0001 10 162 112 291 0
64 4 8 1 1 2 1 ttt 61430 4054428 332 0.0001 10 158 110 286 0
64 4 8 1 1 2 1 ttt 63891 4152938 339 0.0001 9 149 109 274 0
64 4 32 1 1 2 1 ttt 63699 4204182 345 0.0001 4 43 179 240 0
64 4 32 1 1 2 1 ttt 63326 4116218 336 0.0001 4 45 183 248 0
64 4 32 1 1 2 1 ttt 62654 4135211 340 0.0001 4 48 178 248 0
64 4 32 1 1 2 1 ttt 63192 4107506 339 0.0001 4 49 175 245 0
64 4 32 1 1 2 1 ttt 62707 4138666 345 0.0001 4 46 179 245 0
64 4 64 1 1 2 1 ttt 60968 3962929 323 0.0001 4 46 191 256 0
64 4 64 1 1 2 1 ttt 58765 3819787 305 0.0001 4 50 196 267 0
64 4 64 1 1 2 1 ttt 58946 3831499 308 0.0001 5 52 187 260 0
64 4 64 1 1 2 1 ttt 60646 3942047 321 0.0001 4 47 187 254 0
64 4 64 1 1 2 1 ttt 59723 3882044 311 0.0001 4 46 201 269 0
64 8 8 1 1 2 1 ttt 63414 4185382 382 0.0001 33 517 109 643 0
64 8 8 1 1 2 1 ttt 62429 4057899 372 0.0001 33 538 110 667 0
64 8 8 1 1 2 1 ttt 60622 3940452 384 0.0001 33 556 115 689 0
64 8 8 1 1 2 1 ttt 64433 4188192 369 0.0001 33 519 110 644 0
My goal is to be able to plot various combinations (choose which, in different charts) of the columns before the "ttt", with the average and standard deviation of the columns (choose which) after "ttt" (by grouping them by the before "ttt" columns).
Is this possible in GNUPlot and if yes how? If not, do you have any alternate suggestions regarding my problem?
Here is a completely revised and more general version.
Since you want to filter by 3 columns you need to have 3 properties to distinguish the data in the plot. This would be for example color, x-position and pointtype. What the script basically does:
Generates random data for testing (take your file instead)
$Data looks like this:
8 64 57773 0
4 32 64721 2
8 32 56757 1
4 16 56226 2
8 8 56055 1
8 64 59874 0
8 32 58733 0
4 16 55525 2
8 32 58869 0
8 64 64470 0
4 32 60930 1
8 64 57073 2
...
the variables ColX, ColC, ColP, and ColS define which columns are taken for x-position, color, pointtype and statistics.
find unique values of ColX, ColC, ColP, (check help smooth frequency) and put them to datablocks $ColX, $ColC, and $ColP.
put the unique values to arrays ArrX, ArrC, ArrP
loop all possible combinations and do statistics on ColS and put it to $Data2. Add 3 columns at the beginning for color, x-position and pointtype.
$Data2 looks like this:
1 1 1 0 8 4 61639.4 2788.4
1 1 2 0 8 8 59282.1 2740.2
1 2 1 0 16 4 59372.3 2808.6
1 2 2 0 16 8 60502.3 2825.0
1 3 1 0 32 4 59850.7 2603.8
1 3 2 0 32 8 60617.7 1979.8
1 4 1 0 64 4 60399.4 3273.6
1 4 2 0 64 8 59930.7 2919.8
2 1 1 1 8 4 59172.6 2288.2
2 1 2 1 8 8 58992.2 2888.0
2 2 1 1 16 4 59350.1 2364.6
2 2 2 1 16 8 61034.0 2368.5
2 3 1 1 32 4 59920.8 2867.6
2 3 2 1 32 8 59711.9 3464.2
2 4 1 1 64 4 60936.7 3439.7
2 4 2 1 64 8 61078.7 2349.3
3 1 1 2 8 4 58976.0 2376.3
3 1 2 2 8 8 61731.5 1635.7
3 2 1 2 16 4 58276.0 2101.7
3 2 2 2 16 8 58594.5 3358.5
3 3 1 2 32 4 60471.5 3737.6
3 3 2 2 32 8 59909.1 2024.0
3 4 1 2 64 4 62044.2 1446.7
3 4 2 2 64 8 60454.0 3215.1
Finally, plot the data. I couldn't figure out how plotting style with yerror works properly together with variable pointtypes. So, instead I split it into two plot commands with vectors and with points. The third one keyentry is just to get an empty line in the legend and the forth one is to get the pointtype into the legend.
I hope you can figure out all the other details and adapt it to your data.
Code:
### grouped statistics on filtered (unsorted) data
reset session
set colorsequence classic
# generate some random test data
rand1(n) = 2**(int(rand(0)*2)+2) # values 4,8
rand2(n) = 2**(int(rand(0)*4)+3) # values 8,16,32,64
rand3(n) = int(rand(0)*10000)+55000 # values 55000 to 65000
rand4(n) = int(rand(0)*3) # values 0,1,2
set print $Data
do for [i=1:200] {
print sprintf("% 3d% 4d% 7d% 3d", rand1(0), rand2(0), rand3(0), rand4(0))
}
set print
print $Data # (just for test purpose)
ColX = 2 # column for x
ColC = 4 # column for color
ColP = 1 # column for pointtype
ColS = 3 # column for statistics
# get unique values of the columns
set table $ColX
plot $Data u (column(ColX)) smooth freq
unset table
set table $ColC
plot $Data u (column(ColC)) smooth freq
unset table
set table $ColP
plot $Data u (column(ColP)) smooth freq
unset table
# put unique values into arrays
set table $Dummy
array ArrX[|$ColX|-6] # gnuplot creates 6 extra lines
array ArrC[|$ColC|-6]
array ArrP[|$ColP|-6]
plot $ColX u (ArrX[$0+1]=$1)
plot $ColC u (ArrC[$0+1]=$1)
plot $ColP u (ArrP[$0+1]=$1)
unset table
print ArrX, ArrC, ArrP # just for test purpose
# define filter function
Filter(c,x,p) = ArrX[x]==column(ColX) && ArrC[c]==column(ColC) && \
ArrP[p]==column(ColP) ? column(ColS) : NaN
# loop all values and do statistics, write data into $Data2
set print $Data2
do for [c=1:|ArrC|] {
do for [x=1:|ArrX|] {
do for [p=1:|ArrP|] {
undef var STATS*
stats $Data u (Filter(c,x,p)) nooutput
if (exists('STATS_mean') && exists('STATS_stddev')) {
print sprintf("% 3d% 3d% 3d% 3d% 3d% 3d% 9.1f % 7.1f", c, x, p, ArrC[c], ArrX[x], ArrP[p], STATS_mean, STATS_stddev)
}
}
}
print ""; print ""
}
set print
# print $Data2 # just for testing purpose
set xlabel sprintf("Column %d", ColX)
set ylabel sprintf("Column %d", ColS)
set xrange[0.5:|ArrX|+1]
set xtics () # remove all xtics
do for [x=1:|ArrX|] { set xtics add (sprintf("%d",ArrX[x]) x)} # set xtics "manually"
# function for x position and offsets,
# actually not dependent on 'n' but to shorten plot command
# columns in $Data2: 1=color, 2=x, 3=pointtype
width = 0.5 # float number!
step = width/(|ArrC|-1)
PosX(n) = column(2) - width/2.0 + step*(column(1)-1) + (column(3)-1)*step*0.3
plot \
for [c=1:|ArrC|] $Data2 u (PosX(0)):($7-$8):(0):(2*$8) index c-1 w vectors \
heads size 0.04,90 lw 2 lc c ti sprintf("%g",ArrC[c]),\
for [c=1:|ArrC|] '' u (PosX(0)):7:($3*2+4):(c) index c-1 w p ps 1.5 pt var lc var not, \
keyentry w p ps 0 ti "\n", \
for [p=1:|ArrP|] '' u (0):(NaN) w p pt p*2+4 ps 1.5 lc rgb "black" ti sprintf("%g",ArrP[p])
### end of code
Result:
I do not think gnuplot can produce exactly what you are asking for in a single plot command. I will show you two alternatives in the hope that one or both is a useful starting point.
Alternative 1: standard boxplot
spacing = 1.0
width = 0.25
unset key
set xlabel "Column 3"
set ylabel "Column 9"
plot 'data' using (spacing):9:(width):3 with boxplot lw 2
This collects points based on the value in column 3 and for each such value it produces a boxplot. This is a widely used method of showing the distribution of point values in a category, but it is a quartile analysis not a display of mean + standard deviation.
Alternative 2: calculate mean and standard deviation for categories known in advance
The boxplot analysis has the advantage that you do not need to know in advance what values may be present in column 3. Gnuplot can calculate mean and standard deviation based on a column 3 value, but you need to specify in advance what that value is. Here is a set of commands tailored to the specific example file you provided. It calculates, but does not plot, the requested categorical mean and standard deviation. You can use these numbers to construct a plot, but that will require additional commands. You could, for example, save the values for each category in a new file, or array, or datablock and then go back and plot these together.
col3entry = "8 32 64"
do for [i in col3entry] {
stats "data" using ($3 == real(i) ? $9 : NaN) name "Condition".i nooutput
print i, ": ", value("Condition".i."_mean"), value("Condition".i."_stddev")
}
output:
8: 62345.1111111111 1259.34784220021
32: 63115.6 392.552977316438
64: 59809.6 881.583711283279
Ive been looking at implementing GLCM within MATLAB using graycomatrix. There are two arguments that I have discovered (NumLevels and GrayLimits) but in in my research and implementation they seem to achieve the same result.
GrayLimits specified bins between a range set [low high], causing a restricted set of gray levels.
NumLevels declares the number of gray levels in an image.
Could someone please explain the difference between these two arguments, as I don't understand why there would be two arguments that achieve the same result.
From the documentation:
'GrayLimits': Range used scaling input image into gray levels, specified as a two-element vector [low high]. If N is the number of gray levels (see parameter 'NumLevels') to use for scaling, the range [low high] is divided into N equal width bins and values in a bin get mapped to a single gray level.
'NumLevels': Number of gray levels, specified as an integer.
Thus the first parameter sets the input gray level range to be used (defaults to the min and max values in the image), and the second parameter sets the number of unique gray levels considered (and thus the size of the output matrix, defaults to 8, or 2 for binary images).
For example:
>> graycomatrix(img,'NumLevels',8,'GrayLimits',[0,255])
ans =
17687 1587 81 31 7 0 0 0
1498 7347 1566 399 105 8 0 0
62 1690 3891 1546 298 38 1 0
12 335 1645 4388 1320 145 4 0
2 76 305 1349 4894 959 18 0
0 16 40 135 965 7567 415 0
0 0 0 2 15 421 2410 0
0 0 0 0 0 0 0 0
>> graycomatrix(img,'NumLevels',8,'GrayLimits',[0,127])
ans =
1 9 0 0 0 0 0 0
7 17670 1431 156 50 31 23 15
1 1369 3765 970 350 142 84 92
0 128 1037 1575 750 324 169 167
0 46 361 836 1218 747 335 260
0 16 163 330 772 1154 741 547
0 10 74 150 370 787 1353 1208
0 4 67 136 294 539 1247 21199
>> graycomatrix(img,'NumLevels',4,'GrayLimits',[0,255])
ans =
28119 2077 120 0
2099 11470 1801 5
94 1829 14385 433
0 2 436 2410
As you can see, these parameters modify the output in different ways:
In the first case above, the range [0,255] was mapped to columns/rows 1-8, putting 32 different input grey values into each.
In the second case, the smaller range [0,127] was mapped to 8 indices, putting 16 different input grey values into each, and putting the remaining grey values 128-255 into the 8th index.
In the third case, the range [0,255] was mapped to 4 indices, putting 64 different input grey values into each.
I want to first apologize for the biological nature of this post. I thought I should post some background first. I have a set of gene files that contain anywhere from one to five DNA sequences from different species. I used a bash shell script to perform blastn with each gene file as a query and a file of all transcriptome sequences (all_transcriptome_seq.fasta) from the five species as the subject. I now want to process these output files (and there are many) so that I can get all subject sequences that hit into one file per gene, with duplicate sequences removed (except to keep one), and ensure I'm getting the length of the sequences that actually hit the query.
Here is what the blastn output looks like for one gene file (columns: qseqid qlen sseqid slen qframe qstart qend sframe sstart send evalue bitscore pident nident length)
Acur_01000750.1_OFAS014956-RA-EXON04 248 Apil_comp17195_c0_seq1 1184 1 1 248 1 824 1072 2e-73 259 85.60 214 250
Acur_01000750.1_OFAS014956-RA-EXON04 248 Atri_comp5613_c0_seq1 1067 1 2 248 1 344 96 8e-97 337 91.16 227 249
Acur_01000750.1_OFAS014956-RA-EXON04 248 Acur_01000750.1 992 1 1 248 1 655 902 1e-133 459 100.00 248 248
Acur_01000750.1_OFAS014956-RA-EXON04 248 Btri_comp17734_c0_seq1 1001 1 1 248 1 656 905 5e-69 244 84.40 211 250
Btri_comp17734_c0_seq1_OFAS014956-RA-EXON04 250 Atri_comp5613_c0_seq1 1067 1 2 250 1 344 96 1e-60 217 82.33 205 249
Btri_comp17734_c0_seq1_OFAS014956-RA-EXON04 250 Acur_01000750.1 992 1 1 250 1 655 902 5e-69 244 84.40 211 250
Btri_comp17734_c0_seq1_OFAS014956-RA-EXON04 250 Btri_comp17734_c0_seq1 1001 1 1 250 1 656 905 1e-134 462 100.00 250 250
I've been working on a perl script that would, in short, take the sseqid column to pull out the corresponding sequences from the all_transcriptome_seq.fasta file, place these into a new file, and trim the transcripts to the sstart and send positions. Here is the script, so far:
#!/usr/bin/env perl
use warnings;
use strict;
use Data::Dumper;
############################################################################
# blastn_post-processing.pl v. 1.0 by Michael F., XXXXXX
############################################################################
my($progname) = $0;
############################################################################
# Initialize variables
############################################################################
my($jter);
my($com);
my($t1);
if ( #ARGV != 2 ) {
print "Usage:\n \$ $progname <infile> <transcriptomes>\n";
print " infile = tab-delimited blastn text file\n";
print " transcriptomes = fasta file of all transcriptomes\n";
print "exiting...\n";
exit;
}
my($infile)=$ARGV[0];
my($transcriptomes)=$ARGV[1];
############################################################################
# Read the input file
############################################################################
print "Reading the input file... ";
open (my $INF, $infile) or die "Unable to open file";
my #data = <$INF>;
print #data;
close($INF) or die "Could not close file $infile.\n";
my($nlines) = $#data + 1;
my($inlines) = $nlines - 1;
print "$nlines blastn hits read\n\n";
############################################################################
# Extract hits and place sequences into new file
############################################################################
my #temparray;
my #templine;
my($seqfname);
open ($INF, $infile) or die "Could not open file $infile for input.\n";
#temparray = <$INF>;
close($INF) or die "Could not close file $infile.\n";
$t1 = $#temparray + 1;
print "$infile\t$t1\n";
$seqfname = "$infile" . ".fasta";
if ( -e $seqfname ) {
print " --> $seqfname exists. overwriting\n";
unlink($seqfname);
}
# iterate through the individual hits
for ($jter=0; $jter<$t1; $jter++) {
(#templine) = split(/\s+/, $temparray[$jter]);
$com = "./extract_from_genome2 $transcriptomes $templine[2] $templine[8] $templine[9] $templine[2]";
# print "$com\n";
system("$com");
system("cat temp.3 >> $seqfname");
} # end for ($jter=0; $jter<$t1...
# Arguments for "extract_from_genome2"
# // argv[1] = name of genome file
# // argv[2] = gi number for contig
# // argv[3] = start of subsequence
# // argv[4] = end of subsequence
# // argv[5] = name of output sequence
Using this script, here is the output I'm getting:
>Apil_comp17195_c0_seq1
GATTCTTGCATCTGCAGTAAGACCAGAAATGCTCATTCCTATATGGCTATCTAATGGTATTATTTTTTTCTGATGTGCTGATAATTCAGACGAAGCTCTTTTAAGAGCCACAAGAACTGCATACTGCTTGTTTTTTACTCCAACAGTAGCAGCTCCCAGTTTTACAGCTTCCATTGCATATTCGACTTGGTGCAGGCGTCCCTGGGGACTCCAGACGGTAACGTCAGAATCATACTGGTTACGGAACA
>Atri_comp5613_c0_seq1
GAGAATTCTAGCATCAGCAGTGAGGCCTGAAATACTCATGCCTATGTGACTATCTAGAGGTATTATTTTTTTTTGATGAGCTGACAGTTCAGAAGAAGCTCTTTTGAGAGCTACAAGAACTGCATACTGTTTATTTTTTACTCCAACTGTTGCTGCTCCAAGCTTTACAGCCTCCATTGCATATTCCACTTGGTGTAAACGCCCCTGAGGACTCCATACCGTAACATCAGAATCATACTGATTACGGA
>Acur_01000750.1
GAATTCTAGCGTCAGCAGTGAGTCCTGAAATACTCATCCCTATGTGGCTATCTAGAGGTATTATTTTTTCTGATGGGCCGACAGTTCAGAGGATGCTCTTTTAAGAGCCACAAGAACTGCATACTCTTTATTTTTACTCCAACAGTAGCAGCTCCAAGCTTCACAGCCTCCATTGCATATTCCACCTGGTGTAAACGTCCCTGAGGGCTCCATACCGTAACATCAGAATCATACTGGTTACGGAACA
>Btri_comp17734_c0_seq1
GAATCCTTGCATCTGCAGTAAGTCCAGAAATGCTCATTCCAATATGGCTATCTAATGGTATTATTTTTTTCTGGTGAGCAGACAATTCAGATGATGCTCTTTTAAGAGCTACCAGTACTGCAAAATCATTGTTCTTCACTCCAACAGTTGCAGCACCTAATTTGACTGCCTCCATTGCATACTCCACTTGGTGCAATCTTCCCTGAGGGCTCCATACCGTAACATCAGAATCATACTGGTTACGGAACA
>Atri_comp5613_c0_seq1
GAGAATTCTAGCATCAGCAGTGAGGCCTGAAATACTCATGCCTATGTGACTATCTAGAGGTATTATTTTTTTTTGATGAGCTGACAGTTCAGAAGAAGCTCTTTTGAGAGCTACAAGAACTGCATACTGTTTATTTTTTACTCCAACTGTTGCTGCTCCAAGCTTTACAGCCTCCATTGCATATTCCACTTGGTGTAAACGCCCCTGAGGACTCCATACCGTAACATCAGAATCATACTGATTACGGA
>Acur_01000750.1
GAATTCTAGCGTCAGCAGTGAGTCCTGAAATACTCATCCCTATGTGGCTATCTAGAGGTATTATTTTTTCTGATGGGCCGACAGTTCAGAGGATGCTCTTTTAAGAGCCACAAGAACTGCATACTCTTTATTTTTACTCCAACAGTAGCAGCTCCAAGCTTCACAGCCTCCATTGCATATTCCACCTGGTGTAAACGTCCCTGAGGGCTCCATACCGTAACATCAGAATCATACTGGTTACGGAACA
>Btri_comp17734_c0_seq1
GAATCCTTGCATCTGCAGTAAGTCCAGAAATGCTCATTCCAATATGGCTATCTAATGGTATTATTTTTTTCTGGTGAGCAGACAATTCAGATGATGCTCTTTTAAGAGCTACCAGTACTGCAAAATCATTGTTCTTCACTCCAACAGTTGCAGCACCTAATTTGACTGCCTCCATTGCATACTCCACTTGGTGCAATCTTCCCTGAGGGCTCCATACCGTAACATCAGAATCATACTGGTTACGGAACA
As you can see, it's pretty close to what I'm wanting. Here are the two issues I have and cannot seem to figure out how to resolve with my script. The first is that a sequence may occur more than once in the sseqid column, and with the script in its current form, it will print out duplicates of these sequences. I only need one. How can I modify my script to not duplicate sequences (i.e., how do I only retain one but remove the other duplicates)? Expected output:
>Apil_comp17195_c0_seq1
GATTCTTGCATCTGCAGTAAGACCAGAAATGCTCATTCCTATATGGCTATCTAATGGTATTATTTTTTTCTGATGTGCTGATAATTCAGACGAAGCTCTTTTAAGAGCCACAAGAACTGCATACTGCTTGTTTTTTACTCCAACAGTAGCAGCTCCCAGTTTTACAGCTTCCATTGCATATTCGACTTGGTGCAGGCGTCCCTGGGGACTCCAGACGGTAACGTCAGAATCATACTGGTTACGGAACA
>Atri_comp5613_c0_seq1
GAGAATTCTAGCATCAGCAGTGAGGCCTGAAATACTCATGCCTATGTGACTATCTAGAGGTATTATTTTTTTTTGATGAGCTGACAGTTCAGAAGAAGCTCTTTTGAGAGCTACAAGAACTGCATACTGTTTATTTTTTACTCCAACTGTTGCTGCTCCAAGCTTTACAGCCTCCATTGCATATTCCACTTGGTGTAAACGCCCCTGAGGACTCCATACCGTAACATCAGAATCATACTGATTACGGA
>Acur_01000750.1
GAATTCTAGCGTCAGCAGTGAGTCCTGAAATACTCATCCCTATGTGGCTATCTAGAGGTATTATTTTTTCTGATGGGCCGACAGTTCAGAGGATGCTCTTTTAAGAGCCACAAGAACTGCATACTCTTTATTTTTACTCCAACAGTAGCAGCTCCAAGCTTCACAGCCTCCATTGCATATTCCACCTGGTGTAAACGTCCCTGAGGGCTCCATACCGTAACATCAGAATCATACTGGTTACGGAACA
>Btri_comp17734_c0_seq1
GAATCCTTGCATCTGCAGTAAGTCCAGAAATGCTCATTCCAATATGGCTATCTAATGGTATTATTTTTTTCTGGTGAGCAGACAATTCAGATGATGCTCTTTTAAGAGCTACCAGTACTGCAAAATCATTGTTCTTCACTCCAACAGTTGCAGCACCTAATTTGACTGCCTCCATTGCATACTCCACTTGGTGCAATCTTCCCTGAGGGCTCCATACCGTAACATCAGAATCATACTGGTTACGGAACA
The second is the script is not quite extracting the right base pairs. It's super close, off by one or two, but its not exact.
For example, take the first subject hit Apil_comp17195_c0_seq1. The sstart and send values are 824 and 1072, respectively. When I go to the all_transcriptome_seq.fasta, I get
AAGATTCTTGCATCTGCAGTAAGACCAGAAATGCTCATTCCTATATGGCTATCTAATGGTATTATTTTTTTCTGATGTGCTGATAATTCAGACGAAGCTCTTTTAAGAGCCACAAGAACTGCATACTGCTTGTTTTTTACTCCAACAGTAGCAGCTCCCAGTTTTACAGCTTCCATTGCATATTCGACTTGGTGCAGGCGTCCCTGGGGACTCCAGACGGTAACGTCAGAATCATACTGGTTACGGAAC
at that base pair range, not
GATTCTTGCATCTGCAGTAAGACCAGAAATGCTCATTCCTATATGGCTATCTAATGGTATTATTTTTTTCTGATGTGCTGATAATTCAGACGAAGCTCTTTTAAGAGCCACAAGAACTGCATACTGCTTGTTTTTTACTCCAACAGTAGCAGCTCCCAGTTTTACAGCTTCCATTGCATATTCGACTTGGTGCAGGCGTCCCTGGGGACTCCAGACGGTAACGTCAGAATCATACTGGTTACGGAACA
as outputted by my script, which is what I'm expecting. You will also notice that the sequence outputted by my script is slightly shorter than it should be. Does anyone know how I can fix these issues in my script?
Thanks, and sorry for the lengthy post!
Edit 1: a solution was offered that work for some of the infiles. However, some were causing the script to output fewer sequences than expected. Here is one such infile with 9 hits, from which I was expecting only 4 sequences.
Note: this issue has been largely resolved based on the solution provided below the answer section
Apil_comp16418_c0_seq1_OFAS000119-RA-EXON01 1587 Apil_comp16418_c0_seq1 2079 1 1 1587 1 416 2002 0.0 2931 100.00 1587 1587
Apil_comp16418_c0_seq1_OFAS000119-RA-EXON01 1587 Atri_comp13712_c0_seq1 1938 1 1 1587 1 1651 75 0.0 1221 80.73 1286 1593
Apil_comp16418_c0_seq1_OFAS000119-RA-EXON01 1587 Ctom_01003023.1 2162 1 1 1406 1 1403 1 0.0 1430 85.07 1197 1407
Atri_comp13712_c0_seq1_OFAS000119-RA-EXON01 1441 Apil_comp16418_c0_seq1 2079 1 1 1437 1 1866 430 0.0 1170 81.43 1175 1443
Atri_comp13712_c0_seq1_OFAS000119-RA-EXON01 1441 Atri_comp13712_c0_seq1 1938 1 1 1441 1 201 1641 0.0 2662 100.00 1441 1441
Atri_comp13712_c0_seq1_OFAS000119-RA-EXON01 1441 Acur_01000228.1 2415 1 1 1440 1 2231 797 0.0 1906 90.62 1305 1440
Ctom_01003023.1_OFAS000119-RA-EXON01 1289 Apil_comp16418_c0_seq1 2079 1 3 1284 1 1714 430 0.0 1351 85.69 1102 1286
Ctom_01003023.1_OFAS000119-RA-EXON01 1289 Acur_01000228.1 2415 1 1 1287 1 2084 797 0.0 1219 83.81 1082 1291
Ctom_01003023.1_OFAS000119-RA-EXON01 1289 Ctom_01003023.1 2162 1 1 1289 1 106 1394 0.0 2381 100.00 1289 1289
Edit 2: There is still an occasional output lacking fewer sequences than expected, although not as many after incorporating modifications to my script from Edit 1 suggestion (i.e., accounting for reverse direction). I cannot figure out why the script would be outputting fewer sequences in these other cases. Below the infile in question. The output is lacking Btri_comp15171_c0_seq1:
Apil_comp19456_c0_seq1_OFAS000248-RA-EXON07 2464 Apil_comp19456_c0_seq1 3549 1 1 2464 1 761 3224 0.0 4551 100.00 2464 2464
Apil_comp19456_c0_seq1_OFAS000248-RA-EXON07 2464 Btri_comp15171_c0_seq1 3766 1 1 2456 1 3046 591 0.0 1877 80.53 1985 2465
Btri_comp15171_c0_seq1_OFAS000248-RA-EXON07 2457 Apil_comp19456_c0_seq1 3549 1 1 2457 1 3214 758 0.0 1879 80.54 1986 2466
Btri_comp15171_c0_seq1_OFAS000248-RA-EXON07 2457 Atri_comp28646_c0_seq1 1403 1 1256 2454 1 1401 203 0.0 990 81.60 980 1201
Btri_comp15171_c0_seq1_OFAS000248-RA-EXON07 2457 Btri_comp15171_c0_seq1 3766 1 1 2457 1 593 3049 0.0 4538 100.00 2457 2457
You can use hash to remove duplicates
The bellow code remove duplicates depending on their subject length (keep larger subject length rows).
Just update your # iterate through the individual hits part with
# iterate through the individual hits
my %filterhash;
my $subject_length;
for ($jter=0; $jter<$t1; $jter++) {
(#templine) = split(/\s+/, $temparray[$jter]);
$subject_length = $templine[9] -$templine[8];
if(exists $filterhash{$templine[2]} ){
if($filterhash{$templine[2]} < $subject_length){
$filterhash{$templine[2]}= $subject_length;
}
}
else{
$filterhash{$templine[2]}= $subject_length;
}
}
my %printhash;
for ($jter=0; $jter<$t1; $jter++) {
(#templine) = split(/\s+/, $temparray[$jter]);
$subject_length = $templine[9] -$templine[8];
if(not exists $printhash{$templine[2]})
{
$printhash{$templine[2]}=1;
if(exists $filterhash{$templine[2]} and $filterhash{$templine[2]} == $subject_length ){
$com = "./extract_from_genome2 $transcriptomes $templine[2] $templine[8] $templine[9] $templine[2]";
# print "$com\n";
system("$com");
system("cat temp.3 >> $seqfname");
}
}
else{
if(exists $filterhash{$templine[2]} and $filterhash{$templine[2]} == $subject_length ){
$com = "./extract_from_genome2 $transcriptomes $templine[2] $templine[8] $templine[9] $templine[2]";
#print "$com\n";
system("$com");
system("cat temp.3 >> $seqfname");
}
}
} # end for ($jter=0; $jter<$t1...
Hope this will help you.
Edit part update
for negative stand you need to replace
$subject_length = $templine[9] -$templine[8];
with
if($templine[8] > $templine[9]){
$subject_length = $templine[8] -$templine[9];
}else{
$subject_length = $templine[9] -$templine[8];
}
You also need to update your extract_from_genome2 code for negative strand sequences.
I had two arrays, (data1 is a header array of strings and data2 is the data array of numbers)
data1 = {'#','Area','C Xp','C Yp','Length','B #','R','L','Ch','E1 Xp','E1 Yp','E2 Xp','E2 Yp'};
data2 = [1 939 -397 586 99 2 2 0 -1 -450 588 -352 572
2 1185 -287 294 145 2 1 1 0 -317 359 -235 244
3 592 -242 486 77 3 2 1 0 -278 488 -202 477
4 818 -144 480 60 2 0 2 1 -181 488 -135 451
5 377 -23 -443 37 1 0 1 0 -42 -459 -12 -460
6 923 32 -234 67 1 0 0 0 -3 -260 60 -212
7 812 150 -148 54 1 0 1 0 136 -130 169 -161
8 5968 428 432 402 3 3 0 -1 224 468 622 356
9 617 714 13 63 1 0 1 0 687 35 702 -22
csvwrite('file.xlsx', data1, 0, 0);
csvwrite('file.xlsx', data2, 0, 1);
My first problem is data1 prints to the spreadsheet as an array of chars (example: '#','A','r','e','... each in their own cells). How do I get it to print as the strings I am passing?
My second problem is when I csvwrite data2, data1's info is erased or overwritten. How can I write both to the same file?
Hidden away in the Tips section of the csvwrite documentation:
csvwrite does not accept cell arrays for the input matrix M. To export a cell array that contains only numeric data, use cell2mat to convert the cell array to a numeric matrix before calling csvwrite. To export cell arrays with mixed alphabetic and numeric data, where each cell contains a single element, you can create an Excel® spreadsheet (if your system has Excel installed) using xlswrite. For all other cases, you must use low-level export functions to write your data. For more information, see Export Cell Array to Text File in the MATLAB® Data Import and Export documentation.
I'd say, use xlswrite.
If you can't use xlswrite, it looks like you are stuck doing it manually as described on the page, Export Cell Array to Text File. Something along the lines of:
% write headers
fid = fopen('test.csv','w');
fprintf(fid,'%s,',data1{:});
fprintf(fid,'\n');
% write data...
fprintf(fid,[repmat('%d,',1,numel(data1)) '\n'],data2);
fclose(fid)
I've got a large file with diffrent lines.
The lines i am interested in, are looking alike:
lcl|NC_005966.1_gene_59 scaffold441.6 99.74 390 1 0 1 390 34065 34454 0.0 715
lcl|NC_005966.1_gene_59 scaffold2333.4 89.23 390 42 0 1 390 3114 2725 1e-138 488
lcl|NC_005966.1_gene_60 scaffold441.6 100.00 186 0 0 1 186 34528 34713 1e-95 344
Now i want to get the lines after the pattern 'lcl|NC_' but just if the third word(or the nth word in the line) is smaller than 100.
(In this case the first two lines, since they just got a number of 99.74 and 89.23)
Next they should be saved into a new file.
This can make it:
$ awk '$1 ~ /^lcl\|NC_/ && $3<100' file
lcl|NC_005966.1_gene_59 scaffold441.6 99.74 390 1 0 1 390 34065 34454 0.0 715
lcl|NC_005966.1_gene_59 scaffold2333.4 89.23 390 42 0 1 390 3114 2725 1e-138 488
It checks both things:
- 1st field starting with lcl|NC_: $1 ~ /^lcl\|NC_/ does it. (Thanks Ed Morton for improving the previous $1~"^lcl|NC_")
- 3rd field being <100: $3<100.
To save into a file, you can do:
awk '$1 ~ /^lcl\|NC_/ && $3<100' file > new_file
^^^^^^^^^^