I've copy the exact example in q for mortals as follows:
q)h:hopen `:D:/q4m/raw
q)h[42]
548i
q)h 10 20 30
548i
q)hclose h
q)get `:D:/q4m/raw
'D:/q4m/raw
[0] get `:D:/q4m/raw
Look into the directory, the file was created there. Why can't I get it?
Instead, if I do:
q)h:hopen `:D:/q4m/L
q)h[42]
628i
q)h[10 20 30]
628i
q)hclose h
q)get `:D:/q4m/L
0 1 2 3 4 42 10 20 30
Things get normal, why?
After testing the given code I believe your issue may be in how you intialise the file.
I assume in the code that works that you use some variation of
`:D:/q4m/L set til 5
before.
However this is not done for
`:D:/q4m/raw
If you were to use
`:D:/q4m/raw set til 5
or alternatively
.[`:D:/q4m/raw;();:;()]
beforehand then the first set of code will work.
Additionally, if we look at the binary using
read1 `:D:/q4m/raw
and
read1 `:D:/q4m/L
and the output does not include 07 near the beginning then it is not being recognised as a proper kdb list. That is, hopen simply appends to the binary file instead of amending it. (If you notice the 05 byte that indicates length of the list, this doesn't increase when you add via the handle).
eg.
The first method you get
q)read1 `:D:/q4m/raw
0x2a000000000000000a0000000000000014000000000000001e00000000000000
which dosen't really mean anything in q.
The second method gives
q)read1 `:D:/q4m/L
0xfe2007000000000005000000000000000000000000000000010000000000000002000000000..
which is a proper kdb list (notice the 07 which indicates type).
If you wish to instead just read in /q4m/raw then I suggest setting an empty list, hopen to that list and pass it `:D:/q4m/raw as follows
q)`:empty set 0#0
`:empty
q)h:hopen `:empty
q)h read1 `:D:/q4m/raw
3i
q)get `:empty
42 10 20 30
This will only work if all entries are the same type.
Related
q) ({2*x};{3*x})
How can I apply the list of functions to an input, e.g. 4, something like:
({2*x};{3*x})[4]
8 12
You should be able to use apply (#) each left (\:)
({2*x};{3*x})#\:4
Alternative approach to apply each left
q)({2*x};{3*x})[;4]
8 12
Just to generalise Michaels answer, if your function takes more than one input/parameter then you'd need to use dot-apply (.) rather than #. Dot-apply would work in both cases using:
q)({2*x};{3*x}).\:(),4
8 12
q)({y+2*x};{y+3*x}).\:(),4 100
108 112
I am completely new to this type of coding/command lines, so I am sorry if I am asking this question in a wrong way.
I want to loop over all files in a directory (I am quality trimming DNA sequencing files (.fastq format))
I have written this loop:
for i in *.fastq; do
perl /apps/prinseqlite/0.20.4/prinseq-lite.pl -fastq $i -min_len 220 -max_len 240 -min_qual_mean 30 -ns_max_n 5 -trim_tail_right 15 -trim_tail_left 15 -out_good /proj/forhot/qfiltered/looptest/$i_filtered.fastq -out_bad null; done
The code itself seems to work, I can see in my terminal that it is taking the right files and it is doing the trimming (it is writing a summary log in the terminal as it goes), but no output files are generated - i.e these ones:
-out_good /proj/forhot/qfiltered/looptest/$i_filtered.fastq
If I run the code in a non-loop way, just on one file it works (= the output is generated). link this example:
prinseq-lite.pl -fastq 60782_merged_rRNA.fastq -min_len 220 -max_len 240 -min_qual_mean 30 -ns_max_n 5 -trim_tail_right 15 -trim_tail_left 15 -out_good 60782_merged_rRNA_filt_codeTEST.fastq -out_bad null
Is there a simple reason/answer to this?
This problem has nothing to do with Perl at all.
/proj/forhot/qfiltered/looptest/$i_filtered.fastq is read by the shell as interpolating the contents of i_filtered. There is no such shell variable, so this argument turns into /proj/forhot/qfiltered/looptest/.fastq ($i_filtered turns into nothing).
Therefore all of your prinseq-lite.pl executions place their output in the same file, which (because its name starts with a .) is "hidden": You need to use ls -a to see it, not just ls.
Fix
... -out_good /proj/forhot/qfiltered/looptest/${i}_filtered.fastq
Note that this would give you e.g. 60782_merged_rRNA.fastq_filtered.fastq for an input file of 60782_merged_rRNA.fastq. If you want to get rid of the duplicate .fastq part, you need something like:
... -out_good /proj/forhot/qfiltered/looptest/"${i%.fastq}"_filtered.fastq
I have a quick question. Its most likely user error so I apologize before I begin.
I’m trying to setup threshold for a device so that it will alert us when one of our printers is in a certain state. (Jammed, out of toner, no paper etc) I’ve found the specific oid that handles this. (1.3.6.1.2.1.25.3.5.1.2.1) The specific oid is called hrPrinterDetectedErrorState under the HOST-RESOURCE-MIB. I’ve verified that I can see the oid via SNMPWALK. My problem is interpreting the data its spitting out. What i'm reading in the MIB and what i'm seeing via SNMPWALK are different.
Here is the description of oid from the MIB:
"This object represents any error conditions detected
by the printer. The error conditions are encoded as
bits in an octet string, with the following
definitions:
Condition Bit #
lowPaper 0
noPaper 1
lowToner 2
noToner 3
doorOpen 4
jammed 5
offline 6
serviceRequested 7
inputTrayMissing 8
outputTrayMissing 9
markerSupplyMissing 10
outputNearFull 11
outputFull 12
inputTrayEmpty 13
overduePreventMaint 14
Bits are numbered starting with the most significant
bit of the first byte being bit 0, the least
significant bit of the first byte being bit 7, the
most significant bit of the second byte being bit 8,
and so on. A one bit encodes that the condition was
detected, while a zero bit encodes that the condition
was not detected.
This object is useful for alerting an operator to
specific warning or error conditions that may occur,
especially those requiring human intervention."
The odd part is that SNMPWALK says the oid is a Hex-String while the MIB specifies that it should be a Octet-String. Are the two different? Do I need to convert the data that is output by SNMPWALK somehow to get it to match up with what the MIB is saying?
To test everything, I put the printer into several different “States.” I then ran an SNMPWALK on the device to see what the oid output. Here are the results. As you will see, these results don’t match up to what the MIB specifies.
Case 1: Opened the toner door
Expected Output based on MIB: 4
SNMP Output: 08
Case 2: Removed Toner & Closed the door
Expected Output based on MIB: 1
SNMP Output: 10
Case 3: Placed 1 sheet of paper and printed a 2 page document. The printer ran out of paper.
Expected Output based on MIB: 0 or 1
SNMP Output: #
I’m confused at the output. I just need to know how to read the oid so I can setup thresholds so that when it sees, for example, a 08 it performs a certain action.
Thanks for your help!
You are reading this wrong. The data you receive back should actually be interpreted as a bit array and every bit is its own value for the specific alarm in your case
Expected Output based on MIB: 4
SNMP Output: 08
You actually get back the output:
00001000 00000000
The first byte here covers those values
lowPaper 0
noPaper 1
lowToner 2
noToner 3
doorOpen 4
jammed 5
offline 6
serviceRequested 7
So lowPaper is bit 0, noPaper is bit 1, lowToner is bit 2, etc. And doorOpen is bit 4 and as you can see that bit is set, indicating that the door is open.
EDIT:
This is very dependent on the device and implementation. To know how to parse it right involves a lot of trial and error (at least from my experience). As an example if you get back the message 9104, this could be either be
91 and 04 are separate so you first translate 91 to binary from hex and then the
same thing with 04
91: 10010001
04: 00000100
Which would mean low paper, noToner, service requested and inputTrayEmpty. This looks like the most likely way this works.
If you only get one byte back this then means you should only look for alarms in the first 8 bits. As a example of things you need to look out for: If the only alarm present is doorOpen you could be getting back only 08 but it would actually be 0008 where the first 2 hex chars are actually the second part of the alarms but aren't shown because they are none present. So in your case you actually first have to switch the bytes (if there are 4) parse the first two and the second two on their own and then you get the actual result.
As I said there is no real standard here from what i have seen and you have to just work with it until you are sure you know how the data is sent and how to parse it.
Powershell function to interpret the hrPrinterDetectedErrorState octetstring. (I feel like there's some more often used way to do this?) I've been using this module, but it doesn't even resolve hostnames to ip addresses.
The data property returned is just a string not an octetstring type. None of the printers are snmp v3.
https://www.powershellgallery.com/packages/SNMP/1.0.0.1
A single byte would have to be padded with a zero. Most of the time only one byte is sent. The bytes are in the, I think confusing, order that it's documented in. Converting two bytes to an integer results in a different order (depending on big endian or little endian order).
function snmpmessage {
param($data)
[flags()] Enum hrPrinterDetectedErrorState
{
# more common group
LowPaper = 0x0080
NoPaper = 0x0040
LowToner = 0x0020
NoToner = 0x0010
DoorOpen = 0x0008
Jammed = 0x0004
Offline = 0x0002
ServiceRequested = 0x0001
InputTrayMissing = 0x8000
OutputTrayMissing = 0x4000
MarkerSupplyMissing = 0x2000
OutputNearFull = 0x1000
OutputFull = 0x0800
InputTrayEmpty = 0x0400
OverduePreventMaint = 0x0200
}
$bytes = [byte[]][char[]]$data
if ($bytes.count -eq 1) { $bytes = $bytes[0],0 } # pad 0
$code = [bitconverter]::ToUInt16($bytes, ($startIndex=0))
[hrPrinterDetectedErrorState]$code
}
snmpmessage -join [char[]](1,4)
ServiceRequested, InputTrayEmpty
snmpmessage '#' # 0x40
NoPaper
Or:
$hrPrinterDetectedErrorState = '1.3.6.1.2.1.25.3.5.1.2.1'
$hostname = 'myprinter01'
$ip = (Resolve-DnsName $hostname).ipaddress
$result = Get-SnmpData -ip $ip -oid $hrPrinterDetectedErrorState -v v1
snmpmessage $result.data
LowToner
# ?
# $octetstring = [Lextm.SharpSnmpLib.OctetString]::new($result.data)
I'm running a bunch (~320) computational chemistry experiments and I need to pull a small amount of the data out of each of the files so that I can do some work on it in MatLab.
I'm pretty sure I can use sed to make this work, but try as I might I don't seem to be able to do so.
I need all of the data starting at the line beginning with "1 1" and ending with the line starting with "33 33".
I J FI(I,J) k(I,J) K(I,J)
1 1 -337.13279 -0.06697 -0.00430
2 2 3804.89120 8.52972 0.54787
3 3 3195.69653 6.01702 0.38648
4 4 3189.18684 5.99253 0.38490
5 5 3183.73262 5.97205 0.38359
6 6 3174.47525 5.93737 0.38136
7 7 3167.88746 5.91275 0.37978
8 8 1628.80868 1.56311 0.10040
9 9 1623.56055 1.55306 0.09975
10 10 1518.21620 1.35806 0.08723
11 11 1476.93012 1.28520 0.08255
12 12 1341.24087 1.05990 0.06808
13 13 1312.30373 1.01466 0.06517
14 14 1264.73004 0.94242 0.06053
15 15 1185.62592 0.82822 0.05320
16 16 1175.54013 0.81419 0.05230
17 17 1170.41211 0.80710 0.05184
18 18 1090.20196 0.70027 0.04498
19 19 1039.29190 0.63639 0.04088
20 20 1015.00116 0.60699 0.03899
21 21 1005.05773 0.59516 0.03823
22 22 986.55965 0.57345 0.03683
23 23 917.65537 0.49615 0.03187
24 24 842.93089 0.41863 0.02689
25 25 819.00146 0.39520 0.02538
26 26 758.39720 0.33888 0.02177
27 27 697.11173 0.28632 0.01839
28 28 628.75684 0.23292 0.01496
29 29 534.75856 0.16849 0.01082
30 30 499.35579 0.14692 0.00944
31 31 422.01320 0.10493 0.00674
32 32 409.30255 0.09870 0.00634
33 33 227.12411 0.03039 0.00195
33 2nd derivatives larger than 0.371D-04 over 561
MatLab is not a fan of text, so I'd like to not use text delimiters (though there are some in the header of this data section) and keep the data contained to only the numeric lines.
The data files contain a lot of other numbers as well, so I need to match the occurrence of "1 1" at the start of the line and "33 33" as the end of the copy. These 'indices' exist only in this block of info.
I attempted to use
% sed -n /"1 1"/,/"33 33"/p input.file > output.file
But I get a WHOLE BUNCH of data in the output file as it copies everything that shows up between any "1" and "33"
Is there any way to do what I'm looking for?
Also, I'm using the tcsh as that is what my servers run.
How about using awk
awk '$1=="1"&&$2=="1"{t=1};t;$1=="33"&&$2=="33"{t=0}' file
Recommand by #mklement0, if there is only one block, to avoid processing the remainder of the file you can update the command to:
awk '$1=="1"&&$2=="1"{t=1};t;$1=="33"&&$2=="33"{exit}' file
Your problem is twofold. First, there are two blanks between the ones, but your regex only allows for one (judging from the now indented code). Second, you are probably not precise enough; the /1 1/ pattern matches 11 11, for example, and 111 111 and so on.
So, you should consider:
sed -n -e '/^ *1 *1 /,/^33 *33 /p' -e '/^33 33 /q' input.file > output.file
The patterns are anchored to the start of line by the ^ (caret). The numbers are separated by one or more blanks (there are other, longer-winded ways of writing that in standard sed; the + option is not standard sed but is widely available). And the numbers are terminated by a blank. The chances are that the first expression alone will give you what you want. The second expression terminates the search early when it recognizes the 33 33 input line, which can save a significant amount of file I/O and hence processing time if the input file is big enough.
If the lines with ID numbers in the hundreds have some different format, then it should be fairly straight-forward to tweak the regexes to match what is used. If the data contains tabs instead of (or as well as) blanks, you can tweak the regexes to manage that, too.
If you data is all formatted exactly the same as this file, then you can use sed to just read the 3rd through the 35th line (rows 1 1 - 33 33). This is a lot easier than parsing the values, but does require that the files have a standard format:
sed -n 3,35p data.txt
Another cheap way would be to grep for only numeric lines, and take only the first 33:
grep "^[0-9 ][0-9 .-]*$" data.txt | head -n 33
In order to debug a very complex set of functions, I want to isolate a subfunction from the workspace in order to make different test. Therefore a need selected values from the function workspace to be defined already. By setting a break point at the specific position I can "look" into the current workspace by displaying the values in the console, like the variable HF33
HF33 =
1.0777 0.0865 0.0955
-0.1891 0.8110 -0.1889
0.0935 0.0846 1.0755
Is there some function / script that could convert this result to a new Matlab expression that can be pasted somewhere else (for example at the head of a new script), e.g.:
HF33 = [ 1.0777, 0.0865, 0.0955;
-0.1891, 0.8110, -0.1889;
0.0935, 0.0846, 1.0755 ];
With that I could test the subfunction and its behavior by easily changing the given values and see whats happening without having the huge debug workspace running.
Is there some easy function like res2exp(HF33)?
First: Create this function to get the variable name
function out = varname(var)
out = inputname(1);
end
you can print it direct to console:
fprintf('%s =%s\n',varname(varToSave),mat2str(varToSave));
Or use fopen and fprint to write it in a file
fop = fopen('filename','w');
fprint(fop,'%s = %s' ,varname(varToSave),mat2str(varToSave));
fclose(fop);
I think this will help you
It might be a function like mat2str() you are looking for but it will not give exactly the printout you are asking for. Here is an example of how it could be used:
>> A = magic(4)
A =
16 2 3 13
5 11 10 8
9 7 6 12
4 14 15 1
>> B = mat2str(A)
B =
[16 2 3 13;5 11 10 8;9 7 6 12;4 14 15 1]
And if you want the output to be totally copy/paste-able you could use:
disp(['C = ',mat2str(A)])
C = [16 2 3 13;5 11 10 8;9 7 6 12;4 14 15 1]
I made this up just now. It is not formatted beautifully, but it achieves what you are trying to do - if I understand you correctly.
a = [ 2 3 4 5
4 5 5 6
3 4 5 6];
fprintf('\nb = [\n\n');
disp(a);
fprintf(']\n\n');
Copy and paste this and see if it does what you want. It's also very simple code, so you could modify it if the spacing and newline characters aren't where you want them.
You could also make a small function out of this if you wanted to.
If you want me to make a function of it, let me know... I can do it tomorrow. But you can probably figure it out.
Ehh, I just made the function. It didn't take long.
function reprint_matrix(matrix)
var_name = inputname(1);
fprintf('\n%s = [\n\n', var_name);
disp(matrix);
fprintf(']\n\n');
end
I'm not sure what you are looking for, but I think this will help you:
http://www.mathworks.com/matlabcentral/fileexchange/24447-generate-m-file-code-for-any-matlab-variable/content/examples/html/gencode_example.html
Did not use it because I use mat-files to transfer data.
You can combine it with the clipboard function:
clipboard('copy',gencode(ans))
Though there are several ways to write variables to text, saving variables as text is definitely bad practice if it can be avoided. Hence, the best advice I can give you is to solve your problem in a different way.
Suppose you want to use HF33 in your subfunction, then here is what I would recommend:
First of all, save your variable of interest:
save HF33 HF33
Then when you are in the function where you want to use this variable:
load HF33
This assumes that your working directory (not workspace) is the same in both cases, but otherwise you can simply add the path in your save or load command. If you want to display it you can now simply call the variable HF33 without a semicolon (this is probably the only safe way to display it exactly the way you expect in all cases).
Note that this method can easily be adapted to transfer multiple variables at once.