python subprocess pipe comparing string - removing rubbish - subprocess

I have a c program that reads a rfid tag. I am trying to get output from that c program (it uses sudo and args) and compare it to a string
Here is my code as well as some debugging information.
import subprocess
#args - the c program and its args
args = ["sudo", "./rc522", "-r", "-b", "1"]
process = subprocess.Popen(args,stdout=subprocess.PIPE,stderr=None)
rfidRead=process.communicate()[0]
rfidRead=rfidRead.decode('utf-8')
print (len(rfidRead))
rfidRead = rfidRead[14:440]
print (len(rfidRead))
print (rfidRead)
if "49.4f.09.0a.0c.0f.00.00.00.00.00.00.00.00.00.00" in rfidRead:
print("unlocked")
else:
print("Not unlocked")
here is the output...
446
426
49.4f.09.0a.0c.0f.00.00.00.00.00.00.00.00.00.00
Not unlocked
I just cant find the hidden characters and things from the pipe output
Can you help please?
This is the rfidRead output using repr:
'\x1b[1;93m4\x1b[00m\x1b[1;93m9\x1b[00m.\x1b[1;93m4\x1b[00m\x1b[1;93mf\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93m9\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93ma\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93mc\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93mf\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93m0\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93m0\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93m0\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93m0\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93m0\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93m0\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93m0\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93m0\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93m0\x1b[00m.\x1b[1;93m0\x1b[00m\x1b[1;93m0'

I worked out the answer with the help of jasonharper.
After doing the repr(...) I decided to google \x1b and found a stackoverflow article about vt100 and that this is vt100 code, so I did another google search to decode vt100, and I got a regular expression to remove the escape codes... here is the article I found. How can I remove the ANSI escape sequences from a string in python

Related

What is the '...' in sklearn.feature_extraction.text TfidfVectorizer return output?

When I used TfidfVectorizer to encode a paragraph, I receive the output that include '...' inside, this look like the matrix is too large and python cut off something. This problem happens occasionally, but not always, and cause error in another code because '...' has no meaning.
I run in on Conda with Python 36
My source code is like this:
vectorizer = TfidfVectorizer(stop_words='english',use_idf=True)
# print(data)
X = vectorizer.fit_transform(data).todense()
with open("spare_matrix.txt","w") as file:
file.write(str(X))
file.close()
Please suggest me some ideas for that with thanks

How to recode missing genotype code is " '-' " in the ped file of plink

I'm trying to impute genotype data from the public reference panels but my files fail the file sanity check on Sanger Imputation server and it gives the following error:
failed sanity check :
of Non-ACGTN alternate allele at 1:4635556 .. REF_SEQ:'(null)' vs VCF:'-'
I have tried fixing this in the plink with the following command ./plink --bfile chr1 --recode vcf --out chr1_vcf --missing-genotype -
but then it gives error Underscore(s) present in sample IDs.
--recode vcf to chr1_vcf.vcf ... done.
but I still see '_' in the new coded file.
I would appreciate any help, suggestions and comments.
Thanks
Jasdeep
You will have to replace _ with a different character in your PLINK files before running your code.
See below from PLINK manual
When using --recode vcf, sample IDs are formed by merging the FID and IID and placing an underscore between them. When the FID or IID already contains an underscore, this may make it difficult to reconstruct them from the VCF file; you may want to replace underscores with a different character in PLINK files (Unix tr is handy here).

Perl interface with Aspell

I am trying to identify misspelled words with Aspell via Perl. I am working on a Linux server without administrator privileges which means I have access to Perl and Aspell but not, for example, Text::Aspell which is a Perl interface for Aspell.
I want to do the very simple task of passing a list of words to Aspell and having it return the words that are misspelled. If the words I want to check are "dad word lkjlkjlkj" I can do this through the command line with the following commands:
aspell list
dad word lkjlkjlkj
Aspell requires CTRL + D at the end to submit the word list. It would then return "lkjlkjlkj", as this isn't in the dictionary.
In order to do the exact same thing, but submitted via Perl (because I need to do this for thousands of documents) I have tried:
my $list = q(dad word lkjlkjlkj):
my #arguments = ("aspell list", $list, "^D");
my $aspell_out=`#arguments`;
print "Aspell output = $aspell_out\n";
The expected output is "Aspell output = lkjlkjlkj" because this is the output that Aspell gives when you submit these commands via the command line. However, the actual output is just "Aspell output = ". That is, Perl does not capture any output from Aspell. No errors are thrown.
I am not an expert programmer, but I thought this would be a fairly simple task. I've tried various iterations of this code and nothing works. I did some digging and I'm concerned that perhaps because Aspell is interactive, I need to use something like Expect, but I cannot figure out how to use it. Nor am I sure that it is actually the solution to my problem. I also think ^D should be an appropriate replacement for CTRL+D at the end of the commands, but all I know is it doesn't throw an error. I also tried \cd instead. Whatever it is, there is obviously an issue in either submitting the command or capturing the output.
The complication with using aspell out of a program is that it is an interactive and command-line driver tool, as you suspect. However, there is a simple way to do what you need.
In order to use aspell's command list one needs to pass it words via STDIN, as its man page says. While I find the GNU Aspell manual a little difficult to get going with, passing input to a program via its STDIN is easy enough and we can rewrite the invocation as
echo dad word lkj | aspell list
We get lkj printed back, as due. Now this can run out of a program just as it stands
my $word_list = q(word lkj good asdf);
my $cmd = qq(echo $word_list | aspell list);
my #aspell_out = qx($cmd);
print for #aspell_out;
This prints lines lkj and asdf.
I assemble the command in a string (as opposed to an array) for specific reasons, explained below. The qx is the operator form of backticks, which I prefer for its far superior readability.
Note that qx can return all output in a string, if in scalar context (assigned to a scalar for example), or in a list when in list context. Here I assign to an array so you get each word as an element (alas, each also comes with a newline, so may want to do chomp #aspell_out;).
Comment on a list vs string form of a command
I think that it's safe to recommend to use a list-form for a command, in general. So we'd say
my #cmd = ('ls', '-l', $dir); # to be run as an external command
instead of
my $cmd = "ls -l $dir"; # to be run as an external command
The list form generally makes it easier to manage the command, and it avoids the shell altogether.
However, this case is a little different
The qx operator doesn't really behave differently -- the array gets concatenated into a string, and that runs. The very fact that we can pass it an array is incidental, and not even documented
We need to pipe input to aspell's STDIN, and shell does that for us simply. We can use a shell with command's LIST form as well, but then we'd need to invoke it explicitly. We can also go for aspell's STDIN by means other than the shell but that's more complex
With a command in a list the command name must be the first word, so that "aspell list" from the question is wrong and it should fail (there is no command named that) ... except that in this case it wouldn't (if the rest were correct), since for qx the array gets collapsed into a string
Finally, apsell nicely exposes its API in a C library and that's been utilized for the module you mention. I'd suggest to install it as a user (no privileges needed) and use that.
You should take a step back and investigate if you can install Text::Aspell without administrator privilige. In most cases that's perfectly possible.
You can install modules into your home directory. If there is no C-compiler available on the server you can install the module on a compatible machine, compile and copy the files.

Extract the £ (pound) currency symbol and the amount (56) from an html file

Extract the £ (pound) currency symbol and the amount (56) from an html file. It is printing the amount as £56 and prints the currency as Â. How can I print only 56, without the symbol? It is working fine with a $ sign.
Part of the code:
cost= "£56"
currencySymbol = cost[0]
print (currencySymbol, cost[1:])
The output I am getting:
Â: £56
there are many ways that you can do it, you can use split, regex and one method that I did below:
Hope it helps you
import re
cost= "£560,000"
match = re.search(r'([\D]+)([\d,]+)', cost)
output = (match.group(1), match.group(2).replace(',',''))
print (output);
output -->('£', '560000')
check here (https://ideone.com/Y053Vb)
Resolved: i tried to run below code in separate file in eclipse and given error about utf-8.
i search the error and got answer, it is eclipse who is changing unicode style to avoid i used to run in python IDLE, i think we can change unicode in eclipse?.
Thanks to Martijn Pieters
[SyntaxError: Non-UTF-8 code starting with '\x91'
cost= "£56"
currencySymbol = cost[0]
print (currencySymbol, cost[1:])
#resolution :when using file use encoding
#with open('index.html', encoding="UTF-8") as productFile:

Spotify Tech Puzzle - stdin in Python

I'm trying to solve the bilateral problem on Spotify's Tech Puzzles. http://www.spotify.com/us/jobs/tech/bilateral-projects/ I have something that is working on my computer that reads input from a file input.txt, and it outputs to ouput.txt. My problem is that I cannot figure out how to make my code work when I submit it where it must read from stdin. I have looked at several other posts and I don't see anything that makes sense to me. I see some people just use raw_input - but this produces a user prompt?? Not sure what to do. Here is the protion of my code that is suposed to read the input, and write the output. Any suggestions on how this might need changed? Also how would I test the code once it is changed to read from stdin? How can I put test data in stdin? The error i get back from spotify says Run Time Error - NameError.
import sys
# Read input
Input = []
for line in sys.stdin.readlines():
if len(line) <9:
teamCount = int(line)
if len(line) > 8:
subList = []
a = line[0:4]
b = line[5:9]
subList.append(a)
subList.append(b)
Input.append(subList)
##### algorithm here
#write output
print listLength
for empWin in win:
print empWin
You are actually doing ok.
for line in sys.stdin.readlines():
will read lines from stdin. It can however be shortened to:
for line in sys.stdin:
I don't use Windows, but to test your solution from a command line, you should run it like this:
python bilateral.py < input.txt > output.txt
If I run your code above like that, I see the error message
Traceback (most recent call last):
File "bilateral.py", line 20, in <module>
print listLength
NameError: name 'listLength' is not defined
which by accident (because I guess you didn't send in that) was the error the Spotify puzzle checker discovered. You have probably just misspelled a variable somewhere.