Stata: Systematically replace characters in a string variable - substring

I have observations which list criminal codes as string variables, but not in the format I need. Using Stata 12, I want to replace some substrings in a string variable. For example, I need to change all instances of CC to 18, VC to 75, and PC to 35. Like so:
Orginal Variable
CC547A1 | VC549F| PC5297
New Variable
18547A1 | 75549F | 355297
The characters I need to change are always in the beginning. Some original variables do not need to be changed.
I tried figuring this out using the substring command, but I just couldn't adapt the code correctly.

The substr() function (not substring(); not a command) is not as helpful here as its sibling, subinstr(). Documented in the same place: start at help functions.
. clear
. input str7 myvar
myvar
1. CC547A1
2. VC549F
3. PC5297
4. end
. replace myvar = subinstr(myvar, "CC", "18", .)
(1 real change made)
. replace myvar = subinstr(myvar, "VC", "75", .)
(1 real change made)
. replace myvar = subinstr(myvar, "PC", "35", .)
(1 real change made)
. list
+---------+
| myvar |
|---------|
1. | 18547A1 |
2. | 75549F |
3. | 355297 |
+---------+
N.B. previous thread How do I find and replace a part of a string variable in Stata?
EDIT The implication of the question appears to be that the strings to be replaced occur just once. If that were ever false, and you only wanted the first occurrence to be replaced, then the advice is to do something like subinstr(myvar, "CC", "18", 1) where the last argument 1 stipulates the (maximum) number of replacements to be made. In other problems this detail could be crucial.

Related

Powershell number format

I am creating a script converting a csv file in an another format.
To do so, i need my numbers to have a fixed format to respect column size : 00000000000000000,00 (20 characters, 2 digits after comma)
I have tried to format the number with -f and the method $value.toString("#################.##") without success
Here is an example Input :
4000000
45817,43
400000
570425,02
15864155,69
1068635,69
128586256,9
8901900,04
29393,88
126858346,88
1190011,46
2358411,95
139594,82
13929,74
11516,85
55742,78
96722,57
21408,86
717,01
54930,49
391,13
2118,64
Any hints are welcome :)
Thank you !
tl;dr:
Use 0 instead of # in the format string:
PS> $value = 128586256.9; $value.ToString('00000000000000000000.00')
00000000000128586256.90
Note:
Alternatively, you could construct the format string as an expression:
$value.ToString('0' * 20 + '.00')
The resulting string reflects the current culture with respect to the decimal mark; e.g., with fr-FR (French) in effect, , rather than . would be used; you can pass a specific [cultureinfo] object as the second argument to control what culture is used for formatting; see the docs.
As in your question, I'm assuming that $value already contains a number, which implies that you've already converted the CSV column values - which are invariably strings - to numbers.
To convert a string culture-sensitively to a number, use [double]::Parse('1,2'), for instance (this method too has an overload that allows specifying what culture to use).
Caveat: By contrast, a PowerShell cast (e.g. [double] '1.2') is by design always culture-invariant and only recognizes . as the decimal mark, irrespective of the culture currently in effect.
zerocukor287 has provided the crucial pointer:
To unconditionally represent a digit in a formatted string and default to 0 in the absence of an available digit, use 0, the zero placeholder in a .NET custom numeric format string
By contrast, #, the digit placeholder, represents only digits actually present in the input number.
To illustrate the difference:
PS> (9.1).ToString('.##')
9.1 # only 1 decimal place available, nothing is output for the missing 2nd
PS> (9.1).ToString('.00')
9.10 # only 1 decimal place available, 0 is output for the missing 2nd
Since your input uses commas as decimal point, you can split on the comma and format the whole number and the decimal part separately.
Something like this:
$csv = #'
Item;Price
Item1;4000000
Item2;45817,43
Item3;400000
Item4;570425,02
Item5;15864155,69
Item6;1068635,69
Item7;128586256,9
Item8;8901900,04
Item9;29393,88
Item10;126858346,88
Item11;1190011,46
Item12;2358411,95
Item13;139594,82
Item14;13929,74
Item15;11516,85
Item16;55742,78
Item17;96722,57
Item18;21408,86
Item19;717,01
Item20;54930,49
Item21;391,13
Item22;2118,64
'# | ConvertFrom-Csv -Delimiter ';'
foreach ($item in $csv) {
$num,$dec = $item.Price -split ','
$item.Price = '{0:D20},{1:D2}' -f [int64]$num, [int]$dec
}
# show on screen
$csv
# output to (new) csv file
$csv | Export-Csv -Path 'D:\Test\formatted.csv' -Delimiter ';'
Output in screen:
Item Price
---- -----
Item1 00000000000004000000,00
Item2 00000000000000045817,43
Item3 00000000000000400000,00
Item4 00000000000000570425,02
Item5 00000000000015864155,69
Item6 00000000000001068635,69
Item7 00000000000128586256,09
Item8 00000000000008901900,04
Item9 00000000000000029393,88
Item10 00000000000126858346,88
Item11 00000000000001190011,46
Item12 00000000000002358411,95
Item13 00000000000000139594,82
Item14 00000000000000013929,74
Item15 00000000000000011516,85
Item16 00000000000000055742,78
Item17 00000000000000096722,57
Item18 00000000000000021408,86
Item19 00000000000000000717,01
Item20 00000000000000054930,49
Item21 00000000000000000391,13
Item22 00000000000000002118,64
I do things like this all the time, usually for generating computernames. That custom numeric format string reference will come in handy. If you want a literal period, you have to backslash it.
1..5 | % tostring 00000000000000000000.00
00000000000000000001.00
00000000000000000002.00
00000000000000000003.00
00000000000000000004.00
00000000000000000005.00
Adding commas to long numbers:
psdrive c | % free | % tostring '0,0' # or '#,#'
18,272,501,760
"Per mille" character ‰ :
.00354 | % tostring '#0.##‰'
3.54‰

Converting numbers into timestamps (inserting colons at specific places)

I'm using AutoHotkey for this as the code is the most understandable to me. So I have a document with numbers and text, for example like this
120344 text text text
234000 text text
and the desired output is
12:03:44 text text text
23:40:00 text text
I'm sure StrReplace can be used to insert the colons in, but I'm not sure how to specify the position of the colons or ask AHK to 'find' specific strings of 6 digit numbers. Before, I would have highlighted the text I want to apply StrReplace to and then press a hotkey, but I was wondering if there is a more efficient way to do this that doesn't need my interaction. Even just pointing to the relevant functions I would need to look into to do this would be helpful! Thanks so much, I'm still very new to programming.
hfontanez's answer was very helpful in figuring out that for this problem, I had to use a loop and substring function. I'm sure there are much less messy ways to write this code, but this is the final version of what worked for my purposes:
Loop, read, C:\[location of input file]
{
{ If A_LoopReadLine = ;
Continue ; this part is to ignore the blank lines in the file
}
{
one := A_LoopReadLine
x := SubStr(one, 1, 2)
y := SubStr(one, 3, 2)
z := SubStr(one, 5)
two := x . ":" . y . ":" . z
FileAppend, %two%`r`n, C:\[location of output file]
}
}
return
Assuming that the "timestamp" component is always 6 characters long and always at the beginning of the string, this solution should work just fine.
String test = "012345 test test test";
test = test.substring(0, 2) + ":" + test.substring(2, 4) + ":" + test.substring(4, test.length());
This outputs 01:23:45 test test test
Why? Because you are temporarily creating a String object that it's two characters long and then you insert the colon before taking the next pair. Lastly, you append the rest of the String and assign it to whichever String variable you want. Remember, the substring method doesn't modify the String object you are calling the method on. This method returns a "new" String object. Therefore, the variable test is unmodified until the assignment operation kicks in at the end.
Alternatively, you can use a StringBuilder and append each component like this:
StringBuilder sbuff = new StringBuilder();
sbuff.append(test.substring(0,2));
sbuff.append(":");
sbuff.append(test.substring(2,4));
sbuff.append(":");
sbuff.append(test.substring(4,test.length()));
test = sbuff.toString();
You could also use a "fancy" loop to do this, but I think for something this simple, looping is just overkill. Oh, I almost forgot, this should work with both of your test strings because after the last colon insert, the code takes the substring from index position 4 all the way to the end of the string indiscriminately.

Extracting values from a single file

I have a file with multiple lines; but a specific line contains tons of information, with several repeated expressions. I'm trying to extract some specific values. I first tried some commands with sed, for instance, but with no success. So, I was wondering if you could give me some insights.
So, here you have one fraction of the unique line of the given document I mentioned:
[...]6[&length_range={0.19
[... a lot of more information here in between ...]
0.01},habitat.set.prob={0.01,0.03,0.56,0.01,0.01,0.34,0.01,0.01,0.01},DLOOP.rate_median=0.04131395026396427,length=
[...]
10[&length_range={0.19
[... a lot of more information here in between ...]
0.01},habitat.set.prob={0.21,0.33,0.56,0.01,0.01,0.33,0.01,0.01,0.61},DLOOP.rate_median=0.04131395026396427,length=
[...]
My aim here is first to extract all the values that is between the brackets, after "habitat.set.prob={". and put them in a single line in a text file.
Also, it would be important to extract the numbers that appears just before the expression "[&length_range=]", which in this case are "6" and "10". They are the label of the set of numbers after "prob={"
So the set of numbers I want to extract always appears between "habitat.set.prob={" and "},DLOOP.rate_median", while the other number (the label) is always rigth before "[&length_range="; but what is before the label is not the same expression; actually it is a random number.
The goal then is end up with a file with the following characteristcs:
6 0.21,0.33,0.56,0.01,0.01,0.33,0.01,0.01,0.61
10 0.21,0.33,0.56,0.01,0.01,0.33,0.01,0.01,0.61
and so on …
What do you think? Is this possible?
I started with this very basic command at least to try to extract the set of numbers, but it didn't work
sed -n "/habitat.set.prob={/,/},DLOOP.rate_median=/ p"
| Well... I got some improvement.
I was able to get the values at least:
awk '{gsub("habitat.set.prob={","\n");printf"%s",$0}' filename | awk -F'},' '{print $1"}"}' | grep -iv "TREE" > stats.txt
|
Many thanks in advance.
Cheers,
Luiz
Something like that:
sed -rn '/.*[0-9]+\[&length_range=\{/,/habitat.set.prob=\{/{s/.*\b([0-9]+)\[&length_range.*/\1/p; s/.*habitat.set.prob=\{([^D]+)\},DLOOP.rate.*/\1/p}' habitat
6
0.01,0.03,0.56,0.01,0.01,0.34,0.01,0.01,0.01
10
0.21,0.33,0.56,0.01,0.01,0.33,0.01,0.01,0.61
The first part '/.a./,/.b./' searches from pattern a to b, distributed over multiple lines. The -n told sed to do non-printing as default.
In '/.a./,/.b./{s/.c./.d./p; s/.e./.f./p}'
there are two substitution commands with p=print in curly braces.
I am not sure if you really digged a little, so not providing the complete answer, but let's hope this would help you:
for the first part: getting the no(which you call as label) you didn't mention if there is any specific pattern, so try this (data is the file which contains the actual input) - you need to work on how to get the number and tweak the RE a bit
sed -n 's/.*\([0-9][0-9]*\).*length_range.*/\1/p' data
For the other part which gives the numericals between habitat and DLOOP:
sed -n 's/.*habitat.set.prob=\(.*\),DLOOP.*/\1/pg' data | tr '{' ' ' | tr '}' ' '
Now, try to take this as a starter and work on your output to get your desired result!
To explain a bit:
In the first section - I am trying to capture the numericals between anything(.*) and (.*)length_range [you can escape the character [ and & by using \ in front of them]
In the second section: I am capturing pattern in between habitat.set.prob and DLOOP and then doin a tr to remove the brackets.
#include <iostream>
using namespace std;
int main()
{
string p = "1:2:3:4"; //input your string
int arr[4] = {}; //create a new empty integer array to put the integers in it
for(int i=0, j=0; i <p.length(); i++){//loop on the string to extract integers
if( p[i] == ':'){continue;}//if the value = ':' skip it and continue
arr[j]=(int)p[i]-48;j++;//put the integer in the array we created
}
cout << "String={"<<arr[0]<<" "<<arr[1]<<" "<<arr[2]<<" "<<arr[3]<<"}";//print the array
return 0;
}

Brainfuck challenge

I have a any challenge. I must write brainfuck-code.
For a given number n appoint its last digit .
entrance
Input will consist of only one line in which there is only one integer n ( 1 < = n < = 2,000,000,000 ) , followed by a newline ' \ n' (ASCII 10).
exit
On the output has to find exactly one integer denoting the last digit of n .
example I
entrance: 32
exit: 2
example II:
entrance: 231231132
exit: 2
This is what I tried, but it didn't work:
+[>,]<.>++++++++++.
The last input is the newline. So you have to go two memory positions back to get the last digit of the number. And maybe you don't have to return a newline character, so the code is
,[>,]<<.
Nope sorry, real answer is
,[>,]<.
because your answer was getting one too far ;)
Depending on the interpreter, you might have to escape the return key by yourself. considering the return key is ASCII: 10, your code should look like this :
>,----- -----[+++++ +++++>,----- -----]<.
broken down :
> | //first operation (just in case your interpreter does not
support a negative pointer index)
,----- ----- | //first entry if it's a return; you don't even get in the loop
[
+++++ +++++ | //if the value was not ASCII 10; you want the original value back
>, | //every next entry
----- ----- | //check again for the the return,
you exit the loop only if the last entered value is 10
]
<. | //your current pointer is 0; you go back to the last valid entry
and you display it
Your issue is that a loop continues for forever until at the end of the loop the cell the pointer is currently on in equal to 0. Your code never prints in the loop, and never subtracts, so your loop will never end, and all that your code does is take an ASCII character as input, move one forward, take an ASCII character as input, and so on. All of your code after the end of the loop is useless, because that your loop will never end.

xText Variable/Attribute Assignment

I built a grammar in xText to recognize formal expressions of a specific format
and to use the generated object tree in Java.
This is what it looks like:
grammar eu.gemtec.device.espa.texpr.Texpr with org.eclipse.xtext.common.Terminals
generate texpr "http://www.gemtec.eu/device/espa/texpr/Texpr"
Model:
(expressions+=AbstractExpression)*
;
AbstractExpression:
MatcherExpression | Assignment;
MatcherExpression:
TerminalMatcher ({Operation.left=current} operator='or' right= MatcherExpression)?
;
TerminalMatcher returns MatcherExpression:
'(' MatcherExpression ')' | {MatcherLiteral} value=Literal
;
Literal:
CharMatcher | ExactMatcher
;
CharMatcher:
type=('text'|'number'|'symbol'|'whitespace') ('(' cardinality=Cardinality ')')?
;
/* Kardinalitäten für CharMatcher*/
Cardinality:
CardinalityMin | CardinalityMinMax | CardinalityMax| CardinalityExact
;
CardinalityMin: min=INT '->';
CardinalityMinMax: min=INT '->' max=INT;
CardinalityMax: '->' max=INT;
CardinalityExact: exact=INT;
ExactMatcher:
(ignoreCase='ignoreCase''(' expected=STRING ')') | expected=STRING
;
/* Variablenzuweisung
*
* z.B. $myVar=number
* */
Assignment:
'$' name=ID '=' expression=MatcherExpression
;
Everything works fine except for the 'cardinality' assignment.
The Expressions look like this:
text number(3) - (an arbitrary amount of letters followed by exactly 3 numbers)
symbol number(2->) - (an arbitrary amount of special characters followed by at least 2 numbers)
whitespace number(->4) - (an arbitrary amount of whitespaces followed by a maximum of 4 numbers)
number(3->6) - (at least 3 numbers but not more than 6)
When I run Eclipse with this grammar (so that my language is recognized and has code completion and so on), everything I type is shown in the "Outline"-tab as a tree-structure as it should, except for the cardinality values.
When I add a cardinality statement to a CharMatcher, the little plus appears before it, but when I click on it it just disappears.
Can anyone tell me why this does not work?
I found the solution myself, I think the problem was that the compiler could not decide which class to create at this point:
Cardinality:
CardinalityMin | CardinalityMinMax | CardinalityMax| CardinalityExact
;
CardinalityMin: min=INT '->';
CardinalityMinMax: min=INT '->' max=INT;
CardinalityMax: '->' max=INT;
CardinalityExact: exact=INT;
So I simplified the whole thing a little, it now looks like this:
Cardinality:
CardinalityMinMax | CardinalityExact
;
CardinalityMinMax: (min=INT '..' max=INT) | (min=INT '..') | ('..' max=INT);
CardinalityExact: exact=INT;
It is still not shown in the "Outline"-Tab, but I suppose that is a problem of the visualisation.
The generated classes now work as intended.