Trying to pass sum or multiply number to subtract from date dynamically in blue pipeline as below:
#concat(
'RANGE:',
1+((1-1)*(variables('totalcount')/20)),
':',
variables('totalcount'),
':50'
)
The above expression says Unrecognized expression: 1+((1-1)*(variables('totalcount')/20))
The math functions in Data Factory are add, mul, div, etc. You cannot use the *,+,/ literals. Here is the converted expression
#concat(
'RANGE:',
string(add(1,mul(sub(1,1),div(int(variables('totalcount')),20)))),
':',
variables('totalcount'),
':50'
)
which gives the result "RANGE:1:18000:50". You did not specify the expected result so don't know if that is what you want. The expression I changed will always result in a value of '1' because of the 1-1 part.
Related
Using q’s like function, how can we achieve the following match using a single regex string regstr?
q) ("foo7"; "foo8"; "foo9"; "foo10"; "foo11"; "foo12"; "foo13") like regstr
>>> 0111110b
That is, like regstr matches the foo-strings which end in the numbers 8,9,10,11,12.
Using regstr:"foo[8-12]" confuses the square brackets (how does it interpret this?) since 12 is not a single digit, while regstr:"foo[1[0-2]|[1-9]]" returns a type error, even without the foo-string complication.
As the other comments and answers mentioned, this can't be done using a single regex. Another alternative method is to construct the list of strings that you want to compare against:
q)str:("foo7";"foo8";"foo9";"foo10";"foo11";"foo12";"foo13")
q)match:{x in y,/:string z[0]+til 1+neg(-/)z}
q)match[str;"foo";8 12]
0111110b
If your eventual goal is to filter on the matching entries, you can replace in with inter:
q)match:{x inter y,/:string z[0]+til 1+neg(-/)z}
q)match[str;"foo";8 12]
"foo8"
"foo9"
"foo10"
"foo11"
"foo12"
A variation on Cillian’s method: test the prefix and numbers separately.
q)range:{x+til 1+y-x}.
q)s:"foo",/:string 82,range 7 13 / include "foo82" in tests
q)match:{min(x~/:;in[;string range y]')#'flip count[x]cut'z}
q)match["foo";8 12;] s
00111110b
Note how unary derived functions x~/: and in[;string range y]' are paired by #' to the split strings, then min used to AND the result:
q)flip 3 cut's
"foo" "foo" "foo" "foo" "foo" "foo" "foo" "foo"
"82" ,"7" ,"8" ,"9" "10" "11" "12" "13"
q)("foo"~/:;in[;string range 8 12]')#'flip 3 cut's
11111111b
00111110b
Compositions rock.
As the comments state, regex in kdb+ is extremely limited. If the number of trailing digits is known like in the example above then the following can be used to check multiple patterns
q)str:("foo7"; "foo8"; "foo9"; "foo10"; "foo11"; "foo12"; "foo13"; "foo3x"; "foo123")
q)any str like/:("foo[0-9]";"foo[0-9][0-9]")
111111100b
Checking for a range like 8-12 is not currently possible within kdb+ regex. One possible workaround is to write a function to implement this logic. The function range checks a list of strings start with a passed string and end with a number within the range specified.
range:{
/ checking for strings starting with string y
s:((c:count y)#'x)like y;
/ convert remainder of string to long, check if within range
d:("J"$c _'x)within z;
/ find strings satisfying both conditions
s&d
}
Example use:
q)range[str;"foo";8 12]
011111000b
q)str where range[str;"foo";8 12]
"foo8"
"foo9"
"foo10"
"foo11"
"foo12"
This could be made more efficient by checking the trailing digits only on the subset of strings starting with "foo".
For your example you can pad, fill with a char, and then simple regex works fine:
("."^5$("foo7";"foo8";"foo9";"foo10";"foo11";"foo12";"foo13")) like "foo[1|8-9][.|0-2]"
I'm trying to understand the indexof expression(function) of Azure Data Factory.
Example
This example finds the starting index value for the "world" substring in the "hello world" string:
indexOf('hello world', 'world')
And returns this result: 6
I'm confused by what is meant by the 'index value' and how the example arrived at the result 6.
Also, using the above example, can someone let me know what would be the answer for the following expression?
#if(greater(indexof(string(pipeline().parameters.Config),'FilenameMask'),0),pipeline().parameters.Config.FilenameMask,'')
indexof
{"FilenameMask":"accounts*."}
'Config' represents a field in sql database
Per the docs:
Return the starting position or index value for a substring. This function is not case-sensitive, and indexes start with the number 0.
hello world
01234567890
^
+--- "world" found starting at position 6
Regarding the 2nd part of your question. Here's the expression re-written for a bit of clarity:
#if( greater(indexof(string(pipeline().parameters.Config),'FilenameMask'),0)
,pipeline().parameters.Config.FilenameMask
,'')
which can be read as follows:
if the index of the string "FilenameMask" within x is greater than 0 then
return x.Filenamemask
else
return an empty string
where x is pipeline().parameters.Config, which is the value of your "Config" column from the database table. It will hold values such as
{"sparkConfig":{"header":"true"},"FilenameMask":"cashsales*."}
and
{"FilenameMask":"accounts*."}
The ADF expression can also be read as follows:
if the JSON in the Config column contains a "FilenameMask" key then
return the value of the FilenameMask key
else
return an empty string
Trying to convert the number numeric value in this string to a percent. Is there any easy way to do this in powershell?
"Percentage of records ","0.02"
So, the output would look like :
Percentage of records , 2%
Thanks in advance for any suggestions you can provide.
Yes, you can convert the string to a float data type (single, double, decimal), and then convert it back using a format string, like so:
"Percentage of records ", ([double]'0.02').ToString('P0')
And if you want it to output in a single line, you could use:
"Percentage of records: $(([double]'0.02').ToString('P0'))"
Explanation:
Convert your string to a float datatype: [double]'0.02'
Convert that float back into a string: .ToString()
But we want to format it as a percentage, so we supply P0 as a parameter.
i. P - means to format the value as a percentage, this performs the N * 100 operation for you and then adds on the percent sign
ii. 0 - controls the number of decimal places to show. In your case, you want to show zero decimal places.
Note: The percentage format string will round your value to the nearest decimal that you specify.
Example:
0.021.ToString('P0')
# returns 2%
0.025.ToString('P0')
# returns 3%
As #mklement0 pointed out in the comments. I hadn't considered that your sample may be a single string, like:
'"Percentage of records ","0.02"'
I assumed it was two strings, which you separated with a comma.
In the event it is a single string, then you need to extract the number to use it. Once you have isolated the number, then you can use my advice above:
$yourString = '"Percentage of records ","0.02"'
# probably the more "proper" way
$pctValue = ($yourString -split ',' -replace '"')[1]
# or
# a hacky way I just thought of that happens to work in this scenario
$pctValue = (iex $yourString)[1]
Explanation of first example:
-split ',' - Take the string, and break it out into multiple strings, separating them by comma
-replace '"','' - Replace all instances of " with blank. The second parameter is optional since you are removing. Could be written as -replace '"'
(...)[1] - This is saying to take the SECOND string that it returned (starts at zero). In this case it would be your 0.02 value.
Explanation of second example (this is a bit of a hack, but thought it would be fun to include anyway):
iex - alias for Invoke-Expression - it's telling powershell to run whatever is inside of the string verbatim. So it's the equivalent of typing "Percentage of records ","0.02" into powershell and hitting enter. Which in PowerShell terms, that is the equivalent of passing it a list of strings.
Use -f (format operator) in powerhsell for build your string :
"Percentage of records, {0:0%} " -f 0.02
or in percentage :
"Percentage of records, {0:P0} " -f 0.02
I would like to have an expression equal to MyParmater + '_' + utcnow()
My current attempt is: #{pipeline().parameters.Col}_{utcnow()}
but iy fails
Use the concat function:
#concat(pipeline().parameters.Col,'_',utcNow())
I have a script which does not fully work:
inputfield=input('Which field would you like to see: ','s')
if isfield(package, inputfield)
fprintf('The value of the %s field is: %c\n',inputfield,...
eval(['package.' inputfield]))
else
fprintf('Error: %s is not valid field\n', inputfield)
end
First I define a structure in matlab and then i use the script on the structure:
package=struct('item_no',123,'cost',19.99,'price',39.95,'code','g')
package =
item_no: 123
cost: 19.9900
price: 39.9500
code: 'g'
structurevalue
Which field would you like to see: cost
inputfield =
cost
The value of the cost field is: 1.999000e+001
structurevalue
Which field would you like to see: item_no
inputfield =
item_no
The value of the item_no field is: {
why cant it read value for item_no?
Try:
fprintf('The value of the %s field is: %s\n',inputfield,...
num2str(package.(inputfield)))
There were two issues with your version.
You were passing both numbers and strings into the %c field in your fprintf string. When a decimal goes in, it is interpreted as a number and displayed in full precision, which is why 19.99 got displayed as 1.999000e+001. But when an integer goes in, it gets interpreted as a character, which is why 123 got displayed as '{' (ASCII character 123). Use num2str to convert numbers to strings for display. Also, use %s for a string of any length, rather than %c for a character.
In general, it's not a good idea to use eval unless you have to. In this case, it's more convenient to use inputfield as a dynamic field name of package.