Using the toInteger function with locale and format parameters - azure-data-factory

I've got a dataflow with a csv file as source. The column NewPositive is a string and it contains numbers formatted in European style with a dot as thousand seperator e.g 1.019 meaning 1019
If I use the function toInteger to convert my NewPositive column to an int via toInteger(NewPositive,'#.###','de'), I only get the thousand cipher e.g 1 for 1.019 and not the rest. Why? For testing I tried creating a constant column: toInteger('1.019','#.###','de') and it gives 1019 as expected. So why does the function not work for my column? The column is trimmed and if I compare the first value with equality function: equals('1.019',NewPositive) returns true.
Please note: I know it's very easy to create a workaround by toInteger(replace(NewPositive,'.','')), but I want to learn how to use the toInteger function with the locale and format parameters.
Here is sample data:
Dato;NewPositive
2021-08-20;1.234
2021-08-21;1.789

I was able to repro this and probably looks to be a bug to me . I have reported this to the ADF team , will let you know once I hear back from them . You already have a work around please go ahead that to unblock yourself .

Related

How do I prevent users to use thousands separator in FileMaker Pro?

In FileMaker Pro, when using number field, the user can choose to use a thousand separator or not. For example, if I have a database with a field for the price of an item, the user can either enter 1,000 or 1000.
I am using my database to generate an XML file that needs to be uploaded. The thing is, that my XML scheme dictates that only a value of 1000 is allowed and not 1,000. Therefore, I want to either automatically remove the comma, or (my preference in this case) alert the user when trying to enter a value with a thousand separator.
What I tried is the following.
For the field, I am setting Validation options. For example:
Require Strict data type: Numeric Only
Validated by calculation: Position ( Self ; ","; 1 ; 1 ) = 0
Validated by calculation: Self = Substitue ( Self, ",", "")
Auto-enter calculation: Filter( Self ; "0123456789." )
Unfortunately, none of these work. As the field is defined as a number (and I want to keep it like this, as I am also performing calculations based on this number), the Position function and the Substitute function apparently ignore the thousand separator!
EDIT:
Note that I am generating my XML by concatenating a string, for example:
"<Products><Product><Name>" & Name & "</Name><Price>" & Price & "</Price></Product></Product>"
The reason is that what I am exporting is dependent on the values in my database. Therefore, I am not using the [File][Export records...] function.
Auto-enter calculation will work, but you need to uncheck the box "Do not replace existing value of field" (which is checked by default).
I'd suggest using the calculation GetAsNumber(self) as the auto-enter calc. If it should only contain integers, wrap that in a call to Int()
I am using my database to generate an XML file that needs to be uploaded. The thing is, that my XML scheme dictates that only a value of 1000 is allowed and not 1,000.
If this is only a problem when you export, why not handle it when exporting?
If you are exporting as XML using XSLT, you can add an instruction to
your stylesheet to remove the comma from all number fields;
Alternatively, you can export from a layout where the field is
formatted to display without the comma and select the Apply current's layout data formatting to exported data option when
exporting.
Added:
Perhaps I should have clarified. I am not using the export function to generate the XML as there is some logic involved in how the XML should be formatted (dependent on the data that I want to export). What I do instead is that I make a string where I combine XML-tags and actual values from the database.
IMHO, you're making a mistake by not taking advantage of the built-in XML/XSLT export option. Any imaginable logic can be implemented this way, without burdening your solution with the fragile task of creating a valid XML.
In any case, if you're using the field in a calculation, you can replace all references to it with:
GetAsNumber (YourField )
to get an unformatted, numeric-only, value.
Your question puzzles me. As far as I know, FileMaker does not store the thousands separator, but rather offers it only as a display option.
That's also why those functions can't find it.
Are you sure you are exporting the raw data and not a "formatted as layout" variant?

Why does Open XML API Import Text Formatted Column Cell Rows Differently For Every Row

I am working on an ingestion feature that will take a strongly formatted .xlsx file and import the records to a temp storage table and then process the rows to create db records.
One of the columns is strictly formatted as "Text" but it seems like the Open XML API handles the columns cells differently on a row-by-row basis. Some of the values while appearing to be numeric values are truly not (which is why we format the column as Text) -
some examples are "211377", "211727.01", "209395.388", "209395.435"
what these values represent is not important but what happens is that some values (using the Open XML API v2.5 library) will be read in properly as text whether retrieved from the Shared Strings collection or simply from InnerXML property while others get sucked in as numbers with what appears to be appended rounding or precision.
For example the "211377", "211727.01" and "209395.435" all come in exactly as they are in the spreadsheet but the "209395.388" value is being pulled in as "209395.38800000001" (there are others that this happens to as well).
There seems to be no rhyme or reason to which values get messed up and which ones which import fine. What is really frustrating is that if I use the native Import feature in SQL Server Management Studio and ingest the same spreadsheet to a temp table this does not happen - so how is that the SSMS import can handle these values as purely text for all rows but the Open XML API cannot.
To begin the answer you main problem seems to be values,
"209395.388" value is being pulled in as "209395.38800000001"
Yes in .xlsx file value is stored as 209395.38800000001 instead of 209395.388. And it's the correct format to store floating point numbers; nothing wrong in it. You van simply confirm it by following code snippet
string val = "209395.38800000001"; // <= What we extract from Open Xml
Console.WriteLine(double.Parse(val)); // < = Simply pass it to double and print
The output is :
209395.388 // <= yes the expected value
So there's nothing wrong in the value you extract from .xlsx using Open Xml SDK.
Now to cells, yes cell can have verity of formats. Numbers, text, boleans or shared string text. And you can styles to a cell which would format your string to a desired output in Excel. (Ex - Date Time format, Forced strings etc.). And this the way Excel handle the vast verity of data. It need this kind of formatting and .xlsx file format had to be little complex to support all.
My advice is to use a proper parse method set at extracted values to identify what format it represent (For example to determine whether its a number or a text) and apply what type of parse.
ex : -
string val = "209395.38800000001";
Console.WriteLine(float.Parse(val)); // <= Float parse will be deduce a different value ; 209395.4
Update :
Here's how value is saved in internal XML
Try for yourself ;
Make an .xlsx file with value 209395.388 -> Change extention to .zip -> Unzip it -> goto worksheet folder -> open Sheet1
You will notice that value is stored as 209395.38800000001 as scene in attached image.. So nothing wrong on API for extracting stored number. It's your duty to decide what format to apply.
But if you make the whole column Text before adding data, you will see that .xlsx hold data as it is; simply said as string.

Substring and adding leading zeros to a string value

Hi all – I need some assistance with prefixing leading zeros into a string.
I have had a look around in the forum but did not quite find anything that suits my scenario.
The string in the column is in the following format: ‘INV-ACC-180 Some description etc’
The ‘INV-ACC-180’ bit is always the same format whilst the description can vary.
The challenge is the ‘180’. Its needs to be 4 leading zeros so it shows as 0180. i.e ‘INV-ACC-0180 Some description etc’
Some records may look like ‘INV-ACC-80 Some description etc’. And in this case the new correct format should be ‘INV-ACC-0080 Some description etc’
Many thanks in advance!
Is the 180 stored as a separate value ?
In that case you create 0180 using:
RIGHT('0000' + mynum, 4)
if the complete string is the input:
‘INV-ACC-' + RIGHT('0000'+substring(myinput,9,charindex(' ',myinput)-9),4) +substring(myinput,charindex(' '),1000)

Crystal Formula to exclude if value does not contain a certain format

I'm trying to exclude any value from a certain field (table.value) that does not match this format AA#####A. Example if they entered APT12345T, or PT12345PT and No Value then I want to exclude it from the report. It needs to match example AP12345P. What selection formula can I use to accomplish this.
Any help is greatly appreciated
Thanks in advance.
try reading Crystal's help topics on the mid() and isnumeric() functions.
here's an example from the help file:
Examples The following example is applicable to both Basic and Crystal
syntax:
Mid("abcdef", 3, 2)
Returns "cd".
so, in your case, you want to strip your value into three pieces,
mid(table.value,1,2)
mid(table.value,3,5)
mid(table.value,8,1)
and build up a three-part boolean variable where:
the first piece is not numeric(), or between 'AA' and 'ZZ', or however
else you want to test for letters,
the second part isnumeric(), and
the third part passes the same test as the first part.
where are you getting stuck?
something like this:
not isnumeric(mid({table.field},1,2)) and
isnumeric(mid({table.field},3,5) and
not isnumeric(mid({table.field},8,1))

Excel SUM current column (via Excel::Template)

I'm using Excel::Template to generate a series of Excel files via perl. However, I need to do a SUM function on the current Column. I know I can do
=SUM(3:15)
but that gives the sum of ALL columns in rows 3-15. Is there an easier way to do what I'm trying to do?
=sum(indirect(concatenate(address(<row_start>,column()),":")&address(<row_end>,column())))
gives me exactly what I need. Not exactly sure how it works, but found on MrExcel.com
For column C,
=SUM(C3:C15)
Since =SUM(...) is just a string, you may have to parametrize the column if you don't know it before runtime. For instance
$str = "=SUM(" . col_char . "3:" . col_char . "15)";