I have a csv file where amount and quantity fields are present in each detail record except header and trailer record. Trailer record has a total charge values which is the total sum of quantity multiplied by amount field in detail records . I need to check whether the trailer total charge value is equal to my calculated value of amount and quantity fields. I am using the double data type for all these calculations. When i browsed i am able to understand from the below web link that it might create an issue using double datatype while comparison with decimal points. It's suggesting to using BigDecimal
http://epramono.blogspot.com/2005/01/double-vs-bigdecimal.html
Will i get issues if i use double data type. How can i do the calculations using BigDecimal. Also i am not sure how many digits i will get after decimal points in csv file. Also amount can have a positive or negative value.
In csv file
H,ABC.....
"D",....,"1","12.23"
"D",.....,"3","-13.334"
"D",......,"2","12"
T,csd,123,12.345
------------------------------ While Validation i am having the below code --------------------
double detChargeCount =0;
//From csv file i am reading trailer records charge value
String totChargeValue = items[3].replaceAll("\"","").trim();
if (null != totChargeValue && !totChargeValue.equals("")) {
detChargeCount = new Double(totChargeValue).doubleValue();
if(detChargeCount==calChargeCount)
validflag=true;
-----------------------While reading CSV File i am having the below code
if (null != chargeQuan && !chargeQuan.equals("")) {
tmpChargeQuan=Long(chargeQuan).longValue();
}
if (null != chargeAmount && !chargeAmount.equals("")) {
tmpChargeAmt=new Double(chargeAmount).doubleValue();
calChargeCount=calChargeCount+(tmpChargeQuan*tmpChargeAmt);
}
I had declared the variables tmpChargeQuan, tmpChargeAmt, calChargeCount as double
Especially for anything with financial data, but in general for everything dealing with human readable numbers, BigDecimal is what you want to use instead of double, just as that source says.
The documentation on BigDecimal is pretty straight-forward, and should provide everything you need.
It has a int, double, and string constructors, so you can simply have:
BigDecimal detChargeCount = new BigDecimal(0);
...
detChargeCount = new BigDecimal(totChargeValue);
The operators are implemented as functions, so you'd have to do things like
tmpChargeQuan.multiply(tmpChargeAmt)
instead of simply tmpChargeQun * tmpChargeAmt, but that shouldn't be a big deal.
but they're all defined with all the overloads you could need as well.
It is very possible that you will have issues with doubles, by which I mean the precomputed value and the newly computed value may differ by .000001 or less.
If you don't know how the value you are comparing to was computed, I think the best solution is to define "equal" as having a difference of less than epsilon, where epsilon is a very small number such as .0001.
I.e. rather than using the test A == B, use abs(A - B) < .0001.
Related
when using "chained" double generators with jqwik I get a scale error message java.util.concurrent.ExecutionException: net.jqwik.api.JqwikException: Decimal value -1.6099999999999999 cannot be represented with scale 4..
Can you provide me with some details on how to set this scale and the meaning of this parameter ?
Here is the generator function I use :
#Provide("close doubles")
Arbitrary<Tuple.Tuple2<Double,Double>> closeDoubles(#ForAll() Double aDouble) {
return Arbitraries.doubles()
.between(aDouble-2.5, aDouble+2.5)
.withSpecialValue(aDouble)
.ofScale(4)
.map(num ->Tuple.of(aDouble,num));
}
It is then combined to form a business object instance.
My ultimate goal is to generate 2 doubles that are "close" to each other (here the distance is 2.5).
The problem you encounter is due to rounding errors of double numbers and the fact that jqwik is strict with allowing only upper and lower boundaries that comply with the specified scale.
I see several options to get around that, one is to use BigDecimals for generation and map them to double afterwards. This may look like overhead but actually it is not because that's what jqwik is doing anyway under the hood. This could look like:
#Provide
Arbitrary<Tuple.Tuple2<Double, Double>> closeDoubles(#ForAll #Scale(4) BigDecimal aBigDecimal) {
BigDecimal twoPointFive = new BigDecimal("2.5");
return Arbitraries.bigDecimals().between(aBigDecimal.subtract(twoPointFive), aBigDecimal.add(twoPointFive))
.ofScale(4)
.map(num -> Tuple.of(aBigDecimal.doubleValue(), num.doubleValue()));
}
Mind that the original number should also use the same scale as the target numbers, otherwise it will have a default scale of 2.
Personally, I'd prefer to generate a number and the delta, which has improved shrinking behaviour and will create a tuple with identical numbers more often:
#Provide
Arbitrary<Tuple.Tuple2<Double, Double>> closeDoubles2(#ForAll #Scale(4) BigDecimal aBigDecimal) {
BigDecimal twoPointFive = new BigDecimal("2.5");
return Arbitraries.bigDecimals().between(twoPointFive.negate(), twoPointFive)
.ofScale(4)
.map(num -> Tuple.of(aBigDecimal.doubleValue(), aBigDecimal.add(num).doubleValue()));
}
So I am following this course called "Code With Chris - 14 Day Beginner Challenge (SwiftUI)" (yes I am a beginner), and after each lesson, there is a challenge, I have almost completed the challenge but I couldn't figure out why it wouldn't work, so I checked the dropbox of the completed challenge and I had everything pretty much the same, I have found a solution similar to the source but I still don't understand why my first version (first picture) won't work. I copied everything identically from the source code and it won't work. Is there a possibility that it is the creators of the source code fault, instead of mine?
My expected result is for the "Int" to work just like the "Double" did, The number of people is 5 so I don't see why it wouldn't.
My actual result is an error.
My goal is to complete this challenge:
We’re going to be trying out some math operations in a Swift Playground.
Open Xcode and create a new playground
(File Menu->New->Playground).
From the list of Playground templates, just select “Blank”
Challenge 1
Declare a struct called TaxCalculator
Declare a property inside called tax and set it to a decimal value representing the amount of sales tax where you live
Declare a method inside called totalWithTax that accepts a Double as an input parameter and returns a Double value.
Inside that method, write the code to return a Double value representing the input number with tax included
Challenge 2
Declare a struct called BillSplitter
Declare a method inside called splitBy that:
has an input parameter of type Double representing a subtotal
has an input parameter of type Int representing the number of people
returns a Double value
Inside that method, use an instance of TaxCalculator (from challenge 1 above) to calculate the total with tax and then split the bill by the number of people passed into the method.
Return the amount that each person has to pay.
Challenge 3
Create an instance of BillSplitter
Use the instance to print out the amount that each person pays (Assuming 5 people with a bill of $120)
The Code of the course I am using:
https://www.dropbox.com/sh/7aopencivoiegz4/AADbxSj83wt6mPNNgYcARFAsa/Lesson%2009?dl=0&file_subpath=%2FL9+Challenge+Solution.playground%2FContents.swift&preview=L9+Challenge+Solution.zip&subfolder_nav_tracking=1
an image of the code with an error
an image of the code without an error
//https://learn.codewithchris.com/courses/take/start/texts/18867185-lesson-9-challenge
//Challenge1
struct TaxCalculator{
var tax = 0.15
func totalWithTax(_ subtotal:Double) -> Double{
return subtotal * (1 + tax)
}
}
//Challenge2
struct BillSplitter {
func splitBy(subtotal:Double, numPeople:Int //here is the problem) ->Double {
let taxCalc = TaxCalculator()
let totalWithTax = taxCalc.totalWithTax(subtotal)
return totalWithTax/numPeople
}
}
let Split = BillSplitter()
print(Split.splitBy(subtotal: 120, numPeople: 5))
totalWithTax is a Double. numPeople is an Int.
You need to convert numPeople to a Double too.
return totalWithTax / Double(numPeople)
Operators like / don't work with mismatching types.
Swift is a bit of a pain with scalar types. Most C family languages will quietly "promote" scalar types to other types as long as there is no loss of data.
byte->int->long int->float->double all happen silently.
In C, this code just works:
int a = 2;
double b = 2.5;
double c = a * b;
The value a gets promoted to a double, and the result is that contains the double value 5.0.
Not so with Swift.
In Swift, you have to explicitly cast a to a double. It won't let you multiply an Int and a Double unless you explicitly cast the Int to a Double, as aheze said in their answer:
return totalWithTax / Double(numPeople)
I want to return a percentage of results from a dataset. Being a noob in Scala, tried the following
ds.filter(abs(hash(col("source"))) % 100 < percentage)
but getting abs cannot be applied to (org.apache.spark.sql.Column). I don't want to sample it, I want to return based on the hash of a column so that it's deterministic even when dataset changes.
This works just fine:
ds.filter(abs(hash(col("source"))) % 100 < percentage)
Probabely you have multiple abs in your namespace (e.g. from imports like import math._ etc. To be sure, use
ds.filter(org.apache.spark.sql.functions.abs(hash(col("source"))) % 100 < percentage)
But I think this will not garantee that you get the exact percentage, because hash values may not be equally distributed (think about a dataframe with only 1 unique value of source, hash values will all be the same.... you get either all records or none. To get the exact percentage, you would need something like :
val newDF = df
.withColumn("rnb",row_number().over(Window.orderBy($"source"))) // or order by hash if you wish
.withColumn("count",count("*").over())
.where($"rnb" < lit(fraction)*$"count")
I know this is elementary but I can't seem to figure it out, even after reading other posts.
In a dataset, I want to convert an entire column into a date. The current class is factor.
The value in the field looks like this 12/25/2012
This is what I've tried.
C$DateofDeath=as.Date(C$DateofDeath,'%m/%d/%Y')
Error in as.Date.default(C$DateofDeath, "%m/%d/%Y") :
do not know how to convert 'C$DateofDeath' to class “Date”
C$DateofDeath=as.Date(C$DateofDeath,"%m/%d/%Y")
Error in as.Date.default(C$DateofDeath, "%m/%d/%Y") :
do not know how to convert 'C$DateofDeath' to class “Date”
Claims$DateofDeath=strptime(as.character(Claims$DateofDeath),format= '%m/%d/%Y')
Error in `$<-.data.frame`(`*tmp*`, "DateofDeath", value = list(sec = numeric(0), :
replacement has 0 rows, data has 71616
Claims$DateofDeath=strptime(as.character(Claims$DateofDeath),format= "%m/%d/%Y")
Error in `$<-.data.frame`(`*tmp*`, "DateofDeath", value = list(sec = numeric(0), :
replacement has 0 rows, data has 71616
Use as.POSIXct
C$DateOfDeath<-as.POSIXct(as.character(C$DateOfDeath), format = "%d/%m/%Y")
There are lots of R experts here but you have to specify R as one of your tags to get them to notice your question.
Looks like you have tried a bunch of combinations but not the right one.
> C <- data.frame(DateofDeath="12/25/2012",other=TRUE)
> as.Date(as.character(C$DateofDeath),format="%m/%d/%Y")
[1] "2012-12-25"
Notice that as.Date() takes a character input, not a factor. So you need to convert to character, then to Date.
Your strptime() versions seem fine to me except that you call are referring to the dataframe Claims instead of C. Actually strptime() should convert the factor to character for you, so you don't need the as.character() part with those.
I am new at Matlab and I am currently working with financial data exporting from financial times website. I would like to know how can I get, for example, share price forecast information from this page
http://markets.ft.com/research/Markets/Tearsheets/Forecasts?s=DIS:NYQ
High +34.7 % 85.00
Med +15.7 % 73.00
Low -9.6 % 57.00
And save this information as a variables.
Here's a simple solution using urlread and regexpi:
% Create URL string and read in HTML
ftbaseurl = 'http://markets.ft.com/research/Markets/Tearsheets/Forecasts?s=';
ticksym = 'DIS:NYQ';
s = urlread([ftbaseurl ticksym]);
% Create pattern string for regular expression matching
trspan = '<tr><td class="text"><span class="';
tdspan1 = '</span></td><td><span class="\w\w\w color ">'; % \w\w\w matchs pos or neg
matchstr1 = '(?<percent>[\+|\-]*\d+.\d+)'; % percent: match (+or-)(1+ digits).(1+ digits)
tdspan2 = ' %</span></td><td>';
matchstr2 = '(?<price>\d+\.\d\d)</td></tr>'; % price: match (1+ digits) . 2 digits
pat = [trspan 'high">High' tdspan1 matchstr1 tdspan2 matchstr2 '|' ...
trspan 'med">Med' tdspan1 matchstr1 tdspan2 matchstr2 '|' ...
trspan 'low">Low' tdspan1 matchstr1 tdspan2 matchstr2];
% Match patterns in HTML, case insensitive, put results in struct array
forecasts = regexpi(s,pat,'names');
The result is a 1-by-3 struct array where each element has two fields, 'percent' and 'price', that each contain strings extracted by the regular expression parser. For example
>> forecasts(3)
ans = percent: '-10.3'
price: '57.00'
>> str2double(forecasts(3).percent)
-10.3000
I'll leave it to you to convert the strings to numbers (note that financial software usually stores prices in integer cents (or what ever the lowest denomination is) rather than floating point dollars to avoid numerical issues) and to turn this into a general function. Here's some more information on regular expressions in Matlab.
My comment above still stands. This is very inefficient. You're downloading the entire webpage HTML and parsing it in order to find a few small bits of data. This is fine if this doesn't update very often or if you don't need it to be very fast. Also, this scheme is fragile. If the Financial Times updates their website, it may break the code. And if you try downloading their regular webpages very often they may also have means of blocking you.