I am trying to have some feedback about the best way to organize data for an insurance company (and in general).
I am using MongoDB and i'm unsure about a thing.
lets say i have different prices for different coverages on the same policy, on top of that various cases based on the annual income of the company who will sign the policy.
#1. Coverage -> 5.000.000 - Annual income -> 1.000.000 - price -> 500
#2. Coverage -> 5.000.000 - Annual income -> 1.500.000 - price -> 550
Then i can add extensions to that policy (in this case just 4 types)
#1.Basic extension -> 120 - Advanced extension -> 150
#1.Basic extension -> 150 - Advanced extension -> 170
I was thing about saving every policy in a collection, everyone with their characteristics.
So, lets take this one as an example.
Collection: policies
Entry:
id
name: name of the policy as it is written in the docs
prices: [{id: 0, price: 500, income_range: [0, 1.000.000], coverage_range: [0,5.000.000], extensions: [120, 150]}, {{id: 1, price: 550, income_range: [1.000.001, 1.500.000], coverage_range: [0,5.000.000], extensions: [150, 170]}
other_general_stuff
OR
Collection: policies
Entry:
id
name: name of the policy as it is written in the docs
prices: [{id: 0, price: 500, income_range: 0, coverage_range: 0, extensions: l1}, {{id: 1, price: 550, income_range: 1, coverage_range: 0, extensions: l2}
income_ranges: [{id: 0, range: [0, 1.000.000]}, {id: 1, range: [1.000.001, 1.500.000]}]
cover_ranges: [{id: 0, range: [0, 5.000.000]}, {id: 1, range: [5.000.000, 7.000.000]}]
extensions: {l1: {basic: 120, adv: 150}, l2: {basic: 150, adv: 170}}
I mean considering there are different products this i just a tiny example, but what concerns me is for ex. the ranges property, is it ok in this type of db to scructure it like the second example, is there a way of accessing those property in an easy way, or is it better to write the entire thing directly into the 'prices field'.
I have not much experience, so I am trying to figure out what is the best practice to follow.
I know that there are many possible solutions, but is this entirely wrong? Can someone give me a feedback or point me to some nice resources.
Thank you
Related
I am using the PMD Java API in order to create some statistics.
I am interested in seeing what are the values of some metrics of interest, for example:
CouplingBetweenObjects, ExcessiveClassLength, CyclomaticComplexity, etc.
The problem is that I could not find any report format which displays the metric's value explicitly.
I could parse the output and deduce the value for some of those rules, but I cannot do that for others.
Does anyone know what to do in order to get that value?
For example, this is a json report format:
{
"beginline": 1,
"begincolumn": 1,
"endline": 181,
"endcolumn": 2,
"description": "High amount of different objects as members denotes a high coupling",
"rule": "CouplingBetweenObjects",
"ruleset": "Design",
"priority": 3,
"externalInfoUrl": "https://pmd.github.io/pmd-6.46.0/pmd_rules_java_design.html#couplingbetweenobjects"
},
or
{
"beginline": 10,
"begincolumn": 1,
"endline": 44,
"endcolumn": 1,
"description": "The class \u0027Factory\u0027 has a total cyclomatic complexity of 13 (highest 12).",
"rule": "CyclomaticComplexity",
"ruleset": "Design",
"priority": 3,
"externalInfoUrl": "https://pmd.github.io/pmd-6.46.0/pmd_rules_java_design.html#cyclomaticcomplexity"
},
The cyclomatic complexity can be seen that it's 13 and I can write some code to get that number, but I cannot do that for the coupling between objects.
Exchangers, sites like TradingView.com, etc. Quickly response data for different intervals 15m, 30m, 1h, 4h and more. I have 2 questions about working with data:
They generate large timeframes on the fly using smaller ones or store all intervals in DB?
Recommend the best data structure for NOSQL to store OHLCV data for high performance and data manipulation.
Example structure:
{
symbol: "BTCUSDT",
data: [{
ts: 1,
o: 1,
h: 2,
l: 0.5,
c: 1.5
},
]
}
What can I try next?
I want to make a list of tasks that can change their order, but I am not sure how to store this in a database.
I don't want to use array because I have to do some queries further in future.
Here is the screenshot of my database:
I'm trying to make something like Trello where the user adds tasks and can move tasks upward and downward according to their priority. I need to change the position of the tasks in the database as well to maintain the record. I'm unable to understand how to do that in any database. I'm an experienced developer and I have worked with mongodb and firebase but this is something unique for me.
Here is the code to create and get all tasks. When I try to move some task in collection. I maintained an index in each task.
Let's say when I move a task from the position of index 5 to index 2 then I have to edit all the upcoming indexes by +1. Is there some way I can avoid doing this?
Code Sample
class taskManager {
static let shared = taskManager()
typealias TasksCompletion = (_ tasks:[Task],_ error:String?)->Void
typealias SucessCompletion = (_ error:String?)->Void
func addTask(task:Task,completion:#escaping SucessCompletion){
Firestore.firestore().collection("tasks").addDocument(data: task.toDic) { (err) in
if err != nil {
print(err?.localizedDescription as Any)
}
completion(nil)
}
}
func getAllTask(completion:#escaping TasksCompletion){
Firestore.firestore().collection("tasks")
.addSnapshotListener { taskSnap, error in
taskSnap?.documentChanges.forEach({ (task) in
let object = task.document.data()
let json = try! JSONSerialization.data(withJSONObject: object, options: .prettyPrinted)
var taskData = try! JSONDecoder().decode(Task.self, from: json)
taskData.id = task.document.documentID
if (task.type == .added) {
Task.shared.append(taskData)
}
if (task.type == .modified) {
let index = Task.shared.firstIndex(where: { $0.id == taskData.id})!
Task.shared[index] = taskData
}
})
if error == nil{
completion(Task.shared,nil)
}else{
completion([],error?.localizedDescription)
}
}
}
}
I think the question you're trying to ask about is more about database design.
When you want to be able to keep order with a group of items while being able to reorder them you will need a column to keep the order.
You run into an issue when you try to order them if they are sequentially ordered.
Example
For example if you wanted to move Item1 behind Item4:
Before
An item with an ordering index.
1. Item1, order: 1
2. Item2, order: 2
3. Item3, order: 3
4. Item4, order: 4
5. Item5, order: 5
6. Item6, order: 6
After
Problem: we had to update every record between the item being moved and where it was placed.
Why this is a problem: this is a Big O(n) - for every space we move we have to update that many records. As you get more tasks this becomes more of an issue as it will take longer and not scale well. It would be nice to have a Big O(1) where we have a constant amount of changes or as few as possible.
1. Item2, order: 1 - Updated
2. Item3, order: 2 - Updated
3. Item4, order: 3 - Updated
4. Item1, order: 4 - Updated
5. Item5, order: 5
6. Item6, order: 6
Possible Solution #1 (OK Maybe?) - Spacing
You could try to come up with a crafty method where you try to space the order numbers out so that you have holes that can be filled without updating multiple records.
This could get tricky though, and you may think, "Why not store Item1 at order: 4.5" I added a related question below that goes into that idea and why you should avoid it.
You may be able to verify the safety of the order client side and avoid hitting the database to determine the new order ID of the move.
This also has limitations as you may have to rebalance the spacing or maybe you run out of numbers to items. You may have to check for a conflict and when a conflict arises you perform a rebalance on everything or recursively the items around the conflict making sure that other balancing updates don't cause more conflicts and that additional conflicts are resolved.
1. Item2, order: 200
2. Item3, order: 300
3. Item4, order: 400
4. Item1, order: 450 - Updated
5. Item5, order: 500
6. Item6, order: 600
Possible Solution #2 (Better) - Linked Lists
As mentioned in the related link below you could use a data structure like a linked list. This retains a constant amount of changes to update so it is Big O(1). I will go into a linked list a bit in case you haven't played with the data structure yet.
As you can see below this change only required 3 updates, I believe the max would be 5 as shown in Expected Updates. You may be thinking, "Well it took about that many with the first original problem/example!" The thing is that this will always be a max of 5 updates compared to the possibility of thousands or millions with the original approach [Big O(n)].
1. Item2, previous: null, next: Item3 - Updated // previous is now null
2. Item3, previous: Item2, next: Item4
3. Item4, previous: Item3, next: Item1 - Updated // next is now Item1
4. Item1, previous: Item4, next: Item5 - Updated // previous & next updated
5. Item5, previous: Item1, next: Item4 - Updated // previous is now Item1
6. Item6, previous: Item6, next: null
Expected Updates
Item being moved (previous, next)
Old previous item's next
Old next item's previous
New previous item's next
New next item's previous
Linked Lists
I guess I used a double linked list. You probably could get away with just using a single linked list where it doesn't have a previous attribute and only a next instead.
The idea behind a linked list is to think of it a chain link, when you want to move one item you would decouple it from the link in front of it and behind it, then link those links together. Next you would open up where you would want to place it between, now it would have the new links on each side of it, and for those new links they would now be linked to the new link instead of each other.
Possible Solution #3 - Document/Json/Array Storage
You said you want to stay away from arrays, but you could utilize document storage. You could still have a searchable table of items, and then each collection of items would just have an array of item id/references.
Items Table
- Item1, id: 1
- Item2, id: 2
- Item3, id: 3
- Item4, id: 4
- Item5, id: 5
- Item6, id: 6
Item Collection
[2, 3, 4, 1, 5, 6]
Related Question(s)
Storing a reorderable list in a database
Resources on Big O
A guide on Big O
More on Big O
Wiki Big O
Other Considerations
Your database design will depend on what you're trying to accomplish. Can items belong to multiple boards or users?
Can you offload some ordering to the client side and allow it to tell the server what the new order is? You should still avoid inefficient ordering algorithms on the client side, but you can get them to do some of the dirty work if you trust them and don't have any issues with data integrity if multiple people are working on the same items at the same time (those are other design problems, that may or may not be related to the DB, depending on how you handle them.)
I was stuck on the same problem for a long time. The best solution I found was to order them Lexicographically.
Trying to manage a decimal rank (1, 2, 3, 4...) runs into a lot of problems that are all mentioned in other answers on this question. Instead, I store the rank as a string of characters ('aaa', 'bbb', 'ccc'...) and I use the character codes of the characters in the strings to find a spot between to ranks when adjustments are made.
For example, I have:
{
item: "Star Wars",
rank: "bbb"
},
{
item: "Lord of the Rings",
rank: "ccc"
},
{
item: "Harry Potter",
rank: "ddd"
},
{
item: "Star Trek",
rank: "eee"
},
{
item: "Game of Thrones",
rank: "fff"
}
Now I want to move "Game of Thrones" to the third slot, below "Lord of the Rings" ('ccc') and above "Harry Potter" ('ddd').
So I use the character codes of 'ccc' and 'ddd' to mathematically find the average between the two strings; in this case, that ends up being 'cpp' and I'll update the document to:
{
item: "Game of Thrones",
rank: "cpp"
}
Now I have:
{
item: "Star Wars",
rank: "bbb"
},
{
item: "Lord of the Rings",
rank: "ccc"
},
{
item: "Game of Thrones",
rank: "cpp"
},
{
item: "Harry Potter",
rank: "ddd"
},
{
item: "Star Trek",
rank: "eee"
}
If I run out of room between two ranks, I can simply add a letter to the end of the string; so, between 'bbb' and 'bbc', I can insert 'bbbn'.
This is a benefit over decimal ranking.
Things to be aware of
Do not assign 'aaa' or 'zzz' to any item. These need to be withheld to easily allow for moving items to the top or bottom of the list. If "Star Wars" has rank 'aaa' and I want to move something above it, there would be problems. Solvable problems, but this is easily avoided if you start at rank 'bbb'. Then, if you want to move something above the top rank, you can simply find the average between 'bbb' and 'aaa'.
If your list gets reshuffled frequently, it would be good practice to periodically refresh the rankings. If things are moved to the same spot in a list thousands of times, you may get a long string like 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbn'. You may want to refresh the list when a string gets to be a certain length.
Implementation
The algorithm and an explanation of the functions used to achieve this effect can be found here. Credit for this idea goes to the author of that article.
The code I use in my project
Again, credit for this code goes to the author of the article I linked above, but this is the code I have running in my project to find the average between two strings. This is written in Dart for a Flutter app
import 'dart:math';
const ALPHABET_SIZE = 26;
String getRankBetween({String firstRank, String secondRank}) {
assert(firstRank.compareTo(secondRank) < 0,
"First position must be lower than second. Got firstRank $firstRank and second rank $secondRank");
/// Make positions equal
while (firstRank.length != secondRank.length) {
if (firstRank.length > secondRank.length)
secondRank += "a";
else
firstRank += "a";
}
var firstPositionCodes = [];
firstPositionCodes.addAll(firstRank.codeUnits);
var secondPositionCodes = [];
secondPositionCodes.addAll(secondRank.codeUnits);
var difference = 0;
for (int index = firstPositionCodes.length - 1; index >= 0; index--) {
/// Codes of the elements of positions
var firstCode = firstPositionCodes[index];
var secondCode = secondPositionCodes[index];
/// i.e. ' a < b '
if (secondCode < firstCode) {
/// ALPHABET_SIZE = 26 for now
secondCode += ALPHABET_SIZE;
secondPositionCodes[index - 1] -= 1;
}
/// formula: x = a * size^0 + b * size^1 + c * size^2
final powRes = pow(ALPHABET_SIZE, firstRank.length - index - 1);
difference += (secondCode - firstCode) * powRes;
}
var newElement = "";
if (difference <= 1) {
/// add middle char from alphabet
newElement = firstRank +
String.fromCharCode('a'.codeUnits.first + ALPHABET_SIZE ~/ 2);
} else {
difference ~/= 2;
var offset = 0;
for (int index = 0; index < firstRank.length; index++) {
/// formula: x = difference / (size^place - 1) % size;
/// i.e. difference = 110, size = 10, we want place 2 (middle),
/// then x = 100 / 10^(2 - 1) % 10 = 100 / 10 % 10 = 11 % 10 = 1
final diffInSymbols =
difference ~/ pow(ALPHABET_SIZE, index) % (ALPHABET_SIZE);
var newElementCode = firstRank.codeUnitAt(secondRank.length - index - 1) +
diffInSymbols +
offset;
offset = 0;
/// if newElement is greater then 'z'
if (newElementCode > 'z'.codeUnits.first) {
offset++;
newElementCode -= ALPHABET_SIZE;
}
newElement += String.fromCharCode(newElementCode);
}
newElement = newElement.split('').reversed.join();
}
return newElement;
}
There are several approaches you might follow to achieve such functionality.
Approach #1:
You can give your task distant positions instead of continuous position, something like this:
Date: 10 April 2019
Name: "some task name"
Index: 10
...
Index: 20
...
Index: 30
Here are total 3 tasks with position 10, 20, 30. Now lets say you wanted to move third task in the middle, simply change the position to 15, now you have three task with position 10, 15, 20, I am sure you can sort according to the position when getting all tasks from the database, and I also assume that you can get positions of tasks because user will be re arranging the tasks on a mobile app or web app so you can easily get the positions of surrounding tasks and calculate the middle position of surrounding tasks,
Now lets say you wanted to move the first task(which now have possition index 10) in the middle, simply get the positions of surrounding tasks which is 15 and 20 and calculate the middle which is 17.5 ( (20-15)/2=17.5 ) and here you go, now you have positions 15, 17.5, 20
Someone said there is infinity between 1 and 2 so you are not going to run our of numbers I think, but still of you think you will run out of division soon, you can increase the difference and you can make it 100000...00 instead of 10
Approach #2:
You can save all of your tasks in the same document instead of sperate document in stratified json form, something like this:
Tasks: [ {name:"some name",date: "some date" },{name:"some name",date: "some date"},{name:"some name",date: "some date" } ]
By doing this you will get all task at once on the screen and you will parse the json as local array, when user rearrange the task you will simply change the position of that array element locally and save the stratified version of the tasks in database as well, there are some cons of this approach, if you are using pagination it might be difficult to do so but hopefully you will not be using the pagination in task management app and you probably wanted to show all task on the scree at the same time just like Trello does.
I am using version 3.1.1 of the popular charts library for iOS. I have run into an issue with x-axis labeling that I can't seem to find the answer for online:
Let's say I want to have a chart with one x-axis label for every day of the week (namely: S, M, T, W, T, F, S). Lots of forums I've read suggest taking the approach of setting a custom value formatter on the x-axis as suggested here: https://github.com/danielgindi/Charts/issues/1340
This works for calculating labels on days for which I have data. The issue I'm running into with this approach is that if I don't have data for a specific day, then the label for that day won't get generated.
For example, if I were to use a custom value formatter that looked like this:
public class CustomChartFormatter: NSObject, IAxisValueFormatter {
var days: = ["S", "M", "T", "W", "T", "F", "S"]
public func stringForValue(value: Double, axis: AxisBase?) -> String {
return days[Int(value)]
}
}
and my backing data looked like this: [(0, 15.5), (1, 20.1), (6, 11.1)] where 0, 1, and 6 are representations of days, and 15.5, 20.1, and 11.1 are the data points on those days, then when stringForValue is called, some of the days will never get labels generated for them.
Since value is always based on that backing data, it will never be equal to 2, 3, 4, or 5 in this scenario. As such, labels for "T", "W", "T", and "F" are never generated.
Does anyone know how to force the library to generate 7 labels, one for each day of the week, regardless of what my backing data is? Thank you kindly.
Ok so thanks to #wingzero 's comment, I have been able to get this working. There are a few things required to do so. For simplicity's sake, I am going to explain how to get the "days of the week" labels working as I originally asked. If you follow these steps, however, you should be able to tweak them to format your chart however you like (for example, with months of the year).
1) Make sure that your chart's x-axis minimum and maximum values are set. In this case, you'd want to say: chartView.xAxis.axisMinimum = 0.0 and chartView.axisMaximum = 6.0. This is important for step 2.
2) As Wingzero alluded to, create a subclass of XAxisRenderer that allows us to grab the minimum and maximum values set in step one and determine what values should be passed to our IAxisValueFormatter subclass in step three. In this case:
class XAxisWeekRenderer: XAxisRenderer {
override func computeAxis(min: Double, max: Double, inverted: Bool) {
axis?.entries = [0, 1, 2, 3, 4, 5, 6]
}
}
Make sure to pass this renderer to your chart like this: chartView.xAxisRenderer = XAxisWeekRenderer()
3) Create a subclass of IAxisValueFormatter that takes the values we passed to the chart in step two ([0, 1, 2, 3, 4, 5, 6]) and gets corresponding label names. This is what I did in my original question here. To recap:
public class CustomChartFormatter: NSObject, IAxisValueFormatter {
var days: = ["S", "M", "T", "W", "T", "F", "S"]
public func stringForValue(value: Double, axis: AxisBase?) -> String {
return days[Int(value)]
}
}
4) Set the labelCount on your graph to be equal to the number of labels you want. In this case, it would be 7. I show how to do this, along with the rest of the steps, below the last step here.
5) Force the labels to be enabled
6) Force granularity on the chart to be enabled and set granularity to 1. From what I understand, setting the granularity to 1 means that if the data your chart passes to stringForValue is not in round numbers, the chart will essentially round said data or treat it like it is rounded. This is important since if you passed in 0.5, it's possible that your stringForValue might not produce the right strings for your labels.
7) Set the value formatter on the xAxis to be the custom formatter you created in step 3.
Steps 4-7 (plus setting the formatter created in step 3) are shown below:
chartView.xAxis.labelCount = 7
chartView.xAxis.forceLabelsEnabled = true
chartView.xAxis.granularityEnabled = true
chartView.xAxis.granularity = 1
chartView.xAxis.valueFormatter = CustomChartFormatter()
First, have you debugged return days[Int(value)] on your side? From your screenshot, it seems obvious that your value after int cast looses the precision. e.g. 2.1 and 2.7 will be 2, which always shows you T. You have to look at your value first.
If you are sure you only get 7 xaxis labels all the time, a tricky way is to force computeAxisValues to have [0,1,2,3,4,5,6] all the time.
Meaning, you make sure your data x range is [1,7] (or [0,6]), and in #objc open func computeAxisValues(min: Double, max: Double), you should be able to see min is 1 and max is 7.
Then you override this method to set axis.entries = [Double]() to be [0,1,2,3,4,5,6], without any calculation. This should gives you the correct mapping.
However, before doing this, I suggest you take some time to debug this method first, to understand why you didn't get the expected values.
Given the following indices:
{
id: 1,
distance: [4, 10]
},
{
id: 2,
distance: [1, 2]
},
{
id: 3,
distance: [9, 10]
}
After enabling faceting on distance to be able to apply numerical filters on it, I want to apply additional filtering on a derived attribute that I'll call distance_range. It derives its value through custom logic on an existing attribute for example:
// In JS:
distance_range: Math.floor(( distance - 0.5 ) / 5) * 5 + "-" + Math.ceil(( distance ) / 5) * 5
Which would transform each distance number into a range "0-5", "5-10", etc.
{
id: 1,
distance: [4, 10],
distance_range: ["0-5", "5-10"]
},
{
id: 2,
distance: [1, 2]
distance_range: ["0-5"]
},
{
id: 3,
distance: [9, 10]
distance_range: ["5-10"]
}
What is the best way to arrive at such an index? Do I have the process the data beforehand, or does algolia have such a feature to transform attributes on the fly?
Algolia uses a custom data structure to index your data in order to deliver results as fast as possible.
Rebuilding an index in real-time would be too long to get an answer, which is why, even for sorting, Algolia requires you to duplicate your data into multiple indices.
If you only had to compute the new attribute for display, it would definitely be achievable with front-end logic, both autocomplete.js and instantsearch.js have a method to transform the data before displaying it.
However, since you want to filter on it, Algolia needs this information in the data structure, so you'll have to do this computation beforehand.
Also, don't forget to add this new attribute to attributesForFaceting.