Finding Closest Users Without Looping Through Many Records - swift

When users log in to my app, I want them to be able to see the number of users within a selected radius. I was planning to start storing coordinates in my database and looping through each record (~50,000), running userCoordinates.distance(from: databaseCoordinateValue). However, during testing, I've found that this process takes a long time and is not a scalable solution. Do you have any advice on how to quickly query database items within a defined radius?
I am using:
Swift 4
Firebase (Firestore beta)
Xcode 10
Example of database structure and how data gets stored
database.collection("users").document((Auth.auth().currentUser?.uid)!).setData([
"available_tags" : ["milk", "honey"]]) { err in
if let err = err {
print("Error adding document: \(err)")
}
}

Take a look at s2 geometry - http://s2geometry.io/. The basic concept is you encode each location on earth as a 64 bit # with locations close to each other being close #'s. Then, you can look up locations within x distance by finding anything that is +/- a certain # from the location. Now, the actual implementation is a bit more complicated, so you end up needing to creating multiple 'cells' ie. a min and max # on the range. Then, you do lookups for each cell. (More info at http://s2geometry.io/devguide/examples/coverings.)
Here's an example of doing this in node.js / javascript. I use this in the backend and have the frontend just pass in the region/area.
const S2 = require("node-s2");
static async getUsersInRegion(region) {
// create a region
const s2RegionRect = new S2.S2LatLngRect(
new S2.S2LatLng(region.NECorner.latitude, region.NECorner.longitude),
new S2.S2LatLng(region.SWCorner.latitude, region.SWCorner.longitude),
);
// find the cell that will cover the requested region
const coveringCells = S2.getCoverSync(s2RegionRect, { max_cells: 4 });
// query all the users in each covering region/range simultaneously/in parallel
const coveringCellQueriesPromies = coveringCells.map(coveringCell => {
const cellMaxID = coveringCell
.id()
.rangeMax()
.id();
const cellMinID = coveringCell
.id()
.rangeMin()
.id();
return firestore
.collection("User")
.where("geoHash", "<=", cellMaxID)
.where("geoHash", ">=", cellMinID).
get();
});
// wait for all the queries to return
const userQueriesResult = await Promise.all(coveringCellQueriesPromies);
// create a set of users in the region
const users = [];
// iterate through each cell and each user in it to find those in the range
userQueriesResult.forEach(userInCoveringCellQueryResult => {
userInCoveringCellQueryResult.forEach(userResult => {
// create a cell id from the has
const user = userResult.data();
const s2CellId = new S2.S2CellId(user.geoHash.toString());
// validate that the user is in the view region
// since cells will have areas outside of the input region
if (s2RegionRect.contains(s2CellId.toLatLng())) {
user.id = userResult.id;
users.push(user);
}
});
});
return users;
}
S2 geometry has a lot of ways to find the covering cells (i.e. what area you want to look up values for), so it's definitely worth looking at the API and finding the right match for your use case.

Related

Flutter : How to Map values dynmically into list and iterate to its length and how can i do Update and Delete after maping values

I am trying to build json expected output as below through dart programing for my application, I have mapped data to list successfully. But when I trying to delete / add the list, the elements in the list are not getting updated accordingly instead they are hgetting reapeted same data.
Here is my code implimentation
in this set state i am getting required values like Phone, name, email e.t.c
for (int i = 0; i <= selectedContacts.length - 1; i++) {
SimplifiedContact? contact = selectedContacts.elementAt(i);
CreateContestModel(//<-- from here i am getting required values.
contact.phone,
contact.name,
contact.email,
);
}
in below code i am, mapping data and building json
class CreateContestModel {
static List models = [];
String phone = '';
String name = '';
String email;
CreateContestModel(this.phone, this.name, this.email) {
var invitemap = {
'name': name,
'phone': phone,
'email': email,
};
models.add(invitemap);
print(models);
}
}
Output
{
"invitations":[
{
"name":"Acevedo Castro",
"phone":982-475-2009,
"email":"floresjoyner#digifad.com"
},
{
"name":"Acevedo Castro",
"phone":982-475-2009,
"email":"floresjoyner#digifad.com"
},
{
"name":"Abby Webster",
"phone":888-561-2141,
"email":"howardnoel#perkle.com"
},
{
"name":"Abby Webster",
"phone":888-561-2141,
"email":"howardnoel#perkle.com"
},
{
"name":"Abby Webster",
"phone":888-561-2141,
"email":"howardnoel#perkle.com"
}
]
}
As you see above items are not getting updated, but they are getting added more.
Expected Output
{
"invitations":[
{
"name":"Acevedo Castro",
"phone":"982-475-2009",
"email":"floresjoyner#digifad.com"
},
{
"name":"Abby Webster",
"phone":"888-561-2141",
"email":"howardnoel#perkle.com"
}
]
}
That is some seriously flawed program flow. Your object creation as a side effect at the same time fills a static list. And it seems you call your object creation every build. So you would insert into your list whenver the user flips it's phone or drags his browser window.
I'm not entirely sure what you want, but you need state management in your application. There are different ways to do it, you can pick the one you like best from the Flutter documentation on state management.
This is a huge flow issue, You should never do data manipulation in build() function as in flutter build function is called multiple times so the manipulation code will also get called multiple times.
So, the proper way to manipulate data is before the data is being used so make sure that the data is only manipulated in initstate(). In Your case you are also doing something which is not required. You are trying to add data to a list via a constructor to a static list so it will always add the data whenever you call it, This is not a proper way.
class CreateContestModel {
late String phone = '';
late String name = '';
late String email;
CreateContestModel.fromMap(Map<String, String> json) {
phone = json['phone'] ?? '';
phone = json['name'] ?? '';
phone = json['email'] ?? '';
}
}
This is how you should create your class. And always create functions for manipulation if possible.
List<CreateContestModel> createContestModelList =
testData['invitations']!.map(
(data) {
return CreateContestModel.fromMap(data);
},
).toList();
Then use this code to construct your list in initstate() and manipulate this having a static variable in your stateful widget. Make Sure you do not construct the data on build() function.
Need More Help?? here's the Gist link
As per my question i find some work around, i.e., by clear up stack on selecting or updtaing existing contacts because as explained below
in this for loop i am itrating same values on each time, when i hit on submit button
for (int i = 0; i <= selectedContacts.length - 1; i++) { //<--- for every time same elements will be itrated
SimplifiedContact? contact = selectedContacts.elementAt(i);
CreateContestModel(//<-- from here i am getting required values.
contact.phone,
contact.name,
contact.email,
);
}
so the list of sets are not updating as per the displayed list as mentioned in question, so i done some work around in below code while selecting / updating contacts, because my model is static list
void onContactBtnPress(BuildContext context) async { CreateContestModel.models.clear(); //<-- clearing up stack
}

How do I update a MongoDB document with new value using reactors Mono? (Kotlin)

So the context is that I require to update a value in a single document, I have a Mono, the parameter Object contains values such as username (to find the correct user by unique username) and an amount value.
The problem is that this value (due to other components of my application) is the value by which I need to increase/decrease the users balance, as opposed to passing a new balance. I intend to do this using two Monos where one finds the user, then this is combined to the other Mono with the inbound request, where I can then perform a simple sum (i.e balance + changeRequest.amount) then return this to the document database.
override fun increaseBalance(changeRequest: Mono<ChangeBalanceRequestResource>): Mono<ChangeBalanceResponse> {
val changeAmount: Mono<Decimal128> = changeRequest.map { it.transactionAmount }
val user: Mono<User> = changeRequest.flatMap { rxUserRepository.findByUsername(it.username)
val newBalace = user.map {
val r = changeAmount.block()
it.balance = sumBalance(it.balance!!, r!!)
rxUserRepository.save(it)
}
.flatMap { it }
.map { it.balance!! }
return Mono.just(ChangeBalanceResponse("success", newBalace.block()!!))
}
Obviously I'm trying to achieve this in a non-blocking fashion. I'm also open to using only a single Mono if that's possible/optimal. I also appreciate I've truly butchered the example and used .block as a placeholder to illustrate what I'm trying to achieve.
P.S this is my first post, so any tips on how to express my problem clearer would be useful.
Here's how I would do this in Java (Using Double instead of Decimal128):
public Mono<ChangeBalanceResponse> increaseBalance(Mono<ChangeBalanceRequestResource> changeRequest) {
Mono<Double> changeAmount = changeRequest.map(a -> a.transactionAmount());
Mono<User> user = changeRequest.map(a -> a.username()).flatMap(RxUserRepository::findByUsername);
return Mono.zip(changeAmount,user).flatMap(t2 -> {
Double changeAmount = t2.getT1();
User user = t2.getT2();
//assumes User is chained
return rxUserRepository.save(user.balance(sumBalance(changeAmount,user.balance())));
}).map(res -> new ChangeBalanceResponse("success",res.newBalance()))
}

Graph processing increasingly gets slower on titan + dynamoDB (local) as more vertices/edges are added

I am working with titan 1.0 using AWS dynamoDB local implementation as storage backend on a 16GB machine. My use case involves generating graphs periodically containing vertices & edges in the order of 120K. Every time I generate a new graph in-memory, I check the graph stored in DB and either (i) add vertices/edges that do not exist, or (ii) update properties if they already exist (existence is determined by 'Label' and a 'Value' attribute). Note that the 'Value' property is indexed. Transactions are committed in batches of 500 vertices.
Problem: I find that this process gets slower each time I process a new graph (1st graph finished in 45 mins with empty db initially, 2nd took 2.5 hours, 3rd in 3.5 hours, 4th in 6 hours, 5th in 10 hours and so on). In fact, when processing a given graph, it is fairly quick at start time but progressively gets slower (initial batches take 2-4 secs and later on it increases to 100s of seconds for same batch size of 500 nodes; I also see sometimes it takes 1000-2000 secs for a batch). This is the processing time alone (see approach below); commit takes between 8-10 secs always. I configured the jvm heap size to 10G, and I notice that when the app is running it is eventually using up all of it.
Question: Is this behavior to be expected? It seems to me something is wrong here (either in my config / approach?). Any help or suggestions would be greatly appreciated.
Approach:
Starting from the root node of the in-memory graph, I retrieve all child nodes and maintain a queue
For each child node, I check to see if it exists in DB, else create new node, and update some properties
Vertex dbVertex = dbgraph.traversal().V()
.has(currentVertexInMem.label(), "Value",
(String) currentVertexInMem.value("Value"))
.tryNext()
.orElseGet(() -> createVertex(dbgraph, currentVertexInMem));
if (dbVertex != null) {
// Update Properties
updateVertexProperties(dbgraph, currentVertexInMem, dbVertex);
}
// Add edge if necessary
if (parentDBVertex != null) {
GraphTraversal<Vertex, Edge> edgeIt = graph.traversal().V(parentDBVertex).outE()
.has("EdgeProperty1", eProperty1) // eProperty1 is String input parameter
.has("EdgeProperty2", eProperty2); // eProperty2 is Long input parameter
Boolean doCreateEdge = true;
Edge e = null;
while (edgeIt.hasNext()) {
e = edgeIt.next();
if (e.inVertex().equals(dbVertex)) {
doCreateEdge = false;
break;
}
if (doCreateEdge) {
e = parentDBVertex.addEdge("EdgeLabel", dbVertex, "EdgeProperty1", eProperty1, "EdgeProperty2", eProperty2);
}
e = null;
it = null;
}
...
if ((processedVertexCount.get() % 500 == 0)
|| processedVertexCount.get() == verticesToProcess.get()) {
graph.tx().commit();
}
Create function:
public static Vertex createVertex(Graph graph, Vertex clientVertex) {
Vertex newVertex = null;
switch (clientVertex.label()) {
case "Label 1":
newVertex = graph.addVertex(T.label, clientVertex.label(), "Value",
clientVertex.value("Value"),
"Property1-1", clientVertex.value("Property1-1"),
"Property1-2", clientVertex.value("Property1-2"));
break;
case "Label 2":
newVertex = graph.addVertex(T.label, clientVertex.label(), "Value",
clientVertex.value("Value"), "Property2-1",
clientVertex.value("Property2-1"),
"Property2-2", clientVertex.value("Property2-2"));
break;
default:
newVertex = graph.addVertex(T.label, clientVertex.label(), "Value",
clientVertex.value("Value"));
break;
}
return newVertex;
}
Schema Def: (Showing some of the indexes)
Note:
"EdgeLabel" = Constants.EdgeLabels.Uses
"EdgeProperty1" = Constants.EdgePropertyKeys.EndpointId
"EdgeProperty2" = Constants.EdgePropertyKeys.Timestamp
public void createSchema() {
// Create Schema
TitanManagement mgmt = dbgraph.openManagement();
mgmt.set("cache.db-cache",true);
// Vertex Properties
PropertyKey value = mgmt.getPropertyKey(Constants.VertexPropertyKeys.Value);
if (value == null) {
value = mgmt.makePropertyKey(Constants.VertexPropertyKeys.Value).dataType(String.class).make();
mgmt.buildIndex(Constants.GraphIndexes.ByValue, Vertex.class).addKey(value).buildCompositeIndex(); // INDEX
}
PropertyKey shapeSet = mgmt.getPropertyKey(Constants.VertexPropertyKeys.ShapeSet);
if (shapeSet == null) {
shapeSet = mgmt.makePropertyKey(Constants.VertexPropertyKeys.ShapeSet).dataType(String.class).cardinality(Cardinality.SET).make();
mgmt.buildIndex(Constants.GraphIndexes.ByShape, Vertex.class).addKey(shapeSet).buildCompositeIndex();
}
...
// Edge Labels and Properties
EdgeLabel uses = mgmt.getEdgeLabel(Constants.EdgeLabels.Uses);
if (uses == null) {
uses = mgmt.makeEdgeLabel(Constants.EdgeLabels.Uses).multiplicity(Multiplicity.MULTI).make();
PropertyKey timestampE = mgmt.getPropertyKey(Constants.EdgePropertyKeys.Timestamp);
if (timestampE == null) {
timestampE = mgmt.makePropertyKey(Constants.EdgePropertyKeys.Timestamp).dataType(Long.class).make();
}
PropertyKey endpointIDE = mgmt.getPropertyKey(Constants.EdgePropertyKeys.EndpointId);
if (endpointIDE == null) {
endpointIDE = mgmt.makePropertyKey(Constants.EdgePropertyKeys.EndpointId).dataType(String.class).make();
}
// Indexes
mgmt.buildEdgeIndex(uses, Constants.EdgeIndexes.ByEndpointIDAndTimestamp, Direction.BOTH, endpointIDE,
timestampE);
}
mgmt.commit();
}
The behavior you experience is expected. Today, DynamoDB Local is a testing tool built on SQLite. If you need to support high TPS for large and periodic data loads, I recommend you use the DynamoDB service.

arc-jsapi How can I st_geometry using a lookup field?

I am using the st_geometry field and try to view the PNU code value.
I tried two ways.
oracle DB query.
select a.pnu from LP_PA_CBND_4600000000 a
where sde.st_contains(a.shape,
sde.st_point(177566.6728471977,160430.12935426735, 4))=1
When an event occurs, click the map object
i got the evt.mapPoint(x,y).
4 is my srid.
this way is took a long time. and query was down...
i used the arcgis api`s IdentifyParameters
my code is follows.
PoiClick : function(map, evt) {
G_evt =evt;
console.log("ClickPoint ==== "+evt.mapPoint);
var targetLayerId = 'LP_PA_CBND';
var url = map.Layers.getLayerInfo(targetLayerId).SVC_URL;
var map = map.getMap();
//파라미터 설정.
var idParams = new krcgis.core.tasks.IdentifyParameters();
G_idparams =idParams;
idParams.geometry = evt.mapPoint;
idParams.mapExtent = map.extent;
idParams.returnGeometry = true;
idParams.tolerance = 3;
idParams.layerOption = krcgis.core.tasks.IdentifyParameters.LAYER_OPTION_ALL;
idParams.width = map.width;
idParams.height = map.height;
krcgis.Function.GetPoiInfo(url, idParams);
return evt.mapPoint;
},
//POI 정보를 가져온다.
GetPoiInfo : function(url, idParams) {
idTask = new krcgis.core.tasks.IdentyfyTask(url);
idTask
.execute(idParams)
.addCallback(function (response) {
G_response = response;
if (response) {
return response;
}
})
.addErrback(function (error) {
console.log('GetPoiInfo result error=', error);
});
}
It could be obtained in this way code pnu.
However, this way is different from the value that gets pnu code depending on the zoom level.
I want to get a single pnu code in a single x, y values.
how to get pnu code?
Database table :
IdentifyTask can be a little tricky, because it takes into account zoom levels, therefor it doesn't always return the same results.
I suggest that you use QueryTask. Just one problem - it doesn't have tolerance parameter, but you can buffer your evt.mapPoint and get Polygon.

How do I add polyline length to a Bing Maps polygon

I am trying to recreate a tax map within my system using Bing Maps. My problem is in listing the length, in feet, of the sides of the polygons I am creating. I have a good idea of how to get the length of polylines I am creating from the MSSQL 2012 geometry or geography items in my database. I cannot figure out how to present it to the user effectively though. I have two ideas for how I would like to do this.
Place the lengths directly on or adjacent to the polyline in question.
Create an emphasized point on the full polygon and list to the side of the map, the lengths of the sides of the polygon based on a clockwise order.
Either of the 2 options would work as an acceptable solution. I used this tutorial to create my current environment so I would be looking to integrate the solution into it in some way:
How to create a spatial web service that connects a database to Bing Maps using EF5
Note that my implementation only uses the countries part of the code so I do not need to deal with single points like cities that are in that tutorial.
The relevant piece of code that handles drawing on the map that I would need to edit can be found here:
Bing Maps v7 WellKnowTextModule
If you want to get the perimeter of a polygon in SQL2012 you can grab the exterior ring of it. The exterior ring will be a LineString i.e. "#g.STExteriorRing()". Then measure the length along that line. i.e. "#g.STExteriorRing().STLength()". However, countries are usually not just single Polygons, they can be MultiPpolygons, or GeometryCollections. So to calculate these lengths we have to do a bit more work. Here is a helper method you can add to the service to calculate the perimeters of these shapes:
private double CalculateLength(SqlGeometry geom)
{
double length = 0;
if(string.Compare(geom.STGeometryType().Value, "polygon", true) == 0)
{
}
else if (string.Compare(geom.STGeometryType().Value, "multipolygon", true) == 0)
{
int numPolygon = geom.STNumGeometries().Value;
for(int i = 1; i <= numPolygon; i++){
length += geom.STGeometryN(i).STExteriorRing().STLength().Value;
}
}
else if (string.Compare(geom.STGeometryType().Value, "geometrycollection", true) == 0)
{
int numGeom = geom.STNumGeometries().Value;
for (int i = 1; i <= numGeom; i++)
{
length += CalculateLength(geom.STGeometryN(i));
}
}
return length;
}
To get the length info from the server side to the client add a property to the Country or BaseEntity class like this:
[DataMember]
public double Perimeter { get; set; }
From here you can populate this value after the linq query is used to get the response results using a simple loop that calls the helper method from earlier:
for (int i = 0; i < r.Results.Count;i++)
{
var geom = SqlGeometry.STGeomFromText(new System.Data.SqlTypes.SqlChars(r.Results[i].WKT), 4326);
r.Results[i].Perimeter = CalculateLength(geom);
}
As for displaying the information on the map. An easy way to place the information on a polyline is to choose a coordinate along the line, perhaps the middle one, just get the # or coordinates in the line and find the middle index and use that coordinate for a pushpin. You can then create a custom push using either a background image with text, or using custom HTML:
http://www.bingmapsportal.com/ISDK/AjaxV7#Pushpins4
http://www.bingmapsportal.com/ISDK/AjaxV7#Pushpins15
Wanted to add an addendum to the answer I accepted as I feel it changes it a bit.
While working on this I found that I was not actually able to get each line segment's length via entity framework. This is due to the fact that the query required changing the geography I had back to a geometry then parse it to its base line segments and then change those line segments back to geographies. The query, even in SQL, would take minutes so it was not an option to run dynamically in EF.
I ended up creating another table in my database containing the parsed line segments for each side of each polygon I had. Then I could use the centroids of the line segments as faux cities. I then added this logic into the DisplayData javascript function from the tutorial mentioned in the question after the for loop in the method.
if (shape.getLength) {
} else {
var chkPolygon = data.Results[0].WKT.substring(0, data.Results[0].WKT.indexOf('(', 0));
chkPolygon = chkPolygon.replace(/\s/g, '');
switch (chkPolygon.toLowerCase()) {
case 'point':
case 'polygon':
var latlonCheck = map.getCenter();
var setSides = window.location.origin + "/SpatialService.svc/FindNearBy?latitude=" +
latlonCheck.latitude + "&longitude=" + latlonCheck.longitude +
"&radius=" + data.Results[0].ID + "&layerName=" + "city" + "&callback=?";
CallRESTService(setSides, DisplaySides);
default:
break;
}
}
the data.Results[0].ID would find all the line segments in the new table for that specific country. Then the DisplaySides function is used to overlay the html pushpins as "cities" over the appropriate points for each side on the map
function DisplaySides(getSides) {
infobox.setOptions({ visible: false });
if (getSides && getSides.Results != null) {
for (var i = 0; i < getSides.Results.length; i++) {
var sideLenFtShort = Math.round(getSides.Results[i].LengthFeet * 100) / 100;
var htmlLenString = "<div style='font-size:14px;border:thin solid black;background-color:white;font-weight:bold;color:black;'>" + sideLenFtShort.toString(); + "</div>";
var testString = {
pushpinOptions: { width: null, height: null, htmlContent: htmlLenString }
};
var sideCtr = WKTModule.Read(getSides.Results[i].WKT, testString);
dataLayer.push(sideCtr);
}
}
else if (getSides && getSides.Error != null) {
alert("Error: " + getSides.Error);
}
}