I easily can store geolocation Data into MongoDB with an Eve RESTful API Server running.
So I store data like:
loc : {
lng: 13.01111,
lat: 51.01111
}
Validation etc. works nicely.
But: I was unable to find a way to get geolocation data out of the REST API.
There are sample queries over there working fine at the command-line, but there seems to be no way to query the API a appropriate way.
Is there a way to throw MongoDB like Queries against the REST API or
which is the prefered way to customize the API for such a purpose.
To make things clear: There is already a 2d index and geoWithin queriey at the mongo-cmd are working fine.
It's just about how to query via the REST API.
It's not mentioned but it should be supported. I'm not into geo stuff but I just tried to issue a $near query and it returned an operation failure because my database was missing the necessary 2dindex. This means that the command was correctly passed to the database.
If you are using a rest client like Postman the syntax should be something like this (I'm using $near for simplicity):
?where={"loc": {"$near": {"$geometry":{"type": "Point", "coordinates": [13,51]}}, "$maxDistance": 100}}
If you are using app.get method remember to json.loads your query. Hope this helps.
Points in MongoDB can be stored as either GeoJSON Point Objects,
loc : {
type: 'Point',
coordinates: [13.0111, 51.0111]
};
or legacy coordinate pairs.
loc : [13.01111, 51.01111]
The collection should have an index (either 2dsphere or 2d, respectively) to handle queries efficiently.
I'm pretty sure EVE has a built-in geoWithin operator. It's just not working because you're querying an invalid location format.
Related
So I am working on an idea for a "dating app" (I know who would have thought) and searching for what kind of Serverless Service I should use for the BE.
Until now I worked with GCP and Firebase, that's why I thought I could also use Firestore as my Database.
The problem is, that I would like to do complex filtering like the following:
.where(“date”, “>=”, today).where(“date”, “<=”, today + 3)
.where(“heightMin”, “>=”, 165).where(“heightMax”, “<=”, 185)
.where(“ageMin”, “>=”, 20).where(“ageMax”, “<=”, 30)
.where(“type”, “==”, “running”).where(“smoker”, “==”, false)
In the Cloud Firestore Documentation, it is stated that such compound queries are not possible. To me this seems like a basic feature, thus I think I might have misinterpreted the documentation. Similar post here Firestore compound query with <= & >=
This is how a document could look like:
{
"date": "2022-03-12",
"time": "18:30:00",
"withFilters": "true",
"smoker": "false",
"heightMin": "165",
"heightMax": "165",
"ageMin": "20",
"ageMax":"30",
"type": "running",
"latitude" : "y",
"longitude : "x",
"participants": "2",
}
I know that I could probably fetch everything and then filter it in the FE but this is suboptimal and costly.
Question:
Is there any way to have such filtering with Firestore even by re-arranging the data structure?
If not, what kind of another service would you recommend? Perhaps AWS Amplify?
Firebase also offers an Algolia Extension that can easily sync your Firestore data with Algolia. Firestore natively does not support full text search or compound queries with many filters but Algolia does.
For example, once you've copied all your existing data in Algolia, you can the query as shown below:
index.search('query', { // <-- requires some query string
filters: 'ageMin > 25 AND heightMin > 175' // ... other constraints
]
}).then(({ hits }) => {
console.log(hits);
});
This might be the easiest and scalable way as your data is present in Firestore and both are managed services. Also also has a client SDK and you can create custom API keys to restrict user access.
MongoDB does support these queries natively but you might have to restructure existing database and migrate the data. You would also have to create indexes to support your queries manually (Algolia creates them automatically unless you want to modify them).
I'm new to StreamSets, thanks in advance for any help.
In my pipeline records (JSON) I have a field with Geo-coordinates (lat, lon) and I'm trying to add more meta-data to them. I'm wondering if it's possible to use a HTTP Client Processor to realize the operation described here https://nominatim.org/release-docs/develop/api/Reverse/ using the lat, lon values of my records. If it is, could you point me for some documentation or article that describes how to do it?
I've been able to use HTTP Client as Origins in other occasions, but i can't figure out how to use the values in the URL.
For example if the values of my record were
{
"lat": 41.195519,
"lon":-8.666526,
"format": "jsonv2"
}
The url should look like:
https://nominatim.openstreetmap.org/reverse?format=jsonv2&lat=41.226599&lon=-8.709737
Figured out, simply GET to
https://nominatim.openstreetmap.org/reverse?format=${record:value('/format')}&lat=${record:value('/lat')}&lon=${record:value('/lon')}
How to pass complex get query parameters such as where order etc into GET https://firestore.googleapis.com/v1beta1/{name=projects/*/databases/*/documents/*/**}(firestore docs) ?
You can order and limit data in a Cloud Firestore query after using the Get Data method like this:
myDinoRef.where("type", "==", "TREX").orderBy("size", "desc").limit(2)
This will return all TRex Dinos and order by descending "size". It will also limit the result set to two rows.
Google's documentation is pretty good and ever evolving. It's also important to point out that the Cloud Firestore is still in beta (as of today)
In the context of ArangoDB, there are different database shells to query data:
arangosh: The JavaScript based console
AQL: Arangodb Query Language, see http://www.arangodb.org/2012/06/20/querying-a-nosql-database-the-elegant-way
MRuby: Embedded Ruby
Although I understand the use of JavaScript and MRuby, I am not sure why I would learn, and where I would use AQL. Is there any information on this? Is the idea to POST AQL directly to the database server?
AQL is ArangoDB's query language. It has a lot of ways to query, filter, sort, limit and modify the result that will be returned. It should be noted that AQL only reads data.
(Update: This answer was targeting an older version of ArangoDB. Since version 2.2, the features have been expanded and data modification on the database is also possible with AQL. For more information on that, visit the documentation link at the end of the answer.)
You cannot store data to the database with AQL.
In contrast to AQL, the Javascript or MRuby can read and store data to the database.
However their querying capabilities are very basic and limited, compared to the possibilities that open up with AQL.
It is possible though to send AQL queries from javascript.
Within the arangosh Javascript shell you would issue an AQL query like this:
arangosh> db._query('FOR user IN example FILTER user.age > 30 RETURN user').toArray()
[
{
_id : "4538791/6308263",
_rev : "6308263",
age : 31,
name : "Musterfrau"
}
]
You can find more info on AQL here:
http://www.arangodb.org/manuals/current/Aql.html
I am wondering if it is possible to use a mongodb geospatial index with Meteor architecture.
Minimongo does not implement geospatial indices, but does this mean that we cannot use this mongo feature on the server side?
For example, with the todos app, if we use location on the todo, will it be possible to do:
// Publish complete set of lists to all clients.
Meteor.publish('todos', function (lon,lat) {
return Todos.find({loc: {$near:[lon,lat]}}).limit(2);
});
And on the client side :
Meteor.subscribe('todos', lon, lat );
Yes, you can use the MongoDB geospatial index within Meteor, and you can create that index from within your Meteor app too.
- Geospatial Search
I'm using the $within operator below, as opposed to the $near operator mentioned above, but this still applies:
Meteor.publish('places', function(box) {
return Places.find({ loc : { $within : { $box : box }}});
});
Reminder: These kinds of geo queries are only available on the server (currently).
- Creating a Geospatial Index from within Meteor (rather than in a MongoDB shell)
Places._ensureIndex({ loc : "2d" });
e.g. You could use the above in your bootstrap.js.
Also, you'll probably want to put your ensureIndex in Meteor.startup, or perhaps when you're inserting some initial data.
Warning: As mentioned here, the above method of calling ensureIndex is a work around for want of an official way to call it, so please expect that this might change.
Update: now reflects changes in Meteor 0.5.0, see #Dror's comment below.
Yes, I think you can.
On the server side, Meteor delegates find/update/.. into node-mongo-native call. You can take a look the code in packages/mongo-livedata/mongo_driver.js. And as I know, node-mongo-native supports geospatial index.