I have some data in MapQuest Data Manager v2 (DMv2) and I am trying to search this hosted data set using their search API(Web Service) but I am not getting desired results. Following are the snapshot of my data set and queries I am using :
These are the fields :
"storeid","brandname","onlineorderingenabled","street","zipcode","state","geocodereturncode","city","country","mqap_geography","mqap_quality","landmark","county","mqap_id","storename","longitude","latitude"
and this is sample row :
"1","Chili's","true","12815 Preston Rd","75230-1302","TX","","Dallas","US","POINT (-96.80363 32.92329)","P1AAA","","Dallas","ca6b6bae-945f-45fc-a8d1-3d512796150d","Preston/LBJ-Chili's","",""
Sample search queries :
http://www.mapquestapi.com/search/v2/search?key=[My_Key here]&shapePoints=-80,26&outFormat=json&hostedData=hostedData=mqap.121144_BrinkerStores|storename ILIKE ?|Preston/LBJ-Chili's|storeid,storename,city
http://www.mapquestapi.com/search/v2/radius?key=[My_Key]&origin=Dallas&inFormat=json&json={hostedData:[{tableName:mqap.121144_BrinkerStores,extraCriteria:City ILIKE ?,parameters:[Dallas],columnNames:[storeid,storename,city,state]}]}
I am not getting storedid,storenam,city,state. Please help me.
In the search/v2/search request, make sure your shapePoints are latititude,longitude and not the other way around. And the apostrophe in the storename needs to be double apostrophe'd for the postgres backend.
http://www.mapquestapi.com/search/v2/search?key=KEY&shapePoints=32.778149,-96.795403&hostedData=mqap.121144_BrinkerStores|storename=?|Knox-Chili%27%27s
In the search/v2/radius request, include the origin in the json and rename hostedData to hostedDataList.
www.mapquestapi.com/search/v2/radius?key=KEY&inFormat=json&json={"origin":"Dallas,TX","hostedDataList":[{"tableName":"mqap.121144_BrinkerStores","extraCriteria":"City ILIKE ?","parameters":["Dallas"],columnNames:["storeid","storename","city","state"]}]}
Related
I am trying to pull all glaciers as entered in OSM in a given country but am noticing that I am only pulling a fraction of what is available. For example, when I run this following code:
import overpass from shapely.geometry
import shape, Polygon
api = overpass.API()
api = overpass.API(endpoint="https://overpass.myserver/interpreter")
api = overpass.API(timeout=600)
query = 'area["ISO3166-1"="IS][admin_level=2];(way["natural"="glacier"](area););'
result = api.get(query, verbosity='geom')
import geopandas
results = geopandas.GeoDataFrame.from_features(result['features'])
The result has 132 features and appears as so:
Iceland Glaciers
I know this is missing one large glacier (Vatnajökull) which does appear in OSM under osm id 406429.
Any thoughts as to why this is not appearing as a result from my query?
OSM Wiki tag documentation is a helpful starting point when writing Overpass queries. Here is the documentation for natural=glacier. The tag/value is applied to nodes and closed ways based on the documentation and also appears to apply to relations based on community preference (even though this is discouraged in the documentation).
To query for nodes, ways, and relations, you can use the abbreviation nwr instead of the union (node[natural=glacier];way[natural=glacier];relation[natural=glacier];);. As a side note, you can drop admin_level=2 since ISO3166-1 codes are unique identifiers.
Here is the Python request:
query = 'area["ISO3166-1"="IS"];nwr[natural=glacier](area);out geom;'
response = api.get(query)
I am trying to use the VSTS.Feed() function in Power BI to read WorkItemSnapshot data. There are multiple problems. If I build the entire URL into a single string and call VSTS.Feed () with that, I get the correct information in Power BI desktop, but it will not refresh in Power BI online. I have been told to use the (undocumented) Query parameter, as shown below, but it is clear that this parameter is ignored. I can see that the select parameter is ignored on smaller projects, because all columns are returned. I can see that the filter parameter is ignored because the query fails on larger projects.
Does anyone have a working example of using the Query parameter with VSTS.Feed()?
let
BaseURL = "https://server.analytics.visualstudio.com/DefaultCollection/project/_odata/WorkItemSnapshot",
Select = "DateSK,WorkItemId,State,WorkItemType",
Filter = "WorkItemType eq Bug and State ne Closed and State ne Removed and DateSK ge 20180517 and DateSK le 20180615",
Source = VSTS.Feed(BaseURL, [Query=[select=#"Select",filter=#"Filter"]])
in
Source
Update:
With the query above, the message I get is shown below. As I said earlier, it is clearly not using the Filter parameter, and I'm assuming it is not using the Select parameter, either. I can't query everything because there is too much data, and I can't use a filter because I can't figure out a way to get the Options parameter to work. With VSTS.AccountContents, the options parameter works well, but those API endpoints don't use $ in parameter names.
Error: Query result contains 36,788,023 rows and it exceeds maximum allowed size of 300,000. Please reduce the number of records by applying additional filters
Details:
DataSourceKind=Visual Studio Team Services
ActivityId=881f7988-9863-4e03-8375-0489028f28f3
Url=https://server.analytics.visualstudio.com/DefaultCollection/Project/_odata/WorkItemSnapshot
error=Record
The query that started this whole line of questioning is simply one with a variable for a start date.
let
startDate = DateTimeZone.ToText (Date.AddDays(DateTimeZone.UtcNow(), -45), "yyyyMMdd"),
URL = "https://server.analytics.visualstudio.com/DefaultCollection/project/_odata/WorkItemSnapshot?$select=DateSK,WorkItemId,State,WorkItemType&$filter=WorkItemType eq 'Bug' and State ne 'Closed' and State ne 'Removed' and DateSK gt " & startDate,
Source = VSTS.Feed(URL)
in
Source
While this query mostly works in Power BI desktop (the select clause is ignored), the message I get when the data source is refreshed online is:
You can't schedule refresh for this dataset because one or more sources currently don't support refresh.
Discover Data Sources
Query contains unknown or unsupported data sources.
The documentation for VSTS.Feed() contradicts itself, saying both
The VSTS.Feed function has the same arguments, options and return value format as OData.Feed.
and
'VSTS.Feed' provides a subset of the Arguments and Options available through 'OData.Feed'.
To to summarize, I know that I can't combine data sources in Power BI. Does VSTS.Feed() support the options parameter? If so, how do I pass a Filter and Select clause to it?
To get WorkItemSnapshot by vsts.feed, please refer below query:
let
Source = OData.Feed("https://account.analytics.visualstudio.com/project/_odata/v1.0-preview", null, [Implementation="2.0"]),
WorkItemSnapshot_table = Source{[Name="WorkItemSnapshot",Signature="table"]}[Data]
in
WorkItemSnapshot_table
Note: the URL format should be https://account.analytics.visualstudio.com/project/_odata/v1.0-preview, or https://account.analytics.visualstudio.com/_odata/v1.0-preview.
And you can refer below documents:
Connect to VSTS using the Power BI OData feed
Connect using Power Query and Visual Studio Team Services (VSTS) functions
I want to get OSM data after some timestamp - in other words the last records after a certain timestamp. I have downloaded the osm file of the area. I went through the osmosis documentation but could not find a way to filter it by time. The result should be same as when we use the timestamp-argument. Well how to do that:
I could use the overpass but the area is large and overpass timed out many times
I could use the osmconvert-tool (cf the manual: m.m.i24.cc/osmconvert.c )
Some of the following statements might be useful for the task:
"--timestamp=<date_time> add a timestamp to the data\n"
"--timestamp=NOW-<seconds> add a timestamp in seconds before now\n"
What I have tried is the following;
./osmfilter austria-latest.osm --keep="$key=$school" |
./osmconvert - --all-to-nodes --csv="#id #lat #lon #timestamp $key name" --csv-headline |
but this fails. How to get the data out of the osm-pbf-file. Should I use the statements drop! or should i name a certain time from timestamp to timestamp!?
Since version 0.7.50 Overpass API provides a way to query for data, which changed since a given timestamp or in a given timeframe. It is even possible to restrict the change analysis to certain tags (or filter criteria). Please check the Overpass API Wiki page for more details on "diff" and "adiff" keywords.
Working with Overpass API ina way is much more convenient than trying to process a full planet history, which takes at least 35GB to download and requires more complex post-processing.
You want to process OSM history planet (extracts): https://wiki.openstreetmap.org/wiki/Planet.osm/full
My requirement:
fetch ONE object (e.g RetainInfo ) from table RETAIN_INFO if VERSION column has max value
Does CRUD repository support for an interface method like
findOneByMaxRetVersionAndCountry("DEFAULT")
Equivalent db2 sql:
select RET_ID, max(ri.RET_VERSION) from RETAIN_INFO ri where ri. COUNTRY='DEFAULT' group by RET_ID fetch first 1 rows only;
This query selects an ID, but I would actually want the RetainInfo object corresponding the SINGLE row returned by the query.
I prefer to get that without using custom query, i.e using findBy or some other method/interface supported by Spring CRUD.
You could use limiting in combination with sorting (spring data reference:limit query results). Declare a method similar to the following in your CrudRepository interface :
RetainInfo findTopByCountryOrderByRetVersionDesc(String country);
You can also use findFirst to get the first result. Before getting the result, make sure to use Orderby and then the ascending(Asc) or descending(Desc). As an example if you want to order by version and retrieve based on productName
RetainInfo findFirstByProductNameOrderByVersionDesc(String productName);
Spring Data doesn't provide an expression to select a max value. All supported query parts could be found in the Spring 1.2.0.RELEASE docs: Appendix A. Namespace reference or line 182 of org.springframework.data.repository.query.parser.Part.
Also feel free to create a feature request at Spring's Jira page.
For more dynamism, I would like to add a random part on my app.
Here is what I would have done in other techs, and what is not working in play :
long id = JPA.execute("select id from Realisation r order by RANDOM() LIMIT 1");
And here is the stack :
unexpected token: LIMIT near line 1, column 55
Comments :
Either in app or database, makes no difference to me.
About hundred "realisations" in database.
All I need is there ID, no need for full object.
MySQL database behind it all.
EDIT
After a little investigation, here is how I've done it :
Define jpa.dialect in application.conf : jpa.dialect=org.hibernate.dialect.MySQLDialect
Fetch a complete object instead of just id with classic Model utilities :
Realisation r = Realisation.find("order by RAND()").first();
After a little investigation, here is how I've done it. Define jpa.dialect in application.conf :
jpa.dialect=org.hibernate.dialect.MySQLDialect
Fetch a complete object instead of just id with classic Model utilities :
Realisation r = Realisation.find("order by RAND()").first();
Not the best way possible, since I only need the ID and not the complete object. Anyway, I have no other solution. If anyone has one for just the ID I will take it.
There is no "limit" clause in JPQL, what you need is paging. You can use Query.setMaxResults instead if this is a JPQL query, which is not entirely clear in the post.