I am working on an app that uses HealthKit framework to pull data and display it to the user. I have been able to check how many meters has the user walked in a given HKWorkoutSession, by creating a HKStatisticsQuery.
Now, I want to get more details about this data. Basically, I want to divide the total distance walked in splits of 1 km, and display how much time each of these splits took. I haven't found any solutions online to achieve this.
Related
TL;DR:
Can I get Grafana to show me the previous data point, when the currently selected time period does not have a data point? I have an example which sounds ridiculous, but at least it's simple to understand: I send data every 1 minute, and I wish to zoom into the last 30 seconds, and still see data. You may ask "why not just zoom out to 2 minutes" but the reason is that other data is on the same graph that has updated more often, and I wish to compare with that data. Also, for the more lengthy reasons below.
If not, how can I achieve what I want to achieve, see below?
Context
For a few years, I have been monitoring the water level in three of our basement sumps (which have pumps installed) by sending this data from Node-RED to InfluxDB, then visualising the sump levels in Grafana. I have set up three waterproof ultrasonic distance sensors, each pointed down a pipe that is inserted vertically into each sump. The water fills the pipe and the distance sensor, connected to an Arduino, sends me the reading. The Arduino also has other sensors connected (temp / humidity) and deals with distance calibrations to calculate the percent full of each sump. All this data is sent to Node-RED. In total, I am sending 4 values per sump: distance measurement in mm, percent full, temp, humidity. So that's 12 fields. Data is sent every 2 seconds, because I wished to have a reasonably high resolution to see nice curves in graphs.
Also I decided to store all this data so that I could later troubleshoot issues (we have had sewage floods resulting in water not being able to be pumped away, etc...) and design some warning systems for these issues based on data.
Storing 12 values for every 2 seconds, over the course of a number of years, takes up a lot of space (8GB).
Nature of the data
Storing this resolution of data has also helped me be able to describe the nature of the data. I will do so here.
(1) Non-meaningful NOISE (see below) - the percent-full reading goes up and down by 1 or 2 percent every couple of seconds:
(2) Meaningful DRIFT (see below) - I don't mean sensor drift, I am referring to actual water levels changing slowly over time, e.g. over 1 day or 1 week. Perhaps condensation on the walls drips down into the sump, or water evaporates from the sump, and the value can waver by a few percent over the course of a day. Each sump has slightly different characteristics.
(3) Meaningful MONITORING DATA - during wet weather, depending on rainfall amount, the sumps fill up over the course of say 30 mins to 3 hours. Then the pumps run and the water level drops again, wavers a bit, then the sumps continue to fill up. If the rain stopped, you can see a lovely curve as the water fills in progressively more slowly (see the green line below):
Solution to downsample
I know Influx has its own downsampling possibilities, however because of the nature of the data (which can hardly vary for 2 months but when it does, I really need to capture it in detail), I don't think lowering the sample rate is a great idea.
I have some understanding of digital filters (e.g. low pass etc) but have never programmed one myself. So I have written a basic filter in javascript (a Node-RED function) to filter the data in realtime as follows: only send each reading when it has changed from the previous one by x amount. (And update the previous one, when that occurs.)
This has already vastly reduced the amount of data being stored, and I can vary x to filter out noise shown in my first graph above, at the expense of resolution when the pumps run. Even if I set the x value to 2, it still vastly reduces data over long periods of dry weather.
So - onto my problem! Now data is not being logged to InfluxDB unless there is some meaningful change. Which means that when I zoom in to e.g. 15 minute timeframe of data, there is nothing to see.
Grafana does have the option of "fill (previous)" but this draws a line between points on the existing graph, rather than showing the previous data as if it hasn't changed since that point. Now my grafana dashboard looks a bit sad :(
One proposed solution is, in addition to sending "delta" data, send "summary" data, that is - send a full suite of data every 1 minute regardless of whether data changed or not. But then we get noise back again, and pointless storage.
Any other ideas?
I am trying to retrieve current speed limit of a given road location data is passed to the Open Street Maps api every 2-3 seconds as the vehicle is in motion.
I have tried the code below using [around] however it only outputs results when the radius is at least 2500 metres
i am seeking to have a an area that is closer to 10 meters to determine the current speed limit of the vehicles location.
If there is an alternative endpoint to achieve the above so that is more robust and responsive please advise
i am basically wanting to replicate what i see on navigation apps where the speed limit value updates within 1-2 seconds of a vehicle passing a new speed sign of value different to the previous speed zone being driven in
I would appreciate any assistance to resolve this issue.
https://lz4.overpass-api.de/api/interpreter?
data=[out:json];nodemaxspeed;out%20meta;
I'm new to Tableau Desktop, so I'm guessing what I want to do is simple, but I don't know how to do it.
Basically, I have basketball data that gives me players total points scored over several seasons with different NBA teams. I'm trying to sort that data by team, based on the amount that each player scored for each specific team.
Right now, I have the data sorted by team, player, and the total number of points scored. The problem is - I don't actually want the total sum. (E.g. right now Shaq is listed first under the Celtics because he has the most career points out of anyone who played for the Celtics, but not for the Celtics themselves.)
Can someone tell me how I would go about sorting by sum points by team?
This is actually such a common issue that Tableau references a solution in their official training material. If my understanding of your requirement is correct, the following should solve the problem.
http://kb.tableau.com/articles/knowledgebase/finding-top-n-within-category
I am using Google Fit REST API (via Google Java Client Library) to post an activity into Google Fit.
In summary what I am doing is creating three DataSets covering the given time period:
"com.google.location.sample" - Location
"com.google.step_count.delta" - Steps
"com.google.calories.expended" - Calories
... then creating a Session, and finaly a DataSet with a single Activity Segment (in this case all the time is walking).
This basically all seems to work - I can looking in http://fit.google.com, and I can see the activity, with the correct time, location, duration, steps and calories. The problem is with the map... all it shows is a shaded circle over the whole area of the walking - it doesn't show the track/path that I included in the location DataSet.
EDIT... Here is an example of what it looks like (in web UI).
Why would this not be showing up correctly, when all of the rest of the activity shows up perfectly?
These are some of my suspicions
My data does not have either altitude or accuracy - which are two of the fields needed by "com.google.location.sample". So I set altitude to 0.0 (metres), and set accuracy to 5.0 (metres). I particularly wonder if Google is reacting badly to me setting the altitude to 0.0 for each point?
My location DataSet has say 100 DataPoints in it, whereas by steps and calories DataSets only have one DataPoint in each - i.e. I only have total steps, and total calories, for the walk. So there's an inconsistency (the earliest start and latest end dates are the same for each data set)
Can anybody give any guidance about why this is happening please?
Think this may be due to conflicting data points. As stated here. Although this is for the API for Android, I think it holds true too when using the REST API.
Each DataPoint in your app's DataSet must have a startTime and an
endTime that defines a unique interval within that DataSet, with no
overlap between DataPoint instances. If your app attepts to insert a
new DataPoint that conflicts with an existing DataPoint instance, the
new DataPoint is discarded. To insert a new DataPoint that may overlap
existing data points, use the HistoryApi.updateData method described in
Update data.
You mentioned that the dates are the same across data points. Thus overriding the others and it is only treated as one.
For your com.google.location.sample data type fields. Think it's better to leave them as is. Try not to place a static value and for altitude and accuracy.
I would like to know if there are some libraries/algorithms/techniques that help to extract the user context (walking/standing) from accelerometer data (extracted from any smartphone)?
For example, I would collect accelerometer data every 5 seconds for a definite period of time and then identify the user context (ex. for the first 5 minutes, the user was walking, then the user was standing for a minute, and then he continued walking for another 3 minutes).
Thank you very much in advance :)
Check new activity recognization apis
http://developer.android.com/google/play-services/location.html
its still a research topic,please look at this paper which discuss the algorithm
http://www.enggjournals.com/ijcse/doc/IJCSE12-04-05-266.pdf
I don't know of any such library.
It is a very time consuming task to write such a library. Basically, you would build a database of "user context" that you wish to recognize.
Then you collect data and compare it to those in the database. As for how to compare, see Store orientation to an array - and compare, the same holds for accelerometer.
Walking/running data is analogous to heart-rate data in a lot of ways. In terms of getting the noise filtered and getting smooth peaks, look into noise filtering and peak detection algorithms. The following is used to obtain heart-rate information for heart patients, it should be a good starting point : http://www.docstoc.com/docs/22491202/Pan-Tompkins-algorithm-algorithm-to-detect-QRS-complex-in-ECG
Think about how you want to filter out the noise and detect peaks; the filters will obviously depend on the raw data you gather, but it's good to have a general idea of what kind of filtering you'd want to do on your data. Think about what needs to be done once you have filtered data. In your case, think about how you would go about designing an algorithm to find out when the data indicates activity (like walking, running,etc.), and when it shows the user being stationary. This is a fairly challenging problem to solve, once you consider the dynamics of the device itself (how it's positioned when the user is walking/running), and the fact that there are very few (if not no) benchmarked algos that do this with raw smartphone data.
Start with determining the appropriate algorithms, and then tackle the complexities (mentioned above) one by one.