I need help with a project. Right now, I have a giant postgreSQL database (~6 million rows, 25 columns) and I need to figure out how to get the following information:
For one specific range for one attribute, "20<np<400":
Find local minimum values in another variable, "itt"
Record whole row of data for that local minimum
Add extra column and add to this the next local maximum of itt
Edit: the attribute are as follows: The database has this schema: id | at | itt | engine_torque | ng | np | fuel_flow | engine_oil_pressure | engine_oil_temp | airspeed | altitude | total_air_temp | weight_on_wheels | p25_p3 | bypass | chip_counter | number_of_flight_counter | number_of_engine_run | run_id | ectm_file_id | aircraft_sn | engine_sn | gas_generator_sn | power_section_sn | tail_number
By 'local minima and maxima' i mean that within a certain range of np, i'm looking for the highest and lowest values of itt in close proximity with respect to time, or "at"
Thanks in advance!!!
Related
I'm trying to export SPSS metadata to a custom format using SPSS syntax. The dataset with value labels contains one or more labels for the variables.
However, now I want to concatenate the value labels into one string per variable. For example for the variable SEX combine or group the rows F/Female and M/Male into one variable F=Female;M=Male;. I already concatenated the code and labels into a new variable using Compute CodeValueLabel = concat(Code,'=',ValueLabel).
so the starting point for the source dataset is like this:
+--------------+------+----------------+------------------+
| VarName | Code | ValueLabel | CodeValueLabel |
+--------------+------+----------------+------------------+
| SEX | F | Female | F=Female |
| SEX | M | Male | M=Male |
| ICFORM | 1 | Yes | 1=Yes |
| LIMIT_DETECT | 0 | Too low | 0=Too low |
| LIMIT_DETECT | 1 | Normal | 1=Normal |
| LIMIT_DETECT | 2 | Too high | 2=Too high |
| LIMIT_DETECT | 9 | Not applicable | 9=Not applicable |
+--------------+------+----------------+------------------+
The goal is to get a dataset something like this:
+--------------+-------------------------------------------------+
| VarName | group_and_concatenate |
+--------------+-------------------------------------------------+
| SEX | F=Female;M=Male; |
| ICFORM | 1=Yes; |
| LIMIT_DETECT | 0=Too low;1=Normal;2=Too high;9=Not applicable; |
+--------------+-------------------------------------------------+
I tried using CASESTOVARS but that creates separate variables, so several variables not just one single string variable. I'm starting to suspect that I'm running up against the limits of what SPSS can do. Although maybe it's possible using some AGGREGATE or OMS trickery, any ideas on how to do this?
First I recreate your example here to demonstrate on:
data list list/varName CodeValueLabel (2a30).
begin data
"SEX" "F=Female"
"SEX" "M=Male"
"ICFORM" "1=Yes"
"LIMIT_DETECT" "0=Too low"
"LIMIT_DETECT" "1=Normal"
"LIMIT_DETECT" "2=Too high"
"LIMIT_DETECT" "9=Not applicable"
end data.
Now to work:
* sorting to make sure all labels are bunched together.
sort cases by varName CodeValueLabel.
string combineall (a300).
* adding ";" .
compute combineall=concat(rtrim(CodeValueLabel), ";").
* if this is the same varname as last row, attach the two together.
if $casenum>1 and varName=lag(varName)
combineall=concat(rtrim(lag(combineall)), " ", rtrim(combineall)).
exe.
*now to select only relevant lines - first I identify them.
match files /file=* /last=selectthis /by varName.
*now we can delete the rest.
select if selectthis=1.
exe.
NOTE: make combineall wide enough to contain all the values of your most populated variable.
I suspect this question is already well-answered but perhaps due to limited SQL vocabulary I have not managed to find what I need. I have a database with many code:description mappings in a single 'parameter' table. I would like to define a query or procedure to return the descriptions for all (or an arbitrary list of) coded values in a given 'content' table with their descriptions from the parameter table. I don't want to alter the original data, I just want to display friendly results.
Is there a standard way to do this?
Can it be accomplished with SELECT or are other statements required?
Here is a sample query for a single coded field:
SELECT TOP (5)
newid() as id,
B.BRIDGE_STATUS,
P.SHORTDESC
FROM
BRIDGE B
LEFT JOIN PARAMTRS P ON P.TABLE_NAME = 'BRIDGE'
AND P.FIELD_NAME = 'BRIDGE_STATUS'
AND P.PARMVALUE = B.BRIDGE_STATUS
ORDER BY
id
I want to produce 'decoded' results like:
| id | BRIDGE_STATUS |
|--------------------------------------|------------ |
| BABCEC1E-5FE2-46FA-9763-000131F2F688 | Active |
| 758F5201-4742-43C6-8550-000571875265 | Active |
| 5E51634C-4DD9-4B0A-BBF5-00087DF71C8B | Active |
| 0A4EA521-DE70-4D04-93B8-000CD12B7F55 | Inactive |
| 815C6C66-8995-4893-9A1B-000F00F839A4 | Proposed |
Rather than original, coded data like:
| id | BRIDGE_STATUS |
|--------------------------------------|---------------|
| F50214D7-F726-4996-9C0C-00021BD681A4 | 3 |
| 4F173E40-54DC-495E-9B84-000B446F09C3 | 3 |
| F9C216CD-0453-434B-AFA0-000C39EFA0FB | 3 |
| 5D09554E-201D-4208-A786-000C537759A1 | 1 |
| F0BDB9A4-E796-4786-8781-000FC60E200C | 4 |
but for an arbitrary number of columns.
I have a text column indicating date i.e. 20170101
UPDATE table_name
SET work_date = to_date(workdate, 'YYYYMMDD');
I used this command to convert it as date. However, I got a odd result. I read though other existing posts but not sure what's wrong here.
+----------+---------------+
| workdate | work_date |
+----------+---------------+
| 20170211 | 2207-05-09 |
| 20170930 | 2209-04-27 |
| 20170507 | 2208-02-29 |
| 20170318 | 2207-08-24 |
+----------+---------------+
I think you must be mistaken about the data you are supplying to to_date.
For example, input to these functions is not restricted by normal ranges, thus to_date('20096040','YYYYMMDD') returns 2014-01-17 rather than causing an error.
Source: https://www.postgresql.org/docs/9.6/static/functions-formatting.html
table A:
lat | long | the_geom | code | sign
13.8433095 | 100.6360357 | 0101000020E61.... | ABC | start_point
13.7544738 | 100.5459646 | 0101000020E6..... | ABC | end_point
13.4124215 | 100.6232332 | 0101000020E61.... | DEF | start_point
13.2423438 | 100.2324426 | 0101000020E6..... | DEF | end_point
table B:
lat | long | the_geom | code
13.7546285 | 100.5458729 | 0101000020E.... | ABC
13.7546698 | 100.5458513 | 0101000020E.... | ABC
13.7547107 | 100.5458233 | 0101000020E.... | DEF
...
I would like to find the nearest point of each point (start and end pojnt) compare with every points with the same code in table B ?
What's the best PostGIS function/PostgreSQL query to solve this ?
If you are using recent versions of the software, you can quickly find the K nearest neighbors to a point in PostGIS using KNN-GiST techniques. Small values of K are fastest, and 1 is about as small as they get, so this should work very well for you. I've only used KNN-GiST with text trigrams, but I know they work with PostGIS, too -- I just don't know the best page to read to get started with it. A web search for "postgis knn gist" shows lots of likely candidates.
After a long time question, i just found the solution of "Nearest Neighbor"
http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgis_nearest_neighbor_generic
I've got a table named FOO with the column ("Porc" |- 3 7 15 50 15 7 3) and I'm copying the numbers to another table, shown below. I'm doing it the hard way, cell for cell, but I was wondering if there is a way to copy that range of the remote table (A2 to the bottom) in a single command.
| Pr (%) | ROE de A | ROE de B |
|--------+----------+----------|
| 3 | -11.43 | -34.29 |
| 7 | 0. | -11.43 |
| 15 | 3.43 | 0. |
| 50 | 12. | 17.14 |
| 15 | 20.57 | 34.29 |
| 7 | 24. | 41.14 |
| 3 | 30.86 | 54.86 |
|--------+----------+----------|
| Média | 11.86 | 16.41 |
| Desvio | 8.37 | 17.61 |
#+TBLFM: #2$1=remote(FOO, A2)::#3$1=remote(FOO, A3)::#4$1=remote(FOO, A4)::etc
Thanks
It seems your answer is in the org-mode manual:
$3 = remote(FOO, ###$2)
copy column 2 from table FOO into
column 3 of the current table For the
second example, table FOO must have at
least as many rows as the current
table. Inefficient for large number of
rows.
A Kind of Corollary: Copying all fields in a given row
So just as:
$3 = remote(FOO, ###$2)
copies all the fields from a given column (col2) into column three of the new table, then:
#3 = remote(FOO, #1$$#)
copies all the fields from a given row (row1) into row 3.
There's something about how this standard reference form #r$c interacts with the ## and $# notation that makes this seem a bit abstruse. e.g. this is all the org manual has to say about this remote reference syntax:
## and $# can be used to get the row or column number of the field where the formula result goes.
Umm…?
Posting this example here because I found it all a bit mystifying and hope this helps some else save a few minutes when dealing with rows and tables in the awesome org-mode