Mapping with two tables using Postgres - postgresql

I have two tables,
Table_1 contains like,
Project_name Date value
P1 15/06/2016
P2 25/04/2017
P3 18/06/2017
P4 12/05/2017
Table_2 contains like,
Name Occ_June_2016 Occ_April_2017 Occ_May_2017 Occ_June_2017
P1 8.1 7.5 6.5 8.2
P2 8.3 7.4 6.0 8.5
P3 8.6 7.1 6.1 8.1
P4 8.8 7.9 6.8 8.9
I want to get the value, based on mapping the project_name and the date.
Here is what i have tried.
Converting table_1 date column to a particular format,
SELECT to_char(to_date(a.date, 'DD-MM-YYYY'), 'mon_YYYY') from table_1 a ;
Output I got as,
jun_2016
apr_2017
jun_2017
may_2017
By using output above i want to search the column in table_2 by matching the column and name,
Am trying to get the columns based on partial match condition,
select column_name from information_schema.COLUMNS as c where c.TABLE_NAME = 'table_2' and c.COLUMN_NAME like '%occ_%';
Here is the output for the above query,
Occ_June_2016
Occ_April_2017
Occ_May_2017
Occ_June_2017
Now i need to take the output of one query to the input of one query
Here is where am stucked. Mapping the date based on name.
My output should be like,
Project_name Date value
P1 15/06/2016 8.1
P2 25/04/2017 7.4
P3 18/06/2017 8.1
P4 12/05/2017 6.8
kindly give me a solution. Thanks in advance

This can be solved with a database function. First, lets create the tables and fill them with example data.
Creating and filling table_1
CREATE TABLE table_1 (project_name TEXT, Date DATE, value TEXT);
INSERT INTO table_1(project_name, Date) VALUES ('P1','06/15/2016'), ('P2','04/25/2017'), ('P3','06/23/2017'), ('P4','05/12/2017') ;
Creating and filling table_2
CREATE TABLE table_2 (name TEXT, occ_june_2016 DECIMAL, occ_april_2017 DECIMAL, occ_may_2017 DECIMAL, occ_june_2017 DECIMAL);
INSERT INTO table_2(name, occ_june_2016, occ_april_2017, occ_may_2017, occ_june_2017) VALUES
('P1', 8.1, 7.5, 6.5, 8.2),
('P2', 8.3, 7.4, 6.0, 8.5),
('P3', 8.6, 7.1, 6.1, 8.1),
('P4', 8.8, 7.9, 6.8, 8.9);
Next, we create the function:
CREATE OR REPLACE FUNCTION getData(projectName TEXT, projectDate DATE)
RETURNS DECIMAL
AS
$$
DECLARE
columnName TEXT := 'Occ_' || trim(to_char(projectDate, 'Month')) || '_' || to_char(projectDate, 'yyyy');
selectQuery TEXT := 'SELECT %s FROM table_2 where name = ''%s'' LIMIT 1';
returnValue DECIMAL;
BEGIN
selectQuery = format(selectQuery, columnName, projectName);
EXECUTE selectQuery INTO returnValue;
RETURN returnValue;
END;
$$ LANGUAGE 'plpgsql';
The function takes 2 arguments. The first one (projectName) represents the name of the project (for example 'P1'). The second one (projectDate) is the date on which we want to retrieve the data (for example '15/06/2016').
We start by generating the name of the date column in table 2 from the the value of the projectDate variable. Once we have a name, we generate a dynamic sql query by replacing the %s placeholders with the values of the columnName and projectName variables respectively.
We execute the query and store the result in the returnValue variable which this function returns.
After we created the tables and the function, we can execute the following query:
SELECT project_name, Date, getData(project_name, Date) FROM table_1;
This query returns the following result:
project_name date getdata
P1 2016-06-15 8.1
P2 2017-04-25 7.4
P3 2017-06-23 8.1
P4 2017-05-12 6.8

Related

Why am I getting error 17410 end of data on my rest api when I add a parameter to the query

I have ORDS 21 installed in a local 19c oracle database. I have created a stored procedure to list all the departments from the dept table with a cursor that lists the employees from the emp table. If I just list all departments and their employees the api works fine, but if I add a parameter to the query to specify which department, I get an error 17410 no data left .
Both queries are backed by plsql stored procedures. I have created many stored procedures using this same format with parameters and nested cursors before without a problem.
MWE for query that works:
create or replace procedure get_dept
as
l_cur sys_refcursor;
begin
open l_cur for
select d.deptno,d.dname,
cursor (select e.empno,e.ename
from emp e
where e.deptno = d.deptno
order by e.deptno,e.empno
) as employees
from dept d
order by d.deptno;
-- return the resultset in json format
apex_json.open_object;
apex_json.write('emps',l_cur);
apex_json.close_object;
end get_dept;
/
MWE for query that does not work:
create or replace procedure get_dept1
(
p_dept_no in varchar2
) as
l_cur sys_refcursor;
begin
open l_cur for
select d.deptno,d.dname,
cursor (select e.empno,e.ename
from emp e
where e.deptno = d.deptno
order by e.deptno,e.empno
) as employees
from dept d
where d.deptno = to_number(p_dept_no)
order by d.deptno;
-- return the resultset in json format
apex_json.open_object;
apex_json.write('emps',l_cur);
apex_json.close_object;
end get_dept1;
/

Query returning different number of rows, results in crosstab error

I get this error with my query, but I can't figure out what is wrong with it:
Query-specified return tuple has 135 columns but crosstab returns 295.
And here's my query: (The part highlighted in bold returns same number of rows when run separately in pgAdmin.)
SELECT X.*, pi.productcode, pi.productitemdesc, pi.retailsalesprice, cat.productcategorydesc FROM (
SELECT * FROM crosstab (
'SELECT a.productitem AS productitemid, l.locationcode, (CASE WHEN SUM(a.netamount) IS NOT NULL THEN SUM(a.netamount) ELSE 0 END) || ''#'' || (CASE WHEN SUM(a.quantity) IS NOT NULL THEN SUM(a.quantity) ELSE 0 END) AS sales_qty FROM invoiceitem a INNER JOIN invoiceinfo b ON a.invoice = b.invoiceid INNER JOIN locationinfo l ON b.location = l.locationid WHERE b.status !=2 AND l.locationtype = 1 AND l.status = 1 AND TO_CHAR (b.invoicedate, ''YYYY-MM-DD'')>=''2018-03-01'' AND TO_CHAR (b.invoicedate, ''YYYY-MM-DD'')<=''2018-03-03'' GROUP BY a.productitem, l.locationcode ORDER BY a.productitem',
'SELECT l.locationcode FROM locationinfo l INNER JOIN invoiceinfo b ON b.location = l.locationid
WHERE b.status !=2 AND l.locationtype = 1 AND l.status = 1 AND TO_CHAR (b.invoicedate, ''YYYY-MM-DD'')>=''2018-03-01'' AND TO_CHAR (b.invoicedate, ''YYYY-MM-DD'')<=''2018-03-03'' GROUP BY l.locationcode order by l.locationcode')
AS (productitemid int, "0007" text,"BE101" text,"BE1013" text,"BE1014" text,"BE102" text,"BE103" text,"BE1034" text,"BE104" text,"BE1040" text,"BE1043" text,"BE1044" text,"BE1045" text,"BE1046" text,"BE105" text,"BE106" text,"BE107" text,"BE108" text,"BE109" text,"BE110" text,"BE111" text,"BE112" text,"BE123" text,"BE1265" text,"BE1266" text,"BE1271" text,"BE1272" text,"BE1273" text,"BE1274" text,"BE1279" text,"BE1280" text,"BE1281" text,"BE1282" text,"BE1351" text,"BE1400" text,"BE1401" text,"BE1404" text,"BE141" text,"BE142" text,"BE193" text,"BE194" text,"BE2125" text,"BE2126" text,"BE2127" text,"BE2128" text,"BE3001" text,"BE3002" text,"BE3005" text,"BE3006" text,"BE3009" text,"BE3010" text,"BE3031" text,"BE3032" text,"BE3121" text,"BE3122" text,"BE3123" text,"BE3124" text,"BE3127" text,"BE3128" text,"BE3131" text,"BE3132" text,"BE3203" text,"BE3204" text,"BE325" text,"BE3253" text,"BE3254" text,"BE326" text,"BE332" text,"BE3503" text,"BE3504" text,"BE355" text,"BE356" text,"BE365" text,"BE366" text,"BE381" text,"BE382" text,"BE383" text,"BE384" text,"BE400" text,"BE401" text,"BE402" text,"BE403" text,"BE405" text,"BE406" text,"BE408" text,"BE409" text,"BE411" text,"BE412" text,"BE4311" text,"BE4316" text,"BE4401" text,"BE4402" text,"BE4521" text,"BE4522" text,"BE4551" text,"BE4552" text,"BE470" text,"BE473" text,"BE475" text,"BE481" text,"BE482" text,"BE601" text,"BE604" text,"BE609" text,"BE610" text,"BE7040" text,"BE7043" text,"BE7045" text,"BE7046" text,"BE7048" text,"BE7049" text,"BE708" text,"BE7111" text,"BE7112" text,"BE7127" text,"BE7128" text,"BE7217" text,"BE7218" text,"BE7307" text,"BE7308" text,"BE7351" text,"BE7352" text,"BE801" text,"BE802" text,"BE803" text,"BE804" text,"BE831" text,"BE832" text,"BE860" text,"BE861" text,"BE862" text,"BE863" text,"BE865" text,"BE981" text,"BE982" text
)) X
LEFT JOIN productitem pi ON X.productitemid = pi.productitemid
LEFT JOIN productcategory cat ON pi.productcategory = cat.productcategoryid
The bold part of
productitemid int, "0007" text,"BE101" text,"BE1013" text,"BE1014" text,"BE102" text,"BE103" text,"BE1034" text,"BE104" text,"BE1040" text,"BE1043" text,"BE1044" text,"BE1045" text,"BE1046" text,"BE105" text,"BE106" text,"BE107" text,"BE108" text,"BE109" text,"BE110" text,"BE111" text,"BE112" text,"BE123" text,"BE1265" text,"BE1266" text,"BE1271" text,"BE1272" text,"BE1273" text,"BE1274" text,"BE1279" text,"BE1280" text,"BE1281" text,"BE1282" text,"BE1351" text,"BE1400" text,"BE1401" text,"BE1404" text,"BE141" text,"BE142" text,"BE193" text,"BE194" text,"BE2125" text,"BE2126" text,"BE2127" text,"BE2128" text,"BE3001" text,"BE3002" text,"BE3005" text,"BE3006" text,"BE3009" text,"BE3010" text,"BE3031" text,"BE3032" text,"BE3121" text,"BE3122" text,"BE3123" text,"BE3124" text,"BE3127" text,"BE3128" text,"BE3131" text,"BE3132" text,"BE3203" text,"BE3204" text,"BE325" text,"BE3253" text,"BE3254" text,"BE326" text,"BE332" text,"BE3503" text,"BE3504" text,"BE355" text,"BE356" text,"BE365" text,"BE366" text,"BE381" text,"BE382" text,"BE383" text,"BE384" text,"BE400" text,"BE401" text,"BE402" text,"BE403" text,"BE405" text,"BE406" text,"BE408" text,"BE409" text,"BE411" text,"BE412" text,"BE4311" text,"BE4316" text,"BE4401" text,"BE4402" text,"BE4521" text,"BE4522" text,"BE4551" text,"BE4552" text,"BE470" text,"BE473" text,"BE475" text,"BE481" text,"BE482" text,"BE601" text,"BE604" text,"BE609" text,"BE610" text,"BE7040" text,"BE7043" text,"BE7045" text,"BE7046" text,"BE7048" text,"BE7049" text,"BE708" text,"BE7111" text,"BE7112" text,"BE7127" text,"BE7128" text,"BE7217" text,"BE7218" text,"BE7307" text,"BE7308" text,"BE7351" text,"BE7352" text,"BE801" text,"BE802" text,"BE803" text,"BE804" text,"BE831" text,"BE832" text,"BE860" text,"BE861" text,"BE862" text,"BE863" text,"BE865" text,"BE981" text,"BE982" text
AND
SELECT l.locationcode FROM locationinfo l INNER JOIN invoiceinfo b ON b.location = l.locationid
WHERE b.status !=2 AND l.locationtype = 1 AND l.status = 1 AND TO_CHAR (b.invoicedate, ''YYYY-MM-DD'')>=''2018-03-01'' AND TO_CHAR (b.invoicedate, ''YYYY-MM-DD'')<=''2018-03-03'' GROUP BY l.locationcode order by l.locationcode
When run seperately, I get 295 results which is correct. However putting it together in the whole query gets the error.
The manual:
The remaining output columns must have the type of the last column of the source_sql query's result, and there must be exactly as many of them as there are rows in the category_sql query's result.
crosstab(text, text) with a dynamic (!) SELECT query as 2nd parameter is very error prone, since the output column list is static. You should also generate the output column list dynamically. Example:
Dynamic alternative to pivot with CASE and GROUP BY
That aside, the different number of rows returned from the same query, may very well be due to a different timezone setting in the two different sessions.
You have the expression TO_CHAR(b.invoicedate, 'YYYY-MM-DD') in your predicates. If b.invoicedate is type timestamptz the result depends on the timezone setting. Consider:
SET timezone = '+10';
SELECT TO_CHAR(timestamptz '2018-04-07 23:30+0', 'YYYY-MM-DD');
to_char
----------
2018-04-08
SET timezone = '-10';
SELECT TO_CHAR(timestamptz '2018-04-07 23:30+0', 'YYYY-MM-DD');
to_char
----------
2018-04-07
Solution
To remove the dependency on the timezone setting, use absolute values.
to_char() is expensive complication in this place to begin with. Drop that and adjust the predicate.
Provide timestamptz values to match your column. This also allows indexes to be used (if applicable).
Return Table Type from A function in PostgreSQL
Use time zone names (not offsets or abbreviations) to avoid more misunderstandings.
Ignoring timezones altogether in Rails and PostgreSQL
Use dollar-quoting to clean up the quote mess.
Insert text with single quotes in PostgreSQL
I suggest this as 2nd parameter:
$$
SELECT l.locationcode
FROM locationinfo l
JOIN invoiceinfo b ON b.location = l.locationid
WHERE b.status <> 2 AND l.locationtype = 1 AND l.status = 1
AND b.invoicedate >= timestamptz '2018-03-01 Europe/Vienna'
AND b.invoicedate < timestamptz '2018-03-04 Europe/Vienna'
GROUP BY 1
ORDER BY 1
$$
Replace Europe/Vienna in my example with the time zone name defining your dates.
Note 2018-03-04 to include all of 2018-03-03 like you intended.

like with nvarchar is not working correctly?

Using Microsoft SQL Server 2012 - 11.0.5058.0 (X64) Express Edition (64-bit) on Windows NT 6.1 (Build 7601: Service Pack 1) (Hypervisor)
with that creation/fill script :
CREATE TABLE tbl (col CHAR (32) )
insert into tbl values ('test')
Those kind of statements :
declare #var varchar(32) = 'test'
delete from tbl where col like #var
or
delete from tbl where col like 'test'
actually delete the line but why this one :
declare #nvar nvarchar(32) = 'test'
delete from tbl where col like #nvar
do not delete the line ?
According to this page... https://msdn.microsoft.com/en-us/library/ms179859.aspx
When you use Unicode data (nchar or nvarchar data types) with LIKE,
trailing blanks are significant;
Since you are using CHAR data type with length 32, the actual data stored is "test" + 28 spaces.
In your comparison, you are mixing CHAR and nvarchar. Because of the differing data types, SQL Server converts the CHAR data type to NCHAR to perform the comparison.
If you change the data type of the column to VARCHAR, your code works. You could also change your code to:
delete from tbl where col like #nvar + '%'

Dynamic pivot - how to obtain column titles parametrically?

I wish to write a Query for SAP B1 (t-sql) that will list all Income and Expenses Items by total and month by month.
I have successfully written a Query using PIVOT, but I do not want the column headings to be hardcoded like: Jan-11, Feb-11, Mar-11 ... Dec-11.
Rather I want the column headings to be parametrically generated, so that if I input:
--------------------------------------
Query - Selection Criteria
--------------------------------------
Posting Date greater or equal 01.09.10
Posting Date smaller or equal 31.08.11
[OK] [Cancel]
the Query will generate the following columns:
Sep-10, Oct-10, Nov-10, ..... Aug-11
I guess DYNAMIC PIVOT can do the trick.
So, I modified one SQL obtained from another forum to suit my purpose, but it does not work. The error message I get is Incorrect Syntax near 20100901.
Could anybody help me locate my error?
Note: In SAP B1, '[%1]' is an input variable
Here's my query:
/*Section 1*/
DECLARE #listCol VARCHAR(2000)
DECLARE #query VARCHAR(4000)
-------------------------------------
/*Section 2*/
SELECT #listCol =
STUFF(
( SELECT DISTINCT '],[' + CONVERT(VARCHAR, MONTH(T0.RefDate), 102)
FROM JDT1
FOR XML PATH(''))
, 1, 2, '') + ']'
------------------------------------
/*Section 3*/
SET #query = '
SELECT * FROM
(
SELECT
T0.Account,
T1.GroupMask,
T1.AcctName,
MONTH(T0.RefDate) as [Month],
(T0.Debit - T0.Credit) as [Amount]
FROM dbo.JDT1 T0
JOIN dbo.OACT T1 ON T0.Account = T1.AcctCode
WHERE
T1.GroupMask IN (4,5,6,7) AND
T0.[Refdate] >= '[%1]' AND
T0.[Refdate] <= '[%2]'
) S
PIVOT
(
Sum(Amount)
FOR [Month] IN ('+#listCol+')
) AS pvt
'
--------------------------------------------
/*Section 4*/
EXECUTE (#query)
I don't know SAP, but a couple of things spring to mind:
It looks like you want #listCol to contain a collection of numbers within square brackets, for example [07],[08],[09].... However, your code appears not to put a [ at the start of this string.
Try replacing the lines
T0.[Refdate] >= '[%1]' AND
T0.[Refdate] <= '[%2]'
with
T0.[Refdate] >= ''[%1]'' AND
T0.[Refdate] <= ''[%2]''
(I also added a space before the AND in the first of these two lines while I was editing your question.)

Loading, listing, and using R Modules and Functions in PL/R

I am having difficulty with:
Listing the R packages and functions available to PostgreSQL.
Installing a package (such as Kendall) for use with PL/R
Calling an R function within PostgreSQL
Listing Available R Packages
Q.1. How do you find out what R modules have been loaded?
SELECT * FROM r_typenames();
That shows the types that are available, but what about checking if Kendall( X, Y ) is loaded? For example, the documentation shows:
CREATE TABLE plr_modules (
modseq int4,
modsrc text
);
That seems to allow inserting records to dictate that Kendall is to be loaded, but the following code doesn't explain, syntactically, how to ensure that it gets loaded:
INSERT INTO plr_modules
VALUES (0, 'pg.test.module.load <-function(msg) {print(msg)}');
Q.2. What would the above line look like if you were trying to load Kendall?
Q.3. Is it applicable?
Installing R Packages
Using the "synaptic" package manager the following packages have been installed:
r-base
r-base-core
r-base-dev
r-base-html
r-base-latex
r-cran-acepack
r-cran-boot
r-cran-car
r-cran-chron
r-cran-cluster
r-cran-codetools
r-cran-design
r-cran-foreign
r-cran-hmisc
r-cran-kernsmooth
r-cran-lattice
r-cran-matrix
r-cran-mgcv
r-cran-nlme
r-cran-quadprog
r-cran-robustbase
r-cran-rpart
r-cran-survival
r-cran-vr
r-recommended
Q.4. How do I know if Kendall is in there?
Q.5. If it isn't, how do I find out what package it is in?
Q.6. If it isn't in a package suitable for installing with apt-get (aptitude, synaptic, dpkg, what have you), how do I go about installing it on Ubuntu?
Q.7. Where are the installation steps documented?
Calling R Functions
I have the following code:
EXECUTE 'SELECT '
'regr_slope( amount, year_taken ),'
'regr_intercept( amount, year_taken ),'
'corr( amount, year_taken ),'
'sum( measurements ) AS total_measurements '
'FROM temp_regression'
INTO STRICT slope, intercept, correlation, total_measurements;
This code calls the PostgreSQL function corr to calculate Pearson's correlation over the data. Ideally, I'd like to do the following (by switching corr for plr_kendall):
EXECUTE 'SELECT '
'regr_slope( amount, year_taken ),'
'regr_intercept( amount, year_taken ),'
'plr_kendall( amount, year_taken ),'
'sum( measurements ) AS total_measurements '
'FROM temp_regression'
INTO STRICT slope, intercept, correlation, total_measurements;
Q.8. Do I have to write plr_kendall myself?
Q.9. Where can I find a simple example that walks through:
Loading an R module into PG.
Writing a PG wrapper for the desired R function.
Calling the PG wrapper from a SELECT.
For example, would the last two steps look like:
create or replace function plr_kendall( _float8, _float8 ) returns float as '
agg_kendall(arg1, arg2)
' language 'plr';
CREATE AGGREGATE agg_kendall (
sfunc = plr_array_accum,
basetype = float8, -- ???
stype = _float8, -- ???
finalfunc = plr_kendall
);
And then the SELECT as above?
Thank you!
Overview
These steps list how to call an R function from PostgreSQL using PL/R.
Prerequisties
You must already have PostgreSQL, R, and PL/R installed.
Steps
Find R Module name (e.g., Kendall)
Change to the database user:
sudo su - postgres
Run R
R
Install R Module (accept $HOME/R/x86_64-pc-linux-gnu-library/2.9/):
install.packages("Kendall", dependencies = TRUE)
Choose a CRAN Mirror, when prompted.
Create the following table:
CREATE TABLE plr_modules (
modseq int4,
modsrc text
);
Insert into that table the directive to load the R Module in question:
INSERT INTO plr_modules
VALUES (0, 'library(Kendall)' );
Restart the database (or SELECT * FROM reload_plr_modules();):
sudo /etc/init.d/postgresql-8.4 restart
Create a wrapper function in PostgreSQL:
CREATE OR REPLACE FUNCTION climate.plr_corr_kendall(
double precision[],
double precision[] )
RETURNS double precision AS
$BODY$
Kendall(arg1, arg2)
$BODY$
LANGUAGE 'plr' VOLATILE STRICT;
Create a function that uses the wrapper function.
Test the new function.
Wrapper Function
This function performs the work of gathering data from the database and creating two arrays. These arrays are passed into the plr_corr_kendall wrapper function.
CREATE OR REPLACE FUNCTION climate.analysis_vector()
RETURNS double precision AS
$BODY$
DECLARE
v_year_taken double precision[];
v_amount double precision[];
i RECORD;
BEGIN
FOR i IN (
SELECT
extract(YEAR FROM m.taken) AS year_taken,
avg( m.amount ) AS amount
FROM
climate.city c,
climate.station s,
climate.station_category sc,
climate.measurement m
WHERE
c.id = 5148 AND
earth_distance(
ll_to_earth(c.latitude_decimal,c.longitude_decimal),
ll_to_earth(s.latitude_decimal,s.longitude_decimal)) <= 30 AND
s.elevation BETWEEN 0 AND 3000 AND
s.applicable AND
sc.station_id = s.id AND
sc.category_id = 1 AND
extract(YEAR FROM sc.taken_start) >= 1900 AND
extract(YEAR FROM sc.taken_end) <= 2009 AND
m.station_id = s.id AND
m.taken BETWEEN sc.taken_start AND sc.taken_end AND
m.category_id = sc.category_id
GROUP BY
extract(YEAR FROM m.taken)
ORDER BY
extract(YEAR FROM m.taken)
) LOOP
SELECT array_append( v_year_taken, i.year_taken ) INTO v_year_taken;
SELECT array_append( v_amount, i.amount::double precision ) INTO v_amount;
END LOOP;
RAISE NOTICE '%', v_year_taken;
RAISE NOTICE '%', v_amount;
RETURN climate.plr_corr_kendall( v_year_taken, v_amount );
END;
$BODY$
LANGUAGE 'plpgsql' VOLATILE
COST 100;
Test
Test the function as follows:
SELECT
*
FROM
climate.analysis_vector();
Result
A number: -0.0578900910913944