Convert pyspark code to snowflake to create row level policy in snowflake - pyspark

I am trying to convert pyspark code to snowflake to create row level policy in snowflake.
I am new to snowflake and not sure how to add split and case statement in snowflake row level policy.
Pyspark code
df=df.withColumn('first_part',upper(split(col('id'),'#').getItem(0)))\
.withColumn('last_part',split(col('id'),'#').getItem(1))
df.createOrReplaceTempView("df_table")
df1.createOrReplaceTempView("df1_table")
joined_df=spark.sql("select c.*, case when first_part == 'emp' then '1' else p.flag end as flag, p.agreement_date from df_table c left join df1_table p on c.last_part = p.empid")
Snowflake part
create or replace row access policy policy.policy_row_hi as (col1 varchar) returns boolean ->
exists
(select 1
from schema.table1 t1
inner join schema.table2 t2 on (t2.oid = t1.oid)
where t2.empid = col1
and t1.flag = '1'
);

Related

Empty data in SSRS Reports

I'm working on ssrs reports. I was able to see the data or result of my stored procedure.unfortunatesly, when used the same as my dataset for report I was unable to see the data instead i'm getting 0 records.what might be the reasons ?My reports structure will be as like below image :
My current result :
Below is my procedure :
ALTER Proc [dbo].[SP_Get_CIPPSubjectMarks_New_HTSTEST] -
-7,'1,17,8','2537,2555,2558,2568'
(
#ReportId int=7,
#SubjectId varchar(200),
#SectionId varchar(200)
)
AS
BEGIN
Create table #temp (Name Varchar(500),Class varchar(50),Section
Varchar(20),enrollno varchar(500),SubjectName varchar(500),TermName
varchar(500),TestName varchar(500),TestGroupName varchar(500),Weightage
int,IsWeight bit,Marks varchar(20),MaxMarks int,IsAbsent bit,SubjectOrder
Varchar(200))
Insert into
#temp(Name,Class,Section,enrollNo,SubjectName,
TermName,TestName,TestGroupName,Weightage,IsWeight,Marks,
MaxMarks,IsAbsent,SubjectOrder)
SELECT DISTINCT CONCAT(d.name,' ',d.surname),cls.Value,sec.Value,
e.enroll_no,
CASE WHEN ISNULL(cxs.subject_alias,'')='' THEN CASE WHEN rtv.value='Second
Language' then '2ND LANGUAGE:' + b.Name WHEN
rtv.value='Third Language' THEN '3rd Language:'+b.name else b.name end
ELSE cxs.subject_alias end as SubjectName,
z.str_termname,c.str_termtestname,i.str_testgroupname,
i.str_testweightage,i.is_weighted_average,a.marks,max_marks,a.is_absent,
CASE WHEN rtv.value='Second Language' THEN 'Second Language' WHEN
rtv.Value='Third Language' THEN 'Third Language' When
ISNULL(cxs.subject_alias,'')='' THEN b.Name
ELSE cxs.subject_alias end as SubOrder
FROM marks_entry_HTS a JOIN subject b ON a.fk_subject_id=b.Id and a.marks
is not null
LEFT JOIN subjectCategory_HTS l ON l.Id= b.subject_categoryID
JOIN class_term_test_mapping_HTS c ON a.fk_class_term_test_mapping_id=c.id
-- added by me
JOIN class_term_test_category_HTS ctc on c.fk_termcategoryid = ctc.id
JOIN reference_type_value rtv ON rtv.id=a.fk_subject_type_id
-- close
JOIN Term_Test_Subject_AssessmentType_HTS m ON m.fk_term_testID=c.id and
m.fk_SubjectID=b.Id
JOIN class_report_types_mapping_test_HTS k ON
k.fk_class_term_test_mapping_id=c.Id
JOIN class_term_mapping_HTS z ON z.id=k.fk_class_term_mapping_id
JOIN Term_Test_Testgroup_aggregate_HTS i ON
i.Id=c.fk_testgroup_aggregateID
JOIN TestGroup_HTS j on j.Id=i.fk_TestGroupID
JOIN student d ON a.fk_student_id=d.Id JOIN student_enroll_no e ON
e.fk_student_id=d.id and IsNULL(e.is_deleted,0)=0
JOIN student_academic f on f.fk_student_enroll_no_id=e.id and
f.fk_academic_year_id=c.fk_academic_year_id
JOIN reference_type_value cls on cls.Id=f.fk_class_id
LEFT JOIN reference_type_value sec ON sec.Id=f.fk_section_id
LEFT JOIN max_marks_entry_HTS h on h.id=a.fk_max_marks_entry_id
join class_xref_subjects cxs ON h.fk_subject_id=cxs.fk_subject_id and
cxs.fk_subject_type_id=h.fk_subject_type_id and
IsNull(cxs.is_deleted,0)=0
and cxs.fk_class_id=h.fk_class_id and
cxs.fk_academic_year_campus_id=h.fk_academic_year_campus_id and
cxs.fk_curriculum_segment_id=h.fk_curriculum_segment_id
where k.fk_class_report_types_mapping_id=#ReportId
and h.fk_section_id in (select * from SplitStringByChar(#SectionId,','))
and a.fk_subject_id in (select * from SplitStringByChar(#SubjectId,','))
select Name,Class,Section,enrollNo,SubjectName,TermName,
TestGroupName as TestName,
Case WHEN IsWeight=1 THEN Round(Cast(((avg(CAST(Marks as
float)/cast(MaxMarks as float)))*Weightage) as decimal(10,0)),0)
else Round(Cast(((cast(max(Marks) as float)/cast(max(MaxMarks) as
float))*Weightage) as decimal(10,0)),0) ENd as Marks ,
SubjectOrder ,sum(maxmarks) as maxmarksare INTO #temp1 from #temp
GROUP BY Name,Class,Section,enrollNo,SubjectName,
TermName,SubjectOrder,IsWeight,Weightage,TestGroupName
Insert into #temp
(Name,Class,Section,enrollNo,SubjectName,
TermName,TestName,Marks,SubjectOrder)
select
Name,Class,Section,enrollNo,SubjectName,TermName,'Total',
SUM(Marks),SubjectOrder from #temp1
GROUP BY Name,Class,Section,enrollNo,SubjectName,TermName,SubjectOrder
Insert into #temp
(Name,Class,SubjectName,Section,enrollNo,TermName,TestName,Marks)
select Name,Class,'Total',Section,enrollNo,TermName,'Total
Marks',SUM(Marks)
from #temp1
GROUP BY Name,Class,Section,enrollNo,TermName
Insert into #temp
(Name,Class,SubjectName,Section,enrollNo,TermName,TestName,Marks)
select
Name,Class,'Total',Section,enrollNo,TermName,'Percentage',
SUM(Marks)*100/sum(m
axmarksare) from #temp1
GROUP BY Name,Class,Section,enrollNo,TermName
select *from #temp
drop table #temp
drop table #temp1
end
My procedure result is as like below image :
As there are a number of things that could be wrong I would do the following.
Create copy of your report
Remove the existing dataset and tablix if you want.
Create some new datasets that just get the basic data (e.g. SELECT top 10 * FROM marks_emtry_HTS). Do not use your parameters yet as we just want to test we can get basic data.
Add some tables to your report to show that the data is being returned
Add a dataset to test you are passing and parsing parameters correctly by using a dataset query like select * from SplitStringByChar(#SectionId,',') and then put tablix on your report to show the results.
Try trimming your parameter values SET #SectionId = LTRIM(RTRIM(#SectionId)) to make sure you're not handling leading or trailing space incorrectly in you split function.
If any parts do not work, run a trace on the SQL Server as you run the report and look at exactly what is being executed on the server.
I know a lot of those steps you will think might be unnecessary but take the time to do it and at least you are certain and you can exclude such basic checks from your investigation.

How Dynamicaly columns in UNPIVOT operator

I currently have the following query:
WITH History AS (
SELECT
kz.*,
kz.__$operation AS operation,
map.tran_begin_time as beginT,
map.tran_end_time as endT
FROM cdc.fn_cdc_get_all_changes_dbo_EXT_GeolObject_KategZalezh(sys.fn_cdc_get_min_lsn('dbo_EXT_GeolObject_KategZalezh'), sys.fn_cdc_get_max_lsn(), 'all') AS kz
INNER JOIN [cdc].[lsn_time_mapping] map
ON kz.[__$start_lsn] = map.start_lsn
where kz.GUID_BalanceHC_Zalezh = 'DDA9AB3A-A0AF-4623-9362-0000C8C83D63'
),
UnpivotedValues AS(
SELECT guid, GUID_another, field, val, operation, beginT, endT
FROM History
UNPIVOT ( [val] FOR field IN
(
area,
oilwidthmin,
oilwidthmax,
efectivwidthmin,
efectivwidthmax,
etc...
))t
),
UnpivotedWithLastValue AS (
SELECT
*,
--Use LAG() to get the last value for the same field
LAG(val, 1) OVER (PARTITION BY guid, GUID_another, field ORDER BY BeginT) LastVal
FROM UnpivotedValues
)
SELECT * FROM UnpivotedWithLastValue WHERE val <> LastVal OR LastVal IS NULL ORDER BY guid
This query returns the changed values for a single table that has CDC (Change Data Capture) enabled.
I want to create a stored procedure that receives the columns to be unpivoted, and the cdc function (e.g. cdc.fn_cdc_get_all_...) as parameters and returns the result set.
The result for this tables must be joined in one report.
In my case parameter 1 is cdc.fn_cdc_get_all_changes_dbo_EXT_GeolObject_KategZalezh(sys.fn_cdc_get_min_lsn('dbo_EXT_GeolObject_KategZalezh'), sys.fn_cdc_get_max_lsn(), 'all'). This is the CDC function.
How should I send the list of fields that i want in the result? How's the string?
Also, is there a way to do without dynamic SQL? Dynamic SQL it is not better solution for performance.
As you know SQL Server is declarative by design and does not support macro substitution.
UNPIVOT would clearly be more performant, but here is a simplified example of a UNPIVOT which does not require Dynamic SQL, but only a little XML.
Example
Let's assume your table/results looks like this:
You may notice that I only we only specify key fields to EXCLUDE in the final WHERE
Declare #YourData table (ID int,Active bit,First_Name varchar(50),Last_Name varchar(50),EMail varchar(50),Salary decimal(10,2))
Insert into #YourData values
(1,1,'John','Smith','john.smith#email.com',85600),
(2,0,'Jane','Doe' ,'jane.doe#email.com',83200)
;with cte as (
-- Replace with your Complex Query
Select * from #YourData
)
Select A.ID
,A.Active
,C.*
From cte A
Cross Apply (Select XMLData=cast((Select A.* for XML RAW) as xml)) B
Cross Apply (
Select Item = attr.value('local-name(.)','varchar(100)')
,Value = attr.value('.','varchar(max)')
From XMLData.nodes('/row') C1(n)
Cross Apply C1.n.nodes('./#*') C2(attr)
Where attr.value('local-name(.)','varchar(100)') not in ('ID','Active')
) C
Returns

multiple where clause in report query ireport jasper

I want to generate a report chart for this data from two tables are required. So i have a query like this ( Am using OrientDB )
select col1,col2 from (select col11,col22 from t1 where col11 = $P{col11}) where col1 = $P{col1} and col2 = $P{col2}
When i run this report i will get following exception
Caused by: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 
    at java.util.ArrayList.rangeCheck(ArrayList.java:635)      at java.util.ArrayList.get(ArrayList.java:411)      at com.orientechnologies.orient.core.sql.filter.OSQLPredicate.bindParameters(OSQLPredicate.java:366) 
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.assignTarget(OCommandExecutorSQLResultsetAbstract.java:182) 
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.assignTarget(OCommandExecutorSQLSelect.java:435) 
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:417) 
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:388) 
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:64) 
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:1163) 
    ... 8 more
As my observation if i have single where condition i.e., either in subquery or outerquery it works, if it is having where clause in both they this is the exception thrown.
Don't know if it solves your problem, but use of aliases could maybe help you? For proper SQL alias on columns and table :
select col1,col2
from ( select col11 as col1, col22 as col2
from t1 where col11 = $P{col11}) as table
where col1 = $P{col1} and col2 = $P{col2}

Bulk update a column in Oracle 11G

I have two tables say Table1 and Table2 that contains the following column with which I should join and perform an update a column of Table1 with the value of the same column present in Table2.
Columns for Join condition:
Table1.mem_ssn and Table2.ins_ssn
Table1.sys_id and Table2.sys_id
Table1.grp_id and Table2.grp_id
Column to update:
Table1.dtofhire=Table2.dtofhire
I need a way to bulk update (using single update query without looping) the above mentioned column in Oracle 11G.
Table1 does not contain any key constraint specified since it will be used as a staging table for Data upload.
Please help me out to update the same.
You can use the MERGE statement.
It should look something like this:
MERGE INTO table1 D
USING (SELECT * FROM table2 ) S
ON (D.mem_ssn = S.ins_ssn and D.sys_id = S.sys_id and D.grp_id=S.grp_id)
WHEN MATCHED THEN
UPDATE SET D.dtofhire=S.dtofhire;
UPDATE:
Since you have more than one row in table2 with the same (ins_ssn,sys_id,grp_id) and you want the max dtofhire, you should change the query in the using clause:
MERGE INTO table1 D
USING (SELECT ins_ssn, sys_id, grp_id, max(dtofhire) m_dtofhire
FROM table2
GROUP BY ins_ssn,sys_id,grp_id) S
ON (D.mem_ssn = S.ins_ssn and D.sys_id = S.sys_id and D.grp_id=S.grp_id)
WHEN MATCHED THEN
UPDATE SET D.dtofhire=S.m_dtofhire;
The query that I used to arrive the functionality is seen below
UPDATE table1 T2
SET dtofhire = (SELECT Max(dtofhire) AS dtofhire
FROM table2 T1
WHERE T2.mem_ssn = T1.ins_ssn
AND T2.sys_id = T1.sys_id
AND T2.grp_id = T1.grp_id
GROUP BY ins_ssn,
sys_id,
grp_id)
WHERE ( mem_ssn, sys_id, grp_id ) IN (SELECT ins_ssn,
sys_id,
grp_id
FROM table2 );

T-SQL Query, combine columns from multiple rows into single column

I have seeen some examples of what I am trying to do using COALESCE and FOR XML (seems like the better solution). I just can't quite get the syntax right.
Here is what I have (I will shorten the fields to only the key ones):
Table Fields
------ -------------------------------
Requisition ID, Number
IssuedPO ID, Number
Job ID, Number
Job_Activity ID, JobID (fkey)
RequisitionItems ID, RequisitionID(fkey), IssuedPOID(fkey), Job_ActivityID (fkey)
I need a query that will list ONE Requisition per line with its associated Jobs and IssuedPOs. (The requisition number start with "R-" and the Job Number start with "J-").
Example:
R-123 | "PO1; PO2; PO3" | "J-12345; J-6780"
Sure thing Adam!
Here is a query that returns multiple rows. I have to use outer joins, since not all Requisitions have RequisitionItems that are assigned to Jobs and/or IssuedPOs (in that case their fkey IDs would just be null of course).
SELECT DISTINCT Requisition.Number, IssuedPO.Number, Job.Number
FROM Requisition
INNER JOIN RequisitionItem on RequisitionItem.RequisitionID = Requisition.ID
LEFT OUTER JOIN Job_Activity on RequisitionItem.JobActivityID = Job_Activity.ID
LEFT OUTER JOIN Job on Job_Activity.JobID = Job.ID
LEFT OUTER JOIN IssuedPO on RequisitionItem.IssuedPOID = IssuedPO.ID
Here's one way to do it using subqueries:
select 'R-' + cast(r.number as varchar(32)) as RequisitionNumber
, (
select 'PO' + CAST(ip.number as varchar(32)) + ';'
from IssuedPO ip
join RequisitionItems ri
on ip.id = ri.IssuedPOID
where ri.RequisitionID = r.id
for xml path('')
) as POList
, (
select 'J-' + CAST(j.number as varchar(32)) + ';'
from Job j
join Job_Activity ja
on j.id = ja.JobID
join RequisitionItems ri
on ri.Job_ActivityID = ja.id
where ri.RequisitionID = r.id
for xml path('')
) as JobList
from Requisition r