I have created a custom classification in Purview which is looking for a specific column name from a SQL Server source.
The scan does pick up the classification and assign it to the column accordingly, however when i delete the classification rule, I select "Delete this rule and delete all applied classifications", the column that was detected still has the classification assigned to it, and I am unable to delete the custom classification as it is still referenced by the column.
The workaround is to manually remove it from the column first which is not ideal, especially if it is assigned to multiple columns.
In purview, create custom classification to detect specific column text.
Perform scan, new classification is applied to column.
Delete custom classification rules and select "Delete this rule and delete all applied classifications"
Try and delete custom classification - unable to as it is still referenced by the column that was detected by the scan.
Related
I am coping data from a rest api to an azure SQL database. The copy is working find but there is a column which isn't being return within the api.
What I want to do is to add this column to the source. I've got a variable called symbol which I want to use as the source column. However, this isn't working:
Mapping
Any ideas?
This functionality is available using the "Additional Columns" feature of the Copy Activity.
If you navigate to the "Source" area, the bottom of the page will show you an area where you can add Additional Columns. Clicking the "New" button will let you enter a name and a value (which can be dynamic), which will be added to the output.
Source(s):
https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview#add-additional-columns-during-copy
Per my knowledge, the copy activity may can't meet your requirements.Please see the error conditions in the link:
Source data store query result does not have a column name that is
specified in the input dataset "structure" section.
Sink data store (if with pre-defined schema) does not have a column
name that is specified in the output dataset "structure" section.
Either fewer columns or more columns in the "structure" of sink
dataset than specified in the mapping.
Duplicate mapping.
I think Mapping Data Flow is your choice.You could add a derived column before the sink dataset and create a parameter named Symbol.
Then set the derived column as the value of Symbol.
You can use the Copy Activity with a stored proc sink to do that. See my answer here for more info.
I'm adding Row-level security to a Power BI model and it's working on all the normal tables, but there's a DAX table created from a SUMMARIZE that's ignoring the security. I've created a small model that reproduces the issue - am I doing something wrong here, or is this behavior intentional?
Here's my data model - "Raw Table" is just a manual table with some numbers in it and a filter column:
"DAX Table" is created with a SUMMARIZE statement that just makes it a copy of the manual table (since every row is unique anyways):
DAX Table = SUMMARIZE('Raw Table', 'Raw Table'[Number], 'Raw Table'[Filter])
Here's what the content of the tables look like when nothing is filtered:
I've added row-level security that filters the Filter table to just the "Show" records:
And through the join, this appears to filter the Raw table normally - when it's enabled, I can only see the rows with "Show" in them (as expected). However, the DAX table continues to show everything, even the filtered rows that I'm not supposed to see:
What am I doing wrong here? Since I'm not allowed (via RLS) to see the rows labelled "Hide" in the raw table, shouldn't those also be excluded from the SUMMARIZE statement? I realize that adding a slicer on the Filter table and slicing the Raw table doesn't impact the SUMMARIZE statement (I still see all the rows in that case) - am I just seeing the same thing here? Since I'm prevented from seeing these rows based on a security setting, I'd expect them to be hidden from my DAX as well.
Assuming I'm doing this correctly and this is intentional, how can I create a DAX table that does respect my security filters (as I need to do this in my full model)?
Here's a link to the testing PBIX file as well.
Calculated Tables (and columns) get calculated when the model is loaded, and materialized. Not at runtime.
You will need to connect that table to your model so security propagates there aswell.
continuing a project I have posted a few questions for already are you able to have a more precise definition of what a dropdown is populated with?
I am using FM Starting Point and have a Projects, Estimate and Task table. A project can hold multiple estimates and tasks. Estimates contain multiple lines (stored in a separate table).
When In a Project you can add a task for that project. What I want to do is replace the field that gives the task a name with a list of lines contained within the estimates related to the project i have selected to add the task from.
I have created a dropdown and using the inspector got it to display all estimate lines but somehow I need to write a function that will only select lines from estimates that are within the project and also only estimates that have a field set to "active" for example.
what I am struggling with is where I can programme such a thing? In the inspector, you are limited to displaying a value and cant programme in a statement. I have tried a script that executes OnObjectEnter or OnObjectModify but that doesn't appear to work.
Any ideas where I can enter a more complex set of rules as to what populates a dropdown?
Create a relationship with the criteria you need. Base the value list for the dropdown on this relationship.
As for display, The value list's second field can be from a calculated field with any data or several other fields stringed together if no single field is sufficient. The first field should be the ID.
Source & target tables are similar.
Target table has a UUID field that is computed in tMap, however the flow should not insert duplicate persons in target i.e unique (firstname,lastname,dob,gender). I tried marking those columns as key in tMap as in below screenshot, but that does not prevent duplicate inserts. How can I avoid duplicate inserts without adding unique constraint on target?
I also tried "using field" in target.
Edit: Solution as suggested below:
The CDC components in the Paid version of Talend Studio for Data Integration undoubtedly address this.
In Open Studio, you'll can roll your own Change data capture based on the composite, unique key (firstname,lastname,dob,gender).
Use tUniqueRow on data coming from stage_geno_patients, unique on the following columns: firstname,lastname,dob,gender
Feed that into a tMap
Add another query as input to the tMap, to perform look-ups against the table behind "patients_test", to find a match on the firstname,lastname,dob,gender. That lookup should "Reload for each row" using looking up against values from the staging row
In the case of no-match, detect it and then do an insert of the staging row of data into the table behind "patients_test"
Q: Are you going to update information, also? Or, is the goal only to perform unique inserts where the data is not already present?
In DB2 9.7, when you CREATE a table and specify a column with identity. You can specify the method in which it will handle the generation. You have two choices GENERATED ALWAYS or GENERATED BY DEFAULT.
Once the table has been created how can you tell which method it's using (without performing an insert)?
From the Control Center, I tried to generate the DDL to see how it would generate it, but I'm not certain this is accurate.
You can find all schema-related information in the DB2 system catalog. The view SYSCAT.COLUMNS holds the core data about columns and their properties. To determine whether a column is GENERATED ALWAYS or GENERATED BY DEFAULT look at the column GENERATED in that column.