Am trying to migrate data from CRM online to Data Lake.
within Copy Activity of ADF, am using FetchXML to read data from CRM (CDM)
FetchXML:
<fetch>
<entity name="cdm_location">
<attribute name="cdm_gisid" />
<attribute name="cdm_locationid" />
<attribute name="cdm_locationtype" />
<attribute name="cdm_name" />
</entity>
</fetch>
Issue is ADF looks and the Sample dataset of above source query and determines the columns to be extracted. In my case column cdm_gisid within the sample dataset is NULL and hence this column is being ignored by ADF.
Is there a way in FetchXML saying if column is null replace with 'Default Value' so that ADF can see the column in the Sample Dataset.
[Or] is there better way in ADF to pull data from CDM, bringing every column irrespective if its null or not null.
You just need to use a column map which can be defined in the Mapping tab of your copy activity. This is a known issue and is recommend in the ADF/D365 documentation.
See the Important note in the following link.
https://learn.microsoft.com/en-us/azure/data-factory/connector-dynamics-crm-office-365#dynamics-as-a-source-type
Related
Am using Azure DataFlow - DerivedColumn to create some new columns.
Ex:
this is my source and can preview the data.
But from DerivedColumn1 i cannot see these column or even in Expression Editor
Expression Editor:
Is something changed in ADF or am I doing something wrong.
According you screenshot, the column name is set as the row. Or you will get the error in Sink column mapping. Please set "first row as header" in the excel dataset.
If you don't check it, the column name will be considered as first row:
For your issue, you could try bellow workarounds:
import the source schema in Projection and Delete the Derived column
active and add again.
Drop the data flow and create a new one. Some time data flow may
have bugs, we refresh the browser or just recreate the data flow, it
will be solved.
There is the "Show fields related to:" feature in Excel:
I want something like this, but return in the following form:
MeasureGroup1: Dimension1, Dimension2, Dimesion3
MeasureGroup2: Dimension2, Dimesion3
...
Can't find out how Excel retrieve this data. I need to write a script which will automatically generate the relationship data for a chosen cube. It can be MDX or XMLA. ADO.NET in the last resort (it should be a portable script after all).
Hope you can help.
The following returns a recordset which Excel uses to determine which dimensions are related to which measure groups:
select *
from $system.MDSCHEMA_MEASUREGROUP_DIMENSIONS
Plus the xmla solution:
<Discover xmlns="urn:schemas-microsoft-com:xml-analysis">
<RequestType>MDSCHEMA_MEASUREGROUP_DIMENSIONS</RequestType>
<Restrictions />
<Properties />
</Discover>
Please guide me about this. I am working on CRM online.
I need report(custom SSRS) which will be invoked from Campaign and it will print all the Leads belongs to that Campaign.
How should i achieve this.
I created a master report based on Campaign, and a child report which is added inside that master report using BIDS.
I am passing campaignId to the child and child report does it work simply.
Is it possible using single report instead of two report...??
Any help would be appreciated.
Thanks
You can create a single report, isn't mandatory to use the Parent-Child relationship that is available with Dynamics CRM.
Inside your report you can create how many dataset you want, just define a dataset with a query (built with FetchXml because you are using CRM Online) that will fetch for the related leads of the selected campaign.
You can use Advanced Find to get the FetchXml to start.
Example:
you have the FetchXml from the Advanced Find, this will retrieve all the leads with the attribute new_campaignid as the selected GUID value.
<fetch version="1.0" output-format="xml-platform" mapping="logical" distinct="false">
<entity name="lead">
<attribute name="fullname" />
<attribute name="companyname" />
<attribute name="telephone1" />
<attribute name="leadid" />
<order attribute="fullname" descending="false" />
<filter type="and">
<condition attribute="new_campaignid" operator="eq" uiname="Test Campaign" uitype="campaign" value="{F7038DE4-B5A5-E211-8417-000C29E20CBC}" />
</filter>
</entity>
</fetch>
after create the dataset, I attached a screenshot of BIDS
Since the requirements are changing,I need to update the DAO's mapper files from time to time using Mybatis. And here is the problem that I got: we've already generated a set of mapper xml files and additionally,we defined our own specific elements in the mapper file.
<select id="selectTop20PlayerOfClassicCard" resultMap="CollectionRankingMap">
SELECT playerid,count(cardid) from re_player_card where type in (0,1) GROUP
BY playerid ORDER BY COUNT(*) DESC limit 20;
</select>
<select id="selectTop20PlayerOfSpecialCard" resultMap="CollectionRankingMap">
SELECT playerid,count(cardid) from re_player_card where type=2 GROUP BY
playerid ORDER BY COUNT(*) DESC limit 20;
</select>
<resultMap id="CollectionRankingMap" type="dymaic.RePlayerCardRanking">
<!-- manually generated -->
<result column="playerid" jdbcType="INTEGER" property="playerid" />
<result column="count(cardid)" jdbcType="INTEGER" property="count" />
</resultMap>
<select id="getCardIdByPlayerId" parameterType="int" resultType="int">
select
re_player_card.cardid
from re_player_card
where playerid = #{playerid}
</select>
The 3 select statements and the resultmap are defined by ourselves. Now I'm going to regenerate the mapper files again.But I don't want these statement blocks to be overwriten. I need to keep them in the mapper file while changing the rest with Mybatis.
I'm using the Mybatis generator eclipse plugin. So is there any way to make it? I'd appreciate for you guys' help.
I don't think MyBatis Generator Tool is capable of updating the partial content of the generated files.
If you are using MyBatis Generator Maven plugin you can add the following property in pom.xml to tell not to override the existing files and generate the files with another name.
<mybatis.generator.overwrite>false</mybatis.generator.overwrite>
PS: My above assumption is wrong.
From MyBatis documentation:
MyBatis generator is designed to run well in an iterative development environment, and can be included as an Ant task, or a Maven plugin, in a continuous build environment. Important things to note when running MBG iteratively include:
MBG will automatically merge XML files if there is an existing file
with the same name as the newly generated XML file. MBG will not
overwrite any custom changes you make to the XML files it generates.
You can run it over and over again without fear of losing custom
changes to you XML.
MBG will replace any XML elements that were
generated in a previous run. MBG will not merge Java files, it can
either overwrite existing files or save newly generated files with a
different unique name. If you make changes to the generated Java files
and run MBG iteratively you will have to merge the changes by hand.
When run as an Eclipse plugin, then MBG can automatically merge Java
files.
Put your changes into another mapper XML file
Take the existing file copy, e.g. MyMapperBits.xml and I choose to change the namespace ..."http://mybatis.org/dtd/mybatis-3-mapper.dtd"> <mapper namespace="MyMapperBitsMapperNameSpace">
Add the MyMapperBits.xml to the references in your main mybatis configuration file. In this example the package name items also contain various XML mappers, that I have configured mybatis generator to create.
<mappers>
<mapper resource="com/example/extended/xmlmapper/MyMapperBits.xml" />
<package name="com.exmaple.generated.xmlmapper.input_output"/>
<package name="com.exmaple.generated.xmlmapper.pg_schema_1"/>
<package name="com.exmaple.generated.xmlmapper.pg_schema_2"/>
</mappers>
You then can access the "SQL" with MyMapperBits.xml by accessing it via the namespace defined in the file
sqlSession . selectList(
"MyMapperBitsMapperNameSpace.selectTop20PlayerOfClassicCard" ) ;
In reports I need to show localised field labels and entity names. For this I need to retrieve some pieces of metadata, namely -- localised entity names.
Can you access metadata in Fetch-based reports? If you can, how severely will performance suffer?
There is a hack to sort of get pick list values and default labels. Quoting from reference:
You must use the distinct clause, and create a seperate dataset to
fetch only the option set field like this.
<fetch version="1.0" output-format="xml-platform" mapping="logical"
distinct="true"> <entity name="account">
<attribute name="accountratingcode" /> </entity> </fetch>
This actually returns 2 columns to BIDS -
accountratingcode
accountratingcodevalue
Create a report parameter for the main report and in the Available
Values tab specify:
Get Values from a query
Dataset: Seperate Dataset with the Fetch XML
above
Value Field: accountratingcodeValue
Label Field:
accountratingcode
Sorry to say that entity metadata is not exposed via the FetchXml interface as neither "attributes" nor their corresponding "labels" are entities, so cannot be queried.
You can prove this by using the FetchXml builder ([details of getting it and making it work with CRM 2011 are here][1]). You will note that none of the available entities contain the information you seek.
Edit: looking at the core requirement (rather than answering the question in isolation as I originally did), it is possible to include "label" values in FetchXml statements, as #skfd notes in his own answer.