I am trying to set a lookup field in the copy data action in Azure Data Factory.
The error I'm getting is
Failure happened on 'Sink' side. ErrorCode=UserErrorTypeConversionFail,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,
Message=,Source=,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The
attribute 'ks_supervisor' with value 'E3BEFA28-C7A6-EA11-A812-000D3A1BB8EF' cannot be converted from
original type 'System.String' to target type 'Microsoft.Xrm.Sdk.EntityReference'.,Source=Microsoft.DataTransfer.ClientLibrary.DynamicsPlugin,''Type=Sy
stem.InvalidCastException,Message=Specified cast is not
valid.,Source=Microsoft.DataTransfer.ClientLibrary.DynamicsPlugin,''Type=Microsoft.DataTransfer.Common.Sh
ared.HybridDeliveryException,Message=The attribute 'ks_supervisor' with value 'E3BEFA28-C7A6-EA11-A812-
000D3A1BB8EF' cannot be converted from original type 'System.String' to target type
'Microsoft.Xrm.Sdk.EntityReference'.,Source=Microsoft.DataTransfer.ClientLibrary.DynamicsPlugin,''Type=Sy
stem.InvalidCastException,Message=Specified cast is not
valid.,Source=Microsoft.DataTransfer.ClientLibrary.DynamicsPlugin,
The key piece is value 'E3BEFA28-C7A6-EA11-A812-000D3A1BB8EF' cannot be converted from
original type 'System.String' to target type 'Microsoft.Xrm.Sdk.EntityReference'.
My question is how do I set a lookup field value in D365/CDS/CRM if it is expecting a value of type EntityReference?
I have tried the CDS connector, Dynamics 365, and the CRM connector, but all result in an identical error. I have also looked for ways to try converting the source value from a string to a GUID, but I can't find a way and I don't know if that will help since it is a GUID not EntityReference. The source column is from a SQL table of type "uniqueidentifier", but it is projecting automatically to a string.
I had to add "#EntityReference" to the destination column name in the column mappings. See the answer provided by MSFT here: https://learn.microsoft.com/en-us/answers/questions/34069/azure-data-factory-copy-data-to-cds-cannot-convert.html?childToView=34732#comment-34732
See the documentation for setting lookup column values here. It also shows how to set a customer column value: https://learn.microsoft.com/en-us/azure/data-factory/connector-dynamics-crm-office-365#writing-data-to-lookup-field
Related
I am currently using Azure Datafactory in that I am creating a Derived column and since the field will always will be blank, so I want the value to be NULL
currently Derived Column I am doing this for adding the expression e.g. toString("null") and toString(null()) but this is appearing as string. I only want null to appear without quotes in Json document
I have reproduced the above and got below results.
I tried to give null() to a column and it gave the error like below.
So, in ADF Dataflow, there should be any wrap over null() with the functions like toInteger() or toString()
When I give toString(null()) when id is 4 in derived column of dataflow and the sink is a JSON, it gave me the below output.
You can see the row with id==4 skipped the null valued key in JSON. If you give toString(null()) same key in every row will be skipped.
You can go through this link by #ShaikMaheer-MSFT to understand more about this.
AFAIK, The workaround for this can be to store the null as 'null' string to get that key in JSON like this and later use this as per your requirement.
I've created a dataflow task in azure data factory and used derived column transformation. One of the source derived column value is '678396' which is extracted through Substring function and datatype "String" by default. I want to convert it into "Integer" because my target column datatype is "Integer".
I've to converted the column in this expression:
ToInteger(Substring(Column_1,1,8))
Please help me with correct expression.
Kind regards,
Rakesh
You don't need to build the expression. If you column data are all like int string "678396", or the output of Substring(Column_1,1,8) are int String
Data Factory can convert the int string to integer data type directly from source to sink. We don't need convert again.
Make sure you set column mapping correctly in sink settings. All things would works well.
Update:
This my csv dataset:
You can choose the Quote character to singe quote, then could solve the problem. See the source data preview in Copy active and Data Flow:
Copy active source:
Data Flow overview:
In data flow, we will get the alert like you said comment, we could ignore it and debug the data flow directly:
HTH.
you don't even need to substruct quotes '', as ToInteger function can convert numbers as string type
I want to use Data Factory to migrate data in the storage account, but data in the original table is a many type ex: some data in column int, String, DateTime.
When I use Data Factory I need to specify the data type, so how I can definite dynamic type and copy column. Because all data migrate parsed to String type, so how can I keep value type of column?
This my data in the original table
enter image description here
Thanks for your help
According my experience in Data factory, Data Factory can not help you keep value type of column in source table. You must specify the data type in sink dataset.
Copy Data:
As you have tried, if you didn't set the sink data type, the column type will passed String in default.
I have an idea is that cope the data twice, each time copy the different entity column. The sink dataset support 'Merge' and 'Replace'.
Hope this helps.
Not sure if I am understanding the question , but let me first put forward my understanding , you want to copy a table lets say sourceT1 to SinkT1 , if that's the case you can always use the copy activity and then map the columns . When you map the columns it does set the data type also .
I'm using OrientDB and trying to create new property after I inserted my data (millions of rows).
I'm trying to create property on V in order to create an index and I'm getting the following error:
The database contains some schema-less data in the property
'V.ACCOUNT_NO' that is not compatible with the type STRING. Fix those
records and change the schema again [ONetworkProtocolHttpDb]
Now part of the fields type is INTEGER but it seems to me that it's very easy to convert the type to STRING.
how can I do it to the entire data?
I tried your case by creating this simple structure in schema-less mode:
These records are a mix of INTEGER and STRING types:
Now you can convert the not string records type by using this query:
UPDATE V SET ACCOUNT_NO = ACCOUNT_NO.asString() WHERE ACCOUNT_NO.type() <> 'STRING'
Output:
About the exception, I got correctly the same error when I try to create a new property V.ACCOUNT_NO of type STRING in schema-full mode and this is correct because the property already exists in the database and contains mixed types of records, although in schema-less mode.
Once all the records were converted, you'll able to create the new property.
Hope it helps
I have created a data type called id which consists of two text values:
id(text, text)
I now need to cast values to this data type before they are inserted into my table. How would I go about doing this?
I created the type as follows:
CREATE TYPE ID AS(id text, source text);
Well, to create a cast you need a function that takes a value of one type as your input and outputs the type you wish to cast to (in this case "ID" - which I would name a little more verbose if I were you). What type do you want to cast from?
Realize without messing with all that, you should be able to use your type according to this page.
Just..
SELECT ROW('foo','bar')::ID ;
You have to tell PostgreSQL how to cast, CREATE CAST
if we are talking about "user defined types" which are realy compatible with each other, so you can cast the type to text first and then to your custom type admin_action::text::"UserAction"
in my case "admin_action" was of type "AdminAction" and couldn't be converted to "UserAction" directly, but I've done it through "text" step.