when we create AzureBatchLinkedService in azure data factory it's give error for azure batch pool i am using cloud service
Error
Entity provisioning failed: Azure Batch operation failed. Code: 'BadRequest' Message: 'The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters. '
Regards,
Manish
The linked service JSON template stats that it wants the Batch Pool ID. This is incorrect. It actually wants the Batch Pool name!... Its one of those great features! Put the name in the ID value.
Also the URI value needs to exclude the account name for it to work!
Hope this helps.
After modifying the json properties. You have put back the accessKey is ASCII format. By default it's ***** . it shall be g62aUSOG6UdHmWQmsATKPOCdtM1RJra..
"accessKey": "**********",
Related
I am trying to connect to snowflake using linked service and copy data from adls to SF using adf pipeline. I created the linked service and tested the connection. It works fine. Even the debug over the pipeline works fine. althogh when I manually try to trigger the pipeline I get "Required property 'connectionString' is not provided in connection properties"
Thanks in advance.
This looks like a configuration issue and this behavior is noticed when the linked service is configured to use parameters (linked service parameterization), or a key vault is used, and if the connection string value isn't passed to the linked serviced during runtime.
Since it is working in debug mode, I would recommend publishing your linked service and in case if it is parameterized or using AKV, then please make sure that the connection string value is being passed and evaluated at runtime.
All, when running a build pipeline using Azure Devops with ARM template, the process is consistently failing when trying to deploy a dataset or a reference to a dataset with this error:
ARM Template deployment: Resource Group scope (AzureResourceManagerTemplateDeployment)
BadRequest: The document creation or update failed because of invalid reference 'dataset_1'.
I've tried renaming the dataset and also recreating it to see if that would help.
I then deleted the dataset_1.json file from the repo and still get the same message so it's some reference to this dataset and not the dataset itself I think. I've looked through all the other files for references to this but they all look fine.
Any ideas on how to troubleshoot this?
thanks
try this
Looks like you have created 'myTestLinkedService' linked service, tested connection but haven't published it yet and trying to reference that linked service in the new dataset that you are trying to create using Powershell.
In order to reference any data factory entity from Powershell, please make sure those entities are published first. Please try publishing the linked service first from the portal and then try to run your Powershell script to create the new dataset/actvitiy.
I think I found the issue. When I went into the detailed logs I found that in addition to this error there was an error message about an invalid SQL connection string, so I though it may be related since the dataset in question uses Azure SQL database linked service.
I adjusted the connection string and this seems to have solved the issue.
I'm new to using Azure release pipelines and have been fighting issues trying to deploy a database project to a new Azure SQL database. Currently the pipeline is giving me the following error...
TargetConnectionString argument cannot be used in conjunction with any other Target database arguments
I've tried deploying with and without the TargetConnectionString included in my publish profile. Any suggestions or something else to try? I'm out of ideas.
TargetConnectionString
Specifies a valid SQL Server/Azure connection string to the target database. If this parameter is specified it shall be used exclusively of all other target parameters. (short form /tcs)
So please remove all other TargetXXX arguments.
(if you don't have them can you show what arguments you have inline and in publish profile - of course without data)
I am using PowerShell to read the Azure Service Bus Topics. The topic name contains some special characters in it. Eg. Topic name is xxx/t000.
When I use:
Get-AzureRmServiceBusSubscription -ResourceGroup 'rg_name' -NamespaceName 'sbname' -TopicName "xxx/t000"
I am getting error:
Operation returned an invalid Status code "Not Found"
So I am predicting the issue with the / in the topic name.
Can someone help me on how to read the topic name which contains "/" in it?
Try putting the topic name in single quotes.
-TopicName 'xxx/t000'
This problem is related to Azure Resource Manager, which doesn't support resource names that have "/". You should be able to reference the topic by replacing '/' with '~'.
Here's a good behind the scenes description about this subject on MSDN: https://blogs.msdn.microsoft.com/servicebus/2017/06/21/azure-service-bus-azure-resource-manager-and-this-character/
On the Service Bus side we will convert "/" to "~" when interfacing
with ARM, but retain the slashes in our back-end so things continue to
work the way they are supposed to.
When I try to upload the zip file to an azure function app using kudu REST API, it throws an error while I try to view the c# code in Function App editor in the browser. The error is:
"Error:
Function ($Source) Error: Microsoft.Azure.WebJobs.Host: Error indexing method 'Functions.Source'. Microsoft.Azure.WebJobs.Host: Value cannot be null.
Parameter name: hostAccount.
Session Id: xxxxxxxxxxx
Timestamp: 2016-12-02T18:35:00.083Z"
Please note that I have automated end to end of Application Insights starting from creation of a resource group till exporting the multi-setep web test results to our Splunk - All using Powershell.
In this process of automation, I am forming a storage connection string and setting it to the app settings of the function app and then providing that key in my function.json binding.
But still I get this error.
Here is the issue I created in the Azure Function App - Git: https://github.com/Azure/azure-webjobs-sdk-script/issues/1020
The error points to missing host configuration (e.g. the host storage account).
If you're automating the creation process, you need to make sure the required environment variables are properly set. The recommended way would be to use an ARM template manage that for you.
I have some details on how you can get the ARM template for a resource (which you could use to look at the required settings for your Function App) here.
We also have some sample templates you can use linked here
I hope this helps!