Azure Data Factory Getting error AccessToOnPremFileSystemDenied when trying to reach the self hosted IR disk - azure-data-factory

I had been using the F: of my self hosted IR in my ADF pipelines when all of a sudden I started getting an error:
Error code
AccessToOnPremFileSystemDenied
Failure type
User configuration issue
Details
ErrorCode=AccessToOnPremFileSystemDenied,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Access to 'F:' is not allowed.,Source=Microsoft.DataTransfer.Common
This is pretty bad of Microsoft to change stuff on the fly.
Found this link here but nothing on stack overflow so I thought I would put something here to help others.
https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-file-system

So in the end I found out that MS introduced this as a safety feature and these were the steps to disable it:
Log on to the Machine where the IR is hosted as an admin.
Run CMD as an admin.
Run cd C:\Program Files\Microsoft Integration Runtime\5.0\Shared
Run dmgcmd -DisableLocalFolderPathValidation

Related

Internal Server Error in PowerShell cmdlets for Microsoft Dynamics CRM

On Windows Server 2012 Datacenter with Microsoft Dynamics CRM 2016 installed, I want to run a deployment command but for every commands I get this error: "(500) Internal Server Error".
I first run this:
Add-PSSnapin Microsoft.Crm.PowerShell
and it will work fine and when I check it with get-pssnapin and Get-Help *Crm*, every thing is fine and every thing that I need is registered. but when I want to run a cmdlets command like these, I face the error: Get-CrmSetting or Get-CrmCertificate or ...
For example for Get-CrmSetting TraceSettings it give me this error:
How can I solve this problem and error?
Thanks
According to this article, you might want to try:
Get-CrmSetting –SettingType TraceSettings
Here are a couple more items to investigate, from this article:
To use the XRM tooling cmdlets, you need PowerShell version 3.0 or
later. To check the version, open a PowerShell window and run the
following command: $Host
Set the execution policy to run the signed PowerShell scripts. To do
so, open a PowerShell window as an administrator and run the
following command: Set-ExecutionPolicy -ExecutionPolicy AllSigned
Verify the (CRMDeploymentServiceAppPool Application Pool identity) has SQL SEVER SysAdmin permission. This is needed to perform any CRM configuration changes and organizational operations.
Note: it does not matter if the account executing the PowerShell is a system admin or SQL server sysadmin because these operations are executed via the deployment web service.
Deployment Web Service (CRMDeploymentServiceAppPool Application Pool identity)
....Sysadmin permission on the instance of SQL Server to be used for the configuration and organization databases.
....
(500) Internal Server Error, refers to a HTTP response status code. This means that the Powershell command is calling a URL and the URL is reporting a error.
You need to know the URL to really find out what the problem is. One way you can get the URL, is downloading Fiddler Classic. Once installed, you have to enable HTTPS decryption.
In my case the URL was...
https://<my-crm-domain>/XrmDeployment/2011/deployment.svc?wsdl
When I ran this URL on the server where CRM is installed, I got an exception stating...
Could not load file or assembly 'Microsoft.Crm.Application.Components.Application'
All this meant, I needed to copy a file, Microsoft.Crm.Application.Components.Application.dll, from C:\Program Files\Dynamics 365\CRMWeb\bin into folder C:\Program Files\Dynamics 365\CRMWeb\XRMDeployment\bin.
Once this was done, the URL worked and therefor my PowerShell command as well.

Access TFS TeamProjectCollection from build step with PowerShell

I'm trying to create a pre-build step with powershell in one of our team projects to increment the build number for all assemblies/applications within that VS solution. This is for TFS2015-update 2.
I found some example code here and there online and have pieced together what I imagine would work. I can run the powershell script manually on the TFS server (authenticated to the server as my windows account) and it works fine, but yet it won't function correctly within the build process.
Here is the sample code in question where I keep getting my error. It has to do with authentication/connection to the TFS server:
$CollectionUrl=New-Object System.Uri("$env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI")
$BuildUrl="$env:BUILD_BUILDURI"
# get the team project collection
$TeamProjectCollection=[Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($CollectionUrl)
# get the build server
$BuildServer=$TeamProjectCollection.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
# get the build detail
$BuildDetail=$BuildServer.GetBuild($BuildUrl)
# get the build definition
$BuildDefinition=$BuildDetail.BuildDefinition
There error I am getting is:
Exception calling "GetService" with "1" argument(s): "TF30063: You are not authorized to access http://servername/TPCName
I have tried using both the call above to get the TPC, and the constructer for it and neither works.
$TeamProjectCollection=New-Object Microsoft.TeamFoundation.Client.TfsTeamProjectCollection($CollectionUrl)
So I started trying to build a credential object based on my login credentials to the server and see if I could connect to the TPC that way. Still doesn't work (on server or in build process), but different error:
$netCred = New-Object System.Net.NetworkCredential($username,$password)
$basicCred = New-Object Microsoft.TeamFoundation.Client.BasicAuthCredential($netCred)
$tfsCred = New-Object Microsoft.TeamFoundation.Client.TfsClientCredentials($basicCred)
# get the team project collection
$TeamProjectCollection=New-Object Microsoft.TeamFoundation.Client.TfsTeamProjectCollection($CollectionUrl, $tfsCred)
Throws the exception:
Exception calling "GetService" with "1" argument(s): "TF400324: Team Foundation services are not available from the server.
I have also tried my best at modifying TFS permissions for the windows account that TFS service runs as to make sure there were no issues there. I'm fairly certain I added it to all roles/groups that my identity was in, but still the build wouldn't authenticate.
I'm at a loss at this point. Can someone help me figure out how to connect to the TFS server correctly using defined credentials, or help me adjust what permissions the service account, identity, configuration choices, etc; needed to access the teamProjectCollection.
You need to specifically instruct TFS/VSTS to allow the executing build to call back to TFS/VSTS. This protects your server for malicious developers exiting build scripts.
You should see an "enable Auth" option in the Options Tab of your Build.
**note: You should use the Task SDK provided: https://github.com/Microsoft/vsts-task-lib

TFS Build "PowerShell on Target Machines" Step Fails: How to debug?

I'm trying to automate the deployment of the solution my team is working on through TFS Build server. One of the steps which executes a PowerShell script on the target machine fails with the following error:
Microsoft ODBC Driver 11 for SQL Server : Login failed for user 'sa'..
The PowerShell script I'm trying to execute does in fact connect to multiple databases using the sa credentials. When I try to execute the same script passing it the exact same arguments by hand (i.e: executing the script from the target machine VM itself) it works like a charm. But when it is being executed as part of the build steps it fails with the aforementioned error.
Is there a way to further debug the issue? It would be great if there is a way to output trace statements from the script so I could have some insight on what is actually going on.
Usually all the related error should reflect in TFS build log. To narrow your issue you can try to connect to the TFS build agent with the credentials used for the build service and manually run the ps script.
If you execute the ps script with your own account, which will not help to the issue. Usually this kind of problems is related to permissions. Your build service account are lack of related permission. Try to add it to Administrator or SQL Administrator group and execute the build again.

OpsHub Visual Studio Online Migration Utility - Service Error on Migration Summary

I have downloaded opshub tool for the migration, trying to migrate from on premises to visualstydio team service. Getting error On the migration summary page with error message:
"Unable to communicate with the required process opshubtfsservice.Because probably it is not running. Restart application & try again"
I verified the opshub service is running as local service. I tried to restart & tried with Network service/local account etc. But no lock.
Can you please help me on this here.
Thank you.
It seems that your machine is behind a proxy and all traffic in-out is being routed through it. (Including the communication of local traffic)
You will have to bypass local addresses from the proxy as well as enable OVSMU to communicate through the proxy.
Please refer the C:\Program Files\OpsHub Visual Studio Migration Utility\Other_Resources\Resources\ProxyUtility.zip to configure OVSMU to utilize your proxy. There should be a user guide document which lists the steps.Keep the bypass local address value as default.

Visual Studio Online - operations failed getUser List

We are trialing a test migration of a project on TFS 2010 (on prem) to #visual-studio-online using #OpsHub . During the migration phase at the user mapping window we are getting the below error message.
2015-07-08 15:10:49,916 [4] ERROR Error : The operation has timed out
at System.Net.HttpWebRequest.GetResponse() at
Microsoft.TeamFoundation.Client.Channels.TfsHttpWebRequest.SendRequestAndGetResponse(HttpWebRequest
webRequest, WebException& webException)
I am not able to find much information on-line about this error. I tried doing the migration as a TFS administrator and as a domain user who has project admin access to the project but still get these time out errors.
I saw online a user upgraded to #OpsHub 1.3 from 1.2 to overcome this issue which I tried but still doesn't help. Do we need to open any other ports for this to work apart from 80 and 443? Any suggestion as to how to over come this issue?
From the logs we find out that error is about time out from your local tfs server.
Server Error : TF400324: Team Foundation services are not available from server tfs.nextdigital.com\Next Digital.
Technical information (for administrator):
The operation has timed out
Here you need to work with your tfs admin to increase timeout parameter of your tfs.
Following thread might help you in this.
http://blogs.msdn.com/b/ablock/archive/2008/09/16/increasing-the-time-out-time.aspx
Thanks & Regards,OpsHub Support