VSTS Build Fails with SVN Forbidden Exception - azure-devops

I have setup a build process in VSTS that is pulling source code from a public facing SVN url. It works great when I manually queue the build from the dashboard or if I schedule a build to run at a certain time. However, when a build gets queued by continuous integration it throws the following exception:
An exception occurred while polling the repository. Error: Microsoft.TeamFoundation.Build2.Server.Extensions.BuildPollingException: Unable to connect to a repository at URL '<Path to the URL I am referencing>' --->
SharpSvn.SvnRepositoryIOForbiddenException: Unable to connect to a repository at URL '<Path to the URL I am referencing>' --->
SharpSvn.SvnRepositoryIOForbiddenException: Access to '<Path to the trunk of the repository>' forbidden
When I look at the event log on the server that Subversion lives on, it shows that the build agent is trying to access the URL with an anonymous login, hence the exception I am seeing.
Has anyone ran into this before? I know I have configured the service endpoint correctly with user credentials since it works when I manually fire off or schedule queue the build. I've done pretty extensive searching on this topic but have not found much info that involves VSTS and SVN issues. Any guidance or questions will be greatly appreciated.

The CI's Polling Job is executed by the Job Agent on the TFS server. Apparently, your Job Agent is configured to run as NT AUTHORITY\NETWORK SERVICE (i.e., as DOMAIN\TFSSERVER$). On the other hand, it looks like you VisualSVN Enterprise instance is configured to use NTLM. In this case, basic credentials are not used by the SharpSvn library the Polling Job uses to access the repository. Unfortunately, that's SharpSvn restriction. You either should provide the computer account (DOMAIN\TFSSERVER$) with the READ access to the SVN repository in the VisualSVN server configuration (to continue using NTLM), or disable NTLM on the VisualSVN server to switch to Basic authentication.
[Few technical details]The anonymous requests you see in SVN logs are part of the normal HTTP request flow implemented in the SharpSvn library:
The library first submits an anonymous request to the server.
If the server rejects the request, the library inspects the WWW-Authenticate response headers to detect the authentication scheme
requested by the server.
The WWW-Authenticate: NTLM header has a higher priority comparing to WWW-Authenticate: Basic. That's why username and password
specified in the connection endpoint are not used at all.
If only the WWW-Authenticate: Basic header is present in the response, the library uses a callback function provided by the client
To request credential information. At this point username and
the password specified in the connection endpoint is provided to the
library by the Polling Job.
[Note 1] The Polling Job does not download any sources from the SVN repository, it only requests the list of files changed since its last successful execution and decided if it's time to schedule a build.
[Note 2] If the Polling job continuously fails for the same reason, it reports each 100th failure to TFS as a failed build providing a reason of the failure.
Alex

I look to be having a similar issue. Using TFS 2015 Update 3 and VisualSVN Server 3.4.6. I get the same error with CI polling. On the SVN Server I get the following errors every time TFS polls: Access denied: 'TFSSERVER$' OPTIONS : [client ]
Here TFSSERVER$ is the computer name of our TFS Server.
So I gave DOMAIN\TFSSERVER$ Read Only permissions at the root of the repository and then things worked OK. Not happy that I had to do that of course.
Filed this bug about it: SVN CI Polling Uses TFS Machine Account Not SVN Account Set in the Service EndPoint

There isn't any big difference between CI and manual/scheduled build for SVN repository. When you configure it to continuous integration, it just keeps checking the SVN repository by the Polling interval you set and then run the build if there is any new change. The only difference between them is that you need to set "Filters" for CI and base on the access forbidden error message you get, you don't have the permission to read the path. So you need to check if your account has the read permission to the path you set in "Filters".

Related

Authentication of REST services not working with jBPM (KIE Server and Business Central) and Keycloak

I am studying jBPM (KIE Server and Business Central) and Keycloak. In isolation, I managed to run all this software with success. However, when I try to integrate between them, the authentication of users on the website works (Keycloak login page to access the Business Central site), however it seems that the authentication of the REST services call, both from the KIE Server and from the Business Central, stops working (unauthorized) and the KIE Server is unable to synchronize with its Controller (Business Central).
Below are more details on the subject.
I have been searching for a few days for a solution to the problem but without success.
Although I have already consulted this excellent site, this is my first question and I do not have much reference on which file is recommended to attach to the question. If you see a specific file, please sign that I will attach it.
I appreciate any help.
Notes (useful information) about the environment and problems encountered:
Only 1 Wildfly 20.0.1 environment (server) hosting KIE Server 7.43.1.Final, Business Central 7.43.1.Final and Keycloak 11.0.2, with the Keycloak adapter installed, on Windows 10.
In Keycloak, Realm was created with the name VHLLEmpresaRealm and 2 clients (applications):
VHLLEmpresaKIEServer: with public access.
VHLLEmpresaBusinessCentral: with confidential access.
In the Keycloak, the user kieserver with password kieserver1! was created and assigned to various roles (both Realm and Clients), among them: admin, rest-all, kie-server, kiemgmt and user.
Authentication with kieserver User and your Password on the Business Central website working properly integrated with Keycloak (Keycloak login page). Authentication with Google (social login) working properly.
Business Central configured to show and manage the users registered in the Keycloak.
Problems observed:
When starting the Wildfly server, the KIE Server is unable to synchronize with the Controller Business Central (see the Startup Log - Doc 06). The attempt below is repeated at startup several times without success, followed by NullPointerException with lines copied below:
17:13:30,057 INFO [org.kie.server.controller.websocket.client.WebSocketKieServerControllerImpl] (KieServer-ControllerConnect) Kie Server points to non Web Socket controller 'http://localhost:8080/business-central/rest/controller', using default REST mechanism
17:13:30,214 ERROR [org.keycloak.services.error.KeycloakErrorHandler] (default task-6) Uncaught server error: java.lang.NullPointerException
at org.keycloak.storage.StorageId.isLocalStorage(StorageId.java:77)
at org.keycloak.credential.UserCredentialStoreManager.getStoreForUser(UserCredentialStoreManager.java:54)
...
17:13:30,214 WARN [org.kie.server.services.impl.controller.DefaultRestControllerImpl] (KieServer-ControllerConnect) Exception encountered while syncing with controller at http://localhost:8080/business-central/rest/controller/server/wildfly-kieserver error Error while sending PUT request to http://localhost:8080/business-central/rest/controller/server/wildfly-kieserver response code 401
Link to complete file:
https://drive.google.com/file/d/1kr47ap9frsCQB8oyQE2dEm2gfl7__s0G/view?usp=sharing
On the Business Central website, on the Deployments page (which shows the Servers too), the KIE Server server does not appear as available (see Doc 07).
Link to complete file:
https://drive.google.com/file/d/1lKiCZdOp7yRvHibfgJ8ZOE_dZIAnkE6U/view?usp=sharing
In the CURL command to access the Business Central REST services, even if the kieserver user and password are passed, the answer is NOT authorized (see Doc 08).
COMMAND: curl -X GET http://kieserver:kieserver1!#localhost:8080/business-central/rest/repositories -v
RESPONSE: ErrorUnauthorized
Link to complete file:
https://drive.google.com/file/d/1V_CH_hiTkhLh-AtgRvP0YkZ3afwBc4tZ/view?usp=sharing
Other Files:
Wildfly Settings - STANDALONE-FULL.XML
https://drive.google.com/file/d/1f0xvRDPiId1RymZkMGIUaIMB_EuGpbud/view?usp=sharing
I'm not sure if this is the correct way to solve the problem, but after continuing my studies and research for a few more days, using the CURL command to call a Business Central service (below), I obtained the Access Token and through the website jwt.io, I noticed in the content of this Token that the audience (aud) field was not filled in with the Keycloak Client ID that requested the Token.
curl -d "grant_type=password" -d "client_id=VHLLEmpresaBusinessCentral" -d "client_secret=020c8841-50ee-437e-af16-XXXXXXXX" -d "username=kieserver" -d "password=kieserverXXXXX" -d "scope=VHLLEmpresaEscopo" http://localhost:8180/auth/realms/VHLLEmpresaRealm/protocol/openid-connect/token
So, I created a Client Scope, I created a Mapper for the Audience to add the Client's own ID and associated this Client Scope to the corresponding Client.
The second change that was necessary was to override the Direct Grant Flow with the "direct grant" value for each Client involved.
With the two changes above, the synchronization between KIE Server and Business Central has returned to work.

Tyring to run VSTS agent thru a proxy which limits sites

Have installed VSTS agent in a very locked down environment. It makes a connection to VSTS, gets job but fails when downloading artefact. Gives error
Error: in getBuild, so retrying => retries pending : 4.
It retries 4 times and fails.
The agent is going thru a proxy. Have setup the proxy using ./config --proxyurl and also set HTTP_PROXY AND HTTPS_PROXY system environment vars.
The proxy is very limiting in that URLS are locked down, there is no authentication required. Does anybody know what URLs the agent accesses? Am hoping if can get a definitive list this will solve the issue. If anybody knows how can get a list would be great. Or maybe I have misconfigured?
Any ideas?
Tyring to run VSTS agent thru a proxy which limits sites
According to the document
I'm running a firewall and my code is in Azure Repos. What URLs does the agent need to communicate with?:
To ensure your organization works with any existing firewall or IP
restrictions, ensure that dev.azure.com and dev.azure.com are open
and update your allow-listed IPs to include the following IP
addresses, based on your IP version. If you're currently allow-listing
the 13.107.6.183 and 13.107.9.183 IP addresses, leave them in place,
as you don't need to remove them.
And With just the organization's name or ID, you can get its base URL using the global Resource Areas REST API (https://dev.azure.com/_apis/resourceAreas). This API doesn't require authentication and provides information about the location (URL) of the organization as well as the base URL for REST APIs, which can live on different domains.
Please check this document Best practices for working with URLs in Azure DevOps extensions and integrations for some more details.
Hope this helps.

Error when publishing Workflow from SharePoint Designer 2013 to SharePoint Online

I am trying to publish a very simple workflow from SharePoint Designer 2013 to SharePoint Online.
The following error appears:
Microsoft.SharePoint.SPPrincipalManagementException: An error occurred
while attempting to execute a principal management operation. Please
contact your administrator. --->
System.ServiceModel.FaultException`1[Microsoft.Online.Administration.WebService.PropertyValidationException]:
Invalid property specified
Server stack trace: at
System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime
operation, ProxyRpc& rpc) at
System.ServiceModel.Channels.ServiceChannel.Cal
How can I handle this?
You can try create a new sub-site from your site collect then upload your workflow to see if its viable.
If not, you can check whether the Central Admin > Manage Service Application-> “App Management Service” is started.
Also heck whether the “Configure service application associations”, ”App Management Service” is already associated.
Then check whether the “Manage Services on server” and the “App Management Service” is started.
In addition, try to re-register Workflow Service.
If the issues still exists, please follow the steps in the Steps to Verify that Server Is Correctly Set Up. After you verified that the server is correctly set up, follow the steps in the Steps to Troubleshoot Workflow Management Service and Troubleshooting the Service Bus for Windows Server then retry your action.
If all above doesn't solve problem, then you should create a service request to Microsoft in SharePoint Online Admin Portal directly. Since issue is more likely related to SharePoint Online Server Back end.

using pact-broker webhooks locally?

I have setup the pact broker locally and able to publish the pacts which are also verified by the provider successfully. I am at the point to use webhook which kicks off a build of the provider project if the pact content has changed since the previous version. Can I use the webhook concept in my local because my consumer and provider are not configured in CI?
You'll need to create a local "CI server" on your machine. It doesn't really have to be a proper CI server, but it does have to be able to accept an HTTP request that will kick off a build somehow.
You should be able to create a very simple ruby/javascript/python HTTP server that will run the provider build in a backgrounded process when it receives a request. Or, you could install a copy of something like Jenkins locally.

Jenkins not responding to github webhooks

Nothing i try seems to trigger my Jenkins webhooks from github
Things to know
My Jenkins instance sits behind a Nginx proxy
I am using the Organization type Jenkins project with a Jenkinsfile
The git user used is using a personal access token and has organization rights and all needed permissions
I have security setup on the Jenkins instance so my Jenkins webhook url is https://foo:bar#jenkins.doctify.co.uk/github-webhook/
The webhook has the accurate set of permissions
This gets the green tick and a 200 response from the Jenkins instance
When using polling to trigger the branch indexing this works correctly
In the logs i only see this sometimes and never a pull request event
INFO: Received POST for https://github.com/doctify/platform
Jul 14, 2016 4:13:40 PM org.jenkinsci.plugins.github.webhook.subscriber.DefaultPushGHEventSubscriber onEvent
My Organization configuration
Jenkins Version 2.13, Git Version 2.7.5
Questions
How do I get this to work?
How do I make it better and have the organization plugin create the webhooks automatically the way I'm sure its meant to do?