How do I select parameters in a KRL webhook? - krl

I want to run some KRL rules when a website is updated. The deploy script will get the following URL after pushing the updates:
http://webhooks.kynetxapps.net/t/_appid_/updated?site=production&version=123456abcdef
The ruleset to handle this webhook starts out like this:
rule site_updated {
select when webhook updated
pre {
site = event:param("site");
version = event:param("version");
}
// do something with site and version
}
From http://docs.kynetx.com/docs/Event_API I could make more specific rules with:
select when webhook updated site "test"
or webhook updated site "production"
Is there a way to get both parameters without a PRE block? What is the best way to use SELECT with a webhook?

Rule filters (like site "test") are regular expressions, and you can set variables using the setting () clause.
http://webhooks.kynetxapps.net/t/_appid_/update?site=production&version=123456abcdef
select when webhook update site "(.*)" setting(site)
causes site to be set to production without using a pre block. As this is a regular expression, you can match anything, like either of two options:
select when webhook update site "(test|production)" setting(site)
will only match with site == test or site == production, and no other times, AND store the value in the site variable in the context of the rule.

Related

TYPO3 v10: How to access TSFE in Backend/Scheduler Task?

The current situation:
I am trying to access the TypoScript configuration of the frontend from within the backend (or rather a scheduler task). Previously with Typo3 v8 and v9, I initialized entire $GLOBALS["TSFE"] object, however this was already hack the last time around (using mostly deprecated calls) and now it has all been removed with the v10 release.
My goal:
Access the TypoScript configuration of the frontend of a certain page (root page of a site would be fine) from within a scheduler job.
Background of the whole project:
I have a periodic scheduler job that sends emails to various users (fe_users). The email contains links to certain pages (configured UIDs in typoscript) as well as file attachments and the likes (generated by other extensions, which are also fully configured via typoscript). Currently, I basically initialize the entire frontend from within the backend, but as I said before, its inefficient, super hacky and I doubt it was the intended way to solve this problem.
Getting TypoScript settings in the backend is ugly, but possible.
You need a page ID and a rootline which you can pass to \TYPO3\CMS\Core\TypoScript\TemplateService::runThroughTemplates().
Something along these lines:
$template = GeneralUtility::makeInstance(TemplateService::class);
$template->tt_track = false;
$rootline = GeneralUtility::makeInstance(
RootlineUtility::class, $pageId
)->get();
$template->runThroughTemplates($rootline, 0);
$template->generateConfig();
$typoScriptSetup = $template->setup;
You can get inspiration from \TYPO3\CMS\Extbase\Configuration\BackendConfigurationManager::getTypoScriptSetup and \TYPO3\CMS\Tstemplate\Controller\TypoScriptTemplateObjectBrowserModuleFunctionController
This won't get any better and is not intended to be done such way. I would use as configuration:
plain PHP, e.g. in $GLOBALS['TYPO3_CONF_VARS]`
YAML site config if depending on various sites
You can build links by using e.g. something like that
protected function generateUrl(int $pageId, int $recordId)
{
$additionalParams = 'tx_xxxx[action]=show&tx_ixxxx[controller]=Job&tx_xxxx[job]=' . $recordId;
return BackendUtility::getPreviewUrl($pageId, '', null, '', '', $additionalParams);
}

How to publish a draft Task Group via Azure DevOps API

I am in the process of converting some tasks inside numerous task groups we have to different tasks. Instead of doing this by hand I've opted for using Powershell along with the Rest API of Azure DevOps to update the JSON bodies of these task groups and send them to the API. The conversion of the tasks is working fine sofar.
sending a PUT to the rest API in order to update the current task group of the set works but I want to build in some retention / version history if some of the new tasks end up working different than expected. So just blatantly updating the existing Task Group under the same major version is not an option.
the UI of Azure DevOps has functionality where you can save changes to a task group as a 'draft' and then publish this draft as either a completely new version (major version + 1) or as a preview
I went ahead and attempted to send a PUT to the Rest API and upticking the major version using the following URI:
PUT https://dev.azure.com/{organization}/{project}/_apis/distributedtask/taskgroups/{taskGroupId}?api-version=5.1-preview.1
Along with the settings:
JSONObject.version.major = $currentversion + 1
JSONObject.preview = true
This results in the API returning an error saying:
Invoke-WebRequest : {"$id":"1","innerException":null,"message":"Task group {TaskGroupID} not found.","typeName":"Microsoft.TeamFoundation.DistributedTask.WebApi.MetaTaskDefinitionNotFoundException, Microsoft.TeamFoundation.DistributedTask.WebApi","type
Key":"MetaTaskDefinitionNotFoundException","errorCode":0,"eventId":3000}
I then went ahead and tried to see if I could create a draft version. When sending a POST to the following URI i was able to create a draft:
POST https://dev.azure.com/{organization}/{project}/_apis/distributedtask/taskgroups?api-version=5.1-preview.1
using the following settings in JSON:
JSONObject.version.major = 1
JSONObject.version.istest = true
JSONObject.id = $null
JSONObject.parentDefinitionId = {TaskGroupID of the taskgroup of which i am trying to make a draft}
This nets in a new Task group in draft state which i am able to view in the UI and modify and publish (with or without preview). When i export the created JSON from UI from a manually made draft and compare it versus the one coming from powershell i see no differences.
This last step is where I am stuck. I can't seem to convert the created draft into a new version of the {parentdefintionid} task group. I've tried the following settings:
Calling the taskgroupid URI of the parent with put while a draft is available
JSONObject.version.major = $currentversion + 1
JSONObject.version.isTest = false
JSONObject.preview = true
Removing ParentDefinitionID from JSONObject
is resulting in the same error where it states that it cannot find the ID of the parent:
Invoke-WebRequest : {"$id":"1","innerException":null,"message":"Task group {TaskGroupID} not found.","typeName":"Microsoft.TeamFoundation.DistributedTask.WebApi.MetaTaskDefinitionNotFoundException, Microsoft.TeamFoundation.DistributedTask.WebApi","type
Key":"MetaTaskDefinitionNotFoundException","errorCode":0,"eventId":3000}
The same is valid with above settings and calling the draftID URI
When i try to call the draftID task group URI with the following settings :
JSONObject.version.major = $currentversion + 1
JSONObject.version.isTest = false
JSONObject.preview = true
JSONObject.id = $ParentDefinitionID
Removing ParentDefinitionID from JSONObject
it results in the following error:
Invoke-WebRequest : {"$id":"1","innerException":null,"message":"The request specifies task group ID {parentTaskGroupID} but the supplied task group has ID {DraftTaskGroupID}.","typeName":"Microsoft.TeamFoundation.DistributedTask.WebApi.Task
GroupIdConflictException, Microsoft.TeamFoundation.DistributedTask.WebApi","typeKey":"TaskGroupIdConflictException","errorCode":0,"eventId":3000}
i've checked the actual JSONObject to an export of a published Task Group in UI and they match exactly so i'm quite positive that content is not the issue here.
The MS documentation is seriously lacking on API usage so i'm really in the dark there hoping to find some clues / solution here
Seems you were updating a already exist task group in Azure DevOps.
If you incremented the revision property to be 1 higher than what is currently deployed.
You need to submit the JSON with the same revision property that the server has.

Unable to flag / trigger "Merge when pipeline succeeds" via Gitlab Api (v3/v4)

So as a part of some tests to automatically accept / merge successful pipelines in our git repository i was running some tests to flag the "merge when pipeline succeeds" feature when the pipeline is still running:
So this button is available when the pipeline is still running and will convert to a green 'Accept merge' button when the pipeline succeeds:
(note that this picture was taken afterwards not to confuse the use-case)
additionally i have set these general settings:
So when checking the Gitlab API Documentation it says i should use the following endpoint:
PUT /projects/:id/merge_requests/:merge_request_iid/merge
when using the parameter ?merge_when_pipleline_succeeds=true it should flag the button.
However when i call this endpoint when the pipeline is still running (i built in a wait for 10 mins while testing this) i get the following result:
i am getting a Method Not Allowed. My assumption is that the endpoint i am using is correct because otherwise i would've gotten a bad request / not found return code.
when checking the gitlab merge request i am seeing that indeed the flag is not set to true:
However, when i manually click the blue button the mergerequest looks like this:
Also if i let the pipeline finish and then proceed to call the merge api (w/ or w/o the merge when pipeline succeeds flag) it will accept the merge. It just does not work when the pipeline is running (which is odd because even the button itself only shows when the pipeline is running)
so i am wondering what I am doing wrong here.
I am using a Powershell module to call the GitLab API. The Accept part of the module is not official and was made by myself because i found this feature missing.
I am using the same credentials for the API w/ a personal access token to authenticate to the API. Other functionality of the API work with this token like creating merge requests, retrieving status of a current MR and accepting a MR when the pipeline is finished.
I have tried the following variants :
Use the V3 api with merge_when_build_succeeds=true --> nets the same
result
Uncheck the "Only allow merge request to be merged if the
pipeline succeeds" --> nets the same result
Use ID of the merge request instead of IID
use /merge_when_pipeline_succeeds instead of ?merge_when_pipeline_succeeds=true
use True instead of true --> nets the same result
I get a similar issue with the python-gitlab library on v4. It works sometimes when I use:
mr.merge(merge_when_pipeline_succeeds=True)
Where mr is a ProjectMergeRequest object. However, if the MR has a merge conflict in it I get that 405 Method Not Allowed error back.
My best guess is to see if I can apply logic before calling mr.merge() to check for problems. Will update this if that works.
UPDATE: Looks like there is no feature to check for conflicts as of today. https://gitlab.com/gitlab-org/gitlab-ce/issues/41762
UPDATE 2: You can check merge_status when looking at the MR information, so either that attribute or an exception then mr.merge() fails would let you identify when it won't work.

MS project server update resources for timesheets

Since some SOAP operations were removed in Project Server 2016,
we are trying to replace the obsolete SOAP Statusing/UpdateStatus API call with the REST API call /Draft/Assignments('assignmentid') in order to assign resources and set the 'actualWork' property. The MSDN documentation says that we can send a MERGE or a PUT request to that URL but it doesn't mention what the request payload should look like.
Can you let me know what the JSON payload for this call should be?
MERGE _api/ProjectServer/Projects('projectid')/Draft/Assignments('assignmentid')
API documentation: https://msdn.microsoft.com/en-us/library/office/jj668054.aspx
replace things in < > with appropriate values for your data
1) CheckOut the project
POST <pwaUrl>/_api/projectserver/projects('<projectId>')/checkout
2) Add enterprise resource to project team
POST <pwaUrl>/_api/projectserver/projects('<projectId>')/draft/projectresources/addenterpriseresourcebyid('<enterpriseResourceId>')
3) Create the assignment to an existing task
POST <pwaUrl>/_api/projectserver/projects('<projectId>')/draft/assignments/add()
{ "parameters":{
"ResourceId":"<enterpriseResourceId>",
"TaskId":"<taskId>"
}
}
4) edit 1 or more assignment properties
PATCH <pwaUrl>/_api/projectserver/projects('<projectId>')/draft/assignments('<draftAssigmentId>')
{ "ActualWorkTimeSpan":"PT24H" }
5a) Publish & check-in:
POST <pwaUrl>/_api/projectserver/projects('<projectId>')/draft/publish(true)
5b) Or just Check-in (if you don’t want to publish):
POST <pwaUrl>/_api/projectserver/projects('<projectId>')/draft/checkin(false)

How can I get a list of all pull requests for a repo through the github API?

I want to obtain a list of all pull requests on a repo through the github API.
I've followed the instructions at http://developer.github.com/v3/pulls/ but when I query /repos/:owner/:repo/pulls it's consistently returning fewer pull requests than displayed on the website.
For example, when I query the torvalds/linux repo I get 9 open pull requests (there are 14 on the website). If I add ?state=closed I get a different set of 11 closed pull requests (the website shows around 20).
Does anyone know where this discrepancy arises, and if there's any way to get a complete list of pull requests for a repo through the API?
You can get all pull requests (closed, opened, merged) through the variable state.
Just set state=all in the GET query, like this->
https://api.github.com/repos/:owner/:repo/pulls?state=all
For more info: check the Parameters table at https://developer.github.com/v3/pulls/#list-pull-requests
Edit: As per Tomáš Votruba's comment:
the default value for, "per_page=30". The maximum is per_page=100. To get more than 100 results, you need to call it multiple itmes: "&page=1", "&page=2"...
PyGithub (https://github.com/PyGithub/PyGithub), a Python library to access the GitHub API v3, enables you to get paginated resources.
For example,
g = Github(login_or_token=$YOUR_TOKEN, per_page=100)
r = g.get_repo($REPO_NUMBER)
for pull in r.get_pulls('all'):
# You can access pulls
See the documentation (http://pygithub.readthedocs.io/en/latest/index.html).
With Github's new official CLI (command line interface):
gh pr list --repo OWNER/REPO
which would produce something like:
Showing 2 of 2 pull requests in OWNER/REPO
#62 Doing something that-weird-branch-name
#58 My PR title wasnt-inspired-branch
See additional details and options and installation instructions.
There is a way to get a complete list and you're doing it. What are you using to communicate with the API? I suspect you may not be doing something correctly. For example (there are only 13 open pull requests currently) using my API wrapper (github3.py) I get all of the open pull requests. An example of how to do it without my wrapper in python is:
import requests
r = requests.get('https://api.github.com/repos/torvalds/linux/pulls')
len(r.json()) == 13
and I can also get that result (vaguely) in cURL by counting the results myself: curl https://api.github.com/repos/torvalds/linux/pulls.
If you, however, run into a repository with more than 25 (or 30) pull requests that's an entirely different issue but most certainly it is not what you're encountering now.
If you want to retrieve all pull requests (commits, comments, issues etc) you have to use pagination.
https://developer.github.com/v3/#pagination
The GET request "pulls" will only return open pull-requests.
If you want to get all pull-requests either you do set the parameter state to all, or you use issues.
Extra information
If you need other data from Github, such as issues, then you can identify pull-requests from issues, and you can then retrieve each pull-request no matter if it is closed or open. It will also give you a couple of more attributes (mergeable, merged, merge-commit-sha, nr of commits etc)
If an issue is a pull-request, then it will contain that attribute. Otherwise, it is just an issue.
From the API: https://developer.github.com/v3/pulls/#labels-assignees-and-milestones
"Every pull request is an issue, but not every issue is a pull request. For this reason, “shared” actions for both features, like manipulating assignees, labels and milestones, are provided within the Issues API."
Edit I just found that issues behaves similar to pull-requests, so one would need to do retrieve all by setting the state parameter to all
You can also use GraphQL API v4 to request all pull requests for a repo. It requests all the pull requests by default if you don't specify the states field :
{
repository(name: "material-ui", owner: "mui-org") {
pullRequests(first: 100, orderBy: {field: CREATED_AT, direction: DESC}) {
totalCount
nodes {
title
state
author {
login
}
createdAt
}
}
}
}
Try it in the explorer
The search API shoul help: https://help.github.com/enterprise/2.2/user/articles/searching-issues/
q = repo:org/name is:pr ...
GitHub provides a "Link" header which specifies the previous, next and last URL to fetch the values.Eg, Link Header response,
<https://api.github.com/repos/:owner/:repo/pulls?state=all&page=2>; rel="next", <https://api.github.com/repos/:owner/:repo/pulls?state=all&page=15>; rel="last"
rel="next" suggests the next set of values.
Here's a snippet of Python code that retrieves information of all pull requests from a specific GitHub repository and parses it into a nice DataFrame:
import pandas as pd
organization = 'pvlib'
repository = 'pvlib-python'
state = 'all' # other options include 'closed' or 'open'
page = 1 # initialize page number to 1 (first page)
dfs = [] # create empty list to hold individual dataframes
# Note it is necessary to loop as each request retrieves maximum 30 entries
while True:
url = f"https://api.github.com/repos/{organization}/{repository}/pulls?" \
f"state={state}&page={page}"
dfi = pd.read_json(url)
if dfi.empty:
break
dfs.append(dfi) # add dataframe to list of dataframes
page += 1 # Advance onto the next page
df = pd.concat(dfs, axis='rows', ignore_index=True)
# Create a new column with usernames
df['username'] = pd.json_normalize(df['user'])['login']