I am using Buildbot 3.5.0 and I am looking for a way to use its REST-API to find all the builds that were started for a given revision.
I know there is the endpoint
http://.../api/v2/builds
and each build has its revision available within the properties element, which is some nested structure. How can I filter that list to get only those builds that mention a given revision inside their properties element?
Or is there some other way to find the relevant builds?
Some example data:
http://.../api/v2/builds
{
"builds": [
{
"builderid": 6,
"buildid": 358,
"buildrequestid": 537,
"complete": true,
"complete_at": 1652550289,
"masterid": 16,
"number": 77,
"properties": {},
"results": 0,
"started_at": 1652550269,
"state_string": "build successful",
"workerid": 2
},
... many more builds here ...
],
"meta": {
"total": 358
}
}
The properties for the above mentioned build can be found here:
http://.../api/v2/builds/358/properties
{
"meta": {},
"properties": [
{
"revision": [
"01234567899a48ecef090efa39647747c0a03ed5",
"Build"
],
"scheduler": [
"any_branch_foobar",
"Scheduler"
],
"workername": [
"someworker",
"Worker"
],
... many more properties here ...
}
]
}
Related
I want to create a script which count how many times a specific file appears in an artifact builds.
I'm trying to get the name of artifact files from Azure DevOps Rest Apis but it seems it doesn't exit any API calls to get that information.
These are the API calls I'm using to get artifact names from a specific pipeline:
This api get all the builds of a specific pipeline: https://learn.microsoft.com/en-us/rest/api/azure/devops/pipelines/runs/run-pipeline?view=azure-devops-rest-7.1
GET https://dev.azure.com/{organization}/{project}/_apis/pipelines/{pipelineId}/runs/{runId}?api-version=7.1-preview.1
When I get all the build-ID of that pipeline, I can start searching artifacts:
GET https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}/artifacts?api-version=7.0
Now I have all the artifacts of a specific build but I don't have idea how the get the filenames string of the artifacts. The only thing I found in Azure DevOps documentation is a REST API call to download specific files of the artifact:
https://learn.microsoft.com/en-us/rest/api/azure/devops/pipelines/artifacts/get?view=azure-devops-rest-7.1
GET https://dev.azure.com/{organization}/{project}/_apis/pipelines/{pipelineId}/runs/{runId}/artifacts?artifactName={artifactName}&$expand={$expand}&api-version=7.1-preview.1
or
https://learn.microsoft.com/en-us/rest/api/azure/devops/build/artifacts/get-artifact?view=azure-devops-rest-7.1
GET GET https://dev.azure.com/{organization}/{project}/_apis/build/builds/{buildId}/artifacts?artifactName={artifactName}&api-version=7.1-preview.5
What I expect is to find a REST API that get also some information of the files inside the artifacts.
Do someone know how to get that information?
You could use the following API which can be captured in the network trace:
POST https://dev.azure.com/{org}/_apis/Contribution/HierarchyQuery?api-version=5.0-preview
Here's the sample request body:
{
"contributionIds": [
"ms.vss-build-web.run-artifacts-data-provider"
],
"dataProviderContext": {
"properties": {
"artifactId": {artifactId},
"buildId": {buildId},
"sourcePage": {
"routeValues": {
"project": "{YourProjectName}"
}
}
}
}
}
And you'll get a response like this:
{
"dataProviderSharedData": {},
"dataProviders": {
"ms.vss-web.component-data": {},
"ms.vss-web.shared-data": null,
"ms.vss-build-web.run-artifacts-data-provider": {
"buildId": 1487,
"buildNumber": "1487",
"definitionId": 176,
"definitionName": "test5-CI",
"items": [
{
"artifactId": 117,
"name": "/TestResults",
"sourcePath": "/TestResults",
"size": 0,
"type": "directory",
"items": null
},
{
"artifactId": 117,
"name": "/a",
"sourcePath": "/a",
"size": 0,
"type": "directory",
"items": null
},
{
"artifactId": 117,
"name": "/b",
"sourcePath": "/b",
"size": 0,
"type": "directory",
"items": null
},
{
"artifactId": 117,
"name": "/s",
"sourcePath": "/s",
"size": 988,
"type": "directory",
"items": [
{
"artifactId": 117,
"name": "/s/README.md",
"sourcePath": "/s/README.md",
"size": 985,
"type": "file",
"items": null
},
{
"artifactId": 117,
"name": "/s/folder1",
"sourcePath": "/s/folder1",
"size": 3,
"type": "directory",
"items": [
{
"artifactId": 117,
"name": "/s/folder1/file1",
"sourcePath": "/s/folder1/file1",
"size": 3,
"type": "file",
"items": null
}
]
}
]
}
]
}
}
}
In postman:
And for your reference, this is the artifact in Azure DevOps:
I got a collection of 10000 ca. docs, where each doc has the following format:
{
"_id": {
"$oid": "631edc6e207c89b932a70a26"
},
"name": "Ethereum",
"auditInfoList": [
{
"coinId": "1027",
"auditor": "Fairyproof",
"auditStatus": 2,
"reportUrl": "https://www.fairyproof.com/report/Covalent"
}
],
"circulatingSupply": 122335921.0615,
"cmcRank": 2,
"dateAdded": "2015-08-07T00:00:00.000Z",
"id": 1027,
"isActive": 1,
"isAudited": true,
"lastUpdated": 1662969360,
"marketPairCount": 6085,
"quotes": [
{
"name": "USD",
"price": 1737.1982544180462,
"volume24h": 14326453277.535921,
"marketCap": 212521748520.66168,
"percentChange1h": 0.62330307,
"percentChange24h": -1.08847937,
"percentChange7d": 10.96517745,
"lastUpdated": 1662966780,
"percentChange30d": -13.49374496,
"percentChange60d": 58.25153862,
"percentChange90d": 42.27475921,
"fullyDilluttedMarketCap": 212521748520.66,
"marketCapByTotalSupply": 212521748520.66168,
"dominance": 20.0725,
"turnover": 0.0674117,
"ytdPriceChangePercentage": -53.9168
}
],
"selfReportedCirculatingSupply": 0,
"slug": "ethereum",
"symbol": "ETH",
"tags": [
"mineable",
"pow",
"smart-contracts",
"ethereum-ecosystem",
"coinbase-ventures-portfolio",
"three-arrows-capital-portfolio",
"polychain-capital-portfolio",
"binance-labs-portfolio",
"blockchain-capital-portfolio",
"boostvc-portfolio",
"cms-holdings-portfolio",
"dcg-portfolio",
"dragonfly-capital-portfolio",
"electric-capital-portfolio",
"fabric-ventures-portfolio",
"framework-ventures-portfolio",
"hashkey-capital-portfolio",
"kenetic-capital-portfolio",
"huobi-capital-portfolio",
"alameda-research-portfolio",
"a16z-portfolio",
"1confirmation-portfolio",
"winklevoss-capital-portfolio",
"usv-portfolio",
"placeholder-ventures-portfolio",
"pantera-capital-portfolio",
"multicoin-capital-portfolio",
"paradigm-portfolio",
"injective-ecosystem"
],
"totalSupply": 122335921.0615
}
Im pulling updated version of it and, to aviod duplicates, im doing the following by using 'update_one'
for doc in new_doc_list:
CRYPTO_TEMPORARY_LIST.update_one(
{ "name" : doc['name']},
{ "$set": {
"lastUpdated": doc['lastUpdated']
}
},
upsert=True)
The problem is it's too slow.
I'm trying to figure out how to improve speed by using update_many but can't figure out how to set it up.
I Basically want to update every document x name. Completely change the doc and not the "lastUpdated" field would b even better.
Thanks guys <3
As part of BCP/Disaster Recover planning, we want to simulate a restoration scenario of the application gateway template we have in bitbucket.
I have the json template as well as parameters file in the repository, but I also see a WAF rule template files for the gateway.
So basically there are 4 files....but the New-AzResourceGroupDeployment only takes in the main template file (-TemplateUri) as well as parameters file (-TemplateParameterUri). so how would i be able to specify the WAF templates as part of the gateway creation?
I do see references to the WAF rules in the main gateway template file but is that enough?
sku information:
"properties": {
"sku": {
"name": "WAF_v2",
"tier": "WAF_v2",
"capacity": 1
You can add WAF policies while creating the WAF v2 Gateway by just adding the below in the Template file in the same way it has been done in this Microsoft Documentation and providing the values as per your requirement :
{
"type": "Microsoft.Network/ApplicationGatewayWebApplicationFirewallPolicies",
"apiVersion": "2020-06-01",
"name": "[variables('AGWafPol01')]",
"location": "[parameters('location')]",
"properties": {
"customRules": [
{
"name": "CustRule01",
"priority": 100,
"ruleType": "MatchRule",
"action": "Block",
"matchConditions": [
{
"matchVariables": [
{
"variableName": "RemoteAddr"
}
],
"operator": "IPMatch",
"negationConditon": true,
"matchValues": [
"10.10.10.0/24"
]
}
]
}
],
"policySettings": {
"requestBodyCheck": true,
"maxRequestBodySizeInKb": 128,
"fileUploadLimitInMb": 100,
"state": "Enabled",
"mode": "Prevention"
},
"managedRules": {
"managedRuleSets": [
{
"ruleSetType": "OWASP",
"ruleSetVersion": "3.1"
}
]
}
}
},
We have a self-hosted Azure DevOps Server 2019 for which I have created my own widget using the tutorial https://learn.microsoft.com/en-us/azure/devops/extend/develop/add-dashboard-widget?view=azure-devops-2019.
I was also able to successfully add the extension to the server and the extension is also visible.
However, when I edit my dashboard, my newly created widget is not displayed. What could be the reason for this?
Furthermore, I also installed the extension VstsDashboardWidgetProjectTemplate (https://github.com/GregTrevellick/VstsDashboardWidgetProjectTemplate) in Visual Studio and also created an extension/widget from it. Unfortunately with the same result that I can install and see the extension, but not the actual widget.
I am attaching the vss-extension.json file as I suspect this is where the error is most likely to be found. However, I can also supply the other files if needed.
{
"manifestVersion": 1,
"id": "MyVeryFirstCustomExtension",
"version": "1.0.0",
"name": "My Very First Custom Extension",
"description": "Samples containing different widgets extending dashboards",
"publisher": "MyCompany",
"categories": [
"Azure Boards"
],
"targets": [
{
"id": "Microsoft.VisualStudio.Services"
}
],
"icons": {
"default": "img/logo.png"
},
"contributions": [
{
"id": "HelloWorldWidget",
"type": "ms.vss-dashboards-web.widget",
"targets": [
"ms.vss-dashboards-web.widget-catalog"
],
"properties": {
"name": "Hello World Widget",
"description": "My first widget",
"catalogIconUrl": "img/CatalogIcon.png",
"previewImageUrl": "img/preview.png",
"uri": "hello-world.html",
"supportedSizes": [
{
"rowSpan": 1,
"columnSpan": 2
}
],
"supportedScopes": [
"project_team"
]
}
}
],
"files": [
{
"path": "hello-world.html",
"addressable": true
},
{
"path": "sdk/scripts",
"addressable": true
},
{
"path": "img",
"addressable": true
}
],
"scopes": [
"vso.work"
]
}
Are there some good examples where I want to aggregate the following using Mongo's Aggregation Pipeline in scala.
Want to get the sum PriceAmt.
Any good URL with some great Scala examples would be of great help..
Following is the JSON that is being retrieved from MongoDB.
{
"_id": ObjectId("5269723bd516ec3a69f3639e"),
"Id": ObjectId("5269723ad516ec3a69f3639d"),
"results": [
{
"Id": "526971f5b5b8b9148404623a",
"pricingResult": {
"TxId": 0,
"PriceId": "Large_Batch_1",
"Errors": [ ],
"Disposition": [
{
"GroupId": 1,
"PriceAmt": 20,
"Status": "Priced Successfully",
"ReasonCode": 0,
"PricingMethodologyId": 2,
"Lines": [{ "Id": 1 }]
}
]
}
},
{
"ClaimId": "526971f4b5b8b91484046229",
"pricingResult": {
"TxId": 0,
"PriceId": "Large_Batch_1",
"Errors": [ ],
"Disposition": [
{
"GroupId": 1,
"PriceAmt": 20,
"Status": "Priced Successfully",
"ReasonCode": 0,
"PricingMethodologyId": 2,
"Lines": [{ "Id": 1 }]
}
]
}
}
]
}
I do not think that this is a valid question, but if you are looking for examples of aggregation framework and scala take a look at this links:
https://bhudgeons.telegr.am/blog_posts/how-to-use-the-new-mongodb-aggregation-framew
http://mongoblog.tumblr.com/post/44108207769/mongodb-aggregation-using-scala-1-2
and after this you can try to replicate some results with scala using examples from mongodb
Casbah, the official Scala driver, support the aggregation framework.
Please take a look at the documentation.