I'm trying to query the specific projects and that project size in whole db, I am able to get the output with project names but I want to know the size of that projects in DB.
I have tried the below query and got the output of the projects which I need:
qtest=> select name from projects where name like 'MLC%' and issampleproject=false and enddate is null and issampleproject=false;
name
----------------------------------------------
MLC Customers
MLC DELIVERY
MLC Delivery - Clone - Copy-Paste Test
MLC (Sydney)
(4 rows)
I need the size of the above projects.
Related
I have json files sitting under differnt subfolders.
folder structures are like this below
/UserData/data/json/A/2021/01/28/03/
A_2021_01_28_03_file1.json
A_2021_01_28_03_file2.json
A_2021_01_28_03_file3.json
/UserData/data/json/A/2021/01/28/02/
A_2021_01_28_02_file1.json
A_2021_01_28_02_file2.json
/UserData/data/json/B/2021/03/27/02/
A_2021_03_27_02_file1.json
A_2021_03_27_02_file2.json
/UserData/data/json/C/2021/04/21/01/
A_2021_04_21_01_file1.json
A_2021_04_21_01_file2.json
I want to merge all the files available under A folder/B folder/C folder
and ingest them as A table, B table and C table in Azure Data Explorer.
the schema is
name string
timestamp date
value string
I don't see merge feature in copy activity, how could I achieve this?
appreciate your help
You need 3 copy activities.
In each copy activity, in source -> select "Wildcard file path"
and choose * to select all files.(see attached picture).
it will copy all the files under the specific folder
please read more: https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-data-lake-storage?tabs=data-factory
how to develop the extension to display query result on a board? Such thing is not possible in the Azure Devops unfortunatelly. I've found two extensions on the marketplace which are doing what I need:
AA Query Board
Query based boards
but this extensions are not updated for a long time and I couldn't contact the authors (I need to change few things in order to be able to use it internally in my company).
I've found also this topic Add tabs on query result pages, so it looks like it's quite easy to add new tab to the query result menu, but I have no idea and I can't find any info how to get data (work items) from query result to display them?
Rest of the extension is just to display this data in grid, so that would be also quite easy, but getting this query result data is blocking me.
There is a Query Results Widget that you can use to display the query results on the Dashboards under Overview.
1, First you need to create a shared query if not exist, and save query to the shared queries folder shown as below screenshot. (You can click the Column options from the Editor page to add and remove columns to be shown on the results)
Or drag and drop the query from My Queries folder to Shared Queries folder.
2, Go to Dashboards under Overview, and Click Edit, then search and add widget Query results
3, Click the gear icon on the Query Results widget to configure it and select the query you want to display. Then the query result will be display on the Dashboards
Update:
There are some other ways to show the query results on the dashboards, For below example:
you can select your shared query and click more actions(3dots) and click Add to dashboards. This will display simple total number of query results.
you can also create different Charts for the query results and add it to Dashboards.
Select your shared query and go to Charts tab, the choose New Chart, select a Chart type, After you configured the chart, you can click the 3dots on the chart and add it to dashboards, check below screenshot:
Eventually I managed to contact the author of the "AA Query Board" extension and it turns out that he has a public repository on GitHub with the source code of the extension, so basically everyone can lookup how it's done or base on it.
Link to the repository: https://github.com/staticnz/aa_query_board
I'd like to get the repositories that make the most active use of milestones and/or projects. By "most active" I mean something like most cards moved on a project board or most issues added to a milestone.
I tried GH Archive which has yearly datasets on Google bigquery. I ran this query
SELECT
JSON_EXTRACT(payload, '$.action')
FROM
[githubarchive:year.2017]
WHERE
type in ("IssuesEvent")
and JSON_EXTRACT(payload, '$.action') in ("milestoned", "labeled", "assigned")
LIMIT
20
and this query
SELECT
type
FROM
[githubarchive:year.2017]
WHERE
type IN ("MilestoneEvent",
"ProjectEvent",
"ProjectCardEvent")
LIMIT
20
Both return zero results. Does GH Archive not import all events? Am I making a mistake in the queries? Is there another source where I can get this information?
So, I'm trying to do some queries using bigquery-public-data:github_repos.files, which was updated on May 25, 2018, 2:07:03 AM, in theory, it contains all files data from github - as it says in the description of the table:
File metadata for all files at HEAD.
Join with [bigquery-public-data:github_repos.contents] on id columns
to search text.
So, I have this tool called goreleaser, to use it, users create a file named .goreleaser.yaml. To have an idea of how many repositories are using it, I was using the github search, something like this a search for filename:goreleaser extension:yaml extension:yml path:/, you can see the results on this link.
This shows 1k+ results, and gets results for all these possible names:
goreleaser.yml
goreleaser.yaml
.goreleaser.yml
.goreleaser.yaml
The problem is, github shows the 1k result count, but you can only paginate until 1k or so. I wrote some code in Go using the API and etc, you see it here.
Anyway, I tried to do something similar with bigquery, here is my foolish attempt:
SELECT repo_name, path
FROM [bigquery-public-data:github_repos.files]
WHERE REGEXP_MATCH(path, r'\.?goreleaser.ya?ml')
This will include the vendored tools, which is not ok, but that's not the problem. The problem is that even with the vendored tools, it only shows ~500 results, not 1k.
PS: I also tried the simplified version matching path with LIKE and etc, same results.
So, either I'm doing something horribly wrong, this table does not include all data as it says it does or github search is lying to me.
Any advice?
Thanks!
Not every project in GitHub is mirrored on BigQuery's repo dataset.
Let's look at all projects that got more than 40 stars in April, vs what we can find mirrored in BigQuery's repos:
SELECT COUNT(name) april_projects_gt_stars, COUNT(repo_name) projects_mirrored
FROM (
SELECT DISTINCT repo_name, name, c
FROM `bigquery-public-data.github_repos.files` a
RIGHT JOIN (
SELECT repo.name, COUNT(*) c
FROM `githubarchive.month.201804`
WHERE type='WatchEvent'
GROUP BY 1
HAVING c>40
) b
ON repo_name=name
)
9522 vs 3995. Why?
Only open source projects are mirrored. This according to the open source detected license - if GitHub can't tell what license a project is using, the project can't be mirrored.
New projects: The pipeline might miss some new projects. Please report them.
I'd like to make an overview of projects.
This contains some fixed info; table projects joined with some other tables.
Now the report needs to have some subqueries: how many sales per salesman, how many is in transit, inventory, ... and so on.
I have a tablix with grouping on project (as to create an excel with one worksheet per projet).
How would I go about executing a new subquery per project (f.e. select owner, sum(totalprice) from opportunities where project=xxx group by owner)
I know I could achieve this with subreports; but as I will have about 10 subreports, I was hoping I could solve this with extra datasets and some filtering (and thus keep all logic in one file).
What's the best way to achieve this?
I would create a dataset with this query:
select owner, project, sum(totalprice) as totalprice from opportunities group by owner, project
Next, in your tablix where you want to display owner and totalprice info, you will have an expression like this:
=LOOKUP(Fields!<FirstDataSetProjectFieldName>.Value, Fields!project.Value, Fields!owner.Value, "<NewDatasetName>")
The above code will send the value of the project you are searching for, match it with a the same field in your new dataset, then return the requested value from the new dataset. You can obviously do this for totalprice as well.
Check out the documentation for LOOKUP to get a better handle on it but I think this is the solution you are looking for.