I am attempting to create a generic configuration variable in Terraform that will allow someone to setup a new repo in Azure DevOps and specify the users or groups that should be assigned to each group or team.
All groups and members come from and are managed by Azure Active directory.
There are two things I am trying to achieve:
Assign AD users and AD groups to the existing built-in Azure DevOps default groups that exist today like (Readers and Contributors) and
Create custom team names and assign AD users and groups to them.
Here is the proposed variable structure. One for the built-in ADO groups and one for the new teams that need to be created:
TF_VAR_ado_groups = {
Readers: {
groups = [
"AD-ReadyOnly"
],
members = [
"user1#example.com",
"user2#example.com"
]
},
Contributors: {
groups = [
"AD-ReadyOnly"
],
members = [
"user1#example.com",
"user2#example.com"
]
}
}
TF_VAR_teams = {
Approvers: {
groups = [
"AD-ReadyOnly",
"AD-Approvers"
],
members = [
"user2#example.com",
"user3#example.com"
]
},
Managers: {
groups = [
"AD-Managers"
],
members = [
"user10#example.com",
"user12#example.com"
]
}
}
Note that the keys inside the TF_VAR_ado_groups variable (Readers,Contributors) are simply being looked up and assigned these principals while the ones in the TF_VAR_teams variable (Approvers, Managers) are team names that will be dynamically created and assigned those principals.
I have some rough code but still trying to wrap my head how to structure this in Terraform code. I know I will need to use resource.azuredevops_user_entitlement, data.azuread_group, and azuredevops_group for the user lookups to get their descriptors from AD or ADO to finally be used by azuredevops_group_membership for the assignments.
The challenge is structuring all of this with the necessary looping constructs and it feels like I would really need to nest the loops and block declarations to do this properly. Any advice on the best approach for this would be much appreciated.
Some rough brainstorm code (non-working at the moment) is below. This mostly represents a working case I had earlier of getting a single AD user and a single AD group assigned to the Readers default group in Azure DevOps.
// Role: Read only
resource "azuredevops_user_entitlement" "reader" {
for_each = var.TF_VAR_ado_groups
principal_name = each.key
}
data "azuread_group" "readonly" {
count = length(var.TF_VAR_reader_ado_group.groups)
display_name = var.TF_VAR_reader_ado_group.groups
}
resource "azuredevops_group" "aad_readonly" {
origin_id = data.azuread_group.readonly.object_id
}
data "azuredevops_group" "readers" {
name = "Readers"
project_id = azuredevops_project.example.id
}
resource "azuredevops_group_membership" "readers" {
group = data.azuredevops_group.readers.descriptor
mode = "add"
members = concat(azuredevops_user_entitlement.reader[*].descriptor, [azuredevops_group.aad_readonly.descriptor])
}
I figured it out after much trial and error. Assume you want to create or update the following Azure DevOps groups or teams and have a variable called permissions like this:
locals {
permissions = {
# Azure DevOps Default Groups
"Project Administrators": {
ad_groups = [
"ActiveDirectoryGroup1",
"ActiveDirectoryGroup2"
]
ado_groups = [
"AzureDevOpsGroup1",
"AzureDevOpsGroup2"
],
users = [
"user1#example.com",
"user2#example.com"
]
},
"Contributors": {
ad_groups = [
"ActiveDirectoryGroup3"
]
ado_groups = [
"My New Team"
],
users = []
},
# Azure DevOps Create New Group
"My New DevOps Group": {
custom_group = true
ad_groups = [
"ActiveDirectoryGroup4"
]
ado_groups = [],
users = []
},
# Create Azure DevOps Teams
"My New Team": {
is_team = true,
team_admins = [
"user1#example.com"
],
ad_groups = [
"ActiveDirectoryGroup5"
],
ado_groups = [],
users = [
"user1#example.com",
"user2#example.com",
"user3#example.com",
"user4#example.com",
"user5#example.com",
]
}
}
}
Then you can use the following terraform code to parse that permissions variable to and create/update groups as well as create teams dynamically:
// ROLE / PERMISSION ASSIGNMENTS
# Aggregate each resource by it's category into a single list
locals {
ad_groups = toset(flatten([for entry in local.permissions : entry.ad_groups]))
ado_groups = toset(flatten([for entry in local.permissions : entry.ado_groups]))
users = toset(flatten([for entry in local.permissions : entry.users]))
}
# Get all users
data "azuredevops_users" "all" {
for_each = local.users
principal_name = each.key
}
# Get active directory descriptors roster
data "azuread_group" "aad" {
for_each = local.ad_groups
display_name = each.key
security_enabled = true
}
resource "azuredevops_group" "aad_groups" {
for_each = toset(values(data.azuread_group.aad)[*].object_id)
origin_id = each.key
}
# Create new groups
resource "azuredevops_group" "new" {
for_each = {for group_name, entry in local.permissions : group_name => entry if !lookup(entry, "is_team", false) && lookup(entry, "custom_group", false)}
scope = azuredevops_project.example.id
display_name = each.key
description = "Managed by Terraform"
}
# Query for all groups including the ones we just created
data "azuredevops_groups" "all" {
project_id = azuredevops_project.example.id
}
# Create descriptor map to make lookups easy
locals {
descriptor_map = merge(
{for entry in flatten(values(data.azuredevops_users.all)[*].users[*]) : entry.principal_name => entry.descriptor},
{for entry in data.azuredevops_groups.all.groups : entry.display_name => entry.descriptor},
{for entry in azuredevops_group.aad_groups : entry.display_name => entry.descriptor},
{for entry in azuredevops_group.new : entry.display_name => entry.descriptor}
)
}
# Create or update teams
resource "azuredevops_team" "teams" {
for_each = {for team_name, entry in local.permissions : team_name => entry if lookup(entry, "is_team", false)}
project_id = azuredevops_project.example.id
name = each.key
description = "Managed by Terraform"
administrators = toset(compact(concat([for user in each.value.team_admins : lookup(local.descriptor_map, user, "")])))
members = toset(compact(concat(
[for user in each.value.users : lookup(local.descriptor_map, user, "")],
[for group in each.value.ad_groups : lookup(local.descriptor_map, group, "")],
[for group in each.value.ado_groups : lookup(local.descriptor_map, group, "")]
)))
}
locals {
descriptor_map2 = merge(
local.descriptor_map,
{for team_name, entry in azuredevops_team.teams : team_name => entry.descriptor}
)
}
# Update group memberships
resource "azuredevops_group_membership" "assignments" {
for_each = {for group_name, entry in local.permissions : group_name => entry if !lookup(entry, "is_team", false)}
group = local.descriptor_map2[each.key]
mode = "overwrite"
members = toset(compact(concat(
[for user in each.value.users : lookup(local.descriptor_map2, user, "")],
[for group in each.value.ad_groups : lookup(local.descriptor_map2, group, "")],
[for group in each.value.ado_groups : lookup(local.descriptor_map2, group, "")]
)))
}
Permission Entry Fields
The following fields are defined and available for each entry within the permissions map:
The map key represents the name of the group or team you are creating or updating. The map value for a given key can contain the following options:
is_team - Required to be set to true when creating a team. Defaults to false.
team_admins - (optional) Used to set the admins of a team.
custom_group - (optional) Used to set to true when creating a new group. This flag helps Terraform discern between existing built-in groups like Contributors/Readers/etc. and custom ones you create. Defaults to false.
ad_groups - (required) An array of Active Directory groups that should be assigned as members. These values are case sensitive. Pass an empty array if no membership assignments.
ado_groups - (required) An array of Azure DevOps groups that should be assigned as members. These values are case sensitive. Pass an empty array if no membership assignments.
users - (required) An array of email addresses. If a valid descriptor is not found for the email address in Active Directory, then this user cannot be associated to the group or team. Pass an empty array if no membership assignments.
Known Limitations
The default team Azure DevOps creates when a new project is created (usually denoted by a team name of "<PROJECT_NAME> Team") is unused and currently unable to be managed in Terraform.
according to terraform doc, I created a user and role as follows:
resource "postgresql_role" "ro_role" {
name = "ro_role"
}
resource "postgresql_role" "ro_user" {
name = "ro_user"
login = true
password = "some sensitive password"
}
which is successfully created.
next, when trying to grand permissions to a role, such as for a readonly role adding SELECT only, it does not add any privileges at all. Also when login to Postgres and trying SELECT query, it gives me a permission denied error.
resource "postgresql_grant" "readonly_tables" {
database = var.database
role = "ro_role"
schema = "public"
object_type = "table"
objects = []
privileges = ["SELECT"]
}
terraform doc for grant permission:
https://registry.terraform.io/providers/cyrilgdn/postgresql/latest/docs/resources/postgresql_grant
You likely didn't grant yourself USAGE on the public schema. Based on the docs you provided, you'll need to define a resource like:
resource "postgresql_grant" "readonly_tables_schema_usage_public" {
database = var.database
role = "ro_role"
schema = "public"
object_type = "schema"
objects = ["public]
privileges = ["USAGE"]
}
the postgresql_role.ro_user resources is also not assinged the ro_role. You will need this for ro_user to actually have the permissions assigned to ro_role:
resource "postgresql_role" "ro_user" {
name = "ro_user"
login = true
password = "some sensitive password",
roles=[postgresql_role.ro_role.role]
}
Is there a way/possible on how we can get the API GAteway ID by name or can we iterate the list and return by its name from AWS CLI, i tried the following way and it doesn't return any thing
aws apigateway get-rest-apis --query 'items[?name==`TestAPI`].value' --output text --region us-east-1
thanks in advance
Updated the list output
"items": [
{
"id": "5aa9gcij77",
"name": "JavaLamdba",
"description": "JavaLamdba",
"createdDate": 1608225655,
"apiKeySource": "HEADER",
"endpointConfiguration": {
"types": [
"REGIONAL"
]
}
},
aws apigateway get-rest-apis --query 'items[?name==`JavaLamdba`].id' --output text --region us-east-1
This should give you the expected result
> select name, description, created_date from aws.aws_api_gateway_rest_api where name = 'lambda-test';
+-------------+-------------+---------------------+
| name | description | created_date |
+-------------+-------------+---------------------+
| lambda-test | lambda-test | 2019-07-25 09:05:16 |
+-------------+-------------+---------------------
https://steampipe.io/
By checking this AWS blog: https://aws.amazon.com/premiumsupport/knowledge-center/users-connect-rds-iam/ I noticed that I need to create a DB user after login with the master username and password:
CREATE USER {dbusername} IDENTIFIED WITH AWSAuthenticationPlugin as 'RDS';
I can see terraform has mysql_user to provision mysql db users: https://www.terraform.io/docs/providers/mysql/r/user.html
However, I couldn't find postgres_user. Is there a way to provision postgres user with IAM auth?
In Postgres, a user is called a "role". The Postgres docs say:
a role can be considered a "user", a "group", or both depending on how it is used
So, the TF resource to create is a postgresql_role
resource "postgresql_role" "my_replication_role" {
name = "replication_role"
replication = true
login = true
connection_limit = 5
password = "md5c98cbfeb6a347a47eb8e96cfb4c4b890"
}
To enable IAM user to assume the role, follow the steps in the AWS docs.
From those instructions, you would end up with TF code looking something like:
module "db" {
source = "terraform-aws-modules/rds/aws"
// ...
}
provider "postgresql" {
// ...
}
resource "postgresql_role" "pguser" {
login = true
name = var.pg_username
password = var.pg_password
roles = ["rds_iam"]
}
resource "aws_iam_user" "pguser" {
name = var.pg_username
}
resource "aws_iam_user_policy" "pguser" {
name = var.pg_username
user = aws_iam_user.pguser.id
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"rds-db:connect"
],
"Resource": [
"arn:aws:rds-db:${var.region}:${data.aws_caller_identity.current.account_id}:dbuser:${module.db.this_db_instance_resource_id}/${var.pg_username}"
]
}
]
}
EOF
}
I got vcs compile error when adding function in declaration of struct. The IEEE doc does not mention if function in struct is allowed.
I also got vcs compile error when trying to assign a default value to a field. But it is allowed in IEEE-1800-2012 7.2.2.
typedef struct {
int a = 1; //compile error here
int b;
function void func();
b = a;
endfunction
} a_struct;
So I add the command line and error info as suggested:
vcs -sverilog a.sv
Error-[V2KIIAD] Invalid initialization at declaration
....
Struct or union member field 'a' cannot be initialized at declaration.
Error-[SE] Syntax error
Following verilog source has syntax error :
"a.sv", 4: token is 'function'
function void func();
^
And my vcs version is 2013.06-SP1-10
Functions declared inside structs are not supported as of IEEE Std 1800-2012. Looking over the syntax for structure declaration, a struct_union_member is a data_type_or_void and a function is not data_type.
§ 7.2 Structures
data_type ::= // from A.2.2.1
...
| struct_union [ packed [ signing ] ] { struct_union_member { struct_union_member } }
{ packed_dimension }
struct_union_member ::=
{ attribute_instance } [random_qualifier] data_type_or_void list_of_variable_decl_assignments ;
data_type_or_void ::= data_type | void
struct_union ::= struct | union [ tagged ]
Expanding data_type from § A.2.2.1 Net and variable types
data_type ::=
integer_vector_type [ signing ] { packed_dimension }
| integer_atom_type [ signing ]
| non_integer_type
| struct_union [ packed [ signing ] ] { struct_union_member { struct_union_member } }
{ packed_dimension }
| enum [ enum_base_type ] { enum_name_declaration { , enum_name_declaration } }
{ packed_dimension }
| string
| chandle
| virtual [ interface ] interface_identifier [ parameter_value_assignment ] [ . modport_identifier ]
| [ class_scope | package_scope ] type_identifier { packed_dimension }
| class_type
| event
| ps_covergroup_identifier
| type_reference
As we know that structures are static and classes are dynamic data types. So in SV, while we are using class,we can assign values intially or in other conditions. But for structure we can't initialise any variable inside the declaration.