Can't make AWS Aurora Postgres RDS publicly available - postgresql

I'm trying to spin up an Aurora Postgres Cluster and I can't seem to make it available over the internet. I'm using Terraform to code the infrastructure.
I've created a security group to allow external access and that is attached to the VPC's subnets used by the Cluster. Still, I can't seem to be able to access the endpoints from my local machine.
I can't figured out what I'm missing.
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = ">=3.11.0"
name = "vpc-auroradb-${var.environment}"
cidr = var.vpc_cidr_block
azs = var.availability_zones
private_subnets = var.vpc_private_subnets
public_subnets = var.vpc_public_subnets
database_subnets = var.vpc_database_subnets
enable_nat_gateway = true
enable_dns_hostnames = true
enable_dns_support = true
create_igw = true
create_database_internet_gateway_route = true
create_database_nat_gateway_route = true
create_database_subnet_group = true
create_database_subnet_route_table = true
}
module "aurora_cluster" {
source = "terraform-aws-modules/rds-aurora/aws"
version = ">=6.1.3"
name = "bambi-${var.environment}"
engine = "aurora-postgresql"
engine_version = "12.8"
instance_class = "db.t4g.large"
publicly_accessible = true
instances = {
1 = {
identifier = "bambi-1"
}
2 = {
identifier = "bambi-2"
}
}
autoscaling_enabled = true
autoscaling_min_capacity = 2
autoscaling_max_capacity = 3
vpc_id = module.vpc.vpc_id
db_subnet_group_name = module.vpc.database_subnet_group_name
create_db_subnet_group = false
create_security_group = false
iam_database_authentication_enabled = true
storage_encrypted = true
apply_immediately = true
monitoring_interval = 30
db_parameter_group_name = aws_db_parameter_group.parameter_group.id
db_cluster_parameter_group_name = aws_rds_cluster_parameter_group.parameter_group.id
vpc_security_group_ids = [aws_security_group.sg_public.id]
enabled_cloudwatch_logs_exports = ["postgresql"]
}
resource "aws_security_group" "sg_public" {
vpc_id = module.vpc.vpc_id
ingress {
from_port = 5432
to_port = 5432
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"] # Allowing traffic in from all sources
}
egress {
from_port = 0 # Allowing any incoming port
to_port = 0 # Allowing any outgoing port
protocol = "-1" # Allowing any outgoing protocol
cidr_blocks = ["0.0.0.0/0"] # Allowing traffic out to all IP addresses
}
}

From the documentation of the used VPC module, in order to have public access for the database, you need the following:
create_database_subnet_group = true
create_database_subnet_route_table = true
create_database_internet_gateway_route = true
enable_dns_hostnames = true
enable_dns_support = true
create_database_nat_gateway_route should not be true. If we take a look at the code for the module on github:
resource "aws_route" "database_internet_gateway" {
count = var.create_vpc && var.create_igw && var.create_database_subnet_route_table && length(var.database_subnets) > 0 && var.create_database_internet_gateway_route && false == var.create_database_nat_gateway_route ? 1 : 0
route_table_id = aws_route_table.database[0].id
destination_cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.this[0].id
timeouts {
create = "5m"
}
}
We can see that the count for the route for the internet gateway will be 0. This means the route which would allow public internet access is not created for the database subnet.
In the other hand setting create_database_internet_gateway_route to true will block access through the NAT gateway as well, since the route table wont have the correct route.
resource "aws_route" "database_nat_gateway" {
count = var.create_vpc && var.create_database_subnet_route_table && length(var.database_subnets) > 0 && false == var.create_database_internet_gateway_route && var.create_database_nat_gateway_route && var.enable_nat_gateway ? var.single_nat_gateway ? 1 : length(var.database_subnets) : 0
route_table_id = element(aws_route_table.database.*.id, count.index)
destination_cidr_block = "0.0.0.0/0"
nat_gateway_id = element(aws_nat_gateway.this.*.id, count.index)
timeouts {
create = "5m"
}
}
Essentially you block all the traffic by setting both variables to true.

Related

AWS RDS for PostgreSQL Connection attempt timed out error

I created postgresql rds in aws with terraform. I'm checking from the aws console, everything seems normal. But I'm trying to connect to database with DBeaver but I can't connect. Likewise, I can't make the ssh connection for the ec2 I created, maybe there is a connection.
The terraform codes I wrote:
# postgres-db/main.tf
resource "aws_db_instance" "default" {
allocated_storage = 20
storage_type = "gp2"
engine = var.engine
engine_version = var.engine-version
instance_class = var.instance-class
db_name = var.db-name
identifier = var.identifier
username = var.username
password = var.password
port = var.port
publicly_accessible = var.publicly-accessible
db_subnet_group_name = var.db-subnet-group-name
parameter_group_name = var.parameter-group-name
vpc_security_group_ids = var.vpc-security-group-ids
apply_immediately = var.apply-immediately
skip_final_snapshot = true
}
module "service-db" {
source = "./postgres-db"
apply-immediately = true
db-name = var.service-db-name
db-subnet-group-name = data.terraform_remote_state.server.outputs.db_subnet_group
identifier = "${var.app-name}-db"
password = var.service-db-password
publicly-accessible = true # TODO: True for now, but should be false
username = var.service-db-username
vpc-security-group-ids = [data.terraform_remote_state.server.outputs.security_group_allow_internal_postgres]
}
resource "aws_security_group" "allow_internal_postgres" {
name = "allow-internal-postgres"
description = "Allow internal Postgres traffic"
vpc_id = aws_vpc.vpc.id
ingress {
from_port = 5432
to_port = 5432
protocol = "tcp"
cidr_blocks = [aws_vpc.vpc.cidr_block, "0.0.0.0/0"] # TODO: Remove public IP
}
}
In the research I did, it was written things like edit the security rules or set it to public, it seems like that anyway.
Security group inbound rules
Public accessible
How can I solve this problem can you please help?
I solved my problem by setting the subnet group to public.
module "service-db" {
source = "./postgres-db"
apply-immediately = true
db-name = var.service-db-name
db-subnet-group-name = data.terraform_remote_state.server.outputs.db_subnet_group_public
identifier = "${var.app-name}-db"
password = var.service-db-password
publicly-accessible = true # TODO: True for now, but should be false
username = var.service-db-username
vpc-security-group-ids = [data.terraform_remote_state.server.outputs.security_group_allow_internal_postgres]
}
resource "aws_db_subnet_group" "private" {
name = "${var.server_name}-db-subnet-group-private"
subnet_ids = aws_subnet.private.*.id
tags = {
Name = "${var.server_name} DB Subnet Group Private"
}
}
resource "aws_db_subnet_group" "public" {
name = "${var.server_name}-db-subnet-group-public"
subnet_ids = aws_subnet.public.*.id
tags = {
Name = "${var.server_name} DB Subnet Group Public"
}
}

Terraform import error retrieving Virtual Machine Scale Set created from an image

I'm trying to import a Linux VM Scale Set that was deployed in the Azure Portal from a custom shared image, also created in the portal. I'm using the following command:
terraform import module.vm_scaleset.azurerm_linux_virtual_machine_scale_set.vmscaleset /subscriptions/00000000-0000-0000-0000-000000000000
/resourceGroups/myrg/providers/Microsoft.Compute/virtualMachineScaleSets/vmss1
Import fails with the following error:
Error: retrieving Virtual Machine Scale Set "vmss1" (Resource Group "myrg"): properties.virtualMachineProfile.osProfile was nil
Below is my VM Scale set module code
data "azurerm_lb" "loadbalancer" {
name = var.lbName
resource_group_name = var.rgName
}
data "azurerm_lb_backend_address_pool" "addresspool" {
loadbalancer_id = data.azurerm_lb.loadbalancer.id
name = var.lbAddressPool
}
data "azurerm_shared_image" "scaleset_image" {
provider = azurerm.ist
name = var.scaleset_image_name
gallery_name = var.scaleset_image_gallery
resource_group_name = var.scaleset_image_rgname
}
resource "azurerm_linux_virtual_machine_scale_set" "vmscaleset" {
name = var.vmssName
resource_group_name = var.rgName
location = var.location
sku = var.vms_sku
instances = var.vm_instances
admin_username = azurerm_key_vault_secret.vmssusername.value
admin_password = azurerm_key_vault_secret.vmsspassword.value
disable_password_authentication = false
zones = var.vmss_zones
source_image_id = data.azurerm_shared_image.scaleset_image.id
tags = module.vmss_tags.tags
os_disk {
storage_account_type = var.vmss_osdisk_storage
caching = "ReadWrite"
create_option = "FromImage"
}
data_disk {
storage_account_type = "StandardSSD_LRS"
caching = "None"
disk_size_gb = 1000
lun = 10
create_option = "FromImage"
}
network_interface {
name = format("nic-%s-001", var.vmssName)
primary = true
enable_accelerated_networking = true
ip_configuration {
name = "internal"
load_balancer_backend_address_pool_ids = [data.azurerm_lb_backend_address_pool.addresspool.id]
primary = true
subnet_id = var.subnet_id
}
}
lifecycle {
ignore_changes = [
tags
]
}
}
The source image was created from a Linux RHEL 8.6 VM that included a custom node.js script.
Examination of the Scale Set in the portal does indeed show that the virtualMachineProfile.osProfile is absent.
I haven't been able to find a solution on any forum. Is there any way to ignore the error and import the Scale Set anyway?

How to get the http_Listener.hostname in azure terraform

Here's my application gateway terraform template:
resource "azurerm_application_gateway" "appgw" {
name = "${var.client_name_prefix}${var.environment_name}${var.location_abr}gw${var.instance}"
resource_group_name = azurerm_resource_group.rg.name
location = var.location
enable_http2 = true
backend_http_settings {
name = "backOfficeAppHttpSetting"
cookie_based_affinity = "Disabled"
port = var.backed_port
protocol = "Http"
probe_name = "nginx-ingress-prob"
request_timeout = var.backend_request_timeout_back_office
pick_host_name_from_backend_address = false
host_name = "local.backoffice.${var.sslcert}"
affinity_cookie_name = "ApplicationGatewayAffinity"
}
http_listener {
name = "backOfficeAppListener"
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_443_name
protocol = "Https"
host_name = "${var.client_name}-${var.environment_name}-admin.${var.sslcert}"
require_sni = true
ssl_certificate_name = var.sslcert
}
http_listener {
name = "frontWebAppListener"
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_443_name
protocol = "Https"
host_name = "${var.client_name}-${var.environment_name}.${var.sslcert}"
require_sni = true
ssl_certificate_name = var.sslcert
}
}
And In key-vault.tf. I want to create to secret in which I want to save the value of first http_listener (HOSTNAME).
BUT, I'm unable to get it. Is there any way to get http_listener.host_name?
I want to know is there any way to get it.
I have tried it but i couldn't get it.

OCI: Create nodes in Kubernetes nodepool with bastion agent configured

I'm trying to deploy a Kubernetes cluster in Oracle Cloud Infrastructure using Terraform.
I want that every node deployed (in private subnet) has the Bastion agent plugin activate in Cloud Agent.
But I cannot see how to define the details of the instance (setting agent_config in the node pool instances).
My code, until now is:
resource "oci_containerengine_cluster" "generated_oci_containerengine_cluster" {
compartment_id = var.cluster_compartment
endpoint_config {
is_public_ip_enabled = "true"
subnet_id = oci_core_subnet.oke_public_api.id
}
kubernetes_version = var.kubernetes_version
name = "josealbarran_labcloudnative_oke"
options {
kubernetes_network_config {
pods_cidr = "10.244.0.0/16"
services_cidr = "10.96.0.0/16"
}
service_lb_subnet_ids = [oci_core_subnet.oke_public_lb.id]
}
vcn_id = var.cluster_vcn
}
# Check doc: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/containerengine_node_pool
resource "oci_containerengine_node_pool" "node_pool01" {
cluster_id = "${oci_containerengine_cluster.generated_oci_containerengine_cluster.id}"
compartment_id = var.cluster_compartment
initial_node_labels {
key = "name"
value = "pool01"
}
kubernetes_version = var.kubernetes_version
name = "lab_cloud_native_oke_pool01"
node_config_details {
size = "${length(data.oci_identity_availability_domains.ads.availability_domains)}"
dynamic "placement_configs" {
for_each = data.oci_identity_availability_domains.ads.availability_domains[*].name
content {
availability_domain = placement_configs.value
subnet_id = oci_core_subnet.oke_private_worker.id
}
}
}
node_shape = "VM.Standard.A1.Flex"
node_shape_config {
memory_in_gbs = "16"
ocpus = "1"
}
node_source_details {
image_id = "ocid1.image.oc1.eu-frankfurt-1.aaaaaaaalgodii3qx3mfasp6ai22bja7mabfwsxiwkzxx7lhdfdbbuyqcznq"
source_type = "IMAGE"
}
ssh_public_key = "ssh-rsa AAAAB3xxxxxxxx......."
timeouts {
create = "60m"
delete = "90m"
}
}
You can use the "cloudinit_config" to run the custom script in OKE node pool in OCI.
second_script_template = templatefile("${path.module}/cloudinit/second.template.sh",{})
More scripts like
data "cloudinit_config" "worker" {
gzip = false
base64_encode = true
part {
filename = "worker.sh"
content_type = "text/x-shellscript"
content = local.worker_script_template
}
part {
filename = "second.sh"
content_type = "text/x-shellscript"
content = local.second_script_template
}
part {
filename = "third.sh"
content_type = "text/x-shellscript"
content = local.third_script_template
}
}
Refer : https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/instructions.adoc#14-configuring-cloud-init-for-the-nodepools
If you are looking forward to just edit the default script : https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/main/docs/cloudinit.adoc

Is there a web redirect method or example using an application gateway using terraform?

im trying to create a service for web redirect through the application gateway using terraform.
I would like to authenticate the application gateway sl with the free certified (azurm_app_service_managed_certified) of the azure app service plan, is there an example?
Currently, thinking about the composition as follows. However, azurem_application_gateway is demanding ssl certification, so I don't know how to work.
Please let me know if there's a way to solve the problem in that way or in another way.
The problem with the script below is that if you want to use https in the application gateway, you have to use certificate, and I want to make and use free certificated in the service plan.
resource "azurerm_application_gateway" "app_gateway" {
provider = azurerm.generic
name = "${local.service_name}-app-gateway"
resource_group_name = azurerm_resource_group.rg.name
location = azurerm_resource_group.rg.location
enable_http2 = true
sku {
name = "Standard_Small"
tier = "Standard" # v1
capacity = 2
}
gateway_ip_configuration {
name = "${local.service_name}-ip-config"
subnet_id = azurerm_subnet.front_subnet.id
}
frontend_port {
name = local.frontend_port_name
port = 80
}
frontend_port {
name = local.backend_port_name
port = 443
}
frontend_ip_configuration {
name = local.frontend_ip_configuration_name
public_ip_address_id = azurerm_public_ip.pub_ip.id
}
backend_address_pool {
name = "${azurerm_virtual_network.vn.name}-beap"
fqdns = [local.host_name]
}
backend_http_settings {
name = local.http_setting_name
cookie_based_affinity = "Disabled"
port = 443
protocol = "Https"
request_timeout = 60
host_name = local.host_name
}
http_listener {
name = "${local.listener_name}-http"
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.frontend_port_name
protocol = "Http"
}
http_listener {
name = "${local.listener_name}-https"
frontend_ip_configuration_name = local.frontend_ip_configuration_name
frontend_port_name = local.backend_port_name
protocol = "Https"
}
request_routing_rule {
name = "${local.request_routing_rule_name}-http"
rule_type = "Basic"
http_listener_name = "${local.listener_name}-http"
backend_address_pool_name = local.backend_address_pool_name
backend_http_settings_name = local.http_setting_name
}
redirect_configuration {
name = local.redirect_configuration_name
redirect_type = "Permanent"
include_path = false
include_query_string = false
target_listener_name = "${local.listener_name}-https"
}
request_routing_rule {
name = "${local.request_routing_rule_name}-https"
rule_type = "Basic"
http_listener_name = "${local.listener_name}-https"
redirect_configuration_name = local.redirect_configuration_name
}
lifecycle {
ignore_changes = [
backend_address_pool,
backend_http_settings,
frontend_port,
http_listener,
request_routing_rule,
ssl_certificate,
redirect_configuration
]
}
}
resource "azurerm_dns_zone" "zone" {
provider = azurerm.generic
for_each = toset(local.dns_zone_names)
name = each.key
resource_group_name = azurerm_resource_group.rg.name
}
resource "azurerm_app_service_plan" "service_plan" {
provider = azurerm.generic
name = "${local.service_name}-service-plan"
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
sku {
tier = "Basic"
size = "B1"
}
}
resource "azurerm_app_service" "service" {
provider = azurerm.generic
name = "${local.service_name}-service"
app_service_plan_id = azurerm_app_service_plan.service_plan.id
location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name
}
resource "azurerm_app_service_custom_hostname_binding" "service_host_bind" {
provider = azurerm.generic
count = length(local.dns_zone_names)
hostname = "${local.dns_zone_names[count.index]}"
app_service_name = azurerm_app_service.service.name
resource_group_name = azurerm_resource_group.rg.name
lifecycle {
ignore_changes = [ssl_state, thumbprint]
}
depends_on = [
azurerm_app_service.service,
azurerm_resource_group.rg
]
}
resource "azurerm_app_service_managed_certificate" "service_manage_cert" {
provider = azurerm.generic
count = length(local.dns_zone_names)
custom_hostname_binding_id = azurerm_app_service_custom_hostname_binding.service_host_bind[count.index].id
}
resource "azurerm_app_service_certificate_binding" "service_certi_bind" {
provider = azurerm.generic
count = length(local.dns_zone_names)
hostname_binding_id = azurerm_app_service_custom_hostname_binding.service_host_bind[count.index].id
certificate_id = azurerm_app_service_managed_certificate.service_manage_cert[count.index].id
ssl_state = "SniEnabled"
}
i want a service that simply directs to another website through dns using terraform, and if there is any other way, please let us know. (include http to https)
To protect and prevent website abuse, we would like to redirect multiple domains to one website.
ex : (adomain.net -> www.target.com, adomain.tv -> www.target.com, bdomain.net -> www.target.com)
Fist of all there is no support for app services managed certificate with application gateway as of now.
Yes, you can do redirection from multiple domains to one domain using system.webserver rewrite rule either inside app services web.config file or application gateway rewrite rule.