The value supplied for parameter 'instanceProfileName' is not valid - aws-cloudformation

Running cdk deploy I receive the following error message:
CREATE_FAILED | AWS::ImageBuilder::InfrastructureConfiguration | TestInfrastructureConfiguration The value supplied for parameter 'instanceProfileName' is not valid. The provided instance profile does not exist. Please specify a different instance profile and try again. (Service: Imagebuilder, Status Code: 400, Request ID: 41f431d7-8544-48e9-9faf-a870b83b0100, Extended Request ID: null)
The C# code looks like this:
var instanceProfile = new CfnInstanceProfile(this, "TestInstanceProfile", new CfnInstanceProfileProps {
InstanceProfileName = "test-instance-profile",
Roles = new string[] { "TestServiceRoleForImageBuilder" }
});
var infrastructureConfiguration = new CfnInfrastructureConfiguration(this, "TestInfrastructureConfiguration", new CfnInfrastructureConfigurationProps {
Name = "test-infrastructure-configuration",
InstanceProfileName = instanceProfile.InstanceProfileName,
InstanceTypes = new string[] { "t2.medium" },
Logging = new CfnInfrastructureConfiguration.LoggingProperty {
S3Logs = new CfnInfrastructureConfiguration.S3LogsProperty {
S3BucketName = "s3-test-assets",
S3KeyPrefix = "ImageBuilder/Logs"
}
},
SubnetId = "subnet-12f3456f",
SecurityGroupIds = new string[] { "sg-12b3e4e5b67f8900f" }
});
The TestServiceRoleForImageBuilder exists and was working previously. Same code was running successfully about a month ago. Any suggestions?
If I remove the CfninfrastructureConfiguration creation part, deployment runs successfully:, but takes at least 2 minutes to complete.
AwsImageBuilderStack: deploying...
AwsImageBuilderStack: creating CloudFormation changeset...
0/3 | 14:24:37 | REVIEW_IN_PROGRESS | AWS::CloudFormation::Stack | AwsImageBuilderStack User Initiated
0/3 | 14:24:43 | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | AwsImageBuilderStack User Initiated
0/3 | 14:24:47 | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata)
0/3 | 14:24:47 | CREATE_IN_PROGRESS | AWS::IAM::InstanceProfile | TestInstanceProfile
0/3 | 14:24:47 | CREATE_IN_PROGRESS | AWS::IAM::InstanceProfile | TestInstanceProfile Resource creation Initiated
1/3 | 14:24:48 | CREATE_IN_PROGRESS | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata) Resource creation Initiated
1/3 | 14:24:48 | CREATE_COMPLETE | AWS::CDK::Metadata | CDKMetadata/Default (CDKMetadata)
1/3 Currently in progress: AwsImageBuilderStack, TestInstanceProfile
3/3 | 14:26:48 | CREATE_COMPLETE | AWS::IAM::InstanceProfile | TestInstanceProfile
3/3 | 14:26:49 | CREATE_COMPLETE | AWS::CloudFormation::Stack | AwsImageBuilderStack
Is it probably some race condition? Should I use multiple stacks to achieve my goal?
Should it be possible to use a wait condition (AWS::CloudFormation::WaitCondition) to bypass the 2 minutes of creation time in case it is intended (AWS::IAM::InstanceProfile resources always take exactly 2 minutes to create)?
Environment
CDK CLI Version: 1.73.0
Node.js Version: 14.13.0
OS: Windows 10
Language (Version): C# (.NET Core 3.1)
Update
Since the cause seems to be AWS internal, I used a pre-created instance profile as a workaround. The profile can be either created through IAM Management Console or CLI. However it would be nice to have a proper solution.

You have to create a dependency between the two constructs. CDK does not infer it when using the optional name parameter, as opposed to the logical id (which doesn't seem to work in this situation).
infrastructureConfiguration.node.addDependency(instanceProfile)
Here are the relevant docs: https://docs.aws.amazon.com/cdk/api/latest/docs/core-readme.html#construct-dependencies

Related

Openstack API not providing precise data

I am using Openstack - Stein in CentOS 7.9
I was using python to collect data about the openstack nova performance, like server names and id in the openstack project, i have 3 instance(server) created, i can see all three instance in openstack cli, but when i connect to api mentioned in openstack, it provides no data or less data.
I refereed openstack documentation here
[root#centos-vm1 kavin(keystone_admin)]# openstack server list
+--------------------------------------+-----------------+--------+----------------------------------------+-------+----------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+-----------------+--------+----------------------------------------+-------+----------+
| 08cf6226-0303-4b4c-ba53-10af79b81dae | test_instance_3 | ACTIVE | test_networ_3=10.150.0.8 | | m1.tiny |
| 9986f205-82b3-4cbb-bcdc-fb32eab97c83 | test_instance_1 | ACTIVE | test_networ_2=10.100.0.5, x.x.x.x | | m1.small |
| d1c0f520-8540-432c-8fe1-554390fd79bf | test_instance_2 | ACTIVE | test_networ_1=10.50.0.8 | | m1.small |
+--------------------------------------+-----------------+--------+----------------------------------------+-------+----------+
My python code:
import requests,json
from six.moves.urllib.parse import urljoin
identity = {
"methods": ["password"],
"password": {
"user": {
"name": "admin",
"domain": { "id": "default" },
"password": "xxxxxxxxxxxxxxx"
}
}
}
OS_AUTH_URL = 'http://x.x.x.x:5000/v3'
data = {'auth': {'identity': identity}}
HEADERS = {'Content-Type': 'application/json', 'scope': 'unscoped'}
r = requests.post(
OS_AUTH_URL+'/auth/tokens',
headers = HEADERS,
json = data,
verify = False
)
auth_token = r.headers['X-Subject-Token'] # i got auth token
# server list
NOVA_URL="http://x.x.x.x:8774/v2.1"
HEADERS = {"X-Auth-Token" : str(auth_token)}
r = requests.get(
NOVA_URL+'/servers',
headers = HEADERS,
)
r.raise_for_status()
print(r.json())
Output :
{'servers': []}
help me, collect accurate data using api calls, thanks
According to api-ref List Servers doc, maybe you should add the project scope in the request.
By default the servers are filtered using the project ID associated with the authenticated request.
In my opinion, you could use openstacksdk to execute the operation, simply with the Connection object and list_servers method.
import openstack
conn = openstack.connect(
region_name='example-region',
auth_url='http://x.x.x.x:5000/v3/',
username='amazing-user',
password='super-secret-password',
project_id='33...b5',
domain_id='05...03'
)
servers = conn.list_servers()

How can I solve a SCRAM authentication issue in Mongodb with Docker and Prisma?

When I try to run any command from the prisma I get this error message:
$ npx prisma migrate reset
Error: MongoDB error
SCRAM failure: Authentication failed.
0: migration_core::state::Reset
at migration-engine/core/src/state.rs:341
When I run docker ps:
CONTAINER ID | IMAGE | COMMAND | CREATED | STATUS | PORTS | NAMES
388a7219da3d | mongo:latest | "docker-entrypoint.s…" | About an hour ago | Up About an hour | 0.0.0.0:27017->27017/tcp, :::27017->27017/tcp | auction-mongodb
my schema.prisma:
// This is your Prisma schema file,
// learn more about it in the docs: https://pris.ly/d/prisma-schema
generator client {
provider = "prisma-client-js"
previewFeatures = ["mongodb"]
}
datasource db {
provider = "mongodb"
url = env("DATABASE_URL")
}
model Test {
id String #id #default(auto()) #map("_id") #db.ObjectId
text String
}
.env:
DATABASE_URL=mongodb://auction-user:password#localhost:27017/auction-db?authSource=admin
Simply add ?authSource=admin to the end of your DATABASE_URL in .env file.
in my case this code solved.
user and pw suppose to have root role.
DATABASE_URL="mongodb://root:example#localhost:3333/liz?retryWrites=true&w=majority&authSource=admin"
DATABASE_URL="mongodb://username:password#ip:port/db?retryWrites=true&w=majority&authSource=admin"
do remove +rsv if you want to connect to your local mongodb

Terraform: Create multiple Kubernetes manifests using templates

I have a requirement to execute multiple Kubernetes manifests of the same kind. Just names, action, and port changes
my networkpolicy.yaml.tpl
${yamlencode(
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
name: ${name}-policy
spec:
action: ${action}
rules:
- to:
- operation:
ports: [for port in ports : "${port}"]
)}
values that need to be populated for each microservice
| -------- | -------- | --------------| --------------|
| name | frontend | backend | middleware |
| action | allow | allow | allow |
| ports | 8080,443 | 4731 | 8751,7542 |
Example networkpolicy.yaml after generation
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
name: frontend-policy
spec:
action: allow
rules:
- to:
- operation:
ports: ["8080", "443"]
how can I achieve this? I am not clear as how-to write main.tf
resource "kubernetes_manifest" "istio-config" {
manifest = yamldecode(templatefile("${path.module}/networkpolicy.yaml.tpl", {
name =
action =
port =
}))
}
What you are looking for is for_each. You can declare a local values block and then loop over that block, replacing what you need. f.e.:
locals {
services = {
frontend = {
action = "allow"
ports = [8080,433]
}
backend = {
...
}
middleware = {
...
}
}
}
resource "kubernetes_manifest" "istio-config" {
for_each = local.services
manifest = yamldecode(templatefile("${path.module}/networkpolicy.yaml.tpl", each.value)
}
You can use the loop
Example
menifest = yamldecode(templatefile("${path.module}/networkpolicy.yaml.tpl", {
name = var.name
})
update the networkpolicy.yaml.tpl file
%{ for s in nameservers ~}
name ${s}
%{ endfor ~}
If you don't want to edit the tpl file you can edit the main.tf directly
name = <<-EOT
%{ for s in var.name ~}
name ${s}
%{ endfor ~}
EOT
Ref : https://www.terraform.io/language/functions/templatefile
Read more about simple loop : https://blog.gruntwork.io/terraform-tips-tricks-loops-if-statements-and-gotchas-f739bbae55f9
You can use the count or for_each loop to simple make it possible.

Cygnus-NGSI won't save data in PostgreSQL

Im trying to save some data in a postgreSQL database using cygnus-ngsi, but nothing happens. Im running all services in a docker container using docker-compose.
docker-compose.yml:
...
cygnus:
image: fiware/cygnus-ngsi:latest
hostname: cygnus
container_name: cygnus_fiware
volumes:
- ./config/cygnus/cygnus.conf:/opt/apache-flume/conf/agent.conf
- ./config/cygnus/grouping_rules.conf:/opt/apache-flume/conf/grouping_rules.conf
links:
- orion
- postgres
expose:
- "5050"
ports:
- "5050:5050"
postgres:
restart: always
image: postgres:latest
container_name: postgres_fiware
volumes:
- ./data/db/postgres:/var/lib/postgresql/data
ports:
- "5432:5432"
expose:
- "5432"
environment:
- POSTGRES_USER=teste
- POSTGRES_DB=newdb
- POSTGRES_PASSWORD=123456789
agent.conf
cygnus-ngsi.sources = http-source
cygnus-ngsi.sinks = postgresql-sink
cygnus-ngsi.channels = postgresql-channel
#=============================================
# source configuration
# channel name where to write the notification events
cygnus-ngsi.sources.http-source.channels = postgresql-channel
# source class, must not be changed
cygnus-ngsi.sources.http-source.type = org.apache.flume.source.http.HTTPSource
# listening port the Flume source will use for receiving incoming notifications
cygnus-ngsi.sources.http-source.port = 5050
# Flume handler that will parse the notifications, must not be changed
cygnus-ngsi.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.NGSIRestHandler
# URL target
cygnus-ngsi.sources.http-source.handler.notification_target = /notify
# default service (service semantic depends on the persistence sink)
cygnus-ngsi.sources.http-source.handler.default_service = default
# default service path (service path semantic depends on the persistence sink)
cygnus-ngsi.sources.http-source.handler.default_service_path = /
# source interceptors, do not change
cygnus-ngsi.sources.http-source.interceptors = ts gi
# TimestampInterceptor, do not change
cygnus-ngsi.sources.http-source.interceptors.ts.type = timestamp
# GroupingInterceptor, do not change
cygnus-ngsi.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor$Builder
# Grouping rules for the GroupingInterceptor, put the right absolute path to the file if necessary
# see the doc/design/interceptors document for more details
cygnus-ngsi.sources.http-source.interceptors.gi.grouping_rules_conf_file = /opt/apache-flume/conf/grouping_rules.conf
# ============================================
# NGSIPostgreSQLSink configuration
# channel name from where to read notification events
cygnus-ngsi.sinks.postgresql-sink.channel = postgresql-channel
# sink class, must not be changed
cygnus-ngsi.sinks.postgresql-sink.type = com.telefonica.iot.cygnus.sinks.NGSIPostgreSQLSink
# true applies the new encoding, false applies the old encoding.
cygnus-ngsi.sinks.postgresql-sink.enable_encoding = false
# true if name mappings are enabled for this sink, false otherwise
cygnus-ngsi.sinks.postgresql-sink.enable_name_mappings = false
# true if the grouping feature is enabled for this sink, false otherwise
cygnus-ngsi.sinks.postgresql-sink.enable_grouping = false
# true if lower case is wanted to forced in all the element names, false otherwise
cygnus-ngsi.sinks.postgresql-sink.enable_lowercase = false
# the FQDN/IP address where the PostgreSQL server runs
cygnus-ngsi.sinks.postgresql-sink.postgresql_host = postgres
# the port where the PostgreSQL server listens for incomming connections
cygnus-ngsi.sinks.postgresql-sink.postgresql_port = 5432
# the name of the postgresql database
cygnus-ngsi.sinks.postgresql-sink.postgresql_database = newdb
# a valid user in the PostgreSQL server
cygnus-ngsi.sinks.postgresql-sink.postgresql_username = teste
# password for the user above
cygnus-ngsi.sinks.postgresql-sink.postgresql_password = 123456789
# how the attributes are stored, either per row either per column (row, column)
cygnus-ngsi.sinks.postgresql-sink.attr_persistence = row
# select the data_model: dm-by-service-path or dm-by-entity
cygnus-ngsi.sinks.postgresql-sink.data_model = dm-by-entity
# number of notifications to be included within a processing batch
cygnus-ngsi.sinks.postgresql-sink.batch_size = 100
# timeout for batch accumulation
cygnus-ngsi.sinks.postgresql-sink.batch_timeout = 30
# number of retries upon persistence error
cygnus-ngsi.sinks.postgresql-sink.batch_ttl = 10
# =============================================
# postgresql-channel configuration
# channel type (must not be changed)
cygnus-ngsi.channels.postgresql-channel.type = memory
# capacity of the channel
cygnus-ngsi.channels.postgresql-channel.capacity = 1000
# amount of bytes that can be sent per transaction
cygnus-ngsi.channels.postgresql-channel.transactionCapacity = 100
Messages from docker-compose:
cygnus_fiware | time=2017-06-19T15:11:15.132Z | lvl=WARN | corr=4af15a0c-5501-11e7-aa0a-0242ac130004 | trans=508576db-1443-4c64-bfc9-629d1a0b250e | srv=espometeo | subsrv=/environment | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[257] : [NGSIRestHandler] Unnecessary header
cygnus_fiware | time=2017-06-19T15:11:15.133Z | lvl=INFO | corr=89ac4c66-5501-11e7-850f-0242ac130004 | trans=3ae0dc99-de51-49a3-937f-42d887b7e7d7 | srv=espometeo | subsrv=/environment | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[282] : [NGSIRestHandler] Starting internal transaction (3ae0dc99-de51-49a3-937f-42d887b7e7d7)
cygnus_fiware | time=2017-06-19T15:11:15.133Z | lvl=INFO | corr=89ac4c66-5501-11e7-850f-0242ac130004 | trans=3ae0dc99-de51-49a3-937f-42d887b7e7d7 | srv=espometeo | subsrv=/environment | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[299] : [NGSIRestHandler] Received data ({ "subscriptionId" : "5947d328e143997a02b11008", "originator" : "localhost", "contextResponses" : [ { "contextElement" : { "type" : "EstacaoMeteo", "isPattern" : "false", "id" : "Estacao3", "attributes" : [ { "name" : "Humidity", "type" : "float", "value" : "35.3", "metadatas" : [ { "name" : "TimeInstant", "type" : "ISO8601", "value" : "2017-06-24T13:03:00" } ] }, { "name" : "Temperature", "type" : "float", "value" : "15.2", "metadatas" : [ { "name" : "TimeInstant", "type" : "ISO8601", "value" : "2017-06-24T13:03:00" } ] } ] }, "statusCode" : { "code" : "200", "reasonPhrase" : "OK" } } ]})
cygnus_fiware | time=2017-06-19T15:11:36.141Z | lvl=INFO | corr=89ac4c66-5501-11e7-850f-0242ac130004 | trans=3ae0dc99-de51-49a3-937f-42d887b7e7d7 | srv=espometeo | subsrv=/environment | comp=cygnus-ngsi | op=persistAggregation | msg=com.telefonica.iot.cygnus.sinks.NGSIPostgreSQLSink[479] : [postgresql-sink] Persisting data at NGSIPostgreSQLSink. Schema (espometeo), Table (environment_estacao3_estacaometeo), Fields ((recvTimeTs,recvTime,fiwareServicePath,entityId,entityType,attrName,attrType,attrValue,attrMd)), Values (('1497885075139','2017-06-19T15:11:15.139Z','/environment','Estacao3','EstacaoMeteo','Humidity','float','35.3','[{"name":"TimeInstant","type":"ISO8601","value":"2017-06-24T13:03:00"}]'),('1497885075139','2017-06-19T15:11:15.139Z','/environment','Estacao3','EstacaoMeteo','Temperature','float','15.2','[{"name":"TimeInstant","type":"ISO8601","value":"2017-06-24T13:03:00"}]'))
cygnus_fiware | time=2017-06-19T15:11:36.142Z | lvl=WARN | corr=89ac4c66-5501-11e7-850f-0242ac130004 | trans=3ae0dc99-de51-49a3-937f-42d887b7e7d7 | srv=espometeo | subsrv=/environment | comp=cygnus-ngsi | op=processNewBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[541] :
Seems like cygnus is getting all the data from the Orion and putting it right, but when i go to the postgresql db, there nothing. Some one already have this problem?
Persistance ERROR message:
cygnus_fiware | time=2017-06-22T09:45:06.092Z | lvl=ERROR | corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=processRollbackedBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[398] : CygnusPersistenceError. -, null. Stack trace: [com.telefonica.iot.cygnus.sinks.NGSIPostgreSQLSink.persistAggregation(NGSIPostgreSQLSink.java:504), com.telefonica.iot.cygnus.sinks.NGSIPostgreSQLSink.persistBatch(NGSIPostgreSQLSink.java:231), com.telefonica.iot.cygnus.sinks.NGSISink.processRollbackedBatches(NGSISink.java:390), com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:372), org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68), org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147), java.lang.Thread.run(Thread.java:748)]
Cygnus start ERRORS:
cygnus_fiware | + exec /usr/lib/jvm/java-6-sun/bin/java -Xms512m -Xmx1g -Dcom.sun.management.jmxremote -Dflume.root.logger=INFO,console -Duser.timezone=UTC -Dfile.encoding=UTF-8 -cp '/opt/apache-flume/conf:/opt/apache-flume/lib/*:/opt/apache-flume/plugins.d/cygnus/lib/*:/opt/apache-flume/plugins.d/cygnus/libext/*' -Djava.library.path= com.telefonica.iot.cygnus.nodes.CygnusApplication -f /opt/apache-flume/conf/agent.conf -n cygnus-ngsi -p 8081
cygnus_fiware | /opt/apache-flume/bin/cygnus-flume-ng: line 232: /usr/lib/jvm/java-6-sun/bin/java: No such file or directory
cygnus_fiware | /opt/apache-flume/bin/cygnus-flume-ng: line 232: exec: /usr/lib/jvm/java-6-sun/bin/java: cannot execute: No such file or directory
There is a problem with PostgreSQL sink when cache is not enabled. It is described at this issue.
I solved the issue by changing false to true in agent.conf file: cygnus-ngsi.sinks.postgresql-sink.backend.enable_cache = true.

Assertion over each item in collection in Pester

I am doing some infrastructure testing in Pester and there is repeating scenario that I don't know how to approach.
Let's say, I want to check whether all required web roles are enabled on IIS. I have a collection of required web roles and for each of them I want to assert it is enabled.
My current code looks like this:
$requiredRoles = #(
"Web-Default-Doc",
"Web-Dir-Browsing",
"Web-Http-Errors",
"Web-Static-Content",
"Web-Http-Redirect"
)
Context "WebRoles" {
It "Has installed proper web roles" {
$requiredRoles | % {
$feature = Get-WindowsOptionalFeature -FeatureName $_ -online
$feature.State | Should Be "Enabled"
}
}
}
It works in the sense that the test will fail if any of the roles are not enabled/installed. But that is hardly useful if the output of such Pester test looks like this:
Context WebRoles
[-] Has installed proper web roles 2.69s
Expected: {Enabled}
But was: {Disabled}
283: $feature.State | Should Be "Enabled"
This result doesn't give any clue about which feature is the Disabled one.
Is there any recommended practice in these scenarios? I was thinking about some string manipulation...
Context "WebRoles" {
It "Has installed proper web roles" {
$requiredRoles | % {
$feature = Get-WindowsOptionalFeature -FeatureName $_ -online
$toCompare = "{0}_{1}" -f $feature.FeatureName,$feature.State
$toCompare | Should Be ("{0}_{1}" -f $_,"Enabled")
}
}
}
which would output:
Context WebRoles
[-] Has installed proper web roles 2.39s
Expected string length 27 but was 28. Strings differ at index 20.
Expected: {IIS-DefaultDocument_Enabled}
But was: {IIS-DefaultDocument_Disabled}
-------------------------------^
284: $toCompare | Should Be ("{0}_{1}" -f $_,"Enabled")
...which is better, but it doesn't feel very good...
Also, there is second problem with the fact that the test will stop on first fail and I would need to re-run the test after I fix each feature...
Any ideas?
Put your It inside the loop like so:
Context "WebRoles" {
$requiredRole | ForEach-Object {
It "Has installed web role $_" {
(Get-WindowsOptionalFeature -FeatureName $_ -online).State | Should Be "Enabled"
}
}
}