I´m having little luck trying to send a PUT request with JSON containing an array of objects to my GRPC Server using REST. Using GRPC however it accepts an array just like expected. This is what I have defined in my proto file:
message UpdateRequest {
repeated Data data = 1;
int32 Id = 2;
}
message UpdateResponse {
}
message Data {
int32 id = 1;
string name = 2;
}
rpc Update(UpdateRequest) returns (UpdateResponse) {
option (google.api.http) = {
put: "/v1/data/{Id}"
body: "*"
};
}
This deploys successfully to GCP Endpoints but according to the GCP enpointsportal the request body is supposed to only contain a single object like:
{
"data": {
}
}
instead of an array of objects like expected:
{
"data": [
{},
{}
]
}
I´ve tried with replacing the "*" in the body with "data"
rpc Update(UpdateRequest) returns (UpdateResponse) {
option (google.api.http) = {
put: "/v1/data/{Id}"
body: "data"
};
}
This also compiles, but fails when trying to deploy to GCP endpoints with the following message:
kind: ERROR
message: "http: body field path 'data' must be a non-repeated message."
Any suggestions as to how I should go about solving this would be greatly appreciated.
Update:
Heres the contents of my .yaml file.
type: google.api.Service
config_version: 3
name: xxx.xxx-xxx.dev
title: xxxx
apis:
- name: x.x
- name: x.y
backend:
rules:
- selector: "*"
address: grpcs://xxx-xxx-app-xxxx-lz.a.run.app
This is a known issue, according to GCP support.
Here is the google issuetracker link: https://issuetracker.google.com/issues/178486575
There seems that this is a bug in GCP endpoints portal. I´m now successfully sending update requests with arrays containing object through CURL and my frontend application, although this does not work through endpoints.
Related
I'm trying to deploy PostgreSQL managed service with bicep and in most cases get an error:
"code": "InvalidParameterValue",
"message": "Invalid value given for parameter databaseName. Specify a valid parameter value."
I've tried various names for the DB, even in last version of the script I add random suffix to made it unique. Anyway it finishes with error, but looks like service is working. Another unexplainable thing is that sometimes script finishes without error... It's part of my IaC scenario, i need to be able to rerun it many times...
bicep code:
param location string
#secure()
param sqlserverLoginPassword string
param rand string = uniqueString(resourceGroup().id) // Generate unique String
param sqlserverName string = toLower('invivopsql-${rand}')
param sqlserverAdminName string = 'invivoadmin'
param psqlDatabaseName string = 'postgres'
resource flexibleServer 'Microsoft.DBforPostgreSQL/flexibleServers#2021-06-01' = {
name: sqlserverName
location: location
sku: {
name: 'Standard_B1ms'
tier: 'Burstable'
}
properties: {
createMode: 'Default'
version: '13'
administratorLogin: sqlserverAdminName
administratorLoginPassword: sqlserverLoginPassword
availabilityZone: '1'
storage: {
storageSizeGB: 32
}
backup: {
backupRetentionDays: 7
geoRedundantBackup: 'Disabled'
}
}
}
Please follow this git issue here for a similar error that might help you to fix your problem.
I've an Azure Key Vault(KV) that has shared secrets and a cert that needs to be pulled into different deployments.
E.g. DEV, TEST, UAT, Production all have their own key vaults BUT need access to the shared KV for wild card ssl cert.
I've tried a number of approaches but each has errors. I'm doing something similar for KV within the deployment resource group without issues
Is it possible to have this and then use it as a module? Something like this...
sharedKV.bicep
var kvResourceGroup = 'project-shared-rg'
var subscriptionId = subscription().id
var name = 'project-shared-kv'
resource project_shared_kv 'Microsoft.KeyVault/vaults#2021-06-01-preview' existing = {
name: name
scope: resourceGroup(subscriptionId, kvResourceGroup )
}
And then uses like:
template.bicep
module shared_kv './sharedKeyVault/template.bicep' = {
name: 'sharedKeyVault'
}
resource add_secrect 'Microsoft.KeyVault/vaults/secrets#2021-06-01-preview' = {
name: '${shared_kv.name}/mySecretKey'
properties: {
contentType: 'string'
value: 'secretValue'
attributes: {
enabled: true
}
}
}
If you need to target a different resourceGroup (and/or sub) than the rest of the deployment, the module's scope property needs to target that RG/sub. e.g.
module shared_kv './sharedKeyVault/template.bicep' = {
scope: resourceGroup(kvSubscription, kvResourceGroupName)
name: 'sharedKeyVault'
params: {
subId: kvSubscription
rg: kvResourceGroupName
...
}
}
Ideally, the sub/rg for the KV would be passed in to the module rather than hardcoded (which you probably knew, but just in case...)
I'm using the terraform kubernetes-provider and I'd like to translate something like this kubectl command into TF:
kubectl create secret generic my-secret --from-file mysecret.json
It seems, however the secret resource's data field expects only a TF map.
I've tried something like
data "template_file" "my-secret" {
template = "${file("${path.module}/my-secret.json")}"
}
resource "kubernetes_secret" "sgw-config" {
metadata {
name = "my-secret"
}
type = "Opaque"
data = "{data.template_file.my-secret.template}"
}
But it complains that this is not a map. So, I can do something like this:
data = {
"my-secret.json" = "{data.template_file.my-secret.template}"
}
But this will write the secret with a top-level field named my-secret.json and when I volume mount it, it won't work with other resources.
What is the trick here?
as long the file is UTF-8 encoded you can use something like this
resource "kubernetes_secret" "some-secret" {
metadata {
name = "some-secret"
namespace = kubernetes_namespace.some-ns.metadata.0.name
labels = {
"sensitive" = "true"
"app" = "my-app"
}
}
data = {
"file.txt" = file("${path.cwd}/your/relative/path/to/file.txt")
}
}
If the file is a binary one you will have an error like
Call to function "file" failed: contents of
/your/relative/path/to/file.txt are not valid UTF-8; use the
filebase64 function to obtain the Base64 encoded contents or the other
file functions (e.g. filemd5, filesha256) to obtain file hashing
results instead.
I tried encoding the file in base64 but then the problem is that the resulting text will be re-encoded in base64 by the provider. So I guess there is no solution for binary files at the moment...
I'll edit with what I find next for binaries.
This might be a bit off-topic, but I've been facing similar problem except that the file might not be present in which case the terraform [plan|apply] fails.
To be exact: I needed to duplicate a secret from one namespace to another one.
I realized that by using hashicorp/external provider.
The steps are pretty simple:
Load data by calling external program
Refer to the data in kubernetes_secret resource
The program should accept (and process) JSON on STDIN and produce valid JSON on STDOUT as response to the parameters passed-in in the STDIN's JSON.
Example shell script:
#!/bin/bash
set -e
/bin/echo -n '{ "token": "'
kubectl get -n consul secrets/hashicorp-consul-bootstrap-acl-token --template={{.data.token}}
/bin/echo -n '"}'
tarraform source:
data "external" "token" {
program = ["sh", "${path.module}/consul-token.sh"]
}
resource "kubernetes_secret" "consul-token" {
depends_on = [data.external.token]
metadata {
name = "consul-token"
namespace = "app"
}
data = {
token = base64decode(data.external.token.result.token)
}
}
and requirements:
terraform {
required_providers {
external = {
source = "hashicorp/external"
version = ">= 2.0.0"
}
}
}
Just use
https://www.terraform.io/docs/providers/kubernetes/r/config_map.html#binary_data
resource "kubernetes_config_map" "example" {
metadata {
name = "my-config"
}
binary_data = {
"my_payload.bin" = "${filebase64("${path.module}/my_payload.bin")}"
}
}
I believe you can use binary_data attribute in the secret now.
e.g.
binary_data = {
"my_payload.bin" = "${filebase64("${path.module}/my_payload.bin")}"
}
reference:
https://github.com/hashicorp/terraform-provider-kubernetes/pull/1228
https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/secret#binary_data
Basically you need to provide a map like this :
resource "kubernetes_secret" "sgw-config" {
metadata {
name = "my-secret"
}
type = "Opaque"
data {
"key1" = "value1"
"key2" = "value2"
}
}
you can refer to your internal variables using
resource "kubernetes_secret" "sgw-config" {
metadata {
name = "my-secret"
}
type = "Opaque"
data {
"USERNAME" = "${var.some_variable}"
"PASSWORD" = "${random_string.root_password.result}"
}
}
It seems if you run the command kubectl create secret generic my-secret --from-file mysecret.json
and then
$ kubectl get secrets my-secret -o yaml
apiVersion: v1
data:
my-secret.json: ewogICA.....
kind: Secret
metadata:
creationTimestamp: "2019-03-25T18:20:43Z"
name: my-secret
namespace: default
resourceVersion: "67026"
selfLink: /api/v1/namespaces/default/secrets/my-secret
uid: b397a29c-4f2a-11e9-9806-000c290425d0
type: Opaque
it stores it similarly with the filename as the single key. When I mount this in a volume/volumeMount it works as expected. I was afraid that it wouldn't but when I create the secret using the --from-file argument, this is exactly how it stores it.
I was trying to create some basic inspec tests to validate a set of HTTP URLs. The way I started is like this -
control 'http-url-checks' do
impact 1.0
title 'http-url-checks'
desc '
Specify the URLs which need to be up and working.
'
tag 'http-url-checks'
describe http('http://example.com') do
its('status') { should eq 200 }
its('body') { should match /abc/ }
its('headers.name') { should eq 'header' }
end
describe http('http://example.net') do
its('status') { should eq 200 }
its('body') { should match /abc/ }
its('headers.name') { should eq 'header' }
end
end
We notice that the URLs are hard-coded in the controls and isn't a lot of fun. I'd like to move them to some 'attributes' file of some sort and loop through them in the control file.
My attempt was to use the 'files' folder structure inside the profile.I created a file - httpurls.yml and had the following content in it -
- url: http://example.com
- url: http://example.net
..and in my control file, I had the construct -
my_urls = yaml(content: inspec.profile.file('httpurls.yml')).params
my_urls.each do |s|
describe http(s['url']) do
its('status') { should eq 200 }
end
end
However, when I execute the compliance profile, I get an error - 'httpurls.yml not found' (not sure about the exact error message though though). The following is the folder structure I had for my compliance profile.
What I am doing wrong?
Is there a better way to achieve what I am trying to do?
The secret is to use profile attributes, as defined near the bottom of this page:
https://www.inspec.io/docs/reference/profiles/
First, create a profile attributes YML file. I name mine profile-attribute.yml.
Second, put your array of values in the YML file, like so:
urls:
- http://example.com
- http://example.net
Third, create an attribute at the top of your InSpec tests:
my_urls = attribute('urls', description: 'The URLs that I am validating.')
Fourth, use your attribute in your InSpec test:
my_urls.each do |s|
describe http(s['url']) do
its('status') { should eq 200 }
end
end
Finally, when you call your InSpec test, point to your YML file using --attrs:
inspec exec mytest.rb --reporter=cli --attrs profile-attribute.yml
There is another way to do this using files (instead of the profile attributes and the --attrs flag). You can use JSON or YAML.
First, create the JSON and/or YAML file and put them in the files directory. A simple example of the JSON file might look like this:
{
"urls": ["https://www.google.com", "https://www.apple.com"]
}
And a simple example of the YAML file might look like this:
urls:
- https://www.google.com
- https://www.apple.com
Second, include code at the top of your InSpec file to read and parse the JSON and/or YAML, like so:
jsoncontent = inspec.profile.file("tmp.json")
jsonparams = JSON.parse(jsoncontent)
jsonurls = jsonparams['urls']
yamlcontent = inspec.profile.file("tmp.yaml")
yamlparams = YAML.load(yamlcontent)
yamlurls = yamlparams['urls']
Third, use the variables in your InSpec tests, like so:
jsonurls.each do |jsonurl|
describe http(jsonurl) do
puts "json url is " + jsonurl
its('status') { should eq 200 }
end
end
yamlurls.each do |yamlurl|
describe http(yamlurl) do
puts "yaml url is " + yamlurl
its('status') { should eq 200 }
end
end
(NOTE: the puts line is for debugging.)
The result is what you would expect:
json url is https://www.google.com
json url is https://www.apple.com
yaml url is https://www.google.com
yaml url is https://www.apple.com
Profile: InSpec Profile (inspec-file-test)
Version: 0.1.0
Target: local://
http GET on https://www.google.com
✔ status should eq 200
http GET on https://www.apple.com
✔ status should eq 200
http GET on https://www.google.com
✔ status should eq 200
http GET on https://www.apple.com
✔ status should eq 200
I get the following error on ApiaryWe are sorry, but the API call failed.
My host is defined as
FORMAT: 1A
HOST: https://test.mynetwork.com/
GET CALL IS DEFINED AS
data urn [models/v2/files/{urn}/data{?guid}]
GET [GET]
Parameters
urn (required, string, ttt)...design urn.
guid (optional, string,067e6162-3b6f-4ae2-a171-2470b63dfe02)...filter by guid.
Response 200 (application/vnd.api+json)
Body
data
{
"version": "1.0",
}
When i invoke this , i get error . Any inputs
I have edited your API Blueprint as follows:
FORMAT: 1A
HOST: https://test.mynetwork.com
# Test API
This is a test API
## data urn [/models/v2/files/{urn}/data{?guid}]
+ Parameters
+ urn: `ttt` (required, string) - design urn
+ guid: `067e6162-3b6f-4ae2-a171-2470b63dfe02` (optional, string) - filter by guid
### test [GET]
+ Response 200 (application/vnd.api+json)
{
"version": "1.0"
}
You can find tutorials here: https://help.apiary.io/
Also not sure what you mean "invoke" the API - are you calling the mock server or are you hitting your own server through Apiary.
Happy to help further via the support option in Apiary itself or email us on support [at] apiary.io.