I have this secret built like this:
apiVersion: v1
kind: Secret
metadata:
name: secrets
type: Opaque
stringData:
PORT: "3000"
MONGODB_HOSTNAME: hostname
MONGODB_USER: fares
MONGODB_PASSWORD: password
MONGODB_TEST_HOSTNAME: localhost
ENCODED_CREDENTIALS: ewog...
ENCODED_CREDENTIALS contains an encoded base64 JSON file. It's decoded in the app but I need it to remain encoded as a secret.
Now, I seal it using my cert and kubeseal, but ArgoCD displays this: ErrUnsealFailed - Failed to unseal: illegal base64 data at input byte 4852.
I feel like it's the fact that ENCODED_CREDENTIALS is already encoded that might be a problem, but I'm not sure.
Are you guys familiar with this kind of problems?
TIA
EDIT:
The decoding (pre-seal) works:
echo -n $test_thing | base64 -d
{
"type": "service_account",
"project_id": "blabla",
"private_key_id": "blabla",
"private_key": "-----BEGIN PRIVATE KEY-----
...
\n-----END PRIVATE KEY-----\n",
"client_email": "john.doe#gserviceaccount.com",
"client_id": "abcd1234",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/bucket-sa%40steve-jobs.iam.gserviceaccount.com"
}
It's only when it seals & unseals with the sealed secrets controller that it fails.
Good news!
I've managed to find where the issue is.
For some reason, adding an (intentionally) encoded value to stringData keys breaks the unsealing process.
For those who are curious, I've raised the bug (?) to the sealed secrets team: #1038
Related
I got a list in yml - credentials. And supposedly each bank has to have a different password that needs to be encrypted. What would be the right way to specify that? As of now I got it configured like this, but that doesn't work.
This is the config.yml
infopoint:
endpoint: https://test.test.com/ws/SSS/Somthing.pl
system: TEST
mock: false
credentials:
- bank: 1111
user: LSSER
existingSecret:
name: infopoint-creds-s1-hb
- bank: 2222
user: TESSER
existingSecret:
name: infopoint-creds-s1
envFrom:
- secretRef:
name: infopoint-creds-s1-hb
- secretRef:
name: infopoint-creds-s1
This is how I created both secret keys on the server.
C:\Users\mks\IdeaProjects>kubectl.exe create secret generic infopoint-creds-s1-hb --from-literal=INFOPOINT_CREDENTIALS_PASSWORD=SOMEPASS -o yaml -n test-env --dry-run=client | kubeseal -o yaml --scope namespace-wide > infopoint-creds-s1-hb.yaml
C:\Users\mks\IdeaProjects>kubectl.exe create secret generic infopoint-creds-s1 --from-literal=INFOPOINT_CREDENTIALS_PASSWORD=SOMEPASS -o yaml -n test-env --dry-run=client | kubeseal -o yaml --scope namespace-wide > infopoint-creds-s1.yaml
This is my Spring configuration.
#Configuration
#ConfigurationProperties(prefix = "infopoint")
class InfopointAPIConfiguration {
lateinit var endpoint: String
var proxyServerName: String? = null
var proxyPortNumber: String? = null
lateinit var system: String
lateinit var mock: String
lateinit var credentials: List<Credentials>
data class Credentials(
var bank: String? = null,
var user: String? = null,
var password: String? = null
)
fun credentialsByBank(bank: Int): Credentials {
return credentials.firstOrNull { it.bank == bank.toString() }
?: error("Could not load credential for bank $bank")
}
}
Kubernetes secrets can be used or configured in applications in multiple ways for example configmaps, sealed secrets and environment variables. Since you got struck with the sealed secrets part I am providing an answer related to the same.
First we need to create a sealed secret in the same namespace with the same name for preventing other users on the same cluster from using your sealed secret. For more information related to sealed secrets please go through this document.
Now we have our secret created, all we need to do is to use it in our application. The secret which we created needs to be referenced in the yaml file. There is a detailed description on how to configure secrets in a spring boot application along with a sample project available here.
I am trying to use ApiextensionsV1beta1Api to create a custom resource definition though kubernetes python client
with kubernetes.client.ApiClient(configuration) as api_client:
self.client_custom_resource_def = kubernetes.client.ApiextensionsV1beta1Api(api_client)
spec = {"group": "kgosalia.com", "scope": "Namespaced",
"metadata": {"name": "kgosaliaconfigs.kgosalia.com"},
"versions": [{"name": "v1", "served": True, "storage": True}],
"names": {"kind": "CustomResourceDefinition", "plural": "kgosaliaconfigs"}}
def create_custom_resource_definition(self, spec):
body = kubernetes.client.V1beta1CustomResourceDefinition(spec=spec)
try:
api_response = self.client_custom_resource_def.create_custom_resource_definition(body)
pprint(api_response)
except ApiException as e:
print("Exception when calling ApiextensionsV1Api->create_custom_resource_definition: %s\n" % e)
When I run this I am getting a 422. Can you help me find the correct format to create spec and name object?
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"CustomResourceDefinition.apiextensions.k8s.io \"\" is invalid: metadata.name: Required value: name or generateName is required","reason":"Invalid","details":{"group":"apiextensions.k8s.io","kind":"CustomResourceDefinition","causes":[{"reason":"FieldValueRequired","message":"Required value: name or generateName is required","field":"metadata.name"}]},"code":422}
Appreciate your help, thank you!
I'm generating a JWT using google-auth-library-nodejs by providing the credentials through env variables, similar to the sample code from here.
const credentials = JSON.parse(JSON.stringify(env.service_account.credentials));
const client = auth.fromJSON(credentials);
const url = 'my_url';
const token = await client.fetchIdToken(url);
When using that token later on for authentication, the request fails because of the following error:
JWT validation failed: Could not find matching key in public key set for kid=7da7843e8637d669bc2a12622cede2a8814d11b1
https://jwt.io/#debugger-io confirms that the kid of the generated JWT is indeed 7da7843e8637d669bc2a12622cede2a8814d11b1, however, the private key id in the json is 6bd5b5d36f9a4225bd814ff3d6909d95e23e0793.
Here's the json
'{
"type": "service_account",
"project_id": "project_id",
"private_key_id": "6bd5b5d36f9a4225bd814ff3d6909d95e23e0793",
"private_key": "-----BEGIN PRIVATE KEY-----\nXXXXXXXXXXX\n-----END PRIVATE KEY-----\n",
"client_email": "my_service_account#project_id.iam.gserviceaccount.com",
"client_id": "REDACTED",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/my_service_account%40project_id.iam.gserviceaccount.com"
}
Should the kid match the privat_key_id?
client_x509_cert_url provided in the question above was not the right x-google-jwks_uri. It actually has to be https://www.googleapis.com/oauth2/v1/certs.
After changing the URI, the kids match.
If anyone has a clue why that is, it would be much appreciated.
Does the following configuration for configMap create the test.json file of type empty array or string []
kind: ConfigMap
apiVersion: v1
metadata:
name: myconfigmap
data:
test.json: |-
[]
The convertor to JSON suggests string:
{
"kind": "ConfigMap",
"apiVersion": "v1",
"metadata": {
"name": "myconfigmap"
},
"data": {
"test.json": "[]"
}
}
My goal is to create configMap file with empty array.
Sure, you can make it whatever string you want, it just has to be a string. The thing you can't do is test.json: [] since that's an array. The fact that your string happens to be valid JSON is not something K8s knows or cares about.
I have a Ansible group_vars directory with the following file within it:
$ cat inventory/group_vars/env1
...
...
ldap_config: !vault |
$ANSIBLE_VAULT;1.1;AES256
31636161623166323039356163363432336566356165633232643932623133643764343134613064
6563346430393264643432636434356334313065653537300a353431376264333463333238383833
31633664303532356635303336383361386165613431346565373239643431303235323132633331
3561343765383538340a373436653232326632316133623935333739323165303532353830386532
39616232633436333238396139323631633966333635393431373565643339313031393031313836
61306163333539616264353163353535366537356662333833653634393963663838303230386362
31396431636630393439306663313762313531633130326633383164393938363165333866626438
...
...
This Ansible encrypted string has a Kubernetes secret encapsulated within it. A base64 blob that looks something like this:
IyMKIyBIb3N0IERhdGFiYXNlCiMKIyBsb2NhbGhvc3QgaXMgdXNlZCB0byBjb25maWd1cmUgdGhlIGxvb3BiYWNrIGludGVyZmFjZQojIHdoZW4gdGhlIHN5c3RlbSBpcyBib290aW5nLiAgRG8gbm90IGNoYW5nZSB0aGlzIGVudHJ5LgojIwoxMjcuMC4wLjEJbG9jYWxob3N0CjI1NS4yNTUuMjU1LjI1NQlicm9hZGNhc3Rob3N0Cjo6MSAgICAgICAgICAgICBsb2NhbGhvc3QKIyBBZGRlZCBieSBEb2NrZXIgRGVza3RvcAojIFRvIGFsbG93IHRoZSBzYW1lIGt1YmUgY29udGV4dCB0byB3b3JrIG9uIHRoZSBob3N0IGFuZCB0aGUgY29udGFpbmVyOgoxMjcuMC4wLjEga3ViZXJuZXRlcy5kb2NrZXIuaW50ZXJuYWwKIyBFbmQgb2Ygc2VjdGlvbgo=
How can I decrypt this in a single CLI?
We can use an Ansible adhoc command to retrieve the variable of interest, ldap_config. To start we're going to use this adhoc to retrieve the Ansible encrypted vault string:
$ ansible -i "localhost," all \
-m debug \
-a 'msg="{{ ldap_config }}"' \
--vault-password-file=~/.vault_pass.txt \
-e#inventory/group_vars/env1
localhost | SUCCESS => {
"msg": "ABCD......."
Make note that we're:
using the debug module and having it print the variable, msg={{ ldap_config }}
giving ansible the path to the secret to decrypt encrypted strings
using the notation -e#< ...path to file...> to pass the file with the encrypted vault variables
Now we can use Jinja2 filters to do the rest of the parsing:
$ ansible -i "localhost," all \
-m debug \
-a 'msg="{{ ldap_config | b64decode | from_yaml }}"' \
--vault-password-file=~/.vault_pass.txt \
-e#inventory/group_vars/env1
localhost | SUCCESS => {
"msg": {
"apiVersion": "v1",
"bindDN": "uid=readonly,cn=users,cn=accounts,dc=mydom,dc=com",
"bindPassword": "my secret password to ldap",
"ca": "",
"insecure": true,
"kind": "LDAPSyncConfig",
"rfc2307": {
"groupMembershipAttributes": [
"member"
],
"groupNameAttributes": [
"cn"
],
"groupUIDAttribute": "dn",
"groupsQuery": {
"baseDN": "cn=groups,cn=accounts,dc=mydom,dc=com",
"derefAliases": "never",
"filter": "(objectclass=groupOfNames)",
"scope": "sub"
},
"tolerateMemberNotFoundErrors": false,
"tolerateMemberOutOfScopeErrors": false,
"userNameAttributes": [
"uid"
],
"userUIDAttribute": "dn",
"usersQuery": {
"baseDN": "cn=users,cn=accounts,dc=mydom,dc=com",
"derefAliases": "never",
"scope": "sub"
}
},
"url": "ldap://192.168.1.10:389"
}
}
NOTE: The above section -a 'msg="{{ ldap_config | b64decode | from_yaml }}" is what's doing the heavy lifting in terms of converting from Base64 to YAML.
References
How to run Ansible without hosts file
https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#filters-for-formatting-data
Base64 Decode String in jinja
How to decrypt string with ansible-vault 2.3.0
If you need a one liner that works with any yaml file (not only in inventory) containing inlined vault vars, and if you are ready to install a pip package for that, there is a solution using yq, a yaml processor built on top of jq
prerequesite: Install yq
pip install yq
Usage
You can get your result with the following command:
yq -r .ldapconfig inventory/group_vars/env1 | ansible_vault decrypt
If you need to type your vault pass interactively, don't forget to add the relevant option
yq -r .ldapconfig inventory/group_vars/env1 | ansible_vault --ask-vault-pass decrypt
Note: the -r option to yq is mandatory to get a raw result without the quotation marks around the value.