Even after configuring the region explicitly, SDK seems to report more snapshots than visible in the AWS console. It seems like it returns snapshot data from all regions for the account.
sess := session.Must(session.NewSessionWithOptions(session.Options{
Config: aws.Config{Region: aws.String("eu-central-1"),Endpoint: aws.String("ec2.eu-central-1.amazonaws.com")},
}))
svc := ec2.New(sess, &aws.Config{
Region: aws.String(endpoints.EuCentral1RegionID),
})
input := &ec2.DescribeSnapshotsInput{
Filters: []*ec2.Filter{
&ec2.Filter{
Name: aws.String("status"),
Values: []*string{aws.String("completed")},
},
},
}
result, err := svc.DescribeSnapshots(input)
if err != nil {
if aerr, ok := err.(awserr.Error); ok {
switch aerr.Code() {
default:
fmt.Println(aerr.Error())
}
} else {
fmt.Println(err.Error())
}
}
for _, snapshot := range result.Snapshots{
fmt.Println(*snapshot.SnapshotId, " : ", *snapshot.VolumeSize)
}
Followed the documentation here and understood that DescribeSnapshots returns three types of snapshots. a) Public Snapshots (Made publicly available for volume creation by somebody else) b) Explicit Snapshots (Made available explicitly to your AWS account by somebody else) c) Implicit Snapshots (Snapshots created by your own account)
Hence, there are lot more results than expected.
Additionaly, if you pass the following filter to DescribeSnapshotsInput:
&ec2.Filter{
Name: aws.String("owner-id"),
Values:[]*string{aws.String("XXXXYYYYYZZZZ")},
},
the results are further reduced to show snapshots that are only available in your region
Related
I'm using the kubebuilder framework in my controller and I want to completely ignore events from a list of given system namespaces. I think predicates must be the best option for this, hence I have the following code
...
WithEventFilter(predicate.Funcs{
GenericFunc: func(e event.GenericEvent) bool {
namespace := e.Object.GetNamespace()
if _, ok := systemNs[namespace]; ok {
// ignore events from system namespaces
return false
}
return true
},
}).
However I still receive events from all namespaces instead of just the ones which I allow.
Is this a correct and valid use case of predicates?
According to controller-runtime
Generic is called in response to an event of an unknown type or a synthetic event triggered as a cron or external trigger request - e.g. reconcile Autoscaling, or a Webhook.
Most of the time reconciliation gets triggered on Create, Update, or Delete so you need to filter on these events.
WithEventFilter(predicate.Funcs{
CreateFunc: func(e event.CreateEvent) bool {
namespace := e.Object.GetNamespace()
if _, ok := systemNs[namespace]; ok {
// ignore events from system namespaces
return false
}
return true
},
UpdateFunc: func(e event.UpdateEvent) bool {
namespace := e.ObjectNew.GetNamespace()
if _, ok := systemNs[namespace]; ok {
// ignore events from system namespaces
return false
}
return true
},
DeleteFunc: func(e event.DeleteEvent) bool {
namespace := e.Object.GetNamespace()
if _, ok := systemNs[namespace]; ok {
// ignore events from system namespaces
return false
}
return true
},
}).
NewPredicateFuncs apply the filter on all events:
WithEventFilter(predicate.NewPredicateFuncs(func(obj client.Object) bool {
_, ok := systemNs[obj.GetNamespace()]
return !ok
})).
I'm trying to modify the Spec of non-owned objects as part of the Reconcile of my Custom Resource, but it seems like it ignores any fields that are not primitives. I am using controller-runtime.
I figured since it was only working on primitives, maybe it's an issue related to DeepCopy. However, removing it did not solve the issue, and I read that any Updates on objects have to be on deep copies to avoid messing up the cache.
I also tried setting client.FieldOwner(...) since it says that that's required for Updates that are done server-side. I wasn't sure what to set it to, so I made it req.NamespacedName.String(). That did not work either.
Here is the Reconcile loop for my controller:
func (r *MyCustomObjectReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
// ...
var myCustomObject customv1.MyCustomObject
if err := r.Get(ctx, req.NamespacedName, &myCustomObject); err != nil {
log.Error(err, "unable to fetch ReleaseDefinition")
return ctrl.Result{}, client.IgnoreNotFound(err)
}
// ...
deployList := &kappsv1.DeploymentList{}
labels := map[string]string{
"mylabel": myCustomObject.Name,
}
if err := r.List(ctx, deployList, client.MatchingLabels(labels)); err != nil {
log.Error(err, "unable to fetch Deployments")
return ctrl.Result{}, err
}
// make a deep copy to avoid messing up the cache (used by other controllers)
myCustomObjectSpec := myCustomObject.Spec.DeepCopy()
// the two fields of my CRD that affect the Deployments
port := myCustomObjectSpec.Port // type: *int32
customenv := myCustomObjectSpec.CustomEnv // type: map[string]string
for _, dep := range deployList.Items {
newDeploy := dep.DeepCopy() // already returns a pointer
// Do these things:
// 1. replace first container's containerPort with myCustomObjectSpec.Port
// 2. replace first container's Env with values from myCustomObjectSpec.CustomEnv
// 3. Update the Deployment
container := newDeploy.Spec.Template.Spec.Containers[0]
// 1. Replace container's port
container.Ports[0].ContainerPort = *port
envVars := make([]kcorev1.EnvVar, 0, len(customenv))
for key, val := range customenv {
envVars = append(envVars, kcorev1.EnvVar{
Name: key,
Value: val,
})
}
// 2. Replace container's Env variables
container.Env = envVars
// 3. Perform update for deployment (port works, env gets ignored)
if err := r.Update(ctx, newDeploy); err != nil {
log.Error(err, "unable to update deployment", "deployment", dep.Name)
return ctrl.Result{}, err
}
}
return ctrl.Result{}, nil
}
The Spec for my CRD looks like:
// MyCustomObjectSpec defines the desired state of MyCustomObject
type MyCustomObjectSpec struct {
// CustomEnv is a list of environment variables to set in the containers.
// +optional
CustomEnv map[string]string `json:"customEnv,omitempty"`
// Port is the port that the backend container is listening on.
// +optional
Port *int32 `json:"port,omitempty"`
}
I expected that when I kubectl apply a new CR with changes to the Port and CustomEnv fields, it would modify the deployments as described in Reconcile. However, only the Port is updated, and the changes to the container's Env are ignored.
The problem was that I needed a pointer to the Container I was modifying.
Doing this instead worked:
container := &newDeploy.Spec.Template.Spec.Containers[0]
Having trouble figuring out what is wrong. I have a remote kubernetes cluster up and have copied the config locally. I know it is correct because I have gotten other commands to work for me.
The one I can't get to work is a deployment patch. My code:
const namespace = "default"
var clientset *kubernetes.Clientset
func init() {
kubeconfig := "/Users/$USER/go/k8s-api/config"
config, err := clientcmd.BuildConfigFromFlags("", kubeconfig)
if err != nil {
log.Fatal(err)
}
// create the clientset
clientset, err = kubernetes.NewForConfig(config)
if err != nil {
panic(err.Error())
}
}
func main() {
deploymentsClient := clientset.ExtensionsV1beta1().Deployments("default")
patch := []byte(`[{"spec":{"template":{"spec":{"containers":[{"name":"my-deploy-test","image":"$ORG/$REPO:my-deploy0.0.1"}]}}}}]`)
res, err := deploymentsClient.Patch("my-deploy", types.JSONPatchType, patch)
if err != nil {
panic(err)
}
fmt.Println(res)
}
All I get back is:
panic: the server rejected our request due to an error in our request
Any help appreciated, thanks!
You have mixed up JSONPatchType with MergePatchType; JSONPatchType wants the input to be RFC 6902 formatted "commands", and in that case can be a JSON array, because there can be multiple commands applied in order to the input document
However, your payload looks much closer to you wanting MergePatchType, in which case the input should not be a JSON array because the source document is not an array of "spec" objects.
Thus, I'd bet just dropping the leading [ and trailing ], changing the argument to be types.MergePatchType will get you much further along
Actually you should use types.StrategicMergePatchType and remove leading([) and trailing(]) parenthesis from patching string.
Merge-patch: With a JSON merge patch, if you want to update a list, you have to specify the entire new list. And the new list completely replaces the existing list.
Strategic-merge-patch: With a strategic merge patch, a list is either replaced or merged depending on its patch strategy. The patch strategy is specified by the value of the patchStrategy key in a field tag in the Kubernetes source code. For example, the Containers field of PodSpec struct has a patchStrategy of merge:
type PodSpec struct {
...
Containers []Container `json:"containers" patchStrategy:"merge" patchMergeKey:"name" ...`
N.B: kubectl by-default uses strategic merge patch to patch kubernetes resources.
I have the following yaml file with me:
nodes: ["1.1.1.1","127.0.0.1","2.2.2.2"]
emailto: ["sample#sample.com","sample#sample.com"]
I want to open the YAML file, iterate over the IPs one by one and do some certain action. If there is an error, then it should automatically take the next ip and perform the same function. I am having trouble as to how to save the IPs to a list or how to iterate in GOLANG.
Also, I have to send an email based to the e-mail IDs present in the YAML file. Which package in GO is used for that and how to do it like SMTPLIB in Python.
It looks like there are three parts to your question:
1. See gopkg.in/yaml.v2 for YAML parsing
import (
// ...
"gopkg.in/yaml.v2"
)
type Doc struct {
Nodes []string `yaml:"nodes"`
Emails []string `yaml:"emailto"`
}
// Parse the YAML document.
doc := Doc{}
err := yaml.Unmarshal(yamlbytes, &doc)
if err != nil {
log.Fatalf("FATAL: could not parse YAML file: %s", err)
}
2. Example of iterating using range, calling a function, and handling errors
// Iterate over each node and do something, handling errors as needed.
for _, node := range doc.Nodes {
err := DoSomethingWithNode(node)
if err != nil {
log.Printf("ERROR: failed to handle node %q: %s", node, err)
} else {
log.Printf("OK: successfully handled node %q.", node)
}
}
3. See the builtin net/smtp package for sending email
See the package example for a complete illustration.
I am working on a SaaS based project on which Merchants can subscribe to set up their online store.
Project Overview
I am using Golang (backend), Mongodb database service and Angular4 (frontend) to build the system. I have multiple merchants that can set up their store. Each merchant has its own url (having its business name as subdomain in the url) to connect to his database.
For Routing, I am using Golang's Gin framework at back end.
Problem
I want to run the cron jobs for the merchant-specific database. In these cron jobs there are some operations that need to connect to the database. But in my routing, until a route for an API is called, the database won't be set. And ultimately, the cron does not run with proper data.
Code
cron.go
package cron
import (
"gopkg.in/robfig/cron.v2"
"controllers"
)
func RunCron(){
c := cron.New()
c.AddFunc("#every 0h1m0s", controllers.ExpireProviderInvitation)
c.Start()
}
Controller function
func ExpireProviderInvitation() {
bookingAcceptTimeSetting, _ := models.GetMerchantSetting(bson.M{"section": "providers", "option_name": "bookings_accept_time"})
if bookingAcceptTimeSetting.OptionValue != nil{
allInvitations, _ := models.GetAllBookingInvitations(bson.M{ "status": 0, "send_type": "invitation", "datetime": bson.M{"$le": float64(time.Now().Unix()) - bookingAcceptTimeSetting.OptionValue.(float64)} })
if len(allInvitations) > 0 {
for _, invitationData := range allInvitations {
_ = GetNextAvailableProvider(invitationData.Bid, invitationData.Pid)
}
}
}
}
router.go
func NewRouter() {
router := gin.Default()
router.Use(gin.Recovery())
router.Use(SetMerchantDatabase)
public := router.Group("/api/v1")
for _, route := range publicRoutes{
switch route.Method {
case "GET" : public.GET(route.Pattern, route.HandlerFunc)
case "POST" : public.POST(route.Pattern, route.HandlerFunc)
case "PUT" : public.PUT(route.Pattern, route.HandlerFunc)
case "DELETE": public.DELETE(route.Pattern, route.HandlerFunc)
default : public.GET(route.Pattern, func(c *gin.Context){
c.JSON(200, gin.H{
"result": "Specify a valid http method with this route.",
})
})
}
}
router.NoRoute(controllers.UnauthorizedAccessResponse)
router.Run(":8080")
}
func SetMerchantDatabase(c *gin.Context){
subdomain := strings.Split(c.Request.Host, ".")
if len(subdomain) > 0{
config.Database = subdomain[0]
config.CurrentBusinessName = subdomain[0]
}else{
errMsg := "Failed: Invalid domain in headers."
response := controllers.ResponseController{
config.FailureCode,
config.FailureFlag,
errMsg,
nil,
}
controllers.GetResponse(c, response)
c.Abort()
}
c.Next()
}
main.go
package main
import (
"cron"
)
func main(){
cron.RunCron()
NewRouter()
}
Explanation of above code
An example route can be:
Route{ "AddCustomer", "POST", "/customer", controllers.SaveCustomer },
An example API url can be:
http://business-name.main-domain.com/api/v1/customer
Where "business-name" is the database which is set whenever an API is called.
I want to run my cron without calling an API route.
Alternative approach
In Shell script, we can run cron by hitting url as a command. For this, I can create a url to run it as a command. But this is my theoratical approach. Also I don't know how will I get different merchant databases.
I am not sure if this approach will work. Any kind of help will be greatly appreciated.
You need to adapt SetMerchantDatabase to work independently of your router. Then you can have it set things for Cron just as well.