I can't find it in the dock(https://concourse-ci.org /) this setting.
There is a type:
source:
interval: 1h #
location: Europe/Moscow
But the problem with this setup is that it starts every hour, and the hour is counted from the moment the previous cycle is completed (I think so, because the schedule is gradually shifting forward, which is generally fast)
Yeah, the time resource doesn't implement precision.
Use the cron resource instead: https://github.com/pivotal-cf-experimental/cron-resource
---
resource_types:
- name: cron-resource
type: docker-image
source:
repository: cftoolsmiths/cron-resource
resources:
- name: 00-trigger
type: cron-resource
source:
expression: "0 * * * *"
location: "Europe/Moscow"
Related
I have a couple (at some point many) k8s cron jobs, each with schedules like
*/5 * * * * # Every five minutes
*/1 * * * * # Every minute
*/1 * * * *
*/2 * * * * # Every two minutes
...
My problem is that k8s seems to start them all at the top of the minute, so there might be a large number of jobs running at the same time. Each job only takes <10 seconds to run, so ideally, I would like to be able to distribute them over the span of a minute, i.e. delay their start.
Any ideas how to do that, given that k8s does not second-based schedule expressions?
You cant start a job indicating the seconds.
What you can do is delaying each job with sleep .
The pods of the cronjobs would start however together, but you can distributed the load of the jobs in this way.
apiVersion: batch/v1
kind: CronJob
metadata:
name: myminutlyCronjob
spec:
schedule: "1 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: mycronjob
image: myjobImage
imagePullPolicy: IfNotPresent
command: ["/bin/sh", "-c"]
args:
- sleep 10;
mycronjobStartCommand;
restartPolicy: OnFailure
Hope this can help you.
I'm trying to set up a Concourse pipeline that will trigger a new deployment. The goal is to only let the pipeline run when new values have been pushed to the git repository AND when the time is within a defined time window.
Currently, the triggers seem to work in an OR fashion. When a new version is pushed, the pipeline will run. When the time is within the window, the pipeline will run.
It seems like the only exception is when both triggers have not succeeded at least once, for example on the first day when the time has not yet passed. This caused the pipeline to wait for the first success of the time-window trigger before running. After this, however, the unwanted behavior of running with each update to the git repository continued.
Below is a minimal version of my pipeline. The goal is to run the pipeline between only 9:00 and 9:10 PM, and preferably only when the git repository has been updated.
resource_types:
- name: helm
type: docker-image
source:
repository: linkyard/concourse-helm-resource
resources:
- name: cicd-helm-values_my-service
type: git
source:
branch: master
username: <redacted>
password: <redacted>
uri: https://bitbucket.org/myorg/cicd-helm-values.git
paths:
- dev-env/my-service/values.yaml
- name: helm-deployment
type: helm
source:
cluster_url: '<redacted>'
cluster_ca: <redacted>
admin_cert: <redacted>
admin_key: <redacted>
repos:
- name: chartmuseum
url: '<redacted>'
username: <redacted>
password: <redacted>
- name: time-window
type: time
source:
start: 9:00 PM
stop: 9:10 PM
jobs:
- name: deploy-my-service
plan:
- get: time-window
trigger: true
- get: cicd-helm-values_my-service
trigger: true
- put: helm-deployment
params:
release: my-service
namespace: dev-env
chart: chartmuseum/application-template
values: ./cicd-helm-values_my-service/dev-env/my-service/values.yaml
Any ideas on how to combine the time-window and the cicd-helm-values_my-service would be greatly appreciated. Thanks in advance!
For that kind of precise time scheduling, the time resource is not adapted. What works well is https://github.com/pivotal-cf-experimental/cron-resource. This will solve one part of your problem.
Regarding triggering with AND, this is not the semantics of a fan-in. The semantics is OR, as you noticed. You might try the gate resource https://github.com/Meshcloud/gate-resource, although I am not sure it would work for your case.
EDIT: Fixed the URL of the gated resource
I set concurrencyPolicy to Allow, here is my cronjob.yaml:
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: gke-cron-job
spec:
schedule: '*/1 * * * *'
startingDeadlineSeconds: 10
concurrencyPolicy: Allow
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 1
jobTemplate:
spec:
template:
metadata:
labels:
run: gke-cron-job
spec:
restartPolicy: Never
containers:
- name: gke-cron-job-solution-2
image: docker.io/novaline/gke-cron-job-solution-2:1.3
env:
- name: NODE_ENV
value: 'production'
- name: EMAIL_TO
value: 'novaline.dulin#gmail.com'
- name: K8S_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
ports:
- containerPort: 8080
protocol: TCP
After reading docs: https://cloud.google.com/kubernetes-engine/docs/how-to/cronjobs
I still don't understand how to use concurrencyPolicy.
How can I run my cron job concurrency?
Here is the logs of cron job:
☁ nodejs-gcp [master] ⚡ kubectl logs -l run=gke-cron-job
> gke-cron-job-solution-2#1.0.2 start /app
> node ./src/index.js
config: { ENV: 'production',
EMAIL_TO: 'novaline.dulin#gmail.com',
K8S_POD_NAME: 'gke-cron-job-1548660540-gmwvc',
VERSION: '1.0.2' }
[2019-01-28T07:29:10.593Z] Start daily report
send email: { to: 'novaline.dulin#gmail.com', text: { test: 'test data' } }
> gke-cron-job-solution-2#1.0.2 start /app
> node ./src/index.js
config: { ENV: 'production',
EMAIL_TO: 'novaline.dulin#gmail.com',
K8S_POD_NAME: 'gke-cron-job-1548660600-wbl5g',
VERSION: '1.0.2' }
[2019-01-28T07:30:11.405Z] Start daily report
send email: { to: 'novaline.dulin#gmail.com', text: { test: 'test data' } }
> gke-cron-job-solution-2#1.0.2 start /app
> node ./src/index.js
config: { ENV: 'production',
EMAIL_TO: 'novaline.dulin#gmail.com',
K8S_POD_NAME: 'gke-cron-job-1548660660-8mn4r',
VERSION: '1.0.2' }
[2019-01-28T07:31:11.099Z] Start daily report
send email: { to: 'novaline.dulin#gmail.com', text: { test: 'test data' } }
As you can see, the timestamp indicates that the cron job is not concurrency.
It's because you're reading the wrong documentation. CronJobs aren't a GKE-specific feature. For the full documentation on CronJob API, refer to the Kubernetes documentation: https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#concurrency-policy (quoted below).
Concurrency policy decides whether a new container can be started while the previous CronJob is still running. If you have a CronJob that runs every 5 minutes, and sometimes the Job takes 8 minutes, then you may run into a case where multiple jobs are running at a time. This policy decides what to do in that case.
Concurrency Policy
The .spec.concurrencyPolicy field is also optional. It specifies how to treat concurrent executions of a job that is created by this cron job. the spec may specify only one of the following concurrency policies:
Allow (default): The cron job allows concurrently running jobs
Forbid: The cron job does not allow concurrent runs; if it is time for a new job run and the previous job run hasn’t finished yet, the cron job skips the new job run
Replace: If it is time for a new job run and the previous job run hasn’t finished yet, the cron job replaces the currently running job run with a new job run
Note that concurrency policy only applies to the jobs created by the same cron job. If there are multiple cron jobs, their respective jobs are always allowed to run concurrently.
I want a job to trigger every 15 minutes but it is consistently triggering every 30 minutes.
UPDATE:
I've simplified the problem by just running:
kubectl run hello --schedule="*/1 * * * *" --restart=OnFailure --image=busybox -- /bin/sh -c "date; echo Hello from the Kubernetes cluster"
As specified in the docs here: https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/
and yet the job still refuses to run on time.
$ kubectl get cronjobs
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
hello */1 * * * * False 1 5m 30m
hello2 */1 * * * * False 1 5m 12m
It took 25 minutes for the command line created cronjob to run and 7 minutes for the cronjob created from yaml. They were both finally scheduled at the same time so it's almost like etcd finally woke up and did something?
ORIGINAL ISSUE:
When I drill into an active job I see Status: Terminated: Completed but
Age: 25 minutes or something greater than 15.
In the logs I see that the python script meant to run has completed it's final print statement. The script takes about ~2min to complete based on it's output file in s3. Then no new job is scheduled for 28 more minutes.
I have tried with different configurations:
Schedule: */15 * * * * AND Schedule: 0,15,30,45 * * * *
As well as
Concurrency Policy: Forbid AND Concurrency Policy: Replace
What else could be going wrong here?
Full config with identifying lines modified:
apiVersion: batch/v1beta1
kind: CronJob
metadata:
labels:
type: f-c
name: f-c-p
namespace: extract
spec:
concurrencyPolicy: Forbid
failedJobsHistoryLimit: 1
jobTemplate:
metadata:
creationTimestamp: null
spec:
template:
metadata:
creationTimestamp: null
labels:
type: f-c
spec:
containers:
- args:
- /f_c.sh
image: identifier.amazonaws.com/extract_transform:latest
imagePullPolicy: Always
env:
- name: ENV
value: prod
- name: SLACK_TOKEN
valueFrom:
secretKeyRef:
key: slack_token
name: api-tokens
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
key: aws_access_key_id
name: api-tokens
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
key: aws_secret_access_key
name: api-tokens
- name: F_ACCESS_TOKEN
valueFrom:
secretKeyRef:
key: f_access_token
name: api-tokens
name: s-f-c
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Never
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
schedule: '*/15 * * * *'
successfulJobsHistoryLimit: 1
suspend: false
status: {}
After running these jobs in a test cluster I discovered that external circumstances prevented them from running as intended.
On the original cluster there were ~20k scheduled jobs. The built-in scheduler for Kubernetes is not yet capable of handling this volume consistently.
The maximum number of jobs that can be reliably run (within a minute of the time intended) may depend on the size of your master nodes.
Isn't that by design?
A cron job creates a job object about once per execution time of its schedule. We say “about” because there are certain circumstances where two jobs might be created, or no job might be created. We attempt to make these rare, but do not completely prevent them. Therefore, jobs should be idempotent.
Ref. https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations
We have configured Grok exporter to monitor errors from various system logs. But it seems changes are reflected once we restart the respective grok instance.
Please see the config.yml below:
global:
config_version: 2
input:
type: file
path: /ZAMBAS/logs/Healthcheck/EFT/eftcl.log
readall: true
poll_interval_seconds: 5
grok:
patterns_dir: ./patterns
metrics:
- type: gauge
name: EFTFileTransfers
help: Counter metric example with labels.
match: '%{WORD:Status}\s%{GREEDYDATA:FileTransferTime};\s\\%{WORD:Customer}\\%{WORD:OutboundSystem}\\%{GREEDYDATA:File};\s%{WORD:Operation};\s%{NUMBER:Code}'
value: '{{.Code}}'
cumulative: false
labels:
Customer: '{{.Customer}}'
OutboundSystem: '{{.OutboundSystem}}'
File: '{{.File}}'
Status: '{{.Status}}'
Operation: '{{.Operation}}'
FileTransferTime: '{{.FileTransferTime}}'
- type: gauge
name: EFTFileSuccessfullTransfers
help: Counter metric example with labels.
match: 'Success\s%{GREEDYDATA:Time};\s\\%{WORD:Customer}\\%{WORD:OutboundSystem}\\%{GREEDYDATA:File};\s%{WORD:Operation};\s%{NUMBER:Code}'
value: '{{.Code}}'
cumulative: false
- type: gauge
name: EFTFileFailedTransfers
help: Counter metric example with labels.
match: 'Failed\s%{GREEDYDATA:Time};\s\\%{WORD:Customer}\\%{WORD:OutboundSystem}\\%{GREEDYDATA:File};\s%{WORD:Operation};\s%{NUMBER:Code}'
value: '{{.Code}}'
cumulative: false
server:
port: 9845
Without restart it doesn't reflects correct matching patterns. Once I restart the grok instance it reflects perfectly.
Is there some parameter I am missing here ?
Thanks
Priyotosh
Simply change readall to false in input section will stop process lines multiple times when grok_exporter is restarted. Please see the docs on Github.