I've setup the ELK stack (Elasticsearch, Logstash and Kibana) by following this Github repo.
Now I want to send data to logstash, for example specifying a pipeline as following:
input { stdin{} } output { stdout{} }
But I don't know how to do that. I searched and tried some ways but I couldn't succeed.
Can someone help me please?
Related
I am trying to setup monitoring in local as mentioned in https://cadenceworkflow.io/docs/operation-guide/monitor/#instructions
Having these errors for http://host.docker.internal:9098/metrics, http://cadence:9090/metrics as shown in below image.
Can please let me know how we can resolve this, Thanks
Endpoints state
9090 is Prometheus itself. Are you configuring a different port? https://github.com/uber/cadence/blob/68fb2e60d1a2bff77c66acf60c954c9d19f9e5f5/docker/docker-compose-es-v7.yml#L14
But anyway, this is not something important so if you like, you can ignore this error.
9098 is the client sdk . The doc is assuming that you are setting it up correctly: https://github.com/uber/cadence-java-samples/blob/cdd43b6a65bf537ef6c77262a56cd22308d75e06/src/main/java/com/uber/cadence/samples/hello/HelloMetric.java#L53
https://github.com/uber-common/cadence-samples/blob/beacf223ab727c7fd114236f40806497c6d0aabd/config/development.yaml#L7
This is my first question posted on stackoverflow, hence in case I did something incorrectly pleaselet me know.
Description
I am currently working on translation of freestyle projects to declarative pipelines in Jenkins (jenkinsfiles kept in Git repo). The original freestyle job was triggering PowerShell script which needed access to Global name/password pairs defined in Mask Passwords plugin section in Configure System. Solution to this problem was an additional tick in project itself (unfortunately I am not allowed adding screenshots to posts yet, hence editor uploaded screen to imgur and pasted link - please see Screenshot 1):
Screenshot 1
Therefore I started looking for possible implementation of such solution to jenkinsfile, however wothout luck.
My problem
When the script is triggered from the pipeline, it errors out stating that it cannot find relevant passwords (powershell refers to those credentials as to environment variables). This works fine when ran from freestyle project.
Which I reckon is caused by pipeline not being able to reach out to previously mentioned credentials.
What I tried
Wrapping the step into below block:
wrap([$class: 'MaskPasswordsBuildWrapper']) {
bat(batch file launching ps script)
}
Then the above block containing relevant step wrapping into
script {
wrap(...)
}
But none of them worked.
I have taken a look at other plugins like Credentials Binding Plugin or Credentials Plugin but those allow to bind/pass one credential per step, and I need to pass all credentials specified in Jenkins (I am open to move saved credentials to any other location within Jenkins).
I have looked at adding environment variable:
credentials('Credentials-ID')
But the problem is the same as with mentioned plugins.
By any chance, have anyone came across similar situation and know what can be done in order to allow pipeline to access/pass to pipeline all the credentials specified in Jenkins instead of binding/passing them one a time?
All tips are very welcome!
You can do this and the env variable will then be available throughout your job. You could define multiple env variable too.
environment {
// Use credentials() to hide the environment variable's output
MY_PERSONAL_TOKEN = credentials('Credentials-ID')
}
stages {
stage('Test Stage') {
steps {
script {
// do what you need to
}
}
}
}
I have tried this tutorial. But it didn't catch the OSSEC log (alerts, syslog, etc), it just give me this message for my Kibana apps.
Couldn't find any Elasticsearch data
You'll need to index some data into Elasticsearch before you can create an index pattern.
I know that there is some tutorial like this. But it's required to use wazuh package and I dont want to use it, I just want to use the pure OSSEC. My OSSEC and ELK apps are located in the samw machine
My question is, How can I integrate OSSEC with ELK ? What configuration do i have to do first before starting connected OSSEC to ELK ?
You need to load the data template so that Elastisearch can understand the format of the alert data. You can use the one made by Wazuh, or you could download it and modify it to "make your own". If you go down this road you will eventually end up trying to re-write Wazuh, which you don't need to do because it is open source. You can just download all the source files and do whatever you want with them.
Command to load template:
curl https://raw.githubusercontent.com/wazuh/wazuh/3.2/extensions/elasticsearch/wazuh-elastic6-template-alerts.json | curl -XPUT 'http://localhost:9200/_template/wazuh' -H 'Content-Type: application/json' -d #-
Download Template:
https://raw.githubusercontent.com/wazuh/wazuh/3.2/extensions/elasticsearch/wazuh-elastic6-template-alerts.json
-OR-
You could just spin up a Docker container that is ready to go:
https://github.com/wazuh/docker-ossec-elk
I have a Message Hub instance on Bluemix, and am able to produce / consume messages off it. I was looking for a quick, reasonable way to browse topics / messages to see what's going on. Something along the lines of kafka-topics-ui.
I installed kafka-topics-ui locally, but could not get it to connect to Message Hub. I used the kafka-rest-url value from the MessageHub credentials in the kafka-topics-ui configuration file (env.js), but could not figure out where to provide the API key.
Alternatively, in the Bluemix UI, under Kibana, I can see log entries for creating the topic. Unfortunately, I could not see log entries for messages in the topic (perhaps I'm looking the wrong place or have wrong filters?).
My guess is I'm missing something basic. Is there a way to either:
configure a tool such as kafka-topics-ui to connect to MessageHub,
or,
browse topic messages easily?
Cheers.
According to Using the Kafka REST API on Bluemix you need an additional header in all API requests:
-H "X-Auth-Token: APIKEY"
A quick solution is to edit the topic-ui code and include your token in every request. Another solution would be to use a Chrome plugin that can inject the above header. For a more formal solution, i have opened a ticket on github
I was testing OpenWhisk on Bluemix. What i want to do is very very simple. I have a js script that works (it's just a console.log). I have a Dockerfile making a docker .. when i run my docker i can see my console.log \o/ (i know it's amazing)
Then i create my action ......
I try to run it from Bluemix UI
I get this :
failed to invoke action XXX/XXXX: Container hash or name expected in makeContainer.
Just what does it means ? I couldn't find anything on it.
ps : If you know where i can find some real doc about how to use docker container for actions i would be very happy :).
Thanks a lot for your help.
Edit : In the end the problem was that i was trying to pull from a private docker repo. I have to say the error message is not explicit x). Thanks for your help.
You can find the documentation about creating docker actions here: https://new-console.ng.bluemix.net/docs/openwhisk/openwhisk_actions.html#openwhisk_actions_docker
Instructions helps you download the docker sdk for OpenWhisk, and allow you to create a docker action that you can invoke from CLI or Web UI.
If you create the action using the Web UI you need to specify the docker Image from Docker hub using the format username/imagename like openwhisk/example
If you need more help feel free to reach the team on Slack for more real time interactions and we should be able to help you and get your feedback on how can we improve the user experience.
Team: https://developer.ibm.com/open/slack-request
Channel: #openwhisk https://dwopen.slack.com/messages/openwhisk