Logstash 5.0 with input websocket - unable to stdout json data - sockets

Question

I have the following environment:
/usr/share/logstash# bin/logstash --path.settings=/etc/logstash -f /etc/logstash/conf.d/stream.conf -V
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties.
logstash 5.0.0
jruby 1.7.25 (1.9.3p551) 2016-04-13 867cb81 on Java HotSpot(TM) 64-Bit Server VM 1.8.0_111-b14 +jit [linux-amd64]
java 1.8.0_111 (Oracle Corporation)
jvm Java HotSpot(TM) 64-Bit Server VM / 25.111-b14
I installed the following 2 repos
logstash-input-websocket version 3.0.2
ruby-ftw http://github.com/jordansissel/ruby-ftw"
I created both gems by running gem install .gemspec file.
My Gemfile under /usr/share/logstash folder was modified to add those 2 lines.
gem "logstash-input-websocket", :path => "/home/xav/source/logstash-input-websocket-master"
gem "ftw", :path => "/home/xav/source/ruby-ftw-master"
I know my logstash configuration is ok ( I checked it with -t option).
Furthermore, on purpose I modified my stream.conf file to omit the url definition for the input websocket to make sure the plugin was being used.
I got the expected error in the /var/log/logstash/logstash-plain.log file as below:
xav#xav-VirtualBox:/var/log/logstash$ tail -f logstash-plain.log
[2016-11-01T11:40:28,998][ERROR][logstash.inputs.websocket] Missing a required setting for the websocket input plugin:
input {
websocket {
url => # SETTING MISSING
...
}
}
[2016-11-01T11:40:29,011][ERROR][logstash.agent ] fetched an invalid config {:config=>"\ninput {\n websocket {\n mode => client\n}\n}\n\noutput {\n\n stdout { }\n}\n\n", :reason=>"Something is wrong with your configuration."
I edited the stream.conf file to add the wss url I want to read the json
output from with:
input {
websocket {
mode => client
url => "wss://<my-url-to-websocket/something"}
}
output {
stdout { }
}
I run logstash again. Everything seems to be working fine BUT I don't get anything in my stdout. The log file output is:
11-01T12:09:25,968][DEBUG][logstash.runner ] -------- Logstash Settings (* means modified) ---------
11-01T12:09:25,980][DEBUG][logstash.runner ] node.name: "xav-VirtualBox"
11-01T12:09:25,981][DEBUG][logstash.runner ] *path.config: "/etc/logstash/conf.d/stream.conf"
11-01T12:09:25,981][DEBUG][logstash.runner ] *path.data: "/var/lib/logstash" (default: "/usr/share/logstash/data")
11-01T12:09:25,981][DEBUG][logstash.runner ] config.test_and_exit: false
11-01T12:09:25,981][DEBUG][logstash.runner ] config.reload.automatic: false
11-01T12:09:25,981][DEBUG][logstash.runner ] config.reload.interval: 3
11-01T12:09:25,982][DEBUG][logstash.runner ] metric.collect: true
11-01T12:09:25,982][DEBUG][logstash.runner ] pipeline.id: "main"
11-01T12:09:25,982][DEBUG][logstash.runner ] pipeline.workers: 1
11-01T12:09:25,982][DEBUG][logstash.runner ] pipeline.output.workers: 1
11-01T12:09:25,983][DEBUG][logstash.runner ] pipeline.batch.size: 125
11-01T12:09:25,983][DEBUG][logstash.runner ] pipeline.batch.delay: 5
11-01T12:09:25,983][DEBUG][logstash.runner ] pipeline.unsafe_shutdown: false
11-01T12:09:25,983][DEBUG][logstash.runner ] path.plugins: []
11-01T12:09:25,984][DEBUG][logstash.runner ] config.debug: false
11-01T12:09:25,984][DEBUG][logstash.runner ] *log.level: "debug" (default: "info")
11-01T12:09:25,984][DEBUG][logstash.runner ] version: false
11-01T12:09:25,984][DEBUG][logstash.runner ] help: false
11-01T12:09:25,984][DEBUG][logstash.runner ] log.format: "plain"
11-01T12:09:25,984][DEBUG][logstash.runner ] http.host: "127.0.0.1"
11-01T12:09:25,984][DEBUG][logstash.runner ] http.port: 9600..9700
11-01T12:09:25,986][DEBUG][logstash.runner ] http.environment: "production"
11-01T12:09:25,986][DEBUG][logstash.runner ] *path.settings: "/etc/logstash" (default: "/usr/share/logstash/config")
11-01T12:09:25,986][DEBUG][logstash.runner ] *path.logs: "/var/log/logstash" (default: "/usr/share/logstash/logs")
11-01T12:09:25,987][DEBUG][logstash.runner ] --------------- Logstash Settings -------------------
11-01T12:09:26,039][DEBUG][logstash.agent ] Agent: Configuring metric collection
11-01T12:09:26,043][DEBUG][logstash.instrument.periodicpoller.os] PeriodicPoller: Starting {:polling_interval=>1, :polling_timeout=>60}
11-01T12:09:26,049][DEBUG][logstash.instrument.periodicpoller.jvm] PeriodicPoller: Starting {:polling_interval=>1, :polling_timeout=>60}
11-01T12:09:26,122][DEBUG][logstash.agent ] Reading config file {:config_file=>"/etc/logstash/conf.d/stream.conf"}
11-01T12:09:26,197][DEBUG][logstash.codecs.json ] config LogStash::Codecs::JSON/#id = "json_982d8975-bd47-4f64-8a68-0da7e7a59a55"
11-01T12:09:26,198][DEBUG][logstash.codecs.json ] config LogStash::Codecs::JSON/#enable_metric = true
11-01T12:09:26,198][DEBUG][logstash.codecs.json ] config LogStash::Codecs::JSON/#charset = "UTF-8"
11-01T12:09:26,202][DEBUG][logstash.inputs.websocket] config LogStash::Inputs::Websocket/#mode = "client"
11-01T12:09:26,202][DEBUG][logstash.inputs.websocket] config LogStash::Inputs::Websocket/#url = "wss://REDACTED"
11-01T12:09:26,202][DEBUG][logstash.inputs.websocket] config LogStash::Inputs::Websocket/#id = "93ebfb1d0936097ee295b418952f2dab3abb3ef8-1"
11-01T12:09:26,203][DEBUG][logstash.inputs.websocket] config LogStash::Inputs::Websocket/#enable_metric = true
11-01T12:09:26,203][DEBUG][logstash.inputs.websocket] config LogStash::Inputs::Websocket/#codec = <LogStash::Codecs::JSON id=>"json_982d8975-bd47-4f64-8a68-0da7e7a59a55", enable_metric=>true, charset=>"UTF-8">
11-01T12:09:26,203][DEBUG][logstash.inputs.websocket] config LogStash::Inputs::Websocket/#add_field = {}
11-01T12:09:26,211][DEBUG][logstash.codecs.line ] config LogStash::Codecs::Line/#id = "line_a255b752-4d76-4933-b4d3-e76e427bbddb"
11-01T12:09:26,214][DEBUG][logstash.codecs.line ] config LogStash::Codecs::Line/#enable_metric = true
11-01T12:09:26,215][DEBUG][logstash.codecs.line ] config LogStash::Codecs::Line/#charset = "UTF-8"
11-01T12:09:26,215][DEBUG][logstash.codecs.line ] config LogStash::Codecs::Line/#delimiter = "\n"
11-01T12:09:26,218][DEBUG][logstash.outputs.stdout ] config LogStash::Outputs::Stdout/#id = "93ebfb1d0936097ee295b418952f2dab3abb3ef8-2"
11-01T12:09:26,219][DEBUG][logstash.outputs.stdout ] config LogStash::Outputs::Stdout/#enable_metric = true
11-01T12:09:26,219][DEBUG][logstash.outputs.stdout ] config LogStash::Outputs::Stdout/#codec = <LogStash::Codecs::Line id=>"line_a255b752-4d76-4933-b4d3-e76e427bbddb", enable_metric=>true, charset=>"UTF-8", delimiter=>"\n">
11-01T12:09:26,219][DEBUG][logstash.outputs.stdout ] config LogStash::Outputs::Stdout/#workers = 1
11-01T12:09:26,238][DEBUG][logstash.agent ] starting agent
11-01T12:09:26,242][DEBUG][logstash.agent ] starting pipeline {:id=>"main"}
11-01T12:09:26,650][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>125}
11-01T12:09:26,654][INFO ][logstash.pipeline ] Pipeline main started
11-01T12:09:26,676][DEBUG][logstash.agent ] Starting puma
11-01T12:09:26,682][DEBUG][logstash.agent ] Trying to start WebServer {:port=>9600}
11-01T12:09:26,688][DEBUG][logstash.api.service ] [api-service] start
11-01T12:09:26,748][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
11-01T12:09:27,038][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-01 12:09:27 -0400}
11-01T12:09:28,053][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-01 12:09:28 -0400}
11-01T12:09:29,060][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-01 12:09:29 -0400}
11-01T12:09:30,100][DEBUG][logstash.instrument.collector] Collector: Sending snapshot to observers {:created_at=>2016-11-01 12:09:30 -0400}
What am I missing?