When i try to setup Apache Samza, i get nc command not found when starting zookeeper.
I'm running the command:
bin/grid bootstrap
i get
/c/..../hello-samza/bin/grid: line 170: nc: command not found
I'm writing the command in git bash console on windows 10 Education.
I'm new to samza and git bash in general. How can i fix this or is there a way to integrate nc commands in git bash basic commands?
or am i running the commands in the wrong console ?
Related
I'd like to have the ability to remote debug a Spark job written in Scala running in a docker container with VsCode. This is what I have so far,
VsCode with scala-metals v0.8 installed
This debug launch configuration
I spin up a spark cluster with docker-compose up
I submit a spark job to the cluster started above with the following command
docker exec -it -e SPARK_SUBMIT_OPTS=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=4000 -w /spark spark-job-starter_master_1 bin/spark-submit --class example.StarterSparkJob /build/example.jar
After doing so I see the output Listening for transport dt_socket at address: 4000
I attempt to launch the debugger from VsCode, but I get the error.
Debugger failed to attach: handshake failed - received >Content-Length< - expected >JDWP-Handshake<
The full details of the code are here https://github.com/aedenj/spark-job-starter/tree/vscode-debug-setup. Your guidance is appreciated.
I am trying to start Confluent 5.3.0 on Ubuntu and need some help.
The first thing I tried was navigating to confluent-5.3.0/bin and running sudo ./confluent start (because that's the way I've been starting Confluent 5.2.1).
This gave an error message: sudo: ./confluent: command not found.
I then checked the folder confluent-5.3.0/bin and found it's missing a shell script named confluent, which is included in 5.2.1.
I checked the Release Notes and the Quickstart. Both of them say I should start Confluent with confluent local start instead of confluent start. (See screenshot at bottom). However, I still get the same error message because the shell script is still missing: sudo: ./confluent: command not found
For those of you who are able to start Confluent 5.3.0, how are you doing it? Can you paste your command line here?
Did you have to write your own confluent shell script for 5.3.0?
Can I just copy my confluent shell script from 5.2.1? I'm assuming it must be incompatible, since they removed it from 5.3.0.
From 5.3.0 you will have to install CLI separately
https://docs.confluent.io/current/cli/installing.html#cli-install
curl -L https://cnfl.io/cli | sh -s -- -b /path-to-directory/bin
Then you can run
confluent local start --path <path-to-directory>
I am trying to use VSCode - Insiders to run code on a docker container in a remote AWS machine using the Remote - SSH plugin. I have opened a terminal and set up port forwarding like so: ssh -L 2201:localhost:2222 user#host -N -i ~/.ssh/id_rsa. Then in VSCode I try to connect to root#localhost and it starts up, but then gives me an error message:
> Found existing installation...
> Found running server...
>
> bash: no job control in this shell
"install" terminal command done
Received install output: bash: no job control in this shell
Failed to parse remote port from server output: bash: no job control in this shell
I started doing this process a couple days ago and it worked. Yesterday it was in and out a bit, and today it's not working at all. I've tried turning it off and on again, but can't get it to work. In case it's relevant, I am on MacOS with the Mojave OS.
Edit:
Magically, it worked today (the following day) the first time. I would still be interested in knowing how to fix this next time it breaks. In case this helps, here's the output from when it is working:
SSH Resolver called for "ssh-remote+7b22686..."
SSH Resolver called for host: root#localhost
Setting up SSH remote "localhost"
Using commit id "473af338..." and quality "insider" for server
Using SSH config file "/Users/user/config"
Install and start server if needed
> Found existing installation...
> Found running server...
>
> bash: no job control in this shell
> 368805d0-03...==38466==
"install" terminal command done
Received install output: 368805d0-03...==38466==
Server is listening on port 38466
Using SSH config file "/Users/user/config"
Spawning tunnel with: ssh -F /Users/user/config root#localhost -N -L localhost:39003:localhost:38466
Spawned SSH tunnel between local port 39003 and remote port 38466
Waiting for ssh tunnel to be ready
Tunneling remote port 38466 to local port 39003
Resolving "ssh-remote+7b22686f737..." to "localhost:39003", attempt: 1
Edit 2: And now (the following following day) it's not working again.
Edit 3: I have a config file at ~/config. Here are the contents:
Host *
User root
Port 2201
IdentityFile ~/id_rsa
In the specific implementation shown above, you have User root in your config and are logging in with root#localhost, so you have your username twice. Leave the config file as is and just enter localhost in VSCode. This still doesn't solve the instability issue, but it does fix one problem.
I have the same issue when configuring my server. It solved by this issue. After save your config file for remote server, change the remote shell path like this issue, and then connect, you will in.
https://github.com/microsoft/vscode-remote-release/issues/220#issuecomment-490374437
How to run kafka rest proxy on windows.
I downloaded confluent-2.0.1-2.11.7.tar.gz
in windows folder i cannot see kafka-rest-start.
Windows isn't currently a supported platform. However, it should work fine if you adapt the script. Even just running java io.confluent.kafkarest.KafkaRestMain with the appropriate classpath should work.
Here's the example of the command they are actually executing in the end of the bash script:
java -Xmx256M -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dlog4j.configuration=file:C:/Dev/kafka/confluent-4.0.0/etc/kafka-rest/log4j.properties -cp .;C:/Dev/kafka/confluent-4.0.0/target/kafka-rest-*-development/share/java/kafka-rest/*;C:/Dev/kafka/confluent-4.0.0/share/java/confluent-common/*;C:/Dev/kafka/confluent-4.0.0/share/java/rest-utils/*;C:/Dev/kafka/confluent-4.0.0/share/java/kafka-rest/* io.confluent.kafkarest.KafkaRestMain C:/Dev/kafka/confluent-4.0.0/etc/kafka-rest/kafka-rest.properties
Make sure you change paths to yours if you want to try it out.
Perhaps, this answer will help anybody who is new to Kafka and stumble upon this situation, like me :).
I was looking for an answer to the very same question a week ago, came across the official suggestion to run jar files(in this path confluent-x.x.x\share\java\kafka-rest) in windows and was NOT successful in doing so.
Always ran into this error no main attribute found with or without specifying the proper classpath and io.confluent.kafkarest.KafkaRestMain.
I even tried running the shell scripts packaged for Linux distribution using [babun]: http://babun.github.io/, but that resulted in the error like Error: Could not find or load main class io.confluent.kafkarest.KafkaRestMain .
Eventually, docker image built with zookeeper, kafka, schema-registry, kafka-rest worked like a charm.
Here is the official page with info about the image name, further reference to it's doc: https://hub.docker.com/r/confluentinc/cp-kafka-rest/
Upon pulling this image, a new virtual machine gets created with four more images inside it(one for each service like zookeeper, Kafka, schem-registry and Kafka-rest). Running the images runs a separate Docker container.
This guide should get you started quickly:
http://docs.confluent.io/current/cp-docker-images/docs/quickstart.html
And finally, if you would like to expose the kafka REST proxy server running as a Docker container to outside network(like windows machine which is part of the separate network than these containers) just mention the Docker host IP(find it by hitting docker-machine ip <hostname>) in KAFKA_REST_LISTENERS and expose the port with -p option.
Like this:
docker run -d \
--net=host \
--name=kafka-rest \
-p 8082:8082 \
-e KAFKA_REST_ZOOKEEPER_CONNECT=localhost:32181 \
-e KAFKA_REST_LISTENERS=http://192.168.99.100:8082 \
-e KAFKA_REST_SCHEMA_REGISTRY_URL=http://localhost:8081 \
-e KAFKA_REST_HOST_NAME=localhost \
confluentinc/cp-kafka-rest:3.2.1
If everything is OK, you will be able to access REST proxy at this url http://<Docker_host_IP>:8082 from the windows machine.
I was able to run the command that #lexler mentioned above, but outside of cygwin. (directly with the windows command prompt.)
Am a newbie in Deployd and MongoDB. I have installed Deployd (www.deployd.com) 0.6.9 on my Windows-XP system. I executed the following command at the prompt as instructed by the book am studying
>dpd create sportsstore
Now, when i run the following command
> dpd –p 5500 sportsstore/app.dpd
to start the Deployd server, i get the following error::
starting deployd v0.6.9...
Failed to start MongoDB
It states that MongoDb has failed to start. I went into the directory "C:\Program Files\Deployd\tools" and found that there is a file called "mongod".
I have never installed MongoDB before on my computer. It is my first time working with Deployd so i don't know whether the "Mongod" file in the tools folder is the same as MongoDB or i have to install MongoDB separately on the computer.
Can someone point me to the right direction.
Thanks.
First of all, you have to install mongodb in your computer. You can do it from the official site for MongoDB. The standard option would be to install the Community version.
After that, you can check if mongodb was properly installed and can be used with Deployd:
If you are in iOS or Linux you can try:
sudo dpd
If you are in Windows try opening a command window with "Run as Administrator" and try:
dpd
For Windows Installation.
1) Run npm install deployd -g
2) Install Mongo DB separatetly using windows installer available on
https://www.mongodb.com/download-center#community
and configure to run it as service as mentioned on below url.
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-windows/
Service option is convenient to start and stop database using net start and net stop command
3) create a deployd module using 'dpd create abcd'.
4) navigate to deployd module you have created (using cd abcd) to run dpd -d command. Before you run dpd -d command you need to ensure the you start the mongodb service from same command prompt running in elevated mode or admin mode.( use the command : 'net start MongoDB')
In case It is still giving path error then try below command:
set path=%PATH%; "C:\Program Files\MongoDB\Server\3.4.1\bin"
this command is alternative for setting path in environment variable (in case user doesn't have permissions to modify environment variable)
Still not working, same path issue ??
run using command --mongod as mentioned below
dpd -m "C:\Program Files\MongoDB\Server\3.4.1\bin\mongod.exe"
After installed the depolyd run:
dpd -e production
To solve the problem of "failed to start mongoDB"
1- I installed mongoDB for windows from this web page (https://docs.mongodb.com/manual/installation/).
2- Then I created a path as shown below
path
3 - I installed the deployd and I run:
dpd -e production