Hadoop on windows with eclipse - eclipse

I read a post here on stack overflow about this link being very good for hadoop deployment on windows - http://v-lad.org/Tutorials/Hadoop/12%20-%20format%20the%20namendoe.html
Problem is when I format the namenode as given on that page, after
bin/hadoop namenode -format
I get the following error:
Maybe its a problem with my environmental variables but I'm not sure.
bin/hadoop: line 330: C:\Program: command not found
bin/hadoop: line 395: C:\Program Files\java\jdk1.7.0\bin/bin/java: No such
file/directory
bin/hadoop: line 395: exec: C:\Program Files\java\jdk1.7.0\bin/bin/java: cannot
execute: No such file or directory

Just set you JAVA_HOME properly, it resolved my issue-
anwar#dell-pc ~/hadoop-0.20.203.0
$ export JAVA_HOME="C:\Program Files\Java\jdk1.6.0_02"

Related

How to solve error on docker:layers_calculator to compute the Merkle tree on private tangle?

I want to setup a private tangle on my own virtual machine with Ubuntu 18.04, 4GB RAM and 20GB memory.
I have follow this instructions: https://docs.iota.org/docs/compass/0.1/how-to-guides/set-up-a-private-tangle. Every command works fine until reach this one: bazel run //docker:layers_calculator.
It shows an error as follows:
Starting local Bazel server and connecting to it...
ERROR: /home/istabraq/compass/third-party/maven_deps.bzl:3:5: Traceback (most recent call last):
File "/home/istabraq/compass/WORKSPACE", line 42
maven_jars()
File "/home/istabraq/compass/third-party/maven_deps.bzl", line 3, in maven_jars
native.maven_jar(<4 more arguments>)
type 'struct' has no method maven_jar()
ERROR: error loading package '': Encountered error while reading extension file 'protobuf_deps.bzl': no such package '#com_google_protobuf_deps//': error loading package 'external': Could not load //external package
ERROR: error loading package '': Encountered error while reading extension file 'protobuf_deps.bzl': no such package '#com_google_protobuf_deps//': error loading package 'external': Could not load //external package
INFO: Elapsed time: 4.743s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
How can I solve this problem? what I have missed?
read carefully the message given after running bazel installer:
Make sure you have "/home/yourusername/bin" in your path. You can also activate bash completion by adding the following line to your :
source /home/yourusername/.bazel/bin/bazel-complete.bash
You can check with: "bazel info" or "bazel version"
Unfortunately, there are further errors:
https://github.com/iotaledger/compass/issues/142
I have solve this issue by using this commands:
Step 3: Set up your environment
If you ran the Bazel installer with the --user flag as above, the Bazel executable is installed in your $HOME/bin directory. It’s a good idea to add this directory to your default paths, as follows:
export PATH="$PATH:$HOME/bin"
You can also add this command to your ~/.bashrc or ~/.zshrc file to make it permanent.
reference:
https://docs.bazel.build/versions/master/install-ubuntu.html

Creating virtualenv inside veracypt error

I'm setting up a project inside veracrypt and it's throwing this error when I try to setup the environment.
admin#kali:/media/veracrypt1$ virtualenv --python=python3 venv
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /media/veracrypt1/venv/bin/python3
Also creating executable in /media/veracrypt1/venv/bin/python
Traceback (most recent call last):
File "/usr/local/bin/virtualenv", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/dist-packages/virtualenv.py", line 870, in main
symlink=options.symlink,
File "/usr/local/lib/python3.7/dist-packages/virtualenv.py", line 1162, in create_environment
install_python(home_dir, lib_dir, inc_dir, bin_dir, site_packages=site_packages, clear=clear, symlink=symlink)
File "/usr/local/lib/python3.7/dist-packages/virtualenv.py", line 1672, in install_python
os.symlink(py_executable_base, full_pth)
PermissionError: [Errno 1] Operation not permitted: 'python3' -> '/media/veracrypt1/venv/bin/python'
I've tried to look for the source of the issue and it seems it's related to how it's a virtualdrive with limited rights
admin#kali:/media/veracrypt1$ ln -s testfile
ln: failed to create symbolic link './testfile': Operation not permitted
Looks like you are running this in an environment with limited permissions.
Some report this behavior when running on Linux,
but in a folder that is mounted to a "FAT32" partition -
see Chris Lope's blog-post:
permissionerror: [errno 1] operation not permitted
I have experienced this behavior while running in an Ubuntu VM
in a folder that was mounted to the host-OS (Windows-NTFS) as type 'vboxsf'.
Solved it by moving to work in a partition that is native Unix.

Run QEMU from within Eclipse External tools

I'd like to ask since I am setting up the Eclipse IDE for Yocto app development and I got stuck to start the QEMU from within Eclipse.
I got working QEMU image OK such as
ubuntu#ubuntu:~/work/community/build-x11$ runqemu qemuarm
tmp/deploy/images/qemuarm/zImage-qemuarm.bin
tmp/deploy/images/qemuarm/fsl-image-multimedia-full-qemuarm.ext4
Within the Eclipse I follow
https://www.yoctoproject.org/docs/2.5/sdk-manual/sdk-manual.html#oxygen-starting-qemu-in-user-space-nfs-mode
But by configuring "External Tools" and try to run QEMU I got following
runqemu - INFO - Running MACHINE=qemuarm bitbake -e...
ERROR: Unable to find conf/bblayers.conf or conf/bitbake.conf. BBAPTH is unset and/or not in a build directory?
runqemu - WARNING - Couldn't run 'bitbake -e' to gather environment information:
runqemu - WARNING - Can't find qemuboot conf file, DEPLOY_DIR_IMAGE is NULL!
runqemu - INFO - Running MACHINE=qemuarm bitbake -e...
ERROR: Unable to find conf/bblayers.conf or conf/bitbake.conf. BBAPTH is unset and/or not in a build directory?
runqemu - WARNING - Couldn't run 'bitbake -e' to gather environment information:
runqemu - INFO - Setting STAGING_DIR_NATIVE to OECORE_NATIVE_SYSROOT (/home/ubuntu/work/community/build-x11/tmp/work/armv5e-fslc-linux-gnueabi/meta-ide-support/1.0-r3/recipe-sysroot-native)
runqemu - INFO - Setting STAGING_BINDIR_NATIVE to /home/ubuntu/work/community/build-x11/tmp/work/armv5e-fslc-linux-gnueabi/meta-ide-support/1.0-r3/recipe-sysroot-native/usr/bin
runqemu - INFO - QB_MEM is not set, use 512M by default
runqemu - INFO - Continuing with the following parameters:
KERNEL: [/home/ubuntu/work/community/build-x11/tmp/deploy/images/qemuarm/zImage-qemuarm.bin]
MACHINE: [qemuarm]
FSTYPE: [nfs]
NFS_DIR: [/home/ubuntu/work/community/build-x11/MY_QEMU_ROOTFS]
CONFFILE: []
/bin/sh: 1: stty: not found
Traceback (most recent call last):
File "/home/ubuntu/work/community/sources/poky/scripts/runqemu", line 1270, in main
config.setup_network()
File "/home/ubuntu/work/community/sources/poky/scripts/runqemu", line 997, in setup_network
self.saved_stty = subprocess.check_output("stty -g", shell=True).decode('utf-8')
File "/usr/lib/python3.5/subprocess.py", line 626, in check_output
**kwargs).stdout
File "/usr/lib/python3.5/subprocess.py", line 708, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'stty -g' returned non-zero exit status 127
Cleanup
Command 'lesspipe' is available in the following places
* /bin/lesspipe
* /usr/bin/lesspipe
The command could not be located because '/bin:/usr/bin' is not included in the PATH environment variable.
lesspipe: command not found
Command 'dircolors' is available in '/usr/bin/dircolors'
The command could not be located because '/usr/bin' is not included in the PATH environment variable.
dircolors: command not found
ubuntu#ubuntu:~/eclipse/cpp-oxygen/eclipse$
I wonder if any has experience such a problem on setting it up "External Tools" with Eclipse?
Thank you
Navigate to the build path and then trigger this command will work.
OR
Run source oe-init-build-env build path.

Cannot find NetLogo model through command line

I'm trying to get to grips with command line operations of NetLogo on a Windows 10 machine. I want to run the Fire.nlogo model provided.
I set the directory with cd C:\Program Files\NetLogo 6.0.2
Then I try to run a simple experiment called experiment1 which I've written beforehand in BehaviourSpace
netlogo-headless --model Fire.nlogo --experiment experiment1
This gives me the following error:
Exception in thread "main" java.io.FileNotFoundException: C:\Program Files\NetLogo 6.0.2\Fire.nlogo (The system cannot find the file specified)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at scala.io.Source$.fromFile(Source.scala:91)
at scala.io.Source$.fromFile(Source.scala:76)
at scala.io.Source$.fromURI(Source.scala:121)
at org.nlogo.fileformat.AbstractNLogoFormat.$anonfun$sections$1(NLogoFormat.scala:37)
at scala.util.Try$.apply(Try.scala:209)
at org.nlogo.fileformat.AbstractNLogoFormat.sections(NLogoFormat.scala:36)
at org.nlogo.fileformat.AbstractNLogoFormat.sections$(NLogoFormat.scala:34)
at org.nlogo.fileformat.NLogoFormat.sections(NLogoFormat.scala:16)
at org.nlogo.api.ModelFormat.load(ModelFormat.scala:53)
at org.nlogo.api.ModelFormat.load$(ModelFormat.scala:51)
at org.nlogo.fileformat.NLogoFormat.load(NLogoFormat.scala:16)
at org.nlogo.api.FormatterPair.load(ModelLoader.scala:26)
at org.nlogo.api.ModelLoader.readModel(ModelLoader.scala:60)
at org.nlogo.api.ModelLoader.readModel$(ModelLoader.scala:57)
at org.nlogo.api.ConfigurableModelLoader.readModel(ModelLoader.scala:90)
at org.nlogo.headless.HeadlessWorkspace.open(HeadlessWorkspace.scala:491)
at org.nlogo.headless.Main$.newWorkspace$1(Main.scala:18)
at org.nlogo.headless.Main$.runExperiment(Main.scala:21)
at org.nlogo.headless.Main$.$anonfun$main$1(Main.scala:12)
at org.nlogo.headless.Main$.$anonfun$main$1$adapted(Main.scala:12)
at scala.Option.foreach(Option.scala:257)
at org.nlogo.headless.Main$.main(Main.scala:12)
at org.nlogo.headless.Main.main(Main.scala)
I notice that the output gives the path as C:\Program Files\NetLogo 6.0.2\Fire.nlogobut the model is actually located at C:\Program Files\NetLogo 6.0.2\app\models\Sample Models\Earth Science\Fire.nlogo
Though I seem to be following the tutorials as they're written here https://ccl.northwestern.edu/netlogo/docs/behaviorspace.html
Any ideas where I'm going wrong here? Thanks.
A quick look suggests that you need to give the full file path to the --model argument. So the command would look like:
netlogo-headless --model "C:\Program Files\NetLogo 6.0.2\app\models\Sample Models\Earth Science\Fire.nlogo" --experiment experiment1
Since you have set cd C:\Program Files\NetLogo 6.0.2 you can probably go with
netlogo-headless --model "app\models\Sample Models\Earth Science\Fire.nlogo" --experiment experiment1
Alternatively, you can go to the directory that contains the model you want to run and instead provide the path (again with quotes) to the .bat file
"c:\Program Files\NetLogo 6.0.2\netlogo-headless.bat" --model Fire.nlogo --experiment experiment1

breaking on unhandled exceptions in pydev/gae

i am using pydev to develop a google app engine application. i followed the steps mentioned here to configure pydev debugger to break on unhandled exception. i could get it to work on a sample pydev project, but when i try the same steps in my pydev gae project, it doesn't work and gives following error:
pydev debugger: warning: psyco not
available for speedups (the debugger
will still work correctly, but a bit
slower) pydev debugger: starting
...
Traceback (most recent call last):
File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 3858, in _HandleRequest
self._Dispatch(dispatcher, self.rfile, outfile, env_dict) File
"c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 3792, in _Dispatch
base_env_dict=env_dict) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 580, in Dispatch
base_env_dict=base_env_dict) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 2918, in Dispatch
self._module_dict) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 2822, in ExecuteCGI
reset_modules = exec_script(handler_path, cgi_path,
hook) File "c:\program
files\google\google_appengine\google\appengine\tools\dev_appserver.py",
line 2702, in ExecuteOrImportScript
exec module_code in script_module.dict File
"C:\Users\siddjain\workspace\rfad\src\main.py",
line 1, in
import pydevd ImportError: No module named pydevd
my debug configuration for gae project is like this:
the sample pydev project where it works is like this and am following same pattern in my gae project:
import pydevd
def f(x,y):
z = y/x;
return z;
def main():
pydevd.set_pm_excepthook()
print f(0,0)
if __name__ == '__main__':
main()
the run config for test project is like this:
the pydevd.py module is under C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc. Although this path is not included in the pythonpath for test project, the breaking works in test. i also tried including this path in pythonpath of gae project to see if that fixes my problem, but it didn't
Its still not fixed although following steps got rid of No module named pydevd error:
1. Create a symlink to C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc:
src>mklink /d debugger C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc
2. Put an empty __init__.py file in C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc. i learnt this from some google link that i can't find now.
3. In source code:
import debugger.pydevd as pydevd
after these steps its able to import pydevd, but still doesn't break on uncaught exception.
The development server runs your application on your local computer for testing your application. The server simulates the App Engine datastore, services and sandbox restrictions (hence step 1). although i don't understand why we need to import pydevd since its pydevd who's running our application in the first place!
In the Run->Debug Configurations->Interpreter tab if I click on "see resulting command line..."
C:\Python25\python.exe -u
C:\eclipse\plugins\org.python.pydev.debug_2.0.0.2011040403\pysrc\pydevd.py
--vm_type python --client 127.0.0.1 --port 0 --file "c:\program
files\google\google_appengine\dev_appserver.py"
The PYTHONPATH that will be used is:
C:\eclipse\plugins\org.python.pydev_2.0.0.2011040403\PySrc\pydev_si...