After installing pig, I can't call hadoop from the command line anymore.
ex:
C:\Users\Andrew\Documents\Hadoop\Karung>hadoop version
'hadoop' is not recognized as an internal or external command,
operable program or batch file.
Here's a look at my Path:
c:\Program Files (x86)\Intel\iCLS Client\;c:\Program Files\Intel\iCLS Client\;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Windows Live\Shared;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x86;C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x64;C:\Python27;C:\Python27\Scripts;C:\Python27\Lib\site-packages;C:\Program Files (x86)\Git\cmd;C:\Program Files (x86)\Git\bin\;C:\Users\Andrew\Documents\bat_files;C:\Users\Andrew\Documents\Hadoop\hadoop-0.23.8\bin;C:\Users\Andrew\Documents\Pig\pig-0.11.1\bin
Also I have the Hadoop, Pig, and Java install variables like so:
HADOOP_INSTALL C:\Users\Andrew\Documents\Hadoop\hadoop-0.23.8
PIG_INSTALL C:\Users\Andrew\Documents\Pig\pig-0.11.1
JAVA_HOME C:\Program Files\Java\jdk1.7.0_21
Everything seems to be setup correctly. They all execute when I run the commands from their directories. Why can't the command line recognize hadoop? Thanks!
Short answer
hadoop is a bash script. The Windows command-line doesn't know how to run this.
Long answer
You're going to need a lot more set up if you want to run Hadoop on Windows. There are a number of tutorials out there, but honestly your best bet is to run it on a unix machine or VM.
Installing Hadoop on Windows
Cloudera QuickStart VM
If you are just targeting windows, You could even run Hadoop and Pig on top of cygwin or mingw to avoid the hassle of workarounds and problems or better to use Hortonworks Data Platform or HDP that could be installed in windows directly.
Related
I am trying to set up my virtual environment to run Python, Node.js, Conda, and Docker. While I am able to run these programs in the Command Prompt or PowerShell, for some reason, when I try them on the virtual environment terminal, none of the commands are recognized, even though they are in my PATH.
I restarted my computer after modifying the path.
In Visual Studio Code, in the developer tools, I used process.env.path and obtained this result.
'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.8\\bin;C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.8\\libnvvp;C:\\Program Files\\Common Files\\Oracle\\Java\\javapath;C:\\Program Files (x86)\\Razer Chroma SDK\\bin;C:\\Program Files\\Razer Chroma SDK\\bin;C:\\Program Files (x86)\\Razer\\ChromaBroadcast\\bin;C:\\Program Files\\Razer\\ChromaBroadcast\\bin;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files\\dotnet\\;C:\\Program Files\\Git\\cmd;C:\\Program Files\\NVIDIA Corporation\\NVIDIA NvDLISR;C:\\Program Files (x86)\\NVIDIA Corporation\\PhysX\\Common;C:\\rtools40\\qpdf-10.3.2\\bin;C:\\Windows\\system32\\config\\systemprofile\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Program Files\\NVIDIA Corporation\\Nsight Compute 2022.3.0\\;C:\\Program Files\\Docker\\Docker\\resources\\bin;C:\\Program Files\\nodejs\\;C:\\Users\\SSosa\\AppData\\Local\\Programs\\Python\\Python311\\Scripts\\;C:\\Users\\SSosa\\AppData\\Local\\Programs\\Python\\Python311\\;C:\\Users\\SSosa\\AppData\\Local\\Programs\\MiKTeX\\miktex\\bin\\x64\\;C:\\Users\\SSosa\\AppData\\Roaming\\TinyTeX\\bin\\win32;C:\\Users\\SSosa\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Program Files\\JetBrains\\PyCharm 2021.2.2\\bin;C:\\Program Files\\JetBrains\\PyCharm Community Edition 2021.2.2\\bin;C:\\rtools40\\qpdf-10.3.2\\bin;C:\\Users\\SSosa\\AppData\\Local\\GitHubDesktop\\bin;C:\\Users\\SSosa\\Downloads\\OneCommander3.4.16.0;C:\\Program Files\\JetBrains\\PyCharm Community Edition 2022.2.4\\bin;;C:\\Users\\SSosa\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\Users\\SSosa\\AppData\\Local\\Keybase\\;C:\\Users\\SSosa\\AppData\\Roaming\\npm;C:\\Program Files\\Docker\\Docker\\resources\\bin;'
How can I fix this issue?
Thanks.
I have generated an install4j Windows installer. I want the installer to run on a remote Windows machine and have to debug the installation process. The installed JVM is 1.8_161. I have tried several option in my vmoptions file:
-Xdebug
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9999
or
-Xdebug
-Xnoagent
-Djava.compiler=NONE
-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9999
Neither configuration works. The vmoptions' file name is the same as the media file's name except the extension.
I hadn't any problems with debugging earlier remote installers.
Best regards
anko
The installer does not read a .vmoptions file for security reasons, because installers are often saved in writable locations when they are downloaded.
You have to pass -J-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9999 on the command line (note the -J prefix)
This question already has answers here:
How can I use PowerShell with the Visual Studio Command Prompt?
(15 answers)
Closed 1 year ago.
I have Visual Studio 9.0 installed but I want to use it manually from PowerShell. It comes with two setup scripts: vcvars32.bat for the 32-bit compiler and vcvars64.bat for the 64-bit compiler. When I open cmd.exe and run one of the scripts, it sets up everything just fine and I can run cl.exe without any problems. When I run one of those setup scripts from PowerShell, though, it doesn't work. The scripts run through fine but trying to run cl.exe afterwards yields a "cl.exe could not be found" error! And looking at the contents of the PATH environment variable after running one of the setup scripts I can see that PATH hasn't actually been modified at all.
So it seems as if the batch files ran from PowerShell maintain their own environment variables state which goes away as soon as the batch file terminates. So is there a way to run batch files from PowerShell and have those batch files affect the actual environment variables of the current PowerShell session? Because that is what I need. All that is done by vcvars32.bit and vcvars64.bit is setting up environment variables after all but it only seems to work from cmd.exe, not from PowerShell.
You should use InvokeEnvironment script to do that. Check its man page:
Invoke-Environment <path_to_>vsvars32.bat
You can furhter generalize this by determining OS bits and crafting the vsvars<OsBits>.bat.
Example:
PS C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools> $env:INCLUDE -eq $null
PS C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools> $true
PS C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools> Invoke-Environment .\vsvars32.bat
PS C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools> $env:INCLUDE
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE;C:\Program Files (x86)\Windows Kits\10\include\10.0.10586.0\ucrt;C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um;C:\Program Files (x86)\Windows Kits\10\include\10.0.10586.0\shared;C:\Program Files (x86)\Windows Kits\10\include\10.0.10586.0\um;C:\Program Files (x86)\Windows Kits\10\include\10.0.10586.0\winrt;
I don't have Visual Studio at hand, but the batch scripts most likely just set variables for the current session. Running them from PowerShell won't do you any good, because they'll be launched in a child CMD process and change the process environment of that process, but not of the parent (PowerShell) process.
I suspect you need to translate the variable definitions to PowerShell, e.g.
set PATH=%PATH%;C:\some\where
set FOO=bar
becomes
$env:Path += ';C:\some\where'
$env:FOO = 'bar'
Write the translated definitions to a .ps1 file and dot-source that file in your PowerShell session:
. C:\path\to\vcvars.ps1
Every time I install a new program that uses command prompt it still does not recognize it. For example, I have installed git and nodejs and added them to the %PATH%, but they are still unrecognizable in my command prompt.
%Path% = C:\Program Files (x86)\WANdisco\Subversion;C:\Program Files\Common Files\Microsoft Shared\Windows Live;C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Live;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Windows Live\Shared;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files (x86)\Microsoft ASP.NET\ASP.NET Web Pages\v1.0\;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files (x86)\Windows Kits\8.0\Windows Performance Toolkit\;C:\Program Files (x86)\Git\cmd;C:\Program Files (x86)\nodejs
If any other system variables need to be posted, let me know! Thanks in advance for the help!
Don't use
%Path% =
Use this:
Path=
and it will be modified for the scope of the batch file.
To make it global so it is available in every cmd prompt you can use SETX or modify the Windows environment variables in control panel, system, advanced.
I have a 64bit Win 7, installed the 64 bit canopy.
The first part installs fine, but if I try to launch canopy to set up the environment from the start menu (or from the desktop icon or "C:\Program Files\Enthought\Canopy\App\Canopy.exe" which is where they point) absolutely nothing happens.
This seems to be the same problem that has been reported a few times before:
Canopy Does not Start on Windows 7 despite successful installation
Install 32 bit Canopy on Win 7, 32-bit machine fails
Enthought canopy not running
but none of these questions have been resolved. What they all have in common is that they relate to Windows 7.
These questions are followed up by a request for the log:
"C:\Users\\AppData\Roaming\Enthought\Canopy\log\"
However there are no logs, there isn't even a
C:\Users\\AppData\Roaming\Enthought directory.
However I have found that if I go into:
C:\Program Files\Enthought\Canopy\App\appdata\canopy-1.1.0.1371.win-x86_64
and run the copy of Canopy.exe in there it allows me to setup the environment and run canopy from within this subdirectory (the desktop and start menu links still fail) . This also creates the
C:\Users\\AppData\Roaming\Enthought directory and logs.
BUT I am worried if I do this I will cause problems for myself further down the line. Can someone tell me why the canopy.exe linked to in my start menu isn't working?
I do have a previous installation of WinPython 2.7 in
C:\Python27\WinPython-64bit-2.7.5.3
thanks
Susie
Existing PYTHONHOME and/or PYTHONPATH environment variables may be interfering with Canopy startup. For details, please see:
https://support.enthought.com/entries/22508619-Windows-Canopy-shortcut-in-Start-menu-doesn-t-do-anything
Thanks for the report and pointers, Susie. No, you have not created any problem for yourself.
To help us get to the bottom of this, could you be so kind as to answer these questions?:
1) What is the full Target of the non-working shortcut? (Right Click / Properties / Shortcut tab / Target)
2) Can you start Canopy by running this?:
C:\Program Files\Enthought\Canopy\App\Canopy.exe
3) In the light of your previous Python installation, what go you see if you type “PATH” at a command prompt?
1) What is the full Target of the non-working shortcut? (Right Click / Properties / Shortcut tab / Target)
The non-working shortcut target is the filepath you give below:
C:\Program Files\Enthought\Canopy\App\Canopy.exe
2) Can you start Canopy by running this?:
C:\Program Files\Enthought\Canopy\App\Canopy.exe
No. I get about a half second of the spinning 'working' icon on my mouse pointer then stops.
If I run it from the command line it skips straight back out to the next line.
3) In the light of your previous Python installation, what go you see if you type “PATH” at a command prompt?
(formatted)
PATH=
C:\Program Files (x86)\AMD APP\bin\x86_64;
C:\Program Files (x86)\AMD APP\bin\x86;
C:\Perl64\site\bin;C:\Perl64\bin;
C:\Program Files\ImageMagick-6.6.7-Q16;
C:\Program Files\Common Files\Microsoft Shared\Windows Live;
C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Live;
C:\windows\system32;
C:\windows;
C:\windows\System32\Wbem;
C:\windows\System32\WindowsPowerShell\v1.0\;
C:\Program Files (x86)\ATI Technologies\ATI.ACE\Core-Static;
C:\Program Files\Broadcom\Broadcom 802.11 Network Adapter\Driver;
C:\Program Files (x86)\Windows Live\Shared;
C:\Program Files\WIDCOMM\Bluetooth Software\;
C:\Program Files\WIDCOMM\Bluetooth Software\syswow64;
C:\Program Files (x86)\Java\jre6\bin;
C:\Program Files\MPICH2\bin;
D:\College\CVsoft\bundler\bin;
D:\College\CVsoft\bundler\pmvs2_64;
D:\College\CVsoft\Win64-VS2010;
C:\Program Files (x86)\Autodesk\Backburner\;
C:\QtSDK\mingw\bin;
C:\QtSDK\QtCreator\bin;
C:\MinGW\bin;
D:\College\CVsoft\APERO\micmac;
C:\Program Files (x86)\QuickTime\QTSystem\;
C:\Program Files\OpenNI\Bin64;
C:\users\susie\appdata\local\enthought\canopy\user\scripts;
C:\Users\susie\AppData\Local\Enthought\Canopy\User;
C:\Users\susie\AppData\Local\Enthought\Canopy\User\Scripts