Please help to understand output of unsucessfull Scalding run on Hadoop.
I got latest Scalding distribution from git:
git clone https://github.com/twitter/scalding.git
After sbt assembly from scalding directory I tried to run tutorial with command:
scripts/scald.rb --hdfs tutorial/Tutorial0.scala
As a result I got the following errors:
scripts/scald.rb:194: warning: already initialized constant SCALA_LIB_DIR
rsyncing 19.8M from scalding-core-assembly-0.10.0.jar to my.host.here in background...
downloading hadoop-core-1.1.2.jar from http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/1.1.2/hadoop-core-1.1.2.jar...
ssh: Could not resolve hostname my.host.here: Name or service not known
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]
Successfully downloaded hadoop-core-1.1.2.jar!
downloading commons-codec-1.8.jar from http://repo1.maven.org/maven2/commons-codec/commons-codec/1.8/commons-codec-1.8.jar...
Successfully downloaded commons-codec-1.8.jar!
downloading commons-configuration-1.9.jar from http://repo1.maven.org/maven2/commons-configuration/commons-configuration/1.9/commons-configuration-1.9.jar...
Successfully downloaded commons-configuration-1.9.jar!
downloading jackson-asl-0.9.5.jar from http://repo1.maven.org/maven2/org/codehaus/jackson/jackson-asl/0.9.5/jackson-asl-0.9.5.jar...
Successfully downloaded jackson-asl-0.9.5.jar!
downloading jackson-mapper-asl-1.9.13.jar from http://repo1.maven.org/maven2/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar...
Successfully downloaded jackson-mapper-asl-1.9.13.jar!
downloading commons-lang-2.6.jar from http://repo1.maven.org/maven2/commons-lang/commons-lang/2.6/commons-lang-2.6.jar...
Successfully downloaded commons-lang-2.6.jar!
downloading slf4j-log4j12-1.6.6.jar from http://repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.6.6/slf4j-log4j12-1.6.6.jar...
Successfully downloaded slf4j-log4j12-1.6.6.jar!
downloading log4j-1.2.15.jar from http://repo1.maven.org/maven2/log4j/log4j/1.2.15/log4j-1.2.15.jar...
Successfully downloaded log4j-1.2.15.jar!
downloading commons-httpclient-3.1.jar from http://repo1.maven.org/maven2/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar...
Successfully downloaded commons-httpclient-3.1.jar!
downloading commons-cli-1.2.jar from http://repo1.maven.org/maven2/commons-cli/commons-cli/1.2/commons-cli-1.2.jar...
Successfully downloaded commons-cli-1.2.jar!
downloading commons-logging-1.1.1.jar from http://repo1.maven.org/maven2/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar...
Successfully downloaded commons-logging-1.1.1.jar!
downloading zookeeper-3.3.4.jar from http://repo1.maven.org/maven2/org/apache/zookeeper/zookeeper/3.3.4/zookeeper-3.3.4.jar...
Successfully downloaded zookeeper-3.3.4.jar!
compiling tutorial/Tutorial0.scala
scalac -classpath /tmp/temp_scala_home_2.9.3_654763/scala-library-2.9.3.jar:/tmp/temp_scala_home_2.9.3_654763/scala-compiler-2.9.3.jar:/home/test/Cascading/scalding/scalding-core/target/scala-2.9.3/scalding-core-assembly-0.10.0.jar:/tmp/maven/hadoop-core-1.1.2.jar:/tmp/maven/commons-codec-1.8.jar:/tmp/maven/commons-configuration-1.9.jar:/tmp/maven/jackson-asl-0.9.5.jar:/tmp/maven/jackson-mapper-asl-1.9.13.jar:/tmp/maven/commons-lang-2.6.jar:/tmp/maven/slf4j-log4j12-1.6.6.jar:/tmp/maven/log4j-1.2.15.jar:/tmp/maven/commons-httpclient-3.1.jar:/tmp/maven/commons-cli-1.2.jar:/tmp/maven/commons-logging-1.1.1.jar:/tmp/maven/zookeeper-3.3.4.jar -d /tmp/script-build tutorial/Tutorial0.scala
ssh: Could not resolve hostname my.host.here: Name or service not known
rsyncing 1.5K from job-jars/Tutorial0.jar to my.host.here in background...
Waiting for 2 background threads...
ssh: Could not resolve hostname my.host.here: Name or service not known
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(600) [sender=3.0.6]
Could not rsync: /home/test/Cascading/scalding/scalding-core/target/scala-2.9.3/scalding-core-assembly-0.10.0.jar to my.host.here:scalding-core-assembly-0.10.0.jar
Could not rsync: /tmp/Tutorial0.jar to my.host.here:job-jars/Tutorial0.jar
* Update *
After changing host in scald.rb I get the followng authentication problem:
$ scripts/scald.rb --hdfs tutorial/Tutorial0.scala
scripts/scald.rb:194: warning: already initialized constant SCALA_LIB_DIR
rsyncing 19.8M from scalding-core-assembly-0.10.0.jar to node7.test.net in background...
The authenticity of host 'node7.test.net (10.1.21.32)' can't be established.
RSA key fingerprint is fa:41:31:ab:b0:46:08:8f:2b:75:0a:18:24:f9:d5:ec.
Are you sure you want to continue connecting (yes/no)? The authenticity of host 'node7.test.net (10.1.21.32)' can't be established.
RSA key fingerprint is fa:41:31:ab:b0:46:08:8f:2b:75:0a:18:24:f9:d5:ec.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node7.test.net' (RSA) to the list of known hosts.
test#node7.test.net's password: Please type 'yes' or 'no':
Permission denied, please try again.
test#node7.test.net's password:
I enter correct pathword, but the authentication error persists. How should I configure rsync?
You did change this
https://github.com/twitter/scalding/blob/develop/scripts/scald.rb#l27
right?
The default host is: my.host.here.
Related
As the title describes, choco install openvpn returns the below error;
ERROR: Unable to import PGP key 'C:\ProgramData\chocolatey\lib\openvpn\tools\openvpn_public_key.asc' in the temporary keyring (C:\ProgramData\chocolatey\lib\openvpn\tools\e090a0df-7709-xxxx-xxxx-xxxxxxxxxxxx\pubring.gpg).
The install of openvpn was NOT successful.
Error while running 'C:\ProgramData\chocolatey\lib\openvpn\tools\chocolateyInstall.ps1'.
See log for details.
Failures
- openvpn (exited -1) - Error while running 'C:\ProgramData\chocolatey\lib\openvpn\tools\chocolateyInstall.ps1'.
See log for details.
I am using a Windows 10, v1803 machine.
Below is the part of the log that shows the complete traced error;
[DEBUG] - Attempting to delete file "C:\ProgramData\chocolatey\.chocolatey\openvpn.2.4.7\.arguments".
[DEBUG] - Attempting to delete file "C:\ProgramData\chocolatey\.chocolatey\openvpn.2.4.7\.extra".
[DEBUG] - Attempting to delete file "C:\ProgramData\chocolatey\.chocolatey\openvpn.2.4.7\.version".
[DEBUG] - Attempting to delete file "C:\ProgramData\chocolatey\.chocolatey\openvpn.2.4.7\.sxs".
[DEBUG] - Attempting to delete file "C:\ProgramData\chocolatey\.chocolatey\openvpn.2.4.7\.pin".
[DEBUG] - Attempting to delete directory "C:\ProgramData\chocolatey\lib-bad\openvpn".
[DEBUG] - Sending message 'HandlePackageResultCompletedMessage' out if there are subscribers...
[ERROR] - The install of openvpn was NOT successful.
[ERROR] - Error while running 'C:\ProgramData\chocolatey\lib\openvpn\tools\chocolateyInstall.ps1'.
See log for details.
[DEBUG] - Moving 'C:\ProgramData\chocolatey\lib\openvpn'
to 'C:\ProgramData\chocolatey\lib-bad\openvpn'
[DEBUG] - Attempting to delete file "C:\Users\XXX\AppData\Local\NuGet\Cache\openvpn.2.4.7.nupkg".
[WARN ] -
Chocolatey installed 0/1 packages. 1 packages failed.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
[INFO ] -
[ERROR] - Failures
[ERROR] - - openvpn (exited -1) - Error while running 'C:\ProgramData\chocolatey\lib\openvpn\tools\chocolateyInstall.ps1'.
See log for details.
[DEBUG] - Sending message 'PostRunMessage' out if there are subscribers...
[DEBUG] - Exiting with -1
I found out the solution which is disabling the git PATH used where as it puts the version of gpg.exe shipped with git ahead of the one from openvpn package on my environment variables.
This could be done in two ways;
The first is editing the environment PATH variables and moving git PATH below the C:\Program Files (x86)\GNU\GnuPG\pub PATH, see the picture below.
The second is removing/deleting the git PATH variables then start a new powershell session, run choco install openvpn then returning the git PATH back to the env variables.
In my case, the fist option worked just fine.
PhpStorm FTP upload failed.
[17-1-16 下午5:17] Failed to transfer file '/a': cant open output connection for file "ftp://192.168.1.229:21/a". Reason: "425 Unable to build data connection: Cannot assign requested address".
[17-1-16 下午5:17] Upload to server completed in less than a minute: 108 files transferred, 3 items failed (541.1 Kb/s)
My PhpStorm running on deepin system (Linux)
net.ipv4.tcp_fin_timeout=30
net.ipv4.tcp_timestamps=1
net.ipv4.tcp_tw_recycle=1
net.ipv4.ip_local_port_range = 10000 65000
I tried to change the number of ports, but still failed to upload.
Who can help me ?
I got the same problem in IntelliJ IDEA.
Changing the connection type from FTP to SFTP helped. I hope it's also possible for you.
I am trying to setup Capifony to deploy on windows however when running cap deploy I get the following output.
Spec
ruby 2.0.0p481
capifony v2.7.0
The error message
servers: ["homestead.app"]
** [deploy:update_code] exception while rolling back: Capistrano::ConnectionError, connection failed for: homestead.app (Net::SSH::Exception: Creation of file mapping failed with error: 998) connection failed for: homestead.app (Net::SSH::Exception: Creation of file mapping failed with error: 998)
If I close down pageant this issue goes away however I require pageant to load the ssh key for the github repo as it required for doing a git ls-remote locally.
Any suggestions/workarounds?
Related issues found
https://github.com/test-kitchen/test-kitchen/issues/448
Resolved my issue by using an older version of ruby (Ruby 1.9.3-p545).
I am trying to install solaris studio 12.3 on a Solaris 11 x86 machine.
I have downloaded oracle cert and key from mentioned link, but while publishing am getting following error.
pkg set-publisher -k Oracle_Solaris_Studio.key.pem -c Oracle_Solaris_Studio.certificate.pem -g **PORTAL** solarisstudio
pkg set-publisher: The origin URIs for 'solarisstudio' do not appear to point to a valid pkg repository.
Please verify the repository's location and the client's network configuration.
Additional details:
Unable to contact valid package repository
Encountered the following error(s):
Unable to contact any configured publishers.
This is likely a network configuration problem.
1: Framework error: code: 7 reason: Failed connect to pkg.oracle.com:443; Connection refused
URL: '**PORTAL**' (happened 4 times)
2: Framework error: code: 28 reason: Connection timed out after 60000 milliseconds
URL: '**PORTAL**' (happened 4 times)
Please let me know if further information is required.
NOTE PORTAL - https://pkg.oracle.com/solarisstudio/release
Configuring correct proxy settings as pointed out by alanc, resolved the issue.
I am trying to get the following PR to view it offline.
The following address https://github.com/twitter/bootstrap/issues/5982 is accessible from every one (no login is needed) but when I run the following code
httrack https://github.com/twitter/bootstrap/issues/5982 -O ~/Sites/github/ -v
I get the following message:
mirroring https://github.com/twitter/bootstrap/issues/5982 with the wizard help..
18:28:22 Warning: Cache: damaged cache, trying to repair
18:28:22 Warning: Cache: 0 bytes successfully recovered in 0 entries
18:28:22 Warning: Cache: error trying to open the cache
18:28:22 Info: No data seems to have been transfered during this session! : restoring previous one!
Done.
Thanks for using HTTrack!