OSRM Extract Silently Fails - osrm

I'm trying to setup a OSRM backend server using Docker. The host machine is a MacBook Pro with 32GB RAM running OS 10.14 (Mojave).
Having downloaded the england-latest.osm.pbf file I've tried to start the process with the following command.
docker run -t -v $(pwd):/data osrm/osrm-backend osrm-extract -p /opt/car.lua /data/england-latest.osm.pbf
The output is as follows:
[info] Parsed 0 location-dependent features with 0 GeoJSON polygons
[info] Using script /opt/car.lua
[info] Input file: england-latest.osm.pbf
[info] Profile: car.lua
[info] Threads: 6
[info] Parsing in progress..
[info] input file generated by osmium/1.8.0
[info] timestamp: 2018-11-02T21:15:02Z
[info] Using profile api version 4
[info] Found 3 turn restriction tags:
[info] motorcar
[info] motor_vehicle
[info] vehicle
[info] Parse relations ...
[info] Parse ways and nodes ...
[info] Using profile api version 4
[info] Using profile api version 4
[info] Using profile api version 4
[info] Using profile api version 4
[info] Using profile api version 4
At this point the process silently fails and doesn't write any of the files you'd expect. If I use the greater-london-latest.osm.pbf file then everything works fine so I'm guessing it's some sort of memory constraint. How do I go about identifying the problem and fixing it?

After running into the same issue on my Mac, I solved it by increasing the memory available to Docker. Look for the whale icon on the menubar, click then select Preferences > Advanced. Change the slider to a higher amount, press Apply & Restart, then try running again.
https://docs.docker.com/docker-for-mac/#memory
I could run within the 2GB default limit with a 415MB file (Canada > British Columbia) but nothing larger. Given the England file is 422MB, you can probably increase to 4GB and have it work. (Trying to run for all of Canada, a 2.4GB file, needed >8GB of RAM.)

Related

Failures when running Keycloak Testsuite with external Keycloak server

I am trying to run the keycloak Testsuite against an external Keycloak server that I have created.
I am using the base tests in the integration-arquillian using the following commands
mvn -f testsuite/integration-arquillian/tests/base/pom.xml clean install --log-file My_testsuite_integration_logs06.txt -Pauth-server-wildfly -Dauth.server.ssl.required=false -Dpageload.timeout=3600000 -Dauth.server.host={my-server-details} -Dauth.server.http.port={port#}
It works when I am using the embedded tests, but when I add the server details as stated in the HOW-TO-RUN.md file its failing.
https://github.com/keycloak/keycloak/tree/stage/testsuite/integration-arquillian
========
[INFO] Running org.keycloak.testsuite.account.AccountFormServiceTest
11:56:38,313 ERROR [org.keycloak.testsuite.account.AccountFormServiceTest] [AccountFormServiceTest] null() FAILED
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.003 s <<< FAILURE! - in org.keycloak.testsuite.account.AccountFormServiceTest
[ERROR] org.keycloak.testsuite.account.AccountFormServiceTest Time elapsed: 0.003 s <<< ERROR!
java.lang.RuntimeException: Arquillian initialization has already been attempted, but failed. See previous exceptions for cause
at org.jboss.arquillian.junit.AdaptorManagerWithNotifier.handleSuiteLevelFailure(AdaptorManagerWithNotifier.java:36)
at org.jboss.arquillian.junit.AdaptorManager.initializeAdaptor(AdaptorManager.java:16)
at org.jboss.arquillian.junit.AdaptorManagerWithNotifier.initializeAdaptor(AdaptorManagerWithNotifier.java:19)
at org.jboss.arquillian.junit.Arquillian.run(Arquillian.java:109)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
Caused by: org.jboss.arquillian.container.spi.client.container.LifecycleException: The java process starting the managed server exited unexpectedly with code [2]
at org.jboss.as.arquillian.container.managed.ManagedDeployableContainer.startInternal(ManagedDeployableContainer.java:152)
at org.jboss.as.arquillian.container.CommonDeployableContainer.start(CommonDeployableContainer.java:123)
...
==
Getting similiar to above for all my tests and not sure why it is failing.
any help be great.
Your description of the issue is quiet vague, but have you tried
mvn -f testsuite/integration-arquillian/tests/base/pom.xml clean install -Pauth-server-remote -Dauth.server.ssl.required=false Dpageload.timeout=3600000 -Dauth.server.host={my-server-details} -Dauth.server.http.port={port#}
The Profile defined could be where the issue lies.
If you take a look at the documentation https://github.com/keycloak/keycloak/blob/stage/testsuite/integration-arquillian/HOW-TO-RUN.md#remote-server-tests
It shows you how to build the keycloak server from source and run it (remote or local) and the above command is then used to run tests against this solution. Take note of
"The testsuite currently doesn't work with port 80."

ejabberd clustering problems and solutions

Setup Details
2 ejabberd nodes with postgresql as database (OS : Ubuntu 16.04)
Trying to do clustering of two ejabberd as mentioned in
https://docs.ejabberd.im/admin/guide/clustering/
After starting the master node the following steps have been performed on the slave node
copy .erlang.cookie to the slave node
copy ejabbed.yml from master to slave.
slave started successfully but shows the below error.
=====Error=========
Eshell V9.2 (abort with ^G)
(ejabberd#gim-Veriton-M6650G)1> 18:29:41.856 [notice] Changed loghwm of /usr/local/var/log/ejabberd/error.log to 100
18:29:41.856 [notice] Changed loghwm of /usr/local/var/log/ejabberd/ejabberd.log to 100
18:29:41.857 [info] Application lager started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.860 [info] Application crypto started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.865 [info] Application sasl started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.871 [info] Application asn1 started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.871 [info] Application public_key started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.880 [info] Application ssl started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.881 [info] Application p1_utils started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.883 [info] Application fast_yaml started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.888 [info] Application fast_tls started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.892 [info] Application fast_xml started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.895 [info] Application stringprep started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.899 [info] Application xmpp started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.903 [info] Application cache_tab started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.910 [info] Application eimp started on node 'ejabberd#gim-Veriton-M6650G'
18:29:41.910 [info] Loading configuration from /usr/local/etc/ejabberd/ejabberd.yml
18:29:41.913 [error] CRASH REPORT Process <0.67.0> with 0 neighbours exited with reason: no case clause matching <<>> in ejabberd_config:get_config_option_key/2 line 473 in application_master:init/4 line 134
18:29:41.913 [info] Application ejabberd exited with reason: no case clause matching <<>> in ejabberd_config:get_config_option_key/2 line 473
(ejabberd#gim-Veriton-M6650G)1>
I've tried re creating mnesia DB also but didn't help.
ejabberdctl status shows ejabberd is not running in that node
Can some oe please look into the issue and help.
Finally I found the solution to the problem
The issue is with the node name as the node name of the master ia FQ name
but the slave node's name is without a domain.
Also added both the node names in the /etc/hosts file
For ejabberd clustering ,Please refer the below steps.
Before starting , configure proper entries in the /etc/hosts files of both nodes.
ie the nodes should resolve each other using their host names.
set ejaberd node name in ejabberd.cfg file , both the nodes should have different node names.
1.cofigure ejabberd in one master node with a proper node name (either a FQDN or just a name of your convenience)
2.Configure slave node with the same config as that of master ie. bot the nodes should have the same configuration in ejabberd.yml file)
3.copy erlang.cookie from master node to slave and the ejabberd user should be ale to read the cookie file.
4.Start the master node in live mode (ejabberdctl live )
5.Start slave node in live mode
6.Check the cookie value in erlang console of both the nodes using the command 'erlang:get_cookie().' , both the nodes should have the same value.
7.If bot the nodes have same value then execute "ejabberdctl --not-timeout join_cluser ejabberd#nodename" in the slave.
change ejabberd#nodename according to your environment.
In my case I ran ejabberd with 'ejabberd' user with node name as ejabberd#cluster-node1 (If you want you can use a FQDN also like ejabberd#example.com)
8.If the abode command executed without any error then the nodes are in cluster
9.Confirm the cluster in any of the erlang console using the command mnesia:info(). here you will get the node details in "running_db_nodes"
10.Hurrayyyy you are done...
For load balancing the cluster you can use HAProxy
Please refer https://blog.onefellow.com/post/76702632637/haproxy-and-ejabberd for details
I've not done load balancing using any hardware load balancer , need to check on that
If anyone have done that please do post here ..

Can't find Raspberry_pi section (rpi gpio ) in node-red after an update

i used node red to control a led from a local server I had a section in nodes called "Raspberry_Pi" where i could add rpi gpio output/input and other interesting stuff
Howerver after upadating node red => "update-nodejs-and-nodered" I couldnt find this section.
Rpi gpio is correctly installed in my machine but the problem is i cant access it from node red The question is How to access Rpi gpio/mouse/keyboard/..... from node red ?
Start Up log
Start Node-RED
Once Node-RED has started, point a browser at http://192.168.1.5:1880
On Pi Node-RED works better with the Firefox or Chrome browser
Use sudo systemctl enable nodered.service to autostart Node-RED at every boot
Use sudo systemctl disable nodered.service to disable autostart on boot
To find more nodes and example flows - go to http://flows.nodered.org
3 May 19:29:02 - [info]
Welcome to Node-RED
===================
3 May 19:29:02 - [info] Node-RED version: v0.16.2
3 May 19:29:02 - [info] Node.js version: v6.10.3
3 May 19:29:02 - [info] Linux 4.9.24-v7+ arm LE
3 May 19:29:03 - [info] Loading palette nodes
3 May 19:29:07 - [info] Settings file : /root/.node-red/settings.js
3 May 19:29:07 - [info] User directory : /root/.node-red
3 May 19:29:07 - [info] Flows file : /root/.node-red/flows_ziedpi.json
3 May 19:29:08 - [info] Server now running at http://127.0.0.1:1880/
3 May 19:29:08 - [info] Starting flows
3 May 19:29:08 - [info] Started flows

ScalaTest: Suites are being executed but no test cases are actually running?

I am using the latest version of Play framework, along with the following test dependencies from by build.sbt file:
"org.scalatest" %% "scalatest" % "3.0.0",
"org.scalatestplus.play" % "scalatestplus-play_2.11" % "2.0.0-M1"
I have a base specification all of my test cases extend from. I return a Future[Assertion] in each one of my clauses, it looks like this:
trait BaseSpec extends AsyncWordSpec with TestSuite with OneServerPerSuite with MustMatchers with ParallelTestExecution
An example spec looks like this:
"PUT /v1/user/create" should {
"create a new user" in {
wsClient
.url(s"http://localhost:${port}/v1/user")
.put(Json.obj(
"name" -> "username",
"email" -> "email",
"password" -> "hunter12"
)).map { response => response.status must equal(201) }
}
}
I decided to rewrite my current tests using the AsyncWordSpec provided by the newer version of ScalaTest, but when I run the test suite, this is the output that I get:
[info] UserControllerSpec:
[info] PUT /v1/user/create
[info] application - ApplicationTimer demo: Starting application at 2016-11-13T01:29:12.161Z.
[info] application - ApplicationTimer demo: Stopping application at 2016-11-13T01:29:12.416Z after 1s.
[info] application - ApplicationTimer demo: Stopping application at 2016-11-13T01:29:12.438Z after 0s.
[info] application - ApplicationTimer demo: Stopping application at 2016-11-13T01:29:12.716Z after 0s.
[info] application - ApplicationTimer demo: Stopping application at 2016-11-13T01:29:13.022Z after 1s.
[info] ScalaTest
[info] Run completed in 13 seconds, 540 milliseconds.
[info] Total number of tests run: 0
[info] Suites: completed 4, aborted 0
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] No tests were executed.
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[success] Total time: 20 s, completed Nov 12, 2016 8:29:13 PM
All of my test classes are found, built, and seemingly run by the test runner when invoking sbt test. I have also tried using the IDEA test runner, and it reports Empty Test Suite under each one of my test classes. I have exhaustively attempted to RTFM but I cannot see what I am doing wrong. The synchronous versions of my tests are running totally fine.
EDIT 1: A friend suggested to attempt doing whenReady() { /* clause */ } on my Future[WSResponse], but this too has failed.
I was having the same problem while using a test suite with multiple traits. I got it to work by removing all the other traits except for AsyncFlatSpec. I will add them back one at a time as I need them.

Changing the Default Play Framework HTTP Port (without Using System Property)

There's an oft-asked question about changing the HTTP port to which a Play application will bind. James Ward's answer is generally accepted as the most complete, but it involves overriding the default by setting a http.port system property. However, is it possible to change this default without having to manually add it to the run command at development time, tweak the environment, or package an override in a runtime configuration?
This can be accomplished by setting the playDefaultPort key, as follows:
import PlayKeys._
playDefaultPort := 9123
Afterwards, you'll be able to run and testProd without needing to remember the desired port.
This works in both development:
$ sbt run
[info] Loading project definition from /Users/michaelahlers/Projects/MyApp/project
[info] Set current project to MyApp (in build file:/Users/michaelahlers/Projects/MyApp/)
--- (Running the application, auto-reloading is enabled) ---
[info] p.c.s.NettyServer - Listening for HTTP on /0:0:0:0:0:0:0:0:9123
(Server started, use Ctrl+D to stop and go back to the console...)
And production modes:
$ sbt testProd
[info] Loading project definition from /Users/michaelahlers/Projects/MyApp/project
[info] Set current project to MyApp (in build file:/Users/michaelahlers/Projects/MyApp/)
[info] Packaging /Users/michaelahlers/Projects/MyApp/target/scala-2.11/MyApp_2.11-1.0.0-SNAPSHOT-web-assets.jar ...
[info] Done packaging.
(Starting server. Type Ctrl+D to exit logs, the server will remain in background)
2016-04-08 13:09:45,594 [info] a.e.s.Slf4jLogger - Slf4jLogger started
2016-04-08 13:09:45,655 [info] play.api.Play - Application started (Prod)
2016-04-08 13:09:45,767 [info] p.c.s.NettyServer - Listening for HTTP on /0:0:0:0:0:0:0:0:9123