Error when using OSMOSIS to extract data from OpenStreetMap data - openstreetmap

I am trying to replicate the example in this page. I downloaded the data and run this code (the example data of that page can be downloaded from here):
osmosis --rbf sample_osmosis.osm.pbf --nkv keyValueList="amenity.school" --wx schools.osm
I am using Linux Mint 20.2 Cinnamon and installed osmosis by using Sypantic (version 0.47).
I downloaded the example in the previous page and run the previous code in the folder containing the .pbf file.
However, I am getting this error:
osmosis --rbf sample_osmosis.osm.pbf --nkv keyValueList="amenity.school" --wx schools.osm
nov. 18, 2021 11:55:56 P. M. org.openstreetmap.osmosis.core.Osmosis run
INFORMACIÓN: Osmosis Version 0.47
nov. 18, 2021 11:55:56 P. M. org.openstreetmap.osmosis.core.Osmosis run
INFORMACIÓN: Preparing pipeline.
nov. 18, 2021 11:55:56 P. M. org.openstreetmap.osmosis.core.Osmosis run
INFORMACIÓN: Launching pipeline execution.
nov. 18, 2021 11:55:56 P. M. org.openstreetmap.osmosis.core.Osmosis run
INFORMACIÓN: Pipeline executing, waiting for completion.
nov. 18, 2021 11:55:56 P. M. org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
GRAVE: Thread for task 1-rbf failed
java.lang.NoClassDefFoundError: com/google/common/util/concurrent/MoreExecutors
at org.openstreetmap.osmosis.pbf2.v0_6.PbfReader.run(PbfReader.java:87)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.MoreExecutors
at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:271)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:247)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:239)
... 2 more
nov. 18, 2021 11:55:56 P. M. org.openstreetmap.osmosis.core.Osmosis main
GRAVE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed.
at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92)
at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:321)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:234)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
at org.codehaus.classworlds.Launcher.main(Launcher.java:47)
Any idea what might be happening?

I just ran with success your exact command with the Osmosis Version 0.48.3 using your example data (osmosis --rbf sample_osmosis.osm.pbf --nkv keyValueList="amenity.school" --wx school.osm): INFOS: Total execution time: 439 milliseconds.
My Osmosis version is possibly different from yours, but the sample_osmosis.osm.pbf file possibly evolved too.
FYI I encountered on 2022-07-20 the same error message ("GRAVE: Thread for task 1-rbf failed") using the latest GeoFabrik OSM data from https://download.geofabrik.de/europe/monaco-220719.osm.pbf with a basic Osmosis command: osmosis --read-pbf file=monaco-latest.osm.pbf --write-xml file=monaco-latest.osm. After analysis and exchange with the GeoFabrik support, it was related to a bug that has been fixed the day after.
Waiting for a format fix, a good workaround has been to "clean" the pbf with osmconvert: osmconvert" monaco-latest.osm.pbf>monaco-new.osm.pbf --out-pbf

Related

Installing CDT into Eclipse with P2 director gives "The target server failed to respond"

I download the 2020-03 Eclipse Platform Runtime Binary (v4.15 from eclipse-platform-4.15-linux-gtk-x86_64.tar.gz), then invoke the p2 director to install CDT package, but I'm always getting multiple caught exceptions of http://download.eclipse.org not responding (http/https switch doesn't have effect on this).
./eclipse -application org.eclipse.equinox.p2.director -profile SDKProfile -noSplash -roaming -repository http://download.eclipse.org/releases/2020-03,http://download.eclipse.org/releases/2020-03/202003181000/,http://download.eclipse.org/eclipse/updates/4.15/,http://download.eclipse.org/releases/latest,http://download.eclipse.org/tools/cdt/releases/9.11 -installIU org.eclipse.cdt.feature.group/9.11.0.202003091030
I tried it with Oracle JRE 8 and with Oracle JDK 14, I tried it on Debian and RHEL machines, tried different network connection (3 different IPS locations), removed all other plugins I install to nail it down to the CDT plugin. I do not know why or what is causing it, and searching for this problems shows to people that ISP fixed or some similar solution. While mine is consistent for about a year. I always eventually finishes, but it will will print out multiple messages:
Installing org.eclipse.cdt.feature.group 9.11.0.202003091030.
Apr 22, 2020 1:44:29 P.M. org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {}->http://download.eclipse.org:80: The target server failed to respond
Apr 22, 2020 1:44:29 P.M. org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->http://download.eclipse.org:80
Apr 22, 2020 1:44:36 P.M. org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {}->http://download.eclipse.org:80: The target server failed to respond
Apr 22, 2020 1:44:36 P.M. org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->http://download.eclipse.org:80
Operation completed in 44091 ms.
Using the HTTPS repositories has similar output:
INFO: I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {s}->https://download.eclipse.org:443: The target server failed to respond
Apr 22, 2020 4:02:50 P.M. org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {s}->https://download.eclipse.org:443
Operation completed in 46934 ms.
It's not critical as it eventually completes and the end result works, but I would like to understand it better what is happening, what has effect on this behaviour. When I shrink the list of repositories to these two, it will still install, but the messages will be present:
http://download.eclipse.org/releases/2020-03,http://download.eclipse.org/tools/cdt/releases/9.11
Is there maybe a resource/documentation which probably I missed and is explaining more in the detail what is happening and why?

Azure pipeline Docker `all pipe instances are busy`

When running a build pipeline with maven and docker-maven-plugin sometimes(more than 50% of the time) it throws following error and pipeline fails.
Jun 18, 2019 1:57:12 AM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (java.io.FileNotFoundException) caught when processing request to {}->npipe://127.0.0.1:1: \\.\pipe\docker_engine (All pipe instances are busy)
Jun 18, 2019 1:57:12 AM org.apache.http.impl.execchain.RetryExec execute
I can see in the log that multiple retry attempts, but all fails. Is there any way that I can overcome this?
I just solved this problem:
I added the right version of docker-maven-plugin 1.4.10
dockerfile-maven-plugin

Extremely Slow Query Execution After Postgres 9.6 Upgrade

I am seeing extremely slow processing times after a Postgres upgrade from 9.5 to 9.6 related to queries Osmosis (the command line Java application for processing OpenStreetMap data). Specifically syncing a local osm database with the master osm repo (Postgis 2.3 from 2.2). The syncing used to happen every day and took about 500 seconds on average. After the upgrade, after 48 hours, it is still not over. I notice very slow (below zero) object processing times (log record below) compared to before. I am fairly certain the problem is with the Postgres update. Can't figure out what to fix. pg_stat_activity gives the following query (currently):
UPDATE ways w
SET linestring = (
SELECT ST_MakeLine(c.geom) AS way_line
FROM (
SELECT n.geom AS geom
FROM nodes n
INNER JOIN way_nodes wn ON n.id = wn.node_id
WHERE (wn.way_id = w.id)
ORDER BY wn.sequence_id ) c )
WHERE w.id IN (
SELECT w.id
FROM ways w
INNER JOIN way_nodes wn ON w.id = wn.way_id
WHERE wn.node_id = $1
GROUP BY w.id )
Log record after the update:
Oct 31, 2017 12:23:59 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.45
Oct 31, 2017 12:24:00 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Oct 31, 2017 12:24:00 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Oct 31, 2017 12:24:00 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
Oct 31, 2017 12:24:16 PM org.openstreetmap.osmosis.core.progress.v0_6.ChangeProgressLogger process
INFO: Processing Node 26022046 with action Modify, 0.06660006660006661 objects/second.
Oct 31, 2017 4:12:49 PM org.openstreetmap.osmosis.core.progress.v0_6.ChangeProgressLogger process
INFO: Processing Node 26022047 with action Modify, 7.292125918680253E-5 objects/second.
Oct 31, 2017 6:54:27 PM org.openstreetmap.osmosis.core.progress.v0_6.ChangeProgressLogger process
INFO: Processing Node 26022053 with action Modify, 1.0311411856040729E-4 objects/second.
Oct 31, 2017 9:39:22 PM org.openstreetmap.osmosis.core.progress.v0_6.ChangeProgressLogger process
INFO: Processing Node 26022056 with action Modify, 1.0106204077408672E-4 objects/second.
Nov 01, 2017 5:07:11 AM org.openstreetmap.osmosis.core.progress.v0_6.ChangeProgressLogger process
INFO: Processing Node 26022057 with action Modify, 3.72178939465691E-5 objects/second.
Nov 01, 2017 7:43:20 AM org.openstreetmap.osmosis.core.progress.v0_6.ChangeProgressLogger process
INFO: Processing Node 26022058 with action Modify, 1.0673723278600615E-4 objects/second.
Log record before the update:
Oct 22, 2017 11:00:01 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.45
Oct 22, 2017 11:00:02 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Oct 22, 2017 11:00:02 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Oct 22, 2017 11:00:02 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
Oct 22, 2017 11:00:17 PM org.openstreetmap.osmosis.core.progress.v0_6.ChangeProgressLogger process
INFO: Processing Node 557521 with action Modify, 0.06700616456714018 objects/second.
Oct 22, 2017 11:00:22 PM org.openstreetmap.osmosis.core.progress.v0_6.ChangeProgressLogger process
INFO: Processing Node 49820266 with action Modify, 58.15347721822542 objects/second.
Oct 22, 2017 11:00:27 PM org.openstreetmap.osmosis.core.progress.v0_6.ChangeProgressLogger process
INFO: Processing Node 81804664 with action Modify, 40.64719810576164 objects/second.
Oct 22, 2017 11:00:32 PM org.openstreetmap.osmosis.core.progress.v0_6.ChangeProgressLogger process
INFO: Processing Node 109690032 with action Modify, 24.57577530719719 objects/second.
Oct 22, 2017 11:00:38 PM org.openstreetmap.osmosis.core.progress.v0_6.ChangeProgressLogger process
INFO: Processing Node 124684125 with action Modify, 58.35490938060148 objects/second.
FYI, I came across with exactly the same issue: I upgraded pg to 9.6 from 9.4 and queries ran much slower.
I had run only vacuum, nothing changed, but then I ran analyse as #harmic suggested and runs fine now (even faster, thanks to pg 9.6 improvements :) )!
After Postgres Upgrade you need to analyze the database for fast query plane.
To analyze the full database run the below query
ANALYZE VERBOSE --for all database analyze
To analyze only a specific table then use this query
ANALYZE VERBOSE [Table_name] --for particular table
query works in this Supported Versions: Current (13) / 12 / 11 / 10 / 9.6 / 9.5
reference: https://www.postgresql.org/docs/9.1/sql-analyze.html

Osmosis not working with mapfilewriter plugin

I've recently downloaded Osmosis to convert .osm.pbf files to .map files. I'm using Windows 7 for this application. I've downloaded the latest zip file from their system and placed the mapfilewriter jar files into the /lib/default/ folder. However, I keep receiving this error when I run this statement in the .bat file:
osmosis --read-pbf file=taiwanlatest.osm.pbf --mapfile-writer file=helloworld.map
Mar 19, 2013 7:34:49 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.42
Mar 19, 2013 7:34:49 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Mar 19, 2013 7:34:50 PM org.mapsforge.map.writer.osmosis.MapFileWriterTask <init>
INFO: mapfile-writer version: mapsforge-map-writer-0.3.0
Mar 19, 2013 7:34:50 PM org.mapsforge.map.writer.osmosis.MapFileWriterTask <init>
INFO: mapfile format specification version: 3
Mar 19, 2013 7:34:50 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Mar 19, 2013 7:34:50 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
Mar 19, 2013 7:34:50 PM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
SEVERE: Thread for task 1-read-pbf failed
java.lang.AbstractMethodError: org.mapsforge.map.writer.osmosis.MapFileWriterTas
k.initialize(Ljava/util/Map;)V
at crosby.binary.osmosis.OsmosisReader.run(OsmosisReader.java:43)
at java.lang.Thread.run(Thread.java:722)
Mar 19, 2013 7:34:50 PM org.openstreetmap.osmosis.core.Osmosis main
SEVERE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed
.
at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForComple
tion(Pipeline.java:146)
at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92)
at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Laun
cher.java:329)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.jav
a:239)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(La
uncher.java:409)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:
352)
at org.codehaus.classworlds.Launcher.main(Launcher.java:47)
I've read up and it seems that this occurs due to the osmosis itself, and that I have to use a lower version's osmosis. I've tried to use Osmosis version 0.40 and I've gotten this error when I run the .bat file:
Error: Could not find or load main class org.codehaus.classworlds.Launcher
Where exactly did I go wrong?
After much trial and error, I finally made it work by downloading Osmosis version 0.40.1 here.
In addition, I've added 4 jar files in the /lib/default folder:
mapsforge-map-writer-0.3.0-jar-with-dependencies.jar
mapsforge-map-0.3.0-jar-with-dependencies.jar
trove-3.0.3.zip
jts-1.8.jar
After which, i created a new conf file named "osmosis-plugins.conf" and added a line " org.mapsforge.map.writer.osmosis.MapFileWriterPluginLoader" in it.
After doing these steps, Osmosis finally works (though I'm not sure if this is the most correct way of doing it).
However, I still don't really understand why the latest version of Osmosis does not work.
Hope this can help those who faced a similar problem as me!
Bumped into this problem as well and here's my solution to it:
Download the mapsforge-map-writer jar file (the one with
dependencies) from https://search.maven.org/search?q=mapsforge-map
Put the downloaded jar file into my osmosis/lib/default directory
Open up cmd, cd to where my osmosis is located and run a --mapfile-writer command.
Hope this helps!

I had used Quartz in my Java web-application but when Quartz will execute after its execution, Tomcat is being off.

I wrote a code in Quartz that will update database on a daily basis exactly once a day.
I am using Eclipse in my web-application so had started the Tomcat through Eclipse.
But after updating the database from Quartz it is forcing the Tomcat to be terminated in Eclipse. So my application is not working after the Quartz portion is running (due to Tomcat termination). The messages I got in Eclipse 4 Quartz is displayed below:
Job name:Event Update1
Group name:mysql
Trigger name:cronTrigger4Event1
Firing Time:Mon Jan 24 14:30:00 IST 2011
dateFormat.format(calendar.getTime()) : 2011-01-24
Inserting values in Mysql database table!
1 row affected
Jan 24, 2011 2:30:00 PM org.apache.coyote.http11.Http11BaseProtocol pause
INFO: Pausing Coyote HTTP/1.1 on http-8080
[INFO] 24 Jan 2011 14:30:05.506 Thread-34 [org.quartz.core.QuartzScheduler]
Scheduler QuartzScheduler_$_NON_CLUSTERED shutting down.
[INFO] 24 Jan 2011 14:30:05.506 Thread-34 [org.quartz.core.QuartzScheduler]
Scheduler QuartzScheduler_$_NON_CLUSTERED paused.
[INFO] 24 Jan 2011 14:30:05.506 Thread-34 [org.quartz.core.QuartzScheduler]
Scheduler QuartzScheduler_$_NON_CLUSTERED shutdown complete.
Any help is appreciated.
i think your problem in tomcat and eclipse not in quartz itself so you have two solutions
1- get the tomcat out of eclipse and deploy you application manually, don't forget to give you application server more memory.
2- get you schedule jobs out of you application and use the Linux system cron jobs.