ogr2ogr producing different results on different system - postgresql

I'm trying to convert the shape file which represents the administrative area of Philippines into geojson. I'm getting the shape file from here . The file I'm trying to convert is this PHL_adm3.shp .
I'm using this bash script:
function shp2geojson() {
ogr2ogr -f GeoJSON "$1.geojson" "$1.shp"
}
for var in *.shp; do shp2geojson ${var%\.*}; done
I ran this script on my server and got the corresponding geojson file. Now I ran the same script on my local and the geometry for many cities were different in my local than what was on my server. Though the coordinates do not differ much, they only differ after 6-7 places of decimal, but they throw an exception when I try to insert them in PostGIS.
Here's an example of the geometry that is producing error on server while inserting it in PostGIS. I'm also posting the geometry that I'm getting on my local of the same city.
Server: Geometry
Local: Geometry
I also ran ST_IsValidReason on both geometries. I got the following error on my server.
SELECT ST_IsValidReason(ST_SetSRID(
ST_GeomFromGeoJSON(my_geometry),4326))
Server : Error [('Self-intersection[120.979159 14.71218]',)]
Local : [('Valid Geometry',)]
I tried the exact same thing on different countries like India, Japan Thailand and for all those countries that I tested, the same thing was happening. On local I didn't found any error but on my server I was getting Invalid geometry.
Can anyone explain what's wrong with my queries and how to fix this ?
P.S.- Both of my systems are 64 bit and running Ubuntu 14.04.

Related

Syncing Invalid Postgis Geometries with SymmetricDS

My SymmetricDS 3.9.2 implementation syncs databases with postgis (2.2.2) geometry on a daily basis. There is invalid postgis geometry in some of the tables. When this data with a geometry column is moved, SymmetricDS gets an error while trying to convert the geometry data from EWKT back into geometry.
For example: failed: ERROR: geometry requires more points
Hint: "...G(268676.801767248 5977299.18090558)" <-- parse error at position 56 within geometry
Is there a way to sync the geometry without writing it to well known text and then converting it back into geometry?
-- this will fail because there is not enough data for a line, however, I don't want it to ever encounter this issue, I just want it to move the invalid geometry.
SELECT ST_GeomFromText('SRID=3401;LINESTRING(152735.018168789 6064398.3443075)');
Below is a geometry. Could we sync it like this so we do not care if it is invalid?
"01060000A0490D00000100000001030000800100000006000000763A90954362EC40C458D64A116056410000000000000000212DA53A9462EC40BB0CB3CF07605641000000000000000073EC95C3F267EC40DCAA4ADDE05F56410000000000000000BF4F7D4B3F74EC407A48E55CE45F56410000000000000000D409C190AF73EC40F052FA69116056410000000000000000763A90954362EC40C458D64A116056410000000000000000"

Get PostGIS geometry field on Drill

I have a table with a geometry column and if I query it using PostGIS, it shows the records right:
PostGIS query image:
The problem is when I execute the query using Apache Drill, because it shows all the records fine except the geometry one, it shows as null.
Drill query image:
Reviewing the logs, it shows the following error:
WARN o.a.d.e.store.jdbc.JdbcRecordReader - Ignoring column that is
unsupported. org.apache.drill.common.exceptions.UserException:
UNSUPPORTED_OPERATION ERROR: A column you queried has a data type that
is not currently supported by the JDBC storage plugin. The column's
name was geom_multipolygon and its JDBC data type was OTHER
I tested creating the Drill storage plugin with postgis-jdbc-2.2.1.jar and postgresql-42.1.4.jar but the same error is shown.
If I use:
cast(geom_multipolygon as varchar(255))
it shows the varchar representation of the geometry, another option is getting the MULTIPOLYGON text and transform to Drill binary using ST_GeomFromText(geom), but we need the binary format directly from PostGIS, so those approaches can't be done.
We have seen this page: https://github.com/k255/drill-gis/issues/1 but the proposed solution doesn't work for us, so I think there is a way to achieve this.
UPDATE: I finally found the way that Drill can show the geo fields, is to change the data type in PostGIS from geometry to bytea. It seems to be a compatibility issue. With this way, we can perform geospatial queries on Drill, but in PostGIS those fields are no longer geometries, so they can not be indexed and treated as such.

PostGIS geography query returns a string value

I have a strange issue. The lonlat column on my app works well on the development server –– its output is in the form of POINT(X Y). But when I move the data to the production server, the output is strange!
ActionView::Template::Error (undefined method `lon' for "0101000020E6100000541B9C887E7A52C02920ED7F80614440":String):
The lonlat value, which is encoded with SRID: 4326, is being read as a string. I am almost certain that there was a corruption in the data during migrating it from development to production because this was not a problem before the migration.
Does anyone know what about the database schema or column may cause this issue?
A geometry field stores its data as WKB. To see the WKT representation you need to change your query to something like
select ST_Astext(the_geom) as geometry from table
However, I don't know why in your development you have some kind of implicit conversion between WKB binary data and WKT strings. ¿What version of postgres and postgis are you using?
What lang is in your app server?
Is that ActiveRecord you're using?
I suggest you try something like
float ST_X(geometry a_point);
To make sure you can read the data properly and determinate if problem is on the data field or somewhere else.
I also would try doing the pg_dump in a single step if you determinate the problem is with the geometry column.
You can use pg_dump with option
--exclude-table-data=reg_expresion_ _tablename_
--exclude-table-data=schema.reg_expresion_ _tablename_
This will bring all the schema definition, but exclude the table data and bring only the data from table you need.
Turns out that when I killed the connection to the server to migrate the data, Rails did not set the schema search path (meaning didn't discover the postgis extension) upon reconnecting. I had to restart the server to solve this problem.

Cannot map an error code returned from unixodbc using PostgreSQL database

I am using PostgreSQL using unixodbc driver, and while trying to get connection, I get error. I am only printing the value of pfNativeError of SQLError, and I get a value of '26'.
I have gone through the error codes returned by postgresql, as listed here: http://www.postgresql.org/docs/8.1/static/errcodes-appendix.html#ERRCODES-TABLE
I wanted to know if unixodbc returns in pfNativeError just the last three characters of the error codes mentioned in link above? If that is true, I assume the only possibility is for the following error code:
22026 STRING DATA LENGTH MISMATCH string_data_length_mismatch
Do let me know if I am thinking in right direction. Also, this issue I've noticed only when the PostgreSQL has millions of rows, and the query which results in connection failure is trying to fetch a lot of data (10Ks). Can someone give any idea to why the problem might be happening?
EDIT 1:
If its of any help, I'm getting following values for szErrorMsg:
Error while executing the query
Could not send Query(connection dead)
EDIT 2:
'26' returned is in INTEGER, and the codes mentioned in link above are in HEX. 26 in decimal corresponds to 1A in hex. Unfortunately, it does not correspond to any thing in the above mentioned documentation. Clearly I am out of ideas! Can someone tell me what different pfNativeError codes correspond to?

PostGIS Conversion Issues

I am having an issue using PostGIS (1.5.4) data. It may be that I'm just not familiar enough with this technology to see the obvious (I'm a regular expert with nearly 4 hours of experience), but I am running into an error that I have been unable to solve with Google.
I have a table which includes Polygon data (and yes, I checked; the column type is geometry, not polygon- the Postgres native type). The problem arises when I am trying to run a query on the table to find which shape contains a particular problem.
I am using the following query:
SELECT *
FROM geo_shape
WHERE ST_Contains(geoshp_polygon, POINT(-97.4388046000, 38.1112251000));
The error I receive is 'ERROR: function st_contains(geometry, point) does not exist'. I tried using a CAST() function, but got 'ERROR: cannot cast type geometry to polygon'. I'm guessing the issue has to do with the way the data is stored- PGAdmin shows it as hex data. I tried using ST_GeomFromHEXEWKB() just on a hunch, but received 'ERROR: function st_geomfromhexewkb(geometry) does not exist'.
I'm fairly confused as to what the issue is here, so any ideas at all would be much appreciated.
st_contains needs a geom,geom as arguments...
Give this a try...
SELECT * FROM geo_shape
WHERE ST_Contains(geoshp_polygon,
GeomFromText('POINT(-97.4388046000 38.1112251000)'));
Editted to correct , issue in the point data. ST_geomfromtext will work, kinda curious what the difference is there
You cannot mix PostgreSQL's geometric data types with PostGIS's geometry type, which is why you see that error. I suggest using one of PostGIS's geometry contstructors to help out:
SELECT *
FROM geo_shape
WHERE ST_Contains(geoshp_polygon,
ST_SetSRID(ST_MakePoint(-97.4388046000, 38.1112251000),4326);
Or a really quick text way is to piece together the well-known text:
SELECT 'SRID=4326;POINT(-97.4388046000 38.1112251000)'::geometry AS geom;
(this will output the WKB for the geometry type).