What is behind PostGISonline.org

June 15th, 2010 by Nicklas Avén

If you are curious and want to know what is behind PostGIS online you can download the database, php-pages and others from:
http://download.jordogskog.no/pgo
There is two files, pgo.sql.tar.gz is the database as a dump and pgo.tar.gz is the rest of the files.

Just so it’s said:
Do whatever you want with this code, but don’t blame me if things goes wrong.
If you develop it I would appreciate to get a note, just of curiosity.

You will find no rocket science but maybe a few ideas that might be interesting. Now it is just presented as a quite raw copy of the site without much explanation so I will give a few notes here.

The basic idea behind the site is to take the sql-string the user gives and create a table from the result and then show that table through MapServer. In the beginning it created a view instead of a table to save some disc writing, but I found that the view had to be evaluted several times to find resulting geometry types,srid and finally to show the view. So my conclusion was that it is worth writing down the table. So what is actually happening when the sql-string is sent is that the “CREATE TABLE” and a table name is put in front.

The table gets the name map1, map2 or map3 depending on what button the user clicked. To make it working with many users the session is is also added after mapX, for example map139853657d34373592f51e5e8c45f06b7.

What also happens is that when a user sends his first sql-string the session id is stored in table adm.sessions with a timestamp. then the timestamp is updated for each new query the user runs. That makes it possible for a function in the database found in file functions.sql to drop all tables belonging to session id’s that has not been used for half an hour.

On the Mapserver side of the story there is a mapfile with definitions of 9 different layers. One for Map1 point, one for Map1 linestring, one for Map1 polygon, one for Map2 point and so on. To make it possible to run queries that return geometry collections that contains different types I use the new function in PostGIS 1.5, ST_CollectionExtract(geometry collection, integer type). It gives som overhead but makes the system working with results containing mixed types and as mentioned even mixed geometry collections. So each geometry is checked both for points, linestrings and polygons. The DATA definition for points in Map1 looks like this for instance:

DATA “the_geom FROM (
select ST_CollectionExtract(ST_Force_Collection(the_geom),1) as the_geom,
idmap1 from userdata.%relation%) as a
USING UNIQUE idmap1 USING SRID=%srid%”

As you see on the above example there is also an unique id added to the table. That id, here called idmap1, is never showed to the user and is just added to the table after it was created to make Mapserver happy.

What I think from this maybe is most noticeable is the last part, how to make Mapserver work with geometry collections. As said earlier it gives some overhead, but can be usable anyway.

That’s it for now. Any comments or suggestion for improvement is very welcome.

Testing PostGIS 2.0 on PostGIS online

June 11th, 2010 by Nicklas Avén

Now it is possible to try and test new functionality soon after it is added in PostGIS trunk. How up to date the version on the site is might vary because there will be changes on the road towards PostGIS 2.0 that can not be automatically updated on the site.

How to use the trunk version you can learn from this tiny tutorial.

To see what new functions is there take a look at the SVN documentation snapshot

Ship to Gaza 2

May 31st, 2010 by Nicklas Avén

I have hijacked my own blog to write some updates of what is happening in Mediterranean sea right now.

From what I have understood from media, what has happened is that Israeli military has boarded all the ships, killed 19 (now they talk about “only”  9 killed) and wounded 60 persons on International water. Boats coming with humanitarian aid and supply.

It looks from pictures on internet like there might have been activists on the ships that started to hit the Israel soldiers when the boarded from helicopter but at the other hand Israel had no right what so ever to board the boats on international water.

Ship to Gaza

May 30th, 2010 by Nicklas Avén

This is not a blog post within my usual area, but since I think this initiative Ship to Gaza seems like an truly humanitarian project I would like to do what I can to put the light on what is happening right now in the Mediterranean sea. On board the ships are besides supplies like medical equipment and building material,  many hundreds of people supporting the sake like parliamentarians, and activists, both Muslims and Jewish among others.

There is a lot of blogs out there following what is happening, but I don’t know which of the English blogs to consider trustworthy so do a search yourself to get more information.

What is happening right now is that Israeli military are following the ships and have promised to stop them. That I have read on a Swedish blog post from one of the boats, but the same blog also send posts in English. A lot of things are happening right now.

PostGISonline on FOSS4G conference in Barcelona

May 28th, 2010 by Nicklas Avén
I am going to Barcelona in September to attend the FOSS4G conference 2010. From an initiative by Marc Jansen I will also take part in a presentation of Postgisonline and a site Marc has created with a similar functionality http://openlayers-buch.de FOSS4G  Conference

Simple SQL tutorial

April 3rd, 2010 by Nicklas Avén

Here comes an attempt to explain some basic SQL queries.

The tutorial is divided in 3 sections. You can find the all here:
http://www.postgisonline.org/tutorials/

PostGIS online

March 28th, 2010 by Nicklas Avén

Maybe you have seen it, it has been up and running for a few weeks now, but now it is also starting to take shape as I want it. What?…
http://www.postgisonline.org, a site for testing, training and showing PostGIS.

It is a web site with the intention to make it possible to find the power in handling spatial data in a relational database environment. You can write any SELECT statement against the database that PostGreSQL/PostGIS latest releases support and get the answer back as a table or as a map if it includes spatial data in a field named “the_geom”.

It is also possible for anyone to write tutorials and run them through the site. The tutorial can present a text, a sql-code, a background picture and a background map for each page. This concept have to be further developed but I think it is a good start and I hope it will be used to show the capabilities of PostGIS. I will use it to continue writing about “How to use the new distance functions”. An instruction, how to write tutorials will show up in the documentation area of the site. So far I have written two of them that can be found here

The spatial functionality is done with MapServer showing tables created by the sql query written by the user.

So, this is a first version. I hope that it will be used and maybe it can become a project with more people involved. There is a lot of functionality that can be added like PL/R with some way to show diagrams and pgrouting and some way to show WKTraster.

Oh, yes the logo…. It is a creation made by my wife :-)

Welcome !

Strange behavior by design of the spatial function Filter in SQL Server 2008

February 27th, 2010 by Nicklas Avén

In SQL Server 2008 there is a spatial function called Filter, documented here:
http://msdn.microsoft.com/en-us/library/cc645883.aspx
This function makes a fast index-based scan for geometry intersection. It guarantees to return all intersecting cases, but might return cases not intersecting as well. So this is a first filtering and as I understand it, it is an internal part of STIntersects. This functionality has nothing to do with the bounding box comparison  PostGIS does as a first filtering before the real ST_intersts calculation. In SQL Server Filter returns a more accurate answer to the intersection question. This discussion explains a lot of how it works:
social.msdn.microsoft.com/Forums/en/sqlspatial/thread/6e1d7af4-ecc2-4d82-b069-f2517c3276c2

The problem with this function is that, as the documentation says:

In cases where an index is not available, or is not used, the method will return the same values as STIntersects() when called with the same parameters.”

Why is this a problem? Well, this means that the function gives different answers with and without an index from the exactly same query and dataset. This I think is a little problematic in itself for a function not being totally internal. But it might become nearly absurd in some cases. Look at this query:

Select a.id, b.id from table1 a ,  table2 b where a.geom.Filter(b.geom)=1;

If the geometries in table1 and table2 are indexed, we will get a fast answer, which might contain more geometry combinations than those actually intersecting with each other.  That is no problem, that is the whole idea. The problem shows if we want to see the result of the Filter function in the select part of the query like this:

Select.geom.Filter(b.geom) asFiltered,  a.id, b.id from table1 a ,  table2 b where a.geom.Filter(b.geom)=1;

Then the interesting thing happens that since the index won’t be used in the select part (using index makes no sense in select part) the Filter function here will give the same result as STIntersects, just as the documentation says. So, even if we “filter away” all cases where Filter returns anything else than 1 we will get rows where the column “Filtered” returns 0.

Here is a picture of my practical example from SQL Server Management console. As you see I get 144 rows back. I use the exactly same function in the where-part and in the select part but apparently get different answers. Fully logical from how the function is designed, but I don’t like it.

I guess that the reason for this design is that it is difficult or impossible to get the same result without the index

Please comment on this one. Have I misunderstood things?
Maybe it is common with functions giving different answers with and without indexes?

But to me it looks quite ugly.

Good news to windows users, PostGIS 1.5 now in Stack builder

February 22nd, 2010 by Nicklas Avén

Today PostGIS 1.5.0 reached Stack builder. That means that if you are using windows it’s now very simple to try the new functionality. Thanks Regina and Leo

That means that a fully functional installation with PostGreSQL 8.4.2 and PostGIS 1.5.0 is just a few clicks away.

Download the oneclick-installer of PostGreSQL from
http://www.enterprisedb.com/products/pgdownload.do#windows

Then, in the end of the installation choose yes to start Stack builder and mark PostGIS 1.5 under spatial extensions.

Ready to go :-)

How to use the new distance related functions in PostGIS Part1

February 7th, 2010 by Nicklas Avén

As described earlier there is also some new distance related functions in new PostGIS release. Since I am in some way responsible for their existance I feel a need to give some examples of how they can be used.

Example 1 Snap GPS-points to roads

If collecting GPS-points along a road you will get them placed on the side of the roadline because of the inaccurancy of the GPS and the road on the map. One way to get the points on the road is to use ST_Closestpoint.

We have a table with our roads called ….. roads (why not) and another table called gpspoints with pour GPS points. The gepspoints are as expected placed, not on the road, but next to the road. There is also a GPS point that probably have been collected when walking in the forest since it is to far away from the road.


roads and GPS points

Now lets do a first attempt to move the points to the roads. We do:

Select ST_Closestpoint(a.the_geom, b.the_geom) as the_geom
from roads a inner join gpspoints b
on ST_Dwithin(a.the_geom, b.the_geom, 100);

We count on that no GPS points should be more than 100 meters away from the road if it is collected from the road. That much error should never be from the GPS, but it is possible if it is just started and so on. The result from our query will be the green dots below:


Gps Points moved to road 1

But there is a problem. Not the point out in the forest. That one we have abandoned since it is more than 100 meters from the road. But the point between two road parcels a little bitdown from the middle of the map. That GPS point did get two new points insted of just one. What has happened is that every road feature closer than 100 meters of a GPS point has had a point from that GPS point.

This can be solved by collecting all road features that is within those 100 meters from the GPS point into a geometry collection which we put into the ST_Closestpoint function. Since grouping by geometries often is a bad idea we group by the id of the road features but also group by the geometriy just to avoid the need of aggregating also the GPS points. The query will look something like:

Select ST_Closestpoint(ST_Collect(a.the_geom), b.the_geom) as the_geom
from roads a inner join gpspoints b
on ST_Dwithin(a.the_geom, b.the_geom, 100)
group by b.gid, b.the_geom;

and the result will be, again like the green dots:


roads3

That’s better. Now it looks like expected.