Home |  MySQL Buzz |  FAQ |  Feeds |  Submit your blog feed |  Feedback |  Archive |  Aggregate feed RSS 2.0 English Deutsch Español Français Italiano 日本語 Русский Português 中文
Showing entries 1 to 26

Displaying posts with tag: GIS (reset)

Using UDFs for geo-distance search in MySQL
+0 Vote Up -0Vote Down

In my previous post about geo-spatial search in MySQL I described (along with other things) how to use geo-distance functions. In this post I will describe the geo-spatial distance functions in more details.

If you need to calculate an exact distance between 2 points on Earth in MySQL (very common for geo-enabled applications) you have at least 3 choices.

  • Use stored function and implement haversine formula
  • Use UDF (user defined function) for haversine (see below)
  • In MySQL 5.6 you can use
  [Read more...]
Importing Raster Based Spatial Data into MySQL 5.7
Employee_Team +2 Vote Up -0Vote Down

Setting up the Geospatial Data Abstraction Library
GDAL 1.10+ (by OS Geo) supports converting spatial data between various raster and vector (points, lines, polygons, etc.) based formats, along with supporting MySQL. MySQL and ESRI Shapefiles are generally the best supported combination today.

Version 1.10 of GDAL has big improvements over older versions, but most linux distros (EL6 has GDAL 1.7) only have packages for the older versions. I would recommend building the latest 1.11.0 release from source because the older


  [Read more...]
Making Use of Boost Geometry in MySQL GIS
+1 Vote Up -0Vote Down
This post talks about how we make use of Boost.Geometry in MySQL to implement reliable and efficient GIS functionality, as well as changes to the GIS features in the lab release. Now that InnoDB is the default storage engine for MySQL, our user base is rapidly transitioning to InnoDB. One capability that they have been demanding is a performant and scalable GIS implementation. Along with adding R-tree index support to InnoDB, we also decided to replace the original GIS algorithms with a more powerful, reliable, effective and efficient geometric engine.
MySQL 5.7 and GIS, an Example
Employee_Team +1 Vote Up -0Vote Down

Summary
This post will offer a very simple demonstration of how you can use MySQL 5.7 for Spatial features within your applications. In order to demonstrate this, I’ll walk through how we can determine the 10 closest Thai restaurants to a particular location. For this example, we’ll be using the apartment that I lived in when I first started working at MySQL, back in 2003.

For more details on all of the new GIS related work that we’ve done in MySQL 5.7, please read through these blog posts from the developers:


  [Read more...]
Why Boost.Geometry in MySQL?
Employee_Team +5 Vote Up -0Vote Down

In the newly released MySQL 5.7 labs release, users will find refactored GIS functions, as well as R-tree indexes in InnoDB. One notable change is that we have replaced a lot of the native code for geometry computation with an external library, namely Boost.Geometry. In fact, 5.7 will be the first MySQL release that depends on Boost. So why did we make such a move?

In recent years, we have seen a tremendous growth in location based services, and consequently GIS has become a strategically important area for MySQL. Our existing customers require more GIS features, and we believe this is a growing market opportunity.

Up to MySQL 5.6, GIS support in MySQL has been very limited, and the geometry algorithms implemented in MySQL were

  [Read more...]
InnoDB Spatial Indexes in 5.7.4 LAB release
Employee_Team +3 Vote Up -0Vote Down

With MySQL 5.7.4 LAB release, InnoDB now supports “Spatial Index” on geometry data.

Before this new feature, InnoDB stores geometry data as BLOB data, and only prefix index can be created on the spatial data. It is very inefficient when comes to spatial search, especially when it comes to complex geometry data. In most cases, table scan are the only way to get the result. This all changed with InnoDB spatial index, which is implemented as R-tree, any spatial search becomes far more efficient.

InnoDB spatial index can be used with all existing syntax that has been developed for MyISAM spatial index. In addition, InnoDB spatial index supports full transaction properties, as well as isolation levels. It employs predicate lock to prevent phantom scenario.

In InnoDB spatial index, only the object’s Minimum Bounding Box is included in the index,

  [Read more...]
Building MySQL with Boost
Employee_Team +1 Vote Up -0Vote Down

We have a new April labs release that you can download from labs.mysql.com. There are more labs releases. Please try them all, but for now let’s focus on the one called “MySQL GIS, InnoDB R-tree, Parser Refactoring”.

The release contains InnoDB R-trees, the first step of our new GIS implementation, a refactored and faster parser and improved condition filtering cost calculations. You can download and run the binaries just like before, but if you want to build it from source, there are some details you should know. Otherwise you may fail to build MySQL (bug #72172).

This labs release requires Boost to build. Specifically, it requires Boost 1.55. But don’t go running off to boost.org looking for the latest tarball just yet — there’s no need for that. The MySQL

  [Read more...]
Creating GEO-enabled applications with MySQL 5.6
+0 Vote Up -0Vote Down

In my previous post I’ve showed some new MySQL 5.6 features which can be very helpful when creating geo-enabled applications. In this post I will show how we can obtain open-source GIS data, convert it to MySQL and use it in our GEO-enabled applications. I will also present at the upcoming Percona Live conference on this topic.

Data sources (US)

For the U.S. we may look at 2 major data sources:

1. ZIP codes with latitude, longitude and zip code boundaries (polygon). This can be downloaded from the U.S. Census website:

  [Read more...]
jQuery and GIS distance in MariaDB
+0 Vote Up -0Vote Down

I’ve continued building on my MariaDB GIS and node.js example application that I wrote about two weeks back, https://blog.mariadb.org/node-js-mariadb-and-gis/. The application shows how to load GPX information into MariaDB, using some MariaDB GIS functionality, and making use of the node.js platform together with MariaDB’s non-blocking client.

With the GPX data converted into GIS points in the MariaDB database, I wanted to further expand a little on both the GIS aspect and also look at how some additional data could be shown in the application by using jQuery’s Ajax calls to update a piece of the web based application UI.

To start with, an interesting thing to do when you have a bunch of GIS

  [Read more...]
Node.js, MariaDB and GIS
+1 Vote Up -0Vote Down

The availability of the node.js binding for MariaDB’s non-blocking client library together with the GIS capabilities of MariaDB inspired me to make an example of using node.js and MariaDB to import so-called GPX tracks to a MariaDB database and then show them on a map. GPX tracks are what are stored by many GPS devices including running watches and smartphones.

My project makes use of MariaDB’s non-blocking client library together with the node.js platform and on top of that uses the GIS functionality found in MariaDB 5.5 and 10.0.

To start with let’s go through the software and components I’m using:

  • Node.js – The popular Node.js platform built on Chrome’s JavaScript runtime. An event-driven and non-blocking
  [Read more...]
Geo Coding on the Cheap
+0 Vote Up -1Vote Down

Almost every business process or decision includes a WHERE!

Like IP addresses encode the destination of a domain name, with a geographic information system (GIS) you can encode and analyze the location data allowing you to reveal relationships, patterns, and trends.

Weather Decision Technologies

Show embedded map in full-screen mode

The world of GIS is confusing. Here is my understanding of how the peaces go together and how you can get started.

We describe locations on the earth in terms of



  [Read more...]
Comparing open source GIS implementations
+0 Vote Up -0Vote Down

In my quest to understand spatial GIS functionality, I have come to the ultimate goal: evaluation the actual database products themselves.

PostgreSQL / PostGIS

PostGIS is a variant of PostgreSQL with spatial extensions. The main reason for maintaining the GIS feature set outside of PostgreSQL proper seems to be licensing: the spatial extensions are LGPL GPL licensed.

PostGIS is widely recognized as the most mature and feature-rich GIS implementation for SQL databases (and perhaps any database), matched only by the costly Spatial extension for the Oracle Enterprise database. (See comparisons in the links below.)

read more

The OpenGIS standard
+0 Vote Up -0Vote Down

While the underlying index should be opaque to the user of a DBMS with spatial features, the API used to define spatial types and operate on them is of course more visible. The relevant standard in this space is often referred to as "OpenGIS", however the Open Geospatial Consortium in fact defines a long list of standards. The standard relevant to SQL databases is known more precisely as "OpenGIS Implementation Specification for Geographic information - Simple feature access - Part 2: SQL option" aka "Simple feature access".

It is not meaningful to recite the standard at length in my blog, my focus is instead on actual implementations that I will blog about later. The following points are however worth noting:

read more

Spatial data structures
+1 Vote Up -0Vote Down

I work for a company that is the leading supplier of automotive maps, and wants to be the leading supplier of online maps. So it was only a matter of time that I needed to learn more about how spatial extensions work in different open source databases. Let's start from the beginning, understanding various spatial data structures that are used in implementations...

Links are provided to Wikipedia articles - which are both comprehensive, yet easy to understand - for those who want to get a deeper understanding of each structure. All Wikipedia articles on spatial indexes are listed here: http://en.wikipedia.org/wiki/Spatial_index#Spatial_index

B-tree

read more

Implementing efficient Geo IP location system in MySQL
+0 Vote Up -0Vote Down

Often application needs to know where a user is physically located. The easiest way to figure that out is by looking up their IP address in a special database. It can all be implemented in MySQL, but I often see it done inefficiently. In my post I will show how to implement a complete solution that offers great performance.

Importing Geo IP data

First you will require a database mapping network addresses to real locations. There are various resources available, but I chose the one nginx web server uses with its geoip module. GeoLite City comes in CSV format and is available for download with no charge from MaxMind.

The archive contains two files. GeoLiteCity-Blocks.csv lists all IP

  [Read more...]
Screencast: MariaDB GIS demo
+5 Vote Up -0Vote Down

Here’s another MariaDB screencast, this time highlighting some of the GIS functionality in MariaDB.

(I recommend watching it in full screen 720p, so you can see the details.)

Some links and notes:

Acknowledgments:

  [Read more...]
Wrapping up MariaDB 2011
+4 Vote Up -0Vote Down

Parts of the world are already celebrating Christmas Eve and it’s time to relax and spend time with family and friends. Even if you don’t celebrate Christmas this is when there is time for less work. Here are a few words to round off MariaDB’s current state and where it’s heading.

This year culminated in MariaDB 5.3.3, the release candidate of 5.3. This is a significant release that makes years of work available by default in the database server. Earlier releases still required features to be explicitly switched on, but thanks to thorough testing assuring the quality of the new functionality we have now enabled them. It’s still called a release candidate which means it’s ready for general usage, but we want more user feedback before calling it stable. Make yourself familiar with the MariaDB

  [Read more...]
Finding things within some distance in SQL
+1 Vote Up -0Vote Down

One of the query optimization scenarios I’ve seen a lot over the years is finding something within some distance from a point. For example, finding people within some distance of yourself, apartments in a radius from a postal code, and so on.

These queries usually use the great-circle formula. That might be because Google finds lots of pages claiming that this is the right way to do a radius search. “The earth is not flat!”, they all say. That’s true, but it doesn’t mean that the great-circle formula is a good approach. It’s usually a really bad approach, in fact. It’s needlessly precise for most things, not precise enough for others, and it’s an expensive query to execute; all the trig functions tend to eat a bunch of CPU, and make it impossible to use ordinary indexes. This is true for all of the

  [Read more...]
MySQL GIS – Part 6
+3 Vote Up -0Vote Down

Is MySQL’s GIS really worth using?

Is GIS worth using in MySQL? In the past few post, I have explored what GIS is and how it is used. GIS encoded data is wonderful and can help with all kinds of cool queries.  I’m late getting this article written so lets get right to it.

The most common geographical  query is for all the point within some distance from a given point. I’ll try to focus on ways to answer this type of query. Accuracy of the answer is always important. Think carefully about your query. Do you want every pizza place within a radius of a port or within a square mile? Or, do you really want it within a miles walking distance?

I’m using the common city_lookup table for these tests. Here is the schema.

CREATE TABLE `city_lookup` (
`city_id` INT(7) NOT NULL DEFAULT '0',
`feature` VARCHAR(20) NULL DEFAULT
  [Read more...]
MySQL GIS – Part 4
+1 Vote Up -0Vote Down

WHAT CAN YOU DO WITH GEO DATA?

Geo spatial indexes are what make this type of data valuable.  With shape and point data you can find relationships between object in our physical world.  How close is the lightning in the storm front?  What homes where hailed on? (WDT) What schools are in my city?  With a list of homes for sale, how fare are they from their nearest school?  What picture where take in this area. (TwitPic)

Lets start with a simple grid of coordinates by creating a table for it call geom, adding our data points in and out of our grid and then searching with a small bounding box. The grid looks like this.

0,0 0,1 0,2 1,0 1,1 1,2 2,0 2,1 2,2
CREATE TABLE geom
  [Read more...]
MySQL GIS – Part 3
+4 Vote Up -0Vote Down

What data is available?

GEO data is expensive to create, so has been created by governments.  In the past governments charged for this data.  In 1980 the USGS was charging $300 (usd) per county for Oklahoma GEO data. (I complained to my congressman.) Today, a quick Internet search turns up lots of free GIS data.

I was hoping to find a neat collection of basic GEO data.   It would be nice if there was one place you could get world political borders (Polygons), postal codes (Polygons) and  points of interest like hospitals and airports.  What you can find is lots of  lists, often collections of odd data created for a virility of complex political purpose.  For example, The Global Change Master Directory is a large list of data sources on earth

  [Read more...]
MySQL GIS – Part 1
+3 Vote Up -1Vote Down

In my business (weather) we use lots map based (Geo) information.  Almost every table has latitude and longitude. Working with this kind of data can be exciting and frustrating.  This should give you a quick start into GIS with MySQL.

“A geographic information system (GIS), or geographical information system, is any system that captures, stores, analyzes, manages, and presents data that are linked to location. In the simplest terms, GIS is the merging of cartography, statistical analysis, and database technology. GIS systems are used in cartography, remote sensing, land surveying,

  [Read more...]
Log Buffer #178, a Carnival of the Vanities for DBAs
+2 Vote Up -1Vote Down
Dave Edwards has offered me to write this week's Log Buffer, and I couldn't help but jump at the opportunity. I'll dive straight into it.

Oracle

I'll start with Oracle, the dust of the Sun acquisition has settled, so maybe it's time to return our attention to the regular issues.

Lets start with Hemant Chitale's Common Error series and his Some Common Errors - 2 - NOLOGGING as a Hint explaining what to expect from NOLOGGING. Kamran Agayev offers us an insight into Hemant's personality with his Exclusive Interview with Hemant K Chitale. My favorite quote is:

Do you





  [Read more...]
Updated MySQL 5.1.35-GIS snapshot binaries are now available
Employee +5 Vote Up -0Vote Down

We've now published a new set of binary packages including the extended GIS functionality from the mysql-5.1-wl1326 source tree.

This release is based on the MySQL 5.1.35 code base and fixes the bugs mentioned below. It includes some improvements to the GIS functionality as well, so please use these packages for future testing of the MySQL GIS functionality.

The following GIS-related bugs were fixed in this version:

  • Bug#31753: Buffer/area functions only return first row of set
  • Bug#32032: Contains() does not work on MultiPolygons, may force a disconnect and/or result in extremely long query times.
  • Bug#32100: contains,

  [Read more...]
Reverse Geocoding using MySQL GIS
+0 Vote Up -0Vote Down
Normal geocoding is the process of taking an address and converting it into latitude and longitude. Reverse geocoding goes the other direction; given a latitude and longitude we want to know the closest street address.

Google Maps API, Yahoo Maps, Microsoft, GeoNames.Org and others can provide both these services, but they either cost, or have restrictive terms of service that makes them unavailable for high volume applications. See Geo::Coder::US (for US addresses only). This uses a local data source, and requires you to download the data from the US Census manually. It can import a format known as

  [Read more...]
MySQL and Geospatial Data
Employee +2 Vote Up -0Vote Down

MySQL has had basic support for Geospatial Data since 4.1, but has lacked some of the features of the OpenGIS specifications since then. The good news is, this is rapidly changing. Our own Holyfoot has been hammering away at WorkLog #1327, to provide precise functions for our GIS support.

Even better, it’s fast. How fast? Well, the good people at Oki Labs, apart from having implemented several new GIS functions for MySQL, have done some benchmarking, and it’s looking good. If you’ll excuse the cliched comparison to Postgres, here are the response times (seconds) of MySQL GIS vs. PostGIS in Oki’s test:

Connections PostGIS MySQL 1 1.817 0.220 100 10.517 0.557  [Read more...]
Showing entries 1 to 26

Planet MySQL © 1995, 2014, Oracle Corporation and/or its affiliates   Legal Policies | Your Privacy Rights | Terms of Use

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.