Saturday, May 14, 2016

Importing Geonames Database

I have recently had to design a system which will require storing of large number of addresses.

Most people get away with storing Country, City, Region names as text fields. Some use a database table for Country names However allowing user to input own text causes same place having many different names. Some people use the full name, some use an abbrevation, some enters with typos etc.

In addition, char fields occupy more space. Think about storing Turku. The proper location is Finland -> Southwest Finland -> Turku. Storing this information as text would require at least 5-20 bytes of storage. If we had a database of locations, we could get away with an unsigned int value which would occupy 4 bytes only.

The best open database, which provided most complete location data was Geonames database. However the documentation of Geonames is rather bad and it was not so easy to import into Laravel.

I found few packages for importing Geonames database into Laravel however they were quite incomplete and slow. Because of this, I decided to write my own package and put it to packagist and github. It is named yurtesen/geonames and I believe it is the fastest, best documented and most complete Geonames importer available.

The installation to Laravel is quite easy:

Please include the following require in your composer.json :
{
    "require": {
        "yurtesen/geonames": "dev-master"
    }
}
or install using command line :

composer require yurtesen/geonames:dev-master

after installation, you will need to add the service provider in your config\app.php to 'providers' array
 
Yurtesen\Geonames\GeonamesServiceProvider::class,

Please follow my wiki for examples and how to import the database.

>>> GeonamesGeoname::City('Turku')->first()->countryInfo;
=> Yurtesen\Geonames\Models\GeonamesCountryInfo {#876
     iso: "FI",
     iso3: "FIN",
     iso_numeric: "246",
     fips: "FI",
     country: "Finland",
     capital: "Helsinki",
     area: 337030,
     population: 5244000,
     continent_code: "EU",
     continent_id: 6255148,
     tld: ".fi",
     currency_code: "EUR",
     currency_name: "Euro",
     phone: "358",
     postal_code_format: "#####",
     postal_code_regex: "^(?:FI)*(\d{5})$",
     languages: "fi-FI,sv-FI,smn",
     geoname_id: 660013,
     neighbors: "NO,RU,SE",
     equivalent_fips_code: "",
   }
>>>