Ariadna — Open Source Geocoder for Transportation Apps

The process of creating a geocoder based on ElasticSearch, which searches for coordinates by synonyms and names of places, looks for crossroads and addresses in a certain radius, and knows how to reverse geocoding and automatically update with new data from drivers. The repository is available by link:

Prehistory

Copy link

When we decided to design a geocoder for the needs of Namba-Taxi, we encountered face with a lack of data.

What we don't have:

• The full map of Yandex, Google or 2Gis
• Confidence in GPS data

What we have:

• Very mixed input;
• We use OpenStreetMap somewhere;
• Our accumulated address database with coordinates.

What can operators enter?

Copy link

Operators can enter addresses in different formats:

1. Street house
2. Intersection
3. Name of institution
4. Point name
5. Housing estate
6. Microdistrict street house

And there are a lot of such options, for example:

1. Kyiv Street 28
2. Kyiv Street/Soviet Street
3. 5–42
4. 5 micro district Soviet 42
5. CSM (Wallmart)
6. Cafe Ashot’s

Designing

Copy link

The following algorithm was laid down:

1. First, we obtain the geometry of large settlements (cities, capitals, villages, residential areas);
2. We unload all possible addresses and correlate them to the necessary residential array, city, and other settlements, setting the desired value;
3. We unload all roads;
4. Looking for the intersection of roads;
5. Put everything in the index;
6. Searching.

Implementation

Copy link

We have OSM as the main data source, so the filters, in order to get data from us, look like this:

1. Place = city, place = village, place = suburb, place = town, place = neighbourhood — get all the neighborhoods.
2. addr: street + addr: housenumber, amenity, shop, addr: housenumber — get addresses and names of institutions.
3. highway — get all the roads.

There were difficulties with the search for English-language names in Russian. As I tried to solve it:

1. Simple automatic transliteration into Russian. As a result, it turned out to be absurd and incorrect. Example of data conversion looked like this: City House -> Цити Хоусе.
2. Get the transcription of the word and, after that, make its transliteration. It turned out something like Adrenaline rush -> Эдреналин Рэш. Possibly, but you need a Russian accent, such as адреналин раш.
3. Automatically transliterate all data using the replacement dictionary. It is the solution. Simple transliteration works tolerably. The dictionary was filled in principle quickly through several runs on the data.

We sorted out this, to this point, we are already getting data that:

1. Normalized and brought to the Russian language;
2. Addresses are given to the format — country, city, village or village, neighborhood or residential area, street, house.

The next part of the quest is to find the intersections of the roads. I made it on a fast and got a very slow implementation, the complexity of O(n²). As a temporary output, I used Postgres+postgis to find the intersections until I found a good algorithm for finding intersections.

As a result, a good data parser with osm has been created, which puts the data into ElasticSearch, which got a simple name “importer.”

Automation

Copy link

Considering that we should constantly pump out and create indexes in the ElasticSearch soon became fed up, and the updater component appeared. There was also an automatic configuration in the JSON format.

The process of downloading the file and importing it into ElasticSearch was automated. Additionally, there was an opportunity to update the data in the ElasticSearch without downtime, thanks to the aliases.

How it works:

1. Updater downloads the file;
2. Recognizes the current version of the index from the config;
3. Increments the version and creates a new index;
4. Fills it with data;
5. Changes aliases;
6. Removes the old index.

I received such benefits from this:

1. Write a config;
2. Run the ./ariadna update;
3. Go to drink coffee;
4. Get the readily customized index.

Also, for convenience, a simple web interface with a map and search capability was attached.

Automatic replenishment of data

Copy link

In addition to the OSM, we still have many drivers and operators who are hammering orders. Accordingly, we have a name and coordinates. So, the following scheme was made:

1. Tracks of drivers are stored in the drivers_data index;
2. Data from the OSM is stored in the osm_data index;
3. They are combined through the alias addresses on which the address is searched.

Data from drivers are recorded if we have an error in certain coordinates more than 200 meters.

What can the Ariadna geocoder do?

Copy link

1. Search for coordinates by synonyms. For example, CVK — ChampagneVinKombinat;
2. Search for addresses in a certain radius (for example, for themselves, with a search for addresses 30 km from the city center);
3. Search by the name of establishments (cafe Ashot’s, for example);
4. Search crossroads;
5. Search for addresses in neighborhoods and lived arrays;
6. Reverse geocoding;
7. Automatically replenished with new data from drivers.

What components does the geocoder consist of:

Copy link

1. Data Importer;
2. Data ancestor;
3. Web interface.

Minuses

Copy link

1. Tested only for Kyrgyzstan;
2. No demo (Although you can see it in the Namba Taxi application when it determines your address by location);
3. No support for all addressing schemes.

Therefore, I hope someone will help him finish and for a good search for other countries and cities.

If someone has found the project interesting, then I’m not against any criticism, a pool of questioners, issues on GitHub, and feedback in general.