Keyword clustering

Keyword clustering is a practice search engine optimization (SEO) professionals use to segment target search terms into groups (clusters) relevant to each page of the website. After keyword research, search engine professionals cluster keywords into small groups which they spread across pages of the website to achieve higher rankings in the search engine results (SERP). Keyword clustering is a fully automated process performed by keyword clustering tools.

The term and the first principles were first introduced in 2015 by the Russian search engine optimization expert Alexey Chekushin. The SERP-based keyword clustering tool Just-Magic was released in the same year in Russia.

Method

Keyword clustering is based on the first ten search results (TOP-10) regardless of the search engine or custom settings. The TOP 10 search results are the first ten listings that a search engine shows for a certain search query. In most cases, the TOP-10 matches the first page of the search results.

The general algorithm of keyword clustering includes four steps that a tool completes to cluster keywords:

1. The tool takes keywords one by one from the list and sends them as search queries to the search engine. It scans the search results, pulls the ten first search listings, and matches them to each keyword from the list.
2. If a search engine returns the same search listings for two different keywords and the number of this listings is enough to trigger clustering, two keywords will be grouped together (clustered).
3. A minimum number of matches in the search results that trigger keyword clustering is called the clustering level. The clustering level is customizable, and most tools allow changing it in the settings prior to the keyword clustering. The clustering level affects the number of groups and keywords in the group after clustering. The higher clustering level produces more groups with fewer keywords in every group. This happens due to a minimum chance to have 9-10 matching documents on the search results page (it would include almost all pages in the TOP-10 of search results). On the opposite, the clustering level 1 or 2 will create a few groups with a lot of keywords in each of them. There are certain exceptions, but they are not common.
4. If a tool finds no matching URLs in the TOP-10 of the search results, these keywords are sent into a separate group.

Apart from the clustering level, there are also different types of the keyword clustering that affect the way all keywords within one group are linked to each other. Similar to the clustering level, the type of keyword clustering can be set prior to the clustering.

Types

Soft

A keyword clustering tool scans the list of keywords and then picks the most popular keyword. The most popular keyword is a keyword with the highest search volume. Then a tool compares the TOP 10 search result listings that showed up for the taken keyword to the TOP10 search results that showed up for another keyword to detect the number of matching URLs. If the detected number matches the selected grouping level, the keywords are grouped together.

As the result, all keywords within one group will be related to the keyword with the highest search volume, but they will not necessarily be related to each other (will not necessarily have matching URLs with each other).

Moderate

A keyword clustering tool scans the list of keywords and then picks a keyword with the highest search volume. Then a tool compares the TOP 10 search result listings that showed up for the taken keyword to the TOP10 search results that showed up for another keyword to detect the number of matching URLs. At the same time, a tool compares all keywords to each other. If the detected number of identical search listings matches the selected grouping level, the keywords are grouped together.

As the result, every keyword within one group will have a related keyword with matching URL or URLs in the same group. But two random pairs of keywords will not necessarily have matching URLs.

Hard

A keyword clustering tool scans the list of keywords and then picks a keyword with the highest search volume. Then a tool compares the TOP 10 search result listings that showed up for the taken keyword to the TOP10 search results that showed up for another keyword to detect the number of matching URLs. At the same time, a tool compares all keywords to each other and all matching URLs in the detected pairs. If the detected number of identical search listings matches the selected grouping level, the keywords are grouped together.

As the result, all keywords within a group will be related to each other by having the same matching URLs.

History

As the major part of the website optimization process, SEO professionals research keywords to get a pool of target search terms which they use to promote their website and get higher rankings in the search results. After they get a list of keywords related to the contents of the website, they segment the list into smaller groups. Each group is usually relevant to a certain page of the website or a certain topic. Originally, SEO professionals had to group out the keyword pool manually, by picking a keyword after keyword and identifying possible clusters. It could be done with the help of Google Adwords Keyword Tool but it still required a lot of manual work. There was a need in an automated algorithm that would segment keywords into clusters on auto-pilot.

Lemma-based keyword grouping

Prior to the keyword clustering, search engine optimization experts developed keyword grouping tools based on the process known as lemmatisation. Lemma is a base or dictionary form of a word (without inflectional endings). In linguistics, lemmatisation is a process of grouping together the different inflected forms of a word so they can be analyzed as a single item.

In search engine optimization, the process of lemmatisation includes four steps:

1. Keywords are picked from the list one-by-one;
2. Keywords are broken down into lemmas;
3. Keywords with the same lemmas are detected;
4. Keywords with matching lemmas are grouped together.

As the result, a search engine optimization specialist gets a list of keyword groups. Each keyword in a certain group has matching lemmas with all other keywords within this group.

SERP-based

Compared to lemma-based keyword grouping, SERP-based keyword clustering produces groups of keywords that might reveal no morphological matches, but will have matches in the search results. It allows search engine professionals getting a keyword structure close to what a search engine dictates.

Soft and Hard type of keyword clustering and the general algorithm was introduced by the Russian SEO expert Alexey Chekush in 2015. In the same year, he developed and introduced the automated tool that could cluster keywords.