What are Lucene indexes and how does a reindex action work?

Lucene indexes

The work of XTM Cloud translation memory and XTM Cloud terminology is based on the so-called Lucene indexes. These are special files which are created on in the back-end side , in the following ordersequence: for each customer → each language used under for this customer.

Reindex action

The reindexing process consists in involves recreating translation memory/terminology Lucene indexes and is used once there are problems with TM matching in projects and a given particular term is not highlighted in XTM Workbench respectively.

Reindex The reindex action is not an update action, as there will not be any change within XTM Cloud or a downtime, : only the Lucene index will be recreated.

In order to make sure To check if the reindex reindexing action has successfully addressed resolved the issue, you can, for instance, check if the same TM entry is displayed in XTM Workbench and in TM Manager (or, in the case of reindexing terminology, see if a given particular term is properly highlighted in blue, in XTM Workbench). No additional tests checks are needed after the process is has finished. Once the action has been performed in the reported projects, after performing and an update has been performed within segments, all correct matches should be available, as they will be returned correctly from the TM database.

Constraints

Reindex The reindex itself is performed per client and per language, meaning that the exemplary example Portuguese (Brazil) language (pt_BR) will have its indexes recreated under for all customers.

...

...

...

...

...

...

The overall total time the process takes depends on the size of how much TM data that has to must be reindexed. It can take from a couple of minutes to several hours. After the process has finished,  the the support agent checks the logs for confirmation if that the action has been performed correctly.

When it comes to reindexing terminology, the action Terminology reindexing is performed per client on , for all language combinations at once - there is no division here, not separately.

...

How to recognize translation memory/terminology indexing issues

...

How to recognize translation memory (TM) indexing issues

...

The most common symptoms of TM indexing issues are as follows:

• A given particular TM match is shown in XTM Workbench, but you cannot find it in the TM Manager.

• A phrase search is not returning as many matches in the TM Manager as it should. When searching If XTM International Support staff search for every phrase from the project/all phrases, or through the phrases by record ID, XTM they will provide you with a list of records, but not when looking they look for a specific phrase.

• Concordance is not displaying matches or is displaying too few matches.

• The reindexing Reindexing action should also be performed after large-scale operations on the translation memory on in the database (for instance, changing the status of multiple TM records from Not approved to Approved). But this is something we should already be well aware of. However, if the batch approval is done through performed via the XTM Cloud UI , then no reindex reindexing is necessary.

XTM Workbench slow performance issue

If XTM Workbench is preforming performing exceptionally slowly, some translations inserted into segments might not be saved, which usually results in the a warning message (see the screenshot below).

...

This situation is in all likelihood caused by the excessive number of most probably occurs because there are too many Lucene indexes files present on the server, which, in turn, might be due to the large number of XTM Cloud customers. For example, let’s say imagine there are 10 customers, and we have 10 languages for each of them we have got 10 languages, which now means , so there are 100 directories with Lucene indexes files, each containing 2 files each. This already results in 200 files for just 10 customers already. In the case of . If there are even more customers with multiple language combinations, at a later time, the number of Lucene directories will increase proportionately. This might cause performance issues because XTM Cloud is having difficulties in accessing files in those directories due to their huge number.

The most effective solution to this is to just remove some of the indexes of those XTM Cloud customers who are either not being used or are inactive (this action, however, needs to be performed by the XTM International Support team because deleting the TM for a given particular customer and/or deleting this customer from the XTM Cloud UI will not remove Lucene indexes files on in the back-end side, as the files are stored even for the deleted customers). These indexes can also be further safely restored by the reindexing process so and do not require deleting their respective customers do not need to be deleted in the UI.

How to recognize terminology indexing issues

...

Actually, there is only one XTM Cloud UI “symptom” "symptom" that would indicate the need for a terminology reindexa need to reindex the terminology. If you have got a certain particular term is visible in the terminology database in the UI for a given particular language combination, make sure that you have included it in your current project (by creating the project under for the customer which that holds this terminology data or by incorporating the relevant resources from another customer to in the project). If you now enter XTM Workbench in the “Edit” "Edit" mode, activate a segment and see that the term in question will not highlight blue, this might constitute be a valid reason to perform a reindexing action.

Furthermore, if you happen to have Also check if the Disable term decoration option is activated globally in Configuration → Settings → Translation → Terminology → Terminology options:

...

You have it switched off during the project creation:

...

What does the XTM International Support team need

...

to start analyzing

...

an issue

...

involving TM indexing?

Please, see the essentials that Check which essential information you need to be provided provide to the XTM International Support team in the case of TM indexing issues:

• the phrase you were searching for in the TM Manager/concordance;,

• the username or ID of a the user who was conducting performing the search;,

• language combination;

• the customer for which the search was done performed (if in concordance).