Semantic search

Semantic search uses an understanding of the meaning of the search phrase to match products with similar semantic content. This approach allows for flexibility in matching, as products do not have to contain every word in the query.

Setting up

Semantic search can be configured using the AI settings endpoints in the API.

The following must be configured:

Please contact your solution partner for help to set this up.

Fallback search

If a Unified Search returns zero results, a fallback search is performed using semantic search along with the Did You Mean suggestion.

Use cases

Semantic search significantly enhances the user experience by understanding the intent and contextual meaning behind search queries. This advanced search capability is particularly beneficial in two primary scenarios.

Synonym recognition

Semantic search inherently understands the relationships between words, including synonyms, without the need for manual entry in the search system.

Complex queries

Semantic search excels at interpreting long-winded or complex queries that may not have exact matches in the product database.

Data usage

Semantic search works by analyzing the semantic meaning embedded in product descriptions. To ensure an accurate and comprehensive semantic representation for search purposes, all attributes marked as searchable are used. Attributes not marked as searchable do not contribute to the semantic search process.

Improving results

Semantic search results can be improved by increasing the quality and specificity of product data. Semantic search is based on understanding the intent and contextual meaning behind a search query, rather than simply matching keywords

Enhance product descriptions

Write detailed and clear product descriptions that accurately reflect the product's features, benefits, and use cases. Ensure that the language used is aligned with the terms customers might use to search for these products.

Intent-based semantic search

Instead of directly embedding the user's search phrase, BAIA makes an informed guess about the user's intent based on the instruction prompt. This inferred intent is then used to enhance the search results.

For instance if the user searches for kornisjons but BAIA have been told through the instruction prompt that the shop is a food and grocery retailer it will search for the semantic meaning of Cornichons Pickled Cucumbers this will make it easier to find the correct products.

Instruction prompt

An important part of intent-based semantic search is to provide BAIA with a good instruction prompt. This will heavily improve the results. A good instruction prompt is just a short per segment description of the shop the search solution is used on.

Intent check

Before a new search phrase is used for semantic search, BAIA describes what the shopper is most likely looking for, and how sure it is.

• Phrases that clearly describe products are used for semantic search. The descriptions are used to rank the products, so the results match what the shopper means.
• Phrases that are about price, delivery or the store itself, or that cannot be read as a search, are not used for semantic search.
• Phrases where BAIA is unsure are held back until a person reviews them.

Search phrases that were already used for semantic search before the intent check was enabled keep working as before.

Reviewing search phrases

The Semantic search page in the Semantic Search menu shows how semantic search handles the search phrases of a segment. It also lets you guide the phrases that the intent check is unsure about.

Summary

The top of the page shows three numbers:

• Matched automatically: phrases that semantic search uses without a review. The number is also shown as a share of all phrases that are not skipped.
• Zero results recovered: phrases that came from searches with no results and are now used by semantic search.
• Skipped automatically: phrases that semantic search does not use, because the search is not for a product.

Review

The Review tab lists the phrases where semantic search is not sure what shoppers mean. The most searched phrases come first. For each phrase, the tab shows how often it was searched in the last 30 days, its source (Top phrase, Zero result or Candidate), and the readings that BAIA suggests. Each reading has a likelihood: Likely, Possible or Unlikely.

For each phrase, you can:

• Accept: choose one or more of the suggested readings, and write your own if none of them fit. A phrase needs at least one reading and can use at most three. When the accept is published, semantic search uses only the chosen readings for the phrase.
• Discard phrase: semantic search does not use the phrase, and shoppers keep the result they see today. The discard can be undone right away.

An accepted phrase stays in the Review tab, greyed out, so the list does not change under you:

• Pending publication: the accept is not published yet. The phrase is not processed before it is published, so neither the preview nor shoppers get a semantic result for it yet.
• Processing: the accept is published, and the phrase is processed in the background. This takes some time after the publication. There is no time limit.

When the phrase is processed, it leaves the Review tab and is listed in the Inspect tab. Phrases that are pending publication or processing are not counted in the number of phrases to review.

A decision made on the page is never overwritten automatically.

Publishing decisions

The decisions you make on the page are part of the publication of the segment, like the other settings. This covers accepting and discarding a phrase, moving a phrase back to review, the chosen readings, and the products you remove or keep. Shoppers keep getting the published result until you publish. The preview, such as the search result dialog and the diagnostics, shows your unpublished decisions before you publish them.

Phrases that semantic search accepts or skips automatically are not part of a publication. These decisions take effect at once, so new phrases get a semantic result also when you seldom publish.

Inspect

The Inspect tab lists every phrase that semantic search uses, with how often it was searched, its inflections and its source. Discarded phrases are listed at the bottom, marked Discarded from review. You can search for a phrase, sort by phrase or by number of searches, and filter by source or by discarded phrases.

The Your changes column shows how many products you removed from the phrase, for example 1 removed, or None.

A phrase can also be marked:

• No semantic matches: the phrase is processed, and semantic search found no product that fits it. Shoppers get keyword results only. If a later run finds matching products, for example after new products are added, the mark goes away.
• Unpublished changes: the phrase has decisions that are not published yet, for example a discard or removed products.

A phrase you discard in the Inspect tab is shown as discarded at once. Shoppers keep getting its semantic result until the discard is published.

Adjust

Select a phrase in the Inspect tab, or its Adjust button, to see:

• The readings the phrase uses, with the most likely first.
• The products found for the phrase. Each product is marked Semantic, Keyword or both, depending on how it was found.

For a discarded phrase, Move back to review returns the phrase to the Review tab. Products cannot be removed from a discarded phrase.

Relevance

Some products are found by keyword search only, because a word in the phrase matches them, even though they are about something else. To help you find them, each product shows how relevant BAIA found it for the phrase:

• Low relevance: keyword search found the product, and BAIA judged that it does not fit the phrase. These products are marked with a warning.
• Unknown relevance: keyword search found the product, but BAIA has not judged it for the phrase.
• Products that BAIA judged relevant have no label.

Bizzkit Ecommerce Search sets the relevance, and you cannot change it.

The dialog header shows how many products shoppers see for the phrase. At the top of the dialog, the Total box shows all products found for the phrase, including the ones you removed. The Low relevance box shows how many products have low relevance, their share of the total, and how many of them you have not decided on yet. When you have removed or kept all of them, the box says All decided. Select a box to show only those products.

Judging keyword matches

When judging of keyword matches is enabled, BAIA also judges the products that keyword search finds for an accepted phrase, not only the products that semantic search finds. A product that BAIA finds relevant is shown together with the semantic matches, and the dialog marks it as found by both keyword and semantic search. A product with low relevance stays in the result until you remove it.

Keyword matches are judged in the background, so a product can still show Unknown relevance for a while. A product also stays unknown when it is not among the products judged for the phrase, or when judging of keyword matches is not enabled. Hover over Unknown relevance in the dialog for a short explanation.

Removing and keeping products

Every product has a Remove and a Keep button, whatever its relevance:

• Remove takes the product out of the search result for the phrase and its inflections, whether keyword search, semantic search or both found it. The product is also left out of the facets. Shoppers still find it when they search for other phrases.
• Keep confirms that the product fits the phrase. It does not change what shoppers see. A kept product is marked Kept, and a kept product with low relevance counts as decided.
• Undo takes back an earlier decision. A removed product stays in the list, marked Removed, so you can undo it later.

Nothing is stored until you select Save changes. Cancel closes the dialog without changing anything.

Removed and kept products are part of the publication. The dialog shows them as soon as you save, because it uses the preview. Shoppers get them when you publish. A phrase can have at most 1000 removed products.

Products can only be removed from or kept for a phrase that semantic search has already processed. If the phrase has no semantic results yet, the changes cannot be saved.

Phrases that are inflections of each other share their removed and kept products. If semantic search stops using a phrase, its removed and kept products are cleared, and you start over if the phrase is used again.

Existing phrases

Phrases that were used for semantic search before the review was available have no suggested readings. Their reading is the search phrase itself, marked Added manually, and they are counted as matched automatically.