Understanding Ecommerce Search

E-commerce is a complex and ever-evolving field, with businesses constantly facing challenges in managing search in an e-commerce solution. Our platform enables businesses to easily manage and update complex dictionaries, while also providing advanced tools for sorting and ranking search results based on commercial values such as popularity, relevance, and profitability. This allows businesses to optimize search results for their customers and improve overall conversion rates.

A significant advantage of Ecommerce Search (ECS) is the user-friendly interface that enables the management of dictionaries, sorting and ranking, as well as defining fields and facets to be returned. Previously, implementing these changes necessitated a developer's involvement to modify the code. Ecommerce Search now empowers search managers to make these adjustments independently, without the need for developer assistance. Furthermore, Ecommerce Search allows users to obtain early feedback through diagnostics and context previews, helping them identify and correct errors before publication. The platform also assists in enhancing weak searches by providing insights via the dashboard. This comprehensive approach ensures an optimized search experience for users while minimizing the reliance on developers.

In this article, we'll explore the different aspects of Ecommerce Search and how it can help businesses overcome these challenges.

Recurring example

This article will provide an overview of Ecommerce Search using a recurring example of a B2C web shop selling Bizzkit merchandise such as coffee mugs, t-shirts, hoodies, and more.

For a basic introduction to the topic, consider the following description of a user searching for a specific item:

• The user opens a web browser and navigates to a web shops homepage, then performs a global search for the term "cup"
• The browser (or backend) sends an API request to ECS, passing the search term, facets (filters) and a so called scope as arguments
• ECS conducts a search for SKUs using both "cup" and "mug" as search terms
• Based on the scope ECS knows which fields to return, including SKU name, price, description, and URL for an image of each SKU found
• ECS sorts and ranks the SKUs based on their contribution margin and availability in stock
• ECS sends the search results, relevant facets and other information back to the browser
• The browser displays the results and facets to the user

Throughout the article, we will utilize this example to highlight various features of Ecommerce Search.

Application interfaces

Ecommerce Search is a cloud-based product that provides businesses with an extensive user interface for managing dictionaries of hypernyms, synonyms, and other relevant terms used in search. The platform also allows search managers to control the sorting and ranking of search results by adjusting commercial properties of specific parameters.

In addition, Ecommerce Search offers three standard REST APIs that enable businesses to manage administrative features through code and facilitate the actual search process. These APIs allow for seamless integration with existing systems and enable developers to easily build custom features to meet the unique needs of their business.

Segments

Segments are a crucial aspect of Ecommerce Search, allowing businesses to differentiate search engine behavior based on the needs of different customer segments. A segment is based on culture, and each segment is restricted to a single language, as the stemming algorithm varies by language.

For businesses operating in different types of markets, such as B2B and B2C, different sets of dictionaries may be required. In such cases, businesses can create two segments with separate dictionaries, one for B2B and one for B2C. This is particularly useful in cases where the vocabulary of B2B customers differs from that of B2C customers. For example, while B2C customers might simply search for a hammer or saw, B2B customers might search for a specific tool they need.

When a segment is created the following features can automatically be created and connected to the new segment:

• Dictionaries is a collection of settings to fine-tune the text analysis of the search engine.
• Scopes defines the type of interaction the client wants with the search engine. The primary function of a scope is to define what fields should be included in a search result.
• Contexts pertain to commercial sorting and comprise user-controlled weights. These weights determine the relevance of products in response to a specific search phrase. ECS identifies an appropriate context based on predefined context conditions.
  • Each segment has a pregenerated fallback context which is used for sorting and ranking if no other matching context can be found.
• Allowed filters control which filters can be used in search requests, scopes, and context facets (facets refer to the filterable properties of a product).
• Searchable fields determine which fields are utilized during a search.

Segments can be created using the Bizzkit Ecommerce Search UI or API.

Text adjustment and analysis

The core of Ecommerce Search lies in the text analysis of product data and search queries. One of the main benefits is that search administrators can customize the search experience without any technical expertise, simply by understanding the language. To provide administrators with a high degree of customizability, Ecommerce Search offers several text analysis tools. These tools enable administrators to modify how product information is matched with search queries, enhancing the relevance and accuracy of search results.

Text analyses are based on dictionaries, and each dictionary has a language associated with it. A segment can utilize multiple dictionaries of the same type, enabling businesses to attach a general dictionary with linguistic data that can be reused across all segments, and a specialized dictionary with linguistic data specific to that segment. It's also possible to share dictionaries between segments. By utilizing this feature, businesses can for example choose to share a common set of synonyms between their B2B and B2C solution.

There exists a dictionary type for each of the following general linguistic data:

• Hypernyms are words that are superset of other words. For example, pasta is a general description, with lasagna, fettuccine, and tortelli being subsets, also known as hyponyms, within the general category of pasta

• Synonyms are words that have the same meaning in the given context. E.g., pants and trousers, mug and cup, or space and universe

• Misspellings are misspellings of other words

• Irregular words are words that do not follow the normal grammatical rules for conjugation. A common case in English is nouns whose plural form does not end in s or es (e.g., woman, women)

• Ignored words are words that will be ignored if given as part of a search query. It is typically common words that have little meaning on their own like the, is, of, or be

More advanced text analysis includes Stemming overrides that are used to override the stemming of a word. Stemming overrides should only be used to prevent words with a different meaning from being stemmed to the same word. E.g., preventing Training being stemmed to Train and Universe being stemmed to Univers.

Recurring example

Expanding upon the previous example, the search manager has entered the specific synonym for "mug" and "cup," causing ECS to conduct a search using both terms.

Ensuring Reliable Search Settings

The management of dictionaries etc. is based on publications, allowing for testing, and reverting of settings before they impact end-users. When a search is conducted, the published settings for the segment will be used, rather than the unpublished settings in the admin interface. This approach enables businesses to test the effects of settings before implementing them and to revert to old settings if new ones cause unexpected issues.

By utilizing publications, businesses can ensure a smooth and reliable search experience for their customers.

Dashboards

Dashboard is a tool in the UI that can be used to start improving search results for end users by showing tables containing key metrics for the search experience of customers.

Ecommerce Search will regularly perform searches based on the ingested metrics to analyze the results and present them as tables. Several widgets are available:

• 0 - Product results (30 days): Displays the most popular searches in the last 30 days with no matching products.
• Popular searches last week: Displays the most popular searches in the last 7 days, along with the number of searches performed.
• Number of products in popular searches last week: Displays the most popular searches in the last 7 days and the number of products that match the search phrase.
• Top phrases to improve: Displays metrics for enhancing search experience. Findability indicates how effectively a phrase yields desired results. Satisfaction combines findability and search frequency to identify key phrases to optimize.
• Tracked phrases since last week: Displays the tracked phrases the last 7 days.

These widgets provide valuable insights into search behavior and can help businesses optimize their search engine performance to better meet their customers' needs.

From a dashboard a search manager can easily create and test misspellings, synonyms, hypernyms, etc.

Diagnostics

Diagnostics is a useful tool for businesses looking to diagnose search engine behavior and test new configurations. It allows users to mimic search behavior on a live site using unpublished data, which means that businesses can test changes before implementing them on the live site. Additionally, pipeline analysis is a feature of diagnostics that provides a deeper understanding of how the search engine interprets search phrases and how different rules influence search results. The pipeline analysis tool allows businesses to see the effects of text analysis rules for the current segment, making it easier to identify modifications needed to improve search results.

Search results and missing products are two additional features of diagnostics. Search results display all products found by the search phrase and provide insight into why specific products were matched by the search engine. Missing products, on the other hand, allow businesses to compare specific products against search phrases and determine why those products are not appearing in search results. By combining missing products with the pipeline analysis tool, businesses can identify which modifications are needed in the text analysis rules to improve the accuracy of their search results. Overall, diagnostics is a powerful tool that can help businesses optimize their search engine performance and improve their customers' search experience.

Suggestions

Suggestions in Ecommerce Search are phrases that can be proposed to users as they type in the search box of a web shop. The aim of these suggestions is to direct users towards phrases that yield the most relevant results. The implementation of these suggestions from the Ecommerce Search API varies, as there are three types to choose from:

• Suggestions are phrases that might interest the user. For example, searching for 'hat' could suggest 'gray hat', 'green hat', 'red hat', etc. Similarly, a search for 'poster' might suggest 'red moon poster'.
• Related tags expand a search phrase by adding a tag, thus creating a new search possibility. For instance, if a user searches for 'red', the related tags could be 'hoodie', 't-shirt', and 'hat', leading to potential new searches like 'red hoodie', 'red t-shirt', and 'red hat'.
• Did you mean suggestions offer users an alternative search phrase in case of minor typos or misspellings. For example, if a user searches for 'grean', the 'did you mean' suggestion might be 'green'.

Managing suggestions and related tags is done through the UI. There's also a straightforward method to automate the creation of suggestions using a suggestion generator. This tool automatically generates suggestions based on fields like product name, color, brand, etc.

Suggestion candidates can originate from various sources, such as product names from ERP/PIM systems, brands, or top searches from engines like Google Analytics. To prevent unwanted suggestions from external sources, taboos are words or phrases that should not appear in search suggestions. These might include products not suitable for children or adolescents, which should only be found via direct product search.

Recurring example

To build upon the previous example, the search manager could create suggestions such as "CMS mug" and "PIM mug". This would enable the web shop solution to send an API request to ECS using a search term like "cms" and receive a relevant suggestion such as "CMS mug" to display to the user.

Commercial sorting

Ecommerce Search offers a valuable feature that allows businesses to sort product results based on commercial properties and user affinities. This is particularly useful when there are multiple products with equal search relevance, as commercial sorting can be used as a tiebreaker to prioritize certain products. Unlike traditional search relevance, which is based solely on the search phrase, commercial sorting considers a product's commercial relevance.

Using commercial sorting, businesses can ensure that the products most likely to be purchased are prominently displayed at the top of the search results. This is especially beneficial for products with a higher profit margin or significant commercial value, such as those that help build brand awareness. By prioritizing these products, businesses can increase their chances of converting customers and boosting sales.

Commercial properties

A product's commercial score is determined by the commercial properties of the product, depending on the current context and user. The context determines the weight of the commercial parameters of the product while the user has an affinity towards certain commercial affinities of the product. These two contributions are combined to calculate the product's commercial score.

Examples of commercial properties include:

• number of clicks
• profit margin
• a trend index
• items in stock
• index representing when a product was added

It could even be a calculated index based on a weather forecast.

Context partitions

Context partitions control the sorting of products and consist of a specific weight multiplied by a parameter partition. Commercial parameters are combined with context partitions on a context to calculate the commercial boost of products. Following types of commercial parameters are supported:

• Rank is a simple index. Examples could be number of clicks or contribution margin.

• Cluster rank is similar to a normal rank normalization, but calculates rank based on a sample of values. Cluster rank normalization is significantly faster than normal rank normalization.

• Range is a range of values. An example is a range indication when a product was added to the shop (like 7 days with values 0-7, 14 days with values 8-15 and 21 days with values from 15-22).

• Term will look at the input value and match the input against the terms defined on partitions to find a fitting partition. An example is terms like "few in stock", "sold out" and "plenty in stock".

• Proxy is a custom value between 0 and 1 so it fits the normalized values generated for other parameters.

Each parameter type is used to determine how ingested values should be normalized.

Pinning

The concept of 'Pinning' serves as a tool for businesses aiming to spotlight specific products within search results. This functionality enables the ability to 'pin' selected products, identified by their unique IDs, to the uppermost part of a search result, giving them immediate visibility. The feature is not designed to replace well-balanced commercial parameters or search relevance boosts. Rather, it's a tool of precision, intended to be used sparingly and strategically to highlight select products in specific circumstances.

Pinned products come with distinct features that make them stand out. Firstly, the order of these pinned products remains fixed; the product placed first in the list will invariably appear first in the search results, followed by the rest in the order they were pinned. This allows businesses to strategically feature products based on factors like popularity, new arrivals, merchandising, or promotional campaigns.

Secondly, a pinned product will be displayed only if it's relevant to the search query. If a product doesn't match the user's search, it won't be unnecessarily thrust upon the user, ensuring that the search results remain pertinent and user-friendly.

Lastly, to maintain the integrity of the search experience, a pinned product appears only once. It doesn't reappear in the general list of sorted products, avoiding repetition and potential clutter in the search results.

The user interface (UI) is designed for the pinning process to be intuitive and user-friendly, simplifying the task of pinning products and rearranging their order as needed.

Contexts

Contexts play a crucial role in determining how product properties are weighted based on the user's context, such as whether they are searching or browsing and what specific category or brand, they are interested in. These contexts can be managed via the Admin UI or Admin API and consist of various elements such as name, conditions, partitions, relevance weights, and facets. Contexts are also related to commercial sorting, which allows for sorting configurations for different scenarios. Without contexts, all products would have the same boost, but with contexts, search requests with specific filters and conditions can sort by different criteria.

Context conditions are used to specify which context will be used in a search. The more context conditions that match data in the search request, the more relevant the context is. The most relevant context which has all context conditions satisfied is used.

The contexts conditions that can be used are:

• Attributes are a central part of distinguishing different SKUs from each other, but also different products from each other. Attributes can be a list of terms, a number, or an interval. Typical examples include color, height, and size.

• Split tests (also known as an A/B test) is a technique used to compare two versions of commercial boosting configuration contexts to determine which one performs better.

• Scopes defines what data should be contained in a search result, including fields to return, default number of results, number of SKUs per product, and domain filters, and is expected to be stable as changing it often directly affects the website's layout.

• Categories contain a list of paths to find categories the product is part of. A product can exist in multiple categories at a time.

• Whether or not the search request contains a phrase.

Ecommerce Search will choose the right context for a search request based on the API call.

Scopes

Scopes in Ecommerce Search define the type of interaction a client desires with the search engine by determining the data included in a search result. Configurable settings include fields to return, default number of results, number of SKUs per product, and scope domain filters. Scopes remain stable throughout a webshop's lifetime, as changes can directly impact the website's layout. They are related to segments, which define the selected webshop's segment, and contexts, which specify the current interaction's specifics.

Scope types, such as Details, Browse, Search, Quick search, Basket, Checkout, Order confirmation, and Bands (e.g., Recommendations and similar products), allow tracking user behavior and intent. Scopes are also part of the publication flow, requiring a published publication before any scope changes can take effect. They are closely connected to contexts, and by adding a context condition with the scope id, a context can be limited to a specific scope.

Pushing parameter values to ECS

For ECS to be able to sort and rank search results effectively, it needs to be able to normalize values associated with each SKU. This is done by pushing the relevant values through the API on a periodic basis. Depending on the specific value, it may need to be pushed several times a day or just once a week. Once ECS receives the new values, it recalculates the index used for sorting and ranking, allowing for more accurate and relevant search results to be returned to users. The frequency with which values need to be pushed can vary depending on the specific use case and the amount of change in the values over time. By regularly pushing values through the API, businesses can ensure that their search results remain up-to-date and relevant to their users.

Controlling the sorting and ranking

Using the UI, a search manager can control the sorting and ranking of search results for a particular context by assigning weights to various parameters. When assigning weight to a single parameter, the value itself is not as important. However, when adding weight to multiple parameters, the weights can have an influence on one another.

Through the UI, it is possible to compare the calculated index in different ways. This includes the ability to compare a couple of SKUs as well as compare different contexts.

Search relevance

To further control sorting a search relevance can be used to configure how important a phrase match is compared to other contributions. Often, a match in the name or short description of a product is more important than a match in long description. Also, a match on the product number is often the most important. A context is created with a set of recommended settings.

Product and User Affinities

Just as commercial values for each SKU can be uploaded to ECS, it's equally important for ECS to understand both product and user affinities. A product affinity might represent a product's vibrant color (e.g., pink or neon) or a numerical score reflecting its "organic" quality. In parallel, user affinities, such as those for "vibrant colors" and "organic products," allow ECS to produce tailored sorting. These user affinities are typically inferred from a range of data, including product purchases, site analytics, and newsletter preferences, among others.

ECS generates this personalization based on a user ID provided within the search request. The degree of personalization, or the "personalization weight," is adjustable by the search manager via the UI. Additionally, this interface allows for the testing of sort orders by simulating specific user interactions.

Context facets

Context facets are used to set up which facets should be returned in a search response. Each context can have one facet for each allowed filter. When selecting a context facet, the match type is specified, and the order of the facets when editing a context reflects the order of facets in search responses.

The shop solution will use facets to make filtering available for the customer.

To summarize

This article discusses the importance of effective Ecommerce Search and how businesses can optimize their search results using Ecommerce Search. The platform provides tools for managing dictionaries of hypernyms, synonyms, and other relevant terms, as well as for controlling the sorting and ranking of search results based on commercial properties such as popularity, relevance, and profitability.

The core of Ecommerce Search lies in the text analysis of product data and search queries. Suggestions in Ecommerce Search are phrases suggested to the user while typing in the search box on a web shop, to guide the user to choose a phrase that gives the most relevant results.

Additionally, Ecommerce Search allows businesses to sort product results based on commercial properties, and the platform provides a variety of commercial parameters to help determine a product's commercial score. Context partitions control the sorting of products, with different weights assigned to different commercial properties based on the user's context.

Overall, Ecommerce Search provides businesses with powerful tools to optimize their search engine performance.