Search is considered one of the primary functions of websites used to retrieve content quickly and appropriately. Search Space has multiple paid or open source frameworks. Many of these frameworks support different approaches to integrating search into your website.
Get a better understanding of the steps taken to integrate the search framework with AEM.
1) Select search tool
Various open source or commercial search frameworks are available. The most used open source search engines are Solr and Elasticsearch. The list of paid categories includes leaders such as Coveo Relevance Platform, Lucidworks Fusion, IBM Watson Discovery, and Sinequa ES. The key factors for choosing a search engine mainly depend on:
- Open source vs Paid
- Tool flexibility
- Integration features such as extensive API support
- Learning curve and development team capabilities
- horizontal and vertical scaling
- Clustering with the cloud
- Maturity and community support
- Rich full-text query syntax
- AI/ML capabilities
- performance and manageability
2) Configuring search engine and indexing options
Once you’ve identified your tools, the next step is to plan for additional features to fully enjoy the benefits they offer. Before setting up your system, you should discuss the following factors with your stakeholders and make appropriate decisions.
- Hosting: This depends on the search tool’s hosting capabilities (on-premises vs. cloud).
- Indexing: Indexing of data based on requirements.Almost all frameworks support customized indexing of content
- Index frequency: How often to index items (daily, monthly, etc.)
3) Configuration
Some basic configuration is required for content indexing to work smoothly. Below are the configurations you can’t miss.
- metadata: Metadata is used for faceting, sorting, ranking, and relevance
- Breadcrumbs and pagination: for better page navigation
- Sorting and faceting: Faceting and sorting improve search result refinement. Recent searches, Suggestions (Did you mean), Autocomplete – these are the basic search aids provided by the framework.
- AI/ML configuration: Modern search tools take advantage of user behavior and patterns to deliver better results based on artificial intelligence (AI) and machine learning (ML) capabilities
4) fine-tuning the results
As a final step, once your search results are stable, you should analyze the results further to fine-tune your search. “Search Quality” – How we improve search results, such as by fine-tuning relevancy, applying analytics and artificial intelligence features. Some of the options provided by the search framework are listed below.
- Promotions & Banners: Based on your configuration, you can drive your results
- Dictionary and synonyms: Dictionaries help configure how terms are stored in the suggestion dictionary. A synonym is a user-constructed word or phrase that means exactly (or nearly) the same as another word or phrase.
- redirect: Directly redirect users to specific pages based on criteria or search terms
Our AEM Framework Search Feature
Bounteous has developed the Activate framework, which provides digital marketing services for the Adobe Experience Cloud. This framework minimizes the cost and time it takes to launch digital customer experiences using Adobe Experience Manager, Adobe Analytics, and Adobe Target. As a native AEM option, Activate has a search feature that leverages AEM’s built-in lucene search.
Decisions about search integration with AEM Framework
Separating search from AEM’s boundaries provides several benefits. For example, on a site with a large number of content pages, searches can degrade her overall AEM performance. Third-party search also offers many additional built-in features such as faceting, sorting options, pagination, auto-suggestions, and AI/ML capabilities.
We decided to integrate AEM with Coveo as our first entry into Activate’s non-native search capabilities. The deciding factor was non-open source, cloud-hosted search with advanced indexing and result fine-tuning capabilities.
Why we chose Coveo
Coveo is a software-as-a-service (SaaS) search engine that provides intelligent search, delivering the most relevant results from potentially large amounts of data. Coveo is also one of the consistently notable leaders in Gartner’s Magic He quadrant over the past several years.
Our requirements were for flexible indexing methods, including REST API support, cloud-hosted solutions with AI/ML capabilities.
The following sections describe high-level concepts of Coveo search integration with AEM CMS.
AEM and Coveo integration
There are multiple ways to integrate AEM with Coveo. Commonly used approaches are the AEM connector provided by Coveo and generic connectors like Sitemap.
Common methods of content indexing within Coveo
Coveo Search supports various indexing options to facilitate smooth indexing of content based on different technical requirements.
Crawler-based indexing
Coveo provides modules to scan and index content and extract that content and index.
Sitemap-based indexing
The sitemap connector helps you index web pages from your sitemap. The connector also supports authentication methods for secure websites. Supported formats are XML, Text, RSS 2.0, and Atom 1.0.
Push API
Suppose you want to customize indexing so that only defined data is indexed. In this case, the Push API gives you more flexibility and control over indexing.
The Push API exposes a set of services that allow you to push items and their permission models to sources and security identities to security identity providers. However, Security IDs have certain limitations regarding request size, request frequency, and push mechanism.
Indexing Content from AEM to Coveo – AEM Connector
Since we are working on AEM-based indexing, we decided to use the AEM connector for content indexing.
AEM Connector is a tool that facilitates the indexing of content from AEM to Coveo. The tool is provided as a package that can be installed from AEM’s package manager. Once installed, you need to configure a series of workflows to enable various indexing within AEM.
Installation and configuration details can be found at here
With minimal configuration, Adobe Experience Manager page-based activities such as publishing, unpublishing, and deleting pages automatically trigger indexing within Coveo.
Coveo to AEM Authentication and Content Retrieval
Commonly used authentication methods are API key authentication (public and private) and search token authentication. Each configured API can restrict or enable access to any section of the resource. This allows you to fine-tune what operations are allowed within Coveo.
JWT-based authentication is recommended for AEM integration as it ensures secure searches.
Coveo supports various search APIs to interact between systems hosted on your website and retrieve content through predefined authentication mechanisms. These APIs support content retrieval by passing several sets of query parameters and providing a wide variety of results including faceting, sorting, auto-suggestion, pagination, and more.
You can find the full swagger search API list here.
Coveo Results
Choosing Coveo as our search framework did not disappoint. As outlined in the previous section, Coveo is much more flexible than some open source frameworks when it comes to grouping results, analytics capabilities, AI/ML capabilities, and reporting dashboards. Coveo definitely has a learning curve, but as you explore and understand the tool better, you’ll get more features and your search could be a lot faster going forward.
