Administering Elasticsearch
This article is only relevant to clients who host Elements on premises.
Introduction
Some time ago, we announced a significant upcoming change in the set of infrastructure prerequisites for running Elements: namely that "new versions of Elements will require the use of a suitably-configured client-hosted Elasticsearch instance."
Although the project was not started until more recently, this change will still be delivered. This support article aims to provide more information on the initiative.
Why will Elements require Elasticsearch?
As Elements evolves, the use of older third-party technologies has an increasing impact on our speed of delivery, quality of support, and ability to deliver new features. So that we can continue to deliver a great product and excellent support, it is necessary to migrate to newer technologies from time to time.
In this case, the older technology is Lucene. Lucene is used by Elements to index and search for data. Such indexing drives many of the page filters and other product behaviours. Its selected replacement technology is Elasticsearch.
The migration to Elasticsearch will allow us to unblock work on several popular search-related feature requests. Elasticsearch also offers many other crucial improvements over Lucene, including support for better solution scalability and availability, more flexible application deployment architectures, improved information security, and many others.
Does my organisation have to supply an Elasticsearch server?
Yes, if you host Elements yourself.
Lucene indexes are stored on the same disks Elements applications are installed to, and are managed by Elements using third-party Lucene code libraries deployed with Elements. This means that thus far there have been no special requirements of clients to manage any infrastructure associated with search functionality.
With Elasticsearch, all search indexes will be migrated into a networked database server. There is a useful direct analogy with Microsoft SQL Server, which you are already administering as the database server that hosts the Elements operational and reporting databases: Elasticsearch is the new type of database server required to host Elements search indexes. Going forward, you will need to install and manage Elasticsearch as a part of the process of hosting Elements on premises.
If your Elements instances are hosted by Symplectic, we will handle all of this for you.
Will Elasticsearch be optional?
No. Starting from a certain future release of Elements, a suitably configured Elasticsearch server will be a mandatory prerequisite for running Elements. Please see the "What is the planned timeline?" section below for more information on which future version will be the first to require Elasticsearch.
Versions of Elements released before the first version to require Elasticsearch will continue without any dependency on Elasticsearch, even when patched.
We host Elements on premises, so what does this mean for my IT team?
Your IT team will already have a team member serving in the role of "DBA" ("database administrator"), charged with maintaining deployments of Microsoft SQL Server associated with your instance(s) of Symplectic Elements.
Such a team member may or may not be fully dedicated to DBA tasks, but they will have been made responsible by your organisation for administering your deployments of SQL Server. Their tasks may include purchasing SQL Server licences from Microsoft, installing SQL Server, securing it using industry best practices, patching it, upgrading it as required, determining and implementing database backup schedules, troubleshooting performance or other issues in their interactions with physical or virtual hosting environments and/or networks as well as with connecting software such as Elements, ensuring SQL Server is configured to the requirements published in the Elements - Installation Prerequisites, and other miscellaneous maintenance tasks.
Continuing the analogy with SQL Server: to run future versions of Elements requiring Elasticsearch, your IT team will also need to ensure someone is serving in the role of "Elasticsearch administrator", charged with all of the same responsibilities for the maintenance of your Elasticsearch deployment(s).
What are Symplectic's technical requirements for Elasticsearch?
As with SQL Server, we will maintain clear specifications of all requirements relating to how Elasticsearch must be configured by your organisation for use with Elements, such as:
the versions of Elasticsearch compatible with any given version of Elements
the configuration of any accounts and permissions required by Elements applications when accessing Elastiscsearch
This will be done in the same way as we have maintained analogous requirements relating to SQL Server, namely:
In the Elements - Installation Prerequisites support article, which lists version and configuration requirements for SQL Server and other dependencies
In the Symplectic Elements End of Support Announcements support article, which provides advance notice of end of support for versions of SQL Server and other dependencies
We will likely initially require Elasticsearch 9.1, though a final confirmation will be posted in the above article(s). It is likely that our requirements will be very similar to those listed for SQL Server, and include:
Elasticsearch version requirements
Any Elasticsearch feature configuration requirements
Elasticsearch security configuration requirements - e.g. configuration of Basic Auth on the Elasticsearch API, and permissions and credentials for Elasticsearch accounts for use by Elements applications
There will be one major simplifying difference between the management of SQL Server databases and Elasticsearch indexes:
You will not be required by Elements to maintain backups of Elasticsearch indexes, though you are free to do so. Should the Elasticsearch indexes maintained by Elements be lost, Elements will automatically rebuild them from data in the operational SQL Server database. Of course, this takes time, which may influence your decisions on backup strategy.
What is the planned timeline?
As with all changes delivered on the Elements product roadmap, targeted timings are subject to change. At the time of last update (Q1 2026), the following timeline represents the road already travelled, and our current best estimates for the road ahead:
Q2 2023 - Q2 2024: Preparatory work was performed to alter the way indexing work is processed by Elements. This was necessary to simplify interactions with indexing technologies, lowering the complexity of a migration away from Lucene and providing several other important benefits, too.
Q1 2024: Further preparatory work was performed to reduce the amount of data indexed by Elements. This work was necessary to further simplify the upcoming migration away from Lucene to Elasticsearch, but also provides an increase in the speed of indexing. This improvement was released with v6.17 of Elements.
Q1 2024 - Q3 2025: Removal of all Lucene code and replacement with code making use of Elasticsearch. This work is complete - development and test Elasticsearch cluster environments have been built. User interactions have been refactored.
Q3 2024 - Q3 2026: The Symplectic Infrastructure team is developing robust hosting frameworks for managing production-ready Elasticsearch clusters. Technologies and approaches have been researched and selected, initial performance testing completed, and cost analyses performed. Kubernetes clusters are being used in AWS Autoscaling Groups for management automation and resilience reasons. Monitoring and logging infrastructure is also in development.
September 2026: Release of the first version of Elements requiring Elasticsearch
How much notice will I get of the first target release requiring Elasticsearch?
The target release for the first version of Elements to require Elasticsearch is the September release of 2026. Depending on progress, the actual release could be later, but will not be earlier. We will update our estimate here as development progresses.
Sep 2024: The target release quarter was updated to Q3 2025 as a result of our infrastructure-related research and analyses performed during Q3 2024.
May 2025: The target release quarter was updated to Q4 2025 as a result of velocity measurements during infrastructure cluster management framework development.
September 2025: The target release date was updated to Early April 2026 to account for the reprioritisation of several projects across Symplectic.
February 2026: The target release date was updated to September 2026 to account for some additional complexity discovered during development.
When will this document be updated?
We will update this document each quarter with the latest information, and more frequently than that if we have any significant updates to provide.
Will Symplectic help manage our Elasticsearch server(s)?
No. As with SQL Server, clients who opt for on-premises hosting are responsible for all aspects of the maintenance of Elements’ installation prerequisites, including all servers hosting components of Elements. This includes Elasticsearch server(s).
Symplectic offers a Cloud-hosted option for Elements that fully removes all requirements on clients for the maintenance of any server infrastructure.
Is Symplectic willing to host and maintain Elasticsearch while we continue to host the rest of Elements?
No. Clients who opt to host Elements themselves are responsible for all aspects of the hosting of Elements, including running Elasticsearch infrastructure.
Will Elements be compatible with multi-node Elasticsearch clusters?
Yes. Symplectic will use its own high-availability Elasticsearch clusters for Symplectic-hosted instances of Elements.
Which versions of Elasticsearch will be supported?
At the time of writing (10 Sep 2025), we are targeting support for Elasticsearch v9.1.
Can Elements and Elasticsearch be hosted on the same server?
Yes, as long as the server is sufficiently provisioned.
Will Symplectic continue to support clients who do not immediately upgrade to a version of Elements that requires an Elasticsearch server?
Yes. Please see the following support article for details of when each version of Elements reaches its end-of-support date:
