Introduction to Custom Integrations
Elements provides out-of-the-box integrations with many well-known external systems such as the Web of Science, PubMed, and various institutional repository platforms including DSpace, EPrints and Hyrax. Enabling these integrations is often a matter of properly configuring the connection parameters within the Elements web application, and then switching the integration on.
Where the ability to integrate with a system is not provided for within Elements in the form of a ready-to-use connector, customers can create their own rich custom integrations using external scripts or programs separate from Elements that interface with the Elements API or the Elements reporting database.
Common examples of custom integrations include those which might regularly:
Export Elements data into a public-facing institutional web profile system
Import grants data into Elements from a separate institutional grants management system
Export Elements data into an institutional data warehouse
This article discusses the typical lifecycle of such integrations, from development and testing, through to deployment, monitoring and regular review. Maintenance of the health of a custom integration through an Elements upgrade and considerations for how to maintain your custom integrations when and if your institution chooses to migrate from a self-hosted Elements instance to a cloud-hosted instance are also discussed.
Development and testing
Custom integrations are implemented by developing, and then typically regularly executing, your own custom scripts or programs (simply referred to as scripts for the remainder of this article) to interface with the Elements API (which supports both data import and data export) or with the Elements reporting database (which supports data export only). Please see the following rich documentation on those two systems integration interfaces:
The API
The Reporting Database
Both integration interfaces support secure authenticated connections so that only your scripts may connect to and use them. You will typically wish to develop and test your scripts against a test instance of Elements rather than directly against your production instance, to avoid introducing unexpected problems to your production environment before you know your scripts are ready for production use.
You can code your own scripts, or you might engage Symplectic or a third party to help you write your scripts.
Ownership and general responsibilities
Irrespective of who authors and/or executes your custom integration scripts, they are owned by you. It is your institution's responsibility to develop, test, and maintain them, store master copies of them for backup purposes, execute them, and monitor them for correct function.
In practice of course, you might commission Symplectic, or a third party to help you develop, test and perhaps even host and run your custom integration scripts for you. However, custom integration scripts do not form a part of Elements, and the responsibility to ensure their proper maintenance and execution ultimately remains yours.
Executing your integration scripts
It is common to want to run your scripts to a regular schedule, such as nightly. You can run your scripts from any of your institution's own machines - they do not need to be run from the same server on which Elements is installed (indeed, for a Symplectic-hosted instance of Elements you do not have access to the Elements server to do this).
You can alternatively approach Symplectic, or a third party, to host and execute your scripts for you, to your desired schedule.
Altering custom integrations to work with newer versions of Elements
Whoever executes your scripts, it is your responsibility to make any alterations to the code in your scripts that may become necessary as a result of changes in Elements itself, whether those changes are associated with version upgrades or alterations to system configuration that you make. For example, you should ensure that testing the ongoing viability of your custom integrations is a standard part of your acceptance testing processes ahead of upgrading an instance of Elements to a newer version.
Both the API and the reporting database provide strategies for helping you in maintaining your integrations when upgrading to newer versions of Elements. The API provides backwards-compatible endpoints for up to two years after their obsolescence, and where the occasional breaking change is made to the reporting database, the version of Elements which introduces it is released with clear step-by-step instructions for suitably migrating your reporting database SQL queries to work with the newer version of the reporting database. Please see the documentation above on each integration interface for more information.
Migrating your instance of Elements to be hosted by Symplectic
You may at some point consider migrating a self-hosted instance of Elements to be hosted by Symplectic (or a third party), and wish to know how this might affect any custom integrations you have developed and are running.
In short, the hosting of Elements is not related to the development, ongoing maintenance or hosting of your custom integration scripts, since they are not a part of Elements.
You have a number of choices regarding each custom integration:
Continue with your current arrangements for their execution. Whether you are executing your scripts locally, or have engaged Symplectic, or a third party to execute them, you can continue with this approach. You may of course need to make small tweaks to ensure suitable network connectivity is maintained to the migrated instance of Elements. But after this, your custom integrations will simply continue to function in their current setup, whoever is currently executing them.
Ask Symplectic, or a third party, to execute them for you.
In all cases, your scripts will remain yours, and all responsibilities outlined in the previous section also remain yours.
Unless specifically agreed otherwise, any quotes issued by Symplectic relating to the hosting of Elements exclude provision for the hosting of your custom integrations.
FAQs
Can I ask Symplectic to develop my custom integration scripts?
Yes. Symplectic will happily consider quoting for services to develop and test a custom integration script. At the end of this kind of service provision, source code (and any compiled binaries, where relevant) is handed over to your institution, along with with all responsibilities to maintain them going forwards. Alternatively, you can engage a third party, or develop and test your custom integrations entirely yourselves.
You can approach Symplectic again at any time thereafter for further rounds of development should you require any alterations to be made (e.g. in order to keep your scripts in working order against a newer version of Elements), or you can make the required changes yourselves.
Whichever party executes your scripts, Symplectic is not currently able to commit to any standing arrangements to automatically maintain your custom script code in working order against newer versions of Elements. The process of ensuring the working order of your scripts through changes to Elements must be anticipated and initiated by your institution.
Can I ask Symplectic to host and run my custom integration scripts?
Yes. Symplectic will happily consider quoting for services to host and run your scripts either as a one-off execution or to a regular schedule. Alternatively, you can engage a third party, or host and run your scripts yourselves, no matter were your Elements instance is hosted.
As would be the case if you hosted and ran your scripts yourselves, any hosting and running of your custom scripts by Symplectic is performed entirely separately to the hosting and running of instances of Elements. Because of this, it is not possible to arrange for the configuration of execution of your custom integration scripts by Symplectic as a part of the process of installing or upgrading of Elements, though you can ask Symplectic to schedule the regular running of your scripts against a nominated instance of Elements. Similarly, if you ask Symplectic to perform significant alterations to the hosting of Elements (e.g. by commissioning an additional hosted test instance of Elements), you should consider whether a quote for corresponding changes to the running of your hosted scripts should also be requested (e.g. by hosting and running copies of your customisation scripts against the new test instance of Elements).
