Repository Tools 2: Configuring Digital Commons

Digital Commons is an institutional repository developed and hosted by Bepress. You will find Digital Commons as one of the listed data sources.

Allowing Digital Commons to be used as a data source for Elements is a part of a longer vision for integrating institutional repositories as data sources in general.

Please see our Repository Tools 2 article for more information.

Support for Digital Commons is provided with the following advice and guidelines for its proper use.

Enabling this data source will allow your instance of Elements to automatically and regularly harvest bibliographic data from your institution's instance of Digital Commons into Elements.

Begin by disabling the Synchroniser service on the Scheduled Jobs page. This will help keep the ordering of events simple and prevents Elements from jumping the gun while you are altering settings.

Then enable Digital Commons by following instructions on the Manage Data Sources page:

Contact Bepress and ask for the OAI-PMH URL of the OAI-PMH interface of your Digital Commons instance. See this website for a public listing of registered OAI-PMH data providers for example base URLs. You might even find your own one there. You may need to contact bepress to enable the OAI-PMH interface associated with your instance of Digital Commons.

Elements now needs to be configured by you to assign Digital Commons items to the appropriate authors in Elements. Review the Default Data Sources (reached via the System Admin menu) for users of Elements, to make sure that your users have access to Digital Commons. You may also wish to review access on a per-group basis as indicated in the help text there.

Users now have access, in principle, to search Digital Commons data, but you might want to opt in whole groups of users to actually enable search against Digital Commons for them. To do this, visit the "bulk update search settings" link on the Manage Data Sources page:

On the page that leads you to, select Digital Commons, the groups of users for whom you wish to enable Digital Commons data searches, and choose to "Enable search for the selected users".

Please note that you cannot bulk update the entire group structure by selecting the top-most node. Each Primary Group must be selected individually.

Review the search settings of at least one relevant user, to confirm that their search settings now indicate that Digital Commons data should be searched for the user:

Please note that currently, unlike for other data sources, only system administrators are able to alter the state of the checkbox shown above. We would appreciate feedback on whether this restriction is a useful one for institutional repositories in particular.

Now you can restart the Synchroniser service on the Scheduled Jobs page.

Shortly after, you should then expect to see the publicly available content of your Digital Commons instance arriving in Elements as publication records, often alongside records from other data sources. Keep an eye on the Data Source Management page to see how many Digital Commons items have been harvested into Elements, and compare this with the number you know to be in Digital Commons:

You can review the activity logs on the data source management settings page or in the  "digital-commons notes YYYY-MM-DD.txt" file in your <elements-installation-directory>/Sync.Synchronise/Logs folder for more details on how harvesting has proceeded. For example, the following snippet shows that a Digital Commons item with ID ending in "1026" was imported as Elements publication 36753. You can use this information to directly substitute the Elements publication ID in the URL of an Elements browser session open at any existing publication, as a quick way to navigate to and view the harvested publication in your browser. 

Separately to the harvesting of Digital Commons records, Elements will then periodically analyse the imported records and assign them to the pending publication lists of users who it believes might have authored them. For this to happen for a given user, the user must have enabled Digital Commons in their search settings, as with any data source for which they wish to receive pending publications. 

Elements does its best to assign publications to the right users, but we have to tread a careful line between generating too many false positives, and missing correct assignments altogether, particularly when the only data to go on is a surname and initials.

You can see progress analysing Digital Commons content on the Digital Commons data source page:

In line with our Agile development approach, we are providing access to an early release of Digital Commons interoperability functionality. It is intended that the Digital Commons data source is enabled only for testing purposes, and not in a live environment.

We are very keen to receive feedback from interested customers who are willing to enable this functionality in a test instance of Elements about how we can improve the interoperability between Elements and Digital Commons. The Testing Digital Commons as a Data Source community thread would be a great place to provide that feedback. Please note that Elements will not alter any of your data in Digital Commons as a result of enabling Digital Commons as a data source for Elements.

Elements will not have had a chance to assign Digital Commons content to users of the system until it has had time to analyse it, so please be patient when reviewing analysis progress. There is currently no way for you to prompt re-analysis of data via the Elements user interface (for example, after altering users' search settings or group access to data sources), but for those who wish to help test this functionality, we can provide a way to work around this.

• It can take a long time to cycle through all of the publications in Elements, analysing them, and generating pending links to the users. We are aware that this is a slow and mostly invisible process at the moment. The timing of analysis of Digital Commons records and their assignment into relevant users' pending publications lists is not currently very controllable, and is only minimally observable via the "Data analysis" summary above. This is partly because the mechanism used by Elements to do this (looping through previously imported Digital Commons publications and executing an internal search against the users of Elements for each in turn) is not the same as the traditional synchronisation queue mechanism used for most other data sources (looping through the users of Elements and executing an online search at the data source for each in turn). This means that it may take some time for relevant Digital Commons publications to appear in users' pending publications lists. Troubleshooting the assignments of Digital Commons content to the correct users' pending publication lists will be a little tricky until we are able to add more support for it.

• For technical reasons related to the implementation of OAI-PMH, not all fields from Digital Commons are currently crosswalked into suitable fields in Elements (including DOI), and neither are the crosswalks currently configurable by the customer. In particular, this means that custom fields configured in Digital Commons are not currently imported into Elements. We are very keen to work with partner institutions to make improvements in this area.

• Because OAI-PMH is a protocol for exposing publicly accessible data, only publicly accessible records in Digital Commons can be imported into Elements.

• We coded and tested against the CU Boulder instance of Digital Commons when developing this functionality. We expect that there should be a good experience when Elements is configured to interact with other instances of Digital Commons, but we need your feedback!

The intended and proper usage of the Digital Commons data source is for you to connect Elements to your own institution's dedicated Digital Commons institutional repository. Although Elements cannot prevent you from connecting to an instance of Digital Commons that does not belong to your institution, this is not a supported use of the functionality.

The code behind the harvesting of data is optimised under the assumption that the harvested data are all outputs of your institution. Amongst other things, this enables us to infer more about the likelihood that any given repository item was authored by a member of your staff with a matching name, something that is critical to not overpopulating users' pending publication lists with false positives, and also not bloating and slowing down the Elements database with irrelevant publications.

If you are interested in connecting to an instance of Digital Commons that contains shared content with other institutions, please discuss this with Symplectic to ascertain whether your particular usage is supported.

Subject to the availability of partner institutions to work with, and the capabilities of the Digital Commons API, we hope to continue to improve the Digital Commons data harvesting functionality first presented in v5.1 of Elements. In no particular order, we hope to:

• Improve the provided crosswalks

• Allow customisation of the crosswalks

• Use Digital Commons data to drive the Elements Open Access Monitor

In early 2017 Symplectic carried out a feasibility study into depositing to Digital Commons in real-time via Elements; unfortunately this is not possible with the current version of Digital Commons.

Please either submit support tickets to the support site; or post to the Testing Digital Commons as a Data Source community thread.

• Having gained permission and had time to test against only one instance of Digital Commons, we are particular interested in feedback letting us know where improvements can be made when harvesting data at other sites.

• We would like feedback to help improve this guide

• Please periodically review, and send back useful contents of the "digital-commons warnings.txt" files, found in the Elements installation directory in the "Sync.Sychronise\Logs" folder. We expect this file to contain lots of warnings if your instance of Digital Commons is using fields in a way that we haven't seen before. These warnings can help us improve the way data is harvested in future releases.

• Please let us know how effective the assignment of Digital Commons publications to the correct users of Elements is, focusing on providing us with rich enough information that we can reproduce any unexpected behaviour that you are seeing. This information should include a screenshot of the publication metadata and a screenshot of the user's search settings. We are particularly interested in any obvious failures to assign Digital Commons content to the users who have authored them, and in any obviously incorrect assignments of Digital Commons publications to the pending lists of users who have not authored them. It is generally not possible be 100% accurate when implementing matching algorithms that take a best guess at which users correspond to which names on a bibliographic record, but it's always possible to do better, and we're happy to discuss approaches.