Category: DCC News

DMP inspiration down under

 
I’ve had a number of inspiring DMP discussions over the last few weeks, both Australian and European based. Here in Oz, funders do not require data management plans like they do in the UK and USA. This has led to the growth of quite different tools as institutions fit the DMP to local priorities.
 
CSIRO, QCIF and the University of Queensland all have data management tools with a strikingly similar feature set. Research Data Planner, RedBox and UQ Research Data Manager are more akin to data management systems than DMP tools. They integrate with other institutional systems and prioritise storage allocation, metadata capture and data publishing as incentives to engage researchers. It’s heartening to see that they have learned lessons from overseas – much attention has been given to streamlining questions and providing tailored guidance or pre-filled answers. This point came up at Macquarrie University too which is currently developing a DMP tool and will provide default answers that should suit most use cases. They are focusing on sensitive data as that’s the biggest risk and institutional concern. Indeed, institutions here seem very risk adverse and defensive of IP.
 
This prevailing institutional competitiveness is a weakeness for the data management field in my opinion. Three teams have developed very similar DMP tools while the sector as a whole would have been much better served by a coordinated national effort. Admittedly this is easier to say than do. Parallel DMPonline and DMPTool developments ran in the UK and USA for nearly 6 years before we started the DMPRoadmap partnership to have a common open source codebase from which to run each of our services.
 
Australian DMP tools are very impressive and there are a lot of ideas I plan to take back to inform DMPonline developments. I really like the API plug and play approach to allow organisations to join up whatever systems they have in place. I hope to coordinate a co-located workshop during the RDA Plenary on 18-20 March 2020 in Melbourne to discuss global DMP initiatives and what opportunities there are for wider collaboration. These could be around the common standard for DMP, sharing user requirements, code, developer peer exchange, training or more. If you are involved in DMP work and want to get involved please reach out to me.
 
There have been interesting European DMP talks over the last few weeks too. Benjamin Faure and colleagues at DMP OPIDoR in France have made a number of useful extensions to the DMPRoadmap codebase. These include one click plan creation from the public templates page, an API extension to pull out themes, and adding a dataset component to the underlying data model. We have also continued our DMPonline outreach, running drop-ins and scheduling the next user group for 17th September in London. This will follow a full day RDMF on costing data management on 16th at the British Library – register here. We are also growing the DMPonline team and held interviews for a new developer on Monday.
 
I’ll be giving a DMP webinar for ARDC on lessons from Europe tomorrow. Slides are available and a video is forthcoming. 
 
 

All the fun of the FAIR: FAIR WG inaugural meeting held

Re-posted from the EOSCSecretariat website
 
Thursday, 4 July was the inaugural meeting of the EOSC FAIR Working Group. We have 26 members representing 20 different Member States and Associated Countries and a remit which covers FAIR practice, a …

DMPonline subscription model – revisited

Last November we launched a subscription model for DMPonline. Thank to all who have signed up in the first 6 months. We currently have over 40 institutional subscribers in the UK, Netherlands and Sweden. Their stories are being featured in the Knowled…

DMPonline release notes: April 2019

DMPonline release notes: April 2019
We do appreciate your feedback, as it is very valuable for us to know what works well for you, or anything else you wish to be fixed, improved or deployed! If you are interested to see what we are currently working o…

Sam Rust explains benefits of API for DMPonline

Sam Rust – explains benefits of DMPonline API

 

DMPonline has a REST API to enable you to query some statistics about your organisation, download your guidance, check the templates in the tool, and even create plans for your users.  This blog post will detail some clients to use APIs in general, the data you can pull from our api, and run through a more detailed example.  We will also talk about some of the planned extensions to the API to enable you to run your own queries on your organisation’s data.

What is an API?

An API, or “Application Programming Interface”, is simply a defined format for multiple systems to communicate over.  It defines the language for information interchange, both that of the requests from the client, and that of the response from the server.  A REST, or “Representational State Transfer”, API means that the API responds to a predefined set of stateless operations.

How do I use an API?

A variety of clients can be used to communicate with an API.  At a basic level, any system which can make a HTTP request can use an API.  Commonly, this would be any programming language, for example Python and the requests package it uses to make web requests.  For a more user-friendly approach, A GUI application like Postman can be used.  For the purposes of this tutorial, we will use Postman to request data from the API, so please install that now if you want to follow-along.

Who can use the DMPonline API?

Currently, the DMPonline API usage is available to all org-admins within the tool.  To check if you have permission to use the API, go to your ‘edit profile’ page within DMPonline by clicking your name in the top-right corner.  Scrolling down, under your Institutional Credentials, if you have API access, you will see two sections `API token` and `API Information`.  If you are an org-admin and do not see these sections, please get in touch and we can extend this access to your account.

What can the DMPonline API do?

The DMPonline API has a variety of endpoints, all of which are detailed in full on our wiki.

In this guide, we will focus on the statistics endpoints, but the others are listed at the above link.  Currently, API users on DMPonline can pull out the number of users which have joined their organisation; the number of plans by any user based off of your organisation’s non-customized templates; the number of plans created by your organisation’s users grouped by template; and some generic metadata about all of the plans created by your organisation’s users.  Additionally, all of these endpoints can be passed a start and end date to scope the results to those created within the specified time-period.  All DMPonline endpoints currently respond only in JSON, but if there is a demand for other outputs, these could be developed.  If you have need for a different format, please let us know.

An example API use case

Here, we will walk through the process of querying the users-joined so far in 2019 for our organisation.  In this example, we will detail the specifics for using Postman, but the steps are transferable to the application or language of your choice.  Assuming you’ve installed Postman, create a new Request object using the orange ‘New’ button in the top-left.

Screenshot 1: New Request Button

To do so, we’ll want to give the request a name and an optional description. 

Since we are querying the users who have joined our system, I’ll call the request ‘Users Joined’. Here we must also choose to create a collection (essentially a folder) to keep our request grouped with like requests.

Screenshot 2: New Request Dialogue

After creating the request, we next need to add a request URL.  We want to use our base URL for DMPonline, ‘https://dmponline.dcc.ac.uk/’, along with the URL-partial given in the api wiki page in the statistics section, for the command users_joined, ‘/api/v0/statistics/users_joined’. 

Screenshot 3: Request URL

Now that we have told Postman the request we want to make, we next need to turn towards authentication and authorization.  The wiki section on shared headers details the convention used here.  We do Authorization via the ‘API token’ which you can find, as described above, on your ‘edit profile’ page.  The second header, `Content-Type`, lets the API know that your want a JSON response.  To put these into Postman, we will need to go to the ‘Headers’ tab, immediately below where we input the request URL.

These headers will be key/value pairs.  The first key “Authorization” will have a value of “Token token=<API_TOKEN>”, where <API_TOKEN> is the token from your edit profile page, as described above.  This token is used to both authenticate us as the user, and to authorize us to view this API endpoint.  The latter key, “Content-Type” should have a value of “application/json”.  No matter which API endpoint we use, we will want to keep these two headers the same.

If we now hit send, the above is sufficient to query the API for the number of users in your organisation, and you should get a response in JSON.  Congratulations, you’ve just made your first successful API request.  

Don’t get too happy just yet, because we said that for this example, we are only interested in users joining throughout 2019, so we need to scope this query to only include users created between those dates.

 

The API Wiki Section on Statistics details how this is done.  Essentially, we are allowed to specify two additional arguments as a query string, ‘start_date’ and ‘end_date’, and specify these values as YYYY-MM-DD.  In Postman, we will need to navigate to the ‘Params’ tab where we had previously selected ‘Headers’.  Here we will add a key, ‘start_date’, and a value, ‘2019-01-01’.  We do not need to specify an end_date, as 2019 has not yet ended so an open-ended query is sufficient for our purpose.

Re-running the query, we now see that a smaller total number of users has joined throughout 2019 rather than over the whole year.

Planned Extensions

We have a planned extensions to the API which should be released in the near future.  We are going to be creating an export of the full-text of all your organisation’s plans in the system as a (large) JSON object.  This should enable organisations to run their own, more interesting, analyses on their data, whether statistical, semantic analysis, or otherwise.  It is also much more flexible than us defining the set of questions which you are allowed to ask about your data.

To keep up with DMPonline news, you can subscribe to the RSS feed to receive our blogs and tweets, and watch GitHub for code updates. You are also welcome to discuss our new features on the user group mailing list.

DMPonline user group: where next?

We’ve run a couple of user group sessions over the last few weeks. Many thanks to those who attended in Amsterdam and Manchester. It was particularly helpful to hear how you are using the tool and what changes you would like. The DCC team met on Monday to process your feedback and have set the following priorities:

  1. Full text API (#2086)
    You wanted an extension of the plans API to allow authorised admin users to pull out the full text of DMPs for their organisation. Sam has already developed this but it takes a long time to run in real-time so we are going to harvest the data overnight. This requires some changes to our infrastructure but we should be able to release the API later this month.
  2. Reviewer admin permission (#2087)
    It seems that many of you have multiple reviewers of plans and would like this to be a separate role that can be assigned by the org admin. We’re in the process of adding this, and will adjust our contract so you’re not paying more for multiple plan reviewers.
  3. Adding a field for School / Division / Department (#2088)
    We plan to add one field to the edit profile page for which you can name and define a controlled value list. This will allow you to identify the sub-unit affiliation of your users. In time we could extend the functionality to allow you to pull stats on this (e.g. plans by school) and to allow customisations to be done by sub-unit level. Currently it is only the guidance that works in their vein.
  4. Conditional questions (#1772)
    This is a bigger piece of feature development, which we have scheduled for the summer months. It was top priority for you so we will add it first.
  5. Custom section on funder templates (#2072)
    Many of you have the same custom section on funder templates and want to create it once and apply across all / a selection. This will be provided as a new feature in Summer 2019.
  6. Plan versioning 
    Again, this is a bigger piece of feature development, which we have scheduled for the summer months.

There were a number of smaller items raised during discussion too. These have been added as tickets (see the ‘user group’) label and will be addressed during the coming few sprints.

Other areas of interest which represent larger feature development and are to be placed into development plans have been captured on the ‘Future enhancements’ section of the wiki for now.
 
Manchester’s case study on using DMPonline was really inspiring and we wanted to ask how we can support others to do some similar things. We realise not everyone has development effort in-house so we wondered whether it would help if we wrote a script to send you weekly/monthly email notifications of new users and plans so you can monitor usage and start to interact with and assist users more? Let us know what you think.
 
We are due to push out a new release immediately after Easter. Amongst other things, this includes:
  • bug fixes for guidance / comment display when text runs outside the box
  • adjustments to comment notification so users don’t receive multiple emails when you provide feedback
  • display of themed guidance in ‘customise template’ preview
  • table-styling fixes
  • plans by template statistics

You can see these on the test site now and in live deployment soon.

All feedback welcome!
 
Magdalena, Diana, Sam & Sarah
 

More new jobs at the DCC – closing date 10th May

New projects in which we’re advancing the FAIR agenda and continued growth of our consultancy work mean that we are recruiting again at the DCC. If that’s enough to pique your interest, follow the link now and ensure that your application reaches us by the closing date of 10th May 2019. If not, read on and let me try to persuade you why this could be the opportunity for you.

We’re recruiting at least two new staff to join our distributed team at the DCC. The successful applicants will spend much of their time on one or other of the FAIRs FAIR or RDA Europe 4 projects, but will also have the opportunity to engage with other areas of DCC work such as training and consultancy. You will already have some knowledge of issues around data reuse, which you may have acquired as someone who generates data, who uses it or who supports those who do either of these things, in either the public or private sector. We’re not looking for experts – just for people who have some relevant knowledge and have the capacity to expand that through working with our friendly and knowledgeable team. There is a significant amount of travel associated with most of our work and you will have the opportunity to take advantage of this, along with the opportunity to work with leading organisations around the world.

Depending on your skills and interests you’ll spend time on other areas of DCC activity as well. In recent months, these have included training  for universities and researchers in the UK, Japan, Malaysia, Norway and South Africa, and work for funders such as the ERC and Wellcome Trust and for UK national research facilities. We run services such as DMPonline for customers around the world, and there’s also our annual conference, IDCC, and much more. We offer flexible working practices and a supportive working environment with a diverse and friendly team in Edinburgh and Glasgow. We encourage applications from all communities.

You’ll need to use the University of Edinburgh’s online recruitment system to apply. This requires you to register; please don’t leave it until the last minute to enter your application. (You can revise an application that’s already been submitted as long as you do so before the closing date.) If you would like to discuss these posts informally, please contact me (Kevin Ashley – director@dcc.ac.uk), Sarah Jones (sarah.jones@glasgow.ac.uk) or the DCC helpdesk (info@dcc.ac.uk) Applications will close at 1700 UTC on May 10th. For the right candidates, this is an ideal opportunity to broaden your knowledge and experience. That candidate is very possibly you – we look forward to hearing from you.

 

DMPonline – information about phasing out for non subscribers and comparison of subscription models

Dear DMPonline Administrators,
In October 2018, we announced that DMPonline is going to charge its administrative users. We have now formalised the phasing out DMPonline for non-subscribers, which will begin in April 2019. For organisations that d…