As the National Institutes of Health (NIH) new Policy for Data Management and Sharing goes into effect, NIH-funded researchers may be wondering how to ensure they comply with the new requirements. Dryad’s curated data publishing service provides …
Category: data management plan
Different hosting options – which way forward?
The Meeting of the Waters where the Amazon river starts. The darker Rio Negro waters and the sandy Solimões take 6km to fully merge due to different temperatures & pH levels.
Over the past few months we’ve been reconsidering DMPonline hosting – should we move to Amazon Web Services or remain with the University of Edinburgh. Brexit and ensuring we can meet our Service Level Agreements were two major concerns in this decision-making process. After investigating options, we have decided to remain with University of Edinburgh hosting. This blog post outlines our thoughts.
Keeping data within Europe
With all the uncertainties surrounding Brexit and a likely no-deal crash out, Amazon hosting was part of our contingency planning. Ironically it was after a week-long conference on the Amazon river that I sat in Sao Paolo airport and debated the issues with Edinburgh Legal. It transpired that hosting on Amazon Web Services wouldn’t solve any concerns. In contrast, the DCC would be considered as an external processor and any work we do via remote access would be deemed a data transfer. In the case of a no-deal-Brexit, the University has some model clauses which we will sign with overseas clients. These uphold us to European regulations such as GDPR to ensure the same protections are granted.
Controlling hosting
Another concern we had was ensuring the DCC team has full control over deployments. Over the past few years we have been contracting out technical support to EDINA, but with the growth of the developer team, we’re moving everything in-house. If the servers go down or any technical issues occur, we want to be able to liaise on fixes directly. Since we are containerising the application, it gives us more flexibility on deployment strategy. We’ve been investigating two main hosting routes – using Edinburgh University Information Services infrastructure and
Amazon Web Services
Investigating Edinburgh infrastructure and AWS
Ray and Sam met with the Edinburgh infrastructure team to understand what options are available for local hosting. There are several routes, varying the level of central and local control. We have opted for a centrally managed virtual machine to ensure all security updates are managed by the University and we can focus on maintaining the application. The University also has a forthcoming Docker Container service which may prove useful once out of test, as we use a dockerised setup.
As part of our planning process, we also took time to deploy a basic instance of the application to AWS. This helped us understand the technical options and anticipate workloads. AWS provides a large number of services which can be put together in a variety of ways. The options are changing rapidly, which adds to the complexity. We may end up needing to commit significant developer resource to continually monitor and maintain the deployment. Customers also raised several concerns about the implications of a move to AWS in terms of data access and permissions. Both the technical deployment and the legalities seem a bit of a rabbit warren which we’d rather avoid.
Permissions for AWS
Thanks to all the subscribers who gave permission for us to host on Amazon Web Services. The contract conditions required that we obtain explicit consent, hence initiating that process so we could keep all options open. While we are not going with AWS at the moment we will keep that in reserve as part of our disaster recovery planning.
As noted earlier, our final decision is to retain hosting at the University of Edinburgh but to redeploy on to IS infrastructure rather than work through EDINA. In the event of a no-deal-Brexit, the University has some model clauses which we will sign with overseas clients to uphold us to European regulations. We intend to change our local hosting arrangement in late 2019 / early 2020. There will be a small amount of downtime which we will announce in advance. Users will not notice any differences to the service.
RDA-DMP movings and shakings
An update on RDA and our Active DMP work, courtesy of Stephanie Simms
RDA Plenary 9
We had another productive gathering of #ActiveDMPs enthusiasts at the Research Data Alliance (RDA) plenary meeting in Barcelona (5-7 Apr). Just prior to the meet…
Active, actionable DMPs
Roadmap project IDCC briefing
We had a spectacularly productive IDCC last month thanks to everyone who participated in the various meetings and events focused on the DMPRoadmap project and machine-actionable DMPs. Thank you, thank you! Sarah has since …
Roadmap retrospective: 2016
Here’s an update on DMPRoadmap, courtesy of Stephanie Simms at CDL
2016 in review
The past year has been a wild ride, in more ways than one… Despite our respective political climates, UC3 and DCC remain enthusiastic about our partnership and the future of DMPs. Below is a brief retrospective about where we’ve been in 2016 and a roadmap (if you will…we also wish we’d chosen a different name for our joint project) for where we’re going in 2017. Jump to the end if you just want to know how to get involved with DMP events at the International Digital Curation Conference (IDCC 2017, 20–23 Feb in Edinburgh, register here).
In 2016 we consolidated our UC3-DCC project team, our plans for the merged platform (see the roadmap to MVP), and began testing a co-development process that will provide a framework for community contributions down the line. We’re plowing through the list of features and adding documentation to the GitHub repo—all are invited to join us at IDCC 2017 for presentations and demos of our progress to date (papers, slides, etc. will all be posted after the event). For those not attending IDCC, please let us know if you have ideas, questions, anything at all to contribute ahead of the event!
DMPs sans frontières
Now we’d like to take a minute and reflect on events of the past year, particularly in the realm of open data policies, and the implications for DMPs and data management writ large. The open scholarship revolution has progressed to a point where top-level policies mandate open access to the results of government-funded research, including research data, in the US, UK, and EU, with similar principles and policies gaining momentum in Australia, Canada, South Africa, and elsewhere. DMPs are the primary vehicle for complying with these policies, and because research is a global enterprise, awareness of DMPs has spread throughout the research community. Another encouraging development is the ubiquity of the term FAIR data (Findable, Accessible, Interoperable, Reusable), which suggests that we’re all in agreement about what we’re trying to achieve.
On top of the accumulation of national data policies, 2016 ushered in a series of related developments in openness that contribute to the DMP conversation. To name a few:
- More publishers articulated clear data policies, e.g., Springer Nature Research Data Policies apply to over 600 journals.
- PLOS now requires an ORCID for all corresponding authors at the time of manuscript submission to promote discoverability and credit.
- The Gates Foundation reinforced support for open access and open data by preventing funded researchers from publishing in journals that do not comply with its policy, which came into force at the beginning of 2017; this includes non-compliant high-impact journals such as Science, Nature, PNAS, and NEJM.
- Researchers throughout the world continued to circumvent subscription access to scholarly literature by using Sci-Hub (Bohannon, 2016).
- Library consortia in Germany and Taiwan canceled (or threatened to cancel) subscriptions to Elsevier journals because of open-access related conflicts, and Peru canceled over a lack of government funding for expensive paid access (Schiermeier and Rodríguez Mega, 2017).
- Reproducibility continued to gain prominence, e.g., the US National Institutes of Health (NIH) Policy on Rigor and Reproducibility came into force for most NIH and AHRQ grant proposals received in 2016.
- The Software Citation Principles (Smith et al., 2016) recognized software as an important product of modern research that needs to be managed alongside data and other outputs.
This flurry of open scholarship activity, both top-down and bottom-up, across all stakeholders continues to drive adoption of our services. DMPonline and the DMPTool were developed in 2011 to support open data policies in the UK and US, respectively, but today our organizations engage with users throughout the world. An upsurge in international users is evident from email addresses for new accounts and web analytics. In addition, local installations of our open source tools, as both national and institutional services, continue to multiply (see a complete list here).
Over the past year, the DMP community has validated our decision to consolidate our efforts by merging our technical platforms and coordinating outreach activities. The DMPRoadmap project feeds into a larger goal of harnessing the work of international DMP projects to benefit the entire community. We’re also engaged with some vibrant international working groups (e.g., Research Data Alliance Active DMPs, FORCE11 FAIR DMPs, Data Documentation Initiative DMP Metadata group) that have provided the opportunity to begin developing use cases for machine-actionable DMPs. So far the use cases encompass a controlled vocabulary for DMPs; integrations with other systems (e.g., Zenodo, Dataverse, Figshare, OSF, PURE, grant management systems, electronic lab notebooks); passing information to/from repositories; leveraging persistent identifiers (PIDs); and building APIs.
2017 things to come
This brings us to outlining plans for 2017 and charting a course for DMPs of the future. DCC will be running the new Roadmap code soon. And once we’ve added everything from the development roadmap, the DMPTool will announce our plans for migration. At IDCC we’ll kick off the conversation about bringing the many local installations of our tools along for the ride to actualize the vision of a core, international DMP infrastructure. A Canadian and a French team are our gracious guinea pigs for testing the draft external contributor guidelines.
There will be plenty of opportunities to connect with us at IDCC. If you’re going to be at the main conference, we encourage you to attend our practice paper and/or join a DMP session we’ll be running in parallel with the BoFs on Wednesday afternoon, 22 Feb. The session will begin with a demo and update on DMPRoadmap; then we’ll break into two parallel tracks. One track will be for developers to learn more about recent data model changes and developer guidelines if they want to contribute to the code. The other track will be a buffet of DMP discussion groups. Given the overwhelming level of interest in the workshop (details below), one of these groups will cover machine-actionable DMPs. We’ll give a brief report on the workshop and invite others to feed into discussion. The other groups are likely to cover training/supporting DMPs, evaluation cribsheets for reviewing DMPs, or other topics per community requests. If there’s something you’d like to propose please let us know!
IDCC DMP utopia workshop
We’re also hosting a workshop on Monday, 20 Feb entitled “A postcard from the future: Tools and services from a perfect DMP world.” The focus will be on machine-actionable DMPs and how to integrate DMP tools into existing research workflows and services.
The program includes presentations, activities, and discussion to address questions such as:
- Where and how do DMPs fit in the overall research lifecycle (i.e., beyond grant proposals)?
- Which data could be fed automatically from other systems into DMPs (or vice versa)?
- What information can be validated automatically?
- Which systems/services should connect with DMP tools?
- What are the priorities for integrations?
We’ve gathered an international cohort of diverse players in the DMP game—repository managers, data librarians, funders, researchers, developers, etc.—to continue developing machine-actionable use cases and craft a vision for a DMP utopia of the future. We apologize again that we weren’t able to accommodate everyone who wanted to participate in the workshop, but rest assured that we plan to share all of the outputs and will likely convene similar events in the future.
Keep a lookout for more detailed information about the workshop program in the coming weeks and feel free to continue providing input before, during, and afterward. This is absolutely a community-driven effort and we look forward to continuing our collaborations into the new year!
Getting our ducks in a row
Recent activity on the Roadmap project encompasses two major themes: 1) machine-actionable data management plans and 2) kicking off co-development of the shared codebase.
Image credit: ‘Get Your Ducks in a Row‘ CC-BY-SA by Cliff Johnson
Machine-actionable DMPs
The first of these has been a hot topic of conversation among stakeholders in the data management game for some time now, although most use the phrase “machine-readable DMPs.” So what do we mean by machine-actionable DMPs? Per the Data Documentation Initiative definition, “this term refers to information that is structured in a consistent way so that machines can be programmed against the structure.” The goal of machine-actionable DMPs, then, is to better facilitate good data management and reuse practices (think FAIR: Findable, Accessible, Interoperable, Reusable) by enabling:
- Institutions to manage their data
- Funders to mine the DMPs they receive
- Infrastructure providers to plan their resources
- Researchers to discover data
This term is consistent with the Research Data Alliance Active DMPs Interest Group and the FORCE11 FAIR DMPs group mission statements, and it seems to capture what we’re all thinking: i.e., we want to move beyond static text files to a dynamic inventory of digital research methods, protocols, environments, software, articles, data… One reason for the DMPonline-DMPTool merger is to develop a core infrastructure for implementing use cases that make this possible. We still need a human-readable document with a narrative, but underneath the DMP could have more thematic richness with value for all stakeholders.
A recent Cern/RDA workshop presented the perfect opportunity to consolidate our notes and ideas. In addition to the Roadmap project members, Daniel Mietchen (NIH) and Angus Whyte (DCC) participated in the exercise. We conducted a survey of previous work on the topic (we know we didn’t capture everything so please alert us to things we missed) and began outlining concrete use cases for machine-actionable DMPs, which we plan to develop further through community engagement over the coming months. Another crucial piece of our presentation was a call to make DMPs public, open, discoverable resources. We highlighted existing efforts to promote public DMPs (e.g., the DMPTool Public DMPs list, publishing exemplary DMPs in RIO Journal) but these are just a drop in the bucket compared to what we might be able to do if all DMPs were open by default.
You can review our slides here. And please send feedback—we want to know what you think!
Let the co-development begin!
Now for the second news item: our ducks are all in a row and work is underway on the shared Roadmap codebase.
We open with a wistful farewell to Marta Ribeiro, who is moving on to an exciting new gig at the Urban Big Data Centre. DCC has hired two new developers to join our ranks—Ray Carrick and Jimmy Angelakos—both from their sister team at EDINA. The finalized co-development team commenced weekly check-in calls and in the next week or two we’ll begin testing the draft co-development process by adding three features from the roadmap:
- Enhanced institutional branding
- Funder template export
- OAuth link an ORCID
In the meantime, Brian is completing the migration to Rails 4.2 and both teams are getting our development environments in place. Our intention is to iterate on the process for a few sprints, iron out the kinks, and then use it and the roadmap as the touchstones for a monthly community developer check-in call. We hope this will provide a forum for sharing use cases and plans for future work (on all instances of the tool) in order to prioritize, coordinate, and alleviate duplication of effort.
The DCC interns have also been plugging away at their respective projects. Sam Rust just finished building some APIs on creating plans and extracting guidance, and is now starting work on the statistics use case. Damodar Sójka meanwhile is completing the internationalization project, drawing from work done by the Canadian DMP Assistant team. We’ll share more details about their work once we roll it back into the main codebase.
Next month the UC Berkeley Web Services team will evaluate the current version of DMPonline to flag any accessibility issues that need to be addressed in the new system. We’ve also been consulting with Rachael Hu on UX strategy. We’re keeping track of requests for the new system and invite you to submit feedback via GitHub issues.
Stay tuned to GitHub and our blog channels for more documentation and regular progress updates.
The 20:51 sprint (Roadmap team-building: UK edition)
This week we hosted the DMPTool team to flesh out our plans for ‘roadmap’ – the joint codebase we’re building together based on DMPonline and DMPTool. The key focus was reviewing and prioritising tasks for an initial release. &n…
Hallelujah and praise the LARD! The first London Area Research Data group meeting
This is a guest post by Laurence Horton, Data Librarian at The London School of Economics and Political Science (LSE) It was a hot and stormy day; the rain fell in torrents – except at occasional intervals, when I dodged … Continue reading →
All that Big Data Is Not Going to Manage Itself: Part Two
Yesterday’s blog post described some of the federal government initiatives that have driven data management requirements over the past ten years or so. “Data management” is a hot job area right now, and if you tilt the digital stewardship universe a certain direction, almost everything we do falls under the rubric of “data management.” Data […]
DMPonline at OpenAIRE-COAR 2014
On 20th May 2014 I presented DMPonline at an OpenAIRE-COAR 2014 pre-conference workshop in Athens, Greece.read more