A recording of a presentation at Samvera Connect 2018 described thus, prototyping a core component of our new architecture to be horizontally scalable, designing a new architecture for our digital library with a wide ranging set of requirements and users, Stanford University Library has a robust digital library system called the Stanford Digital Repository. This repository holds a little under 500 TB of materials in preservation, and a little less than that for online access, from our cultural heritage digitization efforts and institutional repository outputs. These materials are managed across 90+ codebases serving a variety of functions from self-deposit web applications, to a nearly 10 year old parallel processing framework, to a digital repository assets publication mechanism leading into our Blacklight, Spotlight, and Geoblacklight applications - among other services and needs. At the core of this system is a Fedora 3 store. With Fedora 3 now end-of-lifed, and our system suffering from limited to no horizontal scalability options, we’re revisiting our system and architecture. We are writing it from the start with a goal to have data-forward, distributed microservices and some event-driven processing components. TACO, our new core management API, is the heart of this new architecture, and is currently being developed as a prototype. This talk will walk through the process of analysing our current system via a dataflows analysis, then planning how to create ‘seams’ in our current system to migrate towards our new system in an evolutionary fashion instead of a turn-key migration. A video recording of this session is available at the 'Related URL' below., and seeing where community technologies like Hyrax, Blacklight, and IIIF will connect
Keyword:
Workflow, Architecture, Repository, Connect 2018, and Samvera
Subject:
Samvera Community
Creator:
Frost, Hannah and Harlow, Christina
Contributor:
Stanford University Libraries and University of Utah
In 2006, Stanford Libraries built the Stanford Digital Repository (SDR). The system has served us well—thirteen years later, SDR contains over 2.0 million objects (~500 terabytes of content). We built SDR using open-source software (including Samvera, Fedora, and Blacklight) and an additional ~300,000 lines of custom code. We believe it is among the largest and most complex repository systems in research libraries, and yet the challenges we face are common. We have grown SDR to a point where it is extremely difficult for us to sustain. Some of our foundational technologies are not only aging but are beyond end-of-life. Meanwhile, we are challenged to continue offering a valuable, performant, highly-available repository service to our stakeholders. Over the past two years, we have analyzed the factors complicating sustainability, that work has led to operational changes that improve the current state and a plan for sustaining repository development combining open-source and custom software. Our presentation highlights the reasons SDR became unsustainable and shares areas where we have made improvements and where we go next. We believe the lessons we have learned are widely applicable to institutions that develop their own repository solutions., and A presentation at Samvera Connect 2019 described thus
Keyword:
Architecture, Repository, Sustainability, Samvera, and Connect 2019