A recording of a presentation at Samvera Connect 2018 described thus, prototyping a core component of our new architecture to be horizontally scalable, designing a new architecture for our digital library with a wide ranging set of requirements and users, Stanford University Library has a robust digital library system called the Stanford Digital Repository. This repository holds a little under 500 TB of materials in preservation, and a little less than that for online access, from our cultural heritage digitization efforts and institutional repository outputs. These materials are managed across 90+ codebases serving a variety of functions from self-deposit web applications, to a nearly 10 year old parallel processing framework, to a digital repository assets publication mechanism leading into our Blacklight, Spotlight, and Geoblacklight applications - among other services and needs. At the core of this system is a Fedora 3 store. With Fedora 3 now end-of-lifed, and our system suffering from limited to no horizontal scalability options, we’re revisiting our system and architecture. We are writing it from the start with a goal to have data-forward, distributed microservices and some event-driven processing components. TACO, our new core management API, is the heart of this new architecture, and is currently being developed as a prototype. This talk will walk through the process of analysing our current system via a dataflows analysis, then planning how to create ‘seams’ in our current system to migrate towards our new system in an evolutionary fashion instead of a turn-key migration. A video recording of this session is available at the 'Related URL' below., and seeing where community technologies like Hyrax, Blacklight, and IIIF will connect
At Stanford libraries we've run hundreds of virtual machines to support dozens of applications. We've found the cost and complexity of patching and maintaining these machines to be untenable. We believe that a serverless infrastructure is our future and so we are using AWS Fargate (Elastic Container Services) and Lambda architecture to reduce our maintenance burden. We will explain the AWS offerings in this space, explain how we can set up a simple distributed system, and point out pitfalls that we've experienced. and A video recording of a presentation at Samvera Connect 2018 described thus