Migration Story – University of Maryland

Migration user stories from institutions are some of the best ways to understand how our users are working with Fedora, what issues they’re experiencing and what is working well for them. These stories are also equally as important for fellow community members looking to understand what is possible at their own institutions.

With the release of Fedora 6 in June 2021, we have been actively following along with community members who are testing out the migration tools and documentation and these early migrations have proven that our work has paid off. The tools are working, the documentation is solid and with help from communication channels like Slack and the mailing lists, institutions are taking the leap and making the move out of their old Fedoras into Fedora 6.

But we can’t forget that migration success stories are not just of those moving to Fedora 6. Migrations are costly, resource intensive and take months or years of planning, and any time an institution is able to secure the preservation of their digital content by migrating to a supported version of Fedora, we consider this a success. So today I am excited to share with you a migration success story from the University of Maryland.

Introduce yourself:

Joshua Westgard (JW): My name is Joshua Westgard and I am the Systems Librarian, Digital Programs & Initiatives at the University of Maryland College Park.

Tell us about the repository you migrated:

JW:  We currently have about 4,780 videos and 6,299 sound recordings.

These represent all of our time-based digital media collections, primarily consisting of digitized content from Special Collections and University Archives, Special Collections in Performing Arts, and the former Library Media Services department. Major holdings include content related to campus athletics, student life, broadcasting archives, and faculty and student recitals and performances.  We also have some digitized versions of content in our physical collections on legacy media types, such as VHS tapes.

Most of the content (~ 9,000 items) was migrated from a legacy system built on Fedora 2 plus an external media streaming service. This instance also includes some newly digitized content that was deposited directly into the new system (~ 2,000 items).

What other types of integrations or software did you use?

JW: This is an instance of Avalon Media System, with authentication through our campus’ central authentication service. We have extended Avalon to allow for IP-based access controls to be applied to the individual items because we needed to allow for unauthenticated access by users who are physically present on campus. We also needed to restrict access to certain buildings for collections that are governed by strict donor agreements. IP-based access controls allow us to do both of these things. 

In addition, we added a feature to Avalon that allows administrative users to download a copy of the master media file for each object, so they can fulfill patron requests directly without needing to ask IT to restore the file from preservation storage. 

Finally, we added a feature that allows administrative users to create access tokens that can be shared with researchers who need to access content that is otherwise restricted. These tokens can be configured to allow streaming or download (or both), and can be scheduled to expire.

Tell us about the challenges did you encountered during the migration process:

JW: Probably the biggest challenge was locating original media for certain very old collections. The legacy system predated our current digital preservation workflow and because the Fedora objects used Fedora’s external relationship feature (rels-ext) to point to the streaming server, the files were not available in our legacy Fedora instance to be extracted for migration. Ultimately we did manage to locate most of the preservation files, and the ones we could not locate were manually extracted from our legacy streaming server.

Overall, our challenges really were not related to Fedora but rather to wrangling our data, much of which was very old.

Do you have any recommendations for the Fedora Team on ways we could have helped?

JW: The migration toolkit met our needs. We extended migration-utils with a python script to convert the extracted data into the Avalon batch import CSV format. 

You can see this fork of the migration-utils for our local changes: https://github.com/umd-lib/migration-utils/tree/feature/LIBAVALON-143

What would you like other institutions to know about your experience?

JW: It was very useful for us to think about the migration not as a straight migration from the old system to the new system, but rather as following the  “Extract/Transform/Load” (ETL) paradigm.  Migration-utils provided the extract to disk (binaries plus CSV), and then significant reorganization and cleaning of the data (the “transform” step) was performed by library staff. Our target was the existing Avalon batch import format (this was the “load” step). One of the challenges this presented was that the Avalon CSV import format, due to the way it handles multi-valued fields, meant that we had to be very careful when moving data around in this extracted form.

You can find the University of Maryland’s AV repository here: https://av.lib.umd.edu

A big thank you to Josh Westgard and Kate Dohe for their contributions to both this post and for sharing their stories at the Fedora Summer Open House in July, 2022. If you have any questions about their migration or any of the information provided above, you can feel free to reach out to Josh at westgard@umd.edu.

Thank you again!

The post Migration Story – University of Maryland appeared first on Duraspace.org.

Migration Story – University of Maryland

Migration user stories from institutions are some of the best ways to understand how our users are working with Fedora, what issues they’re experiencing and what is working well for them. These stories are also equally as important for fellow community members looking to understand what is possible at their own institutions.

With the release of Fedora 6 in June 2021, we have been actively following along with community members who are testing out the migration tools and documentation and these early migrations have proven that our work has paid off. The tools are working, the documentation is solid and with help from communication channels like Slack and the mailing lists, institutions are taking the leap and making the move out of their old Fedoras into Fedora 6.

But we can’t forget that migration success stories are not just of those moving to Fedora 6. Migrations are costly, resource intensive and take months or years of planning, and any time an institution is able to secure the preservation of their digital content by migrating to a supported version of Fedora, we consider this a success. So today I am excited to share with you a migration success story from the University of Maryland.

Introduce yourself:

Joshua Westgard (JW): My name is Joshua Westgard and I am the Systems Librarian, Digital Programs & Initiatives at the University of Maryland College Park.

Tell us about the repository you migrated:

JW:  We currently have about 4,780 videos and 6,299 sound recordings.

These represent all of our time-based digital media collections, primarily consisting of digitized content from Special Collections and University Archives, Special Collections in Performing Arts, and the former Library Media Services department. Major holdings include content related to campus athletics, student life, broadcasting archives, and faculty and student recitals and performances.  We also have some digitized versions of content in our physical collections on legacy media types, such as VHS tapes.

Most of the content (~ 9,000 items) was migrated from a legacy system built on Fedora 2 plus an external media streaming service. This instance also includes some newly digitized content that was deposited directly into the new system (~ 2,000 items).

What other types of integrations or software did you use?

JW: This is an instance of Avalon Media System, with authentication through our campus’ central authentication service. We have extended Avalon to allow for IP-based access controls to be applied to the individual items because we needed to allow for unauthenticated access by users who are physically present on campus. We also needed to restrict access to certain buildings for collections that are governed by strict donor agreements. IP-based access controls allow us to do both of these things. 

In addition, we added a feature to Avalon that allows administrative users to download a copy of the master media file for each object, so they can fulfill patron requests directly without needing to ask IT to restore the file from preservation storage. 

Finally, we added a feature that allows administrative users to create access tokens that can be shared with researchers who need to access content that is otherwise restricted. These tokens can be configured to allow streaming or download (or both), and can be scheduled to expire.

Tell us about the challenges did you encountered during the migration process:

JW: Probably the biggest challenge was locating original media for certain very old collections. The legacy system predated our current digital preservation workflow and because the Fedora objects used Fedora’s external relationship feature (rels-ext) to point to the streaming server, the files were not available in our legacy Fedora instance to be extracted for migration. Ultimately we did manage to locate most of the preservation files, and the ones we could not locate were manually extracted from our legacy streaming server.

Overall, our challenges really were not related to Fedora but rather to wrangling our data, much of which was very old.

Do you have any recommendations for the Fedora Team on ways we could have helped?

JW: The migration toolkit met our needs. We extended migration-utils with a python script to convert the extracted data into the Avalon batch import CSV format. 

You can see this fork of the migration-utils for our local changes: https://github.com/umd-lib/migration-utils/tree/feature/LIBAVALON-143

What would you like other institutions to know about your experience?

JW: It was very useful for us to think about the migration not as a straight migration from the old system to the new system, but rather as following the  “Extract/Transform/Load” (ETL) paradigm.  Migration-utils provided the extract to disk (binaries plus CSV), and then significant reorganization and cleaning of the data (the “transform” step) was performed by library staff. Our target was the existing Avalon batch import format (this was the “load” step). One of the challenges this presented was that the Avalon CSV import format, due to the way it handles multi-valued fields, meant that we had to be very careful when moving data around in this extracted form.

You can find the University of Maryland’s AV repository here: https://av.lib.umd.edu

A big thank you to Josh Westgard and Kate Dohe for their contributions to both this post and for sharing their stories at the Fedora Summer Open House in July, 2022. If you have any questions about their migration or any of the information provided above, you can feel free to reach out to Josh at westgard@umd.edu.

Thank you again!

The post Migration Story – University of Maryland appeared first on Duraspace.org.

Migration user stories from institutions are some of the best ways to understand how our users are working with Fedora, what issues they’re experiencing and what is working well for them. These stories are also equally as important for fellow community members looking to understand what is possible at their own institutions.

With the release of Fedora 6 in June 2021, we have been actively following along with community members who are testing out the migration tools and documentation and these early migrations have proven that our work has paid off. The tools are working, the documentation is solid and with help from communication channels like Slack and the mailing lists, institutions are taking the leap and making the move out of their old Fedoras into Fedora 6.

But we can’t forget that migration success stories are not just of those moving to Fedora 6. Migrations are costly, resource intensive and take months or years of planning, and any time an institution is able to secure the preservation of their digital content by migrating to a supported version of Fedora, we consider this a success. So today I am excited to share with you a migration success story from the University of Maryland.

Introduce yourself:

Josh Westgard (JW): My name is Josh Westgard and I am the Systems Librarian, Digital Programs & Initiatives at the University of Maryland College Park.

Tell us about the repository you migrated:

JW:  We currently have about 4,780 videos, 6,299 sound recordings.

These represent all of our time-based digital media collections, primarily consisting of digitized content from Special Collections and University Archives, Special Collections in Performing Arts, and the former Library Media Services department. Major holdings include content related to campus athletics, student life, broadcasting archives, and faculty and student recitals and performances.  We also have some digitized versions of content in our physical collections on legacy media types, such as VHS tapes.

Most of the content (~ 9000 items) was migrated from a legacy system built on Fedora 2 + a media streaming service. This instance also includes some newly digitized content that was deposited directly into the new system (~ 2,000 items).

What other types of integrations or software did you use?

JW: This is an instance of Avalon Media System, with authentication through campus’ central authentication service. We have extended Avalon to allow for IP-based access controls to be applied to the individual items because we needed to allow unauthenticated access to users physically present on campus. We also needed to restrict access to certain buildings for collections governed by a strict donor agreement.

In addition, we added a feature to Avalon that allows administrative users to download a copy of the master media file for each object, so they can fulfill patron requests directly without needing to ask IT to restore the file from preservation storage. 

Finally, we added a feature that allows administrative users to create access tokens that can be shared with researchers who need to access content that is otherwise restricted. These tokens can be configured to allow streaming or download (or both), and can be scheduled to expire.

Tell us about the challenges did you encountered during the migration process:

JW: Probably the biggest challenge was locating original media for certain very old collections. The legacy system predated our current digital preservation workflow and because the Fedora objects used Fedora’s external relationship feature (rels-ext) to point to the streaming server the files were not available in Fedora for migration. Ultimately we did manage to locate most of the preservation files, and those we could not locate could be manually extracted from the legacy streaming server.

Overall, our challenges really were not related to Fedora but rather to wrangling our data, much of which was very old.

Do you have any recommendations for the Fedora Team on ways we could have helped?

JW: The migration toolkit met our needs. We extended migration-utils with a python script to convert the extracted data into the Avalon batch import CSV format. 

You can see this fork of the migration-utils for our local changes: https://github.com/umd-lib/migration-utils/tree/feature/LIBAVALON-143

What would you like other institutions to know about your experience?

JW: It was very useful for us to think about the migration not as a straight migration from the old system to the new system, but rather as following the  “Extract/Transform/Load” (ETL) paradigm.  Migration-utils provided the extract to disk (binaries plus CSV), and then significant reorganization and cleaning of the data (the “transform” step) was performed by library staff. Our target was the existing Avalon batch import format (this was the “load” step). One of the challenges this presented was that the Avalon CSV import format, due to the way it handles multi-valued fields, meant that we had to be very careful when moving data around in this extracted form.

You can find the University of Maryland’s AV repository here: https://av.lib.umd.edu

A big thank you to Josh Westgard and Kate Dohe for their contributions to both this post and for sharing their stories at the Fedora Summer Open House in July, 2022. If you have any questions about their migration or any of the information provided above, you can feel free to reach out to Josh at westgard@umd.edu.

Thank you again!

The post Fedora Migration Stories: University of Maryland College Park appeared first on Duraspace.org.

We hope that you all enjoyed your July long weekends, whether it was for July 4th in the US or Canada Day north of the border, we hope that the weather was nice, the BBQ’s were hot and drinks were cold. As we move officially in to summer, we wanted to keep you updated on some of the things we have going on in the coming weeks. Read on to find out more. 

News You’re Invited! Fedora Summer Open Houses 

We are officially inviting you to join us at one of our 2 upcoming Fedora Open Houses being held this month. These open houses are meant to be an opportunity to share use cases, ask questions and come together to learn about what we’ve been working on. In the interest of accommodating different time zones, we opted to host two separate events – one catering to North America and one for those in the EU/UK. We hope you can join us for one (or both!) of the following sessions: 

Tuesday, July 26th from 12pm – 1pm Eastern – https://bit.ly/3PkEJ0Y 

Wednesday, July 27th from 10am – 11am Eastern – https://bit.ly/3c2iiiQ 

The Open House events are FREE to attend, so bring your colleagues and friends to listen in on the conversation. The more the merrier! Check out all that is going on in our new Community Calendar! 

IMLS Grant Migration Toolkit 

At the end of June, the first draft of the Migration Toolkit was released to the community for feedback and comments. This toolkit is a result of the IMLS Funded Grant Project – Fedora Migration Paths & Tools: A Pilot Project. Pilot Partners at the University of Virginia and Whitman College collaboratively created the content for the toolkit to provide resources for migrating from Fedora 3 to the newest, most supported version of the software, Fedora 6. It contains instructions for the migration process, templates, metadata remediation best practices and user stories collected throughout the course of the grant. You can also find links to the migration utilities and validation tooling created as a result of the work. 

You can see the Draft Toolkit here: https://docs.google.com/document/d/1ZGrofJU-vAYOitGJabJ10Brcvo_oFjTO46FxEK7vmgM/edit?usp=sharing 

We are asking the community to review the contents, instructions and templates contained within the Toolkit and provide feedback and comments. All comments should be entered before July 29, 2022 at which point the Pilot Partners and Fedora team will review and finalize the document for community-wide distribution and use. We look forward to hearing your feedback! 

Technology Update 

In June, the committer and tech team delivered Fedora 6.2.0, which contained some non-critical bug fixes, improvements to the HMLT UI and a new Statistics API. This Statistics API, a feature of which was requested by community users, will allow you to gather information quickly and easily about the contents of your repository at a given point in time.  

Additionally, the team has continued to work on optimizing performance and resolving bugs. They have focused on updating dependencies and have spent time adding features to both the migration and validation tools. Both tools now support “resume” functionality so that you can restart a migration or validation from where the process previously stopped. You can also now opt for a “head” only (ie. The most recent) migration if you don’t need to return your entire Fedora 3 version history. 

The team continues to meet every Thursday at 11am Eastern if you would like to join us for our weekly Tech Call. This call is open for anyone to join, even if just to listen in on the conversation. All are welcome! 

You can find the zoom link in our community calendar. 

Where to Find Us 

August – IslandoraCon, Charlottetown, PEI Canada 

• Presentation: IMLS Grant – Fedora Migration Paths and Tools: A Pilot Project  

September – iPres 2022, Glasgow, Scotlanld 

• Workshop: Welcome to Fedora 6.0: Features, Migration Support & Integrations for Community Use Cases

October – DLF and affiliated events, Baltimore, MD 

• Learn@DLF Workshop: Welcome to Fedora 6.0: Features, Migration Support & Integrations for Community Use Cases 
• DLF Forum:  Fedora 6.0: Feature Highlight & Community Update
• DigiPres: Fedora 6.0: Features and the OCFL

Access 2022, Ottawa, Canada 

• Presentation: IMLS Grant Update – Fedora Migration Paths & Tools: A Pilot Project 

See a conference you want to attend and have a collaborative idea you’d like to work with the Fedora team on? Email us at arran.griffith@lyrasis.org. We would love to work together on join presentations to help bring Fedora to more users. 

What’s Happening in Our Partner Communities  Samvera 

• Call for Proposals for Samvera Connect is open until Aug. 8, 2022 
  • Submit your panel or presentation proposal here: https://forms.gle/yjb9Rh4XmV1fQozM6 
• Register to attend Samvera Connect here: https://samvera.atlassian.net/wiki/spaces/samvera/pages/1942978658/2022+Samvera+Connect+Registration 
• Changes in the Hyrax Community 
  • Daniel Pierce, Indiana University Libraries, was welcomed as the new Tech Lead for the Hyrax repository engine. 
  • Rebekah Kati, University of North Carolina Chapel Hill, announced as the new Hyrax Product Owner 

Islandora

• IslandoraCon will be held August 2-5,2022 at UPEI in Charlottetown, PEI. 
  • Registration is now open for IslandoraCon – https://2022.islandora.ca/ 
• Join the Code of Conduct Committee! 
  • In advance of IslandoraCon, the community is seeking volunteers for a Code of Conduct Committee lead by Amy Blau and Drew Heles. Sign up to volunteer here
• Islandora Annual General Meeting is taking place August 4, 2022 from 2pm-4pm Eastern 
  • Register here: https://t.co/VxTmXD1NRz 

OCFL 

• Editorial Board released guidelines for maintaining and operating OCLF Projects 
  • Find all the details here: https://bit.ly/3O9kItk 
• OCFL 1.1 Spec is nearly ready for release 
• Most recent community meeting minutes can be found here. 
  • Next meeting to take place Wed. Aug 11 8pm EDT/5pm PDT I Thurs. Aug 12 10am AEST/1am GMT 

Membership – Your support is valuable! 

Fedora is an open-source, community-supported program funded entirely by membership contributions. As we move in to membership renewals for FY2022-23, we wanted to thank all of our financial sponsors who continue to actively support this vital piece of software, used by countless institutions across the globe. Your funding supports our staff who work to develop, teach, engage and support all active Fedora users. Without our members, we would not be able to support the preservation of the vital content contained within the repositories of our users. Find out how you can help. Learn more and become a member today! 

Fedora Registry 

Part of our mission with Fedora 6.0 is to better understand our user install base. As a result, we are reaching out to the community now to help with these efforts. Is your institution in our registry? If so, are all the details up to date? Check out the current Fedora Registry here: https://duraspace.org/fedora/community/fedora-users/ 

Need an update or need to add your instance? Use this link: https://duraspace.org/registry/register-your-site/ 

By understanding our install base and what versions of Fedora are being used, we will be better prepared to provide the support necessary for our entire community. 

Get Involved 

Fedora is designed, built, used, and supported by the community. Join the conversation on our Fedora Slack channel or sign up for our Fedora community mailing list to stay in the loop. You can find more details here. 

The post Fedora Newsletter – July 2022 appeared first on Duraspace.org.