Page History

Scope: Procedure for handling batch creation or editing of MARC Bib, MFHD, and Item Records for records of Digitization-produced related assets slated for delivery to support their management in the Catalog.

Contact:Jasmine Burns Jenn Colt

Unit: Batch Processing, Cataloging, Metadata Services, DCAPS

Date created: 02/17/2017

Date of next review: February 2018

This is a procedure triggered by DCAPS or Digitization work. The template or standard DCAPS procedure that involves this step is documented here.

1. Digitization Project Kick-off Meeting Review

Participant: Jasmine Burns

Input: n/a

Output: Expectations for upcoming, project-based work

Steps Involved:

During the kick-off meeting, a Metadata Services representative (with invited member from Batch or Cataloging, as needed) discusses the:

• Identifier/Record/Metadata Needs & Fit for the resources being digitized 
  • i.e. not all resources being digitized will be a good fit for record/metadata management in the Catalog.
  • Where should the metadata of record exist for the resources being digitized? Are those identifiers stable enough for use in digitization and preservation workflows?
• Assess Metadata/Cataloging/Batch work needed:
  • What metadata already exists, for which manifestation (the analog version, a digitized version, other?), and where is this metadata?
  • Do we need to create digital asset records:
    • from scratch? (Note: Metadata, Cataloging & Batch does not do this, but we can help coordinate the effort with the requesting parties)
    • derived from a physical or analog metadata record?
    • derived from a non-MARC source?
    • converted from a non-MARC source?
  • What turn-around / timeframe is required (Note: Metadata, Cataloging & Batch need at least 2 weeks notice generally).
• Confirm Understanding:
  • The records will be managed in the Catalog by Cataloging, Batch & Metadata staff.
  • We can handle suppressed records, but this is not a preferred situation (keeping suppressed records in the Catalog for digitization and preservation management).
  • We can derive metadata for other delivery or preservation systems (eCommons, SharedShelf, Hydra, CULAR, ...) based off of the records.

2. Digitization Lab Inventory & Digital Assets Identifiers Request

Participants: Jasmine Burns, user-9f226 (as needed), Gary Branch (as needed), Pamela Stansbury (as needed),

Input: Inventory of Items to be digitized - CSV with Call Number, Title, BIBID for physical/analog resource (if analog resource is already cataloged)

Output: Inventory of Items Needing a Catalog Record & Existing Records/Metadata to leverage

Steps Involved:

• The Digitization Lab work hands final inventory of items to be digitized (generally, Call Number, Title, BIBID of the physical/analog) to Metadata for review & routing. Metadata review includes:
  • Checking list for completeness.
  • Logging workflow steps into project tracking systems (probably just Zoho).
  • Capturing the hand-off in some version-controlled documentation source like https://github.com/cmh2166/AV2eCommons.
• According to decisions made at Digital Project Kick-off Meeting (Step 1), the Metadata Contact routes this inventory list and additional context to the appropriate Batch contact for digital (and possibly analog) record creation.
  • If the metadata of record is not destined for the Catalog, then Step 3 is skipped and Batch / Cataloging is not involved in the workflow past a Collection-level record.

3. Digital & Analog (as needed) Asset MARC Record Creation & Loading

Participants: Gary Branch (hands off to Batch staff member), Pamela Stansbury (as needed), Jasmine Burns (as needed)

Depending on required next step, one of 3a, 3b, 3c (or a to be created 3d) is followed:

3a. MARC exists for the Analog/Physical Item, Need to Derive MARC for the Digital Asset

Input: List of Analog Bibliographic Record IDs

Output: Derived, suppressed Digital Asset Bibliographic Record IDs, and a List of those IDs matched to the Analog Bib IDs

• List of Analog/Physical Bibliographic Records are handed to Batch Processing (Gary as point person).
• Batch Processing:
  • Derives Bibliographic Records for the Digital Surrogate from the Analog/Physical Bibliographic Record.
  • The Digital Asset, Derived Bibliographic Records have no holdings and are kept suppressed until reloaded with links (Step 6)
• The Digital Asset, Derived Bibliographic Records have need a unique a flag, like a 995 ignore, so Batch Processing's validation jobs don't flag these as errors
• Bib record derivation specification and data profile:
• based on format, to be documented.
• upon generation of the Digital Asset, Derived Bibliographic Records, Cataloging (Point Person, Pam) is requested to review a sample to check for adequate data.
• The resulting decisions/updates/edits should be added back to the format-specific derivation profile, to inform future derivations.

3b. MARC does not exist for the Analog/Physical Item, Need to Derive MARC for the Analog and the Digital Asset from a provided CSV

Input: CSV including Title (of Work), Date, Barcode (on Item), Part Number, Notes

Output: Generated, minimum-level Physical/Analog Bibliographic, MFHD, & Item Records; and Derived suppressed Digital Asset Bibliographic Record IDs, and a List of those IDs matched to the Analog Bib IDs

• The Requesting Party for the Digital Project creates and provides a CSV with the following information:
  • Title (of Work), Date, Barcode (on Item), Part Number, Notes, Other Metadata Fields as encountered / able to be pulled
• The CSV is handed to Batch Processing (Gary as point person).
  • That CSV is used by Batch (can Metadata help with this at all?) to generate minimum-level MARC Bibliographic, Holdings & Items records for the Physical/Analog Resource
  • the Physical/Analog Bibliographic Records are unsuppressed unless explicitly requested by the originating party (and approved in the kick-off meeting)
• Batch Processing:
  • Derives Bibliographic Records for the Digital Surrogate from the Analog/Physical Bibliographic Records generated above.
  • The Digital Asset, Derived Bibliographic Records have no holdings and are kept suppressed until reloaded with links (Step 6)
• The Digital Asset, Derived Bibliographic Records have need a unique a flag, like a 995 ignore, so Batch Processing's validation jobs don't flag these as errors
• Bib record derivation specification and data profile:
• based on format, to be documented.
• upon generation of the Digital Asset, Derived Bibliographic Records, Cataloging (Point Person, Pam) is requested to review a sample (of the physical and the digital records) to check for adequate data.
• The resulting decisions/updates/edits should be added back to the format-specific derivation profile, to inform future derivations.

3c. MARC exists (but not in Voyager) for the Analog/Physical Asset, Need to Load MARC for Analog/Physical Asset and Derive MARC for the Digital Asset

Input: Minimum level MARC records (Bib with holdings and items information as relevant)

Output: Voyager-loaded Physical/Analog Bibliographic, MFHD, & Item Records; and Derived suppressed Digital Asset Bibliographic Record IDs, and a List of those IDs matched to the Analog Bib IDs

• The Requesting Party for the Digital Project creates and provides a set of MARC Bibliographic records stored somewhere open to them, Metadata and Batch Processing
  • The Physical/Analog Bibliographic Records are loaded into Voyager.
  • These records are unsuppressed unless explicitly requested by the originating party (and approved in the kick-off meeting)
• Batch Processing:
  • Derives Bibliographic Records for the Digital Surrogate from the Analog/Physical Bibliographic Records generated above.
  • The Digital Asset, Derived Bibliographic Records have no holdings and are kept suppressed until reloaded with links (Step 6)
• The Digital Asset, Derived Bibliographic Records have need a unique a flag, like a 995 ignore, so Batch Processing's validation jobs don't flag these as errors
• Bib record derivation specification and data profile:
• based on format, to be documented.
• upon generation of the Digital Asset, Derived Bibliographic Records, Cataloging (Point Person, Pam) is requested to review a sample (of the physical and the digital records) to check for adequate data.
• The resulting decisions/updates/edits should be added back to the format-specific derivation profile, to inform future derivations.

4. Generate Updated Inventory with "eBibs", Hand back to Digitization

Participants: Jasmine Burns, Batch staff

• Batch provides Metadata with a list of Bibs generated for the digital and physical items.
• Metadata adds these identifiers to the original inventory CSV handed over by digitization (see step 2).
• Metadata hands this updated inventory to Digitization to be used for Digitization, Filenaming, & loading into Preservation.

5. Descriptive Metadata Integration

Participants: Jasmine Burns, user-9f226 (as needed)

After digitization magic happens, we need to integrate existing descriptive metadata with the generated files and simple inventory metadata for loading into delivery systems (eCommons, Hydra, SharedShelf, etc.)

• Metadata staff is contacted with updated information for the post-digitization inventory (updated with filenames, digitization notes, and Kaltura IDs if appropriate.
• Metadata staff will integrate KalturaIDs, preservation IDs, and catalog (or other) metadata into one CSV and prepare spreadsheets for delivery assets to be loaded into the specified delivery system.
• Metadata staff will also perform any necessary normalization for preparing metadata for ingest.
• Handoff: Metadata Staff will give metadata ingest spreadsheet for loading delivery assets into repository to Delivery repository contact (determined at Kick-off meeting).

6. Hydrate eBIB Stubs

Participants: Jasmine Burns, user-9f226 (as needed), Gary Branch (or designated Batch Staff member), Pamela Stansbury (as needed)

After the assets are loaded into a delivery system (eCommons, Hydra, SharedShelf, etc.), the digital asset records created need to be unsuppressed and updated with persistent URLs to the resource in the delivery system.

• The Delivery System contact (identified at the kick off meeting) will hand off a spreadsheet with BIBID, eBIBID, and persistent URL for delivery system to Metadata Services contact. The Metadata Services contact will:
  • Check list for completeness.
  • Log workflow steps into project tracking systems (probably just Zoho).
  • Capture updated metadata inventory in some version-controlled documentation source like https://github.com/cmh2166/AV2eCommons.
• If the system of record is the Catalog:
  • Metadata will hand to Batch Processing a CSV of digital asset bib ids and persistent URLs 
  • Batch Processing will un-suppress the digital asset bibliographic record and add the persistent URL.
  • Batch Processing will attach an appropriate Holdings for the Digital Asset, Derived Bibliographic Record (no Item record is attached to Digital Asset Bibliographic/Holdings records)

7. Create Collection-Level Bib Record for the Digital Collection

Participants: Jasmine Burns, Pamela Stansbury

Input: Digital Collection Link in Delivery System & Any Related Description

Output: Catalog BIB Record for the Digital Collection - Exemplar: https://newcatalog.library.cornell.edu/catalog/6790930 

Steps Involved:

2024

Please note that this procedure has been revised extensively as of May 2018, therefore previous references to the procedure may no longer apply.

...

These are the three cases in which DCAPS initiates a Batch process

1. Creation of bib records to provide access point for digitized assets

Participants: Jenn Colt, Batch staff

Scenario: Creation of eBIBs for digitized items in delivery systems. eBIBs will only be created if the following criteria is met:

• Items are individually cataloged (not items where the only representation in the catalog is at the collection-level)
• Access with a URL will be provided (only create eBIBs at the time of provisioning for access)

Input: spreadsheet of existing analog bibids with Delivery URLs

Output: eBIBs are created

• DCAPS emails Jenn Colt a spreadsheet of bib records and URLs
• Jenn Colt passes along to appropriate Batch Processing representative
• Batch processing creates bib records for the digitized materials and notifies Jenn Colt when they are complete

2. Identification in the catalog of bibliographic works that have been digitized

Participants: DCAPS Project Manager, Jenn Colt, Batch representative

Scenario: DCAPS has digitized items and would like the catalog record for the analog item to note that these items have been digitized (single items). There may or may not be a bibliographic record for the digital item, this code is simply to note that the analog item was digitized. Placed on the bibliographic record, this note does not presume to provide any information about which specific holdings were digitized or whether the work was digitized in whole or in part.

Input: spreadsheet of FOLIO instance UUIDs and the date that the items were digitized

Output: Bib records get 899 code

• DCAPS emails spreadsheet of FOLIO instance UUIDs to Jenn Colt to hand over to Batch representative
• Batch places 899 code on the BIB records with a subfield $d for date
• Batch notifies Jenn Colt which 899 code was used

3. Identification in the catalog of items that belong to a digital collection

Participants: DCAPS Project Manager, Jenn Colt, Batch representative

Scenario: DCAPS has a group of records that belong to a single digital collection and would like the catalog records to note that they belong to that collection

1. if representation of these items exists somewhere as a collection (i.e. finding aid, catalog record, etc.), this will be the likely scenario
2. it is strongly preferred that all items from a collection are sent at the same time, please email Jenn Colt prior to the request if this is not possible and must be sent in batches
   
   

Input: spreadsheet of BIBs

Output: BIBs get 899 code

• DCAPS emails spreadsheet of FOLIO instance UUIDs to Jenn Colt to hand over to Batch representative
• Batch places an 899 code on the records
• Batch notifies Jenn Colt which 899 code was used
• Upon receipt of a URL for the digital assets in the delivery system, the Metadata Contact passes that URL and any other relevant, collection-level information on to Cataloging (Pam).
• Cataloging (Pam) creates a collection-level Bib record for the Collection.

PostScript: Linking the Digital Asset Bib Record to the Physical/Analog Asset Bib Record

When deriving digital asset Catalog records from physical or analog asset Catalog records, we want to indicate the derivative relationship between the two resources.Currently:

• a special 035 field is generated by Batch that links the derived, digital asset bibliographic record to the physical/analog bibliographic record (documentation for this?)

Proposed:

• In generating future records as part of this workflow, we'd like to clearly indicate the derivate relationship so it can be better exploited in Metadata batch processes/updates as well as in Discovery interfaces.
• Use...
  • 899 field to relate the two records?
  • or further exploit and add a backwards pointer to the 035 field?
  • or fall more closely in-line with the MARC bibliographic specification by using a 776 field?
• This needs to be decided.

Metadata Examples & Profiles for the Above Workflow

Example: Pre-Digitization Inventory

to be added

Example: Post-Digitization Inventory / Metadata

https://github.com/cmh2166/AV2eCommons/blob/master/data/CUlecturetapes_20160211/post-Kaltura_files/CUlecturetapes_metadata_kalturaIDs.csv

Example: Digital Asset MARC Bibliographic Record

to be added

Example: Digital Collection-level MARC Bibliographic Record

...

• 776  0 8  $i Print version | Online version $w (FOLIO) bibidnum
• 899 ind1 = 0 $a 899code