Scope: This procedure explains the process of the pickup and scripted processing of order files from various vendors and our own POOF! system. It includes the methodology of automated vendor selection and the loading of bibliographic records and creation of purchase orders through Voyagerthe Lehigh App and FOLIO's bulk data import function.
Contact: Natalya Pikulik Jenn Colt
Unit: Batch Processing Automation and Acquisitions
Date reviewed: May 23, 2019 March 2024
Date of next review: April 20202025
...
Bulk import profile will discard records that match too closely
...
Automated orders are processed through a variety of sources: POOF!, Coutts Oasis Ally, Harrassowitz, Casalini, and others. We attempt to make the processing steps from each source as similar as possible, creating variations only when necessary and customization only when practical. The premise is to be able to add new vendors with a minimum of script adjustments.
Pick up file from source:Anchor A A - A POOF file is picked up from a cron job scheduled to run at 87:05 45 each weekday morning.Coutts Ally orders must go through an authorization process to produce a file which is downloaded to an LTS Share space: \\input\vendorRecords\Coutts_Ally That file is then processed by an lstools script. (NB: We are still attempting to get Coutts to automate the authorization and file produce process. When that is accomplished, we will institute a cron job for pickup.)
- A cron job to pick up and process Coutts Oasis orders runs at 6:00 each weekday morning
- Harrassowitz generates a file each day. Delivers file via FTP. Sends notification to an appropriate party. We established a cron job to process Ordering staff retrieve the file.
- Casalini generates a file as titles are selected. Delivers file via FTP (pre-order MARC records)Ordering staff retrieve the file.
-
Processing of files:Anchor B B - Both POOF and Coutts Ally orders are processed through a set of rules commonly called the order matrix. Each line depends on a separate a separate rule and the first time a record matches the criteria of the rule, instructions are in place to handle the record appropriately. For example, early in the table, there is a rule that "discards" records which have in the 008 position 28 an f or s for federal or state publications. They are sorted for manual examination as government publications before further processing. Another example of records removed from automated processing are bib records which contain field 981 subfield n REQ: Those records are sorted to a separate file for probable Amazon orderingAll files are created in an S3 bucket that acquisitions and automation staff have access to.
- After initial special exceptions are handled, the country code in the 008 field is examined, and a vendor assigned accordingly.
- Titles for which no vendor is easily automatically assigned according to country code, will be sent to another LTS share space: \\input\vendorFirmOrd\cts\Assign\ placed in a separate file. Depending on which rule applies, the titles the titles will be sorted as and rules applied based on whether they are electronic records,serials, early imprint dates, requesters, etc.
- MARC records with data errors are placed in a separate folder, Assign_Vendor, file for manual processing.
- Harrassowitz files end with the suffix ".24.ord.mrc" generate a Harrassowitz.purchase order. Cron job scansHarrassowitz server to pick up new order files and bulk import conditionally into catalog and create purchase orders.
- In most cases for which a which a vendor creates a MARC file of ordered titles, MARC records are loaded with the Lehigh app to create bib records and purchase order are generated via a bulk import profile established for that vendororders.
The bulk data import profile will discard bib records that match that match too closely to existing records. They will be placed in vendor-named discard folders in the cts folder on the LTS share space established for each vendor mentioned above.Anchor C C - Discarded "duplicates" are based on complete title matches or ISBN or LCCN.
- Discards are manually examined to ensure discarded title is indeed a duplicate. Discarded duplicates are placed in a folder, REJECTED_ORDERS, and are reported back to selectors the following morning by a cron jobPOs will be created but if the record attempts to attach itself to a previously existing inventory instance, the process to overlay the record and create items and holdings will be stopped. This is to prevent destructive overlay of existing records. These records need to be looked at by the person loading the record. If a new bib needs to be created (ie, a print one instead of an electronic), the PO will have to be relinked to the newly created record. These situations are generally caused by duplicated ISBNs.
Next steps: Follow LTS procedure number #117 for firm order final processing.