Scope: This document details the methods by which these batch processing statistics are derived, including information about tools and files used, and where they may be found. These statistics must be gathered from various Access reports, from multiple databases. Some statistics are required for the department-wide spreadsheet that is shared with Library Administration; some are for inter-departmental use. The department-wide spreadsheet can be found at L://tsstats/LTS_Stats_2005-[current year].xlsx (See image below).
Contact: Nancy Solla
Unit: Batch Processing
Date last updated: 6/25/2019
Date of next review: June 2020
The portion of this spreadsheet devoted to Batch Processing includes these categories:
A. “New records (print)” – This information is found by running the “Batch Activity Summary Report” in the LTS Stats database. The total consists of “total Approvals input” + “total Books input” + “total NPAC input.” (net id in 948 is “batch” for these counts.)
1.) Database used: L://tsstats/1New_culv2-2012-2013.accdb.
2.) Use Macro “5 – Batch Summary Report,” highlighted below
Criteria for this query:
3.) The total consists of “total Approvals input” + “total Books input” + “total NPAC input.” (net id in 948 is “batch” for these counts.) Enter data into LTS_Stats_2005-20[YY].xlsx.
B. “New records (e-journals)” – This information is found in the “Batch Activity Summary Report” in the LTS Stats database. The “Catalog/Format/e-Resources (journals) total” is the desired total. (net id in 948 is “batch” for these counts.)
1.) Database used: L://tsstats/1New_culv2-2012-2013.accdb.
2.) Use Macro “5 – Batch Summary Report,” highlighted below.
Criteria for this query:
3.) The “Catalog/Format/e-Resources (journals) total” is the desired total. (net id in 948 is “batch” for these counts.) Enter data into LTS_Stats_2005-20[YY].xlsx.
C. “New records (e-books)” -- This information is found in the “Batch Activity Summary Report” in the LTS Stats database. The total is derived by adding “Catalog/Format/Computer files” total + “Catalog/Format/e-Resources (books)” total + “Catalog/Format/Sound recordings” total. (net id in 948 is “batch” for these counts.)
1.) Database used: L://tsstats/1New_culv2-2012-2013.accdb.
2.) Use Macro “5 – Batch Summary Report,” highlighted below.
Criteria for this query:
3.) The total is derived by adding “Catalog/Format/Computer files” total + “Catalog/Format/e-Resources (books)” total + “Catalog/Format/Sound recordings” total. (net id in 948 is “batch” for these counts.) Enter data into LTS_Stats_2005-20[YY].xlsx.
D. “Records Exported” – The information for this total comes from the spreadsheet found at “L://Batch_Processing/LoadExtractLogs/VoyagerExtracts[FYdates].xls.” The total for the FY is derived by adding together the totals from the following sheets in the document: “Batchmatch (MARCadia)” + “OCLC Archives & Man. Records” (CRN12) + “OCLC Institutional Records” + “OCLC Local Holdings” (CRN10) + “OCLC Daily Export" (CRN15 and CRN16). (NOTE: These jobs are cron jobs, run against large numbers of records at a time. Batchmatch MARCadia and OCLC Institutional Records scripts ceased to be run and logged as of FYE 1617.)
1.) Use “L://Batch_Processing/LoadExtractLogs/VoyagerExtracts[FYdates].xls.”
2.) The total for the FY is derived by adding together the totals from the following sheets in the document: “OCLC Archives & Man. Records” + “OCLC Local Holdings” + “OCLC Daily Export.” Enter data into LTS_Stats_2005-20[YY].xlsx.
E. “Records enhanced” – This information is found by running the Record Enhancement query in the database 1New_culv2-2012-2013.accdb. It includes info for WorldShare, Batchmatch, ebooksenhanced, and all “Update Backlog Records” (storage items) jobs. This query is run against the Batch_Stats table in the built by PH’s job in LS Tools, mentioned in H, below. To get the total number of records enhanced, run “Records enhanced” query and then run “Get Records Enhancements Grand Total” query. The resulting table will consist of one number: The total number of records enhanced in the FY. (Note: Operator will need to go into first query and edit, to limit for specific FY.)
1.) Database used: L://tsstats/1New_culv2-2012-2013.accdb.
2.) Use QUERY “Record Enhancing.” You will need to scroll to the very bottom of the list of queries to access it. (See image below.)
The criteria for this query looks like this:
The criteria for the “subfield_value” are: Like "WorldShare" Or Like "Fast_updates" Or Like "batchmatch" Or Like "ebookenhanced" Or Like "LVL3" Or Like "ogrm" Or Like "arab" Or Like "oels" Or Like "osea" Or Like "ice" Or Like "ohys" Or Like "oslv" Or Like "ocjk" Or Like "olat" Or Like "orom" Or Like "oeso" Or Like "osap"
3.) Enter data into LTS_Stats_2005-20[YY].xlsx.
F. “Headings maintenance” – Batch Processing staff have each been given an additional Voyager sign-on, to be used when they are running batch processes using Gary Strawn’s LocationChanger or RecordReloader. These sign-ons take the form of “batch[netid]” and will show up in the record histories of any records edited in this fashion. A separate database called “3_BATCHstats.mdb,” a set of queries, and a report called “Batch Bib Summary Report” have been created to harvest this information. For this specific line item in the department spreadsheet, information will be gathered by running the “Batch Bib Summary Report” and using the total of updated records for the operator id of “batchjm17,” “batchstr1” and “GLOBAL”.
1.) Use database 3_BATCHstats.mdb
2.) Use Macro “BATCH BIB Summary Report”
This report runs two queries, one for items created and one for items modified, and displays the retrieved data. Criteria for these two queries are as follows:
And
3.) Use the total of updated records for the operator id of “batchjm17,” “batchstr1” and “GLOBAL." Enter data into LTS_Stats_2005-20[YY].xlsx.
G. “Batch OP ID Stats” – This information will be gathered by running the “Batch Bib Summary Report” found in the “3_BATCHstats.mdb.” Statistics for all batch operator id’s will be gathered for interdepartmental use and entered into the spreadsheet, “BATCHID_Stats_ FY[YY]_[YY].xlsx” found in L:\Batch_Processing\Statistics\BatchIDStats.
1.) Use database 3_BATCHstats.mdb
2.) Use Macro “BATCH BIB Summary Report” (See below)
This report runs two queries, one for items created and one for items modified, and displays the retrieved data. Criteria for these two queries are as follows:
And
3.) Use statistics for all batch operator ID’s. Enter into BATCHID_Stats_FY[YY[_[YY].xlsx (found at L:\Batch_Processing\Statistics\BatchIDStats). This document contains statistics for records created or updated by a Batch Processing staff member using their “batchnetid” log in.
H. “Batch jobs”* – LTS managers would like to determine how many batch processes are run each month. Additionally, Batch Processing unit managers would like to know how many records for each vendor/collection are processed. Not all of this information can be gathered via the existing LTS statistics database, so Pete Hoyt built a job in LS Tools to gather the data. (See query spec, "BatchStats_countXandH.docx.") The job is found under “Administrative Proc” in LS Tools, and is called “TS Stats Summary”. The data from this job should be entered into the spreadsheet “3_Batch_Stats_FY[YY]_[YY].xlsx” found in L:\tsstats\. (Get count from adding together “#of jobs run” from each page of 3_Batch_Stats_FY15_16.xlsx. This grand total will be entered into the “Batch jobs” line item in the LTS_Stats_2005-20[YY].xlsx spreadsheet.) (Do we want to create a report for this data that presents it in some way other than monthly breakouts?)
Staff member must run this LS Tools script, found at Admin Proc/TS Stats Summary (CGI24), for each individual month.
The data from the second section, headed “Batch Stats Summary” must have entries separated by “query” (‘bi’ on ‘Inputting’ page, ‘bm’ on ‘Maintenance’ page, and ‘ind3’ on ‘New’ page) and then by “subfield value,” and must be copied and pasted into the spreadsheet, “3_Batch_Stats_FY[yy]_[yy].xlsx” found in L:\tsstats\Batch Stats.
Sample of a report from “Cornell University Library: Voyager Production System
TS Statistics,” “show details” view, below.
Batch Stats Summary
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
3.) The data from the second section, headed “Batch Stats Summary” must have entries separated by “query” (‘bi’ on ‘Inputting’ page, ‘bm’ on ‘Maintenance’ page, and ‘ind3’ on ‘New’ page) and then by “subfield value,” and must be copied and pasted into the spreadsheet, “3_Batch_Stats_FY[yy]_[yy].xlsx” found in L:\tsstats\Batch Stats.
I. “Fastshelve” – In 1New_culv2-2012-2013.accdb, there is a make table query called “Find fastshelve” (See image below) that can be run to gather bib-level statistics on the number of records fast-shelved by LTS staff. Criteria for the date range and the name of the output table must be manually changed each year.
1.) Use L://tsstats/1New_culv2-2012-2013.accdb.
2.) Run QUERY called "Find fastshelve."
Criteria for this query:
3.) The resulting table should be delivered to the Batch Processing Supervisor for review.