Scope: This document details the methods by which these batch processing statistics are derived, including information about tools and files used, and where they may be found. These statistics must be gathered from various Access reports, from multiple databases. Some statistics are required for the department-wide spreadsheet that is shared with Library Administration; some are for inter-departmental use.
...
.
...
The department-wide spreadsheet can be found at L://tsstats/
...
LTS_Stats_
...
2005-[current year].xlsx (See image below).
...
This query looks at the following criteria:
2.) Record Enhancement Query found under “Queries.” You will need to scroll to the very bottom of the list of queries to access it. (See image below.)
The criteria for this query looks like this:
The criteria for the “subfield_value” are: Like "WorldShare" Or Like "Fast_updates" Or Like "batchmatch" Or Like "ebookenhanced" Or Like "LVL3" Or Like "ogrm" Or Like "arab" Or Like "oels" Or Like "osea" Or Like "ice" Or Like "ohys" Or Like "oslv" Or Like "ocjk" Or Like "olat" Or Like "orom" Or Like "oeso" Or Like "osap"
B. An additional database, 3_BATCHstats.mdb can be found in the same folder. One query is to be run in this database, labeled “Batch Bib Summary Report.” It is found under “Macros.” (See image below.) When run, it produces a report with the title, “Batch Activity by Type Report.
This report runs two queries, one for items created and one for items modified, and displays the retrieved data. Criteria for these two queries are as follows:
And
C. Spreadsheets used for this process:
1.) 3_Batch_Stats_ FY[YY]_[YY].xlsx (found at L://tsstats/Batch Stats)
See Sections III.F and IV of this document for details on the contents of this spreadsheet.
2.) LTS_Stats_2005-20[YY].xlsx (found at L://tsstats)
The portion of this spreadsheet devoted to Batch Processing includes these categories
See Section III of this document for details of how these numbers are determined.
3.) BATCHID_Stats_FY[YY[_[YY].xlsx (found at L:\Batch_Processing\BatchIDStats)
This document contains statistics for records created or updated by a Batch Processing staff member using their “batchnetid” log in. See Section III.G of this document for details of how these statistics are compiled.
4.) VoyagerExtracts20[YY]-[YY].xls (found at L:\Batch_Processing\LoadExtractLogs) – Contains log of OCLC extract jobs run for the FY; includes log data for OCLC Batchmatch (MARCadia), OCLC Manuscript & Archive Records, OCLC Local Holdings, OCLC Daily Export/Delete, and Withdrawals.
II. Batch Processing Statistics Requested For Dept. Spreadsheet (LTS_Stats_2005-20[YY].xlsx, see I.C.2, above.):
...
New records (print)
...
New records (e-journals)
...
New records (e-books)
...
Records exported
...
Records enhanced
...
Headings maintenance
...
Batch jobs
III. How The Values for the Department Spreadsheet Are Gathered:
A. “New records (print)” – This information is found by running the “Batch Activity Summary Report” (I.A.1. above) in the LTS Stats database. The total consists of “total Approvals input” + “total Books input” + “total NPAC input.” (net id in 948 is “batch” for these counts.)
B. “New records (e-journals)” – This information is found in the “Batch Activity Summary Report” (I.A.1. above) in the LTS Stats database. The “Catalog/Format/e-Resources (journals) total” is the desired total. (net id in 948 is “batch” for these counts.)
C. “New records (e-books)” -- This information is found in the “Batch Activity Summary Report” (I.A.1. above) in the LTS Stats database. The total is derived by adding “Catalog/Format/Computer files” total + “Catalog/Format/e-Resources (books)” total + “Catalog/Format/Sound recordings” total. (net id in 948 is “batch” for these counts.)
D. “Records Exported” – The information for this total comes from the spreadsheet found at “L://Batch_Processing/LoadExtractLogs/VoyagerExtracts[FYdates].xls.” (I.C.4. above) The total for the FY is derived by adding together the totals from the following sheets in the document: “Batchmatch (MARCadia)” + “OCLC Archives & Man. Records” + “OCLC Institutional Records” + “OCLC MARC Holdings” + “OCLC Daily Export.” (These jobs are cron jobs, run against large numbers of records at a time.)
...
Contact: Nancy Solla
Unit: Batch Processing
Date last updated: 6/25/2019
Date of next review: June 2020
The portion of this spreadsheet devoted to Batch Processing includes these categories:
A. “New records (print)” – This information is found by running the “Batch Activity Summary Report” in the LTS Stats database. The total consists of “total Approvals input” + “total Books input” + “total NPAC input.” (net id in 948 is “batch” for these counts.)
1.) Database used: L://tsstats/1New_culv2-2012-2013.accdb.
2.) Use Macro “5 – Batch Summary Report,” highlighted below
Criteria for this query:
3.) The total consists of “total Approvals input” + “total Books input” + “total NPAC input.” (net id in 948 is “batch” for these counts.) Enter data into LTS_Stats_2005-20[YY].xlsx.
B. “New records (e-journals)” – This information is found in the “Batch Activity Summary Report” in the LTS Stats database. The “Catalog/Format/e-Resources (journals) total” is the desired total. (net id in 948 is “batch” for these counts.)
1.) Database used: L://tsstats/1New_culv2-2012-2013.accdb.
2.) Use Macro “5 – Batch Summary Report,” highlighted below.
Criteria for this query:
3.) The “Catalog/Format/e-Resources (journals) total” is the desired total. (net id in 948 is “batch” for these counts.) Enter data into LTS_Stats_2005-20[YY].xlsx.
C. “New records (e-books)” -- This information is found in the “Batch Activity Summary Report” in the LTS Stats database. The total is derived by adding “Catalog/Format/Computer files” total + “Catalog/Format/e-Resources (books)” total + “Catalog/Format/Sound recordings” total. (net id in 948 is “batch” for these counts.)
1.) Database used: L://tsstats/1New_culv2-2012-2013.accdb.
2.) Use Macro “5 – Batch Summary Report,” highlighted below.
Criteria for this query:
3.) The total is derived by adding “Catalog/Format/Computer files” total + “Catalog/Format/e-Resources (books)” total + “Catalog/Format/Sound recordings” total. (net id in 948 is “batch” for these counts.) Enter data into LTS_Stats_2005-20[YY].xlsx.
D. “Records Exported” – The information for this total comes from the spreadsheet found at “L://Batch_Processing/LoadExtractLogs/VoyagerExtracts[FYdates].xls.” The total for the FY is derived by adding together the totals from the following sheets in the document: “Batchmatch (MARCadia)” + “OCLC Archives & Man. Records” (CRN12) + “OCLC Institutional Records” + “OCLC Local Holdings” (CRN10) + “OCLC Daily Export" (CRN15 and CRN16). (NOTE: These jobs are cron jobs, run against large numbers of records at a time. Batchmatch MARCadia and OCLC Institutional Records scripts ceased to be run and logged as of FYE 1617.)
1.) Use “L://Batch_Processing/LoadExtractLogs/VoyagerExtracts[FYdates].xls.”
2.) The total for the FY is derived by adding together the totals from the following sheets in the document: “OCLC Archives & Man. Records” + “OCLC Local Holdings” + “OCLC Daily Export.” Enter data into LTS_Stats_2005-20[YY].xlsx.
E. “Records enhanced” – This information is found by running the Record Enhancement query in the database 1New_culv2-2012-2013.accdb. It includes info for WorldShare, Batchmatch, ebooksenhanced, and all “Update Backlog Records” (storage items) jobs. This query is run against the Batch_Stats table in the built by PH’s job in LS Tools, mentioned in H, below.To get the total number of records enhanced, run “Records enhanced” query and then run “Get Records Enhancements Grand Total” query. The resulting table will consist of one number: The total number of records enhanced in the FY. (Note: Operator will need to go into first query and edit, to limit for specific FY.)
1.) Database used: L://tsstats/1New_culv2-2012-2013.accdb.
2.) Use QUERY “Record Enhancing.” You will need to scroll to the very bottom of the list of queries to access it. (See image below.)
The criteria for this query looks like this:
The criteria for the “subfield_value” are: Like "WorldShare" Or Like "Fast_updates" Or Like "batchmatch" Or Like "ebookenhanced" Or Like "LVL3" Or Like "ogrm" Or Like "arab" Or Like "oels" Or Like "osea" Or Like "ice" Or Like "ohys" Or Like "oslv" Or Like "ocjk" Or Like "olat" Or Like "orom" Or Like "oeso" Or Like "osap"
3.) Enter data into LTS_Stats_2005-20[YY].xlsx.
F. “Headings maintenance” – Batch Processing staff have each been given an additional Voyager sign-on, to be used when they are running batch processes using Gary Strawn’s LocationChanger or RecordReloader. These sign-ons take the form of “batch[netid]” and will show up in the record histories of any records edited in this fashion. A separate database called “3_BATCHstats.mdb,” a set of queries, and a report called “Batch Bib Summary Report” have been created to harvest this information. For this specific line item in the department spreadsheet, information will be gathered by running the “Batch Bib Summary Report” and using the total of updated records for the operator id of “batchjm17,” “batchstr1” and “GLOBAL”.
1.) Use database 3_BATCHstats.mdb
2.) Use Macro “BATCH BIB Summary Report”
This report runs two queries, one for items created and one for items modified, and displays the retrieved data. Criteria for these two queries are as follows:
And
3.) Use the total of updated records for the operator id of “batchjm17,” “batchstr1” and “GLOBAL." Enter data into LTS_Stats_2005-20[YY].xlsx.
G. “Batch OP ID Stats” – This information will be gathered by running the “Batch Bib Summary Report” found in the “3_BATCHstats.mdb.” Statistics for all batch operator id’s will be gathered for interdepartmental use and entered into the spreadsheet, “BATCHID_Stats_ FY[YY]_[YY].xlsx” found in L:\Batch_Processing\Statistics\BatchIDStats.
1.) Use database 3_BATCHstats.mdb
2.) Use Macro “BATCH BIB Summary Report” (See below)
This report runs two queries, one for items created and one for items modified, and displays the retrieved data. Criteria for these two queries are as follows:
And
3.) Use statistics for all batch operator ID’s. Enter into BATCHID_Stats_FY[YY[_[YY].xlsx (found at L:\Batch_Processing\Statistics\BatchIDStats). This document contains statistics for records created or updated by a Batch Processing staff member using their “batchnetid” log in.
F. “Headings maintenance” – Batch Processing staff have each been given an additional Voyager sign-on, to be used when they are running batch processes using Gary Strawn’s LocationChanger or RecordReloader. These sign-ons take the form of “batch[netid]” and will show up in the record histories of any records edited in this fashion. A separate database called “3_BATCHstats.mdb,” (I.B. above) a set of queries, and a report called “Batch Bib Summary Report” have been created to harvest this information. For this specific line item in the department spreadsheet, information will be gathered by running the “Batch Bib Summary Report” and using the total of updated records for the operator id of “batchjm17,” “batchstr1” and “GLOBAL”.
G. “Batch OP ID Stats” – This information will be gathered by running the “Batch Bib Summary Report” found in the “3_BATCHstats.mdb.” (I.B. above) Statistics for all batch operator id’s will be gathered for interdepartmental use and entered into the spreadsheet, “BATCHID_Stats_ FY[YY]_[YY].xlsx” (I.C.3., above) found in L:\Batch_Processing\BatchIDStats.
H. “Batch jobs”* – LTS managers would like to determine how many batch processes are run each month. Additionally, Batch Processing unit managers would like to know how many records for each vendor/collection are processed. Not all of this information can be gathered via the existing LTS statistics database, so Pete Hoyt built a job in LS Tools to gather the data. (See query spec, “BatchStats"BatchStats_countXandH.docx,” a copy of which has been placed in the L:\tsstats\Batch Stats folder). .") The job is found under “Administrative Proc” in LS Tools, and is called “TS Stats Summary”. The data from this job should be entered into the spreadsheet “3_Batch_Stats_FY[YY]_[YY].xlsx” found in L:\tsstats\. (Get count from adding together “#of jobs run” from each page of 3_Batch_Stats_FY15_16.xlsx. This grand total will be entered into the “Batch jobs” line item in the LTS_Stats_2005-20[YY].xlsx spreadsheet.) (Do we want to create a report for this data that presents it in some way other than monthly breakouts?)
I. “Fastshelve” – In 1New_culv2-2012-2013.accdb, there is a make table query called “Find fastshelve” that can be run to gather bib-level statistics on the number of records fast-shelved by LTS staff. Criteria for the date range and the name of the output table must be manually changed each year.
...
_Stats_FY15_16.xlsx. This grand total will be entered into the “Batch jobs” line item in the LTS_Stats_2005-20[YY].xlsx spreadsheet.) (Do we want to create a report for this data that presents it in some way other than monthly breakouts?)
Staff member must run this LS Tools script, found at Admin Proc/TS Stats Summary (CGI24), for each individual month.
The data from the second section, headed “Batch Stats Summary” must have entries separated by “query” (‘bi’ on ‘Inputting’ page, ‘bm’ on ‘Maintenance’ page, and ‘ind3’ on ‘New’ page) and then by “subfield value,” and must be copied and pasted into the spreadsheet,
...
“3_Batch_Stats_FY[yy]_[yy].
...
xlsx” found in L:\tsstats\Batch Stats.
Sample of a report from “Cornell University Library: Voyager Production System
TS Statistics,” “show details” view, below.
Batch Stats Summary
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
3.) The data from the second section, headed “Batch Stats Summary” must have entries separated by “query” (‘bi’ on ‘Inputting’ page, ‘bm’ on ‘Maintenance’ page, and ‘ind3’ on ‘New’ page) and then by “subfield value,” and must be copied and pasted into the spreadsheet, “3_Batch_Stats_FY[yy]_[yy].xlsx” found in L:\tsstats\Batch Stats.
I. “Fastshelve” – In 1New_culv2-2012-2013.accdb, there is a make table query called “Find fastshelve” (See image below) that can be run to gather bib-level statistics on the number of records fast-shelved by LTS staff. Criteria for the date range and the name of the output table must be manually changed each year.
1.) Use L://tsstats/1New_culv2-2012-2013.accdb.
2.) Run QUERY called "Find fastshelve."
Criteria for this query:
3.) The resulting table should be delivered to the Batch Processing Supervisor for review.