How to Gather Batch Statistics for the Fiscal Year End

These statistics must be gathered from various Access reports, from multiple databases.  Some statistics are required for the department-wide spreadsheet that is shared with Library Administration; some are for inter-departmental use.

I.     Tools and Documents Used in Compiling Batch Statistics

A. The LTS Stats Database can be found at L://tsstats/1New_culv2-2012-2013.accdb. Queries to be run in this database:

1.) Batch Activity Summary Report, found under “Macros.” (See image below.)

This query looks at the following criteria:

2.) Record Enhancement Query found under “Queries.” You will need to scroll to the very bottom of the list of queries to access it. (See image below.)

The criteria for this query looks like this:

The criteria for the “subfield_value” are: Like "WorldShare" Or Like "Fast_updates" Or Like "batchmatch" Or Like "ebookenhanced" Or Like "LVL3" Or Like "ogrm" Or Like "arab" Or Like "oels" Or Like "osea" Or Like "ice" Or Like "ohys" Or Like "oslv" Or Like "ocjk" Or Like "olat" Or Like "orom" Or Like "oeso" Or Like "osap"

B. An additional database, 3_BATCHstats.mdb can be found in the same folder. One query is to be run in this database, labeled “Batch Bib Summary Report.” It is found under “Macros.” (See image below.) When run, it produces a report with the title, “Batch Activity by Type Report.

This report runs two queries, one for items created and one for items modified, and displays the retrieved data. Criteria for these two queries are as follows:

And

C. Spreadsheets used for this process:

1.)    3_Batch_Stats_ FY[YY]_[YY].xlsx (found at L://tsstats/Batch Stats)

See Sections III.F and IV  of this document for details on the contents of this spreadsheet.

2.) LTS_Stats_2005-20[YY].xlsx (found at L://tsstats)

The portion of this spreadsheet devoted to Batch Processing includes these categories

See Section III of this document for details of how these numbers are determined.

3.) BATCHID_Stats_FY[YY[_[YY].xlsx (found at L:\Batch_Processing\BatchIDStats)

This document contains statistics for records created or updated by a Batch Processing staff member using their “batchnetid” log in. See Sections III.F and IV of this document for details of how these statistics are compiled.

4.) VoyagerExtracts20[YY]-[YY].xls (found at L:\Batch_Processing\LoadExtractLogs) – Contains log of OCLC extract jobs run for the FY; includes log data for  OCLC Batchmatch (MARCadia), OCLC Manuscript & Archive Records, OCLC Local Holdings, OCLC Daily Export/Delete, and Withdrawals.

II.   Batch Processing Statistics Requested For Dept. Spreadsheet (LTS_Stats_2005-20[YY].xlsx, see I.C.2, above.):

III.  How The Values for the Department Spreadsheet Are Gathered:

A. “New records (print)” – This information is found by running the “Batch Activity Summary Report” (I.A.1. above) in the LTS Stats database. The total consists of “total Approvals input” + “total Books input” + “total NPAC input.” (net id in 948 is “batch” for these counts.)

B. “New records (e-journals)” – This information is found in the “Batch Activity Summary Report” (I.A.1. above) in the LTS Stats database. The “Catalog/Format/e-Resources (journals) total” is the desired total. (net id in 948 is “batch” for these counts.)

C. “New records (e-books)” -- This information is found in the “Batch Activity Summary Report” (I.A.1. above) in the LTS Stats database. The total is derived by adding “Catalog/Format/Computer files” total + “Catalog/Format/e-Resources (books)” total + “Catalog/Format/Sound recordings” total. (net id in 948 is “batch” for these counts.)

D. “Records Exported” – The information for this total comes from the spreadsheet found at “L://Batch_Processing/LoadExtractLogs/VoyagerExtracts[FYdates].xls.” (I.C.4. above) The total for the FY is derived by adding together the totals from the following sheets in the document: “Batchmatch (MARCadia)” + “OCLC Archives & Man. Records” + “OCLC Institutional Records” + “OCLC MARC Holdings” + “OCLC Daily Export.” (These jobs are cron jobs, run against large numbers of records at a time.)

E. “Records enhanced” – This information is found by running the Record Enhancement query in the database 1New_culv2-2012-2013.accdb (I.A.2. above). It includes info for WorldShare, Batchmatch, ebooksenhanced, and all “Update Backlog Records” (storage items) jobs.  This query is run against the Batch_Stats table in the built by PH’s job in LS Tools, mentioned in H, below. To get the total number of records enhanced, run “Records enhanced” query and then run “Get Records Enhancements Grand Total” query. The resulting table will consist of one number: The total number of records enhanced in the FY. (Note: Need to go into first query and edit to limit for specific FY.)

 F. “Headings maintenance” – Batch Processing staff have each been given an additional Voyager sign-on, to be used when they are running batch processes using Gary Strawn’s LocationChanger or RecordReloader. These sign-ons take the form of “batch[netid]” and will show up in the record histories of any records edited in this fashion. A separate database called “3_BATCHstats.mdb,” (I.B. above) a set of queries, and a report called “Batch Bib Summary Report” have been created to harvest this information. For this specific line item in the department spreadsheet, information will be gathered by running the “Batch Bib Summary Report” and using the total of updated records for the operator id of “batchjm17,” “batchstr1” and “GLOBAL”.

G. “Batch OP ID Stats” – This information will be gathered by running the “Batch Bib Summary Report” found in the “3_BATCHstats.mdb.” (I.B. above) Statistics for all batch operatorid’s will be gathered for interdepartmental use and entered into the spreadsheet, “BATCHID_Stats_ FY[YY]_[YY].xlsx” (I.C.1., above) found in L:\Batch_Processing\BatchIDStats.

H. “Batch jobs”* – LTS managers would like to determine how many batch processes are run each month. Additionally, Batch Processing unit managers would like to know how many records for each vendor/collection are processed. Not all of this information can be gathered via the existing LTS statistics database, so Pete Hoyt built a job in LS Tools to gather the data. (See query spec, “BatchStats_countXandH.docx,” a copy of which has been placed in the L:\tsstats\Batch Stats folder). The job is found under “Administrative Proc” in LS Tools, and is called “TS Stats Summary”. The data from this job should be entered into the spreadsheet “3_Batch_Stats_FY[YY]_[YY].xlsx” found in L:\tsstats\. (Get count from adding together “#of jobs run” from each page of 3_Batch_Stats_FY15_16.xlsx. This grand total will be entered into the “Batch jobs” line item in the LTS_Stats_2005-20[YY].xlsx spreadsheet.)  (Do we want to create a report for this data that presents it in some way other than monthly breakouts?)

I. “Fastshelve” – In 1New_culv2-2012-2013.accdb, there is a make table query called “Find fastshelve” that can be run to gather bib-level statistics on the number of records fast-shelved by LTS staff. Criteria for the date range and the name of the output table must be manually changed each year.

*IV. Section H of “Batch Stats for FY End,” Explained

Staff member must run this LS Tools script found at Admin Proc/TS Stats Summary for each individual month. 

The data from the section headed “Summary for [date] through [date]” must be copied and pasted into the spreadsheet,

The data from the section headed “Batch Stats Summary” must have entries separated by “query” (‘bi’ on ‘Inputting’ page, ‘bm’ on ‘Maintenance’ page, and ‘ind3’ on ‘New’ page) and then by “subfield value,” and must be copied and pasted into the spreadsheet, “3_Batch_Stats_FY[yy]_[yy].xlsx” found in L:\tsstats\Batch Stats.

Sample of a report from “Cornell University Library: Voyager Production System
TS Statistics,” “show details” view, below.

A. Summary for 2017-11-01 through 2017-11-30 --- 463885 Activity records