Page History

Scope: This document details the methods by which these batch processing statistics  are derived, including information about tools and files used, and where they may be found. These statistics must be gathered from various Access reports, from multiple databases.  Some statistics are required for the department-wide spreadsheet that is shared with Library Administration; some are for inter-departmental use.

I.     Tools and Documents Used in Compiling Batch Statistics

...

. The department-wide spreadsheet can be found at L://tsstats/

...

LTS_Stats_

...

2005-[current year].xlsx (See image below

...

).

Contact: Nancy Solla

Unit: Batch Processing

Date last updated: 6/25/2019

Date of next review: June 2020

Image Added

The portion of this spreadsheet devoted to Batch Processing includes these categories:

Image Added

A. “New records (print)” – This information is found by running the “Batch Activity Summary Report” in the LTS Stats database. The total consists of “total Approvals input” + “total Books input” + “total NPAC input.” (net id in 948 is “batch” for these counts.)

1.) Database used: L://tsstats/1New_culv2-2012-2013.accdb.

2.) Use Macro “5 – Batch Summary Report,” highlighted below

Image Added

Criteria for this query:

Image Added

3.) The total consists of “total Approvals input” + “total Books input” + “total NPAC input.” (net id in 948 is “batch” for these counts.) Enter data into LTS_Stats_2005-20[YY].xlsx.

B. “New records (e-journals)” – This information is found in the “Batch Activity Summary Report” in the LTS Stats database. The “Catalog/Format/e-Resources (journals) total” is the desired total. (net id in 948 is “batch” for these counts.)

1.) Database used: L://tsstats/1New_culv2-2012-2013.accdb.

2.) Use Macro “5 – Batch Summary Report,” highlighted below.

Image Added

Criteria for this query:

Image Added

3.) The “Catalog/Format/e-Resources (journals) total” is the desired total. (net id in 948 is “batch” for these counts.) Enter data into

Image Removed

This query looks at the following criteria:

Image Removed

2.) Record Enhancement Query found under “Queries.” You will need to scroll to the very bottom of the list of queries to access it. (See image below.)

Image Removed

The criteria for this query looks like this:

Image Removed

The criteria for the “subfield_value” are: Like "WorldShare" Or Like "Fast_updates" Or Like "batchmatch" Or Like "ebookenhanced" Or Like "LVL3" Or Like "ogrm" Or Like "arab" Or Like "oels" Or Like "osea" Or Like "ice" Or Like "ohys" Or Like "oslv" Or Like "ocjk" Or Like "olat" Or Like "orom" Or Like "oeso" Or Like "osap"

B. An additional database, 3_BATCHstats.mdb can be found in the same folder. One query is to be run in this database, labeled “Batch Bib Summary Report.” It is found under “Macros.” (See image below.) When run, it produces a report with the title, “Batch Activity by Type Report.

Image Removed

This report runs two queries, one for items created and one for items modified, and displays the retrieved data. Criteria for these two queries are as follows:

Image Removed

And

Image Removed

C. Spreadsheets used for this process:

1.)    3_Batch_Stats_ FY[YY]_[YY].xlsx (found at L://tsstats/Batch Stats)

Image Removed

See Sections III.F and IV  of this document for details on the contents of this spreadsheet.

2.) LTS_Stats_2005-20[YY].xlsx (found at L://tsstats)

Image Removed

The portion of this spreadsheet devoted to Batch Processing includes these categories

Image Removed

See Section III of this document for details of how these numbers are determined.

3.) BATCHID_Stats_FY[YY[_[YY].xlsx (found at L:\Batch_Processing\BatchIDStats)

Image Removed

This document contains statistics for records created or updated by a Batch Processing staff member using their “batchnetid” log in. See Sections III.F and IV of this document for details of how these statistics are compiled.

4.) VoyagerExtracts20[YY]-[YY].xls (found at L:\Batch_Processing\LoadExtractLogs) – Contains log of OCLC extract jobs run for the FY; includes log data for  OCLC Batchmatch (MARCadia), OCLC Manuscript & Archive Records, OCLC Local Holdings, OCLC Daily Export/Delete, and Withdrawals.

...

.

C. “New records (e-books)” -- This information is found in the “Batch Activity Summary Report”  in the LTS Stats database. The total is derived by adding “Catalog/Format/Computer files” total + “Catalog/Format/e-Resources (books)” total + “Catalog/Format/Sound recordings” total. (net id in 948 is “batch” for these counts.)

1.) Database used: L://tsstats/1New_culv2-2012-2013.accdb.

2.) Use Macro “5 – Batch Summary Report,” highlighted below.

Image Added

Criteria for this query:

Image Added

3.) The total is derived by adding “Catalog/Format/Computer files” total + “Catalog/Format/e-Resources (books)” total + “Catalog/Format/Sound recordings” total. (net id in 948 is “batch” for these counts.) Enter data into LTS_Stats_2005-20[YY].xlsx.

D. “Records Exported” – The information for this total comes from the spreadsheet found at “L://Batch_Processing/LoadExtractLogs/VoyagerExtracts[FYdates].xls.” The total for the FY is derived by adding together the totals from the following sheets in the document: “Batchmatch (MARCadia)” + “OCLC Archives & Man. Records” (CRN12) + “OCLC Institutional Records” + “OCLC Local Holdings” (CRN10)  + “OCLC Daily Export" (CRN15 and CRN16). (NOTE: These jobs are cron jobs, run against large numbers of records at a time. Batchmatch MARCadia and OCLC Institutional Records scripts ceased to be run and logged as of FYE 1617.)

1.) Use “L://Batch_Processing/LoadExtractLogs/VoyagerExtracts[FYdates].xls.”

2.) The total for the FY is derived by adding together the totals from the following sheets in the document: “OCLC Archives & Man. Records” + “OCLC Local Holdings” + “OCLC Daily Export.” Enter data into LTS_Stats_2005-20[YY].xlsx

...

New records (print)

...

New records (e-journals)

...

New records (e-books)

...

Records exported

...

Records enhanced

...

Headings maintenance

...

Batch jobs

.

E. “Records enhanced

III.  How The Values for the Department Spreadsheet Are Gathered:

A. “New records (print)” – This information is found by running the “Batch Activity Summary Report” (I.A.1. above) Record Enhancement query in the LTS Stats database. The total consists of “total Approvals input” + “total Books input” + “total NPAC input.” (net id in 948 is “batch” for these counts.)

B. “New records (e-journals)” – This information is found in the “Batch Activity Summary Report” (I.A.1. above) in the LTS Stats database. The “Catalog/Format/e-Resources (journals) total” is the desired total. (net id in 948 is “batch” for these counts.)

C. “New records (e-books)” -- This information is found in the “Batch Activity Summary Report” (I.A.1. above) in the LTS Stats database. The total is derived by adding “Catalog/Format/Computer files” total + “Catalog/Format/e-Resources (books)” total + “Catalog/Format/Sound recordings” total. (net id in 948 is “batch” for these counts.)

D. “Records Exported” – The information for this total comes from the spreadsheet found at “L://Batch_Processing/LoadExtractLogs/VoyagerExtracts[FYdates].xls.” (I.C.4. above) The total for the FY is derived by adding together the totals from the following sheets in the document: “Batchmatch (MARCadia)” + “OCLC Archives & Man. Records” + “OCLC Institutional Records” + “OCLC MARC Holdings” + “OCLC Daily Export.” (These jobs are cron jobs, run against large numbers of records at a time.)

...

database 1New_culv2-2012-2013.accdb. It includes info for WorldShare, Batchmatch, ebooksenhanced, and all “Update Backlog Records” (storage items) jobs.  This query is run against the Batch_Stats table in the built by PH’s job in LS Tools, mentioned in H, below.To get the total number of records enhanced, run “Records enhanced” query and then run “Get Records Enhancements Grand Total” query. The resulting table will consist of one number: The total number of records enhanced in the FY. (Note: Operator will need to go into first query and edit, to limit for specific FY.)

1.) Database used: L://tsstats/1New_culv2-2012-2013.accdb.

    2.) Use QUERY “Record Enhancing.”  You will need to scroll to the very bottom of the list of queries to access it. (See image below.)

Image Added

The criteria for this query looks like this:

Image Added

The criteria for the “subfield_value” are: Like "WorldShare" Or Like "Fast_updates" Or Like "batchmatch" Or Like "ebookenhanced" Or Like "LVL3" Or Like "ogrm" Or Like "arab" Or Like "oels" Or Like "osea" Or Like "ice" Or Like "ohys" Or Like "oslv" Or Like "ocjk" Or Like "olat" Or Like "orom" Or Like "oeso" Or Like "osap"

3.) Enter data into LTS_Stats_2005-20[YY].xlsx.

 F. “Headings maintenance” – Batch Processing staff have each been given an additional Voyager sign-on, to be used when they are running batch processes using Gary Strawn’s LocationChanger or RecordReloader. These sign-ons take the form of “batch[netid]” and will show up in the record histories of any records edited in this fashion. A separate database called “3_BATCHstats.mdb,” a set of queries, and a report called “Batch Bib Summary Report” have been created to harvest this information. For this specific line item in the department spreadsheet, information will be gathered by running the “Batch Bib Summary Report” and using the total of updated records for the operator id of “batchjm17,” “batchstr1” and “GLOBAL”.

1.) Use database 3_BATCHstats.mdb

2.) Use Macro “BATCH BIB Summary Report”

Image Added

This report runs two queries, one for items created and one for items modified, and displays the retrieved data. Criteria for these two queries are as follows:

Image Added

And

Image Added

3.)  Use the total of updated records for the operator id of “batchjm17,” “batchstr1” and “GLOBAL." Enter data into LTS_Stats_2005-20[YY].xlsx.

G. “Batch OP ID Stats” – This information will be gathered by running the “Batch Bib Summary Report” found in the “3_BATCHstats.mdb.” Statistics for all batch operator id’s will be gathered for interdepartmental use and entered into the spreadsheet, “BATCHID_Stats_ FY[YY]_[YY].xlsx” found in L:\Batch_Processing\Statistics\BatchIDStats.

1.)    Use database 3_BATCHstats.mdb

2.)    Use Macro “BATCH BIB Summary Report” (See below)

Image Added

This report runs two queries, one for items created and one for items modified, and displays the retrieved data. Criteria for these two queries are as follows:

Image Added

And

Image Added

3.)  Use statistics for all batch operator ID’s. Enter into BATCHID_Stats_FY[YY[_[YY].xlsx (found at L:\Batch_Processing\Statistics\BatchIDStats). This document contains statistics for records created or updated by a Batch Processing staff member using their “batchnetid” log in. 

Image Added

 F. “Headings maintenance” – Batch Processing staff have each been given an additional Voyager sign-on, to be used when they are running batch processes using Gary Strawn’s LocationChanger or RecordReloader. These sign-ons take the form of “batch[netid]” and will show up in the record histories of any records edited in this fashion. A separate database called “3_BATCHstats.mdb,” (I.B. above) a set of queries, and a report called “Batch Bib Summary Report” have been created to harvest this information. For this specific line item in the department spreadsheet, information will be gathered by running the “Batch Bib Summary Report” and using the total of updated records for the operator id of “batchjm17,” “batchstr1” and “GLOBAL”.

G. “Batch OP ID Stats” – This information will be gathered by running the “Batch Bib Summary Report” found in the “3_BATCHstats.mdb.” (I.B. above) Statistics for all batch operatorid’s will be gathered for interdepartmental use and entered into the spreadsheet, “BATCHID_Stats_ FY[YY]_[YY].xlsx” (I.C.1., above) found in L:\Batch_Processing\BatchIDStats.

H. “Batch jobs”* – LTS managers would like to determine how many batch processes are run each month. Additionally, Batch Processing unit managers would like to know how many records for each vendor/collection are processed. Not all of this information can be gathered via the existing LTS statistics database, so Pete Hoyt built a job in LS Tools to gather the data. (See query spec, “BatchStats"BatchStats_countXandH.docx,” a copy of which has been placed in the L:\tsstats\Batch Stats folder). .") The job is found under “Administrative Proc” in LS Tools, and is called “TS Stats Summary”. The data from this job should be entered into the spreadsheet “3_Batch_Stats_FY[YY]_[YY].xlsx” found in L:\tsstats\. (Get count from adding together “#of jobs run” from each page of 3_Batch_Stats_FY15_16.xlsx. This grand total will be entered into the “Batch jobs” line item in the LTS_Stats_2005-20[YY].xlsx spreadsheet.)  (Do we want to create a report for this data that presents it in some way other than monthly breakouts?)

I. “Fastshelve” – In 1New_culv2-2012-2013.accdb, there is a make table query called “Find fastshelve” that can be run to gather bib-level statistics on the number of records fast-shelved by LTS staff. Criteria for the date range and the name of the output table must be manually changed each year.

*IV. Section H of “Batch Stats for FY End,” Explained

Staff member must run this LS Tools script found at Admin Proc/TS Stats Summary for each individual month. 

The data from the section headed “Summary for [date] through [date]” must be copied and pasted into the spreadsheet,

Staff member must run this LS Tools script, found at Admin Proc/TS Stats Summary (CGI24), for each individual month. 

The data from the second section, The data from the section headed “Batch Stats Summary” must have entries separated by “query” (‘bi’ on ‘Inputting’ page, ‘bm’ on ‘Maintenance’ page, and ‘ind3’ on ‘New’ page) and then by “subfield value,” and must be copied and pasted into the spreadsheet, “3_Batch_Stats_FY[yy]_[yy].xlsx” found in L:\tsstats\Batch Stats.

Sample of a report from “Cornell University Library: Voyager Production System
TS Statistics,” “show details” view, below.

A. Summary for 2017-11-01 through 2017-11-30 --- 463885 Activity records

].xlsx” found in L:\tsstats\Batch Stats.

Sample of a report from “Cornell University Library: Voyager Production System
TS Statistics,” “show details” view, below.