Bootcamp: Introduction
Teaching: 15 min
Exercises: 0 min
Questions
what do library workers gain from code?
Objectives
understand terms, phrases, and concepts in software development and data science
identify and use best practice in data structures
use regular expressions in searches
Overall Bootcamp Schedule
- (Meta)Data Basics - including touching modeling, representations, and structures
- Using the command line (in the *nix Shell and a little Bash) to interact with (meta)data and run metadata scripts
- Versioning & collaborating on (meta)data using Git & GitHub
- Querying & updating (meta)data contained in traditional MySQL databases (a popular database selection at CUL & elsewhere)
Workshop Logistics
...
| Table of Contents |
|---|
Part 1: Jargon Busting
Teaching: 15 min
Exercises: 30 min
Questions
- what terms, phrases, or ideas around code or software development have you come across and perhaps feel you should know better?
Objectives
- understand terms, phrases, and concepts in software development and data science
Lesson
Requirements
boards/pads
pens
sticky notes*
Purpose
icebreaker
finding confidence level
expectation management
Task
This group task is an opportunity for everyone to get help understanding terms, phrases, or ideas around metadata, data, and scripting/software development.
...
Teaching: 30 min
Exercises: 15 min
Questions
- what best practice and generic skills underpin you encounters with software skills?
Objectives
- identify and use best practice in data structures
Foundations
Before we crack on with using the computational tools at our disposal, I want to spend some time on some foundation level stuff - a combination of best practice and generic skills.
The Computer is Stupid
This does not mean that the computer isn’t useful. Given a repetitive task, an enumerative task, or a task that relies on memory, it can produce results faster, more accurately, and less grudgingly than you or I. Rather when I say that you should keep in mind that the computer is stupid, I mean to say that computer only does what you tell it to. If it throws up an error it is often not your fault, rather in most cases the computer has failed to interpret what you mean because it can only work with what it knows (ergo, it is bad at interpreting). This is not to say that the people who told the computer what to tell you when it doesn’t know what to do couldn’t have done a better job with error messages, for they could. So keep in mind as we go along that if you find an error message frustrating, it isn’t the computer’s fault that it is giving you an archaic and incomprehensible error message, it is a human person’s.
Why take an automated or computational approach
Otherwise known as the ‘why not do it manually?’ question. To start with, I’m not anti-manual. I do plenty of things manually that a machine could do in an automated way because either a) I don’t know how to automate the task or b) I’m unlikely to repeat the task and estimate that automating it would take longer. However once you know you’ll need to repeat a task, you have a compelling reason to consider automating it. This is one of the main areas in which programmatic ways of doing outside of IT service environments are changing library practice. Andromeda Yelton, a US based librarian closely involved in the Code4Lib movement, put together an excellent American Library Association Library Technology Report called “Coding for Librarians: Learning by Example.” The report is pitched at a real world relevance level, and in it Andromeda describes scenarios library professionals told her about where learning a little programming, usually learning ad-hoc, had made a difference to their work, to the work of their colleagues, and to the work of their library.
Main lessons:
Borrow, Borrow, and Borrow again. This is a mainstay of programming and a practice common to all skill levels, from professional programmers to people like us hacking around in libraries;
The correct language to learn is the one that works in your local context. There truly isn’t a best language, just languages with different strengths and weaknesses, all of which incorporate the same fundamental principles;
Consider the role of programming in professional development. That is both yours and of those you manage;
Knowing (even a little) code helps you evaluate projects that use code. Programming can seem alien. Knowing some code makes you better at judging the quality of software development or planning activity that include software development
Automate to make the time to do something else! Taking the time to gather together even the most simple programming skills can save time to do more interesting stuff! (even if often that more interesting stuff is learning more programming skills …)
Why Automate?: see Geeks and repetitive tasks image.
Keyboard shortcuts are your friend
Though we will get more computational over the course of the programme, we can start our adventure into programming - as many of you will have already - with very simple things like keyboard shortcuts. We all have our favourites. Labour saving but also exploiting this stupid machine in the best possible way. Alongside the very basic ones (ctrl+s for save; ctrl+c for copy; ctrl+x for cut; ctrl+v for paste) my favourite (in a Windows or Linux machines) is alt+tab, a keyboard shortcut that switches between programs {Trainer note: ask other helpers what their favourites are}. You can do all the lessons in Library Carpentry without keyboard shortcuts, but note that they’ll likely come up a lot.
Plain text formats are your friend
Why? Because computers can process them!
...
??? was also chosen by me because it doesn’t clash with existing schemes. Though it is likely that notation schemes will emerge from existing individual practice, existing schema are available to represent headers, breaks, et al. One such scheme is Markdown, a lightweight markup language. Markdown files are as .md, are machine readable, human readable, and used in many contexts - GitHub for example, renders text via Markdown. An excellent Markdown cheat sheet is available on GitHub for those who wish to follow – or adapt – this existing schema. Notepad++ http://notepad-plus-plus.org/ is recommended for Windows users as a tool to write Markdown text in, though it is by no means essential for working with .md files. Mac or Unix users may find Komodo Edit, Text Wrangler, Kate, or Atom helpful. Combined with pandoc, a markdown file can be exported to PDF, LaTeX or other formats, so it is a great way to create machine-readable, easily searchable documents that can be repurposed in many ways. It is a universal document converter.
Naming files sensible things is good for you and for your computers
Working with data is made easier by structuring your stuff in a consistent and predictable manner.
...
The crucial bit for our purposes, however, is the file naming convention you choose. The name of a file is important to ensuring it and its contents are easy to identify. ‘Data.xslx’ doesn’t fulfil this purpose. A title that describes the data does. And adding dating convention to the file name, associating derived data with base data through file names, and using directory structures to aid comprehension strengthens those connection.
Key Points
- data structures should be consistent and predictable
- consider using semantic elements or data identifiers to data directories
- fit and adapt your data structure to your work
- apply naming conventions to directories and file names to identify them, to create associations between data elements, and to assist with the long term readability and comprehension of your data structures
Regular Expressions
Introduction to Data - Handout and Quiz
Introduction to Data - Handout Answers