Take Two: Checking in from the University of Illinois

This fall we are piloting the workshop curriculum at all five partner institutions. In this post, librarian Eleanor Dickson shares her thoughts after leading a DDRF workshop at the University of Illinois at Urbana-Champaign. Feedback from all of the partners along with the assessment forms collected from attendees will inform development of our next iteration of workshop materials for spring 2017. You can read more about the modular DDRF curriculum in this update.

Q. Was there anything that worked particularly well in the workshop?

A. This was our second time leading at DDRF workshop at Illinois, and our session this time was an hour longer than our first pilot workshop in spring 2016. It worked better to have three hours to teach as opposed to two hours. We felt much less rushed. If we were to teach a two hour DDRF workshop again, I would be more selective about what was included and what was left out. This semester we also piloted the “Visualizing Text Data” module activity using the HathiTrust+Bookworm (HT+BW) tool. It was interesting to see how the librarians in the audience approached HT+BW, and it seemed they benefited from the opportunity to bring their own research question to the tool.

Q. Was there anything that surprised you during the workshop?

A. Both times we have taught the workshop, I have been surprised by the range of experience levels with various technical topics among attendees. Experience level does not easily break down along lines of job titles or years in the field. The range of base-line knowledge in the room during the workshop can make it challenging to pace the activities appropriately, but we have also seen that some attendees with greater experience have been willing to help their less experienced neighbors through activities.

Q. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

A. I would continue to improve our hands-on activities. I feel that they have gotten better from our first iteration of the workshop, but I’d still like to see us expand and modify them as we go forward. I would like to make them more interactive, with more time for discussion and reflection. I would also like to find ways to draw on attendees’ past experience with digital scholarship projects and reference questions. This change might ground and contextualize the research examples we provide during the introduction.

Q. What tips would you give to somebody else teaching similar workshops?

A. Be prepared: run through examples and activities ahead of time and try to anticipate where attendees might have difficulty. For example, this semester we provided all the Python commands to attendees in a text file. The file rendered well on our Mac laptops, but displayed poorly on the Windows computers in the lab we used. That said, something always seems to go wrong during technical workshops. These issues may be unavoidable, and often are neither your fault nor the fault of the attendees. Try to help attendees understand when their errors are things they can fix (such as entering a command properly) and when they are largely out of their control (such as software installing in an unexpected way).

Q. How would you encourage a fellow librarian to play a more active role in supporting data-driven research?

A. I would encourage them to look for opportunities to play with various tools and methods for their own inquiries. Hands-on experience is key when learning technical topics, and by applying their experimentation to a question of interest to them, the skills are more likely to stick.

How’d it go?: Checking in from Northwestern University

This fall we are piloting the workshop curriculum at all five partner institutions. In this post, librarian Geoffrey Morse shares his thoughts after leading his first DDRF workshop at Northwestern University. Feedback from all of the partners along with the assessment forms collected from attendees will inform development of our next iteration of workshop materials for spring 2017. You can read more about the modular DDRF curriculum in this update.

Q. Was there anything that worked particularly well in the workshop?

A. The workshop attendees seemed to follow along and replicate the results in the HTRC portions of the workshop without much difficulty. All of the attendees were able to create accounts, create work sets, and run algorithms.  While all had been familiar with the HathiTrust Digital Library, not all were aware of the HTRC prior to the workshop.

Q. Was there anything that surprised you during the workshop?

A. The range of participants in the workshop was broader than might have initially been expected. Librarians and library staff from all areas of the library were interested in learning more about this topic. Not only did library liaisons to academic departments attend, technical services staff including catalogers and staff from acquisitions also attended as did staff members from Repositories and Digital Curation.

Q. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

A. We did not have one change that stood out more than others so we have listed three interrelated changes below:

  • One change we might consider is, after the introduction, taking a single research question and following it through the entire workshop, from identifying and acquiring texts, to evaluating methods and tools, to running the analysis. Along the way we could still introduce other topics, methods, and tools, but this deeper dive into a single question, running like a spine throughout the workshop, might provide a greater sense of continuity throughout the workshop.
  • Another change we might consider would be to start off with a hands-on exercise closer to the beginning of the workshop and then move into the text mining background. The text mining background that we started with was important in terms of establishing context for the workshop but the first part of the workshop was mostly lecture until we got to create the workset in HTRC. Anther change we might consider making is to move the text scraping exercise to the first day of the workshop prior to doing the HTRC portions of the curriculum.  One comment from attendees was that moving from HTRC to the text scraping exercise with PythonAnywhere, and then back to HTRC later in the workshop is confusing for those who are novices in this area; however others liked the order of the workshop.
  • Finally, providing a very brief introduction to scripting in Python—something that shows what’s going on under the hood of the scripts we are running—might be beneficial to the workshop participants.

Q. What tips would you give to somebody else teaching similar workshops?

A. Having all of the files necessary for the hands on portion of the workshop using PythonAnywhere was helpful as was having people register for PythonAnywhere accounts prior to the workshop. Before starting the portion of the workshop using PythonAnywhere it is well worth the time to take ten minutes and make sure everyone has the correct files and the PythonAnywhere account set up.

Q. How would you encourage a fellow librarian to play a more active role in supporting data-driven research?

A. Hands-on experience in workshop environments such as the HTRC workshops is a good way to become familiar with both the tools and concepts involved in data driven research. To be able to support this or any kind of research it is important that librarians have a basic understanding of the concepts behind the research and tools necessary for doing the actual research.  While one need not be an expert in all aspects of text mining to be able to provide support they will need to have a basic foundation of knowledge and skill.

Howdy Partner: Checking in from Lafayette College

This fall we are piloting the workshop curriculum at all five partner institutions. In this post, librarians Lijuan Xu and Sarah Morris share their thoughts after leading a DDRF workshop for their colleagues at Lafayette College. Feedback from all of the partners along with the assessment forms collected from attendees will inform development of our next iteration of workshop materials for spring 2017. You can read more about the modular DDRF curriculum in this update.

Q. Was there anything that worked particularly well in the workshop?

A. Our reorganization of the modules worked well. After a brief introduction, we started with HT+Bookworm, which is relatively straightforward, before discussing the Workset Builder and HTRC algorithms. After the lunch break, we spent the last hour of the three-hour workshop on Extracted Features and gathering and working with text. By first focusing on HTRC’s off-the-shelf tools and then moving on to textual analysis beyond HTDL and HTRC allowed the content to flow more fluidly and seemed to resonate with the participants. Overall, our colleagues were happy with the workshop: they felt better informed about HTDL and HTRC and more confident with textual analysis. They also thought the workshop provided “food for thought” and information for further conversations regarding textual analysis.

The 10 participants in our workshop came from all areas of library work, including research and instruction, digital scholarship and services, special collections and archives, and technical services. The mixture of librarians and staff with various backgrounds and technical expertise resulted in much richer discussion and feedback.

Q. Was there anything that surprised you during the workshop?

A. We were pleasantly surprised that people were able to run Python scripts without any problem. However, it was frustrating that at different moments, individuals experienced errors that we could not account for, for example, while some did not encounter any issues, others had problems logging into their accounts, retrieving their worksets, or viewing results.

Q. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

A. While we were happy with our reorganization of the workshop, we think that the modules could benefit from slides with less text and/or fewer slides. It would also be beneficial to incorporate more concrete examples of how researchers have used HTDL and HTRC tools for research and textual analysis.

Q. What tips would you give to somebody else teaching similar workshops?

A. Build in a long break between the first and second halves of the workshop. The hour-long lunch break we provided allowed participants to recharge and refocus.

Consider team-teaching the workshop. It is challenging to learn, digest, and teach materials that others have developed and that you are not an expert on. Pairing up with another librarian will make it easier. You can bounce ideas off each other and walk through the materials together.

Q. How would you encourage a fellow librarian to play a more active role in supporting data-driven research?

A. Start having conversations among librarians to learn what kind of knowledge and expertise people already have in terms of textual analysis and discuss how to support this type of research.  Learn from each other. Play around with the tools and think about how they could be applied to projects that faculty and students are currently and/or might be engaged in.

The DDRF learning modules

In order to make the workshop materials flexible and reusable, early-on in the development process the DDRF curriculum group decided to draft the content in the form of learning modules. The modules relate to competencies in digital scholarship, as well as tie into particular HTRC tools and services. All except the introductory module have corresponding hands-on activities to reinforce the learning objectives established by the curriculum group.

There are five modules in the current pilot curriculum:

  1. Introduction to the HathiTrust and to text analysis
  2. Gathering textual data
  3. Managing and curating textual data
  4. Performing text analysis
  5. Visualizing textual data with HathiTrust+Bookworm

As we pilot the workshops this fall and gather assessment from attendees, we will have the opportunity to evaluate and modify the modules before we finalize the curriculum in the third year of the grant.

Growing the curriculum

Through the summer, the DDRF curriculum working group has been working with their partners at Lafayette College, Northwestern University, and the University of North Carolina at Chapel Hill to adapt and refine the draft curriculum used in the spring 2016 pilot workshops for events at each of the project institutions this fall. While the workshops will remain fairly consistent from institution-to-institution, there will be slight modifications based on such factors as workshop length and anticipated audience. By teaching another round of pilot workshops this semester, we will be able to better gauge both how well-suited the teaching materials are for different groups of attendees and how easily they can be re-taught by other librarians.

We’ll ask each institution to report back on their experience teaching DDRF workshops this fall and publish their responses here!

Pilot workshops at Indiana and Illinois

Last week, the project team had the opportunity to pilot the curricular materials they have been developing at two workshops held at Indiana University and the University of Illinois on April 12th and 14th, respectively. The workshops were held in the universities’ libraries, and they were open to librarians and graduate students in library and information science. About a dozen people attended each workshop. We think they were a success overall, but we did learn a few lessons about computer lab policies (more restrictive than expected!) and pacing (don’t throw out too many tool names!). Over the next two months, we’ll be refining the curriculum to share with the partners at the University of North Carolina, Lafayette College, and Northwestern University.

Drafting the pilot curriculum

The project’s curriculum working group has developed a preliminary draft of the learning materials to be used in the Digging Deeper, Reaching Further workshops. They shared it with partner institutions this week.

Much of the time in pulling together the preliminary content was devoted to planning, such as thinking about how to meet the needs of the librarian audience and outlining learning objectives. Over the last month, project staff also had the opportunity to speak with organizers of other library training initiatives, including the University of Rochester’s Digital Humanities Institute for Mid-Career Librarians and North Carolina State University’s Data and Visualization Institute for Librarians. These fruitful conversations with other organizers gave the curriculum group some great ideas that they are now integrating into their own plans.

Going forward, the curriculum group will use feedback from the partner institutions to finalize the first draft content for workshops to be piloted at the University of Illinois and Indiana University in mid-April.