Author: E Dickson

1. Was there anything that worked particularly well in the workshop?

We incorporated more discussion questions into this iteration of the workshops and that seemed to go well. It gave participants the chance to talk and reflect, which was a nice break from looking at a screen. The questions could use further refinement, but I was pleased with how the conversations unfolded.

2. Was there anything that you felt was tricky to teach?

We were pressed for time toward the end. When we got to the more technical pieces in the latter-half of the workshop, it felt like we shifted gears too quickly and moved from material participants were easily grasping to that which they needed more time to work through. Unfortunately, we had not allocated enough time to let them dive into the more challenging material. Additionally, there was a wide range of familiarity with programming in the room, which made it especially difficult to target the content appropriately as we rushed through it!

3. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

In this iteration of the workshop materials, we tested out some content related to data management and learned that we had gone too far off-track with those materials. The nice part of iterating through the content is that we are able to test things out and then remove them when we find they didn’t work as well as we had hoped!

4. What changes would make the workshops more accommodating to a diverse range of participants with different roles in the library and varying technology skillsets?

We continue to have an audience with diverse skillsets attending these workshops at our library. Improved opportunities for either paired workshop activities or adding “challenge” activities for people who finish quickly would accommodate our audience better. Hopefully that would make the time we provide to work through the hands-on material better adapted to a range of backgrounds and skills.

1. Was there anything that worked particularly well in the workshop?

We added an introductory slide to the workshop content and framed the workshop as an opportunity for participants to develop some very basic text analysis skills as well as for us to improve the workshop materials. We took out some slides so that the workshop was only three hours (with an hour in between for lunch). The three-hour length seemed perfect to keep people’s attention. At the end of the workshop, we asked people to fill out the assessment survey but also verbally tell us whether the workshop met their expectations and how it could be improved. People were very honest and gave us all kinds of interesting feedback. The discussion went on for 30 minutes, much longer than we expected.

2. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

Include more research and assignment examples, especially those having to do with HTDL content. Participants found the textual analysis examples in Module 1 the most interesting and useful to help them think about how they practice text analysis on their own and/or talk to faculty members about text analysis.

3. What changes would make the workshops more accommodating to a diverse range of participants with different roles in the library and varying technology skillsets?

The challenge we encountered and we think will be very important to keep in mind as we move forward is the institutions that participants are from. Most of our participants in this workshop came from small institutions that are teaching and not research-centered. Having examples of text analysis classroom assignments in addition to research projects would be really beneficial.

4. After several rounds of workshops, do you have any thoughts or comments on our iterative workshop model?

Having people from different institutions teach the same materials to participants with varying text analysis experience and revising the materials based on the feedback definitely help improve the workshop content. Teaching it a couple of times also helps us have a much better understanding and grasp of the materials.

1. Was there anything that worked particularly well in the workshop? 
   This was our third time offering the DDRF workshop at Indiana University Bloomington and I think it was our most successful. This time around we partnered with a faculty member in the School of Information and Library Science and invited students from their Digital Humanities course. We had a great group of curious students (and a few attendees from the library) and the instructor, John Walsh, helped facilitate meaningful discussions throughout the workshop. Compared to past workshops, this round was much more participatory and the attendees seemed more engaged.
2. Was there anything that you felt was tricky to teach? 
   Running algorithms in the portal was a little tricky, mostly because the entire group wanted to see their results all at once! Luckily we have the example prepped and ready to go before the workshop, but sometimes participants are disappointed that they are unable to view their results immediately [because of how the activities are structured].
3. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be? 
   I think we’ve made significant progress in improving our hands-on activities, but I’d still like to see more room for creativity. It’s challenging to leave room for exploration when there’s so much material to cover, but I would like to see a little more tinkering and less step-by-step activities.
4. What changes would make the workshops more accommodating to a diverse range of participants with different roles in the library and varying technology skillsets? 
   I feel confident that the materials are appropriate for a wide audience, but the issue we struggle with is marketing. There is a key group of librarians and library staff members at IU who are already interested in text analysis, so naturally they want to learn more about the HTRC, but it’s tough to reach out to other folks in the library and explain why they should participate.

This fall we are piloting the workshop curriculum at all five partner institutions. In this post, librarian Eleanor Dickson shares her thoughts after leading a DDRF workshop at the University of Illinois at Urbana-Champaign. Feedback from all of the partners along with the assessment forms collected from attendees will inform development of our next iteration of workshop materials for spring 2017. You can read more about the modular DDRF curriculum in this update.

Q. Was there anything that worked particularly well in the workshop?

A. This was our second time leading at DDRF workshop at Illinois, and our session this time was an hour longer than our first pilot workshop in spring 2016. It worked better to have three hours to teach as opposed to two hours. We felt much less rushed. If we were to teach a two hour DDRF workshop again, I would be more selective about what was included and what was left out. This semester we also piloted the “Visualizing Text Data” module activity using the HathiTrust+Bookworm (HT+BW) tool. It was interesting to see how the librarians in the audience approached HT+BW, and it seemed they benefited from the opportunity to bring their own research question to the tool.

Q. Was there anything that surprised you during the workshop?

A. Both times we have taught the workshop, I have been surprised by the range of experience levels with various technical topics among attendees. Experience level does not easily break down along lines of job titles or years in the field. The range of base-line knowledge in the room during the workshop can make it challenging to pace the activities appropriately, but we have also seen that some attendees with greater experience have been willing to help their less experienced neighbors through activities.

Q. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

A. I would continue to improve our hands-on activities. I feel that they have gotten better from our first iteration of the workshop, but I’d still like to see us expand and modify them as we go forward. I would like to make them more interactive, with more time for discussion and reflection. I would also like to find ways to draw on attendees’ past experience with digital scholarship projects and reference questions. This change might ground and contextualize the research examples we provide during the introduction.

Q. What tips would you give to somebody else teaching similar workshops?

A. Be prepared: run through examples and activities ahead of time and try to anticipate where attendees might have difficulty. For example, this semester we provided all the Python commands to attendees in a text file. The file rendered well on our Mac laptops, but displayed poorly on the Windows computers in the lab we used. That said, something always seems to go wrong during technical workshops. These issues may be unavoidable, and often are neither your fault nor the fault of the attendees. Try to help attendees understand when their errors are things they can fix (such as entering a command properly) and when they are largely out of their control (such as software installing in an unexpected way).

Q. How would you encourage a fellow librarian to play a more active role in supporting data-driven research?

A. I would encourage them to look for opportunities to play with various tools and methods for their own inquiries. Hands-on experience is key when learning technical topics, and by applying their experimentation to a question of interest to them, the skills are more likely to stick.

This fall we are piloting the workshop curriculum at all five partner institutions. In this post, librarian Geoffrey Morse shares his thoughts after leading his first DDRF workshop at Northwestern University. Feedback from all of the partners along with the assessment forms collected from attendees will inform development of our next iteration of workshop materials for spring 2017. You can read more about the modular DDRF curriculum in this update.

Q. Was there anything that worked particularly well in the workshop?

A. The workshop attendees seemed to follow along and replicate the results in the HTRC portions of the workshop without much difficulty. All of the attendees were able to create accounts, create work sets, and run algorithms.  While all had been familiar with the HathiTrust Digital Library, not all were aware of the HTRC prior to the workshop.

Q. Was there anything that surprised you during the workshop?

A. The range of participants in the workshop was broader than might have initially been expected. Librarians and library staff from all areas of the library were interested in learning more about this topic. Not only did library liaisons to academic departments attend, technical services staff including catalogers and staff from acquisitions also attended as did staff members from Repositories and Digital Curation.

Q. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

A. We did not have one change that stood out more than others so we have listed three interrelated changes below:

• One change we might consider is, after the introduction, taking a single research question and following it through the entire workshop, from identifying and acquiring texts, to evaluating methods and tools, to running the analysis. Along the way we could still introduce other topics, methods, and tools, but this deeper dive into a single question, running like a spine throughout the workshop, might provide a greater sense of continuity throughout the workshop.
• Another change we might consider would be to start off with a hands-on exercise closer to the beginning of the workshop and then move into the text mining background. The text mining background that we started with was important in terms of establishing context for the workshop but the first part of the workshop was mostly lecture until we got to create the workset in HTRC. Anther change we might consider making is to move the text scraping exercise to the first day of the workshop prior to doing the HTRC portions of the curriculum.  One comment from attendees was that moving from HTRC to the text scraping exercise with PythonAnywhere, and then back to HTRC later in the workshop is confusing for those who are novices in this area; however others liked the order of the workshop.
• Finally, providing a very brief introduction to scripting in Python—something that shows what’s going on under the hood of the scripts we are running—might be beneficial to the workshop participants.

Q. What tips would you give to somebody else teaching similar workshops?

A. Having all of the files necessary for the hands on portion of the workshop using PythonAnywhere was helpful as was having people register for PythonAnywhere accounts prior to the workshop. Before starting the portion of the workshop using PythonAnywhere it is well worth the time to take ten minutes and make sure everyone has the correct files and the PythonAnywhere account set up.

Q. How would you encourage a fellow librarian to play a more active role in supporting data-driven research?

A. Hands-on experience in workshop environments such as the HTRC workshops is a good way to become familiar with both the tools and concepts involved in data driven research. To be able to support this or any kind of research it is important that librarians have a basic understanding of the concepts behind the research and tools necessary for doing the actual research.  While one need not be an expert in all aspects of text mining to be able to provide support they will need to have a basic foundation of knowledge and skill.

This fall we are piloting the workshop curriculum at all five partner institutions. In this post, librarians Lijuan Xu and Sarah Morris share their thoughts after leading a DDRF workshop for their colleagues at Lafayette College. Feedback from all of the partners along with the assessment forms collected from attendees will inform development of our next iteration of workshop materials for spring 2017. You can read more about the modular DDRF curriculum in this update.

Q. Was there anything that worked particularly well in the workshop?

A. Our reorganization of the modules worked well. After a brief introduction, we started with HT+Bookworm, which is relatively straightforward, before discussing the Workset Builder and HTRC algorithms. After the lunch break, we spent the last hour of the three-hour workshop on Extracted Features and gathering and working with text. By first focusing on HTRC’s off-the-shelf tools and then moving on to textual analysis beyond HTDL and HTRC allowed the content to flow more fluidly and seemed to resonate with the participants. Overall, our colleagues were happy with the workshop: they felt better informed about HTDL and HTRC and more confident with textual analysis. They also thought the workshop provided “food for thought” and information for further conversations regarding textual analysis.

The 10 participants in our workshop came from all areas of library work, including research and instruction, digital scholarship and services, special collections and archives, and technical services. The mixture of librarians and staff with various backgrounds and technical expertise resulted in much richer discussion and feedback.

Q. Was there anything that surprised you during the workshop?

A. We were pleasantly surprised that people were able to run Python scripts without any problem. However, it was frustrating that at different moments, individuals experienced errors that we could not account for, for example, while some did not encounter any issues, others had problems logging into their accounts, retrieving their worksets, or viewing results.

Q. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

A. While we were happy with our reorganization of the workshop, we think that the modules could benefit from slides with less text and/or fewer slides. It would also be beneficial to incorporate more concrete examples of how researchers have used HTDL and HTRC tools for research and textual analysis.

Q. What tips would you give to somebody else teaching similar workshops?

A. Build in a long break between the first and second halves of the workshop. The hour-long lunch break we provided allowed participants to recharge and refocus.

Consider team-teaching the workshop. It is challenging to learn, digest, and teach materials that others have developed and that you are not an expert on. Pairing up with another librarian will make it easier. You can bounce ideas off each other and walk through the materials together.

Q. How would you encourage a fellow librarian to play a more active role in supporting data-driven research?

A. Start having conversations among librarians to learn what kind of knowledge and expertise people already have in terms of textual analysis and discuss how to support this type of research.  Learn from each other. Play around with the tools and think about how they could be applied to projects that faculty and students are currently and/or might be engaged in.

In order to make the workshop materials flexible and reusable, early-on in the development process the DDRF curriculum group decided to draft the content in the form of learning modules. The modules relate to competencies in digital scholarship, as well as tie into particular HTRC tools and services. All except the introductory module have corresponding hands-on activities to reinforce the learning objectives established by the curriculum group.

There are five modules in the current pilot curriculum:

1. Introduction to the HathiTrust and to text analysis
2. Gathering textual data
3. Managing and curating textual data
4. Performing text analysis
5. Visualizing textual data with HathiTrust+Bookworm

As we pilot the workshops this fall and gather assessment from attendees, we will have the opportunity to evaluate and modify the modules before we finalize the curriculum in the third year of the grant.

Through the summer, the DDRF curriculum working group has been working with their partners at Lafayette College, Northwestern University, and the University of North Carolina at Chapel Hill to adapt and refine the draft curriculum used in the spring 2016 pilot workshops for events at each of the project institutions this fall. While the workshops will remain fairly consistent from institution-to-institution, there will be slight modifications based on such factors as workshop length and anticipated audience. By teaching another round of pilot workshops this semester, we will be able to better gauge both how well-suited the teaching materials are for different groups of attendees and how easily they can be re-taught by other librarians.

We’ll ask each institution to report back on their experience teaching DDRF workshops this fall and publish their responses here!

Last week, the project team had the opportunity to pilot the curricular materials they have been developing at two workshops held at Indiana University and the University of Illinois on April 12th and 14th, respectively. The workshops were held in the universities’ libraries, and they were open to librarians and graduate students in library and information science. About a dozen people attended each workshop. We think they were a success overall, but we did learn a few lessons about computer lab policies (more restrictive than expected!) and pacing (don’t throw out too many tool names!). Over the next two months, we’ll be refining the curriculum to share with the partners at the University of North Carolina, Lafayette College, and Northwestern University.

The project’s curriculum working group has developed a preliminary draft of the learning materials to be used in the Digging Deeper, Reaching Further workshops. They shared it with partner institutions this week.

Much of the time in pulling together the preliminary content was devoted to planning, such as thinking about how to meet the needs of the librarian audience and outlining learning objectives. Over the last month, project staff also had the opportunity to speak with organizers of other library training initiatives, including the University of Rochester’s Digital Humanities Institute for Mid-Career Librarians and North Carolina State University’s Data and Visualization Institute for Librarians. These fruitful conversations with other organizers gave the curriculum group some great ideas that they are now integrating into their own plans.

Going forward, the curriculum group will use feedback from the partner institutions to finalize the first draft content for workshops to be piloted at the University of Illinois and Indiana University in mid-April.