Project Updates

1. Was there anything that worked particularly well in the workshop?

We incorporated more discussion questions into this iteration of the workshops and that seemed to go well. It gave participants the chance to talk and reflect, which was a nice break from looking at a screen. The questions could use further refinement, but I was pleased with how the conversations unfolded.

2. Was there anything that you felt was tricky to teach?

We were pressed for time toward the end. When we got to the more technical pieces in the latter-half of the workshop, it felt like we shifted gears too quickly and moved from material participants were easily grasping to that which they needed more time to work through. Unfortunately, we had not allocated enough time to let them dive into the more challenging material. Additionally, there was a wide range of familiarity with programming in the room, which made it especially difficult to target the content appropriately as we rushed through it!

3. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

In this iteration of the workshop materials, we tested out some content related to data management and learned that we had gone too far off-track with those materials. The nice part of iterating through the content is that we are able to test things out and then remove them when we find they didn’t work as well as we had hoped!

4. What changes would make the workshops more accommodating to a diverse range of participants with different roles in the library and varying technology skillsets?

We continue to have an audience with diverse skillsets attending these workshops at our library. Improved opportunities for either paired workshop activities or adding “challenge” activities for people who finish quickly would accommodate our audience better. Hopefully that would make the time we provide to work through the hands-on material better adapted to a range of backgrounds and skills.

1. Was there anything that worked particularly well in the workshop?

We added an introductory slide to the workshop content and framed the workshop as an opportunity for participants to develop some very basic text analysis skills as well as for us to improve the workshop materials. We took out some slides so that the workshop was only three hours (with an hour in between for lunch). The three-hour length seemed perfect to keep people’s attention. At the end of the workshop, we asked people to fill out the assessment survey but also verbally tell us whether the workshop met their expectations and how it could be improved. People were very honest and gave us all kinds of interesting feedback. The discussion went on for 30 minutes, much longer than we expected.

2. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

Include more research and assignment examples, especially those having to do with HTDL content. Participants found the textual analysis examples in Module 1 the most interesting and useful to help them think about how they practice text analysis on their own and/or talk to faculty members about text analysis.

3. What changes would make the workshops more accommodating to a diverse range of participants with different roles in the library and varying technology skillsets?

The challenge we encountered and we think will be very important to keep in mind as we move forward is the institutions that participants are from. Most of our participants in this workshop came from small institutions that are teaching and not research-centered. Having examples of text analysis classroom assignments in addition to research projects would be really beneficial.

4. After several rounds of workshops, do you have any thoughts or comments on our iterative workshop model?

Having people from different institutions teach the same materials to participants with varying text analysis experience and revising the materials based on the feedback definitely help improve the workshop content. Teaching it a couple of times also helps us have a much better understanding and grasp of the materials.

1. Was there anything that worked particularly well in the workshop? 
   This was our third time offering the DDRF workshop at Indiana University Bloomington and I think it was our most successful. This time around we partnered with a faculty member in the School of Information and Library Science and invited students from their Digital Humanities course. We had a great group of curious students (and a few attendees from the library) and the instructor, John Walsh, helped facilitate meaningful discussions throughout the workshop. Compared to past workshops, this round was much more participatory and the attendees seemed more engaged.
2. Was there anything that you felt was tricky to teach? 
   Running algorithms in the portal was a little tricky, mostly because the entire group wanted to see their results all at once! Luckily we have the example prepped and ready to go before the workshop, but sometimes participants are disappointed that they are unable to view their results immediately [because of how the activities are structured].
3. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be? 
   I think we’ve made significant progress in improving our hands-on activities, but I’d still like to see more room for creativity. It’s challenging to leave room for exploration when there’s so much material to cover, but I would like to see a little more tinkering and less step-by-step activities.
4. What changes would make the workshops more accommodating to a diverse range of participants with different roles in the library and varying technology skillsets? 
   I feel confident that the materials are appropriate for a wide audience, but the issue we struggle with is marketing. There is a key group of librarians and library staff members at IU who are already interested in text analysis, so naturally they want to learn more about the HTRC, but it’s tough to reach out to other folks in the library and explain why they should participate.

This fall we are piloting the workshop curriculum at all five partner institutions. In this post, librarian Eleanor Dickson shares her thoughts after leading a DDRF workshop at the University of Illinois at Urbana-Champaign. Feedback from all of the partners along with the assessment forms collected from attendees will inform development of our next iteration of workshop materials for spring 2017. You can read more about the modular DDRF curriculum in this update.

Q. Was there anything that worked particularly well in the workshop?

A. This was our second time leading at DDRF workshop at Illinois, and our session this time was an hour longer than our first pilot workshop in spring 2016. It worked better to have three hours to teach as opposed to two hours. We felt much less rushed. If we were to teach a two hour DDRF workshop again, I would be more selective about what was included and what was left out. This semester we also piloted the “Visualizing Text Data” module activity using the HathiTrust+Bookworm (HT+BW) tool. It was interesting to see how the librarians in the audience approached HT+BW, and it seemed they benefited from the opportunity to bring their own research question to the tool.

Q. Was there anything that surprised you during the workshop?

A. Both times we have taught the workshop, I have been surprised by the range of experience levels with various technical topics among attendees. Experience level does not easily break down along lines of job titles or years in the field. The range of base-line knowledge in the room during the workshop can make it challenging to pace the activities appropriately, but we have also seen that some attendees with greater experience have been willing to help their less experienced neighbors through activities.

Q. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

A. I would continue to improve our hands-on activities. I feel that they have gotten better from our first iteration of the workshop, but I’d still like to see us expand and modify them as we go forward. I would like to make them more interactive, with more time for discussion and reflection. I would also like to find ways to draw on attendees’ past experience with digital scholarship projects and reference questions. This change might ground and contextualize the research examples we provide during the introduction.

Q. What tips would you give to somebody else teaching similar workshops?

A. Be prepared: run through examples and activities ahead of time and try to anticipate where attendees might have difficulty. For example, this semester we provided all the Python commands to attendees in a text file. The file rendered well on our Mac laptops, but displayed poorly on the Windows computers in the lab we used. That said, something always seems to go wrong during technical workshops. These issues may be unavoidable, and often are neither your fault nor the fault of the attendees. Try to help attendees understand when their errors are things they can fix (such as entering a command properly) and when they are largely out of their control (such as software installing in an unexpected way).

Q. How would you encourage a fellow librarian to play a more active role in supporting data-driven research?

A. I would encourage them to look for opportunities to play with various tools and methods for their own inquiries. Hands-on experience is key when learning technical topics, and by applying their experimentation to a question of interest to them, the skills are more likely to stick.

This fall we are piloting the workshop curriculum at all five partner institutions. In this post, librarian Amanda Henley shares her thoughts after leading a DDRF workshop at the University of North Carolina at Chapel Hill. Feedback from all of the partners along with the assessment forms collected from attendees will inform development of our next iteration of workshop materials for spring 2017. You can read more about the modular DDRF curriculum in this update.

Q. Was there anything that worked particularly well in the workshop?

A. Our participants were very interested and engaged. They had a lot of questions and expressed interest in learning more. We had two presenters, which also worked well. Jacob Hill is a 5^th year PhD student in the School of Information and Library Science. It was great working with him, and he did an excellent job presenting the material. We also had two floaters that were available to assist with questions as we went through the exercises. Finally, we held the workshop in one of our computer labs, so we were able to load the software and save data locally beforehand to help things go smoothly. It was helpful for everyone to have a uniform computing environment.

Q. Was there anything that surprised you during the workshop?

A. I was surprised at how many UNC Librarians are interested in learning more about text analysis! Our registration filled up almost immediately, and several staff members sent follow up emails asking if it will be taught again. During the workshop, I was surprised that the modules took longer than we expected.  We ran out of time and I had to teach the last module very quickly. Additionally, I didn’t anticipate how difficult it would be to see the command line for the participants.  We used a standard projector and needed to remember to increase the font size for each of the exercises.

Q. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

A. I would like to make the workshop more interactive. Librarians are experts at learning from one another, and we could capitalize on everyone’s expertise if we made the workshop a bit more interactive. It might be interesting to provide participants an assignment in advance: provide a list of text analysis research papers, blog posts, etc. from a variety of disciplines and have the participants choose one, answer a set of questions, and share their findings at the beginning of the workshop. That would make the content more relevant to the participants and it would make it easier on the presenter.

Q. What tips would you give to somebody else teaching similar workshops?

A. If possible, install the software in advance and save the data locally. Remind your participants about the accounts they will need the day before the workshop. Use Ctrl + to increase the font size on the screen.  It is very difficult to see the command line from the back of the room!

Q. How would you encourage a fellow librarian to play a more active role in supporting data-driven research?

A. Play around with the tools and apply digital methods to your own research. If you need incentive, schedule to teach a workshop on something you want to learn. If you can, sit in on consultations given by your peers. Here in the Davis Library Research Hub we are happy to have our colleagues from other departments sit in on our data-intensive consultations. We find that when they do, we learn just as much from them as they learn from us.

This fall we are piloting the workshop curriculum at all five partner institutions. In this post, librarian Geoffrey Morse shares his thoughts after leading his first DDRF workshop at Northwestern University. Feedback from all of the partners along with the assessment forms collected from attendees will inform development of our next iteration of workshop materials for spring 2017. You can read more about the modular DDRF curriculum in this update.

Q. Was there anything that worked particularly well in the workshop?

A. The workshop attendees seemed to follow along and replicate the results in the HTRC portions of the workshop without much difficulty. All of the attendees were able to create accounts, create work sets, and run algorithms.  While all had been familiar with the HathiTrust Digital Library, not all were aware of the HTRC prior to the workshop.

Q. Was there anything that surprised you during the workshop?

A. The range of participants in the workshop was broader than might have initially been expected. Librarians and library staff from all areas of the library were interested in learning more about this topic. Not only did library liaisons to academic departments attend, technical services staff including catalogers and staff from acquisitions also attended as did staff members from Repositories and Digital Curation.

Q. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

A. We did not have one change that stood out more than others so we have listed three interrelated changes below:

• One change we might consider is, after the introduction, taking a single research question and following it through the entire workshop, from identifying and acquiring texts, to evaluating methods and tools, to running the analysis. Along the way we could still introduce other topics, methods, and tools, but this deeper dive into a single question, running like a spine throughout the workshop, might provide a greater sense of continuity throughout the workshop.
• Another change we might consider would be to start off with a hands-on exercise closer to the beginning of the workshop and then move into the text mining background. The text mining background that we started with was important in terms of establishing context for the workshop but the first part of the workshop was mostly lecture until we got to create the workset in HTRC. Anther change we might consider making is to move the text scraping exercise to the first day of the workshop prior to doing the HTRC portions of the curriculum.  One comment from attendees was that moving from HTRC to the text scraping exercise with PythonAnywhere, and then back to HTRC later in the workshop is confusing for those who are novices in this area; however others liked the order of the workshop.
• Finally, providing a very brief introduction to scripting in Python—something that shows what’s going on under the hood of the scripts we are running—might be beneficial to the workshop participants.

Q. What tips would you give to somebody else teaching similar workshops?

A. Having all of the files necessary for the hands on portion of the workshop using PythonAnywhere was helpful as was having people register for PythonAnywhere accounts prior to the workshop. Before starting the portion of the workshop using PythonAnywhere it is well worth the time to take ten minutes and make sure everyone has the correct files and the PythonAnywhere account set up.

Q. How would you encourage a fellow librarian to play a more active role in supporting data-driven research?

A. Hands-on experience in workshop environments such as the HTRC workshops is a good way to become familiar with both the tools and concepts involved in data driven research. To be able to support this or any kind of research it is important that librarians have a basic understanding of the concepts behind the research and tools necessary for doing the actual research.  While one need not be an expert in all aspects of text mining to be able to provide support they will need to have a basic foundation of knowledge and skill.

This fall we are piloting the workshop curriculum at all five partner institutions. In this post, librarian Leanne Nay shares her thoughts after leading a second DDRF workshop at Indiana University Bloomington. Feedback from all of the partners along with the assessment forms collected from attendees will inform development of our next iteration of workshop materials for spring 2017. You can read more about the modular DDRF curriculum in this update.

Q. Was there anything that worked particularly well in the workshop?
A. We were lucky to have three presenters for our workshop. My colleague Nicholae Cline is very articulate and did a fantastic job of presenting some of the background information and history of the HTRC. I have a lot of experience with hands-on technology workshops, so I enjoyed leading the activities. We were also joined by Ewa Zegler-Poleska, our IDEASc fellow, who offered a fresh perspective on the Bookworm tool and also added her own expertise in discussing data management. Our three voices are very different, but complementary. I think it added a nice variety and helped to keep our participants engaged.

Q. Was there anything that surprised you during the workshop?
A. Our attendees were much different than we expected and attendance was fairly low. When we held a pilot workshop earlier this year the attendees were mostly librarians or library staff with little to no previous experience with text analysis. This time around it was a mixed bag and all the participants had already worked on their own DH projects. In the future we plan to reach out to students in our library science program and potentially lead the workshop during a class session. We hope this will get us closer to our target audience of librarians (and future librarians) who are curious about text analysis but not sure how to get started.

Q. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?
A. We’d like to incorporate more open-ended activities that challenge our participants intellectually. The current activities are great for exposing people to the steps involved in text analysis and introducing new tools, but they don’t leave much room for creativity.

Q. What tips would you give to somebody else teaching similar workshops?
A. The best advice I can give is to run through all of the materials beforehand, slide-by-slide, in the space where you will be offering your workshop. At Indiana we piloted a new activity using OpenRefine. We installed the software beforehand and it looked like everything was all set, but when I tested the activity in our computer lab the day before our workshop I realized that we didn’t have administrative privileges to save files. The work-around was easy enough, but I wouldn’t have been able to troubleshoot this in a front of a live audience! If you’re working in a campus computer lab you’re likely to run into issues with downloading software, so it’s best to test this out before the workshop.

Q. How would you encourage a fellow librarian to play a more active role in supporting data-driven research?
A. I think the best way to start is by attending workshops or lectures like these. At Indiana University we have a fantastic lecture series called the Digital Library Brown Bags, where librarians and scholars share digital projects that they’re engaged in. It’s inspiring to hear what our colleagues are working on and it helps to add some context to specific tools that you may have heard of but can’t quite wrap your head around.

This fall we are piloting the workshop curriculum at all five partner institutions. In this post, librarians Lijuan Xu and Sarah Morris share their thoughts after leading a DDRF workshop for their colleagues at Lafayette College. Feedback from all of the partners along with the assessment forms collected from attendees will inform development of our next iteration of workshop materials for spring 2017. You can read more about the modular DDRF curriculum in this update.

Q. Was there anything that worked particularly well in the workshop?

A. Our reorganization of the modules worked well. After a brief introduction, we started with HT+Bookworm, which is relatively straightforward, before discussing the Workset Builder and HTRC algorithms. After the lunch break, we spent the last hour of the three-hour workshop on Extracted Features and gathering and working with text. By first focusing on HTRC’s off-the-shelf tools and then moving on to textual analysis beyond HTDL and HTRC allowed the content to flow more fluidly and seemed to resonate with the participants. Overall, our colleagues were happy with the workshop: they felt better informed about HTDL and HTRC and more confident with textual analysis. They also thought the workshop provided “food for thought” and information for further conversations regarding textual analysis.

The 10 participants in our workshop came from all areas of library work, including research and instruction, digital scholarship and services, special collections and archives, and technical services. The mixture of librarians and staff with various backgrounds and technical expertise resulted in much richer discussion and feedback.

Q. Was there anything that surprised you during the workshop?

A. We were pleasantly surprised that people were able to run Python scripts without any problem. However, it was frustrating that at different moments, individuals experienced errors that we could not account for, for example, while some did not encounter any issues, others had problems logging into their accounts, retrieving their worksets, or viewing results.

Q. If you could change one thing that you did in the workshop to make the sessions more effective, what would it be?

A. While we were happy with our reorganization of the workshop, we think that the modules could benefit from slides with less text and/or fewer slides. It would also be beneficial to incorporate more concrete examples of how researchers have used HTDL and HTRC tools for research and textual analysis.

Q. What tips would you give to somebody else teaching similar workshops?

A. Build in a long break between the first and second halves of the workshop. The hour-long lunch break we provided allowed participants to recharge and refocus.

Consider team-teaching the workshop. It is challenging to learn, digest, and teach materials that others have developed and that you are not an expert on. Pairing up with another librarian will make it easier. You can bounce ideas off each other and walk through the materials together.

Q. How would you encourage a fellow librarian to play a more active role in supporting data-driven research?

A. Start having conversations among librarians to learn what kind of knowledge and expertise people already have in terms of textual analysis and discuss how to support this type of research.  Learn from each other. Play around with the tools and think about how they could be applied to projects that faculty and students are currently and/or might be engaged in.

In order to make the workshop materials flexible and reusable, early-on in the development process the DDRF curriculum group decided to draft the content in the form of learning modules. The modules relate to competencies in digital scholarship, as well as tie into particular HTRC tools and services. All except the introductory module have corresponding hands-on activities to reinforce the learning objectives established by the curriculum group.

There are five modules in the current pilot curriculum:

1. Introduction to the HathiTrust and to text analysis
2. Gathering textual data
3. Managing and curating textual data
4. Performing text analysis
5. Visualizing textual data with HathiTrust+Bookworm

As we pilot the workshops this fall and gather assessment from attendees, we will have the opportunity to evaluate and modify the modules before we finalize the curriculum in the third year of the grant.

Through the summer, the DDRF curriculum working group has been working with their partners at Lafayette College, Northwestern University, and the University of North Carolina at Chapel Hill to adapt and refine the draft curriculum used in the spring 2016 pilot workshops for events at each of the project institutions this fall. While the workshops will remain fairly consistent from institution-to-institution, there will be slight modifications based on such factors as workshop length and anticipated audience. By teaching another round of pilot workshops this semester, we will be able to better gauge both how well-suited the teaching materials are for different groups of attendees and how easily they can be re-taught by other librarians.

We’ll ask each institution to report back on their experience teaching DDRF workshops this fall and publish their responses here!