This is a guest post from Camille Salas, the former Viewshare outreach librarian extraordinaire. Camille completed her internship and temporary assignment with the Library of Congress recently, we hope to again feature her outstanding work as a guest author on this blog once she’s landed a new gig. Best of luck, Camille!
Following is an interview with Merinda Hensley and Thomas Padilla of the University of Illinois Library about their project to create a visualization of conference tweets. Merinda Hensley is the Instructional Services Librarian and Co-Coordinator of the Scholarly Commons at the University of Illinois at Urbana-Champaign. Thomas Padilla is a Graduate Assistant in the Scholarly Commons. Thomas recently created a tutorial on how to use Viewshare and ScraperWiki to capture and display tweets from conferences. The tutorial was posted to the Scholarly Commons blog, Commons Knowledge.
Camille: Please tell us about the Scholarly Commons at the University of Illinois at Urbana Champaign (UIUC). What is its role in the UIUC community?
Merinda: The Scholarly Commons, a unit of the Universityof Illinois Library, opened in August 2010 to serve the emerging needs of faculty, researchers and graduate students pursuing in-depth research and scholarly inquiry. We consult with researchers on issues related to data-intensive work, digital humanities, answer copyright questions, help with the digitization of materials, have workstations to support web and computer usability, and administer a series of open workshops on myriad topics geared towards the research needs of faculty and graduate students. We also partner with several campus constituencies in order to bring together services across campus. Our space offers software and hardware to conduct research including completing tasks such as text-encoding, working with qualitative and quantitative data, and digitization.
Camille: How do you each contribute to the work at the Commons?
Merinda: As Co-Coordinator of the Scholarly Commons, I work to support new and developing services and events related to the advanced research needs of graduate students and faculty. I coordinate our workshop series, the Savvy Researcher, and supervise the pre-professional graduate assistants from the Graduate School of Library and Information Science. Thomas has been an excellent addition to our team with his interests in technology and the humanities.
Thomas: As a Graduate Assistant at the Scholarly Commons, I work with a team of librarians and graduate assistants from the Graduate School of Library and Information Science to provide consultation services to patrons, monitor and learn new tools and resources, market Scholarly Commons services, teach workshops, and contribute to digital humanities projects.
Camille: Thomas created a short tutorial on using ScraperWiki and Viewshare to capture and display conference Twitterstreams for the Scholarly Commons blog. For those unfamiliar with ScraperWiki, please describe what it is.
Thomas: ScraperWiki is a web-based data scraping platform that helps novice and advanced users “scrape” publicly available data from websites like Twitter and Flickr. Novice users can use pre-made “tools” to scrape data and more advanced users can create their own data scrapers.
Camille: Other Viewshare users have described applications they used with Viewshare in previous blog posts. What led you to use ScraperWiki with Viewshare? What is the goal of using them together?
Thomas: Sharing and conversing about tools, resources, methods, and issues like professional ethics are often prominent themes in the conference Twitterstream. While some tools exist to capture this information, I was looking for a method that would capture, display, and make the data underlying the visualizations shareable. I chose ScraperWiki to capture the Twitter data because it offers an accessible method for a novice user to scrape Twitter data – I wanted individuals interested in trying this to face as low of an initial barrier of entry as possible.
I chose Viewshare because it offers a user-friendly interface, it enables spatial and temporal visualizations, it is maintained by a public institution, and it makes the data underlying visualizations shareable in multiple formats, thus affording the possibility for others to interpret, combine, and remix the data as they see fit.
Camille: Briefly walk us through how you would use the two tools together.
Thomas: I use a pre-made ScraperWiki “tool” to scrape all Twitter data associated with a hashtag. After the data is scraped, it is downloaded in Microsoft Excel format. The next step is to clean the data and remove fields that contain data that is not well represented in the overall dataset.
Once the data is ready, the files are uploaded to Viewshare, data types are assigned to the various fields (columns from the spreadsheet), views are built, and widgets are assigned to add search and filtering functions.
Camille: What do you hope conference attendees and even non-attendees might glean from visualizing tweets? Similarly, how do you see projects like this assisting the work of researchers and scholars?
Thomas: By visualizing tweets, conference attendees and non-attendees are provided multiple ways of orienting themselves to topics that arise during an event and to the individuals who are contributing to those topics. The timeline visualization can help users to understand the frequency of tweets as the conference progresses.
This value can be enhanced by filtering the timeline according to factors like the language the tweet occurred in or by hashtag to focus on specific topics, tools, or resources. The map view can give users a sense of local and external participation in the discussion of a topic. Simple pie charts can be used to quickly understand which users tweeted the most, or at an international conference, what languages were used and what proportion of the overall conversation they represent.
While often maligned, the word cloud widget becomes useful in this context because words in the cloud filter the visualizations. Outliers in the word cloud are quickly apparent and the path toward finding the outlier tweets is short. These are just a few of examples of what is possible when using Viewshare to visualize Twitter data gathered by ScraperWiki.
As previous blog posts about Viewshare have shown (Bill Amberg, Jeremy Myntti, Violeta Ilik) the tool can support projects in many different domains and professional settings. While it can handle data in a few different formats, I think it has immediate appeal for a large group of researchers and scholars in its ability to visualize data held in the Microsoft Excel file format.
This is not an endorsement of the file format, rather it is just an acknowledgement that use of the file format is fairly ubiquitous across disciplines, and that lends itself well to a common starting point for many different scholars to use Viewshare to iterate through visualizations of their data, hopefully gaining useful perspective along the way.
Camille: What other visualizations or capabilities could Viewshare offer to enhance your project or future projects at the Scholarly Commons?
Thomas: I agree with Jeremy Myntti that it would be useful to be able to edit metadata within Viewshare rather than exporting, refining, and re-uploading. I think this feature would help users iterate through visualizations in a more streamlined way.
Camille: Have you received any feedback about using these tools together yet? Aside from the Common Knowledge blog post, how do you plan on sharing this tutorial with faculty and students at UIUC?
Thomas: So far we have not received much feedback on using the tools together, though we would definitely welcome it! My post on using Viewshare and ScraperWiki is the first in a three part series. The next post will feature Martin Hawksey’s Twitter Archiving Google Spreadsheet (TAGS) and the post after that will focus on the various insights that can be gained from using either tool. Aside from the blog, I will teach two Viewshare focused workshops this semester. While I will not be talking about combining Viewshare and ScraperWiki in those workshops, I will be focusing on how Viewshare can work with data from different disciplines.
Camille: Merinda, in your role supervising graduate assistants from the GSLIS, what types of skills do you find increasingly important for new librarians to have — especially for those who want to work in library units such as the Scholarly Commons? The tutorial seems like a good example of what new librarians can offer the field.
Merinda: Librarians new to the field can improve their chances of working in an environment similar to the Scholarly Commons by keeping up to date on how technology intersects with research in all disciplines. The Chronicle of Higher Education and Inside Higher Ed are both great places to watch for developments in all fields but getting to know the researchers in your field and the struggles they face is often the best way to begin thinking about how librarians can participate in the solution. We have to be flexible and willing to learn from our mistakes along the way as we adapt to new research strategies. Thomas’ work with Viewshare is an excellent example of exploring options for researchers who are increasingly depending on technology to support their research.