Collecting Case Study Evidence

There are six major sources of evidence discussed in this chapter of Yin (2009). 

Documentation. Such as letters, emails, diaries, agendas, meeting minutes, proposals, progress reports, formal studies or evaluations of the same case and news clippings or other relevant media articles. Documents cannot be accepted as literal recordings of events but their value is in corroborating and augmenting  evidence from other sources. Inferences can be made but must be backed up by observed data, remembering that documents are written for a specific audience and purpose.

Archival records.  Such as census or statistical data, service records, budgets or personnel records, maps and charts, survey data about a particular group. Again these would have been produced for a specific purpose and audience so this must be taken into account when determining the usefulness of such data.

Interviews. These should take the form of guided conversations rather than structured queries. Yin stresses the importance of not asking interviewees the 'why' questions of your study directly but asking 'how'. This has a much softer, less accusatory tone. There are different types of interview, the in-depth and the focused interview. The former asks participants about the facts of the matter as well as their opinion of events. This obviously needs to be corroborated by other sources and also it is wise to search of contrary evidence to strengthen these findings. In the focused interview, you follow a certain set of questions derived from the case study protocol. Care must be taken not to ask leading questions. A third type of interview consists of a more structured questions along the lines of a survey. For all types of interview though, care must be taken with the fact that responses are subject to problems of bias, poor recall and poor or inaccurate articulation. As such, all interview data should be corroborated with other sources.

Direct Observation. This provides opportunities for gathering additional data about the topic under study. multiple observers can increase the reliability of this method though not always a practical option.

Participant-observation. Similar to the previous method, but with the advantage of a more in-depth understanding of the case due to taking on a role within the scenario. There are problems associated with this method however and these are related to potential biases that may occur in getting too involved. There is also the situation where the time to participate leaves little time to observe.

Physical artifact. These can be collected or observed and could include such items as a tool, a work of art, a printout of computer statistics such as usage etc. They do have less potential relevance in most typical case studies but can be used as supplementary evidence sources.

During collection of data from any of these six methods, there are three principles which can maximise the benefits of the data collection phase by dealing with the problem of construct validity and reliability of the case study evidence.

1. Use multiple sources of evidence. This has the advantage of using converging lines of enquiry, a process of triangulation and corroboration. Construct validity is addressed since 'multiple sources of evidence provide multiple measures of the same phenomenon'.(p117)

2. Create a case study database. This concerns how we organise and document the data collected and if done well (systematically) can increase the reliability of the case study. The aim is to allow independent investigation of the data should anyone want to query the findings or final case study report. There are four possible components to a database. 

• case study notes - these may come from interviews, observations, document analysis etc but they must be organised in such a way that they can be easily accessed later on by external parties (and yourself).
• case study documents - this can be weildly  but annotated bibliography of such documents can be useful. another method is to cross-reference documents in interview notes (say). As previously these must be organised for easy access.
• tabular materials - this could be a collection of quantitative results such as survey, observational counts or archival data.
• narratives - this is the case study investigators open-ended answers to the case study protocol's questions. Each answer represents an attempt to integrate the available evidence and to converge upon the facts of the matter or the tentative interpretation.

3. Maintain a chain of evidence. This will allow an external observer to follow the derivation of any evidence from the initial research questions to ultimate case study conclusions and thus increase the reliability of the case study. The chain runs from questions to protocol to evidence to database to final report, each linked forward and backwards (see p123 for a diagram).