Research Design and Research Logistics - Case Study Example

Module Case Research Design and Research Logistics What is the research design used by Dr. Eveland Research design can be ified into three broad categories: experimental research, field research, and observational research. The amount of control a researcher maintains over the conduct of research study determines the category of research design used. A study on 'work group structures and computer support,' by Eveland and Bikson (1988) employed 'field experiment' as their research design methodology. The research design supported: a) random assignment of subjects to predetermined experimental conditions b) control over influencing variables (such as, type of task, type of technology, prior experience with electronic mail, preexisting group structures). c) analyze and interpret performance between two categories of task forces using defined dependent variables From a dilemmatic perspective (McGrath, 1981), the research study by Eveland, presents a set of dilemmas than problems, interlocked with choices to stay with; for instance, the selection of work groups, sample sizes, near-retirees versus young employees, impact of computer support systems or otherwise, to name a few. And within the time and resource constraints for study, maximizing objectives from conflicting desiderata articulated by McGrath seems to be in perfect connivance. And, whether the set of choices made by Eveland was a best strategy is an interesting facet. In the words of Eveland (1988), "we hoped this experiment would illuminate, although, not all of them have been answered" (p. 375). This reaffirms the notion propagated by McGrath, that "the set of choices is systemically circular; it starts with a problem, and gets back to the problem." The cycle of empirical research stands vindicated (Runkel and McGrath, 1972). Eveland's approach in the study under reference seems to follow the methodical strategy, which are generic classes of research settings for gaining knowledge about a research problem. In this context, the problem being, "the impact of computer-supported systems on the work group structures." The research method adopted herein maps into slices I (field experiments and field study, with sample surveys) of the strategy circumplex (Runkel and McGrath, 1972). The point of maximum concern is on the system context and generalizability over the actors. In terms of maximizing generalizability context, Eveland's study used a relatively small sample and inclusion of younger employees was perhaps desirable. In terms of Cronbach (1957) classification of designs, Eveland followed both the comparison and correlation techniques. The study compared the performance between conventional and computer supported work groups; and so also, analysis of variance analysis as in trend analysis of member contact within work group over the study period. The power point slides on research methods ("Research Methods in I/O Psychology", 2008) concisely enumerates the research process, which has also been adopted by Eveland, viz.: ask questions; generate hypothesis; design a study; collect data; data analysis; and interpret-present results. Since the Eveland study used random assignment techniques to select group members, in the terminology of Trochim (2007) the approach could be classified as 'randomized experiment' which helps in deducing casual relationships. 2. After listing and distinguishing a variety of research designs, explain the advantages and limitations of each; why did Dr. Eveland choose the research design he chose Research design provides a formal structure to a research study. A three-fold classification is often used to define a research design: randomized or true experiment, quasi experiment, and non-experiment (Trochim, 2007). A classification of this type helps in describing the internal validity of design. The distinguishing feature of a research design is the aspect of random assignment. When a random assignment is used in design, it leads to a randomized or true experiment scenario. Such a design is useful in studying the cause-effect relationship. A robust experimental design would seek to answer the cause-effect propositions: If a program is administered, then the outcome occurs. If a program is not administered, then the outcome does not occur. A true experimental design is versatile, controllable, has higher internal validity, often costlier, and at times, the interventions used may border around being labeled as unethical. When a random assignment is not used in design, then the distinguishing question to ask is: whether a multiple group, or multiple waves, in measurement is used If affirmative, it leads to a quasi-experimental design, else, it is a non-experimental design. Non-experimental design is often used to study the effects of a natural disaster on survivors, where, a one-shot measurement, generally via interviews, surveys, observation, or case studies. Non-experimental design attempts to relate on variable to another, is quick to administer, but cannot be experimentally manipulated or controlled. Quasi-experimental design is experimentally controllable. It is extensively used in social sciences, where pre-selection and randomization of groups is difficult. The results of such a design can however show general trends. Since there is no randomization in assignment, statistical test are meaningless. Eveland's study used the classical field experiment. The procedure allowed the researchers to randomly assign group members to computer-based versus traditional support systems. The field experiment design facilitated creation of identical work goals, and control the introduction of new information and communications technology. Choosing an experimental design for study enabled the researchers to:- allow interventions for a year to obtain meaningful results with regard to work group restructuring allow introduction of a strong mission focus and make the group activity and communication motivating allow randomly assigned group members to be selected from a common community, sharing similar culture and concerns. 3. What are the main phases of a research project and their normal sequence The main phases of a research project executed in it normal sequence is: Idea-generating phase Literature survey Problem-definition phase Formulate the research question, define the hypothesis Procedures-design phase Research design Definition and type of research design Identify internal and external validity issues Re-phrase research questions, if necessary Redefine hypothesis if necessary Sample design Definition of the Population Definition of Subjects/Samples/Groups Definition of Inclusion/Exclusion Criteria Definition of Selection Procedure Definition of treatments or programs Research methodology Definition of methods and procedures Definition of data collection instruments Reliability issues Limitations Select and validate appropriate research design and methods Human protection and ethical considerations Potential ethical issues and requisite approvals Preparation of informed consent form Planning and Funding Time Line Proposed Budget Application to Research Funding (if necessary) Observation phase Observation and measures Data-analysis phase Appropriate descriptive and inferential statistics Interpretation phase Communication phase Draft reports and reviews Final report 4. What is the relationship between the research question and research design The centrality of the research question in the entire research process is outlined by Rees (1997, p.8) who suggests that "research consists of extending knowledge and understanding through a carefully structured systematic process of collecting information which answers a specific question in a way that is as objective and accurate as possible." In other words, a research question addresses what a researcher wants to know. The overall purpose of research is to find an answer to the research question. Research design provides the rigor and framework to achieve this central objective effectively. It is extremely important that researchers select the most appropriate design for their study. The research design should elucidate the overall plan of how the researcher intends to implement the project in practice. In Eveland's study, the central research questions are (Bikson & Eveland, 1992):- When work groups get access to computer-based media for handling information and communication tasks, do their structures change Do they move closer to or further from formally established organizational structures Do group positions (e.g., leader roles, assistant roles) stay the same or change Do computer-supported groups overcome physical barriers to interaction (e.g., space or time constraints) Do they overcome persisting social barriers (e.g., status differences) Do they form tight clusters ("electronic islands") or are they overlapping and not sharply defined ("loose bundles") How if at all do networked information technologies affect the amount or density of interaction in a group How do they affect extent of members' integration within a group Or centralization across a group How do these new technologies affect social communication among group members How do they affect group members' perceptions and evaluations of the work process Do these media supplant or supplement other means for exchanging information and coordinating group tasks. These research questions have been addressed through a robust research design - the classic field experiment. In the research framework described by Eveland, work groups are recognized and classified on the basis of what they do- their mission or purpose within the broader organizations to which they belong. Also, behavioral focus is on groups, rather than individuals, which leads to a technical focus, not on highly specific electronic tools, but on the broader interactive environment of which the tools are a part. 5. What are the tradeoffs inherent in any design of a field experiment, particularly between site, sample, and experimental procedures Drawing inferences from the validity network schema (McGrath & Brinberg, 1983), it is clear all research studies have inherent limitations. Sampling invariably plays a crucial and complex role in external validity. In Eveland's study, the taskforce comprised recently retired employees, and those on the verge of retirement. The population amongst each subset was assumed to be homogenous, belonging to similar patterns of life, and culture. If such a population of retirees and to-retire, were to be drawn from culturally diverse background, the results of study could have been different. However, from the population, and subsets, selected in the study, random assignment was followed in allocating workforce members to different technology conditions - those with computer support, and otherwise. The population used in the Eveland's study is merely representative and small, and does not reflect the entire workforce. The socio-technical response and workforce dynamics could have been much different, if younger, and entry-level employees (digital natives), were also part of the population. In similar vein, employees representing different industries: manufacturing, hospitality, banking etc. could have produced different behavioral repose to workforce structures. To this extent, sampling and site have enveloped the findings of Eveland's study with limitations. From the standpoint of experimental procedures, four rounds of questionnaires were used predominantly to establish cohesiveness and communication in workforce structure. Photo-identification was heavily relied upon. Since the population size was small, maximum 20 in a subset, the recall of identity through photos may have been fairly simpler. What if, the subset sizes were in hundreds Then could similar methods and inferences help Also more contacts and communication may not necessarily be indicative of stronger productive interactions leading to improved bottom-line and enhanced work climate in an organization. Capturing mail exchanges through logging systems presents an ethical dilemma, where privacy of an individual is being intruded upon. Therefore, internal validity of study findings need not stand good when extrapolated to a larger population, and generalization. Definitely, the conclusions drawn in the study with regard to leveraging computer supported systems in workforce participation will hold in a broader perspective, connecting anytime, anyplace will reflect in communication dynamics, the effect of these may vary across industry and cultural compositions of the population. 6. Internal validity: Were the participants randomly assigned to the experimental or controlled condition In as much as assigning members to computer support or conventional systems is concerned, random assignment was followed in Eveland's study. This makes the work group attributes roughly equivalent. Therefore, any effect observed between work-groups can be linked to the treatment effect, and is not a characteristic of the individuals in the group. This has been the basis of experimental design in Eveland's study: If a certain work group structure changed, then it must be because of different office support conditions - that with computers, and without. 7. What was the major human subjects' protection issues arising in Dr. Eveland's project, including how they might have been addressed by researchers Six key ethical principles are recommended to be addressed, whenever applicable in behavioral research ("Research Ethical Framework", 2005): Research should be designed, reviewed and undertaken to ensure integrity and quality Research staff and subjects must be informed fully about the purpose, methods and intended possible uses of the research, what their participation in the research entails and what risks, if any, are involved. Confidentiality of information supplied by research subjects and the anonymity of respondents must be respected Research participants must participate in a voluntary way, free from any coercion Harm to research participants must be avoided Independence of research must be clear, and any conflicts of interest or partiality must be explicit Eveland's study captured a log of the headers of all network messages exchanged among electronic task force participants over the project year. This log included the sender's ID, the receiver's ID, the message date and time, and, if the message was a reply, the date and time of the original message. Topic identifiers were not retained for confidentiality reasons. The researchers had obtained advance consent to message header logging was obtained prior to the start of the project. (Eveland & Bikson, 1988, p. 369, footer note 1) The common rule requirements ("Human Subjects Research, GAO", 2001) for protecting human subjects include: obtaining a formal written assurance of compliance with federal standards from institutions engaged in research studies setting up criteria for review panels - known as institutional review boards (IRD) obtaining informed written consent from the participants 8. What modifications could have been made to the logistics of the project to improve its quality Research logistics play a crucial role when seeking funds for a research study. Apart from justifying the utility of research theme in a larger context of human prosperity, economic productivity, transportation, communication, protection, environment; costing of a project and alternate proposals also become equally necessary. Most sponsors would seek to know, "What's in it for us" Eveland's study faced a challenge in getting sponsors for funding their project. Ultimately, the research was funded through a grant from a nonprofit organization, 'The Johnand Mary' and 'R. Markle Foundation,' whose two programmatic interests were, aging and adult development, and social uses of media (Eveland & Bikson, 1988, p. 356, footer note 1). The main logistics cost would stem from: Staff costs Non-staff direct costs Space, facilities and equipment Stationery and consumables Indirect costs Inflation / VAT Publishing and conferences costs Travel, boarding and lodging It is plausible to attempt Eveland's study using quasi-experimental design. Community groups with similar average age, gender mix, qualification, experience, age as criteria, can be positioned in two separately hired buildings, one acting as a test group, and the other as control group. Pre and post intervention observations could be captured. Pre-entry criteria tests could ensure homogeneity of sample population. The data collection methods could use palm tops, lap tops connected to a web-based data management system. Perhaps, use of RFID as tags attached to people (ethical issue) could easily establish contacts made. 9. What would you like to know in order to be able to understand and assess the logistics of the project It would be of interest to know the cost of the project: projected, versus actual, with appropriate breakup in categories, person-hours expended internal and external, funding efforts and agencies contacted, bottlenecks, lessons learnt, project delays, downtime of system support, and above all, the morale of staff in the project. Although the last item does not reflect as a logistical entry, it matters most in the success or otherwise, of a project study! Overall, it was an interesting study, and Dr. Eveland's study was certainly no lightweight project. References Bikson, T.K., & Eveland, J.D. (1992). The interplay of work group structures and computer support. The RAND Corporation. Cronbach, L.J. (1957). The two disciplines of scientific psychology. American Psychologist, Vol. 12, pp. 671-684. Eveland, J.D., & Bikson, T.K. (1988). Work group structures and computer support: A field experiment. The RAND Corporation. Human Subjects Research. (2001). U.S. General Accounting Office.GAO-01-775T. McGrath, J.E. (1981). Dilemmatics: The study of research choices and dilemmas. The American Behavioral Scientist. Vol. 25(2), ABI / INFORM Global, pg. 179. McGrath, J.E., & Brinberg, D. (1983). External validity and the research process: A comment on the Calder/Lynch dialogue. The Journal of Consumer Research. Vol. 10(1), pp. 115-124. Rees C. (1997). An introduction to research for midwives. Cheshire: Books for Midwives. Research Ethical Framework. (2005). Economic and Social Research Council. U.K. Research methods in I/O psychology. (2008). Department of Psychology, University of Western Ontario. Runkel, P.J. & McGrath, J.E. (1972). Research on human behavior: A systematic guide to method. Holt, New York. William, T. (2007). Introduction to research design. The Social Research Methods Knowledge Base. Cornell University. Read More