I. INTRODUCTION

This is the evaluators' report of the Workshop funded by grant #R13 HG00793-0142, entitled Women and Genetics in Contemporary Society (WAGICS). The Workshop was held May 16-19, 1996, in Zanesville, Ohio. As the first effort of the Ethical, Legal and Social Implications (ELSI) branch to support a project bringing women's and feminist views to bear on the on-going work of the Human Genome Initiative (HGI), the results of the Workshop will generate interest among a wide variety of constituents. Over 60 women from the fields/communities of genetic counseling and genome research, health advocacy, philosophy and social studies of science, disability activists, women of color and others, participated in a three-day workshop designed to address the social, scientific and ethical issues of the HGI and their relevance to women. The project's main objectives were to extend the analysis of ethical and social questions raised by the HGI by utilizing insights and methodologies from four fields: feminist ethics, bioethics, science analysis and technology studies. This process was to occur through the use of a collaborative methodology to focus on the concerns of women, with extensive input from potential end users of HGI discoveries. The project's goals are listed under goals and in Table 1 and Table 2. The program for the Workshop can be found under program.

The NIH review panels requested that a formal evaluation of the WAGICS Workshop be undertaken. The members of the evaluation team recruited at the time funding was initially sought were unable to participate when the grant was finally obtained, and a new team was assembled, consisting of a cultural anthropologist and a health policy analyst with previous experience on an ELSI project. While originally designed as an outcomes assessment, in consideration of funding actually received, a more focused evaluation objective became to assess achievement of the Workshop's process goals, primarily through written surveys of participants. The question guiding the evaluation was whether the project goals were operationalized so that participants perceived a difference from other workshops and meetings attended, and, whether these differences were attributable to the goals. Capturing participants' expectations before and after, as well as anticipated outcomes, also influenced the effort.

Organization of the Report: Following a review of the evaluation methods used, this report offers a summary of the evaluation results, focusing on six areas: (1) participant background and expectations, (2) pre- and post-Workshop agreement with the Workshop goals, (3) pre-and post-Workshop agreement with the Workshop principles and procedures, (4) assessment of the individual sessions, (5) assessment of the alternative communication techniques, and (6) participants' post-Workshop commentary and reflection. An effort is made to use direct participant feedback wherever possible.

II. METHODOLOGY

The methodology draws on traditional sociological and anthropological qualitative techniques that are adaptable to a variety of field settings, including one-time events such as conferences, workshops or seminars. Three different types of data collection methods were used:

A. Pre- and Post- Workshop participant completed surveys, (Appendix E and F) A three-page questionnaire was mailed to all participants to complete prior to the Workshop. The mailing was timed to occur just after some conference reading material was sent to all participants by the conference organizers. The questionnaire consisted of fixed choice and open- ended items and covered several areas: familiarity and previous involvement with ELSI, expectations of the workshop's significance and impact, and agreement with workshop goals and organizing principles. Respondents were asked to return their surveys by mail; some completed their survey after they arrived in Zanesville, but before the workshop officially began. The return rate was exceedingly good, 89%.

The Post -Workshop Survey (Appendix F) was distributed to participants on the last morning of the Workshop. Its design and content were similar to the Pre-Workshop Survey with certain necessary adjustments. Participants were asked to complete it before the end of the meeting and hand it in or mail it to the Evaluation Team as soon as possible. To boost the return rate a reminder was sent via email to the non-compliers. The final return rate was also very good, 76%.

B. On site observations In the effort to engage process analysis and observations in the course of the conference, each Workshop session was voluntarily evaluated by two different Workshop participants who observed and completed a one-page assessment sheet (Appendix G). Additionally, one member of the Evaluation Team per session also completed an assessment sheet.

The Evaluators also engaged in observation typical of anthropological field work, in which elements of the Workshop's structure, process, interactions, and communications were noted.

C. Alternative communication techniques Alternative communication methods were created for participants who prefer to communicate opinions and views in ways other than in open public discussions or debate. A Suggestion/Comment Box was located near the entry to the conference room, and 5 easels with poster paper were set up inside and outside the room, some titled and some blank. All comments and suggestions were collected.

III. RESULTS OF DATA COLLECTION

A. Participant Views, Backgrounds and Expectations

Based on the pre- and post-Workshop surveys, this section reviews the background of the participants as related to the HGI and ELSI; their expectations of the Workshop prior to the event; and their reasons for participating. Numbers reported are descriptive percentages only, as statistical analysis of the data was not undertaken. The narrative material, used extensively, supplements the quantitative measures.

One area of success for the WAGICS Workshop organizers was in identifying and recruiting a diverse group of women, many of whom had not previously been involved in an ELSI event. This seems noteworthy for its potential to move the discussion and consideration of matters related to ELSI's concerns beyond the core of usual participants. More important, it relates to the goal of incorporating voices of previously uninvolved women. More than one-half (55%) of participants indicated they had not participated in other events sponsored by the ELSI. Among those with previous involvement with ELSI were a variety of events and responsibilities: as grant reviewers, participants in the Cystic Fibrosis Carrier Screening Consortium, the 1991 and 1993 National Conferences on Genetics, Religion and Ethics in Houston, and as principal investigators.

The majority of participants (63%) indicated they know the purpose of ELSI; 26% indicated they are somewhat aware of the purpose, and 10% did not know what ELSI was set up to do. Overall, only 37% claimed to be aware of the issues ELSI has thus far addressed. Of those, quite a few believe that there have been gaps in areas addressed by ELSI projects, notably issues pertaining to women and the problems that average working women face who are not knowledgeable about the HGI. A few asserted that ELSI projects have not been allowed to question the HGI; that a gap persists between the practitioners of science and the ethics projects; and that projects have not been comprehensive enough to do useful work on problems, especially in areas of disability and the effects on pregnant women. Resonating with the goals for WAGICS, it was said that too little attention had been paid to women, people of color, and other marginalized groups.

Results from the pre-Workshop survey showed that practically all participants (85%) expected that the WAGICS undertaking would be different from previous meetings and events on genetics and the HGI. Expectations were that WAGICS would reflect women's perspectives and directly address relevant issues; be less professionally dominated and also less academic; provide opportunities to mold the agenda, ask questions and hold conversations from women's points of view; focus on gender issues and create a feminist model for collaboration; be more subjective, process-oriented, and diverse.

In advance of the Workshop, the overall significance of this unique effort was judged to be potentially profound, with many positive effects anticipated. Comments from participants were about its potential to mobilize marginalized knowledges and people; raise awareness of policy makers and funding agencies about the impact of gene mapping on women and human diversity; stimulate research; provide a vehicle for discussion among feminists from different perspectives/disciplines; and result in national media attention and new publications.

Similarly, many participants regarded the Workshop as an opportunity for personal and professional growth. While a few of the women chose to participate "because they were asked", most identified such interests as updating their current knowledge of genetics and genetic issues, learning about and sharing cross cultural issues, learning more about feminist analysis, integrating new information into their course curriculum, and being in a feminist setting for developing their thinking, as reasons for their participation. Concerns for the core issues of the meeting stimulated thinking about how awareness and knowledge would be broadened. Finding new areas for research was prominent among the professional and personal goals expressed.

In summary, participants generally expressed an overall sense of optimism and positive expectations in the pre-Workshop survey.

B. Agreement with Workshop Goals

Sections of both the pre- and post- surveys attempted to assess how participants understood and viewed the goals of the Workshop as articulated by the Principal Investigator (see Appendix B). In this section we offer a comparative review of those findings.

To try to determine what participants thought about the overall framework of the Workshop, a grid was presented on which participants could indicate their agreement with the goals stated by the organizers.

It appears, therefore, that support softens as the goal or process becomes more complex, theoretical and/or more explicitly ideological. It is not clear if this is due to lack of familiarity or lack of agreement, but is not surprising, given the diversity of participants' ages, occupations, educational levels, involvement with the politics of public policy, and varying self interests. Still, support for the Workshop goals was overwhelmingly positive in advance of the event.

Given the strong support for the Workshop goals and objectives on the Pre-Workshop Survey, what did participants think afterwards had been achieved?

Valuing input from women previously marginalized by race, ethnicity, disability, age, or class, also sustained strong support, by 83% of respondents. It should be noted that during the meeting, several women complimented the Principal Investigator for the substantial number of women of color and disabled participants. Still, this is a 12-point drop from the pre-Workshop survey (95% vs. 83%), and some women were unsure (n=5) or disagreed (n=2) that it had in fact been achieved.

Similarly, on item #3, whether networking among groups was facilitated, 77% said it had been, while beforehand 95% had supported the intention. Five women expressed dissent and five expressed uncertainty (Not Sure). Even though 77% considered this goal realized, there is a need to consider the reasons why this goal did not meet expectations and what might have been done better to facilitate networking among groups. The use of small work groups might have been one way to establish a process to enable women to talk across the boundaries of numerous fields and disciplines. On the Post-Workshop Survey about one-half of respondents indicated that the most significant things happened "during the sessions", rather than at dinner or in between sessions. While this may be support for the strength of the sessions themselves, it also suggests that processes like networking did not happen easily. The experience of the women of color, who organized a separate meeting during one of the periods of unstructured free time, may suggest that a clearer sense of solidarity is needed for coalescing and that connections might have been facilitated had there been time for smaller units to work together on tasks.

Item #4, "Reframed questions to reflect women's life experiences/values", was viewed uncertainly by participants. Once again, in advance 90% supported this goal but in the aftermath, only 60% Agreed or Strongly Agreed it had been achieved, while 33% were uncertain, and 8% disagreed. This drop may reflect the many comments that discussion time was insufficient and that concrete results were not developed. Perhaps, also, once the Workshop was underway, it was not clear how this reframing was to be operationalized.

Whether the Workshop "Utilized collaborative methodology, horizontal decision making and participatory democracy to work toward a dialogue" was also less clearly realized than expected (92% before vs. 75% after). Others were Not Sure (18%), Disagreed (5%), or Strongly Disagreed (2%).

On the last item, "Assessed utility of feminist ethics, medical ethics, science and technology studies to address HGI questions," respondents spread across all opinions. On the pre-Workshop survey, 84% Agreed or Strongly Agreed, and 16% were Not Sure. The post- Workshop results show 54% Agree or Strongly Agree, 22% Not Sure, and 22% each Disagree and Strongly Disagree.

It is worth noting that the goals themselves, as well as any proclaimed agreement or disagreement, rest on several unarticulated assumptions and unmeasurable subjective estimations of their meaning. Within the broad spectrum of participants were multiple understandings of the problems and of the issues the goals were meant to address. There were, furthermore, differences in understandings within and between disciplines over what constitutes the fields of feminist bioethics and science and technology studies. Some participants may not have subscribed to or promoted a feminist perspective, or might not have had a clear grip on the ideas and techniques utilized to run the Workshop (based on ideals of collaboration and non- hierarchy). Among feminists were differences about the kind of feminism most useful to the task. The discussion made these differences evident, and the questionnaire results may reflect, though not always capture exactly, the complex arguments of participants.

C.Agreement with Principles and Procedures

Another focus of the Workshop evaluation was the process of organizing the meeting itself and the theoretical principles that guided the co-Principal Investigators and the planning committee. A second grid sought participant assessment of these principles.

On the post-Workshop survey, the grid on implementation of Workshop principles yielded a few noteworthy findings:

The biggest differences occurred in the last three items. In response to whether use of the principles would produce a different experience, 60% said in advance that they thought it would. On the post Workshop survey, however, more participants, 82%, said that the principles had made the meeting different. In contrast, communication was assessed as less effective afterwards: 86% agreed or agreed strongly that it had been effective in the pre- Workshop Survey; only 65% did so on the post.

Finally, regarding whether the setting and facilities bear on the success of the Workshop, on the pre-Workshop survey, 52% felt the facilities would be important and afterwards, 55% thought they were. The answers are difficult to interpret since issues about access for the disabled, the uneven quality of the food, the relatively unaesthetic environment, and a lack of access to urban amenities, may have been assessed as important or unworkable, but the responses did not clearly register as satisfaction or dissatisfaction. The adequacy of the site on the matter of access for the disabled had been a major area of concern in advance. Many participants criticized the site's lack of various amenities. A problem that produced awkwardness in every session was the need to wait for the microphone to be carried around. Any of these problems could have been reflected in the data.

D. Assessment of Individual Sessions

The method to evaluate the nine individual Workshop sessions was designed to require relatively minor effort on the part of participants for the important reason that it was undesirable to have participants prevented from participating in the sessions due to their own evaluatory efforts. The Evaluation Team asked for two volunteers to complete a form on each session¹. The form covered three items: (1) depth and content of discussion, (2) process and organizational assessment, and (3) factors that enabled or stood in the way of achieving intended objectives. A member of the Evaluation team also completed a form.

The Session Evaluation form (copy in Appendix G) proved difficult to use, in part because some questions were premised on an assumption about the panels' objectives different than that assumed by the panel organizer and participants. Not initially organized with the intent of developing lists of policy points, panels were only sometimes successful in focusing on the task designated in Item 1C: "Did the discussion reach conclusions, such as general agreement, a clear delineation of areas of disagreement, lists of points for further exploration, etc.?" Speakers struggled under time constraints just to present their comments before being cut off. Panels therefore appear to fall short on this. Also, midway through the meeting, the goal to develop policy points was downgraded, further changing the expectations for panelists and evaluators.

Based on the selection of two volunteers per session, the evaluation team paired an individual from a 'humanist' position with one with a 'scientific' position in an effort to get a range of viewpoints on each session. This pairing produced some interesting results, as there is evidence that the varied orientations were reflected in their evaluative remarks. Scientists often indicated that more scientific information should have been presented. Social scientists and philosophers reacted more positively to conflict and controversy, even saying "This is what I came for." These broad generalizations should not be construed as an effort to essentialize any group, but it is useful to recognize that what constitutes an effective session may be different depending on training and orientation. But more important, some of the controversies and disruptions were due in large part to the fact that participants' professional orientations became points for attack. Clashes stemming from the encounter of different disciplines were most intense during the session on Behavioral Genetics (#8), with both evaluators' and participants' comments reflective of the disputes. The following are taken from the session evaluations and the post- Workshop surveys:

"The session did not include any real facts about what is happening, what is possible. Need to reality check. Just paranoid, scare raising rhetoric which is not helpful. Concerns are real but such discussion is not helpful."

"Wish it had happened sooner. This is what I came for."

"Every statement I made was ridiculed by someone. I don't feel good about how I was treated."

"Was important in exposing the divisiveness within the group. It's important to dismantle before we can build but we must have another conference in order to begin rebuilding the process."

"This was the heart of the meeting; this was the most important discussion."

"I wish there had been an overview of Behavioral Genetics. The session seemed to turn into a most unconstructive debate. It was polarizing instead of bridging gap. People were not listening to each other, just telling others what they had to say."

"The discussion on Behavioral Genetics was painful to listen to because so many people were ill-informed. At the end I just got sick of the anti-science comments."

These statements illustrate the interpersonal and interdisciplinary tensions that remained unresolved to the end. The feelings expressed about behavioral genetics research also reflect a good deal about the larger social and ethical divides over the more speculative aspects of the HGI. Fear or criticism of the power of genetic research to define and control social behavior was downplayed by individuals close to the day-to-day functioning of the HGI, while trust in the process and direction of research was seen as naive and inappropriate by many, especially women of color, the disabled and certain lesbians. That the WAGICS workshop could not manage to resolve these areas of contestation does not diminish the undertaking; rather, the nature of the disagreements is better illuminated and the need reinforced to continue to search for ways to encourage encounters and broaden the debate about these important matters.

E. Alternative Communication Techniques

Designed to allow for personal communications that individuals felt either less comfortable publicly verbalizing or felt to be supplemental to the session proceedings, neither the Suggestion Box nor the Poster Boards were extensively used.

In the course of the three days of the Workshop, the suggestion box was used a total of five times. Three submissions were requests related to the setting, for instance the room temperature and food quality. A fourth suggestion requested that presenters be reminded to verbalize any visual presentation material for the visually impaired participants. The fifth submission, received during the final day of the Workshop, offered the viewpoint that the Workshop had been too anti-science, and added that this anonymous contributor herself was not a scientist.

The five poster boards placed in the rear of the main meeting room and in the hall immediately outside the meeting room were available for public display/ comments. Here, again, several comments were of a procedural nature. In addition, a couple of 'conversations' supplemental to the workshop proceedings unfolded. One such 'conversation' follows:

If we are going to lay out all the possibilities for women considering reproduction, shouldn't we also point out to women at a young age that not postponing pregnancy till they are in their late thirties will reduce their chances of infertility, miscarriage, and some kinds of children with disability?

Women's lives don't always work out so that they can 'control' things in order to have children at the 'proper' ages.

We would have to promote greater changes re: career pathways and education so that women's options in non-reproductive areas will not be diminished!

That neither the suggestion box nor poster boards were extensively used may indicate that the sessions themselves provided the necessary space for the articulation of viewpoints and that alternative spaces were thus not necessary.

F. Participants' Post Workshop Commentary and Reflection

In addition to agreement/disagreement grids and the forced choice questions in the participant surveys, the post-Workshop survey also elicited narrative evaluation and commentary from participants. Participants' provided thoughtful and detailed comments revealing how they felt about the Workshop, what influence they expect it to have on ELSI, women's studies and in other areas, and what actions they expect to take as a result of the workshop. Statements frequently reflected the personal and emotional tone that marked the undercurrent of the event.

1. Strengths and Weaknesses

Respondents who had attended other ELSI events confirmed that this one was different; the most common difference noted being the funding level. Participants became aware of the funding situation as the project developed, and this awareness was accompanied by deep dismay that a woman's event received such a poor level of support. Other important differences were the experience of being at an all-women's meeting, organized with a feminist perspective, the significant presence of women of color and disabled women, and the opportunity to meet and network with other women. Especially appreciated were the multiple perspectives and voices:

"An incredible group of women--powerful women! A different kind of conference--an attempt to change the status quo, the process. I thought it was great, in spite of the problems. I'm not anti-male but it was nice not having men around."

"The best part of this whole thing was participation of African American women who spoke their minds."

" I'm glad to have participated--especially benefited from having prominent voices of marginalized/vulnerable groups."

"Excellent experience. I was very impressed by the diversity of participants and their wisdom and articulate presentations."

There were some issues, however, about which there was less enthusiasm. Participants expressed uncertainty about whether important issues were fully addressed. In part this concern was attributed to the lack of time, which contributed to a feeling that all viewpoints were not adequately represented. It also prevented opportunities to communicate across disciplines and groups. Too often the discussion had to end just as issues were laid out, limiting debate and exchange of views. Important issues were raised but fell short of clarity or closure. The tight timetable and structure also prevented moderators from facilitating or a dialogue; their task became simply letting people take turns speaking.

Issues not thoroughly addressed included: the interface between molecular biologists and those in other fields; the question of the resources expended on the HGI as opposed to other areas; the effect of prenatal diagnosis on individuals living "at risk"; the choices offered by feminism; and more thorough exploration of issues related to disability. These could be the basis for another meeting, a discussion group on the Internet, or through some other means.

For women with backgrounds in science and genetics, a common complaint was misinformation and/or lack of sufficient information about HGI, genetic testing, interactions between environment and behavioral genetics that produced a "paranoia about science that interfered with logical discussion."

2. Expectation of Influence on ELSI, Women's Studies, Other Disciplines and Fields

Participants were asked whether they thought the meeting would have any influence on the ELSI program itself, and a wide range of opinions emerged. On the one hand, some commented that ELSI didn't seem likely to be interested, since this Workshop had been funded at such a minimal level. It was suggested that ELSI, as a part of the NIH bureaucracy, will not change its predetermined course, and moreover, within NIH men are vested with most of the power, so there was a sense that they would not listen. In this vein, there was perceived to be an absence of people with power in ELSI at the Workshop, with no well-known social scientists, who often "don't have real power anyway". One person thought ELSI would like to take credit for having a woman-focused multi-perspectival conference but would have a hard time using the ideas that emerged from the meeting. Somewhat self-critically, it was felt that the lack of a concrete product, book, paper, or set of recommendations, would diminish the influence of the Workshop.

But there were also a number of positive ideas and suggestions to work within the system to produce a better result for women. Pragmatic ideas were offered: try to influence policies about directions of funding by nominating new people to serve on review committees; direct research funds to ask previously unasked key questions and embed those in the discourse and then keep adding to it. Increase awareness within ELSI of the power of the emotion generated by these issues and the need to continue dialogue. An urgent sense of engaging in constructive collective activities is a feasible way to do this. Noting that women's voices can be strong, any effort to develop a policy statement would be instrumental.

Views were optimistic about ways women could influence the fields of women's studies, science and technology studies and, potentially, bioethics. Literally dozens of women committed to working, teaching, writing, sharing information, and increasing understanding of issues. A multidisciplinary approach with extensive networking will encourage participants to be aware of issues and concerns. To reach people in the sciences, the admonition is to be "squeaky clean" with facts, and to avoid a condescending tone.

3. Actions to be taken

Participants stated that their writing, teaching and research will be influenced as a result of participating. Greater awareness at the personal level and a commitment to doing things differently are probable results as well. Collaborations among participants previously unknown to each other are planned, as several people found the opportunity to envision a project that will cut across a discipline. The networking started around dinners and in the sessions turned up on the action lists participants want to continue after the Workshop. Networking to try to change frameworks represents still another level of activity intended. Some women want to organize focus groups to raise similar concerns among different ethnic groups. The major demonstration of this effect was the Report from the Women of Color put together by a small caucus.

IV. EVALUATORS' NOTES

As evaluator-observers, we conclude by offering a few remarks about elements of the Workshop structure and process that many participants reacted to and continued to react to after the Workshop. As the previous section showed, a great deal of emotional energy was generated during the three day meeting, much of it constructive to thought and action that women took away to use in their work. But not all of it was constructive, as some participants left the meeting feeling angry, disappointed, dismayed, even victimized.

We believe several different factors contributed to the sense of dissatisfaction over results, and also account for much of the divisiveness and the negatives that came to be associated with professional differences. Many assumptions framed the Workshop, including "we're all feminists" and "critique is a good thing to do." The assumptions about being feminists and using feminism were never really open to negotiation and discussion. This was structurally reinforced with the initial panel on feminism as a presentation--albeit interesting and worthwhile-- rather than a discussion. For some, rather than being edifying, this presented a philosophical position that they did not (necessarily) agree with.

More so the question of critique, a term that has many different meanings in the multiple communities represented. While some people participated in order to question and critique the foundation of the HGI, an equal number were not versed and not invested in that type of activity, but did have their own ideas about how to make the HGI more responsive to the needs of women. Thus, from the start, there was no universally shared perspective on what was being sought, or agreement on the ground rules of interaction or for the depths and directions of the discussion.

Without such agreement, the Workshop de facto became an encounter among members of different professional worlds of humanists, social scientists, laboratory scientists and genetic counselors, and members of health and lay communities. The laissez faire, non-directive approach of "get everyone together and things will be OK" --a magnificent idea seldom realized- -proved insufficient to guide the process toward an affirmative, and affirming outcome. As participants' different cultures and professional worlds became subject to criticism, the lack of management over direction became difficult for many to contend with.

Some other difficulties during the meeting that caused communication problems were structural in nature. For instance, the way the group was held together throughout the meeting when some small group sessions would have benefited the process.

Still, the starkness of the difficulties brought people up against some of the very real issues that are at stake. It forced an emotional confrontation in a way that most surely had a deep impact, and in a way that might not have occurred under a more smoothly run process that did a more surface rendering of the issues. This may well have provoked more personal stock taking than a 'glossier' workshop would have.

Given the deep divides --reflective of the larger social world-- the collaborative efforts that do occur will be all the more important and relevant to the on-going work of ELSI.

Susann Wilkinson, M.S.W., M.P.H., Dept of Social Medicine, Harvard Medical School, Boston 02115(617) 623-2583; swilkins@mit.edu

Melissa Cefkin, Ph.D., 64 Bow Street, #C, Somerville, MA 02143; 617 629-0905; melissa_cefkin@irl.or

Return to the WAGICS home page.