Step 1: Defining Intended Use and Intended Users
- Expected Output
- What are Evaluation Exercises?
- How can Gender be Made Visible in a Project?
- Who are Stakeholders and Intended Users?
- Activity 1.1 Identifying Your Evaluation's Intended Users
- Activity 1.2 Defining Intended Use
- Worksheet 1 Synthesising Intended Users and Intended Use
- Ensuring Gender Considerations are Integrated in Forming an Evaluation Team
- To draw up a list of intended users and their corresponding use to the evaluation results
- To enlist an evaluation team that includes stakeholders who can address gender and ICT issues
Often the concept of evaluation is linked to a donor requirement, an externally imposed system of checking to determine that project objectives were met and resources were wisely utilised. However, there are many other reasons why evaluations are conducted, for example:
- To identify areas of improvement in a project or programme
- To highlight and resolve disagreements
- To set priorities and goals
- To clarify and tackle problems
- To suggest new strategic directions
- To get feedback, appraisal and recognition
- To celebrate project achievements
- To attract resources toward a project
What needs to be examined beyond these more common objectives is a straightforward, yet critical question – How will the evaluation be used.
A well–developed evaluation approach called utilization-focused evaluation (U-FE), states that “evaluations should be judged by their utility and actual use”. Michael Quinn Patton, developer of this approach, points out that use is not an abstraction. Intended use, he says, “concerns how real people in the real world will apply evaluation findings and experience in the real world”. Evaluations should be facilitated and designed having in mind how everything that will be done from the start of the process up to the end will affect intended use. Patton’s extensive study of conducting professional evaluation reveals that the most challenging question concerns identifying what needs to be done to get appropriate results that can be meaningfully used. The focus of the evaluation is on intended use by intended users. [Patton 10, 20-22]
Setting evaluation objectives is about clarifying who intends to use the evaluation and how they intend to use the results. The GEM tool adapts intended use to specifically define evaluation objectives.
Bear in mind that it is important to differentiate intended use or evaluation objectives from project objectives. In a project cycle, formulating objectives are derived from problem identification. An objective may be stated as an expressed intention to address a problem, which can be a statement of intended quantified outcomes to be achieved in a specified time frame. In terms of gender issues, the objective should state the intent to address and eliminate a gender issue, for instance, ending a discriminatory practice that leads to closing a gender gap.
On the other hand, intended use or an evaluation objective may specifically focus on how users intend to use evaluation results. For example, a project objective of GEM tester Fantsuam Foundation was to promote access to ICT facilities and provide ski lls training for women in rural communities in Nigeria through their community learning centres. The evaluation objective of Fantsuam, as the main user of the evaluation, wanted to evaluate how services provided in the community learning centres empowered women and girls.
Overall, the intended use of GEM is to analyse gender issues, perspectives and lessons in ICT projects and initiatives. Your task in STEP 1 is to specifically define your intended users and how you plan to use the results of your gender evaluation.
Gender is a cross-cutting issue that affects all project stakeholders and all aspects of a project activity including evaluation. All evaluations, however, do not necessarily have to include all the groups that have participated, benefited or did not benefit from project activities. Choices will have to be made based on evaluation objectives and gender aspects or issues of project stakeholders as intended users of the evaluation.
In identifying intended users, it is good practice to be as specific as possible and determine “actual primary intended users and their explicit commitments to concrete, specific uses.” [Patton 21] The choice of users will determine whose values and interests will frame the evaluation. Full and active involvement of intended users will lead to the following advantages:
- they will more likely use the evaluation if they understand and feel they own the evaluation process and findings
- they will more likely understand and own the evaluation if they were actively involved in the process
Primary users should be involved in the evaluation since they, in the first place were users or were directly involved in the project. As recipients and/or participants of the project, they are in the best position to determine how to reinforce the intended use of the evaluation every step of the way.
To determine users of your evaluation, think about project stakeholders who will be crucial when examining gender and ICT issues. Consider the following in making your decisions:
- Stakeholders can be both internal and external such as project executors, project staff and management for the former. Project beneficiaries belong in the latter category.
- Particularly in the case of ICT initiatives, stakeholders may be located anywhere in the world.
- Projects have direct stakeholders, those who are/or were directly involved in the project’s activities, and indirect stakeholders or those who did not participate in the project and may or may not have been affected by it. The second group is often critical in an evaluation.
- Other organisations involved in similar projects and activities can also be stakeholders. In fact such organisations can of fer important insights and “sector” commentaries that provide a broader but focused context for your evaluation findings.
Your relationship with and how you approach project stakeholders is as important as the process of identifying them. How these stakeholders participate in the evaluation reflects your organisation’s underlying values or approach to evaluation and the ICT project in general.
Project beneficiaries, the primary target group of the ICT initiative, are integral to the process of uncovering and analysing gender and ICT issues. Remember, gender analysis becomes transformative when arrived at by people who are directly involved in the process.
Below is an example of a list of stakeholders of an ICT project that highlights the core stakeholders who may be identified as users of your evaluation.
Staff of the Telecentre
Community (which refers here to all the groups which can use the telecentre and directly or indirectly benefit from it)
Users and non'users
Stakeholders (which includes all those whose actions interfere with the telecentre operation)
Sponsors, funding agencies, support or operating agencies
Other organisations considering use of telecentres
Source: Monitoring, Evaluation and Impact Assessment )MEIA= of Telecentres: An Initial Framework (Telelac)
Brainstorm a list of ICT initiative stakeholders. Identify core stakeholders and specify those who can address gender and ICT issues. From the list of stakeholders, identify intended users of the evaluation. You may use the example Telecentre Stakeholders as a guide.
According to Patton, it is important to negotiate a shared understanding that is mutually arrived at between the evaluators and the users of the project on how the evaluation results can be used realistically. Naturally, the users themselves should commit to the terms of this shared understanding. For example, conducting an evaluation to influence decisions on how a project can be more gender sensitive would attend to the following questions: what would these decisions be, who will make the decisions, when and what other factors will affect decision-making, and how will the evaluators get to know if the evaluation was used as intended. (82)
Below are a set of questions as a guide for Activity 1.2 which can also help in the actual evaluation process.
Questions to Ask of Intended Users to Establish an Evaluation’s Intended Influence on Forthcoming Decisions
- What decisions, if any, will the evaluation findings expect to influence?
- When will decisions be made? By whom? When, then, must the evaluation findings be presented in order to be timely and influential?
- What is at stake in the decisions? For whom? What controversies or issues surround the decisions?
- What is the history and context of the decision-making process?
- What other factors (values, politics, personalities, promises already made) will affect the decision-making? What could happen to make the decision irrelevant or keep it from being made? In other words, how volatile is the decision-making environment?
- How much influence do you expect will the evaluation have—realistically?
- To what extent has the outcome of the decision already been determined?
- What data and findings are needed to support decision-making?
- What needs to be done to achieve that level of influence?
- How will we know afterward if the evaluation was used as intended?
Source: Utilization-Focused Evaluation by Michael Quinn Patton 
To help synthesise answers in Activities 1.1 and 1.2, look at the examples below and use the sample table for the activities.EXAMPLES OF TWO GEM PARTICIPANTS:
Teleworking in Malaysia is Moms4Moms evaluation of its teleworking project whose members are mothers who work at home. The evaluation aimed to facilitate and foster social change to bring about an environment conducive for women to work at home. It sought to understand problems faced by working women and men, the level of recognition of the triple roles of women (child care, housework, and income-earner), and look into how ICTs can help improve lifestyles to enable women to cope with their multiple roles.
The other example, Neighbourhood Information Unit, is an evaluation of Association for Interdisciplinary Work (ATI) on community telecentres that organised gendersensitivity workshops and ICT trainings in two communities in Bogota, Colombia. The evaluation focused on gender sensitivity strategies in neighbourhood information units in Bogota.
|Teleworking in Malaysia|
INTENDED USER: Project implementer
INTENDED USE: To identify the feasibility of teleworking and setting-up a virtual office within the Malaysian context to promote teleworking to the Malaysian government
|Neighbourhood Information Unit|
INTENDED USERS AND THEIR USE OF THE EVALUATION
1. Members of the local community: To strengthen the local communities’ capacity in directly addressing issues with local government, international agencies and state entities.
2. NGOs: To involve other NGOs to ensure Units include issues of interest to the local community.
3. Project implementers: To generate cooperation between neighbourhood centres.
|BELOW IS A SAMPLE WORKSHEET: SYNTHESISING INTENDED USERS AND INTENDED USE|
|Name of Initiative|
Team membership should be representative of the women and men involved in the projective, activity or initiative.
- At least one member of the team should have an experience and an understanding of gender issues. It would be ideal if the experiences were derived from the organisation initiating the evaluation or a close partner of the organisation. If this is not the case, assign a member of the team to play a significant role as one who will think along gender lines. Make sure to define effective ways of bringing her/him into an understanding of gender issues and concerns.
- It is best for the evaluation team to undergo a gender sensitivity training to facilitate awareness of gender issues that should be addressed in the evaluation process.
- At least one member of the team should have an understanding of ICTs both at the level of using technologies as a tool for delivering project goals as well as uses of technology to empower individuals, organisations and communities.
- Participation of key stakeholder groups or identified users is critical.
The team should be small enough to work together efficiently with a team leader or initiator of the evaluation who assumes the responsibility for driving and leading the group. It should be made up of individuals who are involved in the initiative/project/ organisation from different levels (i.e., project staff, beneficiaries, etc.). It is also best to include hired evaluator/s who are not members of or involved in the project as member/s of the team.