Trust

Introduction

Embracing Diversity is an ongoing preschool project led by the Trust for Social Achievement (TSA) that aims to support Bulgarian preschools serving children in relatively poor communities from a variety of ethnic backgrounds.

The project includes an ongoing teacher capacity building component as well as a research component in collaboration with scholars from the University of Utrecht (UU) and the University of Amsterdam (UA). The latter is designed to collect and analyze data on perspectives and practices from kindergarten teachers, parents/caretakers, and children themselves, with the aim to inform education policy makers how preschools can best support children from a variety of backgrounds thrive in kindergartens to achieve their potential.

This request for proposal pertains to the research component of the project. It outlines the data collection that is being planned and invites experienced data collection agencies to submit proposals to undertake this data collection in November-December 2023. The survey instruments will be designed by TSA in collaboration with the research partners, and with inputs from the data collection agency to be selected.

Data collection

Target Sample

TSA seeks to contract a data collection agency that will conduct interviews with three groups: (1) kindergarten teachers, (2) kindergarten aged children (6 and 7 years old, regardless of kindergarten enrolment status), and (3) their parents/caretakers.

• For teachers: the data collection agency will implement a survey aimed at collecting information on teachers’ practices, including their assessment of children’s performances, the nature of their interactions with their students, teacher attitudes and beliefs, and general background information. The survey will include a short video that teachers will be asked to watch and on which several questions will be based. This video will be developed in advance by TSA.
• For kindergarten aged (6-7 years old, last year of kindergarten) children: The data collection agency will carry out a school readiness test and a short IQ test (e.g. Raven) with the selected children and will also ask them some questions about their self-esteem, self-perceived competence, and mindset and their experiences in kindergarten (for those enrolled).
• For parents/caretakers: the data collection agency will implement a survey of parents/caretakers aimed at collecting information on three different domains: describe the relationship of those parents with their children (parental engagement practices, parental expectations, …); describe the relationship of those parents with their children’s teachers (frequency and nature of contact, beliefs about school quality, etc.) and collect some general background information.

Duration for the one-time assessment

On average, interviews are expected to last about 1 hour each per teacher, 1 hour each per child, and 30 minutes each per parent/caretaker.

Sample Selection

The interviews will take place in 60 relatively poor communities across Bulgaria, with significant shares of Bulgarians from Roma ethnic origin. These communities have been pre-selected by TSA, are spread across Bulgaria, and were all part of a previous TSA project called Springboard for School Readiness.

TSA will provide the selected data collection agency the list of communities and kindergartens.

The sample size breakdown is expected to be as follows:

• Total number of surveys - 1800
• Number of communities - 60
• No of interviews per community - 30

Three groups:

• Teachers - 600
• Parents - 600
• Children - 600

Number of interviews per group per community:

• Teachers - 10
• Parents - 10
• Pupils - 10

The pupils and their parents/caretakers will be selected by carrying out a random walk in the community. The teachers will be (randomly) selected from the nearby kindergarten.

Areas to be covered in the teachers’ survey include:

A draft set of questions has already been created. The data collection agency will be consulted on the instrument and will be expected to pilot the final draft before finalizing the instruments.

1. General questions

• Age
• Gender
• Race/Ethnicity
• City of residence
• Place of work
• Employment status
• Professional experience
• Level of education
• Job satisfaction

2. Teachers’ attitudes

• Expectations and beliefs about their students
• Teaching self-efficacy
• Growth mindset
• Behavioral assessment

Areas covered in the parents’ survey include:

1. Demographic information

• Gender
• Age
• Place of residence
• Race/Ethnicity
• Level of education
• Employment status
• Number of children

2. Income level and monthly expenses

• Household assets
• Dwelling characteristics
• Sources of income

3. Parental investment/support on their children

4. Parental beliefs, expectations, and preferences

• Parental expectations
• Performance vs. learning attitudes
• Growth vs. fixed mindset
• Sufficient level of education for a child
• Roma inclusion
• Willingness to pay for education goods

5. Relationship with their children’s teachers [if child enrolled in school]

• Interaction teacher-parent, initiated by the teacher
• Interaction teacher-parent, initiated by the parent
• Impressions about teachers/school
• Parental assessment of children’s learning outcomes

Areas to be covered in the children survey include:

The child survey will mainly consist of a child assessment covering school readiness domains such as early numeracy, early literacy, and socio-emotional child outcomes. This will be an existing direct assessment where children are asked to respond to certain age-appropriate questions. 

Activities

Please plan the following activities:

1. Test the questionnaires

2. Train and monitor interviewers

3. Collect data from teachers: interview, data entry, quality control and reporting

4. Collect data from parents: interview, data entry, quality control and reporting

5. Collect data from children: interview/screening, data entry, quality control and reporting

Timeline

Data collection will take place in November-December 2023. A precise timeline for data collection will be developed together with the selected agency.

Deliverables

• Provide written feedback of notes and recommendations resulting from tests of the questionnaires (in English). If extensive changes are required, the pretest may have to be repeated and a second feedback email will be requested.
• Training plan, data collection schedule, draft progress report example, and data entry system ready to receive the first data, with all data validation functions coded, signaling the field activities can start.
• Regular weekly update containing the list of visited individuals (or enumerator), the data entered in the database, the back-checks, corrective measures if applied, updated schedule and any other relevant information.
• Database of the dual entry for 10% of the questionnaires.
• Cleaned and validated database in SPSS or Stata with labels, with outliers verified.

Other requirements

1. The agency will seek research/ethical approval for the data collection. The data collection will include an informed consent procedure and a data management plan in line with GDPR and related regulations.

2. Data quality:

• Experienced supervisors managing the fieldwork will carry out the first level of quality control of the survey results. These staff should be experienced in survey methodologies, quality control and procedural aspects of such research projects and will work closely with the company management to monitor the performance of interviewers. Supervisors will:
• Observe and evaluate the first few interviews by each interviewer, and, if necessary, provide additional training, or replace interviewers to ensure the highest quality data collection.
• Organize systemic data verification and validation for 100% of questionnaires, on an ongoing basis during the whole data collection exercise, to identify incomplete, omitted or otherwise erroneous data. If necessary, return to respondents in the case of significant lack of response to parts of the survey or replace incomplete interviews with additional ones; Sign the questionnaire to certify completion. This should be executed concurrently to field activities, not at the end of the data collection.
• Proceed with back-checks of a small number of questions for a random set of questionnaires, to be verified by phone or in person by the data collection supervisors or auditors.
• Also use statistical methods for checking the consistency, accuracy, and quality of work for each interviewer. This requires data entry to start at the beginning of field work.

3. Data entry:

• Start data entry as early as possible after the start of data collection, and no later than two weeks after the start of data collection.
• Design a data entry system that can a) limit the possible values, b) alert data entry operators of aberrant or extreme values caused by typos, and c) conditionally limit or alert when data entered conflicts with other answers (for example skips) for internal coherence. Submit the data entry system for approbation before the start of data collection. The data entry system must be flexible enough to handle minor adjustments to the questionnaire after data collection has started.
• Manually validate the complete database at the end of all data collection exercises by looking at the coherence of the data in a summary form to correct impossible answers and validate the outliers/extreme values by checking both on paper and by making phone calls when the values look suspicious. Provide a report of the verification process for each variable, tallying the method/criteria used to identify the outliers (% verified or multiple of the standard deviation), the number of outliers identified and corrected, and the number of incoherent/impossible answers identified and corrected. Report on the progress of the validation process.
• If paper-based surveys, perform dual entry for at least 10% of all questionnaires. By keypunching twice, the same data, it is possible to estimate the level of mistakes, and results shall be reported to TSA. The dual entry should be undertaken at the end of data collection.  If the level of error/match is satisfactory, data entry is also considered satisfactory. If the data is not satisfactory, corrective measures need to be proposed by the firm and approved by TSA.
• Update progress on a weekly basis during data collection, data entry and data processing/cleaning to the TSA team, including a report of all audits and corrective measures taken when issues arose, in an email of 2-4 paragraphs of description, plus standardized progress tables. A standardized report listing entries by geographic region or by enumerator, to be approved by the TSA, will be used to report the number of interviews, entries, back-checks, and validation made up to date. An update of the estimated dates of completion should also be included in every report to identify as early as possible changes to the schedule.

4. Reporting

• Ensure the TSA team representatives and collaborating researchers can monitor progress at every stage. In the process of monitoring work in progress, the TSA team shall have a right to request and the data collection agency shall within 3 business days provide any information relating to the survey, including information about the progress of the survey, list of households interviewed, list of surveys key punched in the database, a copy of the data currently entered in the database, etc.

The final database for delivery will contain properly labeled variables and properly labeled values. If certain parts of the database do not meet quality requirements, the data collection agency will redo the quality check and if necessary, repeat the fieldwork exercise to replace data that does not meet quality requirements.

Note: if the data is entered electronically directly by the enumerator in the fields (tablets, netbooks, smartphone), some of the above procedures may be amended.

Confidentiality

The data collection agency will at all stages protect the confidentiality of respondents participating in the surveys. None of the intermediate or final outputs of the surveys will bear the names of participating individuals. All completed questionnaires will either be delivered to TSA or destroyed based on the written decision of the TSA task manager. The survey data and all related analysis are the property of TSA and should be used for other purposes only with the written consent of the TSA task manager.

Qualifications of the data collection organization

A reputable data collection agency demonstrating the following qualifications:

• Established track record and demonstrated experience in sample design, survey methodology and survey implementation.
• Experience in conducting household surveys, and the capacity to undertake data entry.
• Capacity to train and mobilize qualified surveyors and data entry personnel. The data collection agency must have the logistical capacity to carry out the survey within the time frame of the study.
• Ability to evaluate the effectiveness of the survey instruments and methodology, and to revise as needed to achieve the best results.
• Proven ability to plan, manage and execute complex projects and to ensure high quality delivery of results.
• Ability to adapt to unexpected work program needs and changing work requirements.
• Demonstrated ability to handle sensitive situations.
• High ethical standards and deep sense of integrity and commitment.
• Staff: supervisors and enumerators will have ample experience collecting these types of data, including experience collecting data with children, and collecting data with vulnerable populations, including Roma.

Submission and Responses

Schedule

• 1 September 2023 - RFP Issued
• 12 September 2023 - Deadline for all questions and clarification inquiries, submitted via e-mail to pzeynelova@tsa-bulgaria.org and ltoneva@tsa-bulgaria.org
• 18 September 2023 - Deadline for all answers to Respondents’ questions
• 29 September 2023 - Proposals due
• 6 October 2023 - Selection of professional / agency completed & notification sent

Instructions for Submission of Responses

The response must be sent by email to pzeynelova@tsa-bulgaria.org and ltoneva@tsa-bulgaria.org no later than COB on 29 September 2023.

1. Technical proposal in English: The Respondent shall provide a cost proposal for the Required Services, which includes a list, by type and amount, of all fees, overhead charges, or reimbursable expenses, together with timeline and estimate of days.

2. Supporting documents in English or Bulgarian:

• Executive Summary: Respondents should provide a summary of their qualification. The agencies are expected to provide a summary of their agency as well. This summary should be a maximum of 2 pages in length.
• Statement of Professional / Agency Qualifications: All responses must include a statement of qualifications, experience and description of the Respondent agency / professional and its history in implementing projects with cultural and ethnic sensitivity.
• Professional / Agency Staff Qualifications: All Respondents must identify the individual(s) who will have primary responsibility for contact and communications with TSA, the person who is authorized to negotiate and contractually-bind the Respondent.

Evaluation Process, Criteria and Selection

Process

TSA shall evaluate each response with timely and complete submission. After review of the responses, interviews with shortlisted candidates might be required.

Selection Criteria

• Prior experience in research in ethnically diverse neighborhoods;
• Cost;
• Appeal of non-specified additions to improve quality of collected data.