Vacancies

Home-based: Senior advisor consultant for Casual Machine Learning to Enhance Policy Design (approx. 30 working days; with travel as needed) - Europe and Central Asia Regional Office (ECARO)

Apply now

Job no: 571733

Contract type: Consultant

Duty Station: Geneva

Level: Consultancy

Location: Switzerland

Categories: Research, Planning, Monitoring and Evaluation

UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential.

Across 190 countries and territories, we work for every child, everywhere, every day, to build a better world for everyone.

And we never give up.

For every child, innovation.

Purpose of Activity/Assignment

The usual approach to policy design roughly follows this pattern:

• Design and implement a policy
• Ask, “did the policy work on average?”
• Expand or adjust the program based on the answer to 2.

While valuable, this approach has limitations. Chief among them is that it struggles at identifying if a policy works differently for one group or another because it relies on averages. Average population effects compounds cases where a policy works better than average for some people with those where it doesn’t work for a different group of people.

There are ways to tease out these differences. For example, we could ask “did the policy work on average for X?”. However, answering this question in a repeated fashion is expensive and time consuming. With this approach, it is operationally impossible to get a complete mapping of how a policy works for all the different groups of people it targets.

Recent developments in machine learning can enable governments to change this paradigm. The most robust evaluation methods rely on RCTs or other quasi-experimental methods to estimate average treatment effects (ATEs). Machine learning enables policymakers to shift to a model that looks at the heterogenous effects of policies in more detail, allowing them to adjust and target policy according to the needs of specific sub-groups. Rather than answer “what works on average”, governments will answer “what works for whom”. This will ideally lead to a more impactful, adaptive, and cost-efficient mode of policy making.

Causal machine learning enables this shift by introducing developments in machine learning to the world of policy evaluation. At a high level, causal forests are composed of decision trees which partition datasets into population clusters. These clusters can then be used to predict and compare the effect of policies at the individual level. This helps policymakers estimate individual average treatment effects (IATE) and group average treatment effects (GATE). Unlike ATES, these two measures do capture heterogeneity in the effect of a policy across population distributions.

Building on the above, UNICEF, along with the Government of Kazakhstan, aims to explore the use of causal machine learning (ML) to enhance social protection policies. Causal ML employs AI algorithms, statistical techniques, and big data to understand the causal effects of interventions such as government programmes. Leveraging Kazakhstan’s administrative data platforms, the project aims to enhance policy effectiveness, generate fiscal savings, and increase the impact of government interventions.

As such and considering the complexity and novelty of the causal machine learning field, UNICEF, together with the government of Kazakhstan, have identified the need for an advisor to build their capacities to implement this technology.

Scope of Work

The exercise will produce a minimum viable open-source pipeline (including data ingestion, algorithm, and analysis of inferred treatment effects) to estimate the causal effect of a policy in Kazakhstan. The model will be trained with government official administrative data. Preliminarily, the government has identified a dataset of about 90 socioeconomic family level indicators, that covers most of the Kazakh population, for the exercise.

Objectives

The expert will advise the data science team within Kazakhstan’s “Ministry of Digital Development, Innovations and Aerospace Industry” to:

a. Assess the quality of its administrative data and select feasible and relevant policy questions to address. UNICEF and the government of Kazakhstan have pre-identified some possible questions, such as: (i) What is the effect of cash transfers on children with disabilities? (ii) What is the effect of cash transfers on the transition of youth from education into the labor market? (iii) What is the effect of cash transfers on education completion?

b. Develop an algorithm, using open-source tools, that uses causal machine learning approaches to answer one or more such questions. The algorithm should estimate heterogeneous treatment effects and output an analysis of these effects to inform decision making and policy recommendations.

c. Create a plan to further improve the government’s capacity to use causal machine learning to complement its already diverse set of policy making tools. This plan will consider both technical (data quality, human resources) and organizational (political buy-in) components.

Guiding questions

The exercise will explore the feasibility to use causal machine learning to answer policy relevant impact evaluation questions. Item “a” above describes a few potential questions, but in general terms the exercise will answer the following evaluation question:

a. What is the effect (on average, by sub-group, and individual) of X policy on Y child-related outcomes? The outcomes Y and the policy X will be defined jointly with UNICEF and the Government of Kazakhstan.

In addition, and considering the capacity building nature of the project, the following questions will also be considered:

a. What is the minimum quality of administrative data needed by countries to effectively implement their own causal machine learning model?

b. To what extent might quality issues lead to biased predictions and hence biased policy?

c. How do causal machine learning models compare to traditional statistical approaches for policy evaluation?

d. What role can causal machine learning play in policy evaluation and design?

e. What kind of child-related policy questions can be addressed using causal machine learning models?

f. What is a replicable and low-cost model for knowledge transfer on causal machine learning to several countries?

g. Would AI policy recommendations be welcomed by policy makers? How can AI be positioned as another tool at their disposal instead of a threat?

Methodology

The above-mentioned questions will be answered primarily by the data science team within Kazakhstan’s “Ministry of Digital Development, Innovations and Aerospace Industry”.

The expert’s role will be that of advising the ministry through all phases of the project, including scoping and assessing data quality, defining feasible policy questions, work-planning, supervising the implementation of the causal forests algorithm, analyzing output and debugging if needed, and presenting results.

Nevertheless, the advisor is expected to have a more hands-on role at key moments in the project. In particular, the advisor will provide remote and in-person training on causal machine learning to the data science team in Kazakhstan. The mix of remote and in-person training can be discussed during the inception phase, but at the moment the expectation is to have 5 days of in-person training in Astana. The advisor may also be required, in agreement with the Government of Kazakhstan, to help review code.

A steering committee with participation from members of the Ministry of Digital Development, Innovations and Aerospace Industry and UNICEF will be established and take an advisory role throughout the whole project.

The exercise will be managed by the Evaluation Specialist from UNICEF’s Europe and Central Asia Regional Office (ECARO) in close coordination with the Child Rights Monitoring and Evaluation Specialist from UNICEF’s Kazakhstan Country Office.

Work Assignment Overview

Tasks / MilestoneDeliverables / OutputsTimeline / DeadlineProject design including selection of policy relevant question, assessment of data quality, and training plansShort inception report1 month after signing contractIn-country training (consider at least 5 days in-country training)Training agenda and materials (PPT, code, sample data, etc.)2 months after signing contractRemote capacity building including structured training sessions, ad-hoc consultations, and continuous advising throughout the projectQuarterly reports with short summaries of training outcomes, PPTs, training recordings (the steering committee will decide if PPTs and recordings are feasible and desirable. The advisor should plan to prepare PPTs at least for the introductory sessions, with later sessions evolving into more open and ad-hoc discussions for which PPTs will be either shorter or non-existent).Every quarterQuality assurance of prototype algorithm and reportTo be included in monthly reports when applicable8 months after signing contractRecommendations to improve the use of causal ML (considering data quality, governance, financial and human capacity, scoping, and any other factor that the advisor considers important)Short (2-3 pages) report with concrete recommendations on next stepsEnd of contract

Approximately 30 working days between June 2024 and March 2025.

Consultant's Work Place and Official Travel

The Consultant will be remote/home-based.

As part of this assignment, some international travels are foreseen. The consultant will arrange her/his travel as and when they take place, and related costs will be reimbursed per UNICEF travel policy.

Travel Clause

• All UNICEF rules and regulations related to travel of Consultants apply.
• All travels shall be undertaken only upon the prior written approval by UNICEF.
• The consultant must be fit to travel, be in a possession of the valid UN BSAFE certificate, obligatory inoculation(s) and have a valid own travel/medical insurance and an immunization/vaccination card.

Estimated Cost of the Consultancy & Payment Schedule

Payment will be made on submission of an invoice and satisfactory completion of the above-mentioned deliverables. UNICEF reserves the right to withhold all or a portion of payment if performance is unsatisfactory, if work/outputs are incomplete, not delivered or for failure to meet deadlines. All materials developed will remain the copyright of UNICEF and UNICEF will be free to adapt and modify them in the future.

The project is estimated to use approximately 30 days of work. The proposal can deviate from this if justified, and we ask candidates to submit a lump sum professional fee (USD) that considers all the project deliverables and provides a high-level breakdown of how the budget will be used (by major activities). Please do not include travel fees as these will be reimbursed as and when they take place.

To qualify as an advocate for every child you will have…

• Master's degree in Economics, statistics, computer science, data science, evaluation, political science, public policy, business, and other relevant areas.
• At least three years of experience in the use and development of causal machine learning methods (five would be an asset).
• At least 10 years of experience in the development and application of causal inference methods broadly speaking.
• Experience in the use of causal machine learning methods for policy analysis and design.
• Experience with the structuring and use of government administrative records for statistical analysis.
• Experience in training and building others’ capacity to use statistics for causal inference.
• Experience in training governments in the use of causal machine learning would be an asset.
• Fluency in English is required.
• The ability to speak Russian or Kazakh would be an asset.

For every Child, you demonstrate…

UNICEF’s core values of Care, Respect, Integrity, Trust, Accountability, and Sustainability (CRITAS), and core competencies in Communication, Working with People and Drive for Results.

To view our competency framework, please visit here.

UNICEF is here to serve the world’s most marginalized children and our global workforce must reflect the diversity of those children. The UNICEF family is committed to diversity and inclusion within its workforce, and encourages all candidates, irrespective of gender, nationality, religious and ethnic backgrounds, including persons living with disabilities, to apply to become a part of the organization.

UNICEF has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. UNICEF also adheres to strict child safeguarding principles. All selected candidates will be expected to adhere to these standards and principles and will therefore undergo rigorous reference and background checks. Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check.

Remarks:

Please include a full CV and Cover Letter in your application. Additionally, indicate your availability and professional fee (in USD) to undertake the terms of reference above. Applications submitted without a professional fee will not be considered. Only shortlisted candidates will be contacted and advance to the next stage of the selection process.

Individuals engaged under a consultancy or individual contract will not be considered “staff members” under the Staff Regulations and Rules of the United Nations and UNICEF’s policies and procedures and will not be entitled to benefits provided therein (such as leave entitlements and medical insurance coverage). Their conditions of service will be governed by their contract and the General Conditions of Contracts for the Services of Consultants and Individual Contractors. Consultants and individual contractors are responsible for determining their tax liabilities and for the payment of any taxes and/or duties, in accordance with local or other applicable laws.

The selected candidate is solely responsible to ensure that the visa (applicable) and health insurance required to perform the duties of the contract are valid for the entire period of the contract. Selected candidates are subject to confirmation of fully-vaccinated status against SARS-CoV-2 (Covid-19) with a World Health Organization (WHO)-endorsed vaccine, which must be met prior to taking up the assignment. It does not apply to consultants who will work remotely and are not expected to work on or visit UNICEF premises, programme delivery locations or directly interact with communities UNICEF works with, nor to travel to perform functions for UNICEF for the duration of their consultancy contracts.

UNICEF offers reasonable accommodation for consultants with disabilities. This may include, for example, accessible software, travel assistance for missions or personal attendants. We encourage you to disclose your disability during your application in case you need reasonable accommodation during the selection process and afterwards in your assignment.

Advertised: W. Europe Daylight Time

Deadline: W. Europe Daylight Time