PakistanMacro Models for Policy
BACKGROUND:
At the request of Minister Finance Ishaq Dar and Minister Planning Ahsan Iqbal, a conference on “Macro Models for Policy and Planning” was jointly organized by PIDE and NIBAF, on Oct 22, 2016, with wide participation from several ministries and academic institutions. Video Recordings of the day long seminar, papers presented, and related materials, are available from: Macro Models for Policy [shortlink: bit.do/pidemm4p]. This post presents an Executive Summary of major findings and recommendations which emerged from the seminar.
The goal was to assess the current state of affairs, in terms of what models are currently in use for policy making, and to get some ideas of how to move forward. That is, how can we improve models currently in use, and how we can spread the use of empirical and evidence based planning more widely. As a result of the presentations and the discussions, the following points emerged as the action agenda for further progress:
- Models for the output gap are available, and in use, at the Finance Ministry, State Bank of Pakistan, and in the academia (PIDE-QAU). Each group is working in isolation, and is unaware of the existence of others working on the same problem. The academia is unaware of the needs of the policy-makers, while the policy makers are in general unaware of the need of models, and how current models could be helpful to their policy decisions.
- In general, models are not used much, and not trusted, in policy decisions. People prefer to use intuition and experience, rather than formal models and empirical evidence. This is due to two factors:
- Flaws and errors in existing models
- Misconceptions regarding models and how to use them for policy.
A substantial amount of training is required on BOTH sides – academics need to learn about how to orient theoretical models for practical use, and policy makers need to learn how to utilize formal model and evidence based procedures to help improve decision making. Some of the key obstacles to progress uncovered are discussed below, together with recommendations on how to overcome these obstacles.
MAJOR MISCONCEPTION: We have choice between using models and NOT using models.
This idea is wrong. Whenever we make plans for the future, or policy decisions, we MUST use models to forecast the likely outcomes and compare them. The choice is always between explicit and articulated models, versus intuitive and hidden models which are not clearly stated. Experience and seat of the pants decisions rely on implicit understanding in the minds of the decision makers which are not made clear. Such policy and decision making processes are extremely personalized, erratic, and hit-or-miss. This informal method of policy and decision making, currently in use everywhere in Pakistan, has the following problems:
- Since the reasons for decision making are not articulated, we can never tell if they were right or wrong.
- There is no possibility of making progress, or learning from past errors.
- Decision making depends on judgment calls, which varies from person to person, and is highly erratic.
- There is no institutional arrangement which puts procedures into place to protect from individual errors and arbitrary decision making for unknown reasons. Institutional growth, learning from past experience, does not take place.
Use of articulated reasons for policy making is the essence of transparency. Whenever a decision is made, it should explicitly and clearly articulate the REASONS for this decision. Such reasons will include forecasts on what is expected to happen when the policy is enacted, and what would go wrong if other decisions are taken. This is currently not done for the obvious reason that if we provide such articulation, we could easily be held to task for mistakes. All the bureaucratic incentives go in the opposite direction – make decisions without explicitly explaining the reasons for the decisions.
RECOMMENDATIONS:
The institutional structure, including bureaucracy and academia, does not encourage or incentivize evidence and model based decision making. Decisions made arbitrarily give freedom to decision makers, do not penalize errors, do not create learning from experience, create dependency on personalities, and do not embed knowledge into institutional structures. Because of this, there is a huge potential for improved policy and decision making in all ministries and departments, but realizing this potential would require coordinated efforts on several fronts, working against inertia, and also changing incentive structures to reward good evidence based decision making.
Recommendation 1: Put in institutional structure to REQUIRE articulation of reasons for decisions. Recorded minutes of policy decision making should require clear expression of expected beneficial outcomes – these are always forecasts based on models, either implicit or explicit. Encouragement should be provided to policy makers to express clearly areas of ignorance – things they would like to know about how the real world works – in order to make better informed policy decisions. It is this expression of ignorance which will create the demand for research.
Recommendation 2: Ask policy makers to express clearly what information they need in order to make better decisions. For example, when contemplating effects of a export promotion strategy, data on previous parallel episodes and their outcomes, as well as theoretical models which provide guidance on predicted effects would be relevant and important to informed decision making. Policy makers should be encouraged to write down the hunches and guesses that they use to guide their decision making. These conjectures can then later be tested using empirical evidence. For instance, one might guess that depreciation of the Rupee would lead to improvement in the Balance of Payments, increased volume of exports, reductions in imports, but the empirical evidence presented at the conference shows that all of these conjectures are wrong.
Recommendation 3: Ask academicians to clearly and simply explain the reasoning behind the policy recommendations made using models. This should bring out and highlight the assumptions behind the model. The current black box approach is harmful, because effectively, academicians are saying – trust us, we are the experts. However, poor performance of models in crises does not lead to confidence in their performance. Academicians need to open up the black box, and explain the reasoning behind their recommendations in sufficiently simple terms so that field experts with knowledge of the real world can understand (and perhaps reject) that reasoning. Current practice does not create confidence. Modellers say that this is the policy recommendation of my model, but all models are wrong, so take it or leave it. They have no skin in the game. They must take responsibility for their recommendations and also explain the logic of the recommendations, instead of putting responsibility on a mechanical model, and thereby avoiding personal responsibility for errors.
Recommendation 4: At the moment, it seems counterproductive to use complex & sophisticated models. This will increase gap and distrust between academia and practitioners. Both sides need to change their practices to bridge the gap and create communication. On the academic side, this involves moving toward simple models based on short causal chains with convincing empirical evidence, rather than fancy black box models.
Recommendation 5: Models have a negative aspect: they inhibit out of the box creative thinking lying outside the range of past experiences. Google has created continuous innovations by using the Google 20% policy, which gives employees one day a week to freely innovate on their own, without any guidance. We also have a huge amount of knowledge, experience and talent bottled up inside long time employees, which remains unutilized due to bureaucratic constraints. We should encourage creative expression of new ideas from experiences and new junior staff from time to time. This will create ownership, generate innovative solutions, and allow for learning from experience.
SPECIFIC RECOMMENDATIONS for Participating Ministries and Organizations
Ministry of Finance: Budgetary issues are of the greatest importance. Projections of Government Expenditure categorized appropriately, as well as Tax Revenues are of key importance. Foreign Exchange needs to be tracked separately, creating a need for models of Exports and Imports. However, this is all basic accounting. For a pro-active approach to management, one needs to creatively assess cost-benefit ratios in different sectors, including sectors not currently in existence. This requires a much higher order of planning. For instance the Korean government created a semi-conductor industry from scratch – there was no way to do accounting for this sector which did not exist. Similarly, the Ministry of Finance in Japan guided the development of Japan by providing window guidance to the banks, directing the flow of investments to different sectors according to their priority in development. There are many ways to systematize thinking in this way, to get pro-active proposals to direct investment in the future.
Ministry of Planning: The largest task is the approval (or dis-approval) of projects requiring financing generated elsewhere. This is a reactive task, responding to demands. A systematic shift to pro-active mode would require some targets for different sectors, and then contacting relevant parties to generate proposals for approval in the target sectors. While this is already being done in an informal way, the relevant cost-benefit analysis could identify the sectors which would generate the greatest pay-offs. To some extent, this planning process could be aided by suitable models, and relevant data. MoPD&R could then provide the relevant targets to Ministry of Finance, which is in a better position to execute these actions. MORE DETAILED recommendation for MoPD&R are available from MPDR Seminar: Improving Planning and Policy
State Bank of Pakistan: Conduct of Monetary Policy requires a fairly good understanding of the mechanisms by which money affects the economy. Unfortunately, many popular theories used for guidance in this respect are empirically invalid. For example, it is believed that lowering exchange rate will worsen balance of payments, but empirical evidence presented at the seminar showed otherwise. It is believed that increasing interest rates will check inflation but empirical evidence presented at the seminar strongly suggests otherwise. General trend in monetary policy is shifting from Keynesian demand management towards Monetary Inflation Targeting; however this shift is not supported by the empirical evidence. We should strive to make evidence based policy grounded in empirical realities; this will require substantial hard work in many dimensions – improvements in models, improvements in data, improvement in understanding of the data and models among the members of the MPC. More intensive training of MPC members in the history and practice of monetary policy, as well as the global experience is required. The documentation of reasons for Monetary Policy Decisions is an excellent step in the direction of transparency, and in line with the recommendations being made here.
Pakistan Bureau of Statistics: As different departments shift towards evidence based policy making, they will demand data required for the empirical verification of affects of their policies. The PBS should be able to provide targeted results. For example, evaluations of all large scale government programs and projects should be within the scope of PBS. Planning Commission is currently concerned with the problem that the long term evaluation of projects, after completion, is not being carried out at all. That is because there is no interest on part of the parties who executed the project to turn in the PC-V which is for the long term evaluation. This problem cannot be fixed by punishment – that is, by denying funding to new project to parties which have not turned in the PC-V. This is because the incentives are not aligned. Any party which does its own appraisal, will have incentive to distort the evaluation. Instead, when the project is assigned, some suitable amount of money (like 10%) must be set aside to be paid to independent auditors, which could be engineers or academics, for evaluation of the project. Without such evaluations, Planning Commission is operating in the dark. With independent third party evaluations paid for by the Planning Commission itself, useful information is likely to emerge.
Ministry of Commerce: Systematic planning requires some amount of forecasting, hand-holding, providing support along different types of dimensions. The idea of free trade is highly misleading. Most industries have developed under protection – the famous infant industry arguments. However, the protection has to be strategically withdrawn, so as to ensure that the industry grows up, and does not remain an infant. There are many other imaginative strategies which can used, like bilateral or trilateral trade agreements, which can save foreign exchange, which is the crucial objective of trade. A deal with Iran, or with Iran and Afghanistan, in terms of oil versus some commodity of use to Iran, could by highly mutually beneficial, and seems more politically feasible now than it was before. Pro-active planning requires more than models, but models can provide some guidance regarding costs and benefits and evaluate and prioritize among different possible mechanisms and schemes. In these calculations, the academia can provide valuable support to the Ministry.
PIDE-QAU and HEC: Incentives for academics are not aligned with the need to produce high quality research useful for applications. The race to publish in impact factor journals is counter-productive in many ways. Research on genuine local Pakistani problems is unlikely to be of interest to high impact factor international journals. Do we want to put our best brains to work on solving European and American problems which will get published in their journals. We should develop our own indigenous impact factor, not based on publication, but based on application. If research is credited with providing policy directions and contributing to further the discussion on relevant and genuine problems of Pakistan, it should receive recognition and reward. To do this will require out-of-the-box thinking, and leadership, since I am not aware of models which could be directly used for this purpose.
CONCLUDING REMARKS
There are no built in incentives in the system to change for the better. Furthermore, there is a lot of potential to dramatically improve the system of decision making currently in place. Only one small part involves greater use of systematic modeling procedures. In fact, realizing this potential will require hard work and strategic interventions in many dimensions. Many of these interventions have already been proposed by the Ministry of Planning – in terms of improving management using KPIs and the like. This external structure or body of the reforms needs to supplemented and supported by the spirit, which is based on motivating people to selflessly serve the nation. The creation of this spirit will produce the energy and drive required to overcome the resistance which is expected to any effort for reforms.
Following talk in URDU, given at MoPD&R on 22 Feb 2017, discusses and motivates this bedrock requirement for transformational change: