EMRG 1995-1997 Activity Report
This report covers two years of activity by the Energy Modelling Research Group, within the Department of Management at the University of Canterbury. Again, the group has been sponsored by the Electricity Corporation of New Zealand, and Trans Power (NZ) Ltd, throughout this period, and we would like to express our appreciation to both organisations for their generous support.
During the current period, two more Ph.D. candidates, Brendan Ring and Glenn Drayton-Bright, have completed successfully, and Tristram Scott has submitted, while Stephen Batstone has joined the group as a Ph.D. candidate. Dr Bruce Lamar has moved on, and not yet been replaced, as have two research assistants, Chris Wallace and Brett Graydon. On the other hand, Tristram Scott remains with us as a research associate, and Andrew Kerr as a research assistant. Thus, the group is now somewhat smaller, with Grant Read and John George remaining as the academic staff responsible for leading a group consisting of 3 full time students and 2 research staff, with the likelihood of a well qualified research fellow being recruited in the near future.
The external environment has changed significantly in this period, too, with the establishment of the New Zealand spot market marking a major milestone in local developments. Thus we are shifting from a period of intense research into how theoretical markets might behave, into an era where interested parties must cope with the realities of an actual market. We believe that the research carried out here has had an important, and beneficial, influence on the development of this new environment, but also expect that "research" activities will now be increasingly performed by commercial parties and consultancies in a real word context. In fact, we note with pleasure the contribution which our own graduates are already making in this regard. Conversely, we are comfortable with the reduced size of our own group in this new environment.
It is evident, too, that the kind of reforms which have been researched, and to some extent pioneered, here are being adopted with increasing momentum elsewhere. Thus, group members have been extensively involved in both electricity and gas market developments in Australia, for example. Further, there has been significant international interest in the New Zealand experience, and ECNZ sponsorship has enabled active participation in the activities of the Energy Modeling Forum at Stanford University. Dr Read, in particular, has contributed papers on the New Zealand experience to several international forums, including - below.
The remainder of this report provides a brief overview of research in a number of specific areas, followed by a brief listing of people involved in the group. Relevant papers are listed in each section, and a list of working papers is appended. Please note, though, that much of the group's recent work is contained in contract reports, which have not been listed for reasons of confidentiality.
- E. G. Read, OR Modelling for a Deregulated Electricity Sector. International Transactions in Operations Research, 3(2), 1996, p. 129-138.
- J. G. Culy, E. G. Read, and B. D. Wright, Structure and Regulation of the New Zealand Electricity Sector. In R. Gilbert and E. Kahn (editors), International Comparison of Electricity Regulation, Cambridge University Press, 1996, p. 312-365.
- E. G. Read, Lessons from Modelling the New Zealand Electricity Market. Invited Presentation to the Energy Modeling Forum WG15, Washington DC, 1996.
- E. G. Read, Transmission Pricing for Operational and Investment Signaling: A New Zealand Perspective. IEEE PES Winter Conference, New York, 1997.
- E. G. Read, Transmission Pricing in New Zealand. Utilities Policy, 6(3), 1997, p. 227-236.
- E. G. Read, Electricity Sector Reform in New Zealand: Lessons From the Last Decade. Pacific Asia Journal of Energy (to appear).
Brendan Ring's doctoral thesis, accepted in 1996, provided the primary focus of our research in this area. As noted in the abstract:
"This thesis investigates the application of marginal cost based spot pricing techniques to the short run coordination of decentralised, and potentially competitive, electricity markets. A Dispatch Based Pricing philosophy is proposed which requires that the dispatcher of a power system determine spot prices which are consistent with both the observed power system dispatch and the offers and bids issued by market participants. Whereas previous research has involved determining prices corresponding to an optimised power system dispatch, Dispatch Based Pricing is more flexible, requiring no such optimality assumption while generating incentives which encourage efficient dispatch. Pricing relationships are formed, and the resulting incentives analysed, by applying duality theory to mathematical programming formulations of the dispatch problem. A detailed theoretical description of a dispatch based pricing model, based on an Optimal Power Flow formulation is presented. This model is an extension of the ex post pricing model of Hogan (1991). As well as presenting a more general representation of dispatch variable relationships, we demonstrate the underlying mathematical relationships which drive the economic interpretation of this model. In addition, we explore the behaviour of transmission flow constraints in cyclic networks, and describe the modifications needed to price for security requirements consistent with current operational practices in New Zealand. We explore the extension of Dispatch Based Pricing to situations beyond the scope of the Optimal Power Flow problem, and even to situations which are strictly incompatible with a pure marginal cost based analysis. We develop a "best compromise" pricing approach which, for (seemingly) economically inconsistent dispatches, minimises the side payments required to account for the difference between the market clearing spot prices and the offers and bids of the market participants. We develop and discuss methods for determining dispatch based prices which are consistent with primal inter-temporal constraints, uncertainty, and integer variables."
With Dr Ring's departure, the group's research work in this area has now largely ceased, although group members have been significantly involved with assisting markets organisations and participants in coming to grips with assessing, and implementing, the theory developed here and elsewhere. Since our last report, though, several papers have appeared in this area. Apart from  and  above, these include , , and .
- B. J. Ring, Dispatch Based Pricing in Decentralised Power Systems. Ph.D. Thesis, Department of Management, University of Canterbury, 1995.
- B. J. Ring and E. G. Read, Short Run Pricing in Competitive Electricity Markets. Journal of the Canadian Economics Association, Special issue XXIX, 1996, p. S313-315.
- W. W. Hogan, E. G. Read, and B. J. Ring, Using Mathematical Programming for Electricity Spot Pricing. International Transactions in Operations Research, 3(3-4), 1996, p. 243-253.
Tristram Scott submitted his Ph.D. entitled "Hydro Reservoir Management for an Electricity Market with Long-term Contracts" in 1997 . To quote from the abstract:
"This Thesis deals with the management of a mixed hydro-thermal system in a competitive electricity market. A notable feature of our market is the presence of long term financial contracts, or options. We model the energy spot market as a Cournot oligopoly, with a non-competitive fringe. The data from the Cournot model is used in an optimisation model based on Dual Dynamic Programming (DDP). The optimisation model produces operating rules in the form of a marginal water value surface, and these rules guide our medium term simulation model. We develop a method for using the Cournot model to produce Demand Curves for Release, which describe the amount of water the hydro manager would want to release in a given period for a range of marginal water values (prices). We show how DDP can be thought of as a process of adding demand curves over time, equating marginal costs between periods. We find that the efficiency of the market is greatly influenced by the size of the contracts, and to a lesser extent by the portfolio of plant that each of the firms has. Increasing contracts lead to increasing output, decreasing spot prices, decreasing profit, increasing consumer surplus, decreasing marginal water values, and increasing storage trajectories. With appropriate choice of contracts the market can be made to mimic perfect competition."
One paper has now appeared on this work , and Tristram is currently working on several papers describing the main areas from his thesis. Among other things, he is also working on the application of Fourier Transforms to the stochastic elements of DDP, and on the extension of the Cournot energy spot market model from a single node to two.
- T. J. Scott, Hydro Reservoir Management for an Electricity Market with Long-term Contracts. Ph.D. Thesis, Department of Management, University of Canterbury, 1997.
- T. J. Scott and E. G. Read, Modelling Hydro Reservoir Operation in a Deregulated Electricity Sector. International Transactions in Operations Research, 3(3-4), 1996, p. 209 221.
Glenn Drayton-Bright's thesis  was submitted in 1996 and accepted in 1997. It built on his earlier work with Bruce Lamar on optimising spinning reserve, and was significantly assisted by experiments conducted by Brett Graydon. As stated in the abstract:
"This thesis addresses a number of questions related to the design of a wholesale electricity market and the decentralisation of a mixed hydro and thermal system.Initially it concentrates on the response to price of a Linear Programming model of a hydro station and existence of a step supply curve consistent with that function. This has implications for the existence of the perfect competition equilibrium in a simplified energy market. An experimental analysis is presented, which attempts to quantify the theorised discrepancy between an 'ideal' centrally coordinated solution and the market's solution.The latter half of this thesis develops a Linear Programming based representation of the joint energy and reserve capability of a generating unit or station, called the Fan Approximation. This approach is used to develop an offering and market-clearing model for energy and reserves which allows hydro, thermal, and interruptible load participants to compete equally."
This work has already been incorporated into the New Zealand market, and is under consideration elsewhere. It has so far been reported to the Operational Research Society of New Zealand (ORSNZ)  and Institute for Operations Research and the Management Sciences (INFORMS) conferences (,).
- G. R. Drayton-Bright, Coordinating Energy and Reserves in a Wholesale Electricity Market. Ph.D. Thesis, Department of Management, University of Canterbury, 1997.
- G. R. Drayton-Bright and E. G. Read, An Integrated Approach to Modelling Power Station Energy and Reserve Dispatch. In Proceedings of 32nd Annual ORSNZ Conference, August 1996, p. 119-124.
- G. R. Drayton-Bright and E. G. Read, An Integrated Framework for Energy and Reserve Market Offers. INFORMS Conference, San Diego, 1997.
- G. R. Drayton-Bright and E. G. Read, An Integrated Approach to Modelling Power Station Energy and Reserve Capability. INFORMS Conference, San Diego, 1997
The main focus of the research in this area has been on testing the performance of heuristics based on common-sense and managerial insights. The basic ideas behind these heuristics is presented in , which was also presented at the 1996 ORSNZ Conference . In , two heuristics were tested in a deterministic setting and using the integer programming model described in  and . (That IP model has also been extended to handle reserve, but this has not been written up as it is essentially the same as the approach taken by Glenn Drayton-Bright in his modelling of generation and reserve). The SAM heuristics focus on the peaks and troughs of the price/target profile, while the PI heuristics integerise the immediate periods, and relax the integer restriction for later periods. These heuristics were compared to the fully integer IP as well as to some naïve heuristics, such as simply rounding the relaxed IP solution and stopping the branch and bound once some tolerance limit, in terms of the objective function value, was reached. Variants of SAM and PI performed very well, in the sense that solution times were lower than for the full IP and objective values were not too dissimilar (in many cases they were identical).
An update of  is being prepared for submission to an academic journal, and a joint paper may be written on the performance of various heuristic approaches to this problem. This area is now receiving less attention due to the perception that the need for heuristic approaches to reducing due to improvements in solution time as a result of upgrades to general purpose MIP software, while decentralised hydro system management seem satisfied with current scheduling methods.
- J. A. George, E. G. Read, R. E. Rosenthal, and A. L. Kerr, Optimal Scheduling of Hydro Stations: An Integer Programming Model. EMRG Working Paper EMRG-WP-95-07, Department of Management, University of Canterbury, 1995.
- J. A. George, E. G. Read, R. E. Rosenthal, and A. L. Kerr, Optimal Scheduling of Hydro Stations: An Integer Programming Model (GAMS Model and Data). EMRG Working Paper EMRG-WP-95-08, Department of Management, University of Canterbury, 1995.
- J. A. George, E. G. Read, R. E. Rosenthal, Unit Scheduling in a Hydro-Electric Power System. INFORMS Conference, Los Angeles CA., 1995.
- A. L. Kerr and E. G. Read, Short-term Hydro Scheduling Using Integer Programming: Management and Modelling Issues. In Proceedings of 32nd Annual ORSNZ Conference, August 1996, p. 111.
- A. L. Kerr, Hydro Scheduling Heuristics: Implementations of SAM and PI Heuristics in a Deterministic Integer Programming Framework, System/Data Descriptions, and GAMS Code Listing. EMRG Working Paper EMRG-WP-97-01, Department of Management, University of Canterbury, 1997.
- A. L. Kerr and E. G. Read, Short-term Hydro Scheduling Using Integer Programming: Management and Modelling Issues. EMRG Working Paper EMRG-WP-97-02, Department of Management, University of Canterbury, 1997.
Grant Read has worked with John Kaye and Shantha Ranatunga (University of New South Wales, Australia), to develop an approach to determine optimal decisions over time using stochastic dynamic programming when the decision maker has some non-linear preference/value function for end-of-horizon outcomes. The technique was applied for a (risk averse) decision maker purchasing and selling forward contracts to hedge against future price uncertainty. By maximising the expected utility of the total benefit stream, adverse total benefit outcomes (over a discrete finite decision horizon) are averted. The non-separability of the objective function caused by utility being non-linear is handled by augmenting the state vector to include a running total of accumulated benefits (wealth).
EMRG have applied that technique to the problem of medium-term reservoir management with inflow uncertainty. The formulation and algorithm are essentially the same as originally developed, with modifications required due to the nature of the reservoir scheduling problem. We have investigated the effect of risk aversion in the context of a traditional cost minimisation approach , with several algorithmic modifications having been identified which can significantly reduce the execution time of the optimisation.
It was observed that applying a revenue maximisation approach would implicitly involve optimisation of "gaming" as well as risk aversion, and therefore we can consider at the situation where the reservoir operates independently in the market and can influence the price received for its spot market generation, assuming that the other players are perfectly competitive price takers. A natural extension to this approach is to allow the company to own a thermal station(s) as well. Other areas of interest include a multiple reservoir formulation, different forms of utility function, optimising contract position, incentive alignment, time scales of risk aversion, different market structures, and gaming. Due to the interest and progress made on this topic, Andrew Kerr will be pursuing his Ph.D. in the area.
- R. J. Kaye, A. L. Kerr, and E. G. Read, Stochastic Dynamic Programming Applied to Medium-term Reservoir Management: Utility Maximisation for a Cost Minimiser. EMRG Working Paper EMRG-WP-97-03, Department of Management, University of Canterbury, 1997.
There are a number of aspects of the electricity supply and distribution infrastructure that can be modeled as network flow problems. Examples include the flow of water through a river chain system, and the flow of electrical power in a transmission grid. Many problems however are complicated by additional restrictions to the problem, and/or economies or diseconomies of scale in the costs associated with the network. Non-convex network optimisation techniques are a promising approach to modelling such problems.
Gavin Bell, in his Ph.D. supervised by Bruce Lamar, has been studying theoretical and algorithmic approaches for solving non-convex optimisation problems. He has developed a theoretical framework for analysing these types of problems. Information obtained from this approach can be used to develop additional techniques that greatly improve the speed of traditional algorithms. His Ph.D. uses this framework to enhance standard "capacity improvement" techniques and develop tighter conditional penalties for this class of problems. Gavin has presented this work at the 31st and 32nd ORSNZ Conference, the later for which he was awarded the ORSNZ Young Practitioner Prize (,). Gavin expects to complete his Ph.D. by early 1998.
In addition to this work, Bruce Lamar has been developing a second extension of capacity improvement, called generalised capacity improvement. Bruce has also developed methods of converting arbitrary nonconvex functions into what are known as "difference of convex" functions that are easier to handle mathematically. This work and the work on "enhanced" capacity improvement has been published and presented internationally (,,,,,).
EMRG has also continued its involvement in the development of NetSpeak, an algebraic modelling language for this class of network flow problems. Versions of NetSpeak have been developed for MS-DOS, Windows 3.1, and Unix operating systems, with the Unix version being used by Gavin Bell to model and solve test problems in his thesis. Bruce has presented NetSpeak to a variety of audiences both nationally and internationally (,,).
- G. J. Bell, A New Penalty for Concave Minimisation over a Polytope. In Proceedings of 31st Annual ORSNZ Conference, August 1995, p. 127-134.
- G. J. Bell, Solution and Application of Nonconvex Network Flow Problems. In Proceedings of 32nd Annual ORSNZ Conference, August 1996, p. 63-68.
- G. J. Bell and B. W. Lamar, Solution Methods for Nonconvex Network Optimization Problems. In P.M. Pardalos, D.W. Hearn and W.W. Hagar (editors), Network Optimization, (Lecture Notes in Economics and Mathematical Systems (450), Springer, Berlin, 1997.
- G. J. Bell and B. W. Lamar, Solution Methods for Nonconvex Network Optimization Problems. Conference on Network Optimization, Center for Applied Optimization, University of Florida, Gainesville FL., February 1996.
- G. J. Bell, B. W. Lamar, and C. A. Wallace, Capacity Improvement, Penalties, and the Fixed Charge Problem. Submitted to Naval Research Logistics.
- B. W. Lamar, Nonconvex Optimization over a Polytope Using Generalized Capacity Improvement. State of the Art in Global Optimization Conference, Princeton NJ., USA, April 1995.
- B. W. Lamar, NETSPEAK: A Mathematical Programming Language for Nonconvex Network Optimization Problems. INFORMS Conference, New Orleans, USA, October 1995.
- B. W. Lamar, Nonconvex Network Optimization: Applications, Algorithms, and Software. Department of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta GA., USA, February 1996.
- B. W. Lamar, and C. A. Wallace, NetSpeak: An Algebraic Modeling Language for Nonconvex Network Optimization Problems. Conference on Network Optimization, Center for Applied Optimization, University of Florida, Gainesville FL., February 1996.
- B. W. Lamar, An Exact Method for Converting Arbitrary Functions into Difference of Convex Functions. INFORMS Conference, Atlanta GA., November 1996.
- B. W. Lamar, and C. A. Wallace, NetSpeak: A Mathematical Programming Language for Nonconvex Network Optimisation Problems. In Proceedings of 31st Annual ORSNZ Conference, August 1995, p. 141-148.
Olly McCahon is writing up his Ph.D. and plans to submit in 1998. This work applies a multi-objective optimisation approach to scenario analysis by treating the objective function under each scenario as one of a set of multiple objectives. The aim of this approach is to relax four major assumptions that must be made if a scenario problem is to be solved using stochastic optimisation:
- That the scenarios can be assigned probabilities. This assumption is difficult to support in the strategic planning arena with its long time frames, and the need to consider events which have no historical precedents.
- That the decision maker is risk neutral. This is not always appropriate, especially for one-off problems in which the situation will not recur. This means that only one outcome can occur, and the result of the decision cannot average out over many realisations of the uncertainty.
- That all significant aspects of the problem have been quantified and included in the model. Without this assumption, the optimal solution produced by the model cannot be assumed to be the optimal solution to the real problem.
- That the objectives of all scenarios can be brought to the same units. This assumption is required for the creation of the stochastic optimisation objective function, which is a weighted average of the scenario objectives.
By formulating a scenario analysis problem as a multi-objective optimisation problem, a set of trade-off efficient solutions can be produced for the decision maker to choose among. This enables the decision maker to express risk preferences directly by his/her choice of decision to implement, and enables the decision maker to include nonquantifiable concerns when making that choice. The generation of an efficient set does not preclude the use of scenario probabilities, instead it enables the decision maker to see how the optimal expected value solution changes as the scenario probabilities are changed.
The theory has been developed for solving problems with continuous variables, and two and three scenarios, and an example code written. This code has been used to solve some small example problems taken from the stochastic programming literature. A branch and bound algorithm has also been developed and coded to solve problems in which the stage one variables are binary decisions, and the stage two variables are continuous. The binary decisions model major strategic decisions, such as which of several possible power stations should be built, and the continuous variables model the operating decisions that will be available after the stage one decisions have been implemented. This algorithm can handle any number of scenarios.
Stephen Batstone has continued to investigate various approaches to optimising a firm's level of contracting as a way of hedging risk . A large part of this work has looked at using results and data from Tristram Scott's Cournot model. This model has the level of contracts fixed, and performing multiple runs with different contracted amounts allows us to look at the nature of various functions with respect to contracts (e.g., profit, spot price, storage, variance of profit etc).
It has become apparent that the nature of these functions in the context of Tristram's analysis has rendered many classical approaches to risk inappropriate. For example, traditional mean-variance utility techniques are based on the assumption that variance of, for example, profit, increases monotonically with the mean. Tristram's results show clearly that this is not the case, at least in the context which he has modelled. Stephen is now investigating ways of handling these "bubbles" of variance.
Stephen's and Andrew's work is closely associated, and they plan to work together on some of the issues relevant to both.
- S. R. J. Batstone, Risk Aversion in a Hydro Electricity Market: Gaming, Storage and Contracts. Ph.D. Proposal, Department of Management, University of Canterbury, 1996.
- E. Grant Read
Grant, who leads the EMRG research team, is a part time Senior Lecturer in the Department of Management, and also a private consultant. In the latter role he currently acts as Principal Consultant with CORE Management Systems and Senior Advisor with Putnam Hayes and Bartlett. After originally specialising in hydro power system modelling, for the last ten years Grant has concentrated on restructuring and market issues, having been a key advisor to the Electricity Sector Task Force and the Wholesale Electricity Market Study in New Zealand, and undertaken consultancies in Europe, Australia and South America.
- John A. George
John is an Associate Professor of Management Science and Head of Department of Management. He is responsible for the administration of EMRG. His research areas include Mathematical Programming models and Heuristic methods, and he has been involved with EMRG projects using Integer Programming and DDP.
- Bruce W. Lamar
Bruce was on the staff of the Department of Management until June 1997 as a Senior Lecturer in Management Science. He has returned to the USA to work for MITRE Corporation. He specialised in Network Optimisation and Mathematical Programming, especially non-convex networks. Bruce is continuing to develop NetSpeak, a network programming language.
- Stephen R. J. Batstone
Stephen is a doctoral student researching the effects of forward contracts on risk management for an electricity generator. He expects to complete his thesis by mid 1999.
- Gavin J. Bell
Gavin's Ph.D. involves theoretical and algorithmic approaches for non-convex network optimisation. These methods will then be applied to model the non-physical loss problem in short-term LP dispatch models. His thesis is due for completion in 1998.
- Glenn R. Drayton
Glenn has been involved in several areas of work in electrical dispatch optimisation. His doctoral research studied the coordination of hydro and thermal stations in a de-centralised electricity market. He completed his thesis in early 1997 and is now employed by Putnam, Hayes, and Bartlett in Wellington.
- Andrew L. Kerr
Andrew is a research assistant and part-time Ph.D. student. Switching from his initial topic of optimising unit commitment, he is now using Stochastic Dynamic Programming, to study the impact of decision maker attitudes towards risk on the way that hydro and thermal plant are operated, as well as on overall system performance. He continues to be employed as a research assistant on various projects.
- Olly C. McCahon
Olly's work applies techniques from multi-objective optimisation to the problem of decision making under uncertainty, when the uncertainty is represented as a small number of scenarios. Olly plans to complete his thesis in 1998.
- Brendan J. Ring
Bendan's doctoral research concentrated on developing pricing models for realistic AC power flow formulations. After graduating in 1996, Brendan left EMRG to join the Wellington office of Putnam Hayes & Bartlett.
- Tristram J. Scott
Since submitting his thesis on the behaviour of hydro generators in competitive electricity markets in 1997, Tristram has continued his involvement as a Research Associate with EMRG, developing extensions of his models, working on papers, and performing specific studies.
- Brett Graydon
Brett, who assisted Glenn extensively with market simulation experiments, has now re-joined the Wellington office of Putnam Hayes & Bartlett.
- Chris A. Wallace
Chris, who assisted Bruce and Gavin in their work on non-convex optimisation, is now pursuing a Ph.D. at the University of Auckland.