Solution Method – Stochastic Dynamic Programming



The following table details the important aspects of the Stochastic Dynamic Programming (SDP) formulation for the client’s problem.



Years                    t = 0, 1, 2, …, T



·        Competitors’ actions, {‘installation not yet begun’, ‘installing’, ‘resting’}

·        Number of smart meters installed by competitors so far

·        Number of old meters remaining to be replaced by competitors

·        Number of smart meters installed by the client so far

·        Number of old meters remaining to be replaced by the client – a pseudo-state, determined according to the values of the other states



Number of meters that the client will install in year t



Backwards (necessary for SDP because the next state in a path through the solution space is uncertain)



·        Smart meters installed so far by the client or competitors: 

·        Competitors’ actions:        

·        Number of old meters remaining for the client or competitors:
      [Dependent upon the rest of the state space]



The client’s installation capacity

Competitors’ installation capacity



Market share and expected annual growth for each party

Costs and benefits for old and smart meters

Relevant state space to search over is determined for each stage

Number of meters that competitors have installed (dependent upon states)



Includes:       Net benefit from old meters

                    Net benefit from smart meters

                    Discounted future reward term


Initial State

The client currently has no smart meters installed

Competitors have a known number of old meters to replace

Neither party has commenced installation


Final State

Many possible final states due to the uncertainty involved.

Taking an expectation of the final stage reward values will account for this.




A flowchart demonstrating how the MATLAB program solves this SDP is available here.


Return to Main Page