Solution Method – Stochastic Dynamic Programming

 

 

The following table details the important aspects of the Stochastic Dynamic Programming (SDP) formulation for the client’s problem.

 

Stages

Years                    t = 0, 1, 2, …, T

 

States

·        Competitors’ actions, {‘installation not yet begun’, ‘installing’, ‘resting’}

·        Number of smart meters installed by competitors so far

·        Number of old meters remaining to be replaced by competitors

·        Number of smart meters installed by the client so far

·        Number of old meters remaining to be replaced by the client – a pseudo-state, determined according to the values of the other states

 

Decision

Number of meters that the client will install in year t

 

Recursion

Backwards (necessary for SDP because the next state in a path through the solution space is uncertain)

 

Transition

·        Smart meters installed so far by the client or competitors: 

·        Competitors’ actions:        

·        Number of old meters remaining for the client or competitors:
      [Dependent upon the rest of the state space]

 

Constraints

The client’s installation capacity

Competitors’ installation capacity

 

Parameters

Market share and expected annual growth for each party

Costs and benefits for old and smart meters

Relevant state space to search over is determined for each stage

Number of meters that competitors have installed (dependent upon states)

 

Reward

Includes:       Net benefit from old meters

                    Net benefit from smart meters

                    Discounted future reward term

 

Initial State

The client currently has no smart meters installed

Competitors have a known number of old meters to replace

Neither party has commenced installation

 

Final State

Many possible final states due to the uncertainty involved.

Taking an expectation of the final stage reward values will account for this.

 

 

 

A flowchart demonstrating how the MATLAB program solves this SDP is available here.

 

Return to Main Page