core¶
 Authors: Chase Coleman, Spencer Lyon, Daisuke Oyama, Tom Sargent,
 John Stachurski
Filename: core.py
This file contains some useful objects for handling a finitestate discretetime Markov chain.
Definitions and Some Basic Facts about Markov Chains¶
Let \(\{X_t\}\) be a Markov chain represented by an \(n \times n\) stochastic matrix \(P\). State \(i\) has access to state \(j\), denoted \(i \to j\), if \(i = j\) or \(P^k[i, j] > 0\) for some \(k = 1, 2, \ldots\); \(i\) and j communicate, denoted \(i \leftrightarrow j\), if \(i \to j\) and \(j \to i\). The binary relation \(\leftrightarrow\) is an equivalent relation. A communication class of the Markov chain \(\{X_t\}\), or of the stochastic matrix \(P\), is an equivalent class of \(\leftrightarrow\). Equivalently, a communication class is a strongly connected component (SCC) in the associated directed graph \(\Gamma(P)\), a directed graph with \(n\) nodes where there is an edge from \(i\) to \(j\) if and only if \(P[i, j] > 0\). The Markov chain, or the stochastic matrix, is irreducible if it admits only one communication class, or equivalently, if \(\Gamma(P)\) is strongly connected.
A state \(i\) is recurrent if \(i \to j\) implies \(j \to i\); it is transient if it is not recurrent. For any \(i, j\) contained in a communication class, \(i\) is recurrent if and only if \(j\) is recurrent. Therefore, recurrence is a property of a communication class. Thus, a communication class is a recurrent class if it contains a recurrent state. Equivalently, a recurrent class is a SCC that corresponds to a sink node in the condensation of the directed graph \(\Gamma(P)\), where the condensation of \(\Gamma(P)\) is a directed graph in which each SCC is replaced with a single node and there is an edge from one SCC \(C\) to another SCC \(C'\) if \(C \neq C'\) and some node in \(C\) has access to some node in \(C'\). A recurrent class is also called a closed communication class. The condensation is acyclic, so that there exists at least one recurrent class.
For example, if the entries of \(P\) are all strictly positive, then the whole state space is a communication class as well as a recurrent class. (More generally, if there is only one communication class, then it is a recurrent class.) As another example, consider the stochastic matrix \(P = [[1, 0], [0,5, 0.5]]\). This has two communication classes, \(\{0\}\) and \(\{1\}\), and \(\{0\}\) is the only recurrent class.
A stationary distribution of the Markov chain \(\{X_t\}\), or of the stochastic matrix \(P\), is a nonnegative vector \(x\) such that \(x' P = x'\) and \(x' \mathbf{1} = 1\), where \(\mathbf{1}\) is the vector of ones. The Markov chain has a unique stationary distribution if and only if it has a unique recurrent class. More generally, each recurrent class has a unique stationary distribution whose support equals that recurrent class. The set of all stationary distributions is given by the convex hull of these unique stationary distributions for the recurrent classes.
A natural number \(d\) is the period of state \(i\) if it is the greatest common divisor of all \(k\)‘s such that \(P^k[i, i] > 0\); equivalently, it is the GCD of the lengths of the cycles in \(\Gamma(P)\) passing through \(i\). For any \(i, j\) contained in a communication class, \(i\) has period \(d\) if and only if \(j\) has period \(d\). The period of an irreducible Markov chain (or of an irreducible stochastic matrix) is the period of any state. We define the period of a general (not necessarily irreducible) Markov chain to be the least common multiple of the periods of its recurrent classes, where the period of a recurrent class is the period of any state in that class. A Markov chain is aperiodic if its period is one. A Markov chain is irreducible and aperiodic if and only if it is uniformly ergodic, i.e., there exists some \(m\) such that \(P^m[i, j] > 0\) for all \(i, j\) (in this case, \(P\) is also called primitive).
Suppose that an irreducible Markov chain has period \(d\). Fix any state, say state \(0\). For each \(m = 0, \ldots, d1\), let \(S_m\) be the set of states \(i\) such that \(P^{kd+m}[0, i] > 0\) for some \(k\). These sets \(S_0, \ldots, S_{d1}\) constitute a partition of the state space and are called the cyclic classes. For each \(S_m\) and each \(i \in S_m\), we have \(\sum_{j \in S_{m+1}} P[i, j] = 1\), where \(S_d = S_0\).

class
quantecon.markov.core.
MarkovChain
(P)[source]¶ Bases:
object
Class for a finitestate discretetime Markov chain. It stores useful information such as the stationary distributions, and communication, recurrent, and cyclic classes, and allows simulation of state transitions.
Parameters: P : array_like or scipy sparse matrix (float, ndim=2)
The transition matrix. Must be of shape n x n.
Notes
In computing stationary distributions, if the input matrix is a sparse matrix, internally it is converted to a dense matrix.
Attributes
P (ndarray or scipy.sparse.csr_matrix (float, ndim=2)) See Parameters stationary_distributions (array_like(float, ndim=2)) Array containing stationary distributions, one for each recurrent class, as rows. is_irreducible (bool) Indicate whether the Markov chain is irreducible. num_communication_classes (int) The number of the communication classes. communication_classes (list(ndarray(int))) List of numpy arrays containing the communication classes. num_recurrent_classes (int) The number of the recurrent classes. recurrent_classes (list(ndarray(int))) List of numpy arrays containing the recurrent classes. is_aperiodic (bool) Indicate whether the Markov chain is aperiodic. period (int) The period of the Markov chain. cyclic_classes (list(ndarray(int))) List of numpy arrays containing the cyclic classes. Defined only when the Markov chain is irreducible. Methods
simulate
(ts_length[, init, num_reps, ...])Simulate time series of state transitions. 
cdfs
¶

cdfs1d
¶

communication_classes
¶

cyclic_classes
¶

digraph
¶

is_aperiodic
¶

is_irreducible
¶

num_communication_classes
¶

num_recurrent_classes
¶

period
¶

recurrent_classes
¶

simulate
(ts_length, init=None, num_reps=None, random_state=None)[source]¶ Simulate time series of state transitions.
Parameters: ts_length : scalar(int)
Length of each simulation.
init : scalar(int) or array_like(int, ndim=1),
optional(default=None)
Initial state(s). If None, the initial state is randomly drawn.
num_reps : scalar(int), optional(default=None)
Number of repetitions of simulation.
random_state : scalar(int) or np.random.RandomState,
optional(default=None)
Random seed (integer) or np.random.RandomState instance to set the initial state of the random number generator for reproducibility. If None, a randomly initialized RandomState is used.
Returns: X : ndarray(int, ndim=1 or 2)
Array containing the sample path(s), of shape (ts_length,) if init is a scalar (integer) or None and num_reps is None; of shape (k, ts_length) otherwise, where k = len(init) if (init, num_reps) = (array, None), k = num_reps if (init, num_reps) = (int or None, int), and k = len(init)*num_reps if (init, num_reps) = (array, int).

stationary_distributions
¶


quantecon.markov.core.
mc_compute_stationary
(P)[source]¶ Computes stationary distributions of P, one for each recurrent class. Any stationary distribution is written as a convex combination of these distributions.
Returns: stationary_dists : array_like(float, ndim=2)
Array containing the stationary distributions as its rows.

quantecon.markov.core.
mc_sample_path
(P, init=0, sample_size=1000, random_state=None)[source]¶ Generates one sample path from the Markov chain represented by (n x n) transition matrix P on state space S = {{0,...,n1}}.
Parameters: P : array_like(float, ndim=2)
A Markov transition matrix.
init : array_like(float ndim=1) or scalar(int), optional(default=0)
If init is an array_like, then it is treated as the initial distribution across states. If init is a scalar, then it treated as the deterministic initial state.
sample_size : scalar(int), optional(default=1000)
The length of the sample path.
random_state : scalar(int) or np.random.RandomState,
optional(default=None)
Random seed (integer) or np.random.RandomState instance to set the initial state of the random number generator for reproducibility. If None, a randomly initialized RandomState is used.
Returns: X : array_like(int, ndim=1)
The simulation of states.