Vertex-reinforced random walk

TLDR: In this article, a class of non-Markovian discrete-time random processes on a finite state space is considered, where transition probabilities at each time are influenced by the number of times each state has been visited and by a fixed a priori likelihood matrix, which is real, symmetric and nonnegative.

Abstract: This paper considers a class of non-Markovian discrete-time random processes on a finite state space {1,...,d}. The transition probabilities at each time are influenced by the number of times each state has been visited and by a fixed a priori likelihood matrix,R, which is real, symmetric and nonnegative. LetS i (n) keep track of the number of visits to statei up to timen, and form the fractional occupation vector,V(n), where $$v_i (n) = {{S_i (n)} \mathord{\left/ {\vphantom {{S_i (n)} {\left( {\sum\limits_{j = 1}^d {S_j (n)} } \right)}}} \right. \kern-\nulldelimiterspace} {\left( {\sum\limits_{j = 1}^d {S_j (n)} } \right)}}$$ . It is shown thatV(n) converges to to a set of critical points for the quadratic formH with matrixR, and that under nondegeneracy conditions onR, there is a finite set of points such that with probability one,V(n)→p for somep in the set. There may be more than onep in this set for whichP(V(n)→p)>0. On the other handP(V(n)→p)=0 wheneverp fails in a strong enough sense to be maximum forH.