- We study subproduct systems in the sense of Shalit and Solel arising from stochastic matrices on countable state spaces, and their associated operator algebras. We focus on the non-self-adjoint tensor algebra, and Viselter's generalization of the Cuntz–Pimsner C*-algebra to the context of subproduct systems. Suppose that X and Y are Arveson–Stinespring subproduct systems associated to two stochastic matrices over a countable set Ω , and let T + ( X ) and T + ( Y ) be their tensor algebras. We show that every algebraic isomorphism from T + ( X ) onto T + ( Y ) is automatically bounded. Furthermore, T + ( X ) and T + ( Y ) are isometrically isomorphic if and only if X and Y are unitarily isomorphic up to a *-automorphism of l ∞ ( Ω ) . When Ω is finite, we prove that T + ( X ) and T + ( Y ) are algebraically isomorphic if and only if there exists a similarity between X and Y up to a *-automorphism of l ∞ ( Ω ) . Moreover, we provide an explicit description of the Cuntz–Pimsner algebra O ( X ) in the case where Ω is finite and the stochastic matrix is essential.