In this paper we derive a Probably Approximately Correct(PAC)-Bayesian error bound for linear time-invariant (LTI) stochastic dynamical systems with inputs. Such boundsare widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points. In particular, the bound derived in this paper relatesfuture average prediction errors with the prediction error generated by the model on the data used for learning.In turn, this allows us to provide finite-sample error bounds fora wide class of learning/system identification algorithms. Furthermore, as LTI systems are a sub-class of recurrent neuralnetworks (RNNs), these error bounds could be a first step towards PAC-Bayesian bounds for RNNs.