the "categorical" way of defining tensor products is essentially "that thing that lets you turn multi-linear maps into linear maps", and linear maps (of finite dimensional vector spaces) are basically matrices anyways. so i don't see it as much of a stretch to say tensors are matrices.
a tensor is a multi-linear map V × ... × V × V* × ... × V* → F, and a multi-linear map V × ... × V × V* × ... × V* → F is the same as a linear map V ⊗ ... ⊗ V ⊗ V* ⊗ ... ⊗ V* → F. and a linear map is ""the same thing as"" a matrix. so in this way, you can associate matrices to tensors. (but the matrices are formed in the tensor space V ⊗ ... ⊗ V ⊗ V* ⊗ ... ⊗ V*, not in the vector space V.)