Semantic memory is the individual's concept of meanings. It includes factual knowledge and general knowledge, right or wrong. This is the raw material of the vocabulary and expression, the basis of formative usage. The 'memory' analogy is an apt description of the quality of this information, which may be blurred confused or skewed by perception of information or translation from communication. (Malapropisms are a good example of where the semantic memory goes off the rails.) It's the quality of memory which defines the semantics.