A hidden Markov model (HMM) is a statistical model for sequences in which the actual states are hidden and can only be inferred from observable signals. It describes the probability of one state following another and of each state producing a given observation. HMMs were long the standard in speech recognition and part-of-speech tagging; today they are still used, for example, in bioinformatics and time-series analysis.
Sequence modelling covers methods that process and predict data in a fixed order, where the position of each element matters. Typical sequences are text, spoken language, time series such as sales or sensor readings, and DNA segments. Applications range from translation and text generation to speech recognition and sales forecasting. Hidden Markov models and recurrent networks have been used for this; today transformer models dominate for language.
Part-of-speech tagging automatically assigns each word in a sentence its grammatical category, such as noun, verb, adjective or preposition. Because many words are ambiguous (“book” as a noun or a verb, “light” as an adjective or a noun), the methods take the sentence context into account. POS tagging is a classic pre-processing step for parsing, information extraction or spell checking and is offered by libraries such as spaCy and NLTK.
A recurrent neural network processes data in order and passes an internal state from each step to the next – a kind of short-term memory. This makes it suitable for sequences such as text, speech or time series, for example to predict the next word or the next sensor reading. Simple RNNs easily forget information from far back; variants such as LSTM and GRU mitigate this. For language models, RNNs have largely been superseded by transformers.
Speech recognition, also called ASR (automatic speech recognition), automatically converts spoken language into written text. Modern systems use neural networks trained on large amounts of audio recordings and cope reasonably well with accents and background noise. Applications include dictation, subtitles, voice assistants and the transcription of meetings and customer calls. Technical terms, proper names and poor audio quality remain common sources of error.
