A bidirectional LSTM (BiLSTM) is a recurrent neural network that reads a sequence both forwards and backwards and combines the two views. This means the context that follows each word is taken into account as well. Before transformers took over, it was a standard choice for tasks such as part-of-speech tagging, named entity recognition and speech recognition. For very long texts and large datasets, transformers now usually perform better.
Long Short-Term Memory (LSTM) is a type of recurrent neural network that can retain information across long sequences. So-called gates decide at each step what to store, forget or output, which mitigates the vanishing gradient problem. For many years LSTMs shaped speech recognition, translation and time-series forecasting. In language processing they have largely been replaced by transformers, but they remain common for sensor data.
A recurrent neural network processes data in order and passes an internal state from each step to the next – a kind of short-term memory. This makes it suitable for sequences such as text, speech or time series, for example to predict the next word or the next sensor reading. Simple RNNs easily forget information from far back; variants such as LSTM and GRU mitigate this. For language models, RNNs have largely been superseded by transformers.
Sequence modelling covers methods that process and predict data in a fixed order, where the position of each element matters. Typical sequences are text, spoken language, time series such as sales or sensor readings, and DNA segments. Applications range from translation and text generation to speech recognition and sales forecasting. Hidden Markov models and recurrent networks have been used for this; today transformer models dominate for language.
Named entity recognition (NER) is a natural language processing technique that finds names of people, companies and places, as well as dates or amounts, in text and assigns each to a category. It turns free text into structured data. Example: the sender’s company, contact person, delivery date and invoice amount are extracted automatically from an email and transferred to a CRM.
