Glossary

Transformer

What is Transformer?

The transformer is a neural network architecture introduced in 2017 in the research paper “Attention Is All You Need”. At its core is the attention mechanism: each word is weighted in relation to all other words in a text, and the computation can be parallelised efficiently. Transformers underpin today's language models such as GPT, Claude and BERT and are also used for images and audio.

How KI.ordo helps

The Anthropic Claude models used by KI.ordo are large language models based on the transformer architecture.

Topics

Architecture
Deep learning
LLM

Try KI.ordo

Agents, flows and templates in one workspace – free 24-hour demo.

Start demo

← All terms

Related terms

Generative pre-trained transformer (GPT)

GPT (generative pre-trained transformer) refers to a series of large language models developed by OpenAI and based on the transformer architecture. The models are first pre-trained on vast amounts of text to predict the next token and then further trained for dialogue and following instructions. The first GPT model appeared in 2018; later generations underpin ChatGPT. In everyday speech, ‘GPT’ is often used for language models in general.

Language model
Transformer
+1

BERT

BERT (Bidirectional Encoder Representations from Transformers) is a language model introduced by Google in 2018. Unlike earlier models, it considers the context to the left and right of a word at the same time, which helps it resolve ambiguity. BERT is designed for understanding text rather than writing longer passages. It is used for search, text classification, named entity recognition and question answering and has shaped many successor models.

Language model
NLP
+1

Large language model (LLM)

A large language model (LLM) is a neural network trained on very large amounts of text, enabling it to understand and generate language. It works by calculating the most likely next token, step by step. LLMs write, summarise, translate, answer questions and generate code; well-known examples include Claude, GPT, Gemini and Llama. Without access to current sources, they only know what was in their training data.

Language models
Generative AI
+1

Hugging Face Transformers

Hugging Face Transformers is an open-source Python library that gives access to pre-trained models for text, images and audio through a single, consistent interface. Developers use it to download models from the Hugging Face Hub, try them out, adapt them to their own data or run them locally. Since version 5 the library has focused primarily on PyTorch (as of October 2026). Example: a team tests a sentiment-analysis model with just a few lines of code.

Open source
Library
+2