What it is
Inference, in the context of artificial intelligence, is the application phase of a trained machine learning model. After a model has completed its training on a dataset, it is deployed to process new, unseen data. During inference, the model takes an input, runs it through its learned parameters, and produces an output, such as identifying an object in an image, translating text, or generating a response in a chatbot. This is distinct from the training phase, which builds the model.
Inference is critical for the real-world deployment of AI, impacting the speed and cost of AI-powered services. The efficiency of inference, measured by latency and throughput, dictates user experience and operational expenses for companies. Investments in specialized hardware, like GPUs, and optimization techniques are driven by the need for faster and cheaper inference, influencing the supply chains for AI chips and the profitability of cloud computing providers.
Why it matters
Inference drives the real-world utility and cost of AI applications, impacting the performance and profitability of tech companies.
Reviewed under editorial standardsUpdated September 26, 2026Not investment advice