What is a large language model?
A large language model (LLM) is an AI model trained on very large amounts of text to predict the next word in a sequence, which lets it follow instructions, generate fluent text, summarize, translate, classify and reason through problems across many subjects and languages, without being built for any single task. Inside an AI agent it is one component, usually the one that reads and writes language.
Large refers to scale: models with billions of parameters, trained on a substantial share of the public web, books and code. Most current LLMs use the transformer, a neural network design published in 2017 that weighs every word in a passage against every other. The same models are often called foundation models, because other applications are built on top of them.
How a large language model works
An LLM's behavior comes from two places: what it learned in training, and what it is given at the moment of use.
| Stage | What happens | What it means in practice |
|---|---|---|
| Pretraining | The model learns patterns by predicting the next token across a very large body of text | Broad knowledge and fluency, fixed at a cutoff date |
| Tuning | Further training on worked examples and on people's ratings of answers | Follows instructions, adopts a style, declines some requests |
| Context | At use time it receives instructions, the conversation and any documents, up to the limit of its context window | Behavior depends heavily on what it is given |
| Generation | It produces output one token at a time, choosing likely continuations | Plausible, fluent text that is not guaranteed to be true |
| Tools and retrieval | It calls search, databases or other software when the application allows | Answers based on current data, and the basis for agents |
A token is a word or a piece of a word. Because generation is probabilistic, the same prompt can produce different answers, and a confident sentence may be wrong. Applications compensate with retrieval from approved sources, fixed output formats, checks on the result and, for anything consequential, logic that sits outside the model.
Why large language models matter
The economics have shifted quickly. Stanford's AI Index Report 2025 found that the inference cost of a system performing at the level of the models that brought generative AI to the mass market dropped more than 280-fold between November 2022 and October 2024. The same report counts 78% of organizations using AI in 2024, up from 55% the year before.
Cheaper inference changes which tasks are worth automating. Reading every inbound email, summarizing every call or checking every application for missing information used to cost too much at volume. Those tasks are now routine candidates.
The limits have not moved as fast. LLMs remain fluent before they are reliable: they can state an invented fact with the same confidence as a true one, and their output shifts with small changes in wording. That is why organizations using them for customer-facing work surround them with structure rather than relying on the model alone.
Where large language models are used
- Drafting and summarizing. Emails, call summaries, reports and first drafts for a person to review.
- Answering from documents. Searching policies, contracts or manuals and answering with references, a method called retrieval-augmented generation.
- Reading and sorting. Classifying requests, extracting details from documents and routing work.
- Translation. Between languages, and between technical and plain language.
- Writing code. Generating, explaining and testing software.
- Inside AI agents. Interpreting what customers say and phrasing replies, while other components decide and act.
LLMs are the wrong tool for calculations of record, final eligibility decisions and statements that must be worded exactly. Those belong to deterministic systems, with the model passing information to them rather than replacing them.
What good use of an LLM looks like
- The right model for the task. Size, speed and cost matched to the job; the largest model is rarely needed everywhere.
- Grounding in approved sources. Answers draw on current, authorized content rather than the model's memory.
- Structured output. Responses follow a defined format that downstream systems can check.
- Evaluation on real cases. A test set drawn from the organization's own work, rerun before every model or prompt change.
- Pinned versions. Providers update models, so the version in use is fixed and changes are deliberate.
- Agreed data handling. What the provider may store, log or train on is settled by contract.
How to measure a large language model in use
- Task accuracy. Correct outputs on a labeled test set drawn from real cases.
- Unsupported statements. The rate of claims not backed by the sources provided, found by sampling.
- Consistency. Whether the same input gives the same substantive answer across runs and versions.
- Latency. Response time at real traffic levels, which matters most on voice.
- Cost per task. Tokens, infrastructure and review time per completed piece of work.
Data-sharing and oversight rules for LLMs
Most rules that touch LLMs concern the data sent to them and the oversight of the companies that run them. The main considerations in the US follow. Financial services is used as the example because its data-sharing rules are the most specific; state privacy laws apply the same logic to other industries.
- Sharing customer data with a model provider. Under Regulation P, a financial institution may share nonpublic personal information with a service provider without offering an opt-out only if a contract bars the provider from using or disclosing it beyond the purpose it was shared for. Logging and training settings need to match that contract.
- Overseeing the provider. The FTC Safeguards Rule requires financial companies under its jurisdiction to select service providers able to protect customer information, require safeguards by contract and assess them periodically.
- Risk frameworks. NIST's Generative AI Profile, published in July 2024 as a companion to its AI Risk Management Framework, is voluntary. It gives organizations a structure for risks specific to generative models, such as confabulation and the exposure of sensitive data.
Frequently asked questions
Is an LLM the same as an AI agent?
No. An LLM interprets and generates language. An agent wraps a model with a goal, tools, memory and decision logic so that it can complete tasks.
Why do LLMs make things up?
They generate the most plausible continuation of text, not a verified fact. When the right answer is missing from the context and the training data, a plausible but wrong one can come out. Grounding and checks reduce this; they do not remove it.
Do LLMs learn from every conversation?
Not by default. A deployed model's weights stay fixed, and it uses only what is in the current context. Whether conversations are kept or used to train future models depends on the provider's terms and the customer's settings.
What is a context window?
The amount of text a model can consider at once, including instructions, the conversation and any documents. Anything outside it is invisible to the model.
Should an organization use one large model or several smaller ones?
Often several. Smaller models are faster and cheaper for narrow tasks such as classification, while larger ones handle open-ended reasoning and writing.
Related terms
A language model is a part. A revenue result is the whole.
Encore AI pairs language models with a structured playbook compiled from your top performers' conversations, so the model handles the talking while approved logic decides what happens next.
Scale your best to be the best.
Book a Demo