Local. Cloud. Hybrid.
Built around operations.

“We need AI infrastructure” does not yet describe a requirement that can be sized. An interactive assistant, overnight document processing and a time-limited training task place different demands on systems. The first step is to record who uses which function, what data is processed and how demand changes throughout the day. Existing applications belong in this picture too: a model endpoint rarely constitutes the complete service that employees or customers actually need.

As a fictional example, consider an industrial supplier. Its service team needs support with technical enquiries during the day. New documents are processed outside the busiest hours. A development team occasionally evaluates additional models using an approved dataset. These tasks share information, but do not necessarily need the same computing environment. A bottleneck in an experiment should not impair the live service; a delayed background job has different consequences from an unavailable customer support function.

Each workload receives service objectives. A Service Level Objective, or SLO, might define the proportion of valid requests that should complete within a specified time. For generative applications, the delay before the first output, time to the complete response and a useful result may matter separately. An average is insufficient: infrequent slow responses can shape the working experience. Measurement and the error budget should therefore reflect how people actually use the service.

Inference means executing an already deployed model. Training and adaptation change model parameters and can create different requirements for memory, computing and data throughput. Preparation tasks such as file extraction, image processing and index construction also consume resources without necessarily occupying the same accelerator throughout. The architecture assigns these activities separate workload profiles. This reveals which capacity is needed continuously and which tasks can wait, run in batches or temporarily execute elsewhere.

The requirements also cover data sensitivity, permitted processing locations, expected concurrency, maintenance windows and available operating expertise. They include whether a ready-made service is sufficient or whether a model needs to be operated in-house. Self-managed operation provides particular design options while adding responsibilities. A small, clearly described initial workload produces better decision evidence than a substantial purchase intended for still-undefined future applications.

Local operation can mean a company server room, a system at a particular site or a customer-managed environment in a data centre. Colocation, for example, moves the physical hosting without automatically transferring software operations. A private cloud is not defined by location alone either: provisioning, usage and the responsibility model matter. Decisions should therefore name ownership, management and actual dependencies explicitly.

Public cloud offerings range from rented compute instances to largely managed AI services. Infrastructure as a Service typically leaves more responsibility for operating systems, runtimes and applications with the customer than a managed model API does. The tasks assumed by a provider must be checked against the particular service and contract. Data classification, access allocation and the suitability of the application do not disappear when execution moves elsewhere.

Self-managed operation can make sense where sustained demand is sufficient, specialised integrations are needed or defined tasks must remain available without an external connection. It requires suitable premises or hosting, maintenance, replacement capacity and an accountable team. Cloud services can support a fast start or provide additional capacity for a limited period. They introduce their own quotas, service boundaries, recurring charges and contractual and technical dependencies. Neither model is superior simply because of its name.

A hybrid design deliberately assigns different tasks to different environments. In the example, the ongoing internal service could execute locally while occasional model comparisons use an external environment with a separate, approved dataset. Handoffs and data paths are specified accordingly. An architecture that requires a cloud call for every local request is not automatically capable of working offline. Conversely, an additional local computer does not by itself provide a working fallback for an external model endpoint.

The interactive topology shows simplified dependencies during normal operation, a connection failure and the loss of a local inference host. Its cloud variant uses an approved data snapshot that has already been transferred; the local interface and data access are separate from the failed inference computer. Hybrid operation separates confidential local requests from cloud-permitted requests without automatically changing the data class during a failure. These assumptions determine which parts can continue. Real deployments need their additional dependencies and recovery paths to be specified explicitly.

Model size is only one part of sizing a local AI environment. Besides model weights, the runtime, intermediate results and concurrent requests require memory. Many generative language models use a KV cache to retain information already computed from the context. Its requirements depend on factors including model architecture, context lengths, data representation and concurrency. A configuration that processes one short request can reach a different limit with longer inputs or several simultaneous requests.

RAM and accelerator memory perform different roles. A large amount of system memory does not automatically replace the fast memory attached to the accelerator. Offloading can enable particular configurations but changes data movement and execution behaviour. Likewise, adding the memory capacities of several devices does not demonstrate that a model will use them efficiently without suitable partitioning and communication. Software support, interconnects and the actual workload profile must be evaluated together. CPU capacity remains important for file extraction, preprocessing and application logic. Whether a task needs an accelerator follows from measured throughput and response behaviour, not simply from the label AI.

Quantisation changes numerical representation and can reduce memory needs. It is not a universally lossless shortcut. Task quality and speed need to be assessed again with the actual model version and runtime. Batching can increase throughput while affecting individual request behaviour. Long inputs, extensive outputs and many concurrent sessions compete for resources. A benchmark should therefore include realistic combinations of these factors rather than documenting only the fastest individual result.

The capacity model considers equivalent requests with an assumed constant processing rate per instance. When incoming demand exceeds that rate, backlog grows in the simplified model. Real execution times and arrival patterns vary; memory limits, scheduling and queue policies may also intervene. The graphic is therefore not a response-time forecast. It explains why a healthy monthly average can conceal a short but commercially important peak. The assumed performance of each additional instance requires independently available, comparable resources. Multiple processes on the same already saturated accelerator do not multiply its capacity.

Sizing also includes power supply, cooling, space, noise and maintenance access. Maximum electrical connection requirements and actual energy consumption under the workload are different quantities. Production planning considers driver maintenance, spare parts, maintenance duration and capacity during a failure. An additional instance may provide maintenance cover or redundancy as well as growth. If reserved for those purposes, it cannot simultaneously be treated as fully available continuous capacity.

The storage plan distinguishes original data, processed data, indexes, model artefacts and operating information. These collections have different availability, access and recovery requirements. Originals may be managed centrally while frequently used working data sits close to processing. A fast local index can support the application, but rebuilding it still requires inputs, versions and enough time. Which data can be recreated is documented rather than merely assumed.

A NAS can provide a useful central repository without being the fastest or only sensible location for every access pattern. Many small operations, concurrent readers and large sequential transfers place different demands on a system. Metadata operations, protocol behaviour and competition from backups or other applications also need attention. Fast local drives can hold working datasets but require a clear policy for synchronisation, rebuilding and loss of the host.

Network design covers throughput, latency, resilience and the actual routes between services. Combining several interfaces does not automatically let one transfer use their aggregate bandwidth. Common LACP configurations distribute connections across individual links. An apparently adequate total connection can therefore remain a constraint for a particular data stream. Measurements should represent the real route, including switches, firewalls, tunnels and the storage systems involved.

When work spans sites and cloud environments, moving data becomes an architectural consideration in its own right. Repeatedly transferring large collections can consume time, bandwidth and money. It is often useful to place processing near existing data and move only defined results or approved subsets. Freshness, deletion and ownership must then remain clear across all copies. Otherwise, an extra storage location primarily increases the number of potentially conflicting versions.

RAID, replication and snapshots serve different purposes from independently protected backups. Redundant storage can absorb some device failures, but may also reproduce accidental deletions or harmful changes. A backup plan therefore identifies protected copies, retention, separate access and recovery procedures. Configuration and necessary keys belong in the plan alongside data. A recovery exercise demonstrates whether the complete application can become usable again within the intended time.

A local installation can be poorly secured, while an external service can provide suitable safeguards. Assessment depends on how identities, data and administrative rights are actually handled. The architecture distinguishes user access, technical service accounts and privileged administration. Permissions follow tasks, are reviewed regularly and are removed when no longer required. Shared credentials with permanently broad authority make access harder to attribute and constrain.

Applications, data stores, development environments and administration receive appropriate network boundaries. A model server does not need to be directly reachable from every network. Controlled ingress and egress support checks on approved services, data categories and authentication. Credentials belong in suitable secret management with rotation and governed access. Encryption protects particular paths and storage states, but does not replace assessment of who can access information while it is being processed.

AI inputs, outputs, diagnostics and logs can contain sensitive information. Logging should explain operations without automatically retaining complete confidential content. Troubleshooting, security analysis and task evaluation may need different data and retention periods. The team defines what is recorded, who may see it and how it is deleted. Convenient development logging is not automatically an appropriate production configuration.

For personal data, purpose, legal basis, necessary scope and safeguards are assessed for the specific use. A service provider introduces questions including contractual roles, subprocessors, processing locations and potential international transfers. Choosing a European storage region does not automatically answer every question about access and processing. Conversely, local storage alone does not make an application GDPR compliant. Approved data use and technical implementation must express the same boundaries.

This also applies to models, runtime packages and container images. Origin, licence, integrity and update paths are reviewed; test and production should use traceable versions. A new package should not enter live operation solely because it worked in an experiment. Changes require ownership and a route back to an approved state. Security requirements thereby become concrete operating tasks that need to be performed and checked in both company-managed and outsourced environments.

An application can fail through identity, database, model runtime, storage, network or an external service. Availability is therefore considered from the use case. Which dependencies must work to produce a valid answer? Which tasks may continue with older data, and which need to wait in a controlled way? An operating diagram should expose these distinctions. Critical paths are examined for single points of failure and shared failure causes, such as a common power supply or one administrative identity. Data processing and management are considered separately: a service can compute locally while depending on external login, licence checks, model downloads or monitoring. These belong in the dependency inventory alongside application data paths.

The responsible teams define recovery objectives. The Recovery Time Objective, RTO, describes the target time to resume operation; the Recovery Point Objective, RPO, concerns the amount of data loss in time that can be accepted. Both arise from the business process. A daily backup does not suit every requirement, and a quickly available replacement host helps little if restoring its required data takes substantially longer. Objectives and procedures are exercised together.

A backup is not automatic failover. A recovery environment needs compatible data, application, model, identities, network permissions and capacity. Switching model providers can also change the quality of results and requires an approved application configuration. Where a complete second operating path would be disproportionate, clearly communicated reduced functionality or a manual process may be more appropriate. That decision is made before a disruption occurs.

Observability connects technical and application signals. These include error rates, waiting times, rejected requests, queue length and resource needs, as well as data freshness and successful background jobs. Fully allocated accelerator memory is not automatically a fault; it may reflect how the runtime operates. A signal becomes useful in relation to a service objective. Alerts need an accountable recipient, an understood response and a sensible priority.

Updates are tested in a controlled environment and introduced progressively. Operating system, drivers, runtime, model and application form a connected configuration. Rollback requires more than an old file: configuration and data must remain compatible. Regular exercises therefore include recovery, credential expiry, connection loss and failure of an important component. Useful exercises also reveal which documentation and contacts must remain accessible outside the affected system during an incident.

A comparison needs an equivalent service scope. Counting only model calls on one side while including storage, redundancy and operating staff on the other does not produce a useful cost comparison. First define the task, quality, demand and availability required. Costs per usable result and over a shared time horizon then refer to that scope. Even an inexpensive technical call is not an economic success if the application repeatedly requires manual correction.

Self-managed operation includes purchase or leasing, setup, integration and any required changes to premises. Recurring expenses include energy, hosting, maintenance, protection and staff effort. Not every cost changes immediately with request volume. A lightly used system still incurs fixed costs, and necessary reserve capacity must also be funded. The assumed useful life and treatment of replacement investment should be explicit.

Cloud services may charge for storage, data transfer, managed components, support or minimum commitments as well as compute time or model usage. The actual tariff determines which quantities and periods are billed. Shutting down a compute instance does not necessarily remove its other associated costs. A favourable long-term commitment can also reduce flexibility. Teams need cost allocation by application or cost centre, budget boundaries and oversight of unexpected usage rather than relying only on a late monthly invoice. Setup, integration and migration can also create one-time costs; usage-based pricing does not automatically make adoption effort-free.

The interactive cost model separates fixed and usage-dependent assumptions for two simplified options. It compares equivalent units of work over a chosen period and shows how demand and assumed costs change the relationship. It contains no current provider prices and does not automatically select an architecture. Missing items are added for a real decision; shared costs must neither be forgotten nor counted twice on one side. A cost calculation also does not establish that the proposed capacity can actually handle the workload.

Several trajectories are more useful than a single point: limited initial use, expected steady operation and growth requiring further resources. It is particularly important to identify when another capacity tier or a contract change becomes necessary. Such changes rarely follow a completely linear path. Hybrid operation can distribute particular workloads economically while introducing its own integration and operating effort. A decision therefore combines costs, technical suitability and available skills rather than simply marking the smallest calculated amount.

Models, runtimes and commercial offerings change faster than many purchasing cycles. Applications should therefore separate important decisions from replaceable components. A clearly defined model interface, versioned configuration and documented data formats make change easier. They do not guarantee arbitrary interchangeability. Two services may expose similar interfaces while differing in context limits, tool calling, output formats, error behaviour and answer quality.

Portability is tested in practice. Can the required dataset be exported with its metadata and permissions? Are model weights, adapters, licences and runtime available for the intended alternative environment? Can images and configurations be deployed reproducibly? A move also requires access, capacity, network connectivity and time for testing. A standard format helps, but does not replace an exercised transition using a realistic subset.

Applications with search indexes must distinguish originals from derived state. An index may depend on a particular model version, configuration or database. A change can require rebuilding it. The time and computing needed belong in the migration plan. Task-level comparisons must also be repeated: a technically reachable endpoint does not show that the existing application will continue working reliably with its new responses.

A useful exit plan identifies triggers, owners, data scope and transition order. Reasons may include changing costs, business requirements, contract terms or an operating model that no longer fits. Parallel operation can support a transition but temporarily adds work and expense. Afterwards, old access, copies and running resources are deliberately retired. Without that final step, both exposure and expenditure remain.

Open components can provide room to adapt, while requiring maintenance and review of their respective terms. Managed services can reduce effort but require transparency about boundaries and dependencies. The architecture keeps this trade-off visible. Its objective is not a theoretically provider-free environment, but a justified commitment with a practical route through the changes likely to matter to the business.

Start with a concise architecture profile: application, data categories, demand pattern, service objectives, sensitivity and owners. Record existing infrastructure and operating skills alongside unresolved assumptions. Develop a small set of comparable options. For the fictional supplier, these could include a managed model endpoint, a self-managed inference service and separate execution of internal daytime work and external experiments. Each option must satisfy the same initial task before additional possibilities are evaluated.

A benchmark uses representative inputs and recorded model and software versions. It tests more than a warm single request: concurrent usage, longer inputs, background work and startup after interruption also matter. Task quality, response behaviour and cost are measured together. Results apply to the tested conditions. A different model version or longer context needs reassessment rather than silently inheriting the previous measurements.

The bounded pilot integrates identity, data access, logging, budget oversight and support. It should demonstrate that the application can be operated in its intended working environment. This includes deliberately interrupting selected dependencies and performing a recovery test. A service that works only with constant development-team supervision has not yet demonstrated operational readiness. Handover therefore makes clear who decides during incidents and which tasks the internal team or a partner will perform.

Capacity boundaries, maintenance procedures and costs are reviewed before expansion. Another department may introduce different data, longer requests or higher availability requirements. Scaling is therefore more than adding computing. It may require new permission models, priorities and operating procedures. Periodic review connects measured usage to the next investment and prevents a small pilot from quietly becoming an unsupported core application.

Scalevity helps connect business applications, software architecture and IT operations: clarify requirements, compare options, design interfaces and data paths, and develop a testable pilot. Physical infrastructure, hosting and specialised security responsibilities are coordinated with the accountable teams and partners. The result is an understandable decision about what runs where, who operates it and which criteria will guide later changes to the architecture.