We’ve all spent the last few years talking a lot about AI, and rightly so. The cloud gives us scale, the latest frontier models, and a constant stream of new capabilities. But somewhere along the way, “AI” became shorthand for “send it to the cloud,” and I think that’s worth talking about. Not because the cloud is wrong, but because it isn’t always the right answer. Sometimes you’re on a train with no signal. Sometimes the data simply isn’t allowed to leave the device. And sometimes you just don’t want to spend tokens on a task your laptop can quietly handle on its own.

These points led me to try building something I can try out for real and then talk about with my colleagues and our customers.

If you’ve invested in Copilot+ PCs, you have already paid for a surprising amount of on-device compute, and increasingly, newer devices ship with even more. So why let it sit idle? In this blog post, I want to show that local AI genuinely works, share two demo agents I built that run entirely offline, and give you a basic comparison of what you gain (and what you trade) when you keep a workload on the device instead of sending it to the cloud.

We will cover the following topics:

• Why local AI deserves a place in your toolkit: compliance, connectivity and cost
• Demo 1: the “Auto Filer” agent with local document triage and filing
• Demo 2: the “Confidential Knowledge Agent”, an offline, vector-based chat agent
• A side-by-side comparison of these agents running local vs. in the cloud
• How to think about mixing and matching the two
• Compliance and security. Some organisations, or some specific datasets within them, simply cannot have information leave the endpoint. Legal documents, client-confidential material, regulated data etc., for these, “the model never sees the cloud” isn’t a nice-to-have, it’s the requirement.
• Connectivity. We design as if everyone has perfect connectivity all the time. They don’t. Field engineers, consultants on customer sites with locked-down guest Wi-Fi, anyone on a rural rail line - they all hit moments where the cloud just isn’t reachable, and the work still needs doing.
• Cost. Not every AI task warrants a frontier model and the token bill that comes with it. Filing a document or answering a quick factual question from a known knowledge base is well within the reach of a small model running locally. Push those high-volume, low-complexity jobs to the device and you reduce token consumption, smooth out your costs, and start to realise a return on your Copilot+ PC investment.
• You define where to look. The “triage” folder can be your Downloads folder, your Documents folder, or any other local location where files tend to pile up.
• You define where things should go. You describe your desired folder structure, including nested hierarchies and the agent extracts the relevant information from each document to decide where it belongs.
• It files things away for you. Documents land exactly where you want them, whether that’s a brand-new local structure or somewhere like your synced OneDrive folder, so they’re still backed up and available across devices.
• It behaves like a normal chat agent. You ask questions in natural language and get grounded answers back, retrieved from your local knowledge base with no online connection required.
• It shows you its confidence scores. This is very useful for hybrid scenarios as we may not have enough local compute and want to know how to deal with that scenario. If a query comes back with low confidence, you immediately know it’s a candidate to re-run against a cloud workload later, once you’re back online and want a more powerful model to take a look. You get a sensible offline answer now, and a clear signal about when the cloud would add value. Even if we can be partially productive locally while offline, it’s still a win!
• IT teams can pre-configure it, or users can refresh it themselves. The vector database can be pre-set and deployed by IT admins to end users. Think of a curated, approved knowledge base pushed out across the fleet. The user can also manually refresh their own vector database through the UI when their reference material changes.
• Local AI works, and it’s an option you should actively consider - especially where compliance, connectivity or cost rule out a cloud round-trip.
• You don’t have to choose one or the other. Mix and match: run routine, sensitive and offline work on the device, and reach for the cloud when you need its power.
• Your Copilot+ PC investment is also a compute investment. Newer devices have increasingly more on-device capability and putting it to work is how you turn that hardware spend into real cost savings.

A problem worth solving

When I started experimenting with on-device agents, I wasn’t trying to replace cloud AI, I was trying to fill some gaps and have real-life examples I can show to my colleagues and our customers, no smoke and mirrors. Three real-world scenarios came up frequently:

The interesting bit is that you don’t have to choose one world over the other, you can mix and match. Run the routine, sensitive or offline work locally, and reach for the cloud when you genuinely need its power.

Demo 1: The “Auto Filer” agent

The first agent solves a problem we all recognise: the chaos of an unmanaged Downloads folder. Auto Filer is a local agent that watches a location you choose, understands what each new document is, and files it away into a folder structure you define, all on the device with no cloud round-trip.

Here’s how it works in practice:

The clever part is that the model reads and reasons about the content locally to make the filing decision. For example, an invoice goes to your finance folder, a signed contract to the right client subfolder, a draft proposal into the active opportunities tree, without you dragging a single file, and without any of those documents being sent anywhere.

Demo 2: The “Confidential Knowledge Agent”

The second agent is a natural follow up from the first agent (in my mind anyway). Confidential Knowledge Agent is a local, vector-based chat agent that looks and feels like the online AI chat experiences we have all become used to, except it runs entirely on the device against a local vector database.

A few things make it genuinely useful rather than just a novelty:

For confidential or regulated knowledge, the content is indexed and queried locally, the answers never leave the device, and you still get a transparent confidence signal to guide when escalation to the cloud makes sense.

Cloud alternative and cost savings

If you're not using Microsoft Copilot (why not) then a common cloud baseline is Azure AI Search (for the vector index) plus something like Azure OpenAI GPT-5.4 (for generation) and text-embeddings. Assuming a knowledge worker running 20 retrieval-augmented queries per day over a 500 MB personal & shared corpus re-indexed monthly, we might see the following per-user annual profile.

All figures are pay-as-you-go list prices (UK region, GBP) at time of writing this blog post with no reserved-capacity or volume discounts applied, directly reflecting the latest official Azure pricing sources.

At a thousand users, this is roughly £258,000 per year of cloud spend for a single workload.

Local vs. cloud: a side-by-side comparison

Some more comparisons to consider:

The local option does well on privacy, cost and resilience, while the cloud wins on raw model power for the genuinely hard, open-ended problems. Neither is “better”, they’re complementary. With each new generation of Copilot+ PCs shipping more on-device compute, the range of tasks that run comfortably and quickly on the local side keeps growing.

To close

In this blog post we challenged the assumption that AI has to mean cloud AI. Local agents running on Copilot+ PCs are a real, practical option and not a compromise. We looked at two demo agents, Auto Filer for local document triage and filing, and the Confidential Knowledge Agent for offline, confidence-aware chat over a private knowledge base, and we compared how each performs locally versus in the cloud.

Some key takeaways:

• Local AI works, and it’s an option you should actively consider - especially where compliance, connectivity or cost rule out a cloud round-trip.
• You don’t have to choose one or the other. Mix and match: run routine, sensitive and offline work on the device, and reach for the cloud when you need its power.
• Your Copilot+ PC investment is also a compute investment. Newer devices have increasingly more on-device capability and putting it to work is how you turn that hardware spend into real cost savings.

For any enquiries contact: info@uk.cdw.com | 020 7791 6000