❌

Reading view

Applying Security by Design and Cyber Immunity principles to protect AI systems | Kaspersky official blog

AI systems are much harder to protect than traditional IT — whether from operator mistakes, malicious interference, or just plain unstable behavior. There are two main reasons for this. First, AI outputs are probabilistic, so the same prompt can give different answers each time you run it. Second, today’s LLMs can’t reliably tell commands apart from data. Despite the demonstrated security issues of AI systems that we’ve covered many times, it’s impossible to ignore the potential economic payoff of adopting them. That’s why many companies are moving beyond simple AI assistants to AI agents. These agents carry out tasks on their own and have direct access to the same tools as humans — from reading databases to changing settings in production processes. Our experts, in their new study, examine how to clearly define and minimize the resulting risks to keep potential problems to a minimum. The approach borrows and adapts the principles of Cyber Immunity and Security by Design.

The basic principles of Security by Design

All the principles of this discipline are universally applicable to the design of any computer system — AI agents included. What differs is how deeply each principle needs to be adapted and reworked for AI scenarios.

  1. Security is a separate and important factor right from the design stage. Security goals and assumptions are set on day one and followed throughout every stage of development. For AI agents, these assumptions should include three key points: (i) the language model is treated as untrusted, (ii) external data of any kind is considered potentially malicious, and (iii) irreversible actions can’t be performed autonomously without mandatory human approval.
  2. The system is designed on the assumption that it will be attacked.
  3. The trusted computing base and attack surface are kept to a minimum. Agents get only the tools they actually need, limited permissions, and the smallest amount of context possible. Security policy enforcement is handled entirely by deterministic components: the policy monitor, the agent harness, and isolation mechanisms (sandboxes).
  4. The principle of least privilege applies. Each agent has its own set of credentials and access privileges (identity), limited to the bare minimum it needs to do its job. If an agent temporarily needs elevated privileges, it gets them only after external approval, and they last exactly as long as the task that requires them.
  5. Components are isolated. In agent-based systems, this is one of the most complex and multi-layered architecture problems. The agent harness should run in an isolated environment (sandbox or virtual machine), and agents should be kept as isolated from each other as possible. Data processing in trusted and untrusted contexts is also kept separate. For example, one agent reads a pull request and writes a draft of the code, while another reviews that work without ever seeing the potentially malicious text of the request.
  6. Default deny is the rule. Security policies follow the rule: anything not explicitly allowed is forbidden. This includes tool calls, interactions between agents, and network traffic.
  7. Defense in depth is in place. Different measures should work at the same time and reinforce each other. Probabilistic defenses — like AI guardrails (a “firewall” for language models) or alignment (the model’s own sense of what’s acceptable and what isn’t) — can’t offer any formal security guarantees. Nevertheless, they make the system more secure overall and serve as a valuable source of telemetry.
  8. The supply chain is protected. Some agents can install skills, libraries, and other software components on their own, so these abilities need strict rules. Either block them completely, or limit them to a closed, pre-approved list of specific versions of specific components. Systems built on cloud LLMs face an extra risk: model updates. One day, the same API request might be answered by a different language model with different security properties.

Pitfalls of AI security

The general principles of Security by Design are well known and look simple on paper. But applying them to AI systems is tricky because agents are incredibly flexible. For example, a security team might believe that an agent on a developer’s computer runs in a sandbox and can’t change anything outside it. But an attacker, or even the agent itself, can come up with dozens of ways around this seemingly solid protection: from running commands through a local MCP server that the sandbox doesn’t restrict, to reconfiguring another AI agent installed on the system. To make things even harder, the range of attack vectors is huge: from arbitrary fields in the agent’s configuration to innocent-looking text strings in a skill file.

One serious problem that designers often underestimate is user fatigue. A busy agent can throw up dozens of requests every minute, asking the user to confirm potentially dangerous actions. Within an hour, the person is clicking Approve on autopilot without reading what they’re approving. That’s why it’s important to carefully tune both how often these requests appear and how they’re worded. Nobody will read a request description that’s too long. And one that’s too short can hide the key details the user needs in order to make the right call.

Vulnerabilities in agents and related tools are a pressing problem — and it’s made worse by how quickly these categories of software evolve and receive updates. Industry-standard platforms and apps — such as LangGraph, MS Semantic Kernel, and Claude Code — have had critical vulnerabilities patched more than once. Architects should always consider swapping a large, do-it-all framework for a compact, stripped-down system that has only the features a specific agent actually needs.

On top of that, carefully implementing standard, long-established security measures in the available tools greatly reduces the damage if an agent is attacked or behaves unexpectedly. One example is a strict allowlist of domains that the web fetch tool can open when asked by the agent. But designers rarely account for the hidden problems and overhead associated with maintaining the list. As a result, it slowly goes out of date and leaves loopholes for attackers. The most striking example is the EchoLeak attack. Microsoft 365 Copilot had a properly implemented and fairly effective allowlist for accessing external sites, but attackers circumvented it through a feature in a secondary tool and the MS Teams domain.

Practical tips for architects and security teams

AI system architects and heads of engineering and security teams should build in agent security from the design stage. They should rely primarily on verifiable, deterministic mechanisms: isolating components, controlling interactions, and keeping the trusted computing base to a minimum.

Key architectural decisions include defining acceptable autonomy for the agents, limiting access to confidential data and dangerous combinations of capabilities, and using sandboxes, separate roles for different models, and access control gateways. Restrictions added as an afterthought reduce risks, but they’re no substitute for secure architecture.

At the organizational and operational levels, companies need to keep an inventory of their agents, coordinate the efforts of security and deployment teams, protect container environments, manage the supply chain (AIBOM), and integrate multi-layered defense.

Observability requires special attention. For AI systems, this should include logging the LLM’s requests, responses, and reasoning, as well as detailed telemetry from the hosts and cloud environments where the agent harness runs.

The hardest part of designing agents is balancing security, performance, and usefulness. To learn how to strike that balance with threat modeling and clearly defined security goals, read the full version of the study.

  •  

💾 Fighting DOGE's Data Grab | EFFector 38.19

When we give our private information to the government to pay taxes, apply for a job, or vote, we should be able to trust that our data won't later be used against us. But in a betrayal of that trust, the federal government has been pooling the information it has about us on an unprecedented scale—and using it for purposes we never agreed to.

EFF and other groups are fighting back. In our latest EFFector newsletter, we're talking about a recent victory in our lawsuit against the government for sharing federal workers' confidential personnel files with DOGE.

JOIN OUR NEWSLETTER

For over 35 years, EFFector has been your guide to understanding the intersection of technology, civil liberties, and the law. This issue covers our free tracker blocker coming to Safari, a call to action for net neutrality in California, and a big win in our lawsuit against the DOGE data grab.

Prefer to listen in? EFFector is now available on all major podcast platforms. This time we're talking to EFF's F. Mario Trujillo about the dangers of government data grabs—and what defenders of privacy are doing to stop it. You can find the episode and subscribe on your podcast platform of choice:

play
Privacy info. This embed will serve content from simplecast.com

Listen on Spotify Podcasts Badge Listen on Apple Podcasts Badge  Subscribe via RSS badge

Want to protect your right to digital privacy? Sign up for EFF's EFFector newsletter for updates, ways to take action, and new merch drops. You can also fuel the fight for privacy and free speech online when you support EFF today!

  •  

HomeWizard maakt oplaadbare temperatuur- en luchtvochtigheidsmeter van 28 euro

HomeWizard introduceert de Temperature Humidity Meter voor het meten van de temperatuur en luchtvochtigheid. Het apparaatje werkt op een oplaadbare accu en vereist geen hub of bridge. De Temperature Humidity Meter is vanaf 27,95 euro verkrijgbaar en wordt vanaf 30 november geleverd.
  •  

Banks are blocking AI agents from customer accounts

This week, Axios reported that major US banks were detecting third-party AI agents operating inside customer accounts. In many cases, customers have given the agents their usernames and passwords so they could act on their behalf. This kind of access can violate a bank’s rules.

Bianco Research president Jim Bianco raised the question during Unchained’s Bits + Bips livestream on October 26:

“Are you protecting me, or are you protecting your own profits?”

Bianco described traditional finance’s business model as “based on laziness.” Customers may leave idle cash in low- or no-interest bank accounts rather than move it into a money market fund. An AI agent could make that move for them. “Now that you’re able to do it, the banks are afraid they’re gonna lose that revenue stream,” he said.

The banks offer a different explanation. JPMorgan Chase CEO Jamie Dimon said the bank wouldn’t allow just any outside agent into its systems because doing so was “too risky for clients and for JPMorgan.”

Both explanations can be true: banks can want to retain deposits while also facing real security problems when software acts through customers’ accounts.

Handing over your banking login introduces another service into the security chain. A vulnerability in that service could expose your credentials or allow your account to be misused. And as Bank of America reportedly pointed out:

 “If you do share your credentials, keep in mind that you’re responsible for activity resulting from that access.”

That’s the bank’s stated position, not a universal ruling on consumers’ legal rights. Still, it’s a warning worth taking seriously.

Pros and cons

An AI agent with permission to make purchases might find and buy an item at a lower price. But giving an agent access to your bank account also means sharing sensitive banking information with a third party. We’ve seen shared AI chats become searchable online too.

Axios reported that many banks allow, or are moving toward allowing, read-only access. This can let a tool analyze transactions without giving it permission to move money.

That’s an important boundary, but it doesn’t make the information harmless. An agent that can read bank statements may still disclose them if it’s compromised or manipulated. Preventing transactions and protecting confidentiality are separate security goals.

How to keep control of your money

For consumers, the practical question is not whether banks have perfectly selfless motives. It’s what access you’re giving away, and what could happen if the agent makes a mistake.

  • Don’t paste banking passwords or security codes into a general-purpose AI assistant. Major banks have explicitly warned customers against sharing their credentials.
  • Use your bank’s approved connection methods rather than giving an agent your login. Where supported, choose read-only access if all you need is analysis or budgeting help.
  • Give agents access to as little sensitive information as possible. An assistant comparing publicly advertised savings rates doesn’t need your bank statements. OpenAI recommends limiting agents to the data and credentials they need for the task.
  • Review consequential actions yourself. If an agent asks for confirmation, check exactly what it intends to do and what information it will share. Don’t treat a confirmation prompt as a formality.

AI can make it easier to compare financial products and spot opportunities you might otherwise miss. But an assistant that finds a better rate and an agent that can act inside your account are different propositions.

Let it help with the homework, but be cautious about handing it the keys.


Let’s face it, an incognito window can only do so much. 
 
Breaches, dark web trading, credit fraud. Malwarebytes Identity Theft Protection monitors for all of it, alerts you fast, and comes with identity theft insurance. 

  •  

Banks are blocking AI agents from customer accounts

This week, Axios reported that major US banks were detecting third-party AI agents operating inside customer accounts. In many cases, customers have given the agents their usernames and passwords so they could act on their behalf. This kind of access can violate a bank’s rules.

Bianco Research president Jim Bianco raised the question during Unchained’s Bits + Bips livestream on October 26:

“Are you protecting me, or are you protecting your own profits?”

Bianco described traditional finance’s business model as “based on laziness.” Customers may leave idle cash in low- or no-interest bank accounts rather than move it into a money market fund. An AI agent could make that move for them. “Now that you’re able to do it, the banks are afraid they’re gonna lose that revenue stream,” he said.

The banks offer a different explanation. JPMorgan Chase CEO Jamie Dimon said the bank wouldn’t allow just any outside agent into its systems because doing so was “too risky for clients and for JPMorgan.”

Both explanations can be true: banks can want to retain deposits while also facing real security problems when software acts through customers’ accounts.

Handing over your banking login introduces another service into the security chain. A vulnerability in that service could expose your credentials or allow your account to be misused. And as Bank of America reportedly pointed out:

 “If you do share your credentials, keep in mind that you’re responsible for activity resulting from that access.”

That’s the bank’s stated position, not a universal ruling on consumers’ legal rights. Still, it’s a warning worth taking seriously.

Pros and cons

An AI agent with permission to make purchases might find and buy an item at a lower price. But giving an agent access to your bank account also means sharing sensitive banking information with a third party. We’ve seen shared AI chats become searchable online too.

Axios reported that many banks allow, or are moving toward allowing, read-only access. This can let a tool analyze transactions without giving it permission to move money.

That’s an important boundary, but it doesn’t make the information harmless. An agent that can read bank statements may still disclose them if it’s compromised or manipulated. Preventing transactions and protecting confidentiality are separate security goals.

How to keep control of your money

For consumers, the practical question is not whether banks have perfectly selfless motives. It’s what access you’re giving away, and what could happen if the agent makes a mistake.

  • Don’t paste banking passwords or security codes into a general-purpose AI assistant. Major banks have explicitly warned customers against sharing their credentials.
  • Use your bank’s approved connection methods rather than giving an agent your login. Where supported, choose read-only access if all you need is analysis or budgeting help.
  • Give agents access to as little sensitive information as possible. An assistant comparing publicly advertised savings rates doesn’t need your bank statements. OpenAI recommends limiting agents to the data and credentials they need for the task.
  • Review consequential actions yourself. If an agent asks for confirmation, check exactly what it intends to do and what information it will share. Don’t treat a confirmation prompt as a formality.

AI can make it easier to compare financial products and spot opportunities you might otherwise miss. But an assistant that finds a better rate and an agent that can act inside your account are different propositions.

Let it help with the homework, but be cautious about handing it the keys.


Let’s face it, an incognito window can only do so much. 
 
Breaches, dark web trading, credit fraud. Malwarebytes Identity Theft Protection monitors for all of it, alerts you fast, and comes with identity theft insurance. 

  •  
❌