Security: Moving from Input Filtering to Action Guarding

Traditional LLM security often relies on input sanitization, which fails in agentic systems because agents interact with a vast, unpredictable surface area (web pages, emails, GitHub issues). When agents operate in group settings, the risk of prompt injection and data leakage increases significantly.

Research indicates that individual "safe" skills can become malicious when combined—for example, an OCR module and a reporting module might independently pass static checks but collude to leak PII to a third party. To mitigate this, security should be implemented at the action boundary rather than the input boundary.

• Deterministic Guards: Use traditional NLP or regex-based approaches to intercept actions (e.g., reading bash variables or exporting data) and flag them for approval.
• Learned Guards: Implement a small, LoRA-fine-tuned model to separate data channels from instruction channels. This model acts as a filter that allows the agent to perform tasks while stripping out malicious content, such as prompt injections that use obfuscation techniques (e.g., interspersed characters) that bypass regex.

Memory: Evolving from Bloated Context to Atomic Graphs

In group settings, agents must manage long-running, multi-user conversations without context bloat. Storing entire chat histories is inefficient and costly. Instead, developers should treat memory as an evolving, curated knowledge base.

• Atomic Extraction: Rather than storing raw transcripts, extract atomic bits of information relevant to future interactions.
• Hierarchical & Temporal Curation: Use auto-raters to evaluate the value of facts, organizing them into graphs. Implement a "relevance scorer" that continuously adapts to the group's evolving context, allowing the agent to forget outdated information and prioritize high-value data.
• KV Cache Awareness: When building memory retrieval systems, ensure the architecture is compatible with KV caching to avoid performance degradation.

Privacy: The New Social Contract

Deploying an agent in a group creates a new social contract where the same data (e.g., a grocery list vs. salary information) may be public or private depending on the context.

• Per-User Adapters: Instead of relying solely on code-based permission layers, use LoRA adapters for each user. This bakes permissions directly into the model's behavior, ensuring that the agent only surfaces information appropriate to the specific user interacting with it.
• Proactive Filtering: Agents in group settings often suffer from being over-articulate. Implementing a classifier to determine the agent's role and when it is appropriate to speak is essential for maintaining a natural, non-intrusive presence, especially on wearable devices like glasses.