What It Takes to Build an AI That Actually Knows You

Video thumbnail: What It Takes to Build an AI That Actually Knows You
Nov 18, 202633m 11s video lengtha16z

The Signal

First Person Labs is developing "Second," a persistent personal-context model designed to run in the background, analyzing continuous computer-event streams to anticipate user needs. While the founder claims it secures data via local and confidential inference, the central tension lies between this always-on data capture and the founder's admission that LLM-based disclosure controls remain inherently imperfect.

The Case

Product Purpose and Design

  • Second is intended as a persistent context store rather than a standalone chatbot, aiming to accumulate a user’s goals, preferences, and values to provide consistent context for specialized external agents.10:22
  • The system performs ambient data capture by monitoring background computer events like application switching, typing patterns, and Wi-Fi status to synthesize high-level interpretations of intent.4:54
  • Hosts describe the product as "reflective" rather than merely reactive, providing unsolicited, useful alerts like weather-based event planning that adapt to context changes without user prompts.7:25

Privacy and Security

  • The product uses a layered security model involving local storage, user-selected inference providers, and hardware-backed confidential computing to attempt to reduce data exposure.12:46
  • The founder explicitly acknowledges that while deterministic controls can block simple data categories, LLM-based reasoning controls cannot definitively prevent unauthorized disclosure, leaving an unresolved risk of model manipulation.16:10
  • Security-conscious users are offered the option to provide their own inference tokens for local data analysis, effectively placing the trust anchor on the hardware provider and the user's own environment.0:36

Scaling and Costs

  • Running 10 agents continuously at high token rates is estimated to cost approximately $1,000 per day on premium models like "Opus 5i," though the founder claims utilizing leaner models like DeepSeek Flash could reduce this to $60 per day.18:31
  • The product is currently available to a limited user base, with the founder emphasizing that wider rollout is contingent on scaling inference capacity and support infrastructure.20:26

The 1 Minute Signal Take

The utility of Second rests on the unverified premise that ambient, continuous tracking of personal habits can be securely mediated by LLMs. Users should view the company’s privacy assurances as architectural designs rather than proven safeguards, especially given the founder’s admission that reasoning-based disclosure limits cannot be fully solved.

Pro Analysis

Why It Matters

This vision represents a shift from 'chatbots' to 'agents as extensions of the self.' If successful, it eliminates the 'context tax' currently paid by users who must re-explain their preferences to every new AI tool, effectively creating a persistent, digital 'Alfred' that works across disparate software silos.

Strategic Implications

The product is an infrastructure play. By building the 'context layer,' First Person Labs hopes to become the standard interface between users and the broader AI ecosystem. If they succeed, they own the user's digital history; if they fail, they are merely an expensive, buggy background process.

Evidence & Hype Audit

This content is high-hype, low-evidence. It is an unverified product pitch from a founder. There are no technical demos, no third-party audits, and no benchmark data to support claims about 'confidential inference' effectiveness or the security of OS-level event streaming. Users should assume these claims are aspirational until proven otherwise.

Counterarguments

The most prominent counterargument is that the 'always-on' model is a privacy nightmare that cannot be solved by technical tricks alone. Even if the data is encrypted, the model itself eventually learns the user's deepest secrets, making the AI itself the ultimate single point of failure.

Who Should Care

  • AI Developers: Should watch how the company implements API-based context retrieval.
  • Security Professionals: Should investigate the feasibility of using confidential computing to wall off personal user models.
  • Power Users: Should monitor the cost-per-day trends to see if this becomes viable for non-enterprise use.

What to Do Next

  • Conduct a personal audit of your digital footprint to see which services currently 'see' your most sensitive data.
  • Research 'Confidential Computing' frameworks to understand the current state of the art in secure, hardware-isolated execution.
  • Monitor the price of smaller, open-weight models, as they are the primary enablers of cost-effective persistent agents.
  • Experiment with local, privacy-focused LLMs to get a feel for the current limitations of local-only context retention.
Time saved:29m 41s

Share this

Tags

Written by: 1 Minute Signal Editorial Team