Topic

Redefining “Compute”: A Chip Architecture Design Methodology for Agentic AI

The continuous reasoning and autonomous decision-making capabilities of Agentic AI are shifting architecture design from “how to compute faster” toward “how to move data more intelligently.” This talk focuses on three critical areas shaping the next generation of AI computing: memory access, communication latency, and Agent-driven self-optimization. It explores how architectural innovation across these areas can accelerate the AI flywheel and provide a new foundation for the Agentic AI era. Outline 1. Challenges in Large-Scale Data Processing An overview of the key challenges facing modern AI systems, including memory access, communication, and the rapid evolution of the software stack. 2. Memory Access Latency and Asynchronous Copy How asynchronous memory operations can decouple computation from data movement and improve pipeline efficiency. 3. Communication Latency with Remote Write and Barrier Mechanisms How Remote Write and hardware-based synchronization barriers can reduce synchronization overhead and enable lower-latency communication across devices. 4. Agent-Driven Automatic Optimization a) Tensor Algebra for Optimal Layout Transformation How mathematical approaches can automatically identify optimal tensor layout transformations while reducing the probability of implementation errors. b) Asynchrony and Pipelining for Performance Diagnosis How asynchronous execution and pipelined architectures can provide Agents with greater opportunities to observe, diagnose, and optimize system performance. Key Takeaways Attendees will leave with a new conceptual framework, three technical keys, and a forward-looking perspective—understanding not only why the paradigm is changing, but also how to respond and where the next opportunities lie. 1. A New Understanding: The Fundamental Paradigm Shift in the Agentic AI Era Agentic AI means that raw compute is no longer the only metric for evaluating a chip. Reducing latency across chips, systems, and the broader software ecosystem is becoming an increasingly fundamental part of effective computing capability. Attendees will gain a deeper understanding of how multi-step reasoning, long-context workloads, and Mixture-of-Experts (MoE) models are pushing traditional architectures toward their limits—and why the goal of “computing faster” is giving way to “moving data more intelligently.” 2. Practical Architecture Insights: Three Keys to Designing for Agentic AI Attendees will explore three practical architectural approaches: 1. Asynchronous Memory Access and Synchronization Understand how mechanisms such as TMA can decouple computation from data loading, enabling pipelined designs that keep compute units continuously supplied with data. 2. Low-Latency Remote Communication Learn how Remote Write combined with hardware barriers can reduce synchronization overhead in inter-device communication. 3. Automated Optimization Methodologies Explore how optimal data layouts can be derived automatically from mathematical principles, and how asynchronous programming frameworks can become a foundation for performance optimization. 3. A Strategic Perspective: Toward Forward-Looking Hardware–Software Co-Design The talk will offer a new perspective on hardware–software co-design: future architectures need to provide observability to the Agents running above them. Tomorrow’s chips will not only need to be fast—they will also need to be transparent. Software and Agents should be able to observe relevant internal hardware states and autonomously adjust their execution and optimization strategies accordingly. For engineers and technical leaders working in system architecture, chip selection, AI infrastructure, or AI platform development, this provides a forward-looking framework for understanding how hardware and software may evolve together in the Agentic AI era.