Robot Picks Grapes Based on Spatial World Model

Explore a brain-inspired model for robots to navigate and reason, currently picking grapes. Learn about its development journey, problem-solving iterations, and future potential compared to LLMs.

Code Base Claude Code NumPy PyTorch JSON Queues text

Overview

We built a new model that is based on the human brain. Right now it can navigate and reason to play Pac Man but we are working on connecting it to a robot and having it pick grapes.

Tech stack

Code

Code.org is the premier global nonprofit platform for computer science education, reaching over 80 million students and 2 million teachers worldwide.

Code.org leads the charge in democratizing computer science through its comprehensive K-12 curriculum and the viral Hour of Code initiative. The platform supports 55 million student accounts and provides high-quality, free resources to help learners master foundational logic, JavaScript, and AI integration. By partnering with major school districts and global governments, Code.org has successfully integrated computer science into the core academic experience for millions of underrepresented students.

https://code.org

View projects
Base

Base is a secure, low-cost Ethereum Layer 2 (L2) network, built on the OP Stack and incubated by Coinbase, designed to scale the blockchain for mass adoption.

Base is Coinbase's flagship L2 solution: an Ethereum-based network built on the open-source OP Stack (Optimism). It delivers full Ethereum Virtual Machine (EVM) compatibility at a fraction of the cost, leveraging Ethereum's mainnet for core security. The primary objective is clear: onboard the next billion users to the cryptoeconomy. Developers gain immediate access to the Coinbase ecosystem, including its 110 million+ verified users and powerful fiat onramps, streamlining the launch and adoption of decentralized applications (dApps).

https://base.org/

View projects
Claude Code

Anthropic's agentic coding tool: Unleash Claude's raw power directly in your terminal or IDE to turn complex, hours-long workflows into a single command.

Claude Code is Anthropic’s powerful agentic coding assistant, designed for high-velocity development. It operates natively within your terminal, IDE (VS Code, JetBrains), or via a web interface, allowing you to delegate complex tasks like feature building, bug fixing, and codebase navigation. The agent plans, edits files, executes commands, and creates commits, maintaining awareness of your entire project structure. Internally, Anthropic engineers using Claude Code reported a 67% increase in productivity, demonstrating its capacity to deliver significant gains for Pro and Max plan users.

https://claude.com/code

View projects
NumPy

NumPy is the fundamental Python package for high-performance scientific computing, centered on the powerful N-dimensional array object (ndarray).

NumPy (Numerical Python) is the foundational library for scientific computing in Python, providing the core `ndarray` (N-dimensional array) object . This object is a homogeneous, fixed-size container that enables highly efficient, vectorized operations: often up to 50x faster than standard Python lists for large datasets . The library includes a comprehensive collection of routines for fast array manipulation, including linear algebra, Fourier transforms, and random simulation . Its performance advantage stems from its core being optimized C and C++ code , establishing it as the universal data structure for data exchange across the entire Python scientific computing ecosystem.

https://numpy.org

View projects
PyTorch

PyTorch is the open-source machine learning framework: it provides a Python-first tensor library with strong GPU acceleration and a dynamic computation graph for building deep neural networks.

PyTorch, developed by Meta AI, is a premier open-source deep learning framework favored in both research and production environments. Its core is a powerful tensor library (like NumPy) optimized for GPU acceleration, delivering 50x or greater speedups for complex computations. The key differentiator is its 'Pythonic' design and dynamic computation graph (eager execution), which allows for rapid prototyping and simplified debugging compared to static-graph frameworks. Leveraging its Autograd system for automatic differentiation, practitioners build and train models for computer vision and NLP; major companies like Tesla (Autopilot) and Microsoft utilize PyTorch for critical AI applications.

https://pytorch.org

View projects
JSON

JSON (JavaScript Object Notation): A lightweight, language-independent data format for structured data interchange, built on universally supported key-value pairs and ordered arrays.

JSON is the standard for modern data exchange, leveraging human-readable text to transmit structured information. Its minimal syntax uses two core types: objects (unordered collections of name:value pairs, like {"user": "alpha"}) and arrays (ordered lists of values, like [1, 2, 3]). This structure, derived from JavaScript, ensures efficient parsing and generation across all major programming languages (e.g., Python, Java, Go). We deploy JSON extensively: it is the de facto payload for RESTful APIs and the preferred format for application configuration files (e.g., package.json). The current specification (RFC 8259) ensures consistent, high-scale interoperability.

https://json.org

View projects
Queues

Decouple microservices and scale asynchronously with high-throughput message buffers.

Queues act as the shock absorbers of distributed systems: they ingest bursts of traffic and protect downstream services from overload. By using a producer-consumer pattern (like AWS SQS or RabbitMQ), you eliminate the need for immediate synchronous responses. This architecture ensures 99.99% availability even during database maintenance or API outages. Whether you are processing 1,000 image uploads per second or scheduling delayed email notifications, queues provide the persistence and retry logic necessary to prevent data loss.

https://aws.amazon.com/sqs/

View projects
text

Text technology (NLP) processes and interprets human language data, enabling applications like sentiment analysis, machine translation, and advanced content classification.

We handle Natural Language Processing (NLP): the core technology for making computers understand text. This is a critical AI subfield, transforming raw text into actionable data through specific steps (tokenization, parsing, classification). Modern systems, leveraging large language models (LLMs) like GPT and BERT, now perform complex tasks: extracting key entities from thousands of legal documents, classifying customer feedback with 90%+ accuracy, and powering sophisticated conversational agents. We are building the engine that turns unstructured language into structured, high-value intelligence for any enterprise.

https://www.nltk.org/

View projects

Robot Picks Grapes Based on Spatial World Model

Related talks