drjimfan starred saulpw/visidata: A terminal spreadsheet multitool for discovering and arranging data
A terminal spreadsheet multitool for discovering and arranging data. Stars: 9029
Chronological feed of everything captured from Jim Fan.
A terminal spreadsheet multitool for discovering and arranging data. Stars: 9029
PyTorch tutorials.. Stars: 9115
The Modular Platform (includes MAX & Mojo). Stars: 25892
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.. Stars: 42133
NVIDIA Isaac GR00T N1.7 - A Foundation Model for Generalist Robots.. Stars: 6675
an ambient intelligence library. Stars: 6134
Bayesian optimization in PyTorch. Stars: 3502
The world's best AI personal assistant for email. Open source app to help you reach inbox zero fast.. Stars: 10458
Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work!. Stars: 42337
Robust Speech Recognition via Large-Scale Weak Supervision. Stars: 97803
A machine learning software for extracting information from scholarly documents. Stars: 4785
Python framework for creating, editing, and running Noisy Intermediate-Scale Quantum (NISQ) circuits.. Stars: 4923
Automated auditing, performance metrics, and best practices for the web.. Stars: 30047
A PyTorch Library for Accelerating 3D Deep Learning Research. Stars: 5073
Unified framework for robot learning built on NVIDIA Isaac Sim. Stars: 6920
Open-source simulator for autonomous driving research.. Stars: 13835
LocateAnything3D reframes 3D object detection as a next-token prediction task in vision-language models using a Chain-of-Sight (CoS) sequence. It generates 2D detections as visual chain-of-thought, followed by 3D box predictions in a near-to-far object order and per-object factorization (center, dimensions, rotation) for stability. This approach achieves SOTA on Omni3D with 38.90 AP_3D, outperforming prior methods by +13.98 even against baselines with ground-truth 2D inputs, while preserving open-vocabulary and zero-shot generalization.
Make your diffs human readable for improved code quality and faster defect detection. :tada:. Stars: 17987
Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.. Stars: 31039
Course notes for AA203. Stars: 157
⚙️ Python API / wrapper for tmux. Stars: 1158
Ongoing research training transformer models at scale. Stars: 16009
The Julia Programming Language. Stars: 48575
Concise, consistent, and legible badges in SVG and raster format. Stars: 26433
Open Source Computer Vision Library. Stars: 87058
Joplin - the privacy-focused note taking app with sync capabilities for Windows, macOS, Linux, Android and iOS.. Stars: 54321
:house_with_garden: Open source home automation that puts local control and privacy first.. Stars: 86009
Fast C++ logging library.. Stars: 28634
List of Computer Science courses with video lectures.. Stars: 80002
🤗 LeRobot: Making AI for Robotics more accessible with end-to-end learning. Stars: 23151
Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞 . Stars: 355578
MindDojo is a novel framework designed to facilitate the development of generalist embodied agents within the open-ended environment of Minecraft. It integrates open-ended tasks, a massive internet-scale knowledge base (YouTube, Wiki, Reddit), and foundation models for agent training. The framework utilizes "MineClip," a contrastive learning model that associates video and language to provide dense reward signals for training agents to perform a diverse set of tasks in Minecraft.
The robotics field, once slow despite being an early AI application, is undergoing rapid transformation due to advancements in large foundation models, scalable data generation through simulation, and increasingly affordable and robust hardware. This convergence is moving robotics from single-purpose, control-driven machines to adaptable, learning-enabled systems capable of performing diverse tasks and interacting safely with the real world, though challenges in cross-embodiment and data diversity remain critical research areas.
Jim Fan, NVIDIA's Research Manager and co-lead of Embodied AI, discusses the development of general-purpose AI agents, called "foundation agents." These agents generalize across multiple skills, embodiments, and realities. Fan highlights three key projects: Mine Dojo for skill acquisition in open-ended environments, Metamorph for multi-body control, and Urea for automated reward function engineering in dexterous manipulation via hybrid gradient architectures, emphasizing the critical role of large-scale data and scalable foundation models.
Jim Fan, a distinguished research scientist at NVIDIA, discusses his career trajectory from early deep learning research to pioneering embodied AI and robotics. He highlights the consistent theme of pursuing "vibe research"—identifying challenging problems and seeking simple, scalable solutions. Fan emphasizes the shift from static computer vision to embodied vision and the critical role of data maximalism and model minimalism in developing robot foundation models. He also introduces the concept of the "physical Turing test" as the grand challenge for general-purpose robot AI, underscoring the difficulties in data collection for robotics compared to large language models. The discussion culminates in predictions for the future of robotics, including programmable factories, self-driving wet labs, and multi-agent fleets, with a target of 2040 for widespread home robot adoption.
NVIDIA is developing a full-stack computing platform for humanoid robots, integrating chip-level hardware, foundation models like Project GR00T, and advanced simulation tools. This initiative, led by Jim Fan's embodied AI research team, aims to create the 'AI brain' for humanoids, leveraging NVIDIA's strengths in compute and simulation to drive general-purpose robot intelligence for a wide range of applications, from household chores to industrial tasks.
Jim Fan, a distinguished scientist at NVIDIA, discusses the evolution and future of AI agents and robotics, drawing insights from his career at OpenAI, Stanford, and NVIDIA. He emphasizes the importance of selecting "hot problems" and seeking "simple solutions" that scale effectively with data and compute. Fan highlights the shift from traditional AI approaches to end-to-end deep learning, stressing the critical role of data collection and generation for advancing robotics toward general-purpose AI.
CaP-X, a project developed in collaboration by NVIDIA, Berkeley, Stanford, and CMU, has been open-sourced under an MIT license. The project provides publicly available code and paper, indicating a contribution to research in its domain.
CaP-X is an open-source agentic robotics system that leverages large language models (LLMs) to enable robots to perform complex tasks zero-shot and improve through reinforcement learning. It integrates a comprehensive toolkit for perception, control, and visualization, and introduces CaP-Gym for standardized evaluation of LLM-driven robotics. The system demonstrates strong performance in both simulation and real-world environments, surpassing learned policies and human expert code in various manipulation tasks.
The traditional academic conference review process is deemed insufficient and irrelevant given the current pace of AI development. The rapid trajectory toward Artificial General Intelligence (AGI) renders the slow cadence of peer review meaningless for real-time technical progress.
The rise of intelligent agents introduces novel and severe cybersecurity vulnerabilities beyond traditional identity theft, as agents can propagate "vibe" contaminations through various digital artifacts like configuration files, skill directories, or seemingly innocuous documents. This expanded attack surface necessitates a new security paradigm, termed "de-vibing," to implement robust guardrails and accountability mechanisms for agentic frameworks, bridging the gap between indiscriminate trust and risky permission bypassing.
EgoVerse leverages egocentric human data to scale robot learning, moving beyond traditional teleoperation. This approach, supported by the EgoScale and dexterity scaling law, uses behavior cloning from human actions to enhance robot capabilities without direct robot interaction during the learning phase. The initiative, a collaboration across research and industry, provides a comprehensive dataset to facilitate both scientific inquiry and practical scaling of robot learning.
This content announces a successful collaboration between an individual named Jim Fan and the "Sharpa team," indicating a positive outcome for a joint project or initiative. The brevity of the message suggests either a final acknowledgment of a completed project or a significant milestone in an ongoing one.
The provided content contains no technical information or substantive claims, consisting only of a well-wish for future endeavors. No knowledge extraction is possible from this source.
This content comprises a brief congratulatory message from Jim Fan to Karina on the occasion of a launch. No further details about the launch or its significance are provided, rendering the content insubstantial for detailed analysis or knowledge extraction.
The provided content is empty, containing only a user note and an emoji. Therefore, no meaningful knowledge extraction or synthesis can be performed. This indicates a potential issue with content ingestion or availability.
The increasing adoption of AI in financial trading raises questions about the sustainability of "alpha" (excess returns). As AI becomes ubiquitous, competitive advantage may shift towards those with access to the most advanced models. This trend suggests a potential future where AI-driven trading becomes a zero-sum game, pushing firms towards a continuous arms race for superior AI.
The provided content consists solely of an affirmation of "the right direction" without any preceding context, elaboration on what "the right direction" refers to, or supporting evidence. This makes it impossible to extract any meaningful, falsifiable claims.
AI systems can be conceptualized by drawing an analogy from human cognitive processes: System 1 for intuitive, "ape-like" intelligence, and System 2 for traditional, analytical VLM (Vision-Language Model) capabilities. This framework suggests a potential architectural division within AI for handling different types of cognitive tasks. It implies that future AI might integrate these two distinct reasoning paradigms to achieve more comprehensive intelligence.