Check Out Raghu Ganti's LinkedIn Stats (Last 30 Days)

🚀 Exciting News! 🚀 In a joint effort between IBM Research, Princeton, CMU, and UIUC, we are thrilled to announce the release of our high-performing hybrid Mamba2 model! This model is trained entirely on open datasets, and we’re releasing intermediate and final checkpoints to enable community experimentation. 🔗 Read more: https://lnkd.in/eUBfMTkW Key Takeaways ⚡ Inference Efficiency The Bamba-9B model delivers significant improvements in throughput and latency, enhancing real-time application performance. Benchmarking with vLLM against Llama 3.1 8B for long contexts shows: 🔹 2.5x throughput improvement 🔹 2x lower latency And this is just the beginning – further optimizations are on the way! 🏆 Competitive Benchmarks Bamba-9B performs competitively with state-of-the-art transformer models like Meta Llama 3.1 8B. It matches average benchmark performance (excluding math and MMLU tasks), with clear opportunities to close gaps through extended training and math-focused datasets. 🤝 Open Collaboration Developed entirely with open data, this effort emphasizes transparency and reproducibility, strengthening the foundations of the open-source AI community. 📂 For details, access to the model, and resources, check out the Bamba GitHub repository: https://lnkd.in/eu5CQUuM Let’s collaborate, experiment, and innovate together! 🔍✨

Very excited that IBM released Apache 2.0 licensed #granite models on Hugging Face - you can find more info on these models at: https://lnkd.in/epTZw-xS * comes in various sizes: 8B, 2B dense and 1B, 3B MoE * datasets and mixtures are transparently shared * power law learning rate and two phase training * mup for improved hyper parameter selection - developed by Microsoft and popularized by Cerebras Systems * comparable performance as other OSS models of similar sizes like #llama3.2.

Want to get 2x throughput improvement on your tuning jobs across various HF models without changing any code and effecting model quality? Now you can simply use Hugging Face transformers and TRL to do this! Read more here: https://lnkd.in/gXpMRb3T Key findings: 1. Simple sequence packing results in model quality deterioration 2. Packing with completion only loss needs 4D attention masks Great work between IBM (Rhui Dih Lee, Achintya Kundu, Laura Wynter, Mayank Mishra) and Hugging Face (Arthur Zucker)!

“All the worlds problems can be solved if only we were willing to think” goes the famous saying from Thomas J Watson. We at the PyTorch team at IBM jointly with the PyTorch team at Meta demonstarated an incredible milestone of training a 7B model to 4T tokens in just two weeks!! 🤯🤯 How we got here: 1. PyTorch FSDP 2. PyTorch Compile 3. SDPA 4. A highly performant data loader with some cool math (Linear Congruential Generator) And we kept those H100 GPUs burning hot at near 70C :) Read all about the details here: https://lnkd.in/e2X5htAS And we open sourced all of it: https://lnkd.in/eFFztgTm Hear about it from Sriram Raghavan and Carole-Jean Wu at #ibm #think today in what’s next in AI on open community based model development. Let’s all use those GPUs well and wisely :) Mudhakar Srivatsa Dakshi Agrawal Brian Vaughan, CFA Linsong Chu Davis Wertheimer Less Wright Antoni Viros Martin Joshua Rosenkranz Priya Nagpurkar Amit Sangani

I have been honored with the title of Distinguished Engineer at IBM, a recognition of my significant technical contributions with global impact, as well as my exemplary leadership among IBM’s technical staff. My fourteen-year tenure at IBM has been remarkable, beginning with the development of an innovative geospatial library for Full Earth functions integrated into over 15 IBM products, customer journey analytics at terabyte scales, and now at the forefront of foundation models and Generative AI using PyTorch. I am profoundly thankful for the opportunity to collaborate with numerous team members on a wide range of challenges, from the intricacies of mathematical problems to scaling complex algorithms across hundreds of nodes. My gratitude extends to our partners for tackling challenging problems that led to impactful use cases, and to IBM’s leadership for their vision and unwavering support. I wish to acknowledge Dakshi Agrawal and Mudhakar Srivatsa for their mentorship since my early days at IBM, Linsong Chu and Joshua Rosenkranz for our enduring technical collaboration. A heartfelt thank you to each of you. Lastly, but most importantly, I am grateful to my wife, Melania Valverde, for her incredible patience and for infusing me with positive energy, as well as to my family and friends for their support and encouragement to reach new heights. Here is to continuing to invent what’s next at IBM Research!! #ibm #ibmresearch