Experiences,  IIITH,  StudentGiri

A wanderer’s tale @Precog

July 20, 2024 /

Summary

My dual degree experience working with Precog and PK has been an immensely fulfilling experience. There’s a lot written about the friendly lab culture at PreCog and rightly so. In this blog I will describe how PK helped me chart my dream undergraduate and master’s journey. TLDR? PK’s remarkable adaptability to new ideas, openness to feedback, caring about students, and overall: research management skills that I will seek inspiration from for the rest of my life, make Precog the best lab in IIIT to explore research.   

Convocation 2024

Unconventional Beginnings

It all began way back when I first “met” PK at the Panini Linguistics Olympiad Selection Camp in 2019, before I even started undergrad. His talk was among the most engaging sessions at the camp, and I was left impressed after my very short interaction with him. So for my first summer internship, I made sure to apply to PK’s lab, which back then was at IIIT Delhi. Despite being in first year, I was shortlisted for the selection task, which was an early sign of Precog’s meritocratic nature — everyone gets a fair chance, and no one is too inexperienced. Unfortunately, I didn’t make it through, but I learnt a lot in the process.  

In my fourth semester, I started working on Machine Unlearning with Ameya Prabhu, an alumnus doing his PhD at Oxford at the time. This was a big deal for me: Ameya was one of the most prolific dual degree graduates from IIIT, and I really vibed with him. I now really wanted a lab where my collaboration with Ameya could count for my dual degree. Luckily, PK was shifting full-time to IIITH, and I saw this as an excellent opportunity to have a cool advisor. At the time, PreCog’s focus was more towards social media analysis and multilingual NLP, and unlearning was only connected through its predominant application being privacy at the time. I was surprised at PK agreeing to a UG2 proposing his own research problem and collaboration. Here’s how he responded:

Starting fast and slow…

From hereon, there was no looking back. I requested PK for twice-a-week meetings to keep my progress accountable. Despite running a big lab with many external commitments, PK agreed. I also requested PK for an Independent Study in 3-1, so I could get a head-start on research, and he approved. One superb thing about PK, he makes himself as available as you want him to be. Our project progressed quite fast in my first summer and 3-1 semester, where I was working remotely as it was still COVID times. We found some questionable assumptions in the nascent domain of unlearning, and tried to correct them through our paper. The paper also benefited a lot from the help of labmates: ‘shepherds’ (labmates reviewing a paper) who helped me fix my writing, and Arjun TH, who was extremely patient in making beautiful illustrations to help explain my fuzzy ideas. By the end of 3-1, we had made our first submission to CVPR, a top-conference, at which point my excitement for research was at its peak. Post-COVID, the previously T-Hub building became our lab, and for me virtually a second home. To date I think it is the best part of campus.

Unfortunately our paper got borderline rejected at CVPR, and then ICML too. In hindsight, our paper, while quite interesting, was not fleshed out enough while being unconventional for unlearning literature at the time. The rejections and quality of reviews were a thrashing for my motivation for Deep Learning (DL) research. Despite the rejections and slow progress in 3-2, PK did not pressure me, only checking in to ensure I stayed on track. In my opinion, how advisors treat you in your lows is the biggest signal, and I am glad PK trusted me to figure it out.

The quirkiness and beauty of the T-Hub building’s lab-space are unparalleled, and will always make me nostalgic.

Exploring, perhaps a little too much 😛

An off-shoot into Social Choice Theory: At the end of 3-2, I got an offer for a research internship in Social Choice Theory, an area I found really intriguing, but was completely different from my existing research. Normally, dual degree students are not recommended to do summer internships outside IIIT. On the back of paper rejections, it went against IIIT norms to allow me to intern in a completely new field. Surprisingly, PK was quite happy and supportive. Indeed, one of the best things about PK is he puts his student’s interests above his own. This not only allowed me a crucial break away from DL research, but I also got to experience a new lab culture and perspective on research. And ofcourse, nothing beats traveling around cities in Europe!

Uncertain research territories: Before leaving for the internship, I encouraged Shashwat Singh to join the lab so we could work together and write a paper with 2 Shashwats. This led to our project on understanding why Language Models at the time were poor at handling “not”, i.e. negation. This research in itself was exploratory, as we did not have clear hypotheses to prove. We were both excited about using interpretability, but we had little experience, and once again the project was quite different from existing topics at the lab. Once again, PK backed us to give it our best shot. Thus, in 4-1 and 4-2, I ended up trying to balance three projects in very different areas: improving the unlearning paper, research continuing from my internship, and the negation project. Needless to say, progress was slow, and often bottlenecked on me, on all three fronts. While it was stressful for me, it was probably much more frustrating for my collaborators, especially PK, who noted this and brought it to my attention. However, rather than intervening to tell you your shiny ambitions are a little too much, PK lets you fall and learn yourself, which I did eventually. Now if I am ever asked how many projects to be a part of simultaneously, I always say at most two. After many delays, we did manage to write and submit papers for all three projects by the end of 4-2.

Tasting a completelely different career path: Right around this time, PK motivated me to try and graduate early, perhaps in 5-1. EARLY? This was the first time I was hearing of an advisor making this suggestion. Around this time, I received an offer for a Quantitative Research internship, which I quite wanted to try as it seemed like the best industry option available in India. Not doing research in the summer would setback any hopes for graduating early, but PK agreed to let me pursue this. During one of our “happiness report” calls at the end of my internship, an opportunity for both the student and PK to give each other one-on-one feedback, PK expressed that I should focus on going for a PhD instead of taking a Quant job. At the time I wasn’t convinced, but a PhD is what I eventually ended up choosing 🙂 So PK was right, but once again let me independently arrive at this conclusion over time.

Converging at Safe AI: After my quant internship, I visited Berkeley for two months as part of the SERI MATS program where I worked with Dan Hendrycks and the Center for AI Safety (CAIS). This experience exposed me to a different style of research, emphasising neglected impactful problems. After the program, PK and I joined a larger collaboration with CAIS and Scale AI, focused on unlearning dual use knowledge from LLMs. Over the years, PK had become increasingly interested in Responsible AI, and many students in the lab wanted to work in this area too. PK decided to dive deeper by offering a new course on Responsible and Safe AI (RSAI) in Spring 2024, and it was inspiring to see his excitement to learn about a new area. I can only wish my eagerness to learn lasts as long 🙂 If an alumnus from a few years ago were to attend a WU (the weekly lab meeting) now, they would probably not recognize it is the same research group from the content discussed. This also makes me proud about my contribution to Precog, from initially exploring this area mostly in isolation, to it growing into a sub-group, and now as of July 2024, it seems to be the lab’s primary focus 🙂 Its amazing to see how fast Precog evolves, made only possible by appreciating new ideas that seem impactful, and the openness to student inputs. 

5-1 ended up being centered around applying for graduate programs, and once again I couldn’t do much research apart from the CAIS collaboration. I think 2022 and 2023 must have been a frustrating experience for PK when it comes to me, as I was exploring way too many different areas and career choices, and not pushing enough on any to form a thesis around it. For me this was a conscious decision, as I believe undergrad is the best time to explore broadly. Changing career paths or even areas (though PK himself is a counterexample to this) seems to gets harder once you join a job or PhD program. I will forever be grateful for PK supporting me in curating what felt pretty close to the ideal undergraduate experience, and here’s an example of PK celebrating my wins:

Exhilarating Final Stretch

So here I was at the start of 5-2, supposed to be my final semester, with only one accepted paper (the negation project), that too at a workshop. Being uncertain about graduating was not a situation either PK or I would have imagined given our fast start. Disappointingly, I didn’t have a lot of output in terms of paper acceptances to show for the effort I’d put into research. January and February were stressful. I was contributing to the CAIS unlearning project being submitted to ICML, working on a new unlearning paper with Ameya, helping curate the first RSAI course in IIIT-H (and perhaps India), all while interviewing for PhD positions. I don’t think I’d be able to pull through if I didn’t love basically living in the lab, which in turn wouldn’t have been possible without some of my closest friends being there the whole day too. And that’s when the fruits of the last 3 years of research started coming through, here’s a timeline:

A big audience of the Precog blogs are students aspiring to take up research. If you are one of them, this final paragraph is for you. I know research can seem daunting, and it can get quite frustrating. Through my time at Precog, I’ve come to realize that if you keep trying to do impactful work that excites you, with the right support structures, the reward does eventually come through. Doing research at Precog is one of the most rewarding experiences you can have at IIIT-H, in terms of both personal growth and profile building. You get the perfect combination of extremely smart and helpful lab-mates, the right mix of freedom and accountability, and most importantly a caring advisor who encourages you to be ambitious and go down rabbit-holes. In the process, you can actually have a lot of fun. One of my favorite memories at IIIT is enjoying a cup of VC coffee and dosa with the view of the city from the lab on a rainy day, in the company of my lab-mates and friends, while trying to consume a stimulating paper or talk on the side. I hope you get the chance to experience this 🙂

When stuck on a problem, try staring out of the lab windows.
It helped me look at the bigger picture 🙂
By  Comments Off on A wanderer’s tale @Precog

You May Also Like

The Journey Known as Precog

September 18, 2017