![\"\"](\"https:\/\/precog.iiit.ac.in\/blog\/wp-content\/uploads\/2024\/07\/IMG_0225-min-768x1024.jpg\")
Unconventional Beginnings<\/h2>\n\n\n\n
It all began way back when I first \u201cmet\u201d PK at the Panini Linguistics Olympiad Selection Camp in 2019<\/a>, before I even started undergrad. His talk was among the most engaging sessions at the camp, and I was left impressed after my very short interaction with him. So for my first summer internship, I made sure to apply to PK\u2019s lab, which back then was at IIIT Delhi. Despite being in first year, I was shortlisted for the selection task, which was an early sign of Precog\u2019s meritocratic nature \u2014 everyone gets a fair chance, and no one is too inexperienced. Unfortunately, I didn\u2019t make it through, but I learnt a lot in the process.\u00a0\u00a0<\/p>\n\n\n\n In my fourth semester, I started working on Machine Unlearning with Ameya Prabhu<\/a>, an alumnus doing his PhD at Oxford at the time. This was a big deal for me: Ameya was one of the most prolific dual degree graduates from IIIT, and I really vibed with him. I now really wanted a lab where my collaboration with Ameya could count for my dual degree. Luckily, PK was shifting full-time to IIITH, and I saw this as an excellent opportunity to have a cool advisor. At the time, PreCog\u2019s focus was more towards social media analysis and multilingual NLP, and unlearning was only connected through its predominant application being privacy at the time. I was surprised at PK agreeing to a UG2 proposing his own research problem and collaboration. Here’s how he responded:<\/p>\n\n\n\n From hereon, there was no looking back. I requested PK for twice-a-week meetings to keep my progress accountable. Despite running a big lab with many external commitments, PK agreed. I also requested PK for an Independent Study in 3-1, so I could get a head-start on research, and he approved. One superb thing about PK, he makes himself as available as you want him to be. Our project progressed quite fast in my first summer and 3-1 semester, where I was working remotely as it was still COVID times. We found some questionable assumptions in the nascent domain of unlearning, and tried to correct them through our paper<\/a>. The paper also benefited a lot from the help of labmates: \u2018shepherds\u2019 (labmates reviewing a paper) who helped me fix my writing, and Arjun TH, who was extremely patient in making beautiful illustrations to help explain my fuzzy ideas. By the end of 3-1, we had made our first submission to CVPR, a top-conference, at which point my excitement for research was at its peak. Post-COVID, the previously T-Hub building became our lab, and for me virtually a second home. To date I think it is the best part of campus.<\/p>\n\n\n\n Unfortunately our paper got borderline rejected at CVPR, and then ICML too. In hindsight, our paper, while quite interesting, was not fleshed out enough while being unconventional for unlearning literature at the time. The rejections and quality of reviews were a thrashing for my motivation for Deep Learning (DL) research. Despite the rejections and slow progress in 3-2, PK did not pressure me, only checking in to ensure I stayed on track. In my opinion, how advisors treat you in your lows is the biggest signal, and I am glad PK trusted me to figure it out.<\/p>\n\n\n\n An off-shoot into Social Choice Theory: <\/strong>At the end of 3-2, I got an offer for a research internship in Social Choice Theory<\/a>, an area I found really intriguing, but was completely different from my existing research. Normally, dual degree students are not recommended to do summer internships outside IIIT. On the back of paper rejections, it went against IIIT norms to allow me to intern in a completely new field. Surprisingly, PK was quite happy and supportive. Indeed, one of the best things about PK is he puts his student\u2019s interests above his own. This not only allowed me a crucial break away from DL research, but I also got to experience a new lab culture and perspective on research. And ofcourse, nothing beats traveling around cities in Europe!<\/p>\n\n\n\n Uncertain research territories: <\/strong>Before leaving for the internship, I encouraged Shashwat Singh to join the lab so we could work together and write a paper with 2 Shashwats. This led to our project<\/a> on understanding why Language Models at the time were poor at handling \u201cnot\u201d, i.e. negation. This research in itself was exploratory, as we did not have clear hypotheses to prove. We were both excited about using interpretability, but we had little experience, and once again the project was quite different from existing topics at the lab. Once again, PK backed us to give it our best shot. Thus, in 4-1 and 4-2, I ended up trying to balance three projects in very different areas: improving the unlearning paper, research continuing from my internship, and the negation project. Needless to say, progress was slow, and often bottlenecked on me, on all three fronts. While it was stressful for me, it was probably much more frustrating for my collaborators, especially PK, who noted this and brought it to my attention. However, rather than intervening to tell you your shiny ambitions are a little too much, PK lets you fall and learn yourself, which I did eventually. Now if I am ever asked how many projects to be a part of simultaneously, I always say at most two. After many delays, we did manage to write and submit papers for all three projects by the end of 4-2.<\/p>\n\n\n\n Tasting a completelely different career path: <\/strong>Right around this time, PK motivated me to try and graduate early, perhaps in 5-1. EARLY? This was the first time I was hearing of an advisor making this suggestion. Around this time, I received an offer for a Quantitative Research internship<\/a>, which I quite wanted to try as it seemed like the best industry option available in India. Not doing research in the summer would setback any hopes for graduating early, but PK agreed to let me pursue this. During one of our \u201chappiness report\u201d calls at the end of my internship, an opportunity for both the student and PK to give each other one-on-one feedback, PK expressed that I should focus on going for a PhD instead of taking a Quant job. At the time I wasn\u2019t convinced, but a PhD is what I eventually ended up choosing \ud83d\ude42 So PK was right, but once again let me independently arrive at this conclusion over time.<\/p>\n\n\n\n Converging at Safe AI: <\/strong>After my quant internship, I visited Berkeley for two months as part of the SERI MATS program <\/a>where I worked with Dan Hendrycks<\/a> and the Center for AI Safety (CAIS)<\/a>. This experience exposed me to a different style of research, emphasising neglected impactful problems. After the program, PK and I joined a larger collaboration with CAIS and Scale AI, focused on unlearning dual use knowledge from LLMs<\/a>. Over the years, PK had become increasingly interested in Responsible AI, and many students in the lab wanted to work in this area too. PK decided to dive deeper by offering a new course on Responsible and Safe AI (RSAI) <\/a>in Spring 2024, and it was inspiring to see his excitement to learn about a new area. I can only wish my eagerness to learn lasts as long \ud83d\ude42 If an alumnus from a few years ago were to attend a WU (the weekly lab meeting) now, they would probably not recognize it is the same research group from the content discussed. This also makes me proud about my contribution to Precog, from initially exploring this area mostly in isolation, to it growing into a sub-group, and now as of July 2024, it seems to be the lab\u2019s primary focus \ud83d\ude42 Its amazing to see how fast Precog evolves, made only possible by appreciating new ideas that seem impactful, and the openness to student inputs.\u00a0<\/p>\n\n\n\n 5-1 ended up being centered around applying for graduate programs, and once again I couldn\u2019t do much research apart from the CAIS collaboration. I think 2022 and 2023 must have been a frustrating experience for PK when it comes to me, as I was exploring way too many different areas and career choices, and not pushing enough on any to form a thesis around it. For me this was a conscious decision, as I believe undergrad is the best time to explore broadly. Changing career paths or even areas (though PK himself is a counterexample to this) seems to gets harder once you join a job or PhD program. I will forever be grateful for PK supporting me in curating what felt pretty close to the ideal undergraduate experience, and here’s an example of PK celebrating my wins:<\/p>\n\n\n\n So here I was at the start of 5-2, supposed to be my final semester, with only one accepted paper (the negation project), that too at a workshop. Being uncertain about graduating was not a situation either PK or I would have imagined given our fast start. Disappointingly, I didn\u2019t have a lot of output in terms of paper acceptances to show for the effort I\u2019d put into research. January and February were stressful. I was contributing to the CAIS unlearning project being submitted to ICML, working on a new unlearning paper with Ameya<\/a>, helping curate the first RSAI course in IIIT-H (and perhaps India), all while interviewing for PhD positions. I don\u2019t think I\u2019d be able to pull through if I didn\u2019t love basically living in the lab, which in turn wouldn’t have been possible without some of my closest friends being there the whole day too. And that\u2019s when the fruits of the last 3 years of research started coming through, here\u2019s a timeline:<\/p>\n\n\n\n A big audience of the Precog blogs are students aspiring to take up research. If you are one of them, this final paragraph is for you. I know research can seem daunting, and it can get quite frustrating. Through my time at Precog, I\u2019ve come to realize that if you keep trying to do impactful work that excites you, with the right support structures, the reward does eventually come through. Doing research at Precog is one of the most rewarding experiences you can have at IIIT-H, in terms of both personal growth and profile building. You get the perfect combination of extremely smart and helpful lab-mates, the right mix of freedom and accountability, and most importantly a caring advisor who encourages you to be ambitious and go down rabbit-holes. In the process, you can actually have a lot of fun. One of my favorite memories at IIIT is enjoying a cup of VC coffee and dosa with the view of the city from the lab on a rainy day, in the company of my lab-mates and friends, while trying to consume a stimulating paper or talk on the side. I hope you get the chance to experience this \ud83d\ude42<\/p>\n\n\n\n Summary My dual degree experience working with Precog and PK has been an immensely fulfilling experience. There\u2019s a lot written about the friendly lab culture at PreCog and rightly so. In this blog I will describe how PK helped me chart my dream undergraduate and master\u2019s journey. TLDR? PK\u2019s remarkable adaptability to new ideas, openness to feedback, caring about students, and overall: research management skills that I will seek inspiration from for the rest of my life, make Precog the best lab in IIIT to explore research. Unconventional Beginnings It all began way back when I first \u201cmet\u201d PK at the Panini Linguistics Olympiad Selection Camp in 2019, before I even started undergrad. His talk was among the most engaging sessions at the camp, and I was left impressed after my very short interaction with him. So for my first summer internship, I made sure to apply to PK\u2019s lab, which back then was at IIIT Delhi. Despite being in first year, I was shortlisted for the selection task, which was an early sign of Precog\u2019s meritocratic nature \u2014 everyone gets a fair chance, and no one is too inexperienced. Unfortunately, I didn\u2019t make it through, but I learnt a lot in the process.\u00a0\u00a0 In my fourth semester, I started working on Machine Unlearning with Ameya Prabhu, an alumnus doing his PhD at Oxford at the time. This was a big deal for me: Ameya was one of the most prolific dual degree graduates from IIIT, and I really vibed with him. I now really wanted a lab where my collaboration with Ameya could count for my dual degree. Luckily, PK was shifting full-time to IIITH, and I saw this as an excellent opportunity to have a cool advisor. At the time, PreCog\u2019s focus was more towards social media analysis and multilingual NLP, and unlearning was only connected through its predominant application being privacy at the time. I was surprised at PK agreeing to a UG2 proposing his own research problem and collaboration. Here’s how he responded: Starting fast and slow\u2026 From hereon, there was no looking back. I requested PK for twice-a-week meetings to keep my progress accountable. Despite running a big lab with many external commitments, PK agreed. I also requested PK for an Independent Study in 3-1, so I could get a head-start on research, and he approved. One superb thing about PK, he makes himself as available as you want him to be. Our project progressed quite fast in my first summer and 3-1 semester, where I was working remotely as it was still COVID times. We found some questionable assumptions in the nascent domain of unlearning, and tried to correct them through our paper. The paper also benefited a lot from the help of labmates: \u2018shepherds\u2019 (labmates reviewing a paper) who helped me fix my writing, and Arjun TH, who was extremely patient in making beautiful illustrations to help explain my fuzzy ideas. By the end of 3-1, we had made our first submission to CVPR, a top-conference, at which point my excitement for research was at its peak. Post-COVID, the previously T-Hub building became our lab, and for me virtually a second home. To date I think it is the best part of campus. Unfortunately our paper got borderline rejected at CVPR, and then ICML too. In hindsight, our paper, while quite interesting, was not fleshed out enough while being unconventional for unlearning literature at the time. The rejections and quality of reviews were a thrashing for my motivation for Deep Learning (DL) research. Despite the rejections and slow progress in 3-2, PK did not pressure me, only checking in to ensure I stayed on track. In my opinion, how advisors treat you in your lows is the biggest signal, and I am glad PK trusted me to figure it out. Exploring, perhaps a little too much \ud83d\ude1b An off-shoot into Social Choice Theory: At the end of 3-2, I got an offer for a research internship in Social Choice Theory, an area I found really intriguing, but was completely different from my existing research. Normally, dual degree students are not recommended to do summer internships outside IIIT. On the back of paper rejections, it went against IIIT norms to allow me to intern in a completely new field. Surprisingly, PK was quite happy and supportive. Indeed, one of the best things about PK is he puts his student\u2019s interests above his own. This not only allowed me a crucial break away from DL research, but I also got to experience a new lab culture and perspective on research. And ofcourse, nothing beats traveling around cities in Europe! Uncertain research territories: Before leaving for the internship, I encouraged Shashwat Singh to join the lab so we could work together and write a paper with 2 Shashwats. This led to our project on understanding why Language Models at the time were poor at handling \u201cnot\u201d, i.e. negation. This research in itself was exploratory, as we did not have clear hypotheses to prove. We were both excited about using interpretability, but we had little experience, and once again the project was quite different from existing topics at the lab. Once again, PK backed us to give it our best shot. Thus, in 4-1 and 4-2, I ended up trying to balance three projects in very different areas: improving the unlearning paper, research continuing from my internship, and the negation project. Needless to say, progress was slow, and often bottlenecked on me, on all three fronts. While it was stressful for me, it was probably much more frustrating for my collaborators, especially PK, who noted this and brought it to my attention. However, rather than intervening to tell you your shiny ambitions are a little too much, PK lets you fall and learn yourself, which I did eventually. Now if I am ever asked how many projects to be a part of simultaneously, I always say at most two. After many delays, we did manage to write and submit papers for all three projects by the end of 4-2. Tasting a completelely different career path: Right around this time, PK motivated me to try and graduate early, perhaps in 5-1. EARLY? This was the first time I was hearing of an advisor making this suggestion. Around this time, I received an offer for a Quantitative Research internship, which I quite wanted to try as it seemed like the best industry option available in India. Not doing research in the summer would setback any hopes for graduating early, but PK agreed to let me pursue this. During one of our \u201chappiness report\u201d calls at the end of my internship, an opportunity for both the student and PK to give each other one-on-one feedback, PK expressed that I should focus on going for a PhD instead of taking a Quant job. At the time I wasn\u2019t convinced, but a PhD is what I eventually ended up choosing \ud83d\ude42 So PK was right, but once again let me independently arrive at this conclusion over time. Converging at Safe AI: After my quant internship, I visited Berkeley for two months as part of the SERI MATS program where I worked with Dan Hendrycks and the Center for AI Safety (CAIS). This experience exposed me to a different style of research, emphasising neglected impactful problems. After the program, PK and I joined a larger collaboration with CAIS and Scale AI, focused on unlearning dual use knowledge from LLMs. Over the years, PK had become increasingly interested in Responsible AI, and many students in the lab wanted to work in this area too. PK decided to dive deeper by offering a new course on Responsible and Safe AI (RSAI) in Spring 2024, and it was inspiring to see his excitement to learn about a new area. I can only wish my eagerness to learn lasts as long \ud83d\ude42 If an alumnus from a few years ago were to attend a WU (the weekly lab meeting) now, they would probably not recognize it is the same research group from the content discussed. This also makes me proud about my contribution to Precog, from initially exploring this area mostly in isolation, to it growing into a sub-group, and now as of July 2024, it seems to be the lab\u2019s primary focus \ud83d\ude42 Its amazing to see how fast Precog evolves, made only possible by appreciating new ideas that seem impactful, and the openness to student inputs.\u00a0 5-1 ended up being centered around applying for graduate programs, and once again I couldn\u2019t do much research apart from the CAIS collaboration. I think 2022 and 2023 must have been a frustrating experience for PK when it comes to me, as I was exploring way too many different areas and career choices, and not pushing enough on any to form a thesis around it. For me this was a conscious decision, as I believe undergrad is the best time to explore broadly. Changing career paths or even areas (though PK himself is a counterexample to this) seems to gets harder once you join a job or PhD program. I will forever be grateful for PK supporting me in curating what felt pretty close to the ideal undergraduate experience, and here’s an example of PK celebrating my wins: Exhilarating Final Stretch So here I was at the start of 5-2, supposed to be my final semester, with only one accepted paper (the negation project), that too at a workshop. Being uncertain about graduating was not a situation either PK or I would have imagined given our fast start. Disappointingly, I didn\u2019t have a lot of output in terms of paper acceptances to show for the effort I\u2019d put into research. January and February were stressful. I was contributing to the CAIS unlearning project being submitted to ICML, working on a new unlearning paper with Ameya, helping curate the first RSAI course in IIIT-H (and perhaps India), all while interviewing for PhD positions. I don\u2019t think I\u2019d be able to pull through if I didn\u2019t love basically living in the lab, which in turn wouldn’t have been possible without some of my closest friends being there the whole day too. And that\u2019s when the fruits of the last 3 years of research started coming through, here\u2019s a timeline: February 15 2024: Received some excellent PhD offers, and also MS admits at CMU February 23 2024: – The social choice theory project I started during my Summer’22 internship won the Outstanding Paper Award (Top 3 out of 12,000+ submissions) at AAAI March 6 2024: The WMDP unlearning project with CAIS and Scale AI got covered by TIME March 7 2024: Our work on Corrective Machine Unlearning accepted at ICLR DMLR Workshop April 10 2024: Recognized as an Exceptional Reviewer at ICLR DMLR Workshop May 1 2024: The WMDP unlearning paper accepted at ICML May 22 2024: Successfully defended my thesis titled: \u201cNew Frontiers for Machine Unlearning\u201d A big audience of the Precog blogs are students aspiring to take up research. If you are one of them, this final paragraph is for you. I know research can seem daunting, and it can get quite frustrating. Through my time at Precog, I\u2019ve come to realize that if you keep trying to do impactful work that excites you, with the right support structures, the reward does eventually come through. Doing research at Precog is one of the most rewarding experiences you can have at IIIT-H, in terms of both personal growth and profile building. You get the perfect combination of extremely smart and helpful lab-mates, the right mix of freedom and accountability, and most importantly a caring advisor who encourages you to be ambitious and go down rabbit-holes. In the process, you can actually have a lot of fun. One of my favorite memories at IIIT is enjoying a cup of VC coffee and dosa with the view of the city from the lab on a rainy day, in the company of my…<\/p>\n","protected":false},"author":123,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":true,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[2,1528,1527],"tags":[],"class_list":["post-5805","post","type-post","status-publish","format-standard","hentry","category-experiences","category-iiith","category-studentgiri"],"_links":{"self":[{"href":"https:\/\/precog.iiit.ac.in\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5805","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/precog.iiit.ac.in\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/precog.iiit.ac.in\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/precog.iiit.ac.in\/blog\/index.php?rest_route=\/wp\/v2\/users\/123"}],"replies":[{"embeddable":true,"href":"https:\/\/precog.iiit.ac.in\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5805"}],"version-history":[{"count":6,"href":"https:\/\/precog.iiit.ac.in\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5805\/revisions"}],"predecessor-version":[{"id":5899,"href":"https:\/\/precog.iiit.ac.in\/blog\/index.php?rest_route=\/wp\/v2\/posts\/5805\/revisions\/5899"}],"wp:attachment":[{"href":"https:\/\/precog.iiit.ac.in\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5805"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/precog.iiit.ac.in\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5805"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/precog.iiit.ac.in\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5805"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![](\"https:\/\/lh7-us.googleusercontent.com\/docsz\/AD_4nXcP3kdmW63gCZ8or6QnFX3f4PhxbO2QRTVpP1RjCXVQ6EULBD8AE4UofCPlsbHV7JlEXAuo1lY67CpKX1YzW1vE1xNITSv__8yDk9FVNsE0KPlOLddDYMCrgbgkAXY41vcWpCuCAjPZackfmmI5PXPONIGX?key=cCwcIXl-1p5EYXfqEp8U9w\")
Starting fast and slow\u2026<\/h2>\n\n\n\n
![\"\"](\"https:\/\/precog.iiit.ac.in\/blog\/wp-content\/uploads\/2024\/07\/IMG20240602123207-min-1-768x1024.jpg\")
<\/figure>![\"\"](\"https:\/\/precog.iiit.ac.in\/blog\/wp-content\/uploads\/2024\/07\/IMG_20230416_115158-min-1-467x1024.jpg\")
Exploring, perhaps a little too much \ud83d\ude1b<\/h2>\n\n\n\n
![](\"https:\/\/lh7-us.googleusercontent.com\/docsz\/AD_4nXdQ-n70RwINzP-9NlU1goJGmC6oKiQU5mXt5VYOEu1jhiQAnECUSX1Akrn4j8RoZCCOCI--pVNt2wszR5dQ7tNrGPPIzURmoo9sWHfiftyi8b8brXAWzwH82ntjj0h090BhaZ-q-cd6g1-AuoWHywwMkvbo?key=cCwcIXl-1p5EYXfqEp8U9w\")
Exhilarating Final Stretch<\/h2>\n\n\n\n
![\"\"](\"https:\/\/precog.iiit.ac.in\/blog\/wp-content\/uploads\/2024\/07\/IMG20240526180623-min-768x1024.jpg\")
<\/figure>![\"\"](\"https:\/\/precog.iiit.ac.in\/blog\/wp-content\/uploads\/2024\/07\/IMG_20240718_182437-min-933x1024.jpg\")
It helped me look at the bigger picture \ud83d\ude42<\/figcaption><\/figure>\n<\/div><\/div>\n","protected":false},"excerpt":{"rendered":"