Publications | Precog Research Group

2025

AAAI

Higher Order Structures For Graph Explanations

Akshit Sinha^*, Sreeram Vennam^*, Charu Sharma, and Ponnurangam Kumaraguru

In The 39th Annual AAAI Conference on Artificial Intelligence, 2025

arXiv Cite Twitter/X Thread

@inproceedings{sinha2024higherorderstructuresgraph,
  title = {Higher Order Structures For Graph Explanations},
  author = {Sinha, Akshit and Vennam, Sreeram and Sharma, Charu and Kumaraguru, Ponnurangam},
  year = {2025},
  eprint = {2406.03253},
  archiveprefix = {arXiv},
  primaryclass = {cs.LG},
  booktitle = {The 39th Annual AAAI Conference on Artificial Intelligence},
}

2024

UniReps @ NeurIPS

Emergence of Text Semantics in CLIP Image Encoders

Sreeram Vennam, Shashwat Singh, Anirudh Govil, and Ponnurangam Kumaraguru

In UniReps: 2nd Edition of the Workshop on Unifying Representations in Neural Models, 2024

Cite PDF Twitter/X Thread

@inproceedings{vennam2024emergence,
  title = {Emergence of Text Semantics in {CLIP} Image Encoders},
  author = {Vennam, Sreeram and Singh, Shashwat and Govil, Anirudh and Kumaraguru, Ponnurangam},
  booktitle = {UniReps: 2nd Edition of the Workshop on Unifying Representations in Neural Models},
  year = {2024},
}

JURIX

InSaAF: Incorporating Safety Through Accuracy and Fairness - Are LLMs Ready for the Indian Legal Domain?

Yogesh Tripathi, Raghav Donakanti, Sahil Girhepuje, Ishan Kavathekar, Bhaskara Hanuma Vedula, Gokul S. Krishnan, Anmol Goel, Shreya Goyal, Balaraman Ravindran, and Ponnurangam Kumaraguru

2024

DOI Cite

@inbook{Tripathi2024,
  title = {InSaAF: Incorporating Safety Through Accuracy and Fairness - Are LLMs Ready for the Indian Legal Domain?},
  isbn = {9781643685625},
  issn = {1879-8314},
  url = {http://dx.doi.org/10.3233/FAIA241266},
  doi = {10.3233/faia241266},
  booktitle = {Legal Knowledge and Information Systems - JURIX 2024: The Thirty-seventh Annual Conference, Brno, Czech Republic, 11-13 December 2024},
  series = {Frontiers in Artificial Intelligence and Applications},
  author = {Tripathi, Yogesh and Donakanti, Raghav and Girhepuje, Sahil and Kavathekar, Ishan and Vedula, Bhaskara Hanuma and Krishnan, Gokul S. and Goel, Anmol and Goyal, Shreya and Ravindran, Balaraman and Kumaraguru, Ponnurangam},
  year = {2024},
}

TMLR

Corrective Machine Unlearning

Shashwat Goel, Ameya Prabhu, Philip Torr, Ponnurangam Kumaraguru, and Amartya Sanyal

Transactions on Machine Learning Research, 2024

arXiv Cite Twitter/X Thread

@article{goel2024corrective,
  title = {Corrective Machine Unlearning},
  author = {Goel, Shashwat and Prabhu, Ameya and Torr, Philip and Kumaraguru, Ponnurangam and Sanyal, Amartya},
  journal = {Transactions on Machine Learning Research},
  issn = {2835-8856},
  year = {2024},
}

MLC @ NeurIPS

LLM Vocabulary Compression for Low-Compute Environments

Sreeram Vennam, Anish R Joishy, and Ponnurangam Kumaraguru

In Workshop on Machine Learning and Compression, NeurIPS 2024, 2024

arXiv Cite Twitter/X Thread

@inproceedings{vennam2024llm,
  title = {{LLM} Vocabulary Compression for Low-Compute Environments},
  author = {Vennam, Sreeram and Joishy, Anish R and Kumaraguru, Ponnurangam},
  booktitle = {Workshop on Machine Learning and Compression, NeurIPS 2024},
  year = {2024},
  url = {https://openreview.net/forum?id=dtcizKLshW},
}

NeurIPS

Random Representations Outperform Online Continually Learned Representations

Ameya Prabhu, Shiven Sinha, Ponnurangam Kumaraguru, Philip Torr, Ozan Sener, and Puneet K. Dokania

In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

Cite Twitter/X Thread

@inproceedings{prabhu2024random,
  title = {Random Representations Outperform Online Continually Learned Representations},
  author = {Prabhu, Ameya and Sinha, Shiven and Kumaraguru, Ponnurangam and Torr, Philip and Sener, Ozan and Dokania, Puneet K.},
  booktitle = {The Thirty-eighth Annual Conference on Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=TZ5k9IYBBf},
}

MathAI @ NeurIPS

Wu’s Method Boosts Symbolic AI to Rival Silver Medalists and AlphaGeometry to Outperform Gold Medalists at IMO Geometry

Shiven Sinha, Ameya Prabhu, Ponnurangam Kumaraguru, Siddharth Bhat, and Matthias Bethge

In The 4th Workshop on Mathematical Reasoning and AI at NeurIPS’24, 2024

Cite Twitter/X Thread

@inproceedings{sinha2024wus,
  title = {Wu{\textquoteright}s Method Boosts Symbolic {AI} to Rival Silver Medalists and AlphaGeometry to Outperform Gold Medalists at {IMO} Geometry},
  author = {Sinha, Shiven and Prabhu, Ameya and Kumaraguru, Ponnurangam and Bhat, Siddharth and Bethge, Matthias},
  booktitle = {The 4th Workshop on Mathematical Reasoning and AI at NeurIPS'24},
  year = {2024},
  url = {https://openreview.net/forum?id=aKRtC45gle},
}

ICML

Representation Surgery: Theory and Practice of Affine Steering

Shashwat Singh, Shauli Ravfogel, Jonathan Herzig, Roee Aharoni, Ryan Cotterell, and Ponnurangam Kumaraguru

In Forty-first International Conference on Machine Learning, 2024

Cite

@inproceedings{singhrepresentation,
  title = {Representation Surgery: Theory and Practice of Affine Steering},
  author = {Singh, Shashwat and Ravfogel, Shauli and Herzig, Jonathan and Aharoni, Roee and Cotterell, Ryan and Kumaraguru, Ponnurangam},
  booktitle = {Forty-first International Conference on Machine Learning},
  year = {2024},
}

ICML
The WMDP Benchmark: Measuring and Reducing Malicious Use with Unlearning

Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew Bo Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, Rishub Tamirisa, Bhrugu Bharathi, Ariel Herbert-Voss, Cort B Breuer, Andy Zou, Mantas Mazeika, Zifan Wang, Palash Oswal, Weiran Lin, Adam Alfred Hunt, Justin Tienken-Harder, Kevin Y. Shih, Kemper Talley, John Guan, Ian Steneker, David Campbell, Brad Jokubaitis, Steven Basart, Stephen Fitz, Ponnurangam Kumaraguru, Kallol Krishna Karmakar, Uday Tupakula, Vijay Varadharajan, Yan Shoshitaishvili, Jimmy Ba, Kevin M. Esvelt, Alexandr Wang, and Dan Hendrycks

In Forty-first International Conference on Machine Learning, 2024

Cite
@inproceedings{li2024the, title = {The {WMDP} Benchmark: Measuring and Reducing Malicious Use with Unlearning}, author = {Li, Nathaniel and Pan, Alexander and Gopal, Anjali and Yue, Summer and Berrios, Daniel and Gatti, Alice and Li, Justin D. and Dombrowski, Ann-Kathrin and Goel, Shashwat and Mukobi, Gabriel and Helm-Burger, Nathan and Lababidi, Rassin and Justen, Lennart and Liu, Andrew Bo and Chen, Michael and Barrass, Isabelle and Zhang, Oliver and Zhu, Xiaoyuan and Tamirisa, Rishub and Bharathi, Bhrugu and Herbert-Voss, Ariel and Breuer, Cort B and Zou, Andy and Mazeika, Mantas and Wang, Zifan and Oswal, Palash and Lin, Weiran and Hunt, Adam Alfred and Tienken-Harder, Justin and Shih, Kevin Y. and Talley, Kemper and Guan, John and Steneker, Ian and Campbell, David and Jokubaitis, Brad and Basart, Steven and Fitz, Stephen and Kumaraguru, Ponnurangam and Karmakar, Kallol Krishna and Tupakula, Uday and Varadharajan, Vijay and Shoshitaishvili, Yan and Ba, Jimmy and Esvelt, Kevin M. and Wang, Alexandr and Hendrycks, Dan}, booktitle = {Forty-first International Conference on Machine Learning}, year = {2024}, url = {https://openreview.net/forum?id=xlr6AUDuJz}, }

CoLLAs-W

Sanity Checks for Evaluating Graph Unlearning

Varshita Kolipaka^*, Akshit Sinha^*, Debangan Mishra, Sumit Kumar, Arvindh Arun, Shashwat Goel, and Ponnurangam Kumaraguru

In Third Conference on Lifelong Learning Agents - Workshop Track, 2024

Cite

@inproceedings{anonymous2024sanity,
  title = {Sanity Checks for Evaluating Graph Unlearning},
  author = {Kolipaka, Varshita and Sinha, Akshit and Mishra, Debangan and Kumar, Sumit and Arun, Arvindh and Goel, Shashwat and Kumaraguru, Ponnurangam},
  booktitle = {Third Conference on Lifelong Learning Agents - Workshop Track},
  year = {2024},
  url = {https://openreview.net/forum?id=dDnjxgFAWh},
}

KIL @ KDD

Towards Infusing Auxiliary Knowledge for Distracted Driver Detection

Ishwar Balappanawar, Ashmit Chamoli, Ruwan Wickramarachchi, Aditya Mishra, and Ponnurangam Kumaraguru

In Fourth Workshop on Knowledge-infused Learning co-located with 30th ACM KDD Conference, Barcelona, Spain, 2024

Cite

@inproceedings{balappanawar2024towards,
  title = {Towards Infusing Auxiliary Knowledge for Distracted Driver Detection},
  author = {Balappanawar, Ishwar and Chamoli, Ashmit and Wickramarachchi, Ruwan and Mishra, Aditya and Kumaraguru, Ponnurangam},
  booktitle = {Fourth Workshop on Knowledge-infused Learning co-located with 30th ACM KDD Conference, Barcelona, Spain},
  year = {2024},
  organization = {ACM},
  url = {https://openreview.net/forum?id=WQdNGavZu3},
}

LREC-COLING
SaGE: Evaluating Moral Consistency in Large Language Models

Vamshi Krishna Bonagiri, Sreeram Vennam, Priyanshul Govil, Ponnurangam Kumaraguru, and Manas Gaur

In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), 2024

Abs Cite

Despite recent advancements showcasing the impressive capabilities of Large Language Models (LLMs) in conversational systems, we show that even state-of-the-art LLMs are morally inconsistent in their generations, questioning their reliability (and trustworthiness in general). Prior works in LLM evaluation focus on developing ground-truth data to measure accuracy on specific tasks. However, for moral scenarios that often lack universally agreed-upon answers, consistency in model responses becomes crucial for their reliability. To address this issue, we propose an information-theoretic measure called Semantic Graph Entropy (SaGE), grounded in the concept of “Rules of Thumb” (RoTs) to measure a model‘s moral consistency. RoTs are abstract principles learned by a model and can help explain their decision-making strategies effectively. To this extent, we construct the Moral Consistency Corpus (MCC), containing 50K moral questions, responses to them by LLMs, and the RoTs that these models followed. Furthermore, to illustrate the generalizability of SaGE, we use it to investigate LLM consistency on two popular datasets – TruthfulQA and HellaSwag. Our results reveal that task accuracy and consistency are independent problems, and there is a dire need to investigate these issues further.
@inproceedings{bonagiri-etal-2024-sage, title = {{S}a{GE}: Evaluating Moral Consistency in Large Language Models}, author = {Bonagiri, Vamshi Krishna and Vennam, Sreeram and Govil, Priyanshul and Kumaraguru, Ponnurangam and Gaur, Manas}, editor = {Calzolari, Nicoletta and Kan, Min-Yen and Hoste, Veronique and Lenci, Alessandro and Sakti, Sakriani and Xue, Nianwen}, booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)}, year = {2024}, address = {Torino, Italia}, url = {https://aclanthology.org/2024.lrec-main.1243/}, pages = {14272--14284}, }

EMNLP Findings

Counter Turing Test (CT^2): Investigating AI-Generated Text Detection for Hindi - Ranking LLMs based on Hindi AI Detectability Index (ADI_hi)

Ishan Kavathekar, Anku Rani, Ashmit Chamoli, Ponnurangam Kumaraguru, Amit P. Sheth, and Amitava Das

In Findings of the Association for Computational Linguistics: EMNLP 2024, 2024

DOI Cite

@inproceedings{kavathekar-etal-2024-counter,
  title = {Counter {T}uring Test ($CT^2$): Investigating {AI}-Generated Text Detection for {H}indi - Ranking {LLM}s based on {H}indi {AI} Detectability Index ($ADI\_{hi}$)},
  author = {Kavathekar, Ishan and Rani, Anku and Chamoli, Ashmit and Kumaraguru, Ponnurangam and Sheth, Amit P. and Das, Amitava},
  editor = {Al-Onaizan, Yaser and Bansal, Mohit and Chen, Yun-Nung},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2024},
  year = {2024},
  address = {Miami, Florida, USA},
  url = {https://aclanthology.org/2024.findings-emnlp.282/},
  doi = {10.18653/v1/2024.findings-emnlp.282},
  pages = {4902--4926},
}

ICWSM

Put Your Money Where Your Mouth Is: Dataset and Analysis of Real World Habit Building Attempts

Hitkul Jangra, Rajiv Shah, and Ponnurangam Kumaraguru

Proceedings of the International AAAI Conference on Web and Social Media, 2024

DOI Cite

@article{article,
  author = {Jangra, Hitkul and Shah, Rajiv and Kumaraguru, Ponnurangam},
  year = {2024},
  pages = {1967-1978},
  title = {Put Your Money Where Your Mouth Is: Dataset and Analysis of Real World Habit Building Attempts},
  volume = {18},
  journal = {Proceedings of the International AAAI Conference on Web and Social Media},
  doi = {10.1609/icwsm.v18i1.31440},
}

LREC-COLING
Multilingual Coreference Resolution in Low-resource South Asian Languages

Ritwik Mishra, Pooja Desur, Rajiv Ratn Shah, and Ponnurangam Kumaraguru

In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), 2024

Abs Cite

Coreference resolution involves the task of identifying text spans within a discourse that pertain to the same real-world entity. While this task has been extensively explored in the English language, there has been a notable scarcity of publicly accessible resources and models for coreference resolution in South Asian languages. We introduce a Translated dataset for Multilingual Coreference Resolution (TransMuCoRes) in 31 South Asian languages using off-the-shelf tools for translation and word-alignment. Nearly all of the predicted translations successfully pass a sanity check, and 75% of English references align with their predicted translations. Using multilingual encoders, two off-the-shelf coreference resolution models were trained on a concatenation of TransMuCoRes and a Hindi coreference resolution dataset with manual annotations. The best performing model achieved a score of 64 and 68 for LEA F1 and CoNLL F1, respectively, on our test-split of Hindi golden set. This study is the first to evaluate an end-to-end coreference resolution model on a Hindi golden set. Furthermore, this work underscores the limitations of current coreference evaluation metrics when applied to datasets with split antecedents, advocating for the development of more suitable evaluation metrics.
@inproceedings{mishra-etal-2024-multilingual, title = {Multilingual Coreference Resolution in Low-resource {S}outh {A}sian Languages}, author = {Mishra, Ritwik and Desur, Pooja and Shah, Rajiv Ratn and Kumaraguru, Ponnurangam}, editor = {Calzolari, Nicoletta and Kan, Min-Yen and Hoste, Veronique and Lenci, Alessandro and Sakti, Sakriani and Xue, Nianwen}, booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)}, year = {2024}, address = {Torino, Italia}, url = {https://aclanthology.org/2024.lrec-main.1031/}, pages = {11813--11826}, }

ICWSM

Tight Sampling in Unbounded Networks

Kshitijaa Jaglan, Meher Pindiprolu, Triansh Sharma, Abhijeeth Singam, Nidhi Goyal, Ponnurangam Kumaraguru, and Ulrik Brandes

Proceedings of the International AAAI Conference on Web and Social Media, 2024

DOI Cite Twitter/X Thread

@article{articlf,
  author = {Jaglan, Kshitijaa and Pindiprolu, Meher and Sharma, Triansh and Singam, Abhijeeth and Goyal, Nidhi and Kumaraguru, Ponnurangam and Brandes, Ulrik},
  year = {2024},
  pages = {704-716},
  title = {Tight Sampling in Unbounded Networks},
  volume = {18},
  journal = {Proceedings of the International AAAI Conference on Web and Social Media},
  doi = {10.1609/icwsm.v18i1.31345},
}

WOAH @ NAACL
X-posing Free Speech: Examining the Impact of Moderation Relaxation on Online Social Networks

Arvindh Arun, Saurav Chhatani, Jisun An, and Ponnurangam Kumaraguru

In Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024), 2024

Abs DOI Cite

We investigate the impact of free speech and the relaxation of moderation on online social media platforms using Elon Musk’s takeover of Twitter as a case study. By curating a dataset of over 10 million tweets, our study employs a novel framework combining content and network analysis. Our findings reveal a significant increase in the distribution of certain forms of hate content, particularly targeting the LGBTQ+ community and liberals. Network analysis reveals the formation of cohesive hate communities facilitated by influential bridge users, with substantial growth in interactions hinting at increased hate production and diffusion. By tracking the temporal evolution of PageRank, we identify key influencers, primarily self-identified far-right supporters disseminating hate against liberals and woke culture. Ironically, embracing free speech principles appears to have enabled hate speech against the very concept of freedom of expression and free speech itself. Our findings underscore the delicate balance platforms must strike between open expression and robust moderation to curb the proliferation of hate online.
@inproceedings{arun-etal-2024-x, title = {{X}-posing Free Speech: Examining the Impact of Moderation Relaxation on Online Social Networks}, author = {Arun, Arvindh and Chhatani, Saurav and An, Jisun and Kumaraguru, Ponnurangam}, editor = {Chung, Yi-Ling and Talat, Zeerak and Nozza, Debora and Plaza-del-Arco, Flor Miriam and R{\"o}ttger, Paul and Mostafazadeh Davani, Aida and Calabrese, Agostina}, booktitle = {Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024)}, year = {2024}, address = {Mexico City, Mexico}, url = {https://aclanthology.org/2024.woah-1.15}, doi = {10.18653/v1/2024.woah-1.15}, pages = {201--211}, }

2023

repl4nlp @ ACL

Probing Negation in Language Models

Shashwat Singh, Shashwat Goel, Saujas Vaduguru, and Ponnurangam Kumaraguru

2023
ASONAM

Together Apart: Decoding Support Dynamics in Online COVID-19 Communities

Hitkul Jangid, Tanisha Pandey, Sonali Singhal, Pranjal Kandhari, Aryamann Tomar, and Ponnurangam Kumaraguru

In Proceedings of the 2023 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, Kusadasi, Turkiye, 2023

Abs DOI

The COVID-19 pandemic that broke out globally in December 2019 put us all in an unprecedented situation. Social media became a vital source of support and information during the pandemic, as physical interactions were limited by people staying at home. This paper investigates support dynamics and user commitment in an online COVID-19 community of Reddit. We define various support classes and observe them along with user behavior and temporal phases for a coherent in the community. We perform survival analysis using Cox Regression to identify factors influencing a user’s commitment to the community. People seeking more emotional and informational support while they are COVID-positive stay longer in the community. Surprisingly, people who give more support in their early phases are less likely to stay. Additionally, contrary to common belief, our findings show that receiving emotional and informational support has little effect on users’ longevity in the community. Our results lead to a better understanding of user dynamics related to community support and can directly impact moderators and platform owners in designing community guidelines and incentive structures.

BDA

Explaining Finetuned Transformers on Hate Speech Predictions Using Layerwise Relevance Propagation

Ritwik Mishra, Ajeet Yadav, Rajiv Shah, and Ponnurangam Kumaraguru

2023

DOI Cite

@inbook{inbook,
  author = {Mishra, Ritwik and Yadav, Ajeet and Shah, Rajiv and Kumaraguru, Ponnurangam},
  year = {2023},
  pages = {201-214},
  title = {Explaining Finetuned Transformers on Hate Speech Predictions Using Layerwise Relevance Propagation},
  isbn = {978-3-031-49600-4},
  doi = {10.1007/978-3-031-49601-1_14},
  booktitle = {Proceedings of the 11th International Conference on Big Data and Artificial Intelligence},
}

ECAI

CAFIN: Centrality Aware Fairness Inducing IN-Processing for Unsupervised Representation Learning on Graphs

Arvindh Arun, Aakash Aanegola, Amul Agrawal, Ramasuri Narayanam, and Ponnurangam Kumaraguru

In Proceedings of the 26th European Conference on Artificial Intelligence, 2023

DOI Cite

@inproceedings{cafin,
  title = {{CAFIN}: {C}entrality {A}ware {F}airness Inducing {IN}-Processing for Unsupervised Representation Learning on Graphs},
  url = {http://dx.doi.org/10.3233/FAIA230259},
  doi = {10.3233/faia230259},
  booktitle = {Proceedings of the 26th European Conference on Artificial Intelligence},
  journal = {Frontiers in Artificial Intelligence and Applications},
  author = {Arun, Arvindh and Aanegola, Aakash and Agrawal, Amul and Narayanam, Ramasuri and Kumaraguru, Ponnurangam},
  year = {2023},
}

ACL
X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agents

Mehrad Moradshahi, Tianhao Shen, Kalika Bali, Monojit Choudhury, Gael Chalendar, Anmol Goel, Sungkyun Kim, Prashant Kodali, Ponnurangam Kumaraguru, Nasredine Semmar, Sina Semnani, Jiwon Seo, Vivek Seshadri, Manish Shrivastava, Michael Sun, Aditya Yadavalli, Chaobin You, Deyi Xiong, and Monica Lam

In Findings of the Association for Computational Linguistics: ACL 2023, 2023

Abs DOI Cite

Task-oriented dialogue research has mainly focused on a few popular languages like English and Chinese, due to the high dataset creation cost for a new language. To reduce the cost, we apply manual editing to automatically translated data. We create a new multilingual benchmark, X-RiSAWOZ, by translating the Chinese RiSAWOZ to 4 languages: English, French, Hindi, Korean; and a code-mixed English-Hindi language.X-RiSAWOZ has more than 18,000 human-verified dialogue utterances for each language, and unlike most multilingual prior work, is an end-to-end dataset for building fully-functioning agents. The many difficulties we encountered in creating X-RiSAWOZ led us to develop a toolset to accelerate the post-editing of a new language dataset after translation. This toolset improves machine translation with a hybrid entity alignment technique that combines neural with dictionary-based methods, along with many automated and semi-automated validation checks. We establish strong baselines for X-RiSAWOZ by training dialogue agents in the zero- and few-shot settings where limited gold data is available in the target language. Our results suggest that our translation and post-editing methodology and toolset can be used to create new high-quality multilingual dialogue agents cost-effectively. Our dataset, code, and toolkit are released open-source.
@inproceedings{moradshahi-etal-2023-x, title = {{X}-{R}i{SAWOZ}: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agents}, author = {Moradshahi, Mehrad and Shen, Tianhao and Bali, Kalika and Choudhury, Monojit and de Chalendar, Gael and Goel, Anmol and Kim, Sungkyun and Kodali, Prashant and Kumaraguru, Ponnurangam and Semmar, Nasredine and Semnani, Sina and Seo, Jiwon and Seshadri, Vivek and Shrivastava, Manish and Sun, Michael and Yadavalli, Aditya and You, Chaobin and Xiong, Deyi and Lam, Monica}, editor = {Rogers, Anna and Boyd-Graber, Jordan and Okazaki, Naoaki}, booktitle = {Findings of the Association for Computational Linguistics: ACL 2023}, year = {2023}, address = {Toronto, Canada}, url = {https://aclanthology.org/2023.findings-acl.174/}, doi = {10.18653/v1/2023.findings-acl.174}, pages = {2773--2794}, }
WASSA @ ACL
PrecogIIITH@WASSA2023: Emotion Detection for Urdu-English Code-mixed Text

Bhaskara Hanuma Vedula, Prashant Kodali, Manish Shrivastava, and Ponnurangam Kumaraguru

In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, 2023

Abs DOI Cite

Code-mixing refers to the phenomenon of using two or more languages interchangeably within a speech or discourse context. This practice is particularly prevalent on social media platforms, and determining the embedded affects in a code-mixed sentence remains as a challenging problem. In this submission we describe our system for WASSA 2023 Shared Task on Emotion Detection in English-Urdu code-mixed text. In our system we implement a multiclass emotion detection model with label space of 11 emotions. Samples are code-mixed English-Urdu text, where Urdu is written in romanised form. Our submission is limited to one of the subtasks - Multi Class classification and we leverage transformer-based Multilingual Large Language Models (MLLMs), XLM-RoBERTa and Indic-BERT. We fine-tune MLLMs on the released data splits, with and without pre-processing steps (translation to english), for classifying texts into the appropriate emotion category. Our methods did not surpass the baseline, and our submission is ranked sixth overall.
@inproceedings{vedula-etal-2023-precogiiith, title = {{P}recog{IIITH}@{WASSA}2023: Emotion Detection for {U}rdu-{E}nglish Code-mixed Text}, author = {Vedula, Bhaskara Hanuma and Kodali, Prashant and Shrivastava, Manish and Kumaraguru, Ponnurangam}, editor = {Barnes, Jeremy and De Clercq, Orph{\'e}e and Klinger, Roman}, booktitle = {Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, {\&} Social Media Analysis}, year = {2023}, address = {Toronto, Canada}, url = {https://aclanthology.org/2023.wassa-1.58/}, doi = {10.18653/v1/2023.wassa-1.58}, pages = {601--605}, }

2022

ACL
SyMCoM - Syntactic Measure of Code Mixing A Study Of English-Hindi Code-Mixing

Prashant Kodali, Anmol Goel, Monojit Choudhury, Manish Shrivastava, and Ponnurangam Kumaraguru

In Findings of the Association for Computational Linguistics: ACL 2022, 2022

Abs DOI Cite

Code mixing is the linguistic phenomenon where bilingual speakers tend to switch between two or more languages in conversations. Recent work on code-mixing in computational settings has leveraged social media code mixed texts to train NLP models. For capturing the variety of code mixing in, and across corpus, Language ID (LID) tags based measures (CMI) have been proposed. Syntactical variety/patterns of code-mixing and their relationship vis-a-vis computational model‘s performance is under explored. In this work, we investigate a collection of English(en)-Hindi(hi) code-mixed datasets from a syntactic lens to propose, SyMCoM, an indicator of syntactic variety in code-mixed text, with intuitive theoretical bounds. We train SoTA en-hi PoS tagger, accuracy of 93.4%, to reliably compute PoS tags on a corpus, and demonstrate the utility of SyMCoM by applying it on various syntactical categories on a collection of datasets, and compare datasets using the measure.
@inproceedings{kodali-etal-2022-symcom, title = {{S}y{MC}o{M} - Syntactic Measure of Code Mixing A Study Of {E}nglish-{H}indi Code-Mixing}, author = {Kodali, Prashant and Goel, Anmol and Choudhury, Monojit and Shrivastava, Manish and Kumaraguru, Ponnurangam}, editor = {Muresan, Smaranda and Nakov, Preslav and Villavicencio, Aline}, booktitle = {Findings of the Association for Computational Linguistics: ACL 2022}, year = {2022}, address = {Dublin, Ireland}, url = {https://aclanthology.org/2022.findings-acl.40/}, doi = {10.18653/v1/2022.findings-acl.40}, pages = {472--480}, }
INLG
PreCogIIITH at HinglishEval : Leveraging Code-Mixing Metrics & Language Model Embeddings To Estimate Code-Mix Quality

Prashant Kodali, Tanmay Sachan, Akshay Goindani, Anmol Goel, Naman Ahuja, Manish Shrivastava, and Ponnurangam Kumaraguru

In Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges, 2022

Abs Cite

Code-Mixing is a phenomenon of mixing two or more languages in a speech event and is prevalent in multilingual societies. Given the low-resource nature of Code-Mixing, machine generation of code-mixed text is a prevalent approach for data augmentation. However, evaluating the quality of such machine gen- erated code-mixed text is an open problem. In our submission to HinglishEval, a shared- task collocated with INLG2022, we attempt to build models factors that impact the quality of synthetically generated code-mix text by pre- dicting ratings for code-mix quality. Hingli- shEval Shared Task consists of two sub-tasks - a) Quality rating prediction); b) Disagree- ment prediction. We leverage popular code- mixed metrics and embeddings of multilin- gual large language models (MLLMs) as fea- tures, and train task specific MLP regression models. Our approach could not beat the baseline results. However, for Subtask-A our team ranked a close second on F-1 and Co- hen‘s Kappa Score measures and first for Mean Squared Error measure. For Subtask-B our ap- proach ranked third for F1 score, and first for Mean Squared Error measure. Code of our submission can be accessed here.
@inproceedings{kodali-etal-2022-precogiiith, title = {{P}re{C}og{IIITH} at {H}inglish{E}val : Leveraging Code-Mixing Metrics {\&} Language Model Embeddings To Estimate Code-Mix Quality}, author = {Kodali, Prashant and Sachan, Tanmay and Goindani, Akshay and Goel, Anmol and Ahuja, Naman and Shrivastava, Manish and Kumaraguru, Ponnurangam}, editor = {Shaikh, Samira and Ferreira, Thiago and Stent, Amanda}, booktitle = {Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges}, year = {2022}, address = {Waterville, Maine, USA and virtual meeting}, url = {https://aclanthology.org/2022.inlg-genchal.4/}, pages = {26--30}, }
LREC
HashSet - A Dataset For Hashtag Segmentation

Prashant Kodali, Akshala Bhatnagar, Naman Ahuja, Manish Shrivastava, and Ponnurangam Kumaraguru

In Proceedings of the Thirteenth Language Resources and Evaluation Conference, 2022

Abs Cite

Hashtag segmentation is the task of breaking a hashtag into its constituent tokens. Hashtags often encode the essence of user-generated posts, along with information like topic and sentiment, which are useful in downstream tasks. Hashtags prioritize brevity and are written in unique ways - transliterating and mixing languages, spelling variations, creative named entities. Benchmark datasets used for the hashtag segmentation task - STAN, BOUN - are small and extracted from a single set of tweets. However, datasets should reflect the variations in writing styles of hashtags and account for domain and language specificity, failing which the results will misrepresent model performance. We argue that model performance should be assessed on a wider variety of hashtags, and datasets should be carefully curated. To this end, we propose HashSet, a dataset comprising of: a) 1.9k manually annotated dataset; b) 3.3M loosely supervised dataset. HashSet dataset is sampled from a different set of tweets when compared to existing datasets and provides an alternate distribution of hashtags to build and validate hashtag segmentation models. We analyze the performance of SOTA models for Hashtag Segmentation, and show that the proposed dataset provides an alternate set of hashtags to train and assess models.
@inproceedings{kodali-etal-2022-hashset, title = {{H}ash{S}et - A Dataset For Hashtag Segmentation}, author = {Kodali, Prashant and Bhatnagar, Akshala and Ahuja, Naman and Shrivastava, Manish and Kumaraguru, Ponnurangam}, editor = {Calzolari, Nicoletta and B{\'e}chet, Fr{\'e}d{\'e}ric and Blache, Philippe and Choukri, Khalid and Cieri, Christopher and Declerck, Thierry and Goggi, Sara and Isahara, Hitoshi and Maegaard, Bente and Mariani, Joseph and Mazo, H{\'e}l{\`e}ne and Odijk, Jan and Piperidis, Stelios}, booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation Conference}, year = {2022}, address = {Marseille, France}, url = {https://aclanthology.org/2022.lrec-1.782/}, pages = {7215--7219}, }

ACM HT

Erasing Labor with Labor: Dark Patterns and Lockstep Behaviors on Google Play

Ashwin Singh, Arvindh Arun, Pulak Malhotra, Pooja Desur, Ayushi Jain, Duen Horng Chau, and Ponnurangam Kumaraguru

In Proceedings of the 33rd ACM Conference on Hypertext and Social Media, Barcelona, Spain, 2022

DOI Cite

@inproceedings{acmht-22,
  author = {Singh, Ashwin and Arun, Arvindh and Malhotra, Pulak and Desur, Pooja and Jain, Ayushi and Chau, Duen Horng and Kumaraguru, Ponnurangam},
  title = {{E}rasing {L}abor with {L}abor: {D}ark {P}atterns and {L}ockstep {B}ehaviors on {G}oogle {P}lay},
  year = {2022},
  isbn = {9781450392334},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3511095.3536368},
  doi = {10.1145/3511095.3536368},
  booktitle = {Proceedings of the 33rd ACM Conference on Hypertext and Social Media},
  pages = {186–191},
  numpages = {6},
  keywords = {Google Play, Fraud, Dark Patterns},
  location = {Barcelona, Spain},
  series = {HT '22},
}

Towards adversarial evaluations for inexact machine unlearning

Shashwat Goel, Ameya Prabhu, Amartya Sanyal, Ser-Nam Lim, Philip Torr, and Ponnurangam Kumaraguru

arXiv preprint arXiv:2201.06640, 2022

Cite

@article{goel2022towards,
  title = {Towards adversarial evaluations for inexact machine unlearning},
  author = {Goel, Shashwat and Prabhu, Ameya and Sanyal, Amartya and Lim, Ser-Nam and Torr, Philip and Kumaraguru, Ponnurangam},
  journal = {arXiv preprint arXiv:2201.06640},
  year = {2022},
}

EMNLP
An Unsupervised, Geometric and Syntax-aware Quantification of Polysemy

Anmol Goel, Charu Sharma, and Ponnurangam Kumaraguru

In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022

Abs DOI Cite

Polysemy is the phenomenon where a single word form possesses two or more related senses. It is an extremely ubiquitous part of natural language and analyzing it has sparked rich discussions in the linguistics, psychology and philosophy communities alike. With scarce attention paid to polysemy in computational linguistics, and even scarcer attention toward quantifying polysemy, in this paper, we propose a novel, unsupervised framework to compute and estimate polysemy scores for words in multiple languages. We infuse our proposed quantification with syntactic knowledge in the form of dependency structures. This informs the final polysemy scores of the lexicon motivated by recent linguistic findings that suggest there is an implicit relation between syntax and ambiguity/polysemy. We adopt a graph based approach by computing the discrete Ollivier Ricci curvature on a graph of the contextual nearest neighbors. We test our framework on curated datasets controlling for different sense distributions of words in 3 typologically diverse languages - English, French and Spanish. The effectiveness of our framework is demonstrated by significant correlations of our quantification with expert human annotated language resources like WordNet. We observe a 0.3 point increase in the correlation coefficient as compared to previous quantification studies in English. Our research leverages contextual language models and syntactic structures to empirically support the widely held theoretical linguistic notion that syntax is intricately linked to ambiguity/polysemy.
@inproceedings{goel-etal-2022-unsupervised, title = {An Unsupervised, Geometric and Syntax-aware Quantification of Polysemy}, author = {Goel, Anmol and Sharma, Charu and Kumaraguru, Ponnurangam}, editor = {Goldberg, Yoav and Kozareva, Zornitsa and Zhang, Yue}, booktitle = {Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing}, year = {2022}, address = {Abu Dhabi, United Arab Emirates}, url = {https://aclanthology.org/2022.emnlp-main.722}, doi = {10.18653/v1/2022.emnlp-main.722}, pages = {10565--10574}, }

2021

CALCS @ ACL
CoMeT: Towards Code-Mixed Translation Using Parallel Monolingual Sentences

Devansh Gautam, Prashant Kodali, Kshitij Gupta, Anmol Goel, Manish Shrivastava, and Ponnurangam Kumaraguru

In Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching, 2021

Abs DOI Cite

Code-mixed languages are very popular in multilingual societies around the world, yet the resources lag behind to enable robust systems on such languages. A major contributing factor is the informal nature of these languages which makes it difficult to collect code-mixed data. In this paper, we propose our system for Task 1 of CACLS 2021 to generate a machine translation system for English to Hinglish in a supervised setting. Translating in the given direction can help expand the set of resources for several tasks by translating valuable datasets from high resource languages. We propose to use mBART, a pre-trained multilingual sequence-to-sequence model, and fully utilize the pre-training of the model by transliterating the roman Hindi words in the code-mixed sentences to Devanagri script. We evaluate how expanding the input by concatenating Hindi translations of the English sentences improves mBART‘s performance. Our system gives a BLEU score of 12.22 on test set. Further, we perform a detailed error analysis of our proposed systems and explore the limitations of the provided dataset and metrics.
@inproceedings{gautam-etal-2021-comet, title = {{C}o{M}e{T}: Towards Code-Mixed Translation Using Parallel Monolingual Sentences}, author = {Gautam, Devansh and Kodali, Prashant and Gupta, Kshitij and Goel, Anmol and Shrivastava, Manish and Kumaraguru, Ponnurangam}, editor = {Solorio, Thamar and Chen, Shuguang and Black, Alan W. and Diab, Mona and Sitaram, Sunayana and Soto, Victor and Yilmaz, Emre and Srinivasan, Anirudh}, booktitle = {Proceedings of the Fifth Workshop on Computational Approaches to Linguistic Code-Switching}, year = {2021}, address = {Online}, url = {https://aclanthology.org/2021.calcs-1.7/}, doi = {10.18653/v1/2021.calcs-1.7}, pages = {47--55}, }

GermEval

Precog-LTRC-IIITH at GermEval 2021: Ensembling Pre-Trained Language Models with Feature Engineering

T. H. Arjun, Arvindh A., and Kumaraguru Ponnurangam

In Proceedings of the GermEval 2021 Shared Task on the Identification of Toxic, Engaging, and Fact-Claiming Comments, 2021

Cite

@inproceedings{germeval-21,
  title = {Precog-{LTRC}-{IIITH} at {G}erm{E}val 2021: Ensembling Pre-Trained Language Models with Feature Engineering},
  author = {Arjun, T. H. and A., Arvindh and Ponnurangam, Kumaraguru},
  booktitle = {Proceedings of the GermEval 2021 Shared Task on the Identification of Toxic, Engaging, and Fact-Claiming Comments},
  year = {2021},
  address = {Duesseldorf, Germany},
  url = {https://aclanthology.org/2021.germeval-1.6},
  pages = {39--46},
}

2012

LBSN

We Know Where You Live: Privacy Characterization of Foursquare Behavior

Tatiana Pontes, Marisa Vasconcelos, Jussara Almeida, Ponnurangam Kumaraguru, and Virgilio Almeida

4th International Workshop on Location-Based Social Networks (LBSN 2012), 2012

Cite

@article{tatiana:we-know-where-you-live:2012:yuqfj,
  author = {Pontes, Tatiana and Vasconcelos, Marisa and Almeida, Jussara and Kumaraguru, Ponnurangam and Almeida, Virgilio},
  journal = {4th International Workshop on Location-Based Social Networks (LBSN 2012)},
  title = {We Know Where You Live: Privacy Characterization of Foursquare Behavior},
  url = {http://precog.iiitd.edu.in/Publications_files/TP_lbsn_2012.pdf},
  year = {2012},
}

Tring! Tring! - An Exploration and Analysis of Interactive Voice Response Systems

Siddharth Asthana, Pushpendra Singh, Ponnurangam Kumaraguru, Amarjeet Singh, and Vinayak Naik

IndiaHCI, 2012
Credibility Ranking of Tweets during High Impact Events

Aditi Gupta, and Ponnurangam Kumaraguru

Workshop on Privacy and Security in Online Social Media, Co-located with WWW, 2012

2011

HotMobile

User Controllable Security and Privacy for Mobile Mashups

Shruthi Adappa, Vikas Agarwal, Sunil Goyal, Ponnurangam Kumaraguru, and Sumit Mittal

Accepted at HotMobile, 2011

Cite

@article{adappa:user-controllable-securit:2011:kxyqv,
  author = {Adappa, Shruthi and Agarwal, Vikas and Goyal, Sunil and Kumaraguru, Ponnurangam and Mittal, Sumit},
  journal = {Accepted at HotMobile},
  title = {{User Controllable Security and Privacy for Mobile Mashups}},
  url = {http://precog.iiitd.edu.in/Publications_files/SA_VA_SG_PK_SM.pdf},
  year = {2011},
}

IEEE SOLI

Enhancing the Rural Self Help Group – Bank Linkage Program

Vikas Agarwal, Vikram Desai, Shalini Kapoor, Ponnurangam Kumaraguru, and Sumit Mittal

IEEE International Conference on Service Operations and Logistics, and Informatics, 2011

Cite

@article{agarwal:enhancing-the-rural-self-:2011:yuqfj,
  author = {Agarwal, Vikas and Desai, Vikram and Kapoor, Shalini and Kumaraguru, Ponnurangam and Mittal, Sumit},
  journal = {IEEE International Conference on Service Operations and Logistics, and Informatics},
  title = {Enhancing the Rural Self Help Group -- Bank Linkage Program},
  year = {2011},
}

Phi.sh/$oCiaL: The Phishing Landscape through Short URLs

Sidharth Chhabra, Anupama Aggarwal, Fabricio Benevenuto, and Ponnurangam Kumaraguru

8th Annual Collaboration, Electronic messaging, Anti-Abuse and Spam Conference, CEAS, 2011
Integrating Linked Open Data with Unstructured Text for Intelligence Gathering Tasks

Archit Gupta, Krishnamurthy Koduvayur Viswanathan, Anupam Joshi, Timothy Finin, and Ponnurangam Kumaraguru

accepted at Information Integration on the Web (IIWeb), Co-located workshop at WWW, 2011
\@Twitter Credibility Ranking of Tweets on Events #breakingnews

Aditi Gupta, and Ponnurangam Kumaraguru

2011
Twitter Explodes with Activity in Mumbai Blasts! A Lifeline or an Unmonitored Daemon in the Lurking?

Aditi Gupta, and Ponnurangam Kumaraguru

2011

precog.iiitd.edu.in/Publications_files/AG_PK_TR_2011.pdf
Home is Safer than the Cloud! Privacy Concerns for Consumer Cloud Storage

Iulia Ion, Niharika Sachdeva, Ponnurangam Kumaraguru, and Srdjan Capkun

In Symposium on Usable Privacy and Security (SOUPS), 2011
Cross-Pollination of Information in Online Social Media: A Case Study on Popular Social Networks

Paridhi Jain, Tiago Rodrigues, Gabriel Magno, Ponnurangam Kumaraguru, and Virgilo Almeida

Accepted at Third International Conference on Social Computing (Short paper), 2011
Marasim: A Novel Jigsaw Based Authentication Scheme using Tagging

Rohit Khot, Srinathan Kannan, and Ponnurangam Kumaraguru

To appear In Proceedings of the 29th ACM International Conference on Human Factors in Computing Systems. CHI., 2011
SMSAssassin : Crowdsourcing Driven Mobile-based System for SMS Spam Filtering

Kuldeep Yadav, Ponnurangam Kumaraguru, Atul Goyal, Ashish Gupta, and Vinayak Naik

Accepted at HotMobile, 2011

2010

Cue : A Framework for Generating Meaningful Feedback in XACML

Sunil Kumar Ghai, Prateek Nigam, and Ponnurangam Kumaraguru

3rd ACM Workshop on Assurable and Usable Security Configuration (in conjunction with CCS 2010), 2010
The Unique Identification Number Project: Challenges and Recommendations

Haricharan Rengamani, Ponnurangam Kumaraguru, Rajarishi Chakraborty, and H. Raghav Rao

The Third International Conference on Ethics and Policy of Biometrics and International Data Sharing (ICEB), 2010
Anti - Terrorism in India

Ponnurangam Kumaraguru, and Ashish Sureka

Terrorism and New Media: Building a Research Network, 2010

2009

Policy framework for security and privacy management

J. Karat, C.-M. Karat, E. Bertino, N. Li, Q. Ni, C. Brodie, J. Lobo, S. B. Calo, L. F. Cranor, P. Kumaraguru, and R. W. Reeder

IBM J. Res. Dev., 2009
Anti-phishing landing page: Turning a 404 into a teachable moment for end users

Ponnurangam Kumaraguru, Lorrie Faith Cranor, and Laura Mather

Conference on Email and Anti-Spam, 2009
School of phish: a real-world evaluation of anti-phishing training

Ponnurangam Kumaraguru, Justin Cranshaw, Alessandro Acquisti, Lorrie Cranor, Jason Hong, Mary Ann Blair, and Theodore Pham

In Proceedings of the 5th Symposium on Usable Privacy and Security, Mountain View, California, 2009

DOI
PhishGuru: A System for Educating Users about Semantic Attacks

Ponnurangam Kumaraguru

School of Computer Science, Carnegie Mellon University, 2009

http://reports-archive.adm.cs.cmu.edu/anon/isr2009/abstracts/09-106.html

2008

A Contextual Method for Evaluating Privacy Preferences

Caroline Sheedy, and Ponnurangam Kumaraguru

In Policies and Research in Identity Management (IDMAN), 2008
Lessons From a Real World Evaluation of Anti-Phishing Training

Ponnurangam Kumaraguru, Steve Sheng, Alessandro Acquisti, Lorrie Faith Cranor, and Jason Hong

e-Crime Researchers Summit, Anti-Phishing Working Group, 2008

2007

A Survey of Privacy Policy Languages

Ponnurangam Kumaraguru, Lorrie Cranor, Jorge Lobo, and Seraphin Calo

Workshop on Usable IT Security Management (USM ’07) at Symposium On Usable Privacy and Security ’07, 2007
Getting Users to Pay Attention to Anti-Phishing Education: Evaluation of Retention and Transfer

Ponnurangam Kumaraguru, Yong Rhee, Steve Sheng, Sharique Hasan, Alessandro Acquisti, Lorrie Faith Cranor, and Jason Hong

e-Crime Researchers Summit, Anti-Phishing Working Group, 2007
Protecting people from phishing: the design and evaluation of an embedded training email system

Ponnurangam Kumaraguru, Yong Rhee, Alessandro Acquisti, Lorrie Faith Cranor, Jason Hong, and Elizabeth Nunge

In CHI ’07: Proceedings of the SIGCHI conference on Human factors in computing systems, San Jose, California, USA, 2007

DOI

2005

Privacy in India: Attitudes and Awareness

Ponnurangam Kumaraguru, and Lorrie Cranor

In Proceedings of the 2005 Workshop on Privacy Enhancing Technologies (PET2005), Dubrovnik, Croatia, 2005