Intelligent Tutors, Robotics and Fun: RISS’18 @ CMU

I’ve (Shiven Mian, B.Tech. Class of 2019) had the privilege of working with the Robotics Institute at Carnegie Mellon University (the world’s largest Robotics facility) for the past 9 months, and spent Summer 2018 (June – August) at CMU’s sprawling campus as a FICCI Research Scholar in the Robotics Institute Summer Scholars (RISS) program (where I was one of 35 undergrads selected around the world). I’m working with Dr. Jack Mostow (Research Professor Emeritus, CMU) on the RoboTutor project (formerly Project LISTEN), which is one of the five $1M Global Finalists in the $15M Global Learning XPRIZE Challenge, funded by Elon Musk. The project is an Android-based Intelligent Tutoring System that enables 7-10 year old children, with no access to schools in remote regions, to learn basic literacy and Math without adult assistance. RoboTutor is currently being beta-tested in Tanzanian villages, and its design as well as the Challenge’s aim could potentially enable millions of deprived children in the future get a basic quality education in their formative years.

If you’d told me 10 months ago I would be writing all this now, I’d have laughed. Seriously. Three months on, the experience still seems surreal today – so this blog would also be a bit long.

I. Applying

Having dabbled with various different fields in the past two and a half years, I really wasn’t sure where to spend my summer experience this year. Though very broadly I had done projects involving Data Science and HCI, I didn’t have one particular area of interest and was mostly versatile. What I did always want though, was that my work should involve (or analyse) interaction with people, and hopefully impact their behavior and lives positively in some way, and I’ve abided by it in most of the work I’ve done during this time. So when I came across RoboTutor and XPRIZE (through a friend on Facebook), I checked if my interests were a match and spoke to a few people in the team, and I realised this is exactly the high-impact social good project I’ve always wanted to be a part of.

Things went uphill quickly from there. I directly emailed Dr. Mostow for applying to RoboTutor, and a couple of interactions later, I started working remotely in Feb 2018, till May. My remote work involved re-designing and developing RoboTutor’s login system for children that utilised faces, streamlining the way we collected logs from the beta test sites, and some server-side work. I also heard about and applied to RISS in the meantime on his suggestion. It’s a fairly standard application – like most others, it requires only an SoP, CV, transcripts and LoRs, along with mentioning what your interests are among the areas RI does research in. Luckily I was one of the 35 selected worldwide in April. To make things sweeter, I also received funding from FICCI for pursuing RISS with RoboTutor. It was pretty flattering for me to be contributing to a project of such massive real-world impact, but being selected in one of the best undergraduate research programs at a place like CMU and being funded for it, however, was well beyond what I had expected. Little did I know that this wasn’t just a research internship – it ended up showing me fields I had never known existed, and impacted my future plans and interests more than anything else I had experienced.

(Cohon University Center, CMU. Glad it wasn’t Winter)

II. Work @ RISS

Since I’d already been working with the team for 4 months, so once I got to CMU in June, I didn’t have much to do to know all the main people in the RoboTutor team, or the massive system architecture RoboTutor had. However, I did work on a different area of RoboTutor for RISS. Me and another fellow RISS scholar, Mononito Goswami, worked on a new analytical tool for aiding RoboTutor’s design improvement process, which we dubbed SPOT. SPOT uses student-interaction data logged by RoboTutor, screen capture videos and AI to automatically detect ‘hot spots’ (major design issues) in the app which are most detrimental to student engagement and learning. The tool then provides designers & developers of RoboTutor with video evidence and causal characteristics of those hot spots – helping them with deriving actionable intelligence to improve RoboTutor accordingly (an example of data-driven iterative design). After getting feedback from Dr. Mostow, the rest of team, and a few other people (especially Ken Koedinger) from CMU and the week-long LearnLab Summer School which we attended, we managed to co-author a working paper on SPOT (for CMU’s Working Papers Journal (WPJ)). We also managed to submit our paper to a top-tier conference. Though this was a great summer outcome, seeing videos of children in our beta sites happily interacting with the things we designed and improved was very satisfying. We knew were contributing to a better learning experience for them, and in future hopefully millions of children. (Update: Accepted to AAAI’19 as a Student Paper 🙂 )

(Part of the RoboTutor Team @ LearnLab. Clockwise: Dr. Mostow, Amogh, me, Corrie, Rebecca, Mononito, Qiwei, Nirmal. Gates-Hillman Centre, CMU)

III. Beyond Tutors

RISS for me wasn’t just a foray into the world of Intelligent Tutoring Systems or Educational Data Mining, it was far more holistic. I am fortunate and privileged to have met some of the brightest peers, and the small cohort size led to us really bonding together as a group. We learned and helped each other in our projects, organised study groups, peer-reviewed each other’s WPJ papers, toured labs, attended RSS’18 (and heard Sergey Levine talk about his work), and even shared lab space (props to Viraj, whose lab space I shared frequently). Being new to many aspects of Robotics, I particularly enjoyed touring my peers’ labs at RI, especially the FRC and the MSL Lab – some of the work being done in the labs was just amazing and futuristic, the kind of stuff I previously saw in those short this-is-the-future videos on YouTube. Lunch talks by RI faculty were hosted every Wednesday as well. Personally, the very first lunch talk (hosted by Red Whittaker) was one of my summer’s highlights because of the impact it left on me (he had spoken about his journey in Robotics and lessons from life), and it was one of the most motivational and insightful talks I had ever heard.

     

(Left: The Search Based Planning Laboratory (SBPL); Right: one of the ballbots at the MSL Lab – the first ballbots ever built)

But the most fun we had as a cohort was in the Robotics workshops. There were three workshops: on ROS (organised by NREC), drones (by DJI) and humanoids (by UBTech). All the workshops were 2-3 days long, and involved full-day tutorials and a one day team competition in the end. It was really exciting to work with the drones and the humanoids, get acquainted with their development environments, and the limited time for the competitions enabled our teams to stay up at nights dabbling with them. It was an amazing experience, and some of us did walk out with some cool stuff to say the least.

(PS: we own them now 🙂 )

RISS also gave us a lot of opportunities to share our work. Apart from the poster session at the end of the program, the lab tours and the WPJ, there were multiple events organised over the summer to share our work with others. I was fortunate to have been one of the panel speakers at the AI4ALL Summer School at CMU, where I interacted and shared my work with RoboTutor with some of the brightest high school STEM students in the US. All in all, seeing the work done in the labs, attending the talks and the workshops, and moreover giving back and sharing what I learned, all massively enhanced my interest in Robotics. It also enhanced my perception that the Robotics Institute (and CMU) is one of the most inclusive research institutes in the world, and more importantly, a place without limits. It truly is.

(AI4ALL Panel Discussion. “How did you get started with AI?”)

IV. The Fun

While it may seem all the various events we had literally every week took up all the time, the 10 weeks at RISS were nothing without the fun. Apart from Pittsburgh (where I was based), I was lucky to get time to explore most of the East Coast – I visited Philadelphia, Washington DC and New York City (which was particularly awesome because I visited on the July 4 week).

 

(Left: Times Square; Right: Fireworks Show over the Hudson River. New York, July 4th 2018)

But the fun I had with the RISS cohort is what I cherish the most. Whether it be the late night parties at each other’s places, checking out different cuisines in Oakland and Shadyside regularly, exploring Downtown, witnessing the Pirates play, going for Arctic Monkeys (!), playing squash or table tennis or ultimate frisbee (and sometimes just FIFA) everyday after work, or simply hanging out near the coffee machines (which are frankly amazing at RI) – we always found a way to stay together as a close group. We all still remain in close touch, and on a personal level, one instance which exemplifies how amazing our bond is (or would turn out to be), was on my birthday when, beyond all my expectations, the entire cohort surprised me with a cake in the middle of one of the talks (and this was barely 2 weeks into RISS). It couldn’t have been any better for me.

     

(Clockwise: AM, one of those random dinners, me with aforementioned cake, Pirates vs Mets at PNC Park)

V. Final week

All good things must unfortunately come to an end, and though I lived the best 10 weeks of my life, time just flew by. The last few days of RISS were both exciting and poignant. The poster presentation sessions went pretty well, where CMU was kind enough to get prominent researchers from both RI and outside of CMU to see our work, and since it was during a time most incoming Graduate Students were moving in, the poster session had an amazing crowd and I made a lot of connections.

(Me, Mononito, Dr. Mostow – RISS Poster Session, Newell-Simon Hall)

The final day had the RISS closing ceremony, which was pretty much a celebration of the summer – our time to say our goodbyes to our advisors and the RISS organisers. The day ended with all of us in the cohort having an amazing farewell party overnight.

 

(Left: receiving my RISS graduation certificate from Dr. John Dolan and Ms. Rachel Burcin, RISS Program Organisers. Right: me with Dr. Mostow)

VI. Acknowledgements

There are few experiences in my career that I could call ‘life-changing’ (it’s also sometimes loosely used), but RISS is comfortably one of them – not just because of the work, the mentorship and the peers, but because it solidified my interests in the fields I liked, exposed me to others I had never imagined existed and helped me be more versatile. I came in as a novice to the field of Robotics and Intelligent Tutors, having never been in a Summer Research Program (or even to the US!). I left more proficient in these fields but more importantly with endless connections, a bag full of memories (and a full SD card), and some invaluable life lessons. I consider myself very lucky and fortunate to have lived this experience.

I’m truly grateful to Dr. Mostow for believing in me through these 9 months, enabling me to live what I lived this Summer at RI and contribute to such an amazing project, but most importantly for the life lessons and anecdotes he shared with me so that I improve as a human and work to my fullest. Whatever happens next for me in my career, these are lessons I will always keep with me. I’m grateful to Mononito for being an amazing colleague, without whose work our summer would never have been as good. I’m also really grateful to Dr.  PK and Dr. Vinayak Naik for recommending me for my RISS application, Indira for helping me on it, and FICCI for sponsoring my stay. And finally, I’m extremely thankful to Precog for providing me an unparalleled environment and exposure to get involved in research at IIITD to begin with and motivating me to aim higher – I couldn’t have been part of a better group here.

(RISS Cohort 2018, Closing Ceremony – University Center, CMU)

Lastly, here’s my 2¢ if you like: Try to work on socially relevant projects if possible, it’s extremely satisfying. Be open to other fields especially during undergrad, you never know when you’ll find something interesting. And more importantly, apply to CMU and RISS. However highly you think of this place, your expectations will be exceeded.

(PS: As always, for any doubts / advice / other help you may require or just a chat, you’re most welcome to contact me through email, Linkedin or Telegram. I’d love to talk!)