publications

2024

1. arXiv

   On the low-shot transferability of [V]-Mamba

   Diganta Misra*, Jay Gala*, and Antonio Orvieto

   arXiv preprint, 2024

   Abs arXiv

   The strength of modern large-scale neural networks lies in their ability to efficiently adapt to new tasks with few examples. Although extensive research has investigated the transferability of Vision Transformers (ViTs) to various downstream tasks under diverse constraints, this study shifts focus to explore the transfer learning potential of [V]-Mamba. We compare its performance with ViTs across different few-shot data budgets and efficient transfer methods. Our analysis yields three key insights into [V]-Mamba’s few-shot transfer performance: (a) [V]-Mamba demonstrates superior or equivalent few-shot learning capabilities compared to ViTs when utilizing linear probing (LP) for transfer, (b) Conversely, [V]-Mamba exhibits weaker or similar few-shot learning performance compared to ViTs when employing visual prompting (VP) as the transfer method, and (c) We observe a weak positive correlation between the performance gap in transfer via LP and VP and the scale of the [V]-Mamba model. This preliminary analysis lays the foundation for more comprehensive studies aimed at furthering our understanding of the capabilities of [V]-Mamba variants and their distinctions from ViTs.

2. arXiv

   Airavata: Introducing Hindi Instruction-tuned LLM

   Jay Gala, Thanmay Jayakumar, Jaavid Aktar Husain, Aswanth Kumar M, Mohammed Safi Ur Rahman Khan, Diptesh Kanojia, and 5 more authors

   arXiv preprint, 2024

   Abs arXiv HTML Code

   We announce the initial release of Airavata, an instruction-tuned LLM for Hindi. Airavata was created by fine-tuning OpenHathi with diverse, instruction-tuning Hindi datasets to make it better suited for assistive tasks. Along with the model, we also share the IndicInstruct dataset, which is a collection of diverse instruction-tuning datasets to enable further research for Indic LLMs. Additionally, we present evaluation benchmarks and a framework for assessing LLM performance across tasks in Hindi. Currently, Airavata supports Hindi, but we plan to expand this to all 22 scheduled Indic languages. You can access all artifacts at https://ai4bharat.github.io/airavata.

3. arXiv

   Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning in Machine Translation

   Everlyn Chimoto, Jay Gala, Orevaoghene Ahia, Julia Kreutzer, Bruce Bassett, and Sara Hooker

   arXiv preprint (coming soon), 2024

   Abs HTML

   Neural Machine Translation models are extremely data and compute-hungry. However, the reality is that not all data is equally significant in driving model generalization. Effectively recognizing and isolating data points crucial to generalization and only training on this subset has the benefit of drastically reducing the compute budget. In this paper, we propose a new data pruning technique Checkpoints Across Time(CAT) that leverages model training signal to identify the most relevant data points for model performance. We benchmark CAT against different data pruning techniques, ranging in complexity from perplexity scores computed by pre-trained models to techniques like COMET-QE (Rei et al., 2020), LASER (Heffernan et al., 2022) and LaBSE (Feng et al., 2022). We find that our CAT techniques outperform far more complex methods with Indo-European languages. Using our best CAT-based method on English-German and English-French translation tasks, we are able to prune up to 50% of the training data while retaining  99% of the model’s performance when trained on the full data. We inspect the data points that our method selects and find that CAT tends to prefer longer sentences and favours sentences with more unique and rare words.

4. arXiv

   An Empirical Study of In-context Learning in LLMs for Machine Translation

   Pranjal A. Chitale*, Jay Gala*, and Raj Dabre

   arXiv preprint, 2024

   Abs arXiv

   Recent interest has surged in employing Large Language Models (LLMs) for machine translation (MT) via in-context learning (ICL) (Vilar et al., 2023). Most prior studies primarily focus on optimizing translation quality, with limited attention to understanding the specific aspects of ICL that influence the said quality. To this end, we perform the first of its kind, exhaustive study of in-context learning for machine translation. We first establish that ICL is primarily example-driven and not instruction-driven. Following this, we conduct an extensive exploration of various aspects of the examples to understand their influence on downstream performance. Our analysis includes factors such as quality and quantity of demonstrations, spatial proximity, and source versus target originality. Further, we also investigate challenging scenarios involving indirectness and misalignment of examples to understand the limits of ICL. While we establish the significance of the quality of the target distribution over the source distribution of demonstrations, we further observe that perturbations sometimes act as regularizers, resulting in performance improvements. Surprisingly, ICL does not necessitate examples from the same task, and a related task with the same target distribution proves sufficient. We hope that our study acts as a guiding resource for considerations in utilizing ICL for MT.

5. arXiv

   RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanization

   Jaavid Aktar Husain, Raj Dabre, Aswanth Kumar, Jay Gala, Thanmay Jayakumar, Ratish Puduppully, and 1 more author

   arXiv preprint, 2024

   Abs arXiv

   This study addresses the challenge of extending Large Language Models (LLMs) to non-English languages, specifically those using non-Roman scripts. We propose an approach that utilizes the romanized form of text as an interface for LLMs, hypothesizing that its frequent informal use and shared tokens with English enhance cross-lingual alignment. Our approach involves the continual pretraining of an English LLM like Llama 2 on romanized text of non-English, non-Roman script languages, followed by instruction tuning on romanized data. The results indicate that romanized text not only reduces token fertility by 2x-4x but also matches if not outperforms native script representation across various NLU, NLG, and MT tasks. Moreover, the embeddings computed on romanized text exhibit closer alignment with their English translations than those from the native script. Our approach presents a promising direction for leveraging the power of English LLMs in languages traditionally underrepresented in NLP research.

2023

1. WMT

   NICT-AI4B’s Submission to the Indic MT Shared Task in WMT 2023

   Raj Dabre, Jay Gala, and Pranjal Chitale

   In Proceedings of the Eighth Conference on Machine Translation, 2023

   Abs HTML

   In this paper, we (Team NICT-AI4B) describe our MT systems that we submit to the Indic MT task in WMT 2023. Our primary system consists of 3 stages: Joint denoising and MT training using officially approved monolingual and parallel corpora, backtranslation and, MT training on original and backtranslated parallel corpora. We observe that backtranslation leads to substantial improvements in translation quality up to 4 BLEU points. We also develop 2 contrastive systems on unconstrained settings, where the first system involves fine-tuning of IndicTrans2 DA models on official parallel corpora and seed data used in AI4Bharat et al, (2023), and the second system involves a system combination of the primary and the aforementioned system. Overall, we manage to obtain high-quality translation systems for the 4 low-resource North-East Indian languages of focus.

2. TMLR

   IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages

   Jay Gala*, Pranjal A. Chitale*, Raghavan AK, Varun Gumma, Sumanth Doddapaneni, Aswanth Kumar, and 8 more authors

   Transactions on Machine Learning Research, 2023

   Abs arXiv HTML Code

   India has a rich linguistic landscape with languages from 4 major language families spoken by over a billion people. 22 of these languages are listed in the Constitution of India (referred to as scheduled languages) are the focus of this work. Given the linguistic diversity, high-quality and accessible Machine Translation (MT) systems are essential in a country like India. Prior to this work, there was (i) no parallel training data spanning all the 22 languages, (ii) no robust benchmarks covering all these languages and containing content relevant to India, and (iii) no existing translation models which support all the 22 scheduled languages of India. In this work, we aim to address this gap by focusing on the missing pieces required for enabling wide, easy, and open access to good machine translation systems for all 22 scheduled Indian languages. We identify four key areas of improvement: curating and creating larger training datasets, creating diverse and high-quality benchmarks, training multilingual models, and releasing models with open access. Our first contribution is the release of the Bharat Parallel Corpus Collection (BPCC), the largest publicly available parallel corpora for Indic languages. BPCC contains a total of 230M bitext pairs, of which a total of 126M were newly added, including 644K manually translated sentence pairs created as part of this work. Our second contribution is the release of the first n-way parallel benchmark covering all 22 Indian languages, featuring diverse domains, Indian-origin content, and source-original test sets. Next, we present IndicTrans2, the first model to support all 22 languages, surpassing existing models on multiple existing and new benchmarks created as a part of this work. Lastly, to promote accessibility and collaboration, we release our models and associated data with permissive licenses at https://github.com/ai4bharat/indictrans2.

3. EACL

   A Federated Approach for Hate Speech Detection

   Jay Gala*, Deep Gandhi*, Jash Mehta*, and Zeerak Talat

   In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 2023

   Abs arXiv Code Slides Talk

   Hate speech detection has been the subject of high research attention, due to the scale of content created on social media. In spite of the attention and the sensitive nature of the task, privacy preservation in hate speech detection has remained under-studied. The majority of research has focused on centralised machine learning infrastructures which risk leaking data. In this paper, we show that using federated machine learning can help address privacy the concerns that are inehrent to hate speech detection while obtaining up to 6.81% improvement in terms of F1-score.

2022

1. NeurIPS

   Expanding Access to ML Research through Student-led Collaboratives

   Deep Gandhi, Raghav Jain, Jay Gala, Jhagrut Lalwani, and Swapneel S Mehta

   In Workshop on Broadening Research Collaborations (NeurIPS), 2022

   Abs HTML

   We present a model of a student-led community of researchers to highlight the impact of pursuing collaborative machine learning research on the group’s members individually as well as towards achieving shared goals. We provide concrete examples of the guiding principles that led to the evolution of the collaborative from a reading group into a research group and eventually launching a non-profit software product to help non-technical stakeholders leverage artificial intelligence (AI), improving access to advanced technologies, and promoting open science. Our goal is to lay out a template to launch similar small-scale collaborative organisations at different institutes around the world.

2. Elsevier

   Combating COVID-19 using object detection techniques for next-generation autonomous systems

   Hrishikesh Shenai*, Jay Gala*, Kaustubh Kekre*, Pranjal Chitale*, and Ruhina Karani

   In Cyber-Physical Systems: AI and COVID-19 (Chapter 4), 2022

   HTML

2021

1. arXiv

   Learning from Mistakes based on Class Weighting with Application to Neural Architecture Search

   Jay Gala, and Pengtao Xie

   arXiv preprint, 2021

   Abs arXiv Slides

   Learning from mistakes is an effective learning approach widely used in human learning, where a learner pays greater focus on mistakes to circumvent them in the future to improve the overall learning outcomes. In this work, we aim to investigate how effectively we can leverage this exceptional learning ability to improve machine learning models. We propose a simple and effective multi-level optimization framework called learning from mistakes using class weighting (LFM-CW), inspired by mistake-driven learning to train better machine learning models. In this formulation, the primary objective is to train a model to perform effectively on target tasks by using a re-weighting technique. We learn the class weights by minimizing the validation loss of the model and re-train the model with the synthetic data from the image generator weighted by class-wise performance and real data. We apply our LFM-CW framework with differential architecture search methods on image classification datasets such as CIFAR and ImageNet, where the results show that our proposed strategy achieves lower error rate than the baselines.

2. Springer

   Improving Image-Based Dialog by Reducing Modality Biases

   Jay Gala, Hrishikesh Shenai, Pranjal Chitale, Kaustubh Kekre, and Pratik Kanani

   In 5th International Conference on Advances in Computing and Data Sciences, 2021

   Abs HTML Code

   Machines cannot outperform human intelligence yet; however, an image-based dialog can enable machines to perceive cues from different modalities and process information in a more human-like manner. The proposed solution is an AI-based agent that can have engaging conversations with humans by considering an image and answering questions about its visual content, taking into account both visual and textual context. Deep learning-based techniques like Recurrent Neural Networks (RNNs) with self-attention mechanisms have been employed. Responses generated by such models, in some cases, are more biased towards dialog history and are not very relevant to the actual question asked. The proposed work focuses on reducing the modality biases without compromising dialog history and improving the visual context through the use of dense captions to describe various entities in the image and generate relevant answers.

2020

1. IVCNZ

   Pothole Detection and Dimension Estimation System using Deep Learning (YOLO) and Image Processing

   Pranjal Chitale, Kaustubh Kekre, Hrishikesh Shenai, Ruhina Karani, and Jay Gala

   In 35th International Conference on Image and Vision Computing New Zealand (IVCNZ), 2020

   Abs HTML Code

   The world is advancing towards an autonomous environment at a great pace and it has become a need of an hour, especially during the current pandemic situation. The pandemic has hindered the functioning of many sectors, one of them being Road development and maintenance. Creating a safe working environment for workers is a major concern of road maintenance during such difficult times. This can be achieved to some extent with the help of an autonomous system that will aim at reducing human dependency. In this paper, one of such systems, a pothole detection and dimension estimation, is proposed. The proposed system uses a Deep Learning based algorithm YOLO (You Only Look Once) for pothole detection. Further, an image processing based triangular similarity measure is used for pothole dimension estimation. The proposed system provides reasonably accurate results of both pothole detection and dimension estimation. The proposed system also helps in reducing the time required for road maintenance. The system uses a custom made dataset consisting of images of water-logged and dry potholes of various shapes and sizes.

2. IEEE

   IoT and ML based Smart System for Efficient Garbage Monitoring: Real Time AQI monitoring and Fire Detection for dump yards and Garbage Management System

   Dev Savla, Amogh Parab, Kaustubh Kekre, Jay Gala, and Meera Narvekar

   In 3rd International Conference on Smart Systems and Inventive Technology (ICSSIT), 2020

   Abs HTML

   There is always a significant amount of challenges associated with waste and its disposal, which can be essentially mitigated by the use of technology. As the urban population increases, the amount of waste disposal is also increasing at an unprecedented rate. The inappropriate disposal of this waste will lead to many hazards including the risk of fires in the dump yards that leverages poisonous smoke in the atmosphere by adversely affecting the safety of nearby residential areas. Monitoring the occurrence of fire in huge dumping grounds manually is a tough task and thus developing an automatic fire extinguishing system is highly required. The advanced technologies can be leveraged to ensure the protection and safety of people by eliminating such hazardous risks. The air quality index (AQI) is an indicator of daily air quality report that shows how air quality affects a person’s life in a very short time. AQI plays a key role in ensuring the safety of residential areas. The proposed system aims to aid the possible hazardous risks associated with the dump yard and waste management.

3. IEEE

   Virtual Farmer: Real Time Crop Prediction and Automatic Irrigation System

   Dev Savla, Amogh Parab, Kaustubh Kekre, Jay Gala, S Ramchandra, and Pankaj Sonawane

   In 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT), 2020

   Abs HTML

   There are a plethora of problems associated with agriculture and farming which require considerable improvements but remain untouched by technology. Farming is considered as one of the strong pillars of any economy. Considering this, very few technologies exist to aid the farmers in selecting the right crops depending on the environmental factors. Moreover most irrigation systems all around the world require at least some form of human intervention. Considering this, the proposed smart farming solution aims to aid the farmers by means of technology so as to increase their yield by suggesting to them the crops that will be most profitable for them as well as automating the irrigation system for them. This in turn will be a major help to the agriculture community as a whole and also set free the farmers from certain rudimentary tasks.