Utilising Large Language Models for Adversarial Attacks in Text-to-SQL: A Perpetrator and Victim Approach

Abstract

This paper investigates the use of Large Language Models (LLMs) for the Text-to-SQL task, both as Perpetrator models for generating adversarial attacks and as Victim models for assessing their robustness. In this study, two state-of-the-art LLMs, Llama3 with 70 billion and Mixtral with 47 billion parameters, were employed as Perpetrators to generate adversarial examples at the character-, word-, and sentence-level. A total of 77,292 adversarial examples were generated from 2,147 data points of the Spider test-set using three additional LLMs as Victims and evaluated thoroughly. These Victim models are based on Llama3 with 8 billion parameters and differ only in the extent of fine-tuning for related benchmark tasks. The results show that attacks at the word-level, particularly through synonym replacements, most significantly impair model performance. Additionally, providing database schemas significantly improves execution accuracy, while fine-tuning does not always enhance robustness against adversarial attacks. This work provides important insights into improving the reliability of Text-to-SQL models in future applications and makes a significant contribution to the further development of these models in research.