How AI Coding Assistants Could Be Compromised Via Rules File (scworld.com) 7
Slashdot reader spatwei shared this report from the cybersecurity site SC World:
: AI coding assistants such as GitHub Copilot and Cursor could be manipulated to generate code containing backdoors, vulnerabilities and other security issues via distribution of malicious rule configuration files, Pillar Security researchers reported Tuesday.
Rules files are used by AI coding agents to guide their behavior when generating or editing code. For example, a rules file may include instructions for the assistant to follow certain coding best practices, utilize specific formatting, or output responses in a specific language.
The attack technique developed by Pillar Researchers, which they call 'Rules File Backdoor,' weaponizes rules files by injecting them with instructions that are invisible to a human user but readable by the AI agent.
Hidden Unicode characters like bidirectional text markers and zero-width joiners can be used to obfuscate malicious instructions in the user interface and in GitHub pull requests, the researchers noted.
Rules configurations are often shared among developer communities and distributed through open-source repositories or included in project templates; therefore, an attacker could distribute a malicious rules file by sharing it on a forum, publishing it on an open-source platform like GitHub or injecting it via a pull request to a popular repository.
Once the poisoned rules file is imported to GitHub Copilot or Cursor, the AI agent will read and follow the attacker's instructions while assisting the victim's future coding projects.
Rules files are used by AI coding agents to guide their behavior when generating or editing code. For example, a rules file may include instructions for the assistant to follow certain coding best practices, utilize specific formatting, or output responses in a specific language.
The attack technique developed by Pillar Researchers, which they call 'Rules File Backdoor,' weaponizes rules files by injecting them with instructions that are invisible to a human user but readable by the AI agent.
Hidden Unicode characters like bidirectional text markers and zero-width joiners can be used to obfuscate malicious instructions in the user interface and in GitHub pull requests, the researchers noted.
Rules configurations are often shared among developer communities and distributed through open-source repositories or included in project templates; therefore, an attacker could distribute a malicious rules file by sharing it on a forum, publishing it on an open-source platform like GitHub or injecting it via a pull request to a popular repository.
Once the poisoned rules file is imported to GitHub Copilot or Cursor, the AI agent will read and follow the attacker's instructions while assisting the victim's future coding projects.
what best practices? (Score:2)
"For example, a rules file may include instructions for the assistant to follow certain coding best practices..."
Isn't the first "coding best practice" writing your own code? And then knowing everything that's in your code?
AI best coding practices are, and always will be, an oxymoron.
So you outsource your coding... (Score:1)
...to a third party black box beyond your understanding or control, and you wonder why it's full of surprises.
Great business model!
Unicode (Score:3, Insightful)
Hidden Unicode characters like bidirectional text markers and zero-width joiners can be used to obfuscate malicious instructions in the user interface and in GitHub pull requests, the researchers noted.
Unicode's mentality is a product of the 90s internet: why think about security adverserially when we're all just friends building utopia?
And what do we have? We have both injection attacks like this and every unicode compatible device wasting valuable bytes carrying data to render gender-ambiguous multiracial poo emojis.
Couldn've all been avoided if everyone on the internet just spoke English...like God intended.
that is to be expected (Score:2)
So many people just blindly follow the machine (Score:1)
I've read many articles about people who mindlessly follow their navigation app and end up in a river or driving off of a cliff or something. "I know the road turned into a goat track but Google said that this is the way to the city!"
I just read a question on another website where a user blindly followed instructions from some AI to update his video driver and now can't log into his machine any more.
And now we're having people writing programs that they don't understand, uncritically following what some AI
But (Score:2)
But the sales guy convinced our executives it was all secure and stuff