Analisi Gli informatici hanno modi dettagliati in cui i sistemi linguistici di intelligenza artificiale, inclusi alcuni in produzione, possono essere indotti a prendere decisioni sbagliate da testo contenente caratteri Unicode invisibili.
Account numbers can be switched around, recipients of transactions changed, and comment moderation bypassed by special hidden characters, we’re told. And it is claimed software built by Microsoft, Google, IBM, and Facebook can be potentially fooled by carefully crafted Unicode.
The issue is that ambiguity or discrepancies can be introduced if the machine-learning software ignores certain invisible Unicode characters. What’s seen on screen or printed out, for instance, won’t match up with what the neural network saw and made a decision on. It may be possible abuse this lack of Unicode awareness for nefarious purposes.
Ad esempio, puoi ottenere l'interfaccia web di Google Translate per trasformare quella che sembra la frase inglese "Send money to account 4321" nel francese "Envoyer de l'argent sur le compte 1234".
Ingannare Google Traduttore con Unicode. clicca per ingrandire
This is done by entering on the English side “Send money to account” and then inserting the invisible Unicode glyph 0x202E, which changes the direction of the next text we type in – “1234” – to “4321.” The translation engine ignores the special Unicode character, so on the French side we see “1234,” while the browser obeys the character, so it displays “4321” on the English side.
It may be possible to exploit an AI assistant or a web app using this method to commit fraud, though we present it here in Google Translate to merely illustrate the effect of hidden Unicode characters. A more practical example would be feeding the sentence…
…into a comment moderation system, where U+8 è l'invisibile Carattere Unicode for delete the previous character. The moderation system ignores the backspace characters, sees instead a string of misspelled words, and can’t detect any toxicity – whereas browsers correctly rendering the comment show, “You are a coward and a fool.”
Thus, you’re able to trash-talk someone without setting off the moderation system using hidden Unicode characters in your message or post. This has been demonstrated, to varying degrees, against IBM’s Toxic Content Classifier and Google’s Perspective API.
Questa malizia ci ricorda gli attacchi del contraddittorio ai sistemi di visione artificiale che hanno causato una Tesla guidare più veloce rispetto al limite di velocità e una mela da essere sbagliato per un iPod.
Crucially, however, these Unicode shenanigans abuse machine-learning systems’ handling of input text rather than exploiting weaknesses within the depths of a neural network.
I nostri attacchi funzionano contro i sistemi commerciali attualmente implementati
Sono stati gli accademici dell'Università di Cambridge in Inghilterra e dell'Università di Toronto in Canada a evidenziare questi problemi, esponendo le loro scoperte in un documento rilasciato su arXiv Nel giugno di quest'anno.
"Scopriamo che con una singola impercettibile iniezione di codifica - che rappresenta un carattere invisibile, un omoglifo, il riordino o la cancellazione - un utente malintenzionato può ridurre significativamente le prestazioni dei modelli vulnerabili e con tre iniezioni la maggior parte dei modelli può essere funzionalmente interrotta", si legge nell'abstract del documento. .
“Our attacks work against currently deployed commercial systems, including those produced by Microsoft and Google, in addition to open source models published by Facebook and IBM.”
Un attacco contraddittorio omoglifo facile da eseguire in Google Translate comporta il passaggio dalla prima lettera dell'alfabeto inglese, a, al cirillico а in una parola. Sembrano uguali all'occhio umano anche se i loro caratteri Unicode sono diversi.
Utilizzando la lettera inglese a nella parola "paypal" e traducendola in russo in Google Translate si ottiene la traduzione corretta "PayPal", ma sostituendo la prima occorrenza di a con la a cirillica, Google sputerà "папа", che significa papà o padre. Potrebbe quindi essere possibile sfruttarlo in un assistente AI o in un'app Web per reindirizzare pagamenti e simili.
Screenshot di Google Translate che scambia la parola inglese paypal per papa in Russia a causa di un attacco con omoglifi
Spam emails may be able to evade detection, and hate speech may be able to slip through moderation, if miscreants use these techniques, Nicolas Papernot, co-author of the paper and an AI security researcher at the University of Toronto’s Vector Institute, told Lui Reg. Papernot referred to these text-based Unicode attacks as “bad characters.”
“The attacks presented in our paper are applicable to real-world applications; as part of our responsible disclosure, a major mail provider made changes to their spam filters and a cloud provider modified their machine-learning-as-a-service offering,” Papernot told us.
“Bad characters [are applicable] everywhere machine learning is used for natural language processing – examples of such systems are toxic content detection, topic extraction, and machine translation. Bad characters are also agnostic to machine learning tasks and pipelines – they exploit discrepancies between visual and logical representation of characters rather than inconsistencies specific to a given model as was targeted by prior work on adversarial examples.
“This makes bad characters more practical to use.”
Potrebbe anche essere possibile utilizzare Unicode invisibile nel bene e nel male, ha aggiunto.
“When machine learning is used for questionable purposes, such as censorship, bad characters could be leveraged by human rights activists to evade censorship,” Papernot told us.
“In another example, law firms that rely on natural language processing to process large corpus of documents efficiently are also exposed: a malicious entity could submit documents with bad characters to evade scrutiny from the law firm.”
Developers of AI-powered software should either filter out special Unicode characters – such as backspaces – entirely, if feasible, or pass the Unicode through a parser before it’s given to a neural network, so that ultimately what the neural net sees and makes a decision on is what the user also sees and interacts with in the browser or user interface. Changes in language, such as from English to Russian, should be detected and handled appropriately.
Dato che i modelli potenzialmente suscettibili a questi attacchi potrebbero essere già ampiamente utilizzati in produzione, potremmo assistere a uno sfruttamento riuscito nel mondo reale. ®
Fonte: https://go.theregister.com/feed/www.theregister.com/2021/08/06/unicode_ai_bug/
