Artificial intelligence (AI) spam

[CarolC1]

There is a topic on this in 3.2 Support already.
viewtopic.php?p=16024309#p16024309

This morning we have a French language post in a 25-post English language topic. The content of the post is a rerun of advice given in previous posts in the topic, concurring with the advice, nothing new.

The username is an English language choice that fits the theme of the forum. I can't evaluate the quality of grammar and usage. The email is not on Stop Forum Spam. The IP is APNIC, Australia.

So the main indicators of possible spam are,
1) they must speak English because they were able to read previous posts and use the ideas, or at least refer to the ideas, to write their post in French, so why didn't they respond in English? The board has a French section with very little posting. It is odd behavior to respond to an all-English topic in French.
2) the content being a restatement of what was already said
3) APNIC IP
4) gmail email address is a favorite of spammers

My current thought is to put them on approval, so if they try to go back and edit in a link we'll see it.

What is the chance this is AI?

I know AI is that good, because just for fun, a member of the forum used an online program and gave it the url of the board. The AI program crawled the index page and apparently some post titles, and created a 17-min "podcast" talking about the board. The script generated by AI was 99% believable. It was delivered by 2 AI voices, not human, that sounded 100% real. He said it took 2 minutes to generate.

So if AI can make a podcast in 2 minutes, surely it can crawl a topic and generate a reply. We've had spammers either copy/paste parts of a topic in a new reply, or reword parts of a topic into a reply. I have a feeling this is in the same vein.

Honestly, it's good enough, there aren't many "tells" to give it away. Is anyone else seeing this? This is harder to catch than previous spam, if that's what it is.

[KevC]

My current thought is to put them on approval, so if they try to go back and edit in a link we'll see it.

Yep I'd do the same. From experience on here and other sites, they will come back and edit it, usually a few days later when it's dropped off the first page but can be weeks later.

Honestly, it's good enough, there aren't many "tells" to give it away. Is anyone else seeing this? This is harder to catch than previous spam, if that's what it is.

I've seen it a lot. One forum I read gets them all the time. I can usually spot them a mile off. They often feel the need to unnecessarily explain a common term that everyone in the topic already knows. The language also usually reads as a bit 'clinical' if you know what I mean. Maybe doesn't have the natural flow of a real person. There are no unnecessary words. Also they often seem to dig up old topics that were long since done with rather than joining in with the current crop of active topics.

I've also seen that they will often shoehorn in a key word that they're likely to edit like route or cargo or sell and that is the word where the link edit will go in a few days time.

Finally it's worth clicking edit to check the text to see if there is already a spam link in there in the form of a 1px font link which you wouldn't normally see or they've edited a . or a ' in to a link.

How can you stop them? That's a lot harder now. But it's where the mod queue comes in to its own because if they go back to edit the post they'll end up back in the mod queue if it's still under the threshold, thus flagging up that it's been changed.

[CarolC1]

Thanks for the tips. They are on approval. The part about certain words like route or cargo or sell is new to me. I'll start being aware of that.

I view the board on a PC and FF underlines even a single punctuation mark that was turned into a link, and you can see it. I clicked the post open anyway to check for a matching bg color link, which we've had a couple of times.

It's very strange. The sound of the post is similar to some other posts we've had in English that kind of reword what was already said and don't add anything. I wonder if those were generated by the same program and they simply messed up using it this time. Quite a few of the spam or suspected spam posts we get have a certain all-knowing and supportive tone that is pretty consistent from post to post, and this one had that same tone.

It's almost as if someone got or was given an advanced software program capable of doing this, and it doesn't auto-detect the target language, or they set it to French because they couldn't tell it was English, or they forgot to set it at all. I don't know what happened.

We don't have a rule requiring members to reply in the language of the OP. If it happens again we might have to discuss that.

Thank you!

[KevC]

It's very strange. The sound of the post is similar to some other posts we've had in English that kind of reword what was already said and don't add anything.

Yes I've seen that too. Either a rewording of the opening post (almost looks like it's been washed through a translator and back again), or simply used AI to reword it and the other thing I've seen is posts that say a lot and actually say nothing. They agree a lot with what someone else has said, usually saying things like 'yes I've had that too and went through the same experience with a shop near me' (ok great but that says nothing really). They also rarely further the conversation because there isn't anything the AI can 'think of' to add to it and push it along like a real person would.

As to why yours is French, could just be a glitch by the spambot. We sometimes get posts on here in other languages when clearly every other post is in English.

[Brf]

the other thing I've seen is posts that say a lot and actually say nothing.

I have seen some on a game forum I frequent where someone will ask if you can do a particular thing. The AI spammer bots will reply with a big post of gobbledygook that boils down to "if the rules allow it, then you can do it, otherwise you cannot". Totally meaningless.

[Helter Skelter]

As to why yours is French, could just be a glitch by the spambot. We sometimes get posts on here in other languages when clearly every other post is in English.

If it is not a bot, its likely just a browser translator plugin. I made a few English posts on a Ukrainian site before I realized that the rest of the posts were automatically translated and were not actually in English

[Holger]

Delete the post and dectivate the user. If the user is legit, he/she/it will contact you.

[battye]

I've noticed the AI bots are becoming more sophisticated and it's a worry.

Like on my forum (called CricketMX), a bot signed up and made a post and ended the message with something along the lines of "What do the long-time CMX-ers think?".

That's quite clever, it's obviously crawled other topics and found people using the shorthand (CricketMX -> CMX) and then adapted it to sound more natural.

In this case it was easily identifiable as a bot because it included links, but I'm a bit concerned about when they become patient enough to become "naturally ingrained" in the community by using those natural-sounding shorthands, nicknames for forum members, etc, and then when they are trusted maybe people will click on those links thinking that they've been interacting with a friend.

Generative AI is interesting and has legitimate uses, but it's going to be a real problem for us to combat in the context of forums and whether users are real or not.

[[Dimetrodon]]

What concerns me is that we may enter an era where all CAPTCHAs become useless, with AI bots becoming so sophisticated that it is impossible to present something a human can figure out while it cannot.

[Holger]

Shut it down.
AI brings some minor advantages, but A LOT more disadvantages to humanity.

[thecoalman]

What concerns me is that we may enter an era where all CAPTCHAs become useless, with AI bots becoming so sophisticated that it is impossible to present something a human can figure out while it cannot.

You can make a request for a DNA blood sampling extension. I'd agree the AI is going to be difficult to manage especially when they start using poor grammar, wrong capitalization etc. However I have faith, for example if Google can get it's act together and stop email account registrations. Perhaps some kind of verified email account.

[Mick]

Is it really Ai though? I have my doubts and worries about it. As I’ve asked before, what happens if and when the Ai starts rewriting its own code, the idea makes me shiver?

Thank heavens the major nuclear arsenals are 50s and 60s technology.