Flood Protection

Discussion and questions about hub software
Locked
Toast

Flood Protection

Post by Toast » 09 Aug 2008, 16:59

I wanna start a discussion on floodprotection and get new ideas on how to implement a more secure environment on (A)DC in general. I really hope all devs thinks "outside of the box" and apply their skills and ideas to help out in this matter.

This idea originated from Big_Muscle on DCDev when he said can we improve this, and i for one think we can.

Client, Hubsoft, Hublist, Bot Developers all are welcome in this discussion to input their thoughts.

Todays flood protection is mostly interval checks there must be a better way to detect floods, connection or chat floods.

Note: A typo was in this topic:"Big_Muscle on DCDev when he said cant we improve this" to "Big_Muscle on DCDev when he said can we improve this" (Shit happens sometimes,sry ;) )

Toast

Re: Flood Protection

Post by Toast » 10 Aug 2008, 10:52

Well since noone has start to comment this thread ill start then (as usual).

First off lets analyze a simple flood program.

The general term for flood bots is:

  • reconnect
  • say
  • reconnect
The way to defeat this bot is:

  • Reconnect interval check
  • Same message detection
  • Max Connections from Same IP
Since this is the most common bot out there and the most easiest bot to make it would be wise to make really good detections for it.

Well next up is socks5 flooding this one is more tricky then a simple flooding app.

The principle is still the same:

  • reconnect
  • say
  • reconnect
The only thing that differs is that every reconnect the bot gets a new ip from its socks5 list

The way to defeat this bot is:

  • Reconnect interval check
  • Same message detection
  • Proxy Block lists

en_dator
Member
Posts: 72
Joined: 01 Apr 2008, 19:24

Re: Flood Protection

Post by en_dator » 15 Aug 2008, 09:28

a way to improve flood protection and avoiding issues with same message detection, ip spoofing, socks etc is to use some simple statistics methods.

For example:
In the hubsoft we keep statistics on number of connections/sec, number of incomming messages/sec etc.
This way we know what is "normal" for our hub (as this can change rather much over time the statistics should only be short term).

As long as all stat values are "normal", we can let down our guard and don't restrict reconnections and such things.

As soon as stat values start to differ from normal we go to defcon 2 and impose restrictions ;) .
(I think that similar ideas already exists in some hubsofts btw)

What I would like to see is a hubsoft that can automatically modify its "flood-settings" so that during normal operation it allows a large number of connections and dont restrict users, but as soon as it detects something it gradually restrics, and as a final stage (if it really is properly flooded) it stops accepting new connections as long as the flood continues and when it stops it gradually returns to the normal settings again.

In combination to this I would like to see three more functions.
1) it can tell the other hubs in my net that it has gone to defcon 2,3 or 4.
2) it can receive information from other hubs in my net about their defcon level and act on that information.
3) provide a scripting interface option that would for example give the possibility to write a script that makes the hub interact with the firewall to block/unblock directly in the firewall and releiving the hubsoft from taking all the heat.

so guys, what u think of that idea?

blastbeat
Member
Posts: 53
Joined: 10 Jan 2008, 19:56
Contact:

Re: Flood Protection

Post by blastbeat » 09 Sep 2008, 17:54

your idea sounds simple but to do the math is not.
first of all you will need some empirical data you dont have when you start a hub. at the beginning there are only some constants like max users/connections, max adc commands per time interval, max incoming data length, max new users/connections per time interval,
max MSG per time interval etc you could define. for example what is a good value for max adc commands per time interval? 5 per second? or is it proportional to actual user count? to determine the factor you need some empirical data you dont have. in reality you will make rough estimates for these values which are much to high, because you have to consider sporadic peeks.
even if you have some empirical data, it is not necessary good enough. often you dont have an adequate amount of random samples and your stats are distorted.
0) worst case
when a value defined above is exceeded, the hub can batten down the hatches. e.g. disconnect/ban users which send too large or too much messages.
1) connect - disconnect problem
you can measure the connects/disconnects per time interval and the avarage online time of an user. but that wont tell you if an connecting user is a flooder. maybe it would help to count high rates of short online times in short time intervals to identify possible connect - reconnect bots.
the old problem - what means short online time? how much are high rates?
but its independent of the identity (IP, CID) of an user, so you dont have to bother with the mentioned issues.
a strategie for a bot would be to staying just long enough in hub to prevent a defcon 2 or even better to connect with 100 bots which altenernate with reconnecting and change their id everytime.
a same message detection wont work when the bots flood with random bytes, and proxy block lists in case of socks5 are also a no-go.
2) flooding mainchat/pm
assume the 100 bots reconnecting and flooding. you can meausure the same thing here: high rates of mainchat message in short time. same problem as above.
3) defcon x
when some limit is reached, you can switch to defcon x. as usual you have some empirical problem.
when you limit is to weak, it takes too long to reach defcon x, or its to strong and some peeks caused by normal users triggers defcon x. a flooding can happen before you have enough data to determine a good limit.
maybe its a good idea to use a nonliniear relation:
defcon x+1 = ln( defcon x ) and the last defcon is 0)
4) warning other hubs
useful feature?
could be done with a kind of hublink adc extension or simply sending a hubcommand like!floodwarning defcon x to trigger something
5) hub - firewall
cannot imagine how to link a hub to every kind of firewall via a hub scripting interface. at least with my firewall it will not work
7) other methods
a simple method to avoid bots which mutilate the hub is to allow only regged users to chat, search, download. of course a "+regme" command would be only useful with i kind of CAPTCHA to check the "human nature" of the user.

en_dator
Member
Posts: 72
Joined: 01 Apr 2008, 19:24

Re: Flood Protection

Post by en_dator » 09 Sep 2008, 20:14

i totally forgot about this one, and I even wrote a reply to it, man I AM getting old an confused :shock:

thanks for the reply blastbeat,

And to tell you the truth I do not know how much work it would take to create the things I wrote about, but when it comes to hub hosting and protecting a hub from floods and other attacks I think that there are a few things that the hub should deliver.

Today in most hubsofts we have two options, either we restrict, and this means that we make our hub more robust against attacks but less responsive to users, or we dont restrict and let the hub & computer handle it as best it can during an attack.

What I would like to see is a hub that is normally not very restricted at all, and that automatically adjust its restrictions if/when it detects something that looks like an attack.

I think you can agree that this would be a good thing?

As you and Toast both pointed out, some of todays methods is not very effective to protect the hub. My idea was that instead of looking at the message, or the ip, I think its better to keep stats on some of the relevant data and use that to modify the hubsoft settings (how and which parameters to monitor is ofcorse another matter, I am by no means an expert on hub/client communication protocol but I do know my way around statistics).

I realize it takes some time before the stats are useful, during hub start they can of course not be trusted, and if I have a large hub and I restart the hubsoft it can take many minutes before all users have reconnected again. But I dont see that as a reason why my idea should not work. I do not mean it as a hubs only defence against attachs, but as an automated way of letting the hubsoft adjust the level of protection on a sliding scale. What would be good start values I have no idea (there probably should not be any), but I believe that when the hub has been running for 5 - 10 minutes the stat values should have "settled" and then they could be trusted enough to be used.

I understand what you say but I don't actually see it as something that cant be handled quite effectively in the hub soft. My major concern really is whether it will create a too large overhead in the soft so it is making it into a huge resource hog instead of a nice and smooth hub :)

Im actually not interested in random, I'm just interested in what is normal for that specific hub, and sporadic peaks dont have to cause a problem if there are a few safe guards in place (so that for example the first to speak in a hub thats been quiet for a day wont get kicked :P )

in reply to your points:

0) yes this is a real possibility.

1) It is not possible to know if a connecting user is an attacher or not, but that is hardly the point, what the hubsoft should do is protect the ones already in the hub by slowing down the connection of new users if the rate of new connections is getting too high (for the computer/connection) to handle. Or in case of message floods it should simply drop those and not send them out to all users (only an example of possible methods).

2) This can be handled by different means, and I don't know if anyone is more effective than the other (one way would be to gradually impose a longer time before a user actually can send messages after login).

3) Yes I agree there is a problem here, but the hub server should be able to handle some abuse so if the detection is a little slow is not a major issue, after all there must be an attack before it can be detected :).
It also depends on what kind of settings there are to adjust, lets say that for main chat messages we have the following possibilities:

* global time between each mc message normal this is not limited, when attacked it would be possible to start dropping and only keep 1 per sec - 1 per 2sec - ... etc
* time between each mc message per user same as above but more agressive limiting perhaps
* lenght of message normal is some owner setting, if attacked cut it down to 128 or someting even smaller
* message content normal could be unfiltered, when attached drop everything that looks like it has ip or addys or characters outside the a-z 0-9 range (to make it language protected the hubowner should probably add a list of allowed characters outside that range, like åäö for swedish).

The first three items there could all be adjusted on a sliding scale, all or separate while the last is more of an on/off thing (and I do realize parsing the data is also something that takes resources, but I believe that is already done in the hubsoft so there should not be that much extra processing involved).

4) Yes I think that this is a useful feature, to be able to warn other hubs in a net that something might be up. And yes the simpler the better :)

5) I did not mean that the hubsoftware should provide interface to lots of different firewalls, I mereley meant that if the scripting part of the hubsoft includes functions that the owner (or another scripter) could use to write those script it would be a good thing.

7) hmm, CAPTCHA-type pattern is an idea, I'll try that and see if users actually understand what to do :). I have seen hubs where the simple +regme causes issues :D


When it comes to methods of preventing or blocking attacks I am quite sure there are plenty that has already been tried and tested, some of them are probably very good against certain attacks and I see no reason for me to (try to) invent the wheel once more. What I instead would want to have is a hub that I feel I can leave alone in the knowledge it will take care of it self if some dude decides he doesnt like my hub :)

looks like I like to repeat myself a lot, but as I still think this is a good idea (until proven otherwise) I will not be offended if you all point out the weaknesses of it all.

Toast

Re: Flood Protection

Post by Toast » 20 Nov 2008, 17:37

i started this thread in an effort to get an open discussion about new techniques that can be implemented into software go get views from both users and developers no matter how absurd so far ive been pleased with the reading that ive taken part of the captha idea is a real smart idea but its not the only solution the imperial idea aint that dumb either in retrospect my idea about same msg detection seems kinda lame going on known ideas instead of thinking in new terms.

i will take some time to think about a feature just wanted to k that read the material so people dont think this thread is dead or forgotten this is still an active discussion of the utmost importance.

There has never been an open discussion of security feats on Direct Connect most this have been decided in the dark.

im gonna focus my attention of other software see how they made solutions in order to get inspired before writing anything new in the topic hope other do that aswell

Locked