|
Attention: Year 2009 is here
Wishing a very Happy New Year to all members of FFI. Our new and improved site is ready. To visit main site, click at faithfreedom.org and to visit our new forum, click at forum09.faithfreedom.org and register again. Do not worry about your old forum posts and PM, everything is saved here till 31st December, 2008 for future references.
|
| View previous topic :: View next topic |
| Author |
Message |
ixolite
Joined: 27 Jan 2006
Posts: 12939
Location: land of pork and beer
|
 Posted: Tue Dec 23, 2008 6:48 am Post subject: |
 |
|
| Quote: |
| I've seen about a total of 20 people on the site, between shown and hidden guests, and the forum slows to a crawl as if there are hundreds of simultaneous view post requests. Obviously, there's more to it than signed in traffic and guest traffic. Some other, more sinister traffic has to be going on. |
FFI is a little bigger than just this forum.
_________________
</islam>
"Never argue with idiots. They bring you down to their level and then beat you with experience." - Goldthwait H. Dorr |
|
| Back to top |
|
 |
DSingh
Joined: 05 May 2008
Posts: 326
Location: Canada
|
| Posted: Tue Dec 23, 2008 6:58 am Post subject: |
|
|
This was the forum count on 19 of December. Quiet a lot of people there, one would expect slow loading times, I have seen it on other big forums before.
_________________
| THHuxley_redux wrote: |
| Watching debates between Christians and Muslims is like watching a boxing match between quadrilateral amputees. |
""Everything I ever needed to learn about Islam, I learned on 911."" |
|
| Back to top |
|
|
Chewchy
Joined: 02 Feb 2008
Posts: 1774
|
| Posted: Tue Dec 23, 2008 7:04 am Post subject: |
|
|
One has to be carefull when choosing a host and especially one that costs much less for than others for the server FFI needs.
I had a really bad experience when I used such a host and they basically took over my site control and wouldn't fulfill their promise when the server went down repeatedly nor would they address my concerns about what was going on.
In order to get a server with all the safeguards needed for a site like FFI and the bandwidth, etc. it's going to cost a bit more and even then one is at the mercy of the host in many ways. |
|
| Back to top |
|
|
doubting_thomas
Joined: 13 Jun 2005
Posts: 7379
Location: Western Hemisphere
|
| Posted: Tue Dec 23, 2008 6:34 pm Post subject: |
|
|
We considered them, but I don't recall offhand why we eventually chose not to use them. I think it may have been because we got a better price with another datacenter.
| Islamis_Allah_Tashit wrote: |
Sure, because they're expecting you to become a sub hosting site for other sites, not just having one tiny site with a forum. As far as traffic, it wouldn't surprise me if at least 50% of it was due to constant DOS attempts. Some places do nothing but security full time and if it were effecting their customers, I wouldn't be surprised if they find a way to minimize the effect.
|
Well, if 50% of our traffic is coming from DoS attacks then they're doing a terrific job of masking it by using hundreds of IP's to make legitimate network requests from the server. Even a distributed attack will generally follow a detectable pattern. What I see most of the time just looks like normal network traffic.
I think you may be thinking that this forum is the only site running on this server. We also have a news site, an Italian site and forum, a German forum, and a Dutch forum. Until recently, we also had an Indonesian forum which got 3 times as much traffic as this forum. The admins of the Indo forum are working on bringing it back with new software.
| Islamis_Allah_Tashit wrote: |
If it were possible to have better security, you wouldn't even need one quarter of a server.
|
Again, you're assuming that the reason for the high server load is DoS attacks. There are times when this is true, but usually the high server load is due to normal internet traffic.
| Islamis_Allah_Tashit wrote: |
Because of DOS attacks, right? I'm guessing that this is due to your auto email post notification failing
|
No, not because of DoS attacks. The high load imposed by the database is because of the size of the database. It has nothing to do with auto email notifications. Every time you access a page on this forum it generates several queries to the database manager. How long would it take your PC to search a 2GB database? Probably a few seconds, right? Imagine if it were receiving a dozen search requests every second? Would it be able to keep up with those requests?
The database manager tries to keep as much of the database cached in memory as possible in order to avoid having to hit the hard drive with every query. Our current server is maxed out on memory, and our database has outgrown it.
| Islamis_Allah_Tashit wrote: |
There's plenty of much larger forums. Are you sure this isn't some sort of ongoing DOS?
|
Sure, and most large forums do not run on shared hosting servers. The really large forums use several servers, along with a firewall and load balancer.
Unless the DoS attack is coming from hundreds of IP's, and each IP is behaving like a regular web user, then I'm certain it's not an ongoing DoS attack.
| Islamis_Allah_Tashit wrote: |
Or a cisco gateway router?
|
A router will not automatically defend against DoS attacks, but Cisco does make some good firewalls.
| Islamis_Allah_Tashit wrote: |
I've seen about a total of 20 people on the site, between shown and hidden guests, and the forum slows to a crawl as if there are hundreds of simultaneous view post requests. Obviously, there's more to it than signed in traffic and guest traffic. Some other, more sinister traffic has to be going on. |
You're looking only at one forum. The typical number of guests on the Indo forum was over 100. Combined with the other sites and forums, there were typically around 250 simultaneous TCP/IP connections to the server. If a significant number of those guests are viewing pages that are generated by php scripts then that represents a pretty significant CPU load. If those php scripts are generating database queries, then that represents a pretty significant hard drive load. That's a lot of work for one server to manage. |
|
| Back to top |
|
|
Islamis_Allah_Tashit
Joined: 12 May 2004
Posts: 4384
Location: A Holiday Inn bathroom
|
| Posted: Tue Dec 23, 2008 11:37 pm Post subject: |
|
|
| doubting_thomas wrote: |
We considered them, but I don't recall offhand why we eventually chose not to use them. I think it may have been because we got a better price with another datacenter.
| Islamis_Allah_Tashit wrote: |
Sure, because they're expecting you to become a sub hosting site for other sites, not just having one tiny site with a forum. As far as traffic, it wouldn't surprise me if at least 50% of it was due to constant DOS attempts. Some places do nothing but security full time and if it were effecting their customers, I wouldn't be surprised if they find a way to minimize the effect.
|
Well, if 50% of our traffic is coming from DoS attacks then they're doing a terrific job of masking it by using hundreds of IP's to make legitimate network requests from the server. Even a distributed attack will generally follow a detectable pattern. What I see most of the time just looks like normal network traffic. |
Isn't the same ability for users here to mask their actual ip address the same thing that is also used for DOS attacks? Some people here have told me about free services they use to bounce their request around to different participating machines, thereby allowing their ip address to appear different every time. But what I explained to this person is that he himself will also be used to fake the address of someone else. It's kind of a limewire situation, where you take and you give also, but the sinister side, is that if you are using this free service, chances are, that in return, they're using your computer to spoof, send spam and email, and/or participate in a DOS attack, and I think that these would probably look like legitimate requests because the legitimate addresses might have opened themselves up the right way to this "free" service (enabled relaying) because it would be a required part of using the spoofing service that they are using to protect their identity, and while thinking that they are only spoofing the specific site that they're on, the free spoofing service is using them as a legitimate seeming address. I'm not even really talking about users on this site, I'm saying that there are tons of people using these free spoofing services while the hidden underground is that they are being used to spoof as well. I'm not an expert security admin, but what I'm saying seems to make sense to me. What do you think?
| Islamis_Allah_Tashit wrote: |
I think you may be thinking that this forum is the only site running on this server. We also have a news site, an Italian site and forum, a German forum, and a Dutch forum. Until recently, we also had an Indonesian forum which got 3 times as much traffic as this forum. The admins of the Indo forum are working on bringing it back with new software. |
Please correct me if I'm wrong, but I thought that any decent sql server can handle more than 1000 seemingly simultaneous requests. Am I way off on this?
P.S., love your name.
_________________
Somebody get me a hairdryer |
|
| Back to top |
|
|
Islamis_Allah_Tashit
Joined: 12 May 2004
Posts: 4384
Location: A Holiday Inn bathroom
|
| Posted: Tue Dec 23, 2008 11:48 pm Post subject: |
|
|
| DSingh wrote: |
This was the forum count on 19 of December. Quiet a lot of people there, one would expect slow loading times, I have seen it on other big forums before. |
319 near simultaneous requests is a lot for a decent sql server? Then again, if there's 300 some people online, you may get about 50 near simultaneous requests. I'm guessing that between all of the forums that are here, you might get 1000 near simultaneous requests, and that might be generous.
_________________
Somebody get me a hairdryer |
|
| Back to top |
|
|
doubting_thomas
Joined: 13 Jun 2005
Posts: 7379
Location: Western Hemisphere
|
| Posted: Wed Dec 24, 2008 3:59 pm Post subject: |
|
|
| Islamis_Allah_Tashit wrote: |
Isn't the same ability for users here to mask their actual ip address the same thing that is also used for DOS attacks? Some people here have told me about free services they use to bounce their request around to different participating machines, thereby allowing their ip address to appear different every time. But what I explained to this person is that he himself will also be used to fake the address of someone else. It's kind of a limewire situation, where you take and you give also, but the sinister side, is that if you are using this free service, chances are, that in return, they're using your computer to spoof, send spam and email, and/or participate in a DOS attack, and I think that these would probably look like legitimate requests because the legitimate addresses might have opened themselves up the right way to this "free" service (enabled relaying) because it would be a required part of using the spoofing service that they are using to protect their identity, and while thinking that they are only spoofing the specific site that they're on, the free spoofing service is using them as a legitimate seeming address. I'm not even really talking about users on this site, I'm saying that there are tons of people using these free spoofing services while the hidden underground is that they are being used to spoof as well. I'm not an expert security admin, but what I'm saying seems to make sense to me. What do you think?
|
You're talking about proxy servers, and it is indeed possible to use proxy servers in a DoS attack. It's even possible for one user to use multiple proxy servers to attempt a distributed attack. Most proxy servers are pretty good at detecting when they're being used this way. They simply detect a lot of requests being sent to the same IP address in a short space of time. They usually either disconnect the user, or the they throttle his access to limit the number of requests.
I've seen attacks of this sort before. What is usually common about them is that they target specific aspects of a website which they have determined to be especially vulnerable to attack. The search system within the phpBB forum software can be abused in this way. This is because it doesn't just send queries for specific rows and columns, but forces the database manager to search through all rows and columns in specific tables looking for matching records. This is a lot of work for the database manager to do, especially if it can't cache all of those tables in RAM, and has to actually scour the database files on the hard disk. A few such requests are all that is needed to bring a server to it's knees. That was the reason for disabling the search feature on this forum. One simple search was taking as much as 3 minutes to complete.
Anyway, my point is that a distributed attack usually has a detectable pattern. For example, attacking a weak point on the server. They don't usually involve multiple IP's pulling up various random pages on multiple sites on the same server. This looks like normal web traffic. If it were a DoS attack then there would be no way to defend against it because each IP is just asking the server to do what it was designed to do. On the other hand, if a particular IP is issuing multiple requests in a short space of time then that's easy to detect because each request will open a new socket connection to the server. A long list of connections from the same IP is pretty solid evidence that a DoS attack is underway. The only exception is if those requests are coming from an IP owned by a search engine.
| Islamis_Allah_Tashit wrote: |
Please correct me if I'm wrong, but I thought that any decent sql server can handle more than 1000 seemingly simultaneous requests. Am I way off on this?
P.S., love your name. |
Sure. The SQL server can handle that many socket connections, and each request can be in various stages of completion at the same time. The problem arises when many of those requests require searching for data on the hard disk. Data coming from the hard disk must be serialized. The hard disk can't read multiple locations on the disk at the same time, and searching the hard disk is dramatically slower than searching for data in memory.
We could see a significant improvement in the performance of this server by either drastically reducing the size of the forum databases so that they could all be cached in memory, or increasing the amount of RAM on the server. The latter is not an option with this particular server. It's several years old, and has the maximum amount of RAM it can accomodate. Our new server has four times as much RAM, and will be running a 64-bit O/S with a multicore processor. |
|
| Back to top |
|
|
Islamis_Allah_Tashit
Joined: 12 May 2004
Posts: 4384
Location: A Holiday Inn bathroom
|
| Posted: Fri Dec 26, 2008 7:28 pm Post subject: |
|
|
| doubting_thomas wrote: |
| Islamis_Allah_Tashit wrote: |
Isn't the same ability for users here to mask their actual ip address the same thing that is also used for DOS attacks? Some people here have told me about free services they use to bounce their request around to different participating machines, thereby allowing their ip address to appear different every time. But what I explained to this person is that he himself will also be used to fake the address of someone else. It's kind of a limewire situation, where you take and you give also, but the sinister side, is that if you are using this free service, chances are, that in return, they're using your computer to spoof, send spam and email, and/or participate in a DOS attack, and I think that these would probably look like legitimate requests because the legitimate addresses might have opened themselves up the right way to this "free" service (enabled relaying) because it would be a required part of using the spoofing service that they are using to protect their identity, and while thinking that they are only spoofing the specific site that they're on, the free spoofing service is using them as a legitimate seeming address. I'm not even really talking about users on this site, I'm saying that there are tons of people using these free spoofing services while the hidden underground is that they are being used to spoof as well. I'm not an expert security admin, but what I'm saying seems to make sense to me. What do you think?
|
You're talking about proxy servers, and it is indeed possible to use proxy servers in a DoS attack. |
Actually, I wasn't talking about proxy servers, I was talking about client machines being used as a zombie to make requests. By using a spoofing service, as many people do, they open their machine up to help others spoof. The machine can also be a zombie just waiting to be activated to make requests. This way, there are many zombie client machines making requests and therefore it looks like normal traffic.
| doubting_thomas wrote: |
It's even possible for one user to use multiple proxy servers to attempt a distributed attack. Most proxy servers are pretty good at detecting when they're being used this way. They simply detect a lot of requests being sent to the same IP address in a short space of time. They usually either disconnect the user, or the they throttle his access to limit the number of requests.
I've seen attacks of this sort before. What is usually common about them is that they target specific aspects of a website which they have determined to be especially vulnerable to attack. The search system within the phpBB forum software can be abused in this way. This is because it doesn't just send queries for specific rows and columns, but forces the database manager to search through all rows and columns in specific tables looking for matching records. This is a lot of work for the database manager to do, especially if it can't cache all of those tables in RAM, and has to actually scour the database files on the hard disk. A few such requests are all that is needed to bring a server to it's knees. That was the reason for disabling the search feature on this forum. One simple search was taking as much as 3 minutes to complete. |
This has been disabled for at least a few years, and there are still major performance problems.
| doubting_thomas wrote: |
Anyway, my point is that a distributed attack usually has a detectable pattern. For example, attacking a weak point on the server. They don't usually involve multiple IP's pulling up various random pages on multiple sites on the same server. This looks like normal web traffic. If it were a DoS attack then there would be no way to defend against it because each IP is just asking the server to do what it was designed to do. |
And that is what I believe could be happening.
| doubting_thomas wrote: |
On the other hand, if a particular IP is issuing multiple requests in a short space of time then that's easy to detect because each request will open a new socket connection to the server. A long list of connections from the same IP is pretty solid evidence that a DoS attack is underway. The only exception is if those requests are coming from an IP owned by a search engine. |
Actually, there must be a way to where the server can be configured to temporarily ignore all requests for 30 seconds or so if the traffic begins to meet certain benchmarks indicating a DOS attack. A message can be displayed explaining that the server is taking a break (or whatever) and will be back in X amount of seconds. This way, a DOS attack would never succeed. This would be similar to what happens when someone attempts x amount of login attempts, then more logins are no longer allowed for x amount of seconds to prevent a password cracker routine from attempting a million different login attempts.
| doubting_thomas wrote: |
| Islamis_Allah_Tashit wrote: |
Please correct me if I'm wrong, but I thought that any decent sql server can handle more than 1000 seemingly simultaneous requests. Am I way off on this?
P.S., love your name. |
Sure. The SQL server can handle that many socket connections, and each request can be in various stages of completion at the same time. The problem arises when many of those requests require searching for data on the hard disk. Data coming from the hard disk must be serialized. The hard disk can't read multiple locations on the disk at the same time, and searching the hard disk is dramatically slower than searching for data in memory. |
Well, that's not a problem with "search" disabled.
| doubting_thomas wrote: |
We could see a significant improvement in the performance of this server by either drastically reducing the size of the forum databases so that they could all be cached in memory, or increasing the amount of RAM on the server. The latter is not an option with this particular server. It's several years old, and has the maximum amount of RAM it can accomodate. Our new server has four times as much RAM, and will be running a 64-bit O/S with a multicore processor. |
That will certainly help. I'd be curious to see how much. Good luck.
_________________
Somebody get me a hairdryer |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2002 phpBB Group
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
