NT responds to UDP segments sent to the broadcast address for a subnet. This means that every NT machine on the network will respond to a UDP segment with the broadcast address. Such a response might be so great as to cause considerable network congestion by itself.
But consider what happens to a victim machine if the UDP segment sent to the broadcast address contains a forged source address of the victim, and if the port to which the segment is sent is port 19, the "chargen" service (pumps out endless characters rotating the starting point)
Root causes
The three way TCP connection establishment handshake can be subverted. TCP packets may arrive with forged addresses.
The attack
An attacker sends, in quick succession, a series of N connection requests to the server being attacked. The SYN segments have a forged address of a machine which is unreachable by the attacked server. This means that when the attacked machine sends a SYN/ACK segment back in response to the bogus SYN segment, the connection hangs until timeout.
The backlog queue (pending connections and established connections which haven't been accept(2)d by a process yet) is of finite length. If it were not, then the server could be kept busy answering connection requests for an even simpler DOS attack. The problem is that when the backlog queue is full, subsequent connection requests are thrown away. If the backlog queue is full of bogus attack requests, real requests will be lost.
The forged address must be to an unreachable host, since otherwise the host that corresponds to the forged return address will get this SYN/ACK out of the blue and respond with a RST segment which resets the connection.
The timeout per connection is 75 seconds, the range of backlog queue size in various OSes is 6 to 128. With a backlog of 10, and a timeout of 75, the server can be made unavailable for 75 seconds just by sending ten quick attacking SYN segments.
Possible fix
A relay process running on a firewall sits between requests and servers. The relay process takes the SYN segment, answers with a SYN/ACK, then if it receives an ACK (i.e. this is a legit connection) passes a SYN request to the server, then responds with an ACK to the subsequent SYN/ACK the server sends. After that the relaying host has to sit in the middle of the connection and translate sequence numbers on all segments.
Software fixes
Increase backlog queue - doesn't fix problem, since attacker just needs to send a few more requests. Note that the TCP SYN flood attack doesn't require sender to send massive amounts of data like other DOS attacks.
Decrease the connection request timeout - makes each bogus request have less impact, but what about legit requests on slow lines?
http://www.op.net/~jaw/syn-fix.html A fix proposed by Dan Bernstein - eliminate the queue of pending connections.
When a SYN arrives, do not queue it, do not allocate a control block data structure for it, in fact, do not maintain any local record of the packet. Send the SYN/ACK, but do not allocate any resources until the confirming ACK is recieved. The means that pending requests don't take resources or have timers running for them.
To prevent bogus ACKs from creating open connections without going through a proper 3-way handshake, set the initial sequence number (iss) on the outgoing SYN/ACK to a secret value and verify the value recieved on the ACK (it should be what was sent plus one).
The iss must be difficult or impossible for an attacker to guess. One way of doing this is by using a cryptographically secure one-way hash of information contained in the incoming packet: source and destination IP addresses and port numbers, and sequence number, a secret key, and an ever changing counter.
Wietse Venema's 1992 USENIX paper describing TCP wrappers
In place of a regular Internet service and it's corresponding process, you put another process. This process accepts the connection, as usual, does some logging, possibly executes some commands, then consults a rule-base to decide if the connection request is legit. If so, it passes the connection to the real server who takes over from that point.
The commands executed can be used to help track hackers. For example, when a request for a local service comes from a remote site, the tcp wrapper could send a FINGER request to the remote site to find out who is logged in and record that information. This might lead to determining which accounts on the remote site have been compromised.
To standardize the ability of a TCP wrapper to get information on an intruder, and to be able to distinguish bad guys from innocent users in a multiuser environment, a new IP service has been defined called ident. The ident service provides user identifying information for a given IP address and port. The TCP wrapper uses the ident service on the machine from which a suspicious packet has just arrived. The suspicious machine provides the IP addres and port (in effect identifying the socket in use) and gets in return some information about the process that owns that socket. The information could include a userid, but is left general in the specs so that ident can work on a variety of OSs.