Date: Sat, 18 Dec 1999 16:28:31 -0500 (EST) From: William Stearns To: Josh Sides , NARULA PRASHANT Cc: ML-ipchains , William Stearns Subject: Proxy arp - theory and examples, was RE: [Ipchains] Configuration Good afternoon, Josh and Narula, On Sat, 18 Dec 1999, Josh Sides wrote: > I'm sorry I should have read thru your response a second time. I have read > the Ipchains Howto and the Masquerade howto several times. I don't think I > have read about anything about this. Where can I read more? (See the resources at the end of this email) > William Stearns: > 2) If they won't do this and you're left with only a single block > of addresses between your firewall and their router, you can still move > some of those addresses to a separate server network. In this case, your > firewall will perform proxy arp for those machines. This means the > firewall will, at the Ethernet level, stand in for those machines so that > the router thinks they're actually on the original cable when in fact > they're on the server lan. I'll present a little of the background and then try to apply it to Narula's configuration below. Proxyarp is a technique that allows a machine that should be on an Ethernet cable to actually be somewhere else. A proxy machine stands in for the missing box and takes responsibility for getting the packets to the missing box. Proxyarp is really a way of working around incorrect or incomplete routing tables. I believe any problem solved with proxyarp could also be solved by putting correct routing tables in place. If one has an uncooperative ISP that won't update their routing, it can come in handy. Proxyarp is also used for the router accepting incoming ppp connections at an ISP; when a client calls and connects, the ISP's router can simply start standing in for that client on it's Ethernet cable. Back to the client end, here's a common example: .2 .3 .4 .5 | | | | ISP Router ------------------ Linux box .1 12.13.14.X/24 .10 (A) (B) Four servers and a Linux box on an Ethernet hub with real IP's. The ISP's router believes that all the machines from 12.13.14.1 to 12.13.14.254 are on the Ethernet network. The (A) and (B) are the Ethernet cards - we'll need these later. I did not include the fact that on the left hand side of the ISP's router is the connection to the Internet; this might be a dial up connection or a dedicated line. Picture an arrow pointing left to the Internet from the ISP's router. Lets say that we want to move two of the boxes behind the Linux box. It could be that you want to protect servers .4 and .5 behind the Linux box so it can act as a firewall for them. Maybe those machines are moving to a different area of the building and you need to put them on a different hub. Here's the new layout: .2 .3 >> .4 .5 | | >> | | ISP Router ------------------ Linux box ---------------- .1 12.13.14.X/24 .10 .11 (also 12.13.14.X/24 addresses) (A) (B) (C) The .4 and .5 servers are on the other side of the Linux box now. Here's the problem. If .2 or .3 need to talk to .4 or the router (.1) has a packet destined to .4, they'll send out a broadcast on the left cable saying "What Ethernet card has .4?" (*1) If .4 was still on the left hand Ethernet network, it would respond with it's own mac (Ethernet card) address and the IP packets would flow. Unfortunately, .4 is no longer on that cable to hear the ARP request, so .1, .2, and .3 think that .4 doesn't exist. (*2) There are two ways to fix this. The first is to explicitly tell .1, .2 and .3 that .4 and 5 can be found on the other side of .10. In other words, add a routing table entry that says "Give all packets with a destination address of .4 to .10 and let .10 figure out how to get them to .4". If .2 or .3 were Linux boxes (you could add these routes to almost any IP capable OS, but I'm just giving a Linux example), you could add the following to their startup scripts (*3): route add -host 12.13.14.4 gw 12.13.14.10 route add -host 12.13.14.5 gw 12.13.14.10 These routes would take precedence over the existing route add -net 12.13.14.0 netmask 255.255.255.0 dev eth0 that already exist because the host routes point at a smaller number of machines (1 each) than the network route (for 256 machines). See "man route". Now .2 and .3 understand that they can't directly reach .4 or .5 on their Ethernet cable. Instead, they give all packets destined for .4 to .10 and hope that it can find .4, which it can. That's fine for the .2 and .3 Linux boxes, but what about the router? The ISP owns it and isn't willing to let us play around with it's configuration. OK, we'll let the router believe that all 254 machines are still on the left hand Ethernet cable. And we'll lie to it. :-) We tell .10 that whenever it hears an ARP request like "I'm 12.13.14.1, can someone tell me what Ethernet card has 12.13.14.4?", it should lie and respond "Yes .1, .4 can be found at my (B) Ethernet card.". .1 will then start giving all packets that should have gone to .4 to .10's (B) Ethernet card instead, thinking it's talking to .4. In fact, those packets have to go through .10 and have one more cable to cross before they reach .4, but .1 doesn't care. (Note: at this point, the Linux box is a router - it's forwarding packets from one IP interface to another. Make sure you have IP forwarding turned on or the packets won't get passed back and forth.) Assuming (B) is eth0 and has a mac address of 00:10:5A:CC:97:BF, here's what you'd type on the Linux box to make this happen: route add -host 12.13.14.4 dev eth1 route add -host 12.13.14.5 dev eth1 Translation: If I need to find .4 or .5, look out the (C) interface instead of the (B) where I'd normally look for 12.13.14.0/24 addresses. arp -i eth0 -s 12.13.14.4 00:10:5A:CC:97:BF pub arp -i eth0 -s 12.13.14.5 00:10:5A:CC:97:BF pub Translation: If I hear incoming arp requests on eth0 (the left hand Ethernet network for .4 or .5, respond that those machines can be reached through the (B) Ethernet card. See "man arp" for more details. If you use this second approach (proxyarp), you no longer have to make any routing changes to .1, .2, or .3. You do have to make a little bit of a change to .4 and .5, however. After booting, you need to do the following on .4: route del -net 12.13.14.0 netmask 255.255.255.0 dev eth0 route add -host 12.13.14.11 dev eth0 route add -host 12.13.14.5 dev eth0 route del default gw 12.13.14.1 route add default gw 12.13.14.11 Translation: take away the incorrect assumption that the entire class C is on the right hand Ethernet cable, register the fact that .11 and .5 _are_ on the right hand Ethernet cable, and hand off all packets going anywhere else to .11 and not the .1 we would have used if we were still on the left hand cable.. .5 would need an almost identical set (the only change being that it needs to be told that _.4_ is on it's cable): route del -net 12.13.14.0 netmask 255.255.255.0 dev eth0 route add -host 12.13.14.11 dev eth0 route add -host 12.13.14.4 dev eth0 route del default gw 12.13.14.1 route add default gw 12.13.14.11 Why all this work and effort? Because you have machines (at least the .1 router) that don't have a correct view of how you network is laid out. You need to get another machine (.10) to lie about how to reach the missing machines (.4 and .5) so that .1 can reach .4 and .5 . Now to Narula's question and a real world example of the above: > [Narula] > dear william, i read your post on the ipchains mailing list. can you > please tell me how to implement proxy arp on RH6.0 kernel 2.2.5. > Will this approach be able to solve my problem which is as below: I > have a real class c network on my external network: 209.100.10.0/24. > The ip address of the router is 209.100.10.1 and the ip address of the > external interface of the Linux firewall is 209.100.10.2. My internal > network is 172.16.0.0/16. The ip address of the internal interface of > the Linux firewall is 172.16.1.1. I have assigned my servers > 172.16.10.1-255 and my client workstations 172.16.11.1-172.16.30.255. > Using ip masq'ing, all machines can see the Internet with no problems. > The reason I am needing nat is so that I can map real ip addresses to > my servers so they will be visible accessible from the internet, yet > still behind my firewall. > > I want to take the real ip addresses 209.100.10.11-20 and map them to > 172.16.10.11-20. > > Address match ups will look like this: > > 209.100.10.11 => 172.16.10.11 > 209.100.10.12 => 172.16.10.12 > 209.100.10.13 => 172.16.10.13 > 209.100.10.14 => 172.16.10.14 > 209.100.10.15 => 172.16.10.15 > 209.100.10.16 => 172.16.10.16 > 209.100.10.17 => 172.16.10.17 > 209.100.10.18 => 172.16.10.18 > 209.100.10.19 => 172.16.10.19 > 209.100.10.20 => 172.16.10.20 > > Simply put, if someone tries to access a web server at 209.100.10.14, > the Linux firewall will translate the address through to 172.16.10.14. > Also, I cannot use ip-masq to return packets because they need to be > returned from 209.100.10.14, not the external interface of my firewall > 209.100.10.1 for this to work properly. I'll assume eth0 is your outside interface on the Linux firewall and that eth1 is the inside interface. The above choices still apply. You can do any of the following: - Tell your router that packets destined to these ten machines need to be handed to the Linux firewall (209.100.10.2). Give the Linux box static routes to these machines on the internal network. Router: route add -host 209.100.10.11 gw 209.100.10.2 (repeat for the rest. You'll need to figure out the syntax to use for your particular router, but the idea is that it should hand off packets to 209.100.10.11, netmask 255.255.255.255 to 209.100.10.2, often called the gateway.) Linux firewall: route add -host 209.100.10.11 dev eth1 (repeat for the rest) ifconfig eth1:1 209.100.10.3 netmask 255.255.255.255 up Servers: Give them the real IP's instead of reserved IP's. route del -net 209.100.10.0/24 dev eth0 route add -host 209.100.10.11 dev eth0 (repeat for all other servers) route del default gw 209.100.10.1 route add default gw 209.100.10.3 - Set up proxy arp so that the Linux firewall acts as a stand-in on the outside cable for the machines that are now on the inside network (.11-.20). Router: (nothing to do) Linux firewall: route add -host 209.100.10.11 dev eth1 (repeat for the rest) ifconfig eth1:1 209.100.10.3 netmask 255.255.255.255 up arp -i eth0 -s 209.100.10.11 00:10:5A:CC:97:BF pub (repeat for all, use the mac address for the Linux firewall's eth0 card instead of the example) Servers: Give them the real IP's instead of reserved IP's. route del -net 209.100.10.0/24 dev eth0 route add -host 209.100.10.11 dev eth0 (repeat for all other servers) route del default gw 209.100.10.1 route add default gw 209.100.10.3 Note that the only difference between the first approach and the second is how the router knows to hand packets destined for the internal servers to the Linux firewall - in the first example, it's told this explicitly, in the second proxyarp example, the Linux firewall convinces the router that it can get packets to the servers. - Give the Linux box the .11-.20 IP addresses and tell it to perform NAT to the boxes on the Internal lan. (Sorry - not all that familiar with how to do formal NAT. It involves bringing up all the real IP's on the Linux firewall itself and using the "ip" command from the "iproute" package to tell the kernel to translate addresses from the real back to the reserved addresses on the servers.) - One more possibility, if you only have a few ports that need to be handed back to the servers, is to use the single real address on your Linux firewall and do port forwarding back to the servers on reserved addresses. This is probably not appropriate if you have more than one web server, more than one mail server, or more than one (any IP service) server. See the IP Masquerading HOWTO and the ipmasq resource page for more information on port forwarding. The above examples haven't been tested - you'll almost certainly have to do some tweaking to make them work. If things don't work at first, use ping to see which machines can reach which machines. Examine the routing tables on all machines involved to see if the outgoing ping requests (icmp echo requests) can get all the way to the destination machine and the ping replies (icmp echo replies) can get all the way back. Cheers, - Bill *1 This is called an ARP (Address Resolution Protocol) query. .1 sends out an Ethernet broadcast packet asking what Ethernet card is associated with 12.13.14.4. .4 is supposed to respond with "I can be found at Mac address 04:05:06:07:08:09" so that .1 can now direct IP packets for .4 to that mac address instead of having to broadcast everything to all Ethernet cards. Run tcpdump on an Ethernet interface sometime ("tcpdump -i eth0 -tn") and you'll see the ARP requests and responses from time to time. *2 Note that ARP packets stay confined to a single lan - we can't ask the Linux box in the center to carry the ARP query over to the right hand cable. *3 All of the commands listed here can go either in your network startup script (perhaps /etc/rc.d/init.d/network for RedHat) or in /etc/rc.d/rc.local . They should be in place before any other services need to reach other machines, so /etc/rc.d/init.d/network is preferred. Place them after the other commands that bring up interfaces in that file. Resources: - The Network Administrators Guide (NAG - see "Address Resolution" and "Checking the ARP Tables") http://metalab.unc.edu/linux/LDP - The IP Masquerading HOWTO, the Proxy ARP Subnetting HOWTO and the (unmaintained) Proxy-arp mini HOWTO at: http://metalab.unc.edu/linux/HOWTO/ - "man arp", "man route", the documentation in the iproute or iproute2 package: ftp://ftp.inr.ac.ru/ip-routing/ --------------------------------------------------------------------------- I'm not tense, just terribly, terribly alert. (Courtesy of "Michael J. Dark" ) -------------------------------------------------------------------------- William Stearns (wstearns@pobox.com). Mason, Buildkernel, named2hosts, and ipfwadm2ipchains are at: http://www.pobox.com/~wstearns/ --------------------------------------------------------------------------