Transparent Caching Proxy with Squid and Just One Ethernet Port

Tags: ARM, Efika, Linux, Network, Squid

Some time ago we had a project which needed a simple transparent caching proxy for use in classrooms in Latin America. The classroom desktops consisted of ten of our Efika's, and we thought why not use an extra one of those to act as the proxy. This is not the kind of high bandwidth environment of a typical installation, so the performance limitations were not really an obstacle. The goal was rather to use the limited internet connection available as efficiently as possible. The configuration of the system using Squid is detailed below.

Efika NetTop

The goals:

  • Install and configure DHCP server on the system for the clients
  • Install and configure Squid as a transparent caching proxy with one Ethernet port
  • Setup of the network interface and iptables rules to route packages appropriately

The Network consists of: 

  • Internet gateway only visible by the Squid server (IP = 192.168.253)
  • The Squid server (IP = 192.169.2.253)
  • Several clients with IP addresses serviced by the same Squid server (Range 10.0.0.10/10.0.0.100)

The first step is to make sure the system (assuming a headless Debian or Ubuntu installation) is up to date (aptitude update followed by aptitude safe-upgrade). It would probably be a good idea to have OpenSSH running so the configuration can be done remotely.

We need the required Squid and DHCP server software installed:

aptitude install squid3 dhcp3-server

Note: there is also a package called squid (without the '3'). We opt to use Squid3 since it is easier to configure and subjectively runs better.

Now, we have to configure the network interface. Since we have only one, we are going to multi-host this interface to serve two networks: the 192.168.2.x and 10.0.0.x ones. Edit /etc/network/interfaces file so it looks like this:

auto lo
iface lo inet loopback
	
auto eth0
iface eth0 inet static
	address 192.168.2.253
	netmask 255.255.255.0
	network 192.168.2.0
	broadcast 192.168.2.255 
	gateway 192.168.2.254

auto eth0:0
iface eth0:0 inet static
	name Ethernet alias LAN card
	address 10.0.0.1
	netmask 255.255.255.0	
	network 10.0.0.0

Both eth0 and eth0:0 are configured static, with eth0 the part that will talk to the internet gateway, and eth0:0 the part that will talk to our local network. After we have configured the network interface, let's continue with the DHCP server so we can serve our clients with IP addresses. Edit /etc/dhcp3/dhcpd.conf and add a subnet as follows:

subnet 10.0.0.0 netmask 255.255.255.0 {
	range 10.0.0.10 10.0.0.100;
	option routers 10.0.0.1;
}

This will provision our clients with IP addresses in the range of 10.0.0.1o to 10.0.0.100, more than enough for this application. Make sure that in the same configuration file you also configure the correct domain name servers for your internet. This should be done in the following directives:

option domain-name "example.org";
option domain-name-servers ns1.example.org, ns2.example.org;

Next comes the configuration of Squid itself. The config file is quite big, but we only need to make some small changes to make it work. Open /etc/squid3/squid.conf in an editor and locate the section that deals with access control lists (acl). There should already be a minimum section like this:

acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1

To this section, add the following acl:

acl our_networks src 10.0.0.0/24

Next, locate the section that sets the http_port variable. It should look like this:

# Squid normally listens to port 3128 http_port 3128

change the value of the http_port variable so that it reads:

# Squid normally listens to port 3128 http_port 10.0.0.1:3128 transparent

The reason for this is twofold: it enables transparent mode so we don't have to configure proxy settings on each client, and it makes sure we only listen on the internal network. Next, we have to configure to allow the clients access to the Squid server. Search for a section that configures http_access rules (or search for "INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS"). There, add the following line:

http_access allow our_networks

Finally, turn on the disk cache mechanism. Search for "Uncomment and adjust the following to add a disk cache directory" and change the line that reads:

#cache_dir ufs /var/spool/squid3 100 16 256

into:

cache_dir ufs /var/spool/squid3 2000 16 256

That's all for Squid. To tie this all together, we have to instruct Linux that it is ok to route packets, and how they have to be routed. For that, a script was written with inline comments explaining the directives which looks like this:

#!/bin/sh

# Squid server IP
SQUID_SERVER="10.0.0.1"

# Interface connected to Internet
INTERNET="eth0"

# Address connected to LAN
LOCAL="10.0.0.0/24"

# Squid port
SQUID_PORT="3128"

# Clean old firewall
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X

# Enable Forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward

# Setting default filter policy
iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT

# Unlimited access to loop back
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

# Allow UDP, DNS and Passive FTP
iptables -A INPUT -i $INTERNET -m state --state ESTABLISHED,RELATED -j ACCEPT

# set this system as a router for Rest of LAN
iptables -t nat -A POSTROUTING -o $INTERNET -j MASQUERADE
iptables -A FORWARD -s $LOCAL -j ACCEPT

# unlimited access to LAN
iptables -A INPUT -s $LOCAL -j ACCEPT
iptables -A OUTPUT -s $LOCAL -j ACCEPT

# DNAT port 80 request comming from LAN systems to squid 3128 ($SQUID_PORT) aka transparent proxy
iptables -t nat -A PREROUTING -s $LOCAL -p tcp --dport 80 -j DNAT --to $SQUID_SERVER:$SQUID_PORT

# if it is same system
iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT

#open everything
iptables -A INPUT -i $INTERNET -j ACCEPT
iptables -A OUTPUT -o $INTERNET  -j ACCEPT

# DROP everything and Log it
iptables -A INPUT -j LOG
iptables -A INPUT -j DROP

Make this script executable and run it at startup, for example within /etc/rc.local. That's it! Make sure you restart all the services and double check the clients get the correct IP address, gateway and name servers. The full configuration files and script are attached. Remove the .txt extension from 'interfaces.txt' and 'firewall.sh.txt'

Attachments

Comments

Can we do this with Linux Namespace

I have installed linux container namespace or Debian ARM architecture .. can you help me to run this conig  in linux namespace

Re: Can we do this with Linux Namespace

It's probably possible, but keep in mind that this article is quite old by now - things might have changed. 
It's been a while since I've played around wtih things like proxies, so I don't think I'll be able to guide you through the process (I also have very limited free time). However the principles are pretty much the same, so you should be able to get there. I think there should be plenty of other resources available online that can help you as well.

Hi

 

I am using ubuntu 18.04.3tls and I have two network cards one local

and one in bridge mode as I can implement the squid in transparent mode.

 

Network card in bridge mode has the network 192.168.1.0/24.With this I have internet access and the card that is in the local network 172.168.1.0/24

Sorry, my English is that I'm Latino and I don't know much.

I am using ubuntu 18.04.3tls and I have two network cards one local and one in bridge mode as I can implement the squid in transparent mode.

 

Hi, This entry is quite old,

Hi,

This entry is quite old, so I'm not sure if all the info is still relevant. However, do keep in mind that with https (TLS) being used more and more, the effectiveness of a proxy becomes less. 

Hope you can find the info in my post useful, but I think you might need to look at some other sites for more up-to-date info as well.

 

Johan.

configuration

first of all 

can you please explain for my configuration

dhcp server settup via wifi

option routers is 192.168.1.1

dynamic allocation is given for clients

configuration

Hi.

It's been a while since I wrote this entry, so this is mostly from (distant) memory. You need to have control over what is serving the IP addresses. The reason I used a single device for that is because of simplicity. You now have a router which is sending out IP addresses, but that means that none of that traffic is going through the proxy. You probably have to configure your wireless access point to route through the proxy. In other words, put the access point behind the proxy and let the proxy communicate to the internet through a switch between your WAN network and the access point. 

How you would do this exactly depends on your equipment though. The principle stays the same, except that instead of having a switch connected to the Ethernet port of the proxy and connect the PC's there, you have a wireless access point on that Ethernet port and have the PC's connect to that.

Hope this helps.

 

Johan.

Routing Issue

Hello,

Thank you so much for your guide and configuration files.

I applied everything on my configuration but the problem is my dhcp provider is core switch. Configuration is like the below scenario

 

Core Router > Firewall > Core Switch > Access Switch (With 22 VLan) > Users.

                                                                    > DMZ Switch > proxy, dns, mail, web.

DMZ zone is 10.100.222.0/24

Users 10.10.0.0/16

Now what will be routing (IPtables Default route or anything)

 

Thanks in advance.

Hi. If I understand you

Hi.

If I understand you correctly, you only have to change the IP addresses on the two Ethernet interfaces (eth0 and eth0:0) so that eth0 is connected to the DMZ and eth0:0 serves your local clients with IP addresses. Except for the IP addresses, nothing else changes.

 

Johan.

directive in squid.conf

Hi, 

Nice tutorial you've made here. Could u explain to me regarding the squid.conf. By referring to line 'http_access deny manager'. Why the config deny manager? I'm confuse. Help me

Best regards,

Tyro

Hi.Glad you like the

Hi.

Glad you like the tutorial!

http_access deny manager controls access rights to the cache manager: http://wiki.squid-cache.org/Features/CacheManager. The reason to deny it in that line is to make sure not just anyone can access the manager. The line right above that one allows for manager access form localhost only, not every device connected to the network the squid proxy is serving.

 

Johan.

Hi Johan, Nice setup. I'm

Hi Johan,

Nice setup. I'm curious - given that this is an small embedded system, how much throughput were you getting with this setup? Did you run any benchmarks comparing bandwidth before and after the proxy was placed inbetween?

Best regards,

John

Hi John, No, I did not run

Hi John,

No, I did not run any benchmarks since the bottle neck for the set-up was the main internet connection to begin with, and the proxy itself was fast enough to make that experience orders of magnitude better. I imagine that the device won't be the best choice for a corporate environment of course, but for small amounts of users with basic internet needs it works great. 

 

Johan.

Https sites not blocking

how to block https sites

Blocking https

Hi,

You can block https with iptables, e.g.:

iptables -t nat -I PREROUTING -m tcp -p tcp --dport 443 -j DROP

Keep in mind that in the set-up discussed in the post I did not make provisions for https in the first place. In order to use squid as a transparent proxy for https as well, complement my post with this one here: http://ajayadas.com/e110body-anchor/

Johan.

doubts in lines of the scrip located in /etc/rc.local and others

Hello Johan, thanks for such a fantastic tutorial, I'm doing a project with a raspberry pi 2b + and 3b +
and I thought it was great to do it with them, but on the way taking the tutorial out,
I came across that the DHCP3 SERVER version did not It was useful so I chose to use ISC-DHCP-SERVER
and everything started to work according to what you propose in the tutorial, the router is an internet service with 2MB of bandwidth
and generates its own ips range so that disable the DHCP of the router, so that my server ISC-DHCP-SERVER deploy in the LAN and WLAN
the ips to my devices and if indeed when observing in the devices the TCP / IP works and I have the transparent proxy with its due gateway ,
I can browse some http sites such as google, youtube, netflix, but wanting to see some other sites I can not through the proxy either http or https,
not if you have to add something to the script in /etc/rc.local I have doubts with some lines for example in the next line:
# DNAT port 80 request comming from LAN systems to squid 3128 ($ SQUID_PORT) aka transparent proxy
iptables -t nat -A PREROUTING -s $ LOCAL -p tcp --dport 80 -j DNAT --to $ SQUID_SERVER: $ SQUID_PORT
Do we have to add another line for port 443 with the same characteristics as with port 80? like the other line that says:
# if it is same system
iptables -t nat -A PREROUTING -i $ INTERNET -p tcp --dport 80 -j REDIRECT --to-port $ SQUID_PORT
Would we have to add another line with port 443?
and also I have doubts my router generates ipv6 and in the scritp you also have a line that says:
# Enable Forwarding
echo 1> / proc / sys / net / ipv4 / ip_forward
Would we have to add for example #Enable Forwarding
echo 1> / proc / sys / net / ipv6 / ip_forwar? I do not know what may be missing and because I do not see some sites with specific domains,
doing more research I found the topic of intercepting https by means of ssl certificates and I see that the configuration becomes more complex,
I do not want to have to be installing certificates on the computers browsers, tablets and smartphones
I would like the transparent proxy to work as the non-transparent works that can see any site, either http or https,
I hope you can enlighten me and terminate my project, in advance very grateful for your help.

Hi, Let me preface this by

Hi,

Let me preface this by saying that this tutorial was written six years ago or so, so several things changed in the mean time. Most importantly is probably the push for SSL everywhere, which is a good thing. Unfortunately, SSL is inherently incompatible with proxy caching software unless you intercept this traffic, decrypt it, and then re-encrypt with your local LAN certificate. I would say that this is not just inefficient, but also leads to security and privacy issues. Sadly, if you want to use a proxy in this day and age, it's the only option and pretty much mandatory since there relatively few non-SSL sites that would benefit from the proxy cache in the first place.

I think the changes you mention in your post should get you to a working proxy, but I won't manage to verify that for you due to lack of time. If I were you however, I would investigate one of the more recent tutorials on the web that include SSL. It's more complex, but if you take them step by step you should be able to get there.

 Johan.