Friday, May 3, 2013

Squid with Antivirus and Traffic Shaping

This is not a gateway/firewall setup, for that my personal favorite is pfSense.

Edit: Jan 30 2019 - This info is probably getting out of date, most http traffic these days is over https, so using Squid to do caching has become largely redundant, and the AV scanning wont work unless you goto the effort of pushing your own root cert to every machine on the LAN, and implementing some form of SSL interception to inspect the traffic. However, the traffic shaping part of this still may be useful in solving a problem. Thou the days of having to engineer stuff like this at work places is largely a thing of the past with large amounts of cheap bandwidth and cloud services a plenty, the average small business would be better off employing a trusted 3rd party service to filter their internet traffic, monitor their network and generally do all this and more for a yearly fee - prices might still seem steep, but it makes it someone else's problem, and shifts the liability to them, in that light its not so bad for piece of mind.
I had the slightly unique situation at a previous work place where, we had only 4mbps symmetrical DSL,  our firewall was not capable of managing bandwidth - the problem we were having is that a single download would consume all the bandwidth and cause latency to increase which would impact VPN and remote desktop sessions. I also wanted to filter HTTP traffic for viruses and malware.
The solution was to run a proxy server for web traffic, filter it for virus and malware, and apply some traffic shaping on both the downstream and upstream to keep things fair. And provide a way to transparently configure computers on the network to use it.
First I investigated delay pools in Squid, which didn't work terribly well. I looked at TC on Linux and this is way too complex to get working properly.
Finally I looked at FreeBSD, and found it's ipfw firewall has traffic shaping baked right into it in the form of pipes and queues. This turned out to be surprisingly simple and easy to get working based an example I found here: FreeBSD ipfw Traffic Shaping Firewall Script

Software Used

Installing FreeBSD

Go with the defaults except:
  • Deselect ports, and games.
  • Manual IP, no IPv6.
  • Yes to sshd, ntp.
  • No crash dumps.
  • Add an account for your self, add the wheel group (allows su -).

Post Installation Configuration

Install Ports Collection

First time:
portsnap fetch
portsnap extract
portsnap update
Keeping ports up-to-date - run this before installing any new programs:
portsnap fetch
portsnap update

Setup firewall and traffic shaping

Enable firewall and start it.
echo 'firewall_enable="YES"' >> /etc/rc.conf
echo 'firewall_type="OPEN"' >> /etc/rc.conf
/etc/rc.d/ipfw start
ipfw.rules traffic shaping script:
# Based on http://bash.cyberciti.biz/security/freebsd-ipfw-traffic-shaping-firewall-script/
# This is simplified and adapted to allow free flow of LAN traffic, but to manage
# upstream and downstream to and from anything outside of of the LAN subnet.

#firewall command
fwcmd="/sbin/ipfw"
 
#interfaces
wire=em0
internal="10.1.21.0/24"


# Force a flush and reload.
# See /etc/rc.firewall for default rules and tweak that as needed.
# Since were internal only run in open mode.
/etc/rc.d/ipfw restart
kldload -nv dummynet
 
# Setup incoming and outgoing pipes to 75% of internet link speed.
# External In
$fwcmd pipe 10 config bw 3Mbit/s
# External Out
$fwcmd pipe 20 config bw 3Mbit/s

################################################################################
#No shaping between internal networks
################################################################################
 
$fwcmd add 2000 skipto 64000 ip from $internal to $internal in via ${wire}
$fwcmd add 2100 skipto 64000 ip from $internal to $internal out via ${wire}
 
################################################################################
# Setup bandwidth shaping queues
# Higher weight, high priorities
################################################################################
 
# High priority queue for tcp ACK
$fwcmd queue 10 config pipe 20 weight 90
 
# Low priority queue for other users
$fwcmd queue 20 config pipe 10 weight 25
$fwcmd queue 30 config pipe 20 weight 25
 
################################################################################
#Traffic shaping
################################################################################

#TCP ACK
$fwcmd add 3000 queue 10 ip from any to any out via ${wire} tcpflags ack iplen 52
 
#General traffic low priority
$fwcmd add 3100 queue 20 ip from any to $internal in via ${wire}
$fwcmd add 3200 queue 30 ip from $internal to any out via ${wire}

echo 'Traffic shaping rules loaded'
Copy to /root/ipfw.rules and execute, this will restart the firewall and then add the shaping rules:
sh ipfw.rules
 This was how I was testing the shaping rules, rather than edit rc.firewall, it was easier to put the shaping rules into different file and have it reload the firewall to flush everything and add the (updated) shaping rules.
So ideally this needs to be tidied up appropriately on a production system..
I think the shaping rules need to go in-to /etc/rc.firewall so they are loaded on bootup.

Software Installation

SSMTP

Replace sendmail with ssmtp and configure to relay email to the company email server so it winds up in an inbox that gets read by someone, probably you.
This breaks it because no one can read the config file! So I chmodded the dir to 755, files to 644. This needs a little more work to get the process of setting up ssmtp and its config just right..
http://log.brandonthomson.com/2010/10/freebsd-use-gmail-instead-of-sendmail.html - The from name for root appears as “Charlie &” or maybe some other weird name. As root run chpass and change the Full Name field to something more useful like “Root at <server_name>”
echo 'sendmail_enable="NO"' >> /etc/rc.conf
echo 'sendmail_submit_enable="NO"' >> /etc/rc.conf
echo 'sendmail_outbound_enable="NO"' >> /etc/rc.conf
echo 'sendmail_msp_queue_enable="NO"' >> /etc/rc.conf

killall sendmail
cd /usr/ports/mail/ssmtp
make install replace clean
 Insert config file

Squid

cd /usr/ports/www/squid32
make install clean
Enable ICAP client.
Configure the cache directory.
chown -R squid:squid /var/squid/cache
squid -z
Configure Squid /usr/local/etc/squid/squid.conf
Make config file writable:
chmod +w /usr/local/etc/squid/squid.conf
Uncomment cache_dir ufs /var/squid/cache/squid 100 16 256 line if needed, if left commented out, Squid will use default settings.
  1. On production change to 1000 or something larger.
  2. On test VM leave at say 100mb and change/add cache_mem 100 MB.
Our own additions:
visible_hostname change.this.to.dns.name.of.server
cache_mgr admin.email@your.mail.server
request_header_access X-Forwarded-For deny all
Test the config file for errors.
squid -f /usr/local/etc/squid/squid.conf -k parse
Enabe Squid at bootup
echo 'squid_enable="YES"' >> /etc/rc.conf
Start squid
/usr/local/etc/rc.d/squid start
Test squid by manually configuring Firefox to use it as a proxy server on port 3128.

Install ClamAV

cd /usr/ports/security/clamav
make install clean
Start ClamAV at bootup:
echo 'clamav_freshclam_enable="YES"' >> /etc/rc.conf
echo 'clamav_clamd_enable="YES"' >> /etc/rc.conf
Configure /usr/local/etc/freshclam.conf
It's not writable by default, so do this:
chmod +w /usr/local/etc/freshclam.conf
Configure a local database mirror, see http://www.iana.org/cctld/cctld-whois.htm for a list of the two letter country codes.
If you're in the US:
echo 'DatabaseMirror db.US.clamav.net' >> /usr/local/etc/freshclam.conf
Update the database:
/usr/local/etc/rc.d/clamav-freshclam start && tail -f /var/log/clamav/freshclam.log
Can take a while if you hit a slow mirror. Once it updates you'll get this error have not started clamav yet ..
WARNING: Clamd was NOT notified: Can't connect to clamd through /var/run/clamav/clamd.sock
Start ClamAV
/usr/local/etc/rc.d/clamav-clamd start

Install SquidClamAV

cd /usr/ports/www/squidclamav
make install clean
c-icap is a dependecy, and will also get installed:
  1. Deselect IPv6 support - otherwise it listens on IPv6 only!
  2. Enable-large-files.
/usr/local/etc/c-icap/squidclamav.conf
clamd_local /var/run/clamav/clamd.sock

/usr/local/etc/c-icap/c-icap.conf
ServerAdmin sys.admin@your.company
ServerName dns.name.of.this.server
Enable c-icap at bootup
echo 'c_icap_enable="YES"' >> /etc/rc.conf
/usr/local/etc/rc.d/c-icap start
icap config for Squid, add this to /usr/local/etc/squid/squid.conf
# icap config
icap_enable on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_encode off
icap_client_username_header X-Authenticated-User
icap_preview_enable on
icap_preview_size 1024
icap_service service_req reqmod_precache bypass=0 icap://127.0.0.1:1344/squidclamav
icap_service service_resp respmod_precache bypass=0 icap://127.0.0.1:1344/squidclamav
adaptation_access service_req allow all
adaptation_access service_resp allow all

Lighttpd

cd /usr/ports/www/lighttpd
make install clean
  1. Add BZIP2
  2. Del IPv6
Create the directories
mkdir /usr/local/www/data
mkdir /usr/local/www/data/cgi-bin
cd /usr/local/www/data/cgi-bin
ln -s /usr/local/libexec/squidclamav/clwarn.cgi clwarn.cgi
chown www:wheel /usr/local/www/data
There are a couple of goofups (does anyone actually test this stuff?) in the config file /usr/local/etc/lighttpd/lighttpd.conf
  1. Change server.use-ipv6 = “enable” to server.use-ipv6 = “disable”
  2. Comment out this line: $SERVER[“socket”] == “0.0.0.0:80” { }
Next enable cgi and the squidclamav warning page when a virus is detected.
http://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_ModCGI
/usr/local/etc/lighttpd/lighttpd.conf
server.modules += ( "mod_cgi" ) 
   cgi.assign = ( ".pl"  => "/usr/bin/perl", ".cgi" => "/usr/bin/perl" )
Enable lighttpd at bootup and start it
echo 'lighttpd_enable="YES"' >> /etc/rc.conf
/usr/local/etc/rc.d/lighttpd start

Testing Things Out

  1. Create /usr/local/www/data/index.html with one line of text, or something fancy if you're in to HTML. Point a web browser at the server to test it out.
  2. Using a web browser confifured to use the server as a proxy go-to http://www.eicar.org/85-0-Download.html and when you try and download the test files you should get redirected to the warning page.

Auto config browser

In the examples given the proxy server is using 10.1.21.252, change this to suit your network.

DHCP Option 252

To create a WPAD entry in the DHCP Server service (Windows Server 2008):
  1. Log on to the server running the DHCP role as a domain administrator.
  2. Click Start, point to Administrative Tools, and then click DHCP.
  3. Expand the name of the Management Server, right-click IPv4, and then click Set Predefined Options.
  4. In the Predefined Options and Values dialog box, click Add.
  5. In the Option Type dialog box, do the following:
    • In Name, type WPAD.
    • In Code, type 252.
    • In Data type, select String, and then click OK.
    • In String, type enter the URL of the wpad.dat file, e.g. if you're hosting it from the proxy server: http://10.1.21.252/wpad.dat, and then click OK.
  6. In the console tree, expand the DHCP scope for which you want to configure WPAD, right-click Scope Options, and then click Configure Options.
  7. Click Advanced, and then in Vendor Class, click Standard Options.
  8. In Available Options, select 252 WPAD, and then click OK.

WPAD.DAT

Create wpad.dat on an internally accessible web server.
Here we instruct the browser to go direct for localhost/loopback addresses, plain hostnames, RFC 1918 private IP addresses, and any specific domain exclusions for broken sites where the devs could not be bothered.
function FindProxyForURL(url, host)
{
    if (dnsDomainIs(host, "localhost")) return "DIRECT";
    if (isInNet(host, "127.0.0.0", "255.0.0.0")) return "DIRECT";
    if (isPlainHostName(host)) return "DIRECT";
    if (isInNet(host, "10.0.0.0", "255.0.0.0")) return "DIRECT";
    if (isInNet(host, "172.16.0.0", "255.240.0.0")) return "DIRECT";
    if (isInNet(host, "192.168.0.0", "255.255.255.0")) return "DIRECT";
    if (dnsDomainIs(host, ".example_domain_to_exclude.com")) return "DIRECT";
    return "PROXY 10.1.21.252:3128";
}

Windows Internet Settings

Enable Internet explorer Automatically detect connection settings using registry key and group policy to apply it.
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections

Monitoring Activity

Watch the requests in real time:
tail -f /var/log/squid3/access.log
You can also filter the log file by piping it through grep, e.g. to see requests from a specific machine:
tail -f /var/log/squid3/access.log | grep "10.1.21.99"
If Munin is installed on the same host, it'll make some simple graphs of Squid's activity and cache hits.

Enhancements

  • Investigate user authentication with AD so usernames show in the logs and internet access controlled via group membership. This is a little involved, the added complexity may not be worth it in a small environment. To track someone down we can do a reverse lookup on the clients IP address recorded in the squid log file (/var/log/squid3/access.log) to get the PC name, see who that PC is assigned to.

References