Web Content Filtering with DansGuardian

 "Learn how to setup a parental blocker to protect yourself and your network from potentially harmful sites."

Requirements:

• A Linux distro
• Internet Access
• Administrative Privileges*

*If you are not logged in as root, you may need to type "sudo" prior to any command - this runs the command as the administrator. A password will need to be provided the first time it's used

DansGuardian:

DansGuardian is software that has "smart" web content filtering. It looks at all of the text in a web page and has different "weights" for specific words. For example: "Breast Cancer". The word "Breast" may have a weight of (+5), and "Cancer" may have a weight of (-10). Combined, the weight is (-5). If the weight of the page exceeds a particular amount, the page is blocked. Together with the proxy server Squid, we can direct ALL HTTP traffic to Dans Guardian for filtering. Normally it would be easy for the user to just reconfigure the proxy settings in their web browser to work around it, but we'll show you how that wouldn't have any effect on our filter.

Step 1: Install "dansguardian"

You can use your package manager to find, download, and install "dansguardian". Using "apt-get" or "yum" would work for most distros. Dansguardian should be in most major repositories (Ubuntu contains it for sure).
If you would like to download the package yourself, you can download it from the DansGuardian download page.

Step 2: Install "squid"

Just as the previous step, look for "squid" in your package manager and download/install it. It should definitely be in your repositories. Otherwise you can download it from their download page.

Step 3: Configure Squid

Next we need to make sure squid is configured properly. We want squid to run "transparently" in the background, so we need to modify its configuration file. To do so, run the following command in the terminal:

 username@localhost:$ sudo gedit /etc/squid/squid.conf

Now search for the line that has "http_port" in it. Ther will be a few in the commented sections, but we're looking for the one that has its default port listed. It should be something like "http_port 3128".
Once you have found this line, add the word "transparent" right after it:
"http_port 3128 transparent"
Make sure you save, then exit. This is all we needed to do to configure Squid.

Step 4: Configure DansGuardian

DansGuardian allows us to make precise configurations in order to make the slightest changes to details according to our needs. We can change a whole ton of things in the configuration file. Here I will take you through the simplest ways in order to just get it running. Later, I'll show you how to change the "Access Denial" page to customize it in anyway you wish (optional, of course). But for now, we need to open up the configuration file by typing the following command:

username@localhost:$ sudo gedit /etc/dansguardian/dansguardian.conf

The first thing we want to do is find the line that tells DansGuardian which port to look for our Squid proxy server that we had just setup. We need to find the line that says "proxyport = ". By default, it may already have the proper port "3128". Make sure it does, and also make sure our "filterport=" is set to "8080": 

"proxyport = 3128"
"filterport = 8080"

The last line we need to edit is the "UNCONFIGURED" line. This line is up near the top of the file, and says something like: "UNCONFIGURED - Please remove this line after configuration". This is DansGuardian's way of knowing that we have setup the configuration file and are ready for it to start using our configurations. You can either remove this line entirely, or comment it out by placing a "#" in front of it.

Once completed, save the file and exit. 

Step 5: Restart Squid and DansGuardian

You can restart Squid and DansGuardian by rebooting your computer, or typing the following lines, in order:

 username@localhost:$ sudo /etc/init.d/squid restart

username@localhost:$ sudo dansguardian -q

username@localhost:$ sudo dansguardian

Now, if you try to go to our "BAD" test page, you should still get through. This is because we need to setup our web browser to follow the proxy server settings.

Step 6: Setup Web Browser's Proxy Server Settings

Each web browser has their settings in a different location - but most are within a "Preferences" or "Options" menu. Locate the settings window and change the settings to the following:

Manual Proxy Settings
HTTP Proxy - "localhost:8080"

This will tell the browser to use the proxy server instead of directly connecting to the internet. Once completed, close out and try accessing this page again. At this point, you should see a denial page from DansGuardian. The question is, if it's that easy to change proxy settings, why can't they just change it back? That's what the next step will take care of...

Step 7: Direct All HTTP Traffic Through Squid

We really want to make sure that all of the HTTP (port 8080) traffic is sent through Squid, and therefore, through DansGuardian. To test this, we can run the following command:

username@localhost:$ iptables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner proxy --dport 80 -j REDIRECT --to-port 8080

This tells the system that for ANY HTTP traffic (any "port 80", including port "8080") should be redirected to port "8080", which is where Squid resides. This way, whether the browsers are set to listen to the proxy, or are told to connect directly to the internet, it will ALWAYS be redirected to our proxy. This is technically modifying iptables, so if you have a separate firewall installed, you may need to configure it. Ubuntu users from a fresh-install (no firewalls added) - this will work just fine as is.

Now you can reconfigure your browser to NOT listen to the proxy (connect directly to the internet) - and try accessng this page again. You should still see the rejection page.

This command will stay in effect until the system is rebooted. So to make it always run upon startup, we need to place an executable script in "/etc/init.d/" with that command in it. Let's call it "tproxy":

username@localhost:$ sudo gedit /etc/init.d/tproxy

This will create that file and bring up gedit. So just paste the above command ("iptables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner proxy --dport 80 -j REDIRECT --to-port 8080") into the file. Save and exit gedit.

Now we need to make it executable. You can do that by running this command:

username@localhost:$ sudo chmod a+x /etc/init.d/tproxy

Lastly, we need to tell the system to run this script at startup. Do this by running the command:

username@localhost:$ sudo update-rc.d tproxy defaults

Now the iptable redirection will occur at every starup. Because Squid and DansGuardian also run at startup, you will constantly have web filtering on your machine - reguardless of what other users may attempt to do. Of course, this is all based on the assumption that the other users don't have the root password!

Step 8: Customizing Rejection Page (Optional).

You can easily customize the rejection page of DansGuardian simply by replacing the one they have by default. The default location of this file is in the following (assuming English version was downloaded):

"/etc/dansguardian/languages/ukenglish/template.html"

You can replace it with something like my example page.

One neat thing about DansGuardian is you can place variable names within your HTML page, and when DansGuardian retrieves the HTML page, it replaces those variables with actual text. Let's take a look at what variables we have:

       

-URL-                                gives the URL the user was trying to access

-REASONGIVEN-            gives the "nice" reason (i.e. not quoting the banned phrase)

REASONLOGGED-         gives the reason that gets logged including full details

-USER-                              gives the username (if known)

-IP-                                     gives the originating IP address   

-HOST-                               gives the originating host name (if known)    

-RAWFILTERGROUP-     gives the group number

-FILTERGROUP-              gives the group name

-SERVERIP-                      gives the IP address on which the filter is running

-BYPASS-                          gives a URL which allows temporary bypass of denied page

-CATEGORIES-                gives the categories assigned to the banned content

Note that all of these "variables" have dashes on either side of them "-variable-". This tells DansGuardian that it's a variable and not plain text.
With these in hand, you can whip up a pretty slick-looking denail page if you know a little HTML. Otherwise you can use the default page, or my page and save it as "template.html" in the language directory. 

Step 9: View Log of Denials (Optional)

So you have all of this setup and everything, but how to we see who was denied what and when? Of course DansGuardian logs everything, and does a pretty good job of it too. And of course, you can specify where it writes the logs to in the configuration file.
By default, it keeps the log file here:

"/var/log/dansguardian/access.log"

To change this, open up the "dansguardian.conf" file as root and find where it says "loglocation = ", and specify where you want it. You even have different options for log file format! You can leave it as default, or you can change that as well (search for "logfileformat = ") and changing it to something like option 4. You can also change WHAT it logs. By default, it logs everything. This can be space-consuming, and make it harder to see what denials have occured. I changed my setting ("loglevel = ") to 1 - which is "just denied".

Once we have this updated, we need to reload the configuration files. We can do this by running the following command:

username@localhost:$ sudo dansguardian -r

 

Step 10: Allowing Blocked Sites/Denying Allowed Sites (Optional)

Ok so DansGuardian does a really good job of blocking sites, sometimes TOO good. We may want to access sites that are blocked, but for reasons that aren't that relavent to us.

DansGuardian has a bunch of various lists that we can use. Depending on what we want to do, there will be a list for it. Keep in mind that in order to modify all lists, we need to have root access (makes sense, huh?)

Here are the lists and what they are used for:

Banned Lists (/etc/dansguardian/lists/)

bannedextensionlist          denies any file with an extension in this list

bannediplist                       denies access to any IP address in this list

bannedmimetypelist          denies access to certain MIME types

bannedphraselist                denies access to a page that contains any phrase in this list

bannedregexpheaderlist     bands certain outgoing HTTP headers in list

bannedregexpurllist           bands regular expression URLs

bannedsitelist                     denies access to particular websites - includes some in blacklists folder

bannedurllist                     denies access to certain pages of a website - such as   

                                             "example.com/badpart/"    where        

                                               "example.com" would still be allowed

Exception Lists (/etc/dansguardian/lists/) 

exceptionextensionlist               allows any file with an extension in this list

exceptioniplist                           allows access to any IP address in this list

exceptionmimetypelist              allows access to certain MIME types

exceptionphraselist                    allows access to a page that contains any phrase in this list

exceptionregexpheaderlist         allows certain outgoing HTTP headers in list

exceptionregexpurllist               allows regular expression URLs

exceptionsitelist                         allows access to particular websites

exceptionurllist                          allows access to certain pages of a website - such as    

                                                   "bad.com/goodpart/"

                                                    where the rest of "bad.com" would still be blocked

                  

There are a handful of more lists in this directory that you may explore for yourself...............

   

 

 

 

 

 

 

 

     

          

                     

          

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Static Routes - Basic Setup

This lesson will illustrate the necessary steps to configure a simple static route on a typical Endian appliance. This will allow us to configure routing (i.e. communication capabilities) between the Endian and other networks that may not live directly on the Endian but on an external router or through a VPN tunnel.

Configuration Example

In this simple example, we'll configure a static route to an external network that lives on a router within the Green network. As you can see the router has one interface in the Green network and another interface for the external network that lives outside the Green network.

Create Static Route

Now we'll add a static route in the Endian under Network > Routing > Static Routing. Here we'll leave the "Source Network" field blank to mean ANY and then supply the destination network (external) and the route gateway IP which is the Green IP of the external router.
Click Add Route and then Apply to complete the process.

Verify Route

Now you can verify the route works properly by pinging a device in the remote network (10.45.1.0/24) and vice versa. Be sure to check the appropriate firewall rules are in place to allow the desired level of communication between the two networks.

Policy Routes - Basic Outbound Load Sharing:

This lesson will illustrate the necessary steps to configure a simple static route on a typical Endian appliance. This will allow us to configure routing (i.e. communication capabilities) between the Endian and other networks that may not live directly on the Endian but on an external router or through a VPN tunnel.

Configuration Example

In this simple example, we'll configure policy routing so that the Green (LAN) and Orange (DMZ) use the primary Internet connection while the Blue (WIFI) network uses a secondary Internet connection This will ensure our Blue (WIFI) users will not take bandwidth from the Green / Orange network traffic.

Create Policy Route #1 (Green / Orange)

Now we'll add a new policy route in the Endian under Network > Routing > Policy Routing. Select the GREEN and ORANGE zones as the Source and we'll leave the Destination network / ports blank ( = ANY) and then configure the route to use the Main Uplink. Notice we've checked the "Use backuplink if uplink fails" as this will provide a failover policy to the backup uplink in the event our main uplink goes down.
Click Create Rule and then Apply to complete the process.

Create Policy Route #2 (Blue)

Now we'll add our second policy route in the Endian for the Blue network. Select the BLUE zones as the Source and we'll leave the Destination network / ports blank ( = ANY) and then configure the route to use the Backup Uplink. Notice we've checked the "Use backuplink if uplink fails" as this will provide a failover policy to the main uplink in the event our backup uplink goes down.
Click Create Rule and then Apply to complete the process.

Verify Routes

Now you can verify the route works properly by performing a traceroute from any workstation in the Green / Orange network to the Internet and you should see the packet leave out of the Main uplink. Then you can do another traceroute from any device in the Blue network to the Internet and you should see that packet leave out of the Backup uplink.

Endain SNAT (Source NAT) - Basic Setup

This lesson will illustrate the necessary steps to configure a very simple Source NAT (SNAT) on a typical Endian appliance. The primary purpose of Source NAT is to take an internal application (IP and port) and manipulate which external IP and/or port is masqueraded to the Internet. By default, the Endian will masquerade all outbound connections to the primary Red interface IP address so you need SNAT in instances where you don't want this to occur.

Configuration Example

In this simple example, we'll setup a Source NAT (SNAT) for an mail server in the Orange (DMZ) network. We'll use one of our (non-primary) statically assigned public IP addresses as our masqueraded IP.

Create SNAT Rule

We'll begin by creating a new SNAT rule at which point we can configure all the necessary fields to correspond to our example network. Notice in the "NAT to source address" dropdown we specifically select our new public IP address to use for masquerading for this rule.
Once done, click the Create Rule button.

Apply the Rule

Once your rule is created, you must apply the rule to the device by clicking the Apply button (shown above).

Verify the Rule

Now that our rule has been applied successfully, we can test that our mail server is being masqueraded to the correct IP address.

Note

You can test a Source NAT rule from any server with a browser by using a site like WhatIsMyIP which will tell you what which public IP you're coming from.

Endian Network Configuration Wizard

This lesson will illustrate the steps necessary to configure all of the network interfaces of the Endian appliance after the initial configuration.

Endian Network Architecture

Before we begin the configuration process, please take a moment to familiarize yourself with the color-coded network zones available in the Endian platform and how they are intended to be used (pictured above).

Choose RED (WAN) Interface Type

The first step is to choose the connection type of your primary WAN interface. In most cases the proper selection is either Ethernet Static or DHCP unless you require one of the other specific connection types. Click the Forward button to continue.

Note

If you require a configuration where you will not need a Red (WAN) interface, you can select Gateway as the connection type and this will allow you to deploy the Endian in a semi-transparent configuration. This option will allow you to deploy the Endian into a network using the Green (LAN) interface as your primary network connection and using an existing gateway that lives within the Green network.

Add Network Zones

The next option will allow you to select any additional network zone you wish to have configured on your Endian appliance. The available options will depend on the total number of available Ethernet NIC's on the Endian device. Your options could include adding the Blue zone (Wifi) or Orange zone (DMZ) or both. Click the Forward button to continue.

Configure Network Zones

The next step involves configuring the actual IP address you want assigned to the Endian device for each existing network zone. The default Green IP is provided for you but you can use any IP address and subnet you wish. You are also allowed to add additional networks that may co-exist within each single network zone. An example of where this might be used is if you host multiple internal subnets that all need to exist within the same network segment (within one zone).
The next item is a graphical representation of the available physical network interfaces and which zone they are mapped to. You may check or uncheck one or more network interfaces to belong to a network zone (at least one is required) and you may have more than one physical interface per zone; however, you cannot have one physical inteface belong to more than one zone. A network zone with multiple network interfaces will act as a bridge and mimic the behavior of a switch though using an actual physical switch is recommended where performance and efficiency are desired. Next to each network interface port is (1) the link status which indicates if there is a device actually connected to the port, (2) NIC device driver description, (3) network MAC address, and (4) the operating system physical device name.
The last two items are the host and domain name you want assigned to the Endian device itself. Click the Forward button to continue.

Note

It is suggested to follow the standards described in RFC 1918 and use only IP addresses contained in the networks reserved for private use by the Internet Assigning Numbers Authority (IANA):

• 10.0.0.0 - 10.255.255.255 (10.0.0.0/8), 16,777,216 addresses
• 172.16.0.0 - 172.31.255.255 (172.16.0.0/12), 1,048,576 addresses
• 192.168.0.0 - 192.168.255.255 ( 192.168.0.0/16), 65,536 addresses

The first and the last IP address of a network segment are the network address and the broadcast address respectively and must not be assigned to any device.

Configure Red (WAN) Interface

Now you can configure the Red (WAN) interface according to your ISP connection type (as selected during Step 1). The configuration is identical to the previous step where you must configure the IP, subnet, and gateway (if necessary), select the appropriate physical inteface to use for the Red (WAN) connection, and fill out any other ISP connection specific fields.
If you have multiple public IP addresses assigned, you may enter each IP in the "Add additional addresses" field (1). You should list each individual IP in either IP/Netmask or IP/CIDR format with one entry per line (Example: 29.150.10.5/24, 29.150.10.6/24, ...)
The options for MTU is to manually enter a custom value for interface MTU size and is not recommended unless instructed by your ISP. The option to Spoof MAC address with is really only useful for situations where your ISP modem has a "sticky" connection and requires that your Internet MAC address always stay the same. This option would allow you to configure the Endian to "forge" it's Red interface MAC address so you do not lose your ISP connection.
Click the Forward button to continue.

Configure DNS

This option is only required if you are not using some form of DHCP for your Red (WAN) connection. You should fill in your ISP-provided or preferred public DNS servers in these fields. Click the Forward button to continue.

Setup Email Information (Optional)

Here you can provide the administrator (recipient) email account along with the Endian (sender) address you want emails from the Endian firewall to use. Also you may specify the address of an email smarthost should you require one. Click the Forward button to continue.

Apply Configuration

The last step is to apply the configuration to the device. Keep in mind, the changes you made may take up to 20 seconds to be fully applied to the device and for dependent services to be restarted so this may impact any internal device(s) ability to access the device or pass traffic through it. You must access the administration interface of the Endian device using the new IP settings either manually or using the link in the Web UI provided.

Endian Web Proxy - Basic Setup (Transparent)

This lesson will illustrate the necessary steps to configure a very simple transparent web proxy on a typical Endian appliance. A transparent web proxy is one that requires no client-side changes to operate effectively (all traffic is tranparently redirected). The primary purpose of the web proxy is to (1) allow for a simple method to filter web traffic to appropriate levels for business and (2) provide accountability for user web traffic.

Enable the Web Proxy

 

The first step is to enable the web proxy by clicking the gray button (which will turn green when enabled). Once this is done, we can configure the networks we want to be filtered transparently (using Green only in this example).

Configure the Log Settings

Since we want to have all web access (allowed and blocked) logged for review purposes, we're going to enable the appropriate logging options.
Click Save and then Apply the changes to proceed.

Configure the Content Filter Profile (Default)

In this example, we're only going to configure web filtering by URL Blacklist (only) for ease and administration purposes. The first thing we'll do is ensure our HTTP antivirus is enabled by checking the appropriate box. You can select the whole category to block by clicking the green arrow or, alternatively, you can drop down the subcategories and select those individually in order to block some and not others. You can also attach custom white- or blacklists to this profile as well.
Click Update Profile and then Apply the changes to proceed.

Note

When you enable web filtering by phrases (Content Filtering) this will block those categories of phrases for content "inside the page" which will result in a more aggressive blocking strategy with potentially a higher false positive rate.

Configure the Access Policy

The last step is to create an access policy which will map the content filtering profile based on a specific network configuration. In the example above, we're creating a simple policy for the Green zone (entire network) that is using the content filtering profile (default) that we just configured in the previous step.
Click Create Profile and then Apply the changes to finalize the configuration.

Test the Web Proxy

You can test your configuration now by browsing the Internet from the Green network and you should see a block page on sites that match the categories selected.

Verify Logging

How to Install Endian Software Firewall

Installation of Endian Community Firewall 

Endian Firewall Community Edition
I’ve been meaning to set up my personal router/firewall for some time now. Currently I’m using your standard off-the-shelf Linksys router to run my network. This being my first home setup I don’t have anything else to compare it to other than a few commercial routers and firewalls. The following is a how-to for the installation and configuration of Endian Firewall (Community Edition), along with explanations on what some setting mean.
Installation:
Boot up is your typical Linux distribution. Once booted up the first screen you come to is the language selection. There are only three languages available: Deutsch, English, Italiano. I’ve selected English.

 Next is confirmation that you want to install the software and where it will be installed. Say “yes” to continue.

 Do you want to enable console over serial. This setting is if you want to be able to locally administer the firewall. This is ideal if you forgot your password and want to reset the password.

Next is the Appliance Installer, this may take a while depending on how fast the transfer is between the installation media and the hard drive.

Configure the GREEN interface. This is the interface that is on your internal network and allows you access to configure the firewall.

Installation is finished. It is important to write down the following address: http://efw-community and https://efw-community:10443 . After selecting “OK” the system will restart.

Boot Progress

This is the menu you are presented with after the system is completely booted. At this point you can remove your keyboard, mouse and monitor. You will be able to do the rest of the configuration remotely. You will configure the passwords through the GUI interface.

To connect to the GUI I started up FireFox (any browser should do) and typed in the address of the machine using https and was presented with this. Select “I understand the Risks.”


Click on “Confirm Security Exception.”

You are now at the Welcome Screen of your firewall.