June 8th, 2013 By arlinton Categories: System Administration

Red Hat has, for quite some time discontinued PAE support in their kernels. For those who would like to use Red Hat Enterprise Linux 6, CentOS 6, or any derivatives – I’ve setup a repo here:

I’ll try to keep up with official releases but as with anything, I cannot guarantee compatibility. It’s provided AS IS :)

Happy Hacking!

October 13th, 2012 By admin Categories: Rants
15:56 < boris[1]> framework programmer is like
15:57 < boris[1]> oxymoronic and/or deceptive
15:57 < boris[1]> it's like calling yourself a lego civil engineer to people
                  who don't know what legos are
April 19th, 2012 By arlinton Categories: System Administration

My friend Meethune was looking for a feature to simplify his google searches, he wrote:

I was looking for a “select -> right click -> search google” feature for gnome-terminal. I didn’t find anything however I did find this little gem. Bind the following one liner to a keyboard shortcut in whatever window manager you use.

 sh -c 'xsel | tr " " "+" | xargs -I %s xdg-open "http://www.google.com/search?q=%s"'

Now I can select something in my terminal window and press “ctrl+shift+g” and a new tab opens in my browser with a query on Google of the selection.

I thought this was cool and wanted to share. I’ve currently use this on my Fedora desktop machines and can select text from any application (I’ve tested with gnome-terminal, xterm, firefox) and can initiate a search on google with a single key combination.

Here’s some info from the rpm package that allows this interaction to be successful. I hope this helps you implement or set this up on the operating system of your choosing:

$ rpm -qi xdg-utils-1.1.0-0.10.20111207.fc16.noarch
Name        : xdg-utils
Version     : 1.1.0
Release     : 0.10.20111207.fc16
Architecture: noarch
Install Date: Mon 13 Feb 2012 03:43:39 PM EST
Group       : System Environment/Base
Size        : 273423
License     : MIT
Signature   : RSA/SHA256, Fri 09 Dec 2011 08:07:36 AM EST, Key ID 067f00b6a82ba4b7
Source RPM  : xdg-utils-1.1.0-0.10.20111207.fc16.src.rpm
Build Date  : Wed 07 Dec 2011 01:45:56 PM EST
Build Host  : x86-14.phx2.fedoraproject.org
Relocations : (not relocatable)
Packager    : Fedora Project
Vendor      : Fedora Project
URL         : http://portland.freedesktop.org/
Summary     : Basic desktop integration functions
Description :
The xdg-utils package is a set of simple scripts that provide basic
desktop integration functions for any Free Desktop, such as Linux.
They are intended to provide a set of defacto standards.
This means that:
*  Third party software developers can rely on these xdg-utils
   for all of their simple integration needs.
*  Developers of desktop environments can make sure that their
   environments are well supported
*  Distribution vendors can provide custom versions of these utilities
The following scripts are provided at this time:
* xdg-desktop-icon      Install icons to the desktop
* xdg-desktop-menu      Install desktop menu items
* xdg-email             Send mail using the user's preferred e-mail composer
* xdg-icon-resource     Install icon resources
* xdg-mime              Query information about file type handling and
                        install descriptions for new file types
* xdg-open              Open a file or URL in the user's preferred application
* xdg-screensaver       Control the screensaver
* xdg-settings          Get various settings from the desktop environment
April 10th, 2012 By randy Categories: Rants

I remember taking an undergraduate, management science, class where in an effort to gauge how much of the class needed an Excel review, the professor asked the question, “How many of you consider yourselves Excel experts?” I couldn’t help but think about the subjective nature of the question. Nearly half the class considered themselves experts; yet, as the class progressed, it became evident that overall, the so-called “experts” were struggling. I’m sure they all thought, with their limited understanding of the breadth of the software, “I know Windows;” “I’ve used Excel;” “yeah I know that…”

I’ve experienced the same phenomenon when interviewing and working with programmers and ‘nix admins. Almost always, the biggest technical dim-wits are those who are quick to say something like, “oh, I know PHP!” or “yeah, I know Linux.” Perhaps they have done a little coding or have worked with Linux before, but their limited exposure to the depth of the field and other competent people, have led them to believe that they are in some way good at what they do.

A serious programmer or sys-admin understands that there is a philosophy driving what they do and how they set things up. Anyone can code and anyone can Google their way through a Linux configuration, but not everyone understands good programming methodology and not everyone can think through a complicated algorithm to solve a difficult problem. Good programmers are confident, but they are also humble because they are aware of the breadth and depth of their field.

The same is true of non-technical managers and small business owners in the position of hiring technical staff. I later discovered that my management science professor was a dim-wit too; the early questions asked to gauge the competency level of the class revealed the fact that he himself didn’t understand the breadth and depth of the topic either. For the entirety of the semester, the class degraded into an Excel tutorial (this was an accredited institution). It wasn’t the first semester-long, waste of time I experienced throughout my academic career. Bored out of my mind, I even tried importing the class material into MySQL and writing Perl scripts to perform the same functions Excel did. I thought my professor would be impressed, but he insisted I use Excel; that’s what he wanted students to walk away from the class knowing. He did not recognize the value in actually understanding the algorithms, or being able to implement the formulas used under the hood.

This is why managers who don’t understand technology go from one bad hire to another. This is why managers and small business owners make the mistake of outsourcing their project to India; it’s cheap, and after-all “a coder is a coder is a programmer,” right? This is why the information security field is in shambles, basic web applications keel under the slightest load- requiring a dozen application servers, and script-kiddie hacker groups are empowered. It’s because for the past decade, our industry has made their killing by writing tools and software that enable dim-wits to say, “yeah, I know that…”

April 6th, 2012 By randy Categories: System Administration

I was asked by an academic client to install Boinc Server on FreeBSD. BOINC is a software platform for volunteer computing and desktop Grid computing. FreeBSD has a boinc-client port, but not a port for the server, so it must be compiled manually. Following the instructions located on the Boinc wiki for a FreeBSD installation would result in some errors.

Here is how to get started in FreeBSD:

Follow the normal prerequisite / prep instructions found here: http://boinc.berkeley.edu/trac/wiki/ServerIntro. All the prerequisites can be found in FreeBSD ports, install them including Python and /usr/ports/databases/py-MySQLdb . I installed this in a Jail, stripped down Apache to the bare minimum and set this up for my client so that Apache would automatically include his httpd.project.conf files after running make_project.

The following are some of the things I did differently compared to the Linux install:

svn co http://boinc.berkeley.edu/svn/branches/server_stable boinc
cd boinc

Without editing the configure script you’ll get the following error while making:

/usr/bin/ld: cannot find -ldl

To fix this, edit the ./configure script:

vi configure

and remove all occurances of “-ldl “:

:%s/-ldl //g

-ldl is not required in FreeBSD, it is available in libc.


./configure --disable-client --disable-manager --with-boinc-platform=x86_64-pc-freebsd

You may need to replace x86_64-pc-freebsd depending on your hardware. Check the list here: http://boinc.berkeley.edu/trac/wiki/BoincPlatforms

If you ran `make` instead of GNU make`gmake` you’d get these errors:

“Makefile”, line 19: Missing dependency operator
“Makefile”, line 23: Need an operator
“Makefile”, line 27: Need an operator
make: fatal errors encountered — cannot continue



That’s it! Follow the rest of the instructions on the Boinc wiki.

April 2nd, 2012 By arlinton Categories: System Administration

What is iSCSI?

The SCSI protocol implemented over TCP/IP. It allows for iSCSI targets (servers) to export block/storage devices over the network to iSCSI initiators (client). For more technical details please checkout:

Some RFCs:

What is gPXE?

gPXE is an open-source bootloader that can run from the Pre-eXecution Environment (PXE) as well as be chainloaded from other bootloaders. PXE usually works well on good network boot roms (caution to those who use certain realtek chips). Once loaded gPXE allows the loading of various types of boot code over various protocols. Protocols such as HTTP, AoE (ATA over Ethernet), iSCSI, and more are supported by gPXE. We will be using gPXE to connect to our iSCSI target and use it as a boot device on our iSCSI initiator (client). For a better overview of gPXE, you can read the wikipedia article on gPXE, http://en.wikipedia.org/wiki/GPXE . For more details the gPXE wiki is the best source, http://etherboot.org/wiki/start .

Setting up ISC DHCPd for the iSCSI boot
Before you begin you should know how to configure DHCPd:

The following code will be found in the host section of the dhcpd.conf. It has a conditional that detects if the dhcp client request is coming from gpxe or not. This is required to prevent an infinite boot loop. When the boot rom from the NIC does it’s dhcp client request the options held with the ‘else’ clause will be run. The ‘else’ clause loads the undionly.kpxe file which is obtained from the gPXE package from your respective distribution. Upon the successful load of the undionly.kpxe code into memory, gPXE is executed and then it issues its own dhcp client request. The conditional confirming the existence of ‘user-class and option user-class = “gPXE”‘ sets the root-path and gpxe.keep-san options. These options allow for the binding of the iSCSI target as a boot device.

Keep in mind that the default gateway is configured in windows as the iSCSI target. If you have multiple NICs I would recommend you use an alternate NIC to access your network and the internet. If you’re on a machine with a single NIC setting up ipforwarding and firewall rules are trivial. This must be done on the iSCSI target.

hardware ethernet ff:ff:ff:ff:ff:ff;
if exists user-class and option user-class = "gPXE" {
  filename "";
  option root-path "iscsi:";
  option gpxe.keep-san 1;
  option routers;
} else {
  filename "/undionly.kpxe";
  option routers;

The following options may be required. I believe they’re optional but if you have any issues put this in the global section of the dhcpd.conf:

# gPXE-specific encapsulated options
option space gpxe;
option gpxe-encap-opts code 175 = encapsulate gpxe;
option gpxe.priority code 1 = signed integer 8;
option gpxe.keep-san code 8 = unsigned integer 8;
option gpxe.no-pxedhcp code 176 = unsigned integer 8;
option gpxe.bus-id code 177 = string;
option gpxe.bios-drive code 189 = unsigned integer 8;
option gpxe.username code 190 = string;
option gpxe.password code 191 = string;
option gpxe.reverse-username code 192 = string;
option gpxe.reverse-password code 193 = string;
option gpxe.version code 235 = string;

# Other options that may be useful
option iscsi-initiator-iqn code 203 = string;

Setting up tftpd
If you need to know how to setup tftpd see this:

This is a matter of getting the gPXE bits into your tftpboot root directory. In Fedora 15 the ‘gpxe-bootimgs’ package has all of the binary images. You can download and compile the source from http://etherboot.org/wiki/download .You will need to copy the ‘undionly.kpxe’ to the tftpboot directory.

Setting up a iSCSI Target

I’ve used the netbsd-iscsi target package. The Fedora 15 package name is ‘netbsd-iscsi’ the source is available here, http://pkgsrc.se/devel/netbsd-iscsi-target . Here is a howto on configuring the iSCSI Target: ftp://ftp.netbsd.org/pub/NetBSD/misc/agc/HOWTO-iSCSI-target.txt . Here is a sample of my config, I am using LVM and the volume size is 650GB:

# Simple file showing 1 extent, mapped straight into 1 target

# extents       file                                                   start   length
extent0         /dev/volume_group/logical_volume             0     655360MB

# target        flags   storage         netmask
target0         rw      extent0

Installing Windows 7
This is an excellent guide on installing Windows over gPXE:

On the machine that will be using the iSCSI have your respective windows installation media. It’s important that you set the CD device to boot after the NIC (remember gPXE sets up the install block device). After gPXE fails to boot your iSCSI target the CD should boot. Proceed with the installation as normal. It’s important that the windows installation has drivers for your NIC or else nothing will work. The windows install should detect the iSCSI drive and you should be able to install to it. At any point during the installation press, Shift+F10, and check the network & status.

Proof of Concept video
Check it out in action, https://www.youtube.com/watch?v=Cnb-Mcq303Q

March 29th, 2012 By randy Categories: Security, System Administration

The solution to the age old problem of locking SFTP users into their home directory is setting up a chroot environment. This normally requires that you copy the necessary binaries and libraries so that your jailed users can make use of the allowed tools for file transfer.

As of OpenSSH 4.9p1, things have gotten a bit easier. OpenSSH has two features that make the task of locking users into their home directories a piece of cake. They are:

  1. A built in SFTP subsystem.
    With a built in SFTP subsystem, you no longer need binaries and the required libraries to provide the services necessary in a chroot environment. OpenSSH provides an internal SFTP subsystem.
  2. The Match keyword.
    This allows you to target specific users or groups in the sshd_config file and specify settings particular to them, like a chroot option and ForceCommand internal-sftp.

Getting it working is simple.

  1. Add a group called sftponly and add the users who you’d like to lock into their home directories to that group.
  2. Edit your sshd_config file (/etc/ssh/sshd_config if you’re on FreeBSD) and add the following to the bottom:
    Match Group sftponly
            X11Forwarding no
            AllowTcpForwarding no
            ForceCommand internal-sftp
            ChrootDirectory %h
  3. That’s it, HUP sshd (/etc/rc.d/sshd restart if you’re running FreeBSD) and test it out.
January 18th, 2012 By randy Categories: System Administration

If you’re running a local mail client to send mail from your web application, you’ve probably already spent hours upon hours wondering why mail always ends up in the recipients spam/junk folder. No matter what combination of custom headers you pass to PHP’s mail function- yahoo, hotmail and maybe even gmail still marks it as spam. Have no fear. The solution is quite simple.

There are two options that are very similar. They are DKIM and DomainKeys. When I was first experimenting, I had installed DomainKeys only to realize it worked with gmail and yahoo, but not hotmail. After some reading I found that DKIMProxy implemented both DKIM and DomainKeys, so I ditched DomainKeys and went with DKIMProxy. This covers all the major email services and ISPs. Switching was easy, because they work exactly the same with only a few minor differences in syntax.

Read more about how those protocols work here:

Essentially, DKIM is a solution for email providers to verify that mail being sent from a particular domain is authorized. The way it works is that the domain owner inserts a TXT record into DNS for that domain. This record contains a public key. Each e-mail that gets sent must be signed using the private key, the signature gets placed into the header of the email. Email providers can then verify the authenticity of each email that is sent using classic pgp-like signature techniques. DKIMProxy is a service that runs on your server, works with your already existing mail server (sendmail, postfix, qmail, etc..) and can automatically sign your mail pieces.

Below I highlight the steps for setting up PostFix + DKIMProxy on FreeBSD to send mail from the local machine ONLY. This will be setup on an application server whose sole purpose is sending mail through the local relay, not receiving or relaying email from other machines. If you’ve already setup PostFix for wider purposes, you’re probably already pretty comfortable with it; just use the same techniques to specify the content_filter for any other listening service and skip my PostFix write-up. There will be some differences if you’re running this instance of FreeBSD in a jail (as I do), however I will document those differences and the security precautions you’ll need to take as well.

If you’re not using FreeBSD, you’ll have to substitute my use of ports with your distribution’s package management command. The location of your config files may also vary, as well as the start-up scripts; otherwise, it’s all the same.

Installing / Configuring Postfix:

cd /usr/ports/mail/postfix
make & make install

Choose yes to modify mailer.conf

Stop sendmail if it’s running:

cd /etc/mail
make stop

Edit your rc.conf file to turn off sendmail and enable postfix. Add the following:


If we only want to send local mail we can comment out the following line in /usr/local/etc/postfix/master.cf :

#smtp      inet  n       -       n       -       -       smtpd

This is beneficial if we are running postfix in a jail that doesn’t have access to a loopback device. If you were running in a jail, but wanted to run an SMTP server as well, you could specify an IP address to listen on by prepending the line with the ip address as follows:      inet  n       -       n       -       -       smtpd

It’s fine to comment out though.

Edit your /usr/local/etc/main.cf file according to your needs. Since I am just using this to send mail from my domain, but am using Google Apps to receive mail I do the following in main.cf :

myhostname = mydomain.com
inet_interfaces = loopback-only
mydestination = localhost

The default main.cf file has those specific parameters documented for your reference (RTFM).

Now start PostFix :

/usr/local/etc/rc.d/postfix start

Check /var/log/maillog for errors. If there is an error about reading /etc/aliases.db, you probably just need to generate it. Run the `newaliases` command and it should solve your problems.

Test sending mail from the local machine to make sure PostFix works. You can either use a PHP script or run sendmail manually. If it doesn’t show in your inbox, check the spam folder and /var/log/maillog .

Installing DKIMProxy:
You need to install the following Perl modules… CPAN doesn’t work well from a jail console, and if you’re like me and don’t want to install SSH or Tmux to fix your tty issues in jails, the following commands will install the needed Perl modules from the command line and works no matter what distribution you use:

perl -MCPAN -e'CPAN::Shell->install("Crypt::OpenSSL::RSA")'
perl -MCPAN -e'CPAN::Shell->install("Digest::SHA")'
perl -MCPAN -e'CPAN::Shell->install("Digest::SHA1")'
perl -MCPAN -e'CPAN::Shell->install("Mail::Address")'
perl -MCPAN -e'CPAN::Shell->install("MIME::Base64")'
perl -MCPAN -e'CPAN::Shell->install("Net::DNS")'
perl -MCPAN -e'CPAN::Shell->install("Net::Server")'
perl -MCPAN -e'CPAN::Shell->install("Test::More")'
perl -MCPAN -e'CPAN::Shell->install("Error")'
perl -MCPAN -e'CPAN::Shell->install("Text::Wrap")'
perl -MCPAN -e'CPAN::Shell->install("Mail::Address")'
perl -MCPAN -e'CPAN::Shell->install("Mail::DomainKeys")'
perl -MCPAN -e'CPAN::Shell->install("Mail::DKIM")'
cd /usr/ports/mail/dkimproxy
make && make install

Add the following to your rc.conf :


We won’t configure “in” because we are only interested in sending mail out right now.

Generate some keys:

cd /usr/local/etc/
openssl genrsa -out privatedkim.key 1024
openssl rsa -in privatedkim.key -pubout -out publicdkim.key

My /usr/local/etc/dkimproxy_out.conf file looks like this:

# specify what address/port DKIMproxy should listen on
# specify what address/port DKIMproxy forwards mail to
# specify what domains DKIMproxy can sign for (comma-separated, no spaces)
domain    mydomain.com
# specify what signatures to add
signature dkim(c=relaxed)
signature domainkeys(c=nofws)
# specify location of the private key
keyfile   /usr/local/etc/privatedkim.key
# specify the selector (i.e. the name of the key record put in DNS)
selector  selector1

You can just copy /usr/local/etc/dkimproxy_out.conf.example to /usr/local/etc/dkimproxy_out.conf and make the appropriate changes. The selector parameter is what we’ll specify later in DNS.

Quick Explanation
dkimproxy is going to listen on the ip address / port (10027) we’ve specified for mail and it’s going to proxy smtp traffic to the ip address / port (10028) we’ve specified, with the appropriate headers attached.

Security concern: If you’re using a jail and/or specified a public facing IP address to listen on, you’ve just opened up relaying for the world even if relaying is turned off in postfix. The reason is because you’ve opened a proxy that will listen publicly and forward internally to postfix. SOOO… If you’ve specified a public facing IP address, either change it to an internal-only or firewall it off.

Back to PostFix Configuration:

Now we need to configure postfix to use the DKIM proxy as a content_filter (sending mail out to it on port 10027) and then receiving it back on 10028.

We’ll be editing /usr/local/etc/postfix/master.cf and since we are only concerned with sending local mail, you can edit the pickup line to reflect the following:

pickup    fifo  n       -       n       60      1       pickup
        -o      content_filter=dksign:[]:10027

and add the following somewhere below:

dksign  unix    -       -       n       -       10      smtp
    -o smtp_send_xforward_command=yes
    -o smtp_discard_ehlo_keywords=8bitmime inet  n  -      n       -       10      smtpd
    -o content_filter=
    -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
    -o smtpd_helo_restrictions=
    -o smtpd_client_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o mynetworks=
    -o smtpd_authorized_xforward_hosts=,mydomain.com,localhost

Now restart postfix and start dkimproxy:

/usr/local/etc/rc.d/postfix reload
/usr/local/etc/rc.d/dkimproxy_out start

Check netstat -na to make sure you’ve got services listening on both 10027 and 10028 . If not, check /var/log/maillog for errors.

Your DNS Records:

You’ll need to add three DNS records:

@       3600    IN      TXT     "v=spf1 a mx ~all"
_domainkey      3600    IN      TXT     "t=y; o=~;"
selector1._domainkey    3600    IN      TXT     "g=*; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDhrfYr2KLFiU7Zo6H06LlhFBEpif/Tb7oBJvKdEIm1uED9FqJump/q6RSt3Yw1iuM3iBaQcPohGbdoGiuaJGOUWMOblsSXkAOWxl4lbI5UQ6zCTBpVdLVDVWJ0E3UW1YJs1crSBdmG9G3WghrvIRkHzxfDMqndIV5gliYt+nmqXQIDAQAB;"

Once your tests are complete you can remove t=y; More info

Where you see p=MIGfMA…, you need to concatenate the publicdkim.key file into one line without the header/footer. So for example my publicdkim.key file originally looks like this:

-----END PUBLIC KEY-----

You can see how my p=… is related.

If you are using a GUI DNS editor tool, you’ll have to figure out how to place these into DNS using that tool.

Now to test it out:

Assuming DNS has propagated, you can try sending a couple e-mails. tail -f /var/log/maillog while you do and you should be able to see the proxy in action. If you send mail to hotmail or google, you can view the message source and see the DKIM and DomainKeys header information as well.

You can also use the following tools which have been extraordinarily helpful to me when setting this up:

  • Send an e-mail from your server to: check-auth@verifier.port25.com

It will respond with some helpful diagnostic information, but make sure you can receive an e-mail response to the reply-to field.

Submit your SPF record to evil Microsoft
If everything passes, you should submit your spf record to Microsoft. They are sort of like the really strong dumb kid on the block who can steal your candy, but can’t perform a DNS lookup on their own. Without submitting to Microsoft your mail will end up in the hotmail / live.com SPAM folder with a senderid temperror. You can sign up for a hotmail account and view full message source to see where it fails, then burn your monitor, keyboard and clothes for having signed up for a Microsoft product.

DomainKeys DNS Testing Tools:

That should get you on your way. I’ve pretty much covered every hurdle I’ve come across when setting this up. You’ll find plenty others who have similar, but slightly different setups by doing a cursory Google search.

December 30th, 2011 By randy Categories: @Home

In a previous post titled, “Home Wireless Router” I walked through my custom built FreeBSD, wireless router at home. In this post, we’ll add web based authentication for guests. Essentially, when an unknown users connects to our network and browses the web, we’ll display our own website with a note letting them know we’re watching. They’ll have to agree to behave before they can actually browse the internet on port 80.

This will build on the previous “Home Wireless Router” post; start there first and make the appropriate changes noted below.

Using dhcpd to setup a split network:

We’ll make the network for trusted users and the network for our untrusted guests. Perhaps, in the future we can even throttle their bandwidth!

Here is what my /usr/local/etc/dhcpd.conf file now looks like.

subnet netmask {
  pool {
          option domain-name-servers;
          deny unknown-clients;
  pool {
        option domain-name-servers;
        allow unknown-clients;
  option domain-name "CANAAN";
  option routers;
  option broadcast-address;
  default-lease-time 600;
  max-lease-time 7200;
group trusted {
        host phone { hardware ethernet 00:0e:08:d5:c9:af;}
        host dell { hardware ethernet 00:11:43:75:0d:89; }
        host android1 { hardware ethernet f8:7b:7a:f0:c8:4b; }
        host android2 { hardware ethernet f8:7b:7a:f0:71:8b; }
        host ipad { hardware ethernet 10:93:e9:41:8f:26; }
        host macmini { hardware ethernet 68:a8:6d:59:9f:c9; }

I’m embarrassed, I really didn’t pay for all of those Mac products. If it isn’t obvious, you’ll need to replace the hosts and their hardware Ethernet addresses with the actual hardware Ethernet addresses of your trusted machines.

Packet Filter

Our packet filter configuration (/etc/pf.conf) is going to look a lot different than it did when we only had a wireless nat, but it should be obvious what the differences do:

allowed_out="{http, https, ssh, domain}"
table <goodboys> {}
set block-policy return
nat on $ext_if from to any -> ($ext_if)
nat on $ext_if from to any port $allowed_out ->($ext_if)
no rdr on $int_if proto tcp from <goodboys> to any port 80
rdr on $int_if proto tcp from to any port 80 -> port 80

In my prior wireless post, I stripped out all my fancy variables. Since we are now referring to the internal and external interfaces quite often, we might as well make our lives a bit easier by using variables.

Nat’ing works as usual for the network, however only allows the ports we’ve specified in $allowed_out. The “rdr on” rule forwards port 80 on our untrusted network to our localhost’s port 80. So now, no matter what website our guests visit, they end up on our webserver! The “no rdr” rule excludes IP addresses listed in the “goodboys” table from our subsequent “rdr on” rule.

All we need to do once a guest authenticates is run a simple command from the command line to add their IP address to the “goodboys” list. That command would look something like this:

pfctl -t goodboys -T add

Setting up the web server

It’s pretty obvious what to do next. Setup a webserver on localhost. You can write any sort of authentication script you like. I’ve placed a simple note letting my guests know that I’m not a sucker, nor a socialist and that by using my network they agree to my terms. The link “I Agree” takes them to a simple php script:

$output=exec("/usr/local/bin/sudo /sbin/pfctl -t goodboys -T add " . $_SERVER["REMOTE_ADDR"]);
header ("location: http://www.mises.org/");

It would be simple enough to do that in any other language.

But wait… pfctl needs to be run as root…

Right.. Install sudo and add this to your /usr/local/etc/sudoers file:

www ALL=NOPASSWD: /sbin/pfctl -t goodboys -T add 10.0.2.[2-254]

I used a regular expression so that a vulnerability in my script couldn’t give a crafty guest free reign over pfctl. I’m sure you could also play with the permissions of /dev/pf and make use of /etc/devfs.conf … but ehh.. I think this does the trick.

Oh.. and don’t forget to change the netmask on your gateway

Our network is now (not /24) in comparison to the previous “Home Wireless” post. You’ll need to make the change in ifconfig and rc.conf .

Maybe not so obvious pitfalls..

  • A crafty user could assign themselves an IP address on the trusted network. If this were a product and we were concerned with real authentication, we wouldn’t have a trusted “backdoor” network.
  • Only port 80 is forced into our authentication scheme. The user could still make use of any other protocol in the $allowed_out variable. Again, if this were a product, we’d keep other ports closed and open them using our “goodboys” table, which contains the list of authenticated users.
October 20th, 2011 By randy Categories: System Administration

This isn’t going to be an AWK how-to, but an AWK-’why’. If you want a quick AWK tutorial check out Grymoires’ site.

System Administrators, DevOps or whatever you want to call them these days often need to parse large amounts of data in log files in order to extract relevant data. In fact, this is an often asked interview quiz question for many ‘nix sys admin jobs.

“Here’s a log file and a ‘nix shell. Write a script that tells me x, y and z.”

Often, smart, sys admins opt for PERL, and there is nothing wrong with that. However, did you know AWK was designed specifically for generating reports of this kind? The principles and techniques will be the same as in PERL, but AWK gives you a neat framework for generating them.

Here is a sample AWK script that parses an Apache log file, and spits out a list of IP addresses that have generated 1000 or more hits and how many hits they’ve generated sorted in descending order.

#!/usr/bin/awk -f
    OFS="\t\t"                 #Set the output field separator
    print "IP Address", "Hits"    
/^[1-9]/ {                     #Parse lines that start with a number (IPv4).
    iphash[$1]++               #Increment IP in our associative array.
    sort="sort -k2 -nr"        #The sort command we'll use. Parameters may
                               #vary depending on your flavor of 'nix.
                               #You may need to replace -k2 with +2.
    for (i in iphash) {
        if (iphash[i] >= 1000) print i, iphash[i] | sort 
                               #AWK's output buffer benefits us here
    close (sort)               #close the sort pipe to flush output buffer
    print "TOTAL", NR          #print total number of records (hits).

Save it as ‘something.awk’, chmod it to executable and run `./something.awk /var/log/your.log`