Perl, Python, SSL/TLS

Performing HTTP SSL server certificate validation from Python or Perl

Update: Since I’ve written this post I’ve switched to using Python Requests, which is a much better way of achieving verified SSL connections.

SSL/TLS, love it or hate it, is the backbone of nearly all online communication. These days most network protocols are usually written atop HTTP, and then wrapped inside TLS (HTTPS) to provide encryption. But HTTPS also provides verification and trust via certificates. In this way you can ensure that not only are you sending your data in an encrypted fashion but you are talking to the real server and not a rogue server instead.

You would think, given the prevalence of systems written using HTTPS as the underlying protocol, that writing a HTTPS client in Python or Perl would be easy and all the complex security and verification would be done for you. Sadly not so. Simply opening HTTPS connections from Python and Perl are both extremely easy – you can use urllib2 in Python and LWP in Perl. Both provide encryption – but certificate verification? Not so easy.

Perl is best placed here because the current version of LWP (or any version from 6.00) performs certificate validation. When you connect to an SSL HTTP server it will validate the certificate and ensure that the CN of the certificate matches the hostname you specified. For more details, see the LWP docs for SSL opts. This feature is relatively recent however – it was only released in March 2011. As such nearly all current stable Linux distributions ship a previous LWP.

Previous releases could be made – with some difficulty – to validate the certificate, but not to verify the CN of the certificate matched the hostname the code was connecting to. That wasn’t all that useful of course because as long as the server had /any/ valid SSL certificate trusted by the client then it would work. So for LWP connections using SSL make sure you update your module to LWP 6 or later, and of course you can do this easily on even old versions of Perl.

What about Python? Surely Python’s famous ease of use and great built in modules will mean it works? Alas, no. The situation is even worse. As the urllib2 documentation for Python 2.7 says, HTTPS connections do not perform certificate verification. Python 3.0 or 3.1 doesn’t support it either. Python 3.2 or later, thankfully, does support performing SSL certificate verification – it has new options for specifying the CA certificate details. It even does hostname CN checking/verification. But Python 3.2 is very new as well – first release in February 2011 – and is in virtually no Linux distributions – certainly no stable or enterprise releases.

However, there is another option. Many programs are written in Python that perform proper SSL certificate checking. A little poking around at the source of some Red Hat utilities revealed the answer – they don’t use urllib, they use libcurl / pycurl instead. The python curl module supports performing SSL verification and hostname verification and as such this module is used by virtually all the tools in Python which need to talk over HTTPS. Sadly, many don’t however and were probably coded with urllib thinking they did.

Sadly, pycurl is really badly documented. So if you want to find out how to do it here is a little example:

import pycurl
curl = pycurl.Curl()
curl.setopt(pycurl.URL, "https://secure.domain.com/")
curl.setopt(pycurl.SSL_VERIFYPEER, 1)
curl.setopt(pycurl.SSL_VERIFYHOST, 2)
curl.setopt(pycurl.CAINFO, "/path/to/certificate-chain-bundle.crt")
curl.perform()

The “SSL_VERIFYPEER” flag means that cURL will check the validity of the certificate against the certificate chain / root CA certificates. The “SSL_VERIFYHOST” flag means that cURL will check the certificate CN matches the hostname you connected to. The latter option must be set to 2 – see the link for more information.

DHCP, Infoblox, PXE

PXE booting to a Windows WDS boot server from a Linux (or Infoblox) DHCP server

The University of Southampton is deploying a new Infoblox-based DHCP and DNS service. Underneath Infoblox is just a custom version of ISC DHCPD and ISC BIND. We also have a Windows-based WDS (Windows Deployment Services) network build server which we want to be used as the PXE build server. If you’re not using IP helpers, which most people don’t, this can be tricky.

The solution is to configure the “next-server” and “pxe-boot” server IP address in Infoblox. As far as I can tell this seems to set DHCP option 66 which is the TFTP boot server IP address. However you also need to set the boot filename – which seems to be DHCP option 67. If you’re using a Linux TFTP server and syslinux/pxelinux that filename should be set to ‘pxelinux.0’. The Windows TFTP server of course requires a different path.

It seems that for Windows WDS the path is ‘/boot/x86/wdsnbp.com’, except Windows uses backslashes, and just putting that path in does not work – you’ll end up with a TFTP access violation error. Instead you need to use backslashes, but escape the backslashes as ISC DHCPD will think you’re using an escape sequence. You also need to terminate the filename with a NUL character.

You’ll thus need something like this in your Infoblox GUI:

Screen Shot 2013-05-29 at 21.02.19

And then, voila, you should be able to use a PXE boot client onto a Windows Deployment Services PXE server from an Infoblox DHCP server.

Bargate, Flask, Web Development

Logging detailed information when Flask deals with exceptions

I use the Flask web development framework, written in Python, as the tool of choice for writing web applications. Its simple, lightweight, easy and well documented. It makes writing web applications a breeze – freeing me to write the important stuff.

Flask integrates into the Python API wherever possible and takes care of most things for you. When something goes badly wrong in your code Flask catches exceptions and sends them through Python’s logging framework. This is very useful – in addition to telling the user, Flask can be configured to send logging to a file and to e-mail, so I get to hear about it even if a user doesn’t report a fault. This is where tools like Sentry also fit in – although I don’t use them.

The problem is that default logging, whilst dumping a full stack trace, does not add any Flask or web-specific details. The log format only contains the default log record attributes:

Message type:       ERROR
Location:           /usr/lib/python2.6/site-packages/flask/app.py:1306
Module:             app
Function:           log_exception
Time:               2013-05-03 15:35:57,170
Logger Name:        bargate
Process ID:         1877

Message:

Exception on /login [POST]

The worst part of the above is the lack of relevant information, but also the fact that because the exception is handled by several wapper functions, the ‘Location’ and ‘Function’ and ‘Module’ parts are all irrelevant because they refer to the function dealing with the exception rather than where it was generated from.

Either way, I wanted to extend the log format (the emails generated) when exceptions occur. A thread on the Flask mailing list led me toward messing with subclassing parts of Python’s logging classes (as indicated by other developers), but this led me astray. After extensive reading of ‘logging’ and thinking about the problem in detail, I realised this was nonsense – there was no way to inject additional custom attributes into exception logs like that.

The way to do it is to inject custom data into a log with the “extra=” kwargs flag when calling “logger.error”. The problem of course is that code you write within Flask doesn’t (generally) call logger.error at all – Flask does this for you within app.log_exception. The solution then is deliciously simple – and a recommended way of using Flask – subclass Flask itself.

Thus I subclassed Flask like so:

class BargateFlask(Flask):
        def log_exception(self, exc_info):
                """Logs an exception.  This is called by :meth:`handle_exception`
                if debugging is disabled and right before the handler is called.
                This implementation logs the exception as error on the
                :attr:`logger` but sends extra information such as the remote IP
                address, the username, etc.

                .. versionadded:: 0.8
                """

                if 'username' in session:
                        usr = session['username']
                else:
                        usr = 'Not logged in'

                self.logger.error("""Path:                 %s 
HTTP Method:          %s
Client IP Address:    %s
User Agent:           %s
User Platform:        %s
User Browser:         %s
User Browser Version: %s
Username:             %s
""" % (
                        request.path,
                        request.method,
                        request.remote_addr,
                        request.user_agent.string,
                        request.user_agent.platform,
                        request.user_agent.browser,
                        request.user_agent.version,
                        usr,

                ), exc_info=exc_info)

In the above example I decided not to use the extra kwargs flag and modify the Formatter object when setting up e-mail alerts as that would mean every call to app.logger.error would require me to send the same extra= flags. Instead I added the information I wanted via the message parameter (which achieves the same end result). This way, error e-mails are much more useful:

A fatal error occured in Bargate.

Message type:       ERROR
Location:           /data/fwa/bargate/__init__.py:104
Module:             __init__
Function:           log_exception
Time:               2013-05-05 09:43:59,942
Logger Name:        bargate
Process ID:         12141

Further Details:

Path:                 / 
HTTP Method:          POST
Client IP Address:    152.78.130.147
User Agent:           Wget/1.11.4 Red Hat modified
User Platform:        None
User Browser:         None
User Browser Version: None
Username:             Not logged in