Securing Linux

July 3, 2003

Few people decide on their careers until they are well into their teens. Russell Coker is an exception. When he built his first computer, a TEC-1, from a kit, at the age of 11, he consciously made a career decision as well - he would be working in that field.

However, the Melbourne-based developer and sysadmin probably never figured that his name would one day end up on the website of America's National Security Agency, the cryptologic organisation.

That's exactly how things have turned out. Russell is among those who have made a sterling contribution to the NSA's SE Linux project. He is listed both among those who have contributed to the upstream line of development and to the community.

On the upstream front, Russell has expanded and improved the example policy configuration, enhanced the run_init and spasswd utilities, developed a devfsd module for managing devfs file contexts, implemented improvements to the setfiles program, and extended strace to trace SELinux system calls.

On the community front, he has ported and packaged SELinux for Debian GNU/Linux and now handed off maintenance of the package for Debian stable to fellow Australian, Brian May.

More recently Russell was asked to develop SE Linux for the Ipaq by a company which is thinking of using it on the handheld device.

Russell in no way resembles the stereotype of a geek. He is clean-shaven, has neatly trimmed hair and, if he had been wearing a suit, would probably have passed off as the CEO of some medium or large corporation (he wore a T-shirt with a VA Linux logo and a jacket of similar origin).

Like many technically competent types, he is quick, almost impatient, with his words and has strong opinions about many things. His answers are not so much abrupt as compact and rounded. You know when he's stopped thinking about something and had his say.

Russell is married - and, fortunately for the Linux community at large, his wife, Faye, is also an experienced sysadmin, and understands the sense of involvement that open source projects generate. Else, the amount of time he has spent developing software and contributing to development would probably have been much less.

Back in Melbourne after working in the Netherlands for a few years, Russell spoke at length about his background, his involvement with open source software in general and the SE Linux project in particular.

The tale in his own words:

Let's have a brief idea about yourself - your area(s) of expertise, what you do, your background...

Schooling is not particularly relevant. The computer education programme at schools is not nearly as technical as it should be. I tried to convince the computer teachers to get a Unix server as buying a single 386 machine would be much cheaper than upgrading the class network of IBM PC-Jr machines to ATs and give better performance on average. Unfortunately they didn't like the idea and I didn't get to use Unix until University.

I completed the Computer Science and Software Engineering degree at Swinburne University because it seemed to be the most technical computer course available at the time.

I have had a lot of experience running ISPs which started when I set up an internet cafe in 1995. I do C and C++ programming, package a lot of software for Debian, and do most Unix and network administration tasks.

How did you get interested in Linux?

In 1993 I wanted to run a public access Unix server. I made an offer for an unused University machine but they wanted too much money. So I bought a 386 and installed Linux on it. Later I used Linux as the server for my internet cafe. Then, after OS/2 became obsolete, I started using Linux for everything.

Moving over from Windows to Linux - how did it come about?

I initially removed Windows 3.0 from my computer when a Microsoft representative told me that it was easier for them to train people to work around bugs in their software than to fix them. I deplore this attitude and I immediately went home, removed Windows from my computer, removed Windows file areas from my BBS, and have had as little as possible to do with Microsoft software ever since.

At what stage of your life did the open source model start to make sense?

In the mid 90s when my programming skills increased I wanted to be involved in larger projects. Commercial projects are almost never done properly and most of the interesting projects are too large for one person to work on alone. So the Open Source model is the only viable option.

What exactly is the SE Linux project?

NSA Security Enhanced Linux is comprised of a kernel patch to add security features, and patches to applications to allow them to determine the security domain in which to run processes. For example, /bin/login selects the domain for user processes according to configuration files and the security policy database.

In SE Linux every process is in a security domain, and every file, network socket, or other object has a security type associated with it. SE Linux uses a "policy database" to determine what access each domain gets to the objects on the system. This means that a "root" process does not get full access to the system unless SE Linux also permits the access. SE Linux is based on the Linux Security Modules (LSM) interface.

How did you come to get interested in this project?

At the Ottawa Linux Symposium 2001, Peter Loscocco from the NSA gave a presentation on SE Linux. After that I spent some time talking to him about the project and came to understand that SE Linux offers significant security benefits. Also SE Linux has the backing of some significant organisations and seems to have the most potential for widespread use.

There are a large number of different kernel security patches available. But most of them seem to have limited use, and many of them seem to have little overall design. SE Linux has a clean design with concrete objectives, good quality code, and good support.

I decided that to improve the state of security in Linux this needed to be integrated into the Debian distribution. I expected such work to take a few weeks and that I would then pass it on to someone else after I got it done. However, I then became involved in the development of SE Linux (through writing security policy for applications and daemons, and through patching applications for SE Linux support). So I have now spent the equivalent of about two years of full-time work on SE Linux.

What kind of feedback have you had from those who have tried out your patches?

The feedback has been very positive overall. The take-up of SE Linux has been slower than I expected (I had expected over 10,000 users of my SE Linux packages by this time, while I only have about 100 users). However, SE Linux is gaining a lot of the smarter Linux users, so I expect that they will recommend it to others and the user-base will increase rapidly in the near future.

I have on many occasions put SE Linux "play" machines on the net with a public root password. (The details are here). Lots of people have tried to break them for fun. Several configuration errors were discovered in the early days, mainly due to the fact that the earlier policy was not designed to be used on a machine with public root access. The use of such machines has resulted in a stronger security policy, so if someone gains unauthorised root access on a SE Linux machine they won't be able to do anything dangerous (and their attempts to damage the machine will be logged clearly).

How long did you take to complete development? Who else was involved?

The SE Linux code was based on security research dating back decades, and it was implemented by NSA employees and by other companies under contract to the NSA. I could not even guess how much work was involved in the research that it is based on. The implementation of the Linux kernel code, application patches, and policy would be at least 10 person-years of work by my most conservative estimates. For a GPL project such as SE Linux it is almost impossible to determine exactly how much work has been involved, when someone produces a small elegant patch to solve a problem you never know if it was 10 minutes work that seemed obvious to them, or whether they spent a month testing out all the alternatives.

A list of public contributors is on the NSA site. Note that some people and organisations do not wish to be publicly credited for their work - a full list of contributors would be longer. The actual amount of work involved in implementing the code could be as much as 30 person-years or more.

In addition to this there have been a number of projects related to writing tools to manage security policy and to check that the kernel code is correct. Large amounts of work have gone into this, it could easily be four person years or more.

You've developed SE Linux for the Ipaq. How did this come about? Is it available for download?

The patches are all on my website. Not all the binaries are available at the moment.

It came about because a company that is considering using iPaQs wanted to have SE Linux available for them, so they paid me to do the port. At the same time someone else was doing a port but couldn't get permission to release their work, so there was some duplication of effort in that regard.

Like any person in their 30s, I'm sure you have dreams of being rich. Yet you stick to the open source model, the same one which many consultants insist will end up leaving people in the doghouse. Are you crazy or is there some logic behind it?

If you are good at your work and you work in a field such as computers where there is a lot of money then you don't have to worry too much about such things.

If you write proprietary software it is very difficult to prove your skills. What do you say at a job interview - "I wrote a large part of a cool program, unfortunately I can't show you the program unless you pay lots of money and I can't prove which bits I wrote but you'll have to trust me that I'm good"?

When you do Open Source programming you can give a potential employer a list of the projects you've worked on and invite them to read the code for a fair assessment of your skills. If your skills are good, then this is a good way of demonstrating them. Some employers routinely ask for a personal web page and lists of SourceForge projects from job applicants.

In the case one company which I applied to, they spent an entire day reviewing my web site (from web logs) and then spent an hour at the job interview questioning me about the code on my site.

Why do so many people in the IT industry hate Microsoft? Is it jealousy? Or is there something else?

Hate is not the correct word. We just dislike low quality software and their attitude towards fixing it. Also Microsoft actively tries to restrict the freedom of choice of the consumers. Every time I purchase a PC I am forced to purchase yet another unwanted licence for Microsoft software that I will never use.

Every new version of Microsoft Office is intentionally incompatible with the file formats used by the old version. This means that when a new version comes out corporations are forced to upgrade (often without budgets to upgrade hardware, so the machines run very slowly). People who don't use Microsoft software have to wait for input filters to be adapted for their favourite programs to import the new file formats.

Finally, there is a significant amount of work involved in running mail servers since Microsoft viruses have become common. Filtering out all the viruses which only exist because of poor design decisions in Outlook requires significant effort.

Microsoft deliberately acts against the best interests of their customers (and everyone else) to pursue their own short-term financial gain.