Continuous Monitoring and an 85% drop in Risk

This Article reminds me of the presentation I saw by Allen Paller at Infosec last year. Alan Paller is the director of Research at SANS. He presented a testimonial of his work within the State Department to congress. He highlighted the development and use of a continuous monitoring methodology, which has led to an 85% drop in measured risk. In part this was achieved by using a continuous monitoring approach which was an IT-driven system and replaced the existing paper-based reporting system. What made this approach effective was the usage of the metrics that were Comparative, Numeric, Reliable and Authoritative. At infosec Paller went into a little more detail about these aspects.

Comparative: Comparative monitoring needs to be something that can be used to show the relative quality between the effort undertaken by different teams. This creates a healthy climate of competition and motivation

Numeric: Existing FISMA standards dictate that the reporting systems should produce a risk report every few months or quarter. This is a terrific delay in terms of response time, and ability to gain instant situational awareness. One important factor in ensuring success was to automate the measurement of these controls. Without it, the overhead would have been a barrier to it being effective. The monitoring period was reduced to 72 hours thus having the effect of allowing a better response time, as well as illustrating gains being made.

Reliability: Based on repeatable tests, two or more evaluators would get the same results.

Authoritative: By getting a consensus from an acknowledged group of experts allows you to get buy-in from the very individuals who will be assessed by the measurements.

It is an interesting approach as it advocates a far more scientific methodology to monitoring . However, Paller also highlighted the fact that the human element needed to be considered as equally as important. Therefore, be fair when measuring metrics. If a team cannot change or effect something, it is not fair to measure it. Finally celebrate success and make progress visible.

MESP and EaaS

I've just watched a fascinating documentary about the Rolls Royce jet engine company. I had always thought of them as a Plain Old Engine Company. To my surprise and delight, it turns out that they're an EaaS and an MESP.

As part of their offering, they provide a service package with each engine. The Airline companies are guaranteed an engine, no matter what, and only pay for the miles they clock up. This is, rather wonderfully, Engine as a Service (EaaS).

In addition, each serviced engine has a near real-time monitoring system (approx 90 sec delay), which allows engineering staff located at a operations centre to follow and respond to any particular issues flagged by the monitoring system. Any reported problems are then dealt with by ground crews at the appropriate airport at which the plane will land. The data sent in to the operations centre is analysed, and deviations from the norm are flagged and followed up. This, of course, sounds exactly like the Managed Security Service Provider (MSSP) model that has expanded a great deal in recent years. However, this instance should probably be called a Managed Engine Service Provider (MESP).

What fascinates me about this model is the emphasis on an engineering approach from the testing, build and deployment of each engine component, right through to the assembly, commissioning, and even in-air monitoring of the system. I wonder how this impacts on the false positive metrics that the operation centre employees must deal with, and whether there is as large a problem as it is with an MSSP engineer monitoring the voluminous output from a computer system.

There's an old(ish) debate within the software engineering community that states there's not enough emphasis on the engineering side. Lack of engineering approaches has led to code defects, and therefore the abundance of security issues that need to be dealt with. Combine that with the fact that we look for security events in stupendous volumes of data that is highly unlikely to yield useful information, and is often not fit-for-purpose, then you've got a serious problem when trying to hunt down serious problems. Not all security problems are based on defects caused by lack of standards, but it would be interesting to see how a modern MSSP would look if we had an operating system equivalent of Rolls Royce.

Top 10 reasons your security sucks

There's a great post over at infosec-island, commenting on the cultural, procedural and technical problems that appear to be still present in infosec environments.

All of the reasons are pretty much spot-on, but the following stuck out from our technology perspective:

6. The tools you use are ineffective (they don’t really work) and inefficient (they cost way too much)

5. Your security vendor is lying to you and why shouldn’t they, you believe them

2. Your dealing with the exact same problems you dealt with a decade ago, only it seems so much worse today then back then

It seems to me that reasons 2, 5 and 6 are interlinked. After speaking with someone we know at a local data centre, as well as folks at a local MSSP, it's clear that the lack of innovation in this domain is stark to say the least. As Niladri highlights, all of the trade shows he's been to in the past, in his other life in other industries, have had some form of innovation. Information Security? Nope. Nada. Nothing. The path that most of these guys are on is BIGGER FASTER MORE of the same old stuff that's becoming increasingly obsolete.