AppSensor Application Logging, Signalling and Dashboards

Last week at AppSec EU, I gave a talk about OWASP AppSensor project.

(Show Full Size Image, 1203x728px, 98kB)

Following a short introduction to the concept, I sked the audience two questions: do you know if your applications are being attacked right now, and have any unknown vulnerabilities been exploited today? Using three attack test cases, I outlined how conventional defences do not protect applications and why building application-specific security controls within the application code is best. I provided references to the project documentation and video presentation by the project leader Michael Coates, for those who wanted further details. But I wanted to focus on the aspects of AppSensor architectures, application logging and signalling information to other systems such as a live dashboard, to illustrate that implementation need not be a huge challenge, and that by starting simply, this can be a good foundation to gradually build up a more comprehensive approach.

I described how AppSensor detection, logging and analysis could all be located on a single-server system and how this is not so dissimilar to a simple three-tier architecture with no failover. Even in this arrangement it may make sense to move analysis, and possibly logging, to a separate system. In clustered environments with load balancing, there may already be separate centralised logging. But detection needs to be added to every application instance, with centralised AppSensor analysis. The analysis engine may also collect information from other detection point locations such as network security devices and other business systems. It may also send (signal) information to some of these systems.

Then I described application logging practices and what additional types of information should be recorded if AppSensor is implemented. I made a distinction between the raw application logs (which may be done locally or by some other device), and signalling which I see as a sub-set of log data. It could also include some form of aggregation of discrete events, such as say a single data points concerning an average function usage over a particular time period.

The use of Common Event Format (CEF) was explained since this can be used for both raw logging and signalling, and benefits from being widely supported. But I noted we should keep our eyes open for developments in Common Event Expression (CEE) and Event Management Automation Protocol (EMAP) - read how these are related.

Moving on, I then described an example ecommerce web site (Schematic Diagram, 932x426px, 53kB), and what an initial base configuration of AppSensor might look like. For the example site, the primary issues were considered to be product pricing errors, discounts and fiddles, order process manipulation, payment card mis-use and personal data theft. Therefore just nine detection points were considered relating to general request processing to filter out clearly malicious payloads, some specific detection in the catalogue, shopping basket and payments functions, together with some detection related to database result sets and database table integrity.

Three (request exception) detection points would monitor every request, and the database detection points would occur in multiple locations (code and tables). Response actions would always include increased logging (e.g. request and response headers), with some events leading to alerts to operational staff, and serious events causing temporary blocking of user accounts, and/or IP address ranges. A dynamic demonstration of signalling and an example live dashboard was used to show how the detection events, responses and information on any locked customer accounts might be displayed. A video (without narration) of this first scenario is available (You Tube Example 1).

I then discussed how that base configuration might be extended to provide greater insight into input validation failures, checks on session management and user/system trend monitoring. The second scenario also included the use of information from third party malware monitoring and a network intrusion protection system, providing additional data which could be used by the analysis engine to make decisions (YouTube Example 2).

The videos should be updated soon with new recordings that have a narrative soundtrack.

If you are going to be in or near Brussels, I will be repeating the talk and live demonstration at OWASP Belgium in two days time (16th June 2011). I am also looking forward to hearing Andreas Falkenberg speak at the same event on "How to Become Twitter's Admin: An Introduction to Modern Web Service Attacks". See you there.