Security Logging Requirements

In How Much Logging, Monitoring and Alerting? I suggested logging implementations are often incorrect for most web sites and web applications.

The logging should be defined in terms of its intended use. We're talking here specifically about information security, so what might the uses and logging be?

1. To confirm data and process integrity and availability:
   • completeness and consistency
   • response times
   • function/process abandonment
   • session timeout
   • up-time
   • data changes
   • data mirroring, back-ups and archiving
2. To identify and provide enough information for investigation of:
   • errors and unexpected conditions
     • code errors
     • database access and performance
     • web server errors
     • third party services
     • lack of storage space
   • data breaches
   • use and mis-use
     • authentication successes and failures
     • access (authorisation) failures
     • excessive use
     • data validation failures
     • fraud and other criminal activities
     • suspicious, unacceptable or unexpected behaviour
   • modifications to configuration
   • security reports from users and third parties
3. To provide data:
   • subject access requests
   • freedom of information requests
   • litigation document requests
   • police and other regulatory investigations
4. To monitor content changes:
   • database fields
   • file contents
   • generated web page content
5. To demonstrate compliance:
   • internal policies and standards (e.g. information security policy, quality standards)
   • contractual obligations (e.g. PCI DSS)
   • change control
   • use of other's intellectual property
   • legislation (e.g. Data Protection Act)
   • regulation (e.g. Financial Services Authority)
   • external standards (e.g. Web Content Accessibility Guidelines [WCAG] 2.0 conformance claim)

It's important the logging is centralised so that alerts and reporting can be drawn from across all sources (web, application, file and database servers, network devices, etc). The scope and extent of logging ought to be be determined by business needs and the threats. For a typical e-retail site, the payment, check-out and any registration facilities will require greater logging than other parts. In some cases it may be appropriate to set particular thresholds for additional logging (e.g. transactions above a certain value, requests from particular clients, users in some geographic locations). This is easier if the requirements can be built into projects at an early stage.

The logging then needs to be tied in with appropriate monitoring, alerting and reporting. If you want alerts raised automatically, you'll have to think of what conditions initiate these. Referring back to specifications, threat models and test cases can be of use with this.