I have been exploring further the possible response actions an application might make once it has detected a suspected or actual attack, as a contribution to the OWASP AppSensor project. There is now a draft document describing response actions, discussed and announced last week.

The draft document AppSensor - Response Actions describes thirteen response actions, provides examples of each, and discusses how they might be categorised in order to help with selection of appropriate responses.

It is still a working document. If you have any suggestions or comments on the draft document, please send them to the AppSensor project's mailing list, or perhaps add them below. In particular, I'd like to discuss whether there are any other responses which aren't covered by the ones already included.

There is additional background information and links relating to web application intrusion detection and the OWASP AppSensor project in my posts about presentations in Newcastle and London, but I hope to present again later in the year.

I have been meaning to write again about web application security logging, but luckily read a paper last week which provides excellent guidance.

How to Do Application Logging Right is the best guidance I have come across to date. Co-written by Anton Chuvakin and Gunnar Peterson for the IEEE Security & Privacy Journal, the paper describes the problems with typical logging systems, what events need logging, and for those, what to include and exclude. They have also provided some broader guidance on log management and protection.

Previously, the most notable application security logging guidance existed buried rather deeply in the documentation for OWASP's ESAPI Java edition, the OWASP Logging Project, and more general guidance in NIST's SP 800-92 Guide to Computer Security Log Management.

If you read those in conjunction with the new paper, and perhaps Chuvakin's and Peterson's own comments, you'll be well up to speed.

The content of the "module", "object" and "action" fields will be dependent upon the degree of granularity required and how much additional event information is collected as additional details (e.g. stack trace, request headers, response body). I believe a transaction ID should always be included so that all events for a single request/response can be more easily correlated—this has a request scope rather than the session scope of a username/id. If I might suggest some other additional items for "what to include", I would also consider:

• host address (e.g. host name and domain, or server IPv4 or IPv6 address) which is useful if clustering is being used, or to confirm logs are from live rather than staging systems
• service (e.g. name, port and protocol)
• full actual entry point URL (protocol, full domain, port, path and further parameters)
• canonicalised entry point URL
• HTTP method (for web applications)
• responses seen by the user and/or taken by the application (e.g. status code, custom text messages, session termination, administrator alerts)
• analytical confidence in the event detection (low, medium, high or a numeric value).

Full request headers and possibly the response body may be worth collecting for some events. But ensure these are sanitised for sensitive input such as passwords, session cookies or credit card numbers.

I would also tend to use a severity scale (0=emergency, 1=alert, ..., 7=debug) rather than the suggested "priority" field, for consistency with syslog protocol. But the paper's authors note that whatever scale is used, it will be different for each organisation due to their own priorities and views on risk.

You may also want to consider how the integrity of the logged information can be determined.

Whatever you log, bear in mind you probably want it to be relatively human-readable, but also done in a way you can share the information with other systems. For the moment, consider Common Event Format (CEF). But Common Event Expression (CEE) is an ongoing collaborative effort to develop an event interoperability format summarised in a presentation, and in more detail in a white paper. The CEE web site includes a description of alternative approaches for sharing data from event producers.

See also my previous web application logging related posts How Much Logging, Monitoring and Alerting?, Security Logging Requirements, Testing the Audit Trail, Don't Stop the Attack (Too Soon), and Application Log Management and Analysis.

Happy application logging!

The UK's Office of Fair Trading (OFT) promotes and protects consumers' interests by ensuring markets work well, and that businesses act fairly and competitively. The government has asked the OFT to develop a longer term national strategy for consumer protection and enforcement on the internet. The strategy is intended to promote a safe and vibrant internet market.

As part of this strategy development, the OFT has launched a consultation on E-consumer Protection. The objectives are to improve the effectiveness of online markets and increase the level of consumer trust, so that consumers have a real option to use the internet for transactions, as equally as any other channel. The aim is also to ensure that enforcement of consumer protection online is as good as anywhere else in the world.

The main consultation document outlines some useful statistics about the UK internet economy using data from the European Commission's Consumer Markets Scoreboard 2010, the OECD and the OFT's Attitudes to Online Markets (publication due shortly). For example, 71% of the UK's retailers use e-commerce/internet sales channel for retail, and internet/online accounted for 9.5% of UK retail trade (£38 billion) in 2009. Apparently UK consumers have a high level of trust in UK sellers/providers' protection of their consumer rights and that they are adequately protected. However, it is not all good news as almost 20% of UK internet users are not transacting online, with a third of these stating concerns about the security of their personal and financial information as the reason. Overall, two-thirds of all internet users are worried about unauthorised access to their personal information. There are also concerns about being conned by companies online. The consultation document outlines how consumers may be becoming complacent about security but that they lack awareness of issues such as mis-use of cookies and behavioural advertising.

The OFT suggests these problems reduce confidence, lead to lower levels of demand, and consequently lower levels of supply. Households can miss out on potential savings and this is especially problematic for low income households (LIH). The consultation document proposes that agencies should work together to empower consumers, promote business compliance and develop effective enforcement. It proposes a number of high-level actions under the themes of consumer education, tool provision and hardening, business information, cooperation and deterrence, and enforcement capability building, coordination and leveraging intelligence.

The outcome of this consultation will have a large impact on organisations in the business-to-consumer (B2C) sector (there is also some discussion of whether C2C should also be addressed). If you are an online retailer, perhaps get in touch with your trade organisation and ask them whether they are responding, or do so yourself.

There are five general response questions, and further more-detailed questions about the high-level actions and monitoring proposed. Responses can be submitted online, by email and by post. The consultation period closes on 13th October 2010.

Fed up with false positives when trying to detect malicious users with network intrusion detection systems (IDS)? Application intrusion detection is the way to go.

Like an advanced robot, applications can build in security protection, detection and response.

Next Thursday 15th July 2010, I will be presenting "Real Time Application Attack Detection and Response" at the next OWASP meeting in London. Like all OWASP chapter meetings, the event is free but prior registration is required.

I will talk about how advanced attackers probe and try to exploit applications, how some common defences against these attacks are of no use, and why we need to use protection that:

• understands the application
• understands normal vs. suspicious use
• can identify and shut down attackers in real time.

Is this possible? Yes. AppSensor specifies how application-based detection points can be used to stop attackers. I will also describe how project leader Michael Coates has demonstrated how real web sites can deploy such measures in practice to protect an application against automated scanners, advanced attackers and build in protection against application worms.

Arrive from 17:30 hrs since the talks start promptly at 18:00. Hope to see you there.

Sometimes when I'm out socially and people ask what I do, the conversation progresses to concerns about their own web site. They may have a hobby site, run a micro-business or be a manager or director of a small and medium-sized enterprise (SME)—there's all sorts of great entrepreneurial activity going on.

It is very common for SMEs not to have much time or budget for information security, and the available information can be poor or inappropriate (ISSA-UK, under the guidance of their Director of Research David Lacey, is trying to improve this). But what can SMEs do about their web presence—and it is very unusual not to have a web site, whatever the size of business.

Last week I was asked "Is using <company> okay for taking online payments?" and then "what else should I be doing?". Remember we are discussing protection of the SME's own web site, not protecting its employees from using other sites. If I had no information about the business or any existing web security issues, this is what I recommend checking and doing before anything else:

• Obtain regular backup copies of all data that changes (e.g. databases, logs, uploaded files) and store these securely somewhere other than the host servers. This may typically be daily, but the frequency should be selected based on how often data changes and how much data the SME might be prepared to lose in the event of total server failure.
  • check backup data can read and restored periodically
  • don't forget to securely delete data from old backups when they are no longer required
• Use a network firewall in front of the web site to limit public (unauthenticated user) access to those ports necessary to access the web site. If other services are required remotely, use the firewall to limit from where (e.g. IP addresses) these can be used.
  • keep a record of the firewall configuration up-to-date
  • limit who can make changes to the firewall
• Ensure the host servers are fully patched (e.g. operating system, services, applications and supporting code), check all providers for software updates regularly and allow time for installing these.
  • remove or disable all unnecessary services and other software
  • delete old, unused and backup files from the host servers
• Identify all accounts (log in credentials) that provide server access (not just normal web page access), such as used for transferring files, accessing administrative interfaces (e.g. CMS admin, database and server management/configuration control panels) and using remote desktop. Change the passwords. Keep a record of who has access and remove accounts that are no longer required and enable logging for all access using these accounts.
  • restrict what each account can do as much as possible
  • add restrictions to the use of these accounts (e.g. limit access by IP address, require written approval for use, keep account disabled by default)
• Check that every agreement with third parties that are required to operate the web site are in the organisation's own name. These may include the registration of domain names, SSL certificates, hosting contracts, monitoring services, data feeds, affiliate marketing agreements and service providers such as for address look-up, credit checks and making online payments.
  • ensure the third parties have the organisation's official contact details, and not those of an employee or of the site's developers
  • make note of any renewal dates
• Obtain a copy of everything required for the web site including scripts, static files, configuration settings, source code, account details and encryption keys. Keep this updated with changes as they are made.
  • verify who legally owns the source code, designs, database, photographs, etc.
  • check what other licences affect the web site (e.g. use of open source and proprietary software libraries, database use limitations).

Do what you can, when you can. Once those are done, then:

• Verify the web site and all its components (e.g. web widgets and other third party code/content) does not include common web application vulnerabilities that can be exploited by attackers (e.g. SQL injection, cross-site scripting).
  • use resources such as the OWASP Top Ten and WASC Threat Classification
  • build security requirements into future developments
• Check what obligations the organisation is under to protect business and other people's data such as the Data Protection Act, guidance from regulators, trade organisation rules, agreements with customers and other contracts (e.g. PCI DSS via the acquiring bank).
  • impose security standards and obligations on suppliers and partner organisations
  • keep an eye open for changes to business processes that affect data
• Document (even just some short notes) the steps to rebuild the web site somewhere else, and to transfer all the data and business processes to the new site.
  • include configuration details and information about third-party services required
  • think about what else will need to be done if the web site is unavailable (does it matter, if so what exactly is important?)
• Provide information to the web site's users how to help protect themselves and their data.
  • point them to relevant help such as from GetSafeOnline, CardWatch and Think U Know
  • provide easy methods for them to contact the organisation if they think there is a security or privacy problem
• Monitor web site usage behaviour (e.g. click-through rate, session duration, shopping cart abandonment rate, conversion rate), performance (e.g. uptime, response times) and reputation (e.g. malware, phishing, suspicious applications, malicious links) to gather trend data and identify unusual activity.
  • web server logs are a start, but customised logging is better
  • use reputable online tools (some of which are free) to help.

That's just the basics. So, what would be next for an SME? If the web site is a significant sales/engagement channel, the organisation has multiple web sites, is in a more regulated sector or one that is targetted particularly by criminals (e.g. gaming, betting and financial), takes payments or does other electronic commerce, allows users to add their own content or processes data for someone else, the above is just the start. Those SMEs probably need to be more proactive.

This helps to protect the SME's business information, but also helps to protect the web site users and their information. After all, the users are existing and potential customers, clients and citizens.

Oh, the best response I had to someone when I was explaining my work: "You're an anti-hacker than?". Well, I suppose so, but it's not quite how I'd describe it.

Any comments or suggestions?

Continuing on the theme of confusing users, Security Dialogs and Graphics discussed the multitude of inconsistent styles for security warnings on web sites, mobile applications and in email.

This is usability hell. Why should each device, browser and application choose how these messages are worded and displayed? I mentioned previously the contributing factor of tab colouring in IE8 and a recent post Tabnabbing: A New Type of Phishing Attack demonstrates how users can be tricked into providing sensitive information to the wrong web site. But it's not just inconsistent technical implementation that matters; the humans in the process matter too.

Last week I came across two web sites in my normal business usage that gave me invalid SSL/TLS secure certificate warnings, and these weren't little businesses that might not know any better—both are multinational enterprises—one a UK bank, and the other a UK mobile phone company.

1. In one case a registration process used a domain www.[subdomain].[company].co.uk whereas the certificate was for [subdomain].[company].co.uk
2. In the other, the company had recently been taken over and the site was using www.[oldcompanyname].com whereas the certificate was for www.[newcompanyname].co.uk

But what surprised me most was the response I received when reporting these problems to the respective site owners. Neither organisation had a clear process for sending details of possible security problems and therefore the responses seem to have been directed through customer support channels.

One provided the initial response:

and:

and even though I re-explained the problem, and that it hadn't stopped me from doing what I wanted:

and then:

and the rather indefinite:

and then back to asking me to supply screenshots, my username and details of my browser:

Mmm.. and still no idea about the issue:

My issue? Their issue I believe.

The other company suggested it was the date/time on my own computer which was the fault:

and some advice about security:

at which point I rang them up, and stumped them by quoting their own tracking number. No further call back from them yet.

I found this all rather depressing. How can we expect customers (end users) to take care with security if the understanding and processes are not in place within the organisation, especially with customer-facing staff? Organisations like Get Safe Online and Card Watch are trying their best to educate people, but the web site owners need to play along too. So here's a quick checklist, including a couple of new items:

1. Obtain your own domain like example.com or example.co.uk (not a sub-domain of someone other company e.g. example.uk.com)
2. Provide security training to web site architects and developers
3. Determine how SSL/TLS will be applied on the site
4. Buy a certificate and apply SSL usage appropriately through the site
5. Verify the proper usage of SSL and session management
6. Verify the correct SSL configuration
7. Include SSL/TLS certificate management in change control processes
8. Provide awareness training to customer support staff about what "secure web site" means and the types of enquiries customers may have
9. Give some basis security advice to customers and direct them to other resources for more information
10. Ensure there is a simple way for people to inform you of security concerns and possible incidents such as phishing, browser security warnings, compromised credentials, account mis-use, etc.

Please—I don't want to have to go through customer support again! At the time of publishing this post, the certificate problems still exist.

Security and audit logging should be defined, implemented and tested for every web application. But what about log management and analysis?

This week Raffael Marty posted an updated item to his blog about a Maturity Scale for Log Management and Analysis. It is an excellent review.

Whilst much of this management and analysis is intended to be external to an application, we need to remember each application needs to record adequate information to feed into these analysis and reporting tools. And why do that? Read the bullet points under return on investment (ROI) at the end of the article. What else? Well perhaps also:

• feedback into the development lifecycle (to improve subsequent patches, versions and other projects)
• greater trust by users
• brand protection
• protection of information assets (not just preventing leaks, but ensuring accuracy and integrity).

Therefore, build adequate logging in from the start. Web server logs are not enough!

Special Publication (SP) 800-122 Guide to Protecting the Confidentiality of Personally Identifiable Information (PII) has been published by the US National Institute of Standards and Technology (NIST). Are you using personal data on your web site?

SP 800-122 provides a useful read for people responsible for assessing privacy and for those designing and implementing privacy controls within information systems and business processes. Importantly it mentions web applications which are increasingly being used as part of business processes. By their nature, data will pass through systems more exposed to public threats.

In the UK, the best starting point for advice is the Information Commissioner's Office guides and other resources, especially the Data Protection Guide and the pages and reports on building privacy in. However, SP 800-122's impact classification methodology, lists of safeguards, examples and scenarios are useful whatever your jurisdiction.

But do note, the definitions, requirements and obligations in NIST SP 800-122 of course relate to US legislation and not to the UK Data Protection Act 1998. In particular they don't cover all eight UK data protection principles. Apart from background reading, they can therefore also be of use for UK organisations considering, or who already have, customers or some other presence in the US.

Last week, Symantec published its latest Internet Security Threat Report.

The 95-page report describes Symantec's methodology, findings and recommendations about internet security threats to businesses and individuals. It describes the financial and other losses possible such as damage to reputation and data theft. There is a strong focus on protecting confidentiality and less about how internet threats affect the integrity of data and availability of information systems and business processes.

In the two chapters on Vulnerabilities and Malicious Code Trends, the importance of publicly accessible services (web, mail and FTP) and vulnerabilities in web browsers and web browser plugins in the malware ecosystem are highlighted and recommendations for protecting these servers are provided. The top Web-based attack in 2009 was associated with malicious PDF activity, which accounted for 49 percent of the total.

The chapter on Phishing, Underground Economy Servers, and Spam Trends provides a good insight into how your users may be targetted by third parties hoping to lure them into visiting other web sites. the report makes the important point that "the use of brand(s) in phishing activity can significantly undermine consumer confidence in its reputation". The financial sector continues to be the primary target for phishing attacks, but all types of organisation can be targetted.

Appendix A describes some best practices that businesses (enterprises) and consumers should follow to reduce the risk from internet threats. Many of these relate to using electronic mail and browsing web sites. The slightly more web application related recommendations include employ defense-in-depth strategies, administrators should limit privileges on systems for users, turn off and remove services that are not needed for normal company network operations, test security regularly to ensure that adequate controls are in place, educate management on security budgeting needs, administrators should update antivirus definitions regularly, always keep patch levels up to date, enforce an effective password policy and ensure that emergency response procedures are in place.

A shorter executive summary of the report is also available.

A recent High Court ruling has reconfirmed the situation that pre or post moderation of user-submitted content may make a site owner liable for the material.

Whether user-generated content is unlawful, offensive or inappropriate such as comment spam (i.e. a danger to the web site, its users or their computer equipment), the advice appears to be not to do anything until a complaint is received, and then block or remove the content expeditiously. Although the meaning of content may still be an issue, the ability for users to submit links and other formatting should certainly be automatically prevented in most cases. That "just" leaves the unlawful and offensive content to deal with. Use of user registration, identity verification, logging and CAPTCHAs can help, but cannot prevent such content being added. It's still a big issue.

Most web site owners will not contemplate unmoderated user-generated content and this means that technical controls are not sufficient. The moderators need training, guidance and escalation procedures with good legal advice backup to ensure the content is suitable, appropriate and lawful. Users of the web site should understand what is acceptable and opt in to appropriate terms of use.

A full description and analysis was posted on the IT and e-commerce legal advice web site Out Law.