The UK's Centre for the Protection of National Infrastructure (CPNI) has updated its guidance on protecting business applications with the publication this month of a new document on developing and implementing secure web applications.

Development and Implementation of Secure Web Applications is a well-written and digestible 81-page A4 document arranged in seven main sections:

• Introduction to web application security
• General aspects of web application security
• Access handling (authentication, session management and access control)
• Injection flaws
• Application users and security
• Thick client security
• Preparing the infrastructure

It appears to replace the good, but somewhat dated document "Briefing 10/2006 - Secure web Applications - Development, Installation and Security Testing" created by their predecessor National Infrastructure Security Co-ordination Centre (NISCC), and issued in April 2006. The new document is more compact and focused, and I think I prefer it. Yes of course it is more up-to-date, and while it would be possible to argue why some things are included and not others, these others things tend to be explained further in the references. It's clear there is considerable overlap with information from OWASP and the Microsoft SDL, but I'm sure the reverse is true to an extent too.

It is very encouraging CPNI have taken the time to produce an updated document, but that probably reflects the types of risks facing their audience. I am especially pleased to see the section on infrastructure, since application security cannot be an island on its own. I would say the guidance is probably on the medium-to-heavy weight side of advice, but that is probably appropriate for critical national infrastructure, and the document does discuss threat modelling initially. It might seem overwhelming to some organisations new to the subject, and that might need some help on what to do first.

I think the document could perhaps do with more cross-referencing to additional information resources elsewhere. Yes, documents can always be improved, and I am sure we will find niggles and faults with use, but threats evolve and so does our knowledge.

In 2005, the US Federal Financial Institutions Examination Council (FFIEC) published Authentication in an Internet Banking Environment building on their earlier work on electronic banking. A supplement to the 2005 guidance has now been published.

The 2005 guidance recommended periodic risk assessments so that control mechanisms can be adjusted to respond to changing internal and external threats. The guidance stated that authentication techniques should be appropriate to the risks associated with the product or service, but that single-factor authentication is inadequate. It also recommended building customer awareness, and minimum supervisory expectations for authentication controls relating to high-risk online transactions involving customer information and the movement of funds to other parties.

On 28th June 2011, the FFIEC announced its Supplemental Guidance on Internet Banking Authentication to reinforce the guidance's risk management framework and update the expectations for authentication, layered security and other controls in an increasingly hostile environment.

The supplementary guidance reiterates the need for risk assessments and authentication for higher-risk transactions. It also recommends a layered security approach "since virtually every authentication technique can be compromised". Its recommendations for layered security controls include fraud detection & monitoring, dual device authentication, out-of-band verification, positive pay & debit blocks, transactional limits, activity limits, geo-location IP address reputation monitoring, policies & processes for handling compromised devices and malicious users, monitoring of account maintenance activities and customer education.

So, useful information, and not just for internet banking. The guidance is a good reference source for anyone considering authentication controls for access to more sensitive information.

Thank you to Alexis Fitzgerald for bringing this document to my attention.

Last Friday, I was going to travel to Oxford by express coach, but there had been a road traffic accident which had caused delays to the normally frequent service. After standing in a very long queue at Marble Arch for 10 minutes, and seeing one full coach go past, I decided to catch the train from Paddington instead.

The earliest "off peak" train was at 19:17 hrs and I was intrigued to see on the display boards "TV Entertainment in coach D". Fearing having to watch a football match for the whole journey, I boarded the train just after coach F, expecting that to be coach E. But no, it was coach D. Rather than some sort of large screen at the end of the carriage, I was surprised to see airline-style back-of-seat type displays, and sat down before the rest of the crowd arrived.

I thought a description of the user interface might be of interest to some readers of this blog, especially considering that apart from headset jacks, a screen power button, volume and brightness controls, all interaction is using the touch screen. The interface constraints, need to allow for train vibration and use by untrained public users affect what's possible.

The video on-demand in-train entertainment system is provided by Volo who suggest on their website the mobile payments are integrated by RingGo. I had a look at the safety video which had stop, start and pause buttons available like a conventional media player. The "try me" button let you navigate the juke-box of television shows categorised by genre. The interface felt like "CD multi-media" rather than "web browser". After a while the nag dialogue box popped up in a couple of permutations.

The adjacent passenger seemed to want to go further, but without a valid payment code, their attempt to "log in" was rejected.

The code entry field indicated the required format with underscores and hyphens once touched, and a backspace key appeared once the first character was entered. It did not seem to be possible to enter longer codes than required.

Entry of an invalid "login" code, led to a 20s delay (see countdown timer bottom right) before another attempt could be made, but there did not seem to be any limit on the number of attempts allowed. It is possible the payment code activation period is time limited as well as being for a fixed duration, and they might even be train-specific. Therefore the 20s delay could have been deemed sufficient to protect against guessing unused codes (brute force attack). I would be intrigued to see the risk assessment for the system. The codes could theoretically be tied to seat numbers, but the "special offer" at the cafe didn't seem to suggest you needed to know your own seat number, and that might put off people going for a day's duration to cover their outward and return journeys.

Presumably codes are issued in real time, rather than from prepared lists, so the system will rely upon a semi-continuous communication connection. But it is nice to know if you have a problem, they have "controllers" rather than "helpdesk staff". The code entry screen also reverted back to the main welcome screen after 60s of inactivity.

Without logging in, not much else seemed to be available to the adjacent passenger, but they did manage to get another slightly amusingly worded message screen when randomly tapping on the screen just after starting to watch the safety video once more.

So some interesting constraints on authentication and access control to deal with. Spelling aside, the resolution and related font size does make some screens a little cramped. Having an on-screen keyboard which is only numerical plus A-F does feel a bit techie, and maybe old-fashioned when people are used to much greater flexibility with touch devices. It might indicate the range of values for each code character, and thus reveal something about how it is generated? The touch screen appears to be the only method of user input—perhaps no other way to increase accessibility. Will it be a success? Hard to tell, but very few early-evening commuters seemed to be interested—many had smartphones, game consoles, netbooks, laptops and e-readers to hand, or were asleep.

Perhaps that is why there apparently weren't any power sockets available at the seats.

On the short journey, maybe passengers would be more tempted by live news, travel and weather, or even web browsing if mobile reception was poor, rather than old television shows. But on longer routes it will have greater appeal.

OWASP Podcast Episode 76 with Bill Cheswick is a great discussion about using passwords for authentication and how account lockout affects both usability and user support costs.

I can't recommend the OWASP Podcasts highly enough—Jim Manico's efforts combined with some guest hosts provide a steady stream of fascinating insights from application security experts. In Episode 76, Matt Tesauro interviewed Bill Cheswick (AT&T Labs) following his keynote at OWASP AppSec US 2010.

I recently mentioned password economics and use of popular passwords, and in both of those there is a common theme that password strength alone is not a sufficient measure of authentication security. Bill Cheswick gives a lively discussion of why passwords are often a "pain in the neck" and why they don't have to be like this; in fact "more fun, safer and easier".

He said users are not facing dictionary attacks on the passwords. Instead, they have to deal with key stroke loggers and phishing sites where it doesn't matter how complex your password is, and site owners should consider:

• account lockout after something like 3, 4 or 5 incorrect guesses
• use of a short lockout period
• ensuring that duplicate errors (e.g. typing the same incorrect guess more than once) don't count against the lockout threshold
• allow previously agreed trusted users (e.g. parents, spouses, teacher, carers, someone with power of attorney) to vouch for each other and allow them to create a password reset
• make usernames hard to guess/find
• use a standard authentication service so the logic does not have to be built again each time for new projects.

I would also add "allowing users to optionally view the password as they type", or at least view the last few characters, if they are in a safer location and are not being overlooked.

Web users should be very familiar with challenge-response CAPTCHA test which are also known as Human Interactive Proofs (HIPs).

A lot of time is spent devising CAPTCHAs and trying to break CAPTCHAs in an automated manner. They are used where website owners want to try to ensure a real person is undertaking a transaction such as logging in or submitting content. Now, most web sites don't need to use CAPTCHAs and there can be accessibility problems, but they are often needed for more high-profile sites or when targetted automated attacks are occurring.

A new paper Attacks and Design of Image Recognition CAPTCHAs examines image recognition CAPTCHAs (IRCs) and analyses the effectiveness and security of the schemes considering:

• ease of use by humans
• difficulty to automate
• universality
• reistance to no-effort attacks
• scalability
• use of a secret database.

The authors describe the lessons learned, some fundamental guidelines for IRCs and propose an alternative IRC which relies on recognising an object by exploiting its surrounding context. This requires the user (hopefully a human) to select an item from a set of objects detached from the original image, and then place it back at its original position in the image. Its usability, robustness and copyright issues are discussed.

A few weeks ago I mentioned two new research papers about the use of passwords on website. Another new paper from Microsoft Research and Harvard University discusses how to avoid, and protect web sites from, users selecting popular passwords.

The paper Popularity is Everything: A New Approach to Protecting Passwords from Statistical-Guessing Attacks describes online and offline threats and defences against the sue of common popular passwords.

Password implementation policies can be guided by legacy approaches and various standards, but as mentioned previously, economics plays a large part too. Following a much publicised successful brute force against Twitter accounts, the company increased its password requirements. But rather than forcing passwords to be more complex, they instead took the decision to prevent the use of 370 common passwords. Whilst the list is culturally-biased, due to other breaches, there is similar data from other sites (e.g. here and here). But how does banning popular passwords help, and if the lists of common passwords are known, does this matter?

Firstly I'll mention here a couple of typical online tools for determining password complexity:

• Password meter providing an indication of complexity
• Hammer of God providing an estimate of how long it would take to obtain the password using a brute force attack

Don't put your real passwords into these sites or any other checkers! But these types of tools do not take into account popularity (e.g. '123456') or common manipulations (e.g. is 'P@ssword' really that much more secure than 'password'?). If attackers try popular passwords first (i.e. a dictionary attack), the time to break into a user's account may be much shorter.

The research paper, which does include some mathematics, suggests that simple passwords should be allowed providing they are not subject to statistical guessing attacks and proposes attack detection methods.

Good reading and inspiration for password-based authentication systems. I'm off to the station now, to get a train to Newcastle which was cancelled last night.

Two great papers on web site password security were published this month. We are swamped with passwords and every daily activity is increasingly linked with an online version, which requires users to register to obtain some additional benefits. Every organisation, resource, activity and event encourages us to visit their own website and sign-up.

Firstly, in Where Do [Password] Security Policies Come From?, Dinei Florêncio and Cormac Herley of Microsoft Research discuss the password policies of 75 different web sites, in an effort to determine password strength requirements with other aspects such as size of site, assets protected, number of users and frequency of attacks.

The authors' findings suggest that none of these are the key factors, and in fact some of the largest sites, most attacked and with higher-value assets have the weakest password policies. The authors suggest stronger policies exist where organisations are more insulated from the consequences of poor usability, whereas online retailers and sites that rely on advertising revenues have to compete rigorously for users and traffic. The paper also discusses how strong passwords need to be, and how this is affected also by what attack methods you are considering (e.g. online vs. offline brute-force), and whether other security controls are implemented (e.g. account lock-out).

This idea of considering the whole password environment is taken further in The Password Thicket: Technical and Market Failures in Human Authentication on the Web by Joseph Bonneau and Sören Preibusch at the Cambridge University Computing Laboratory, and presented at this year's Economics of Information Security (WEIS 2010). Their study included 150 web sites looking at password implementations. the study looked more broadly at the protective measures used— not just complexity requirements—but whether these were applied consistently across the site's functionality (e.g. registration/enrolment, log-in/authentication, password change, password reset/recovery, log-out), encryption during transmission, storage of passwords in clear text, inclusion of passwords in emails, as well as protection from brute-force attacks.

The authors found that stricter security in one area was often undermined by weaknesses in another, suggesting that a lack of standards is harming security. The paper also discusses economic interpretations, such as how deploying passwords might be being used to justify collection of marketing data, and how password insecurity can be a negative externality.

Sometimes when I'm out socially and people ask what I do, the conversation progresses to concerns about their own web site. They may have a hobby site, run a micro-business or be a manager or director of a small and medium-sized enterprise (SME)—there's all sorts of great entrepreneurial activity going on.

It is very common for SMEs not to have much time or budget for information security, and the available information can be poor or inappropriate (ISSA-UK, under the guidance of their Director of Research David Lacey, is trying to improve this). But what can SMEs do about their web presence—and it is very unusual not to have a web site, whatever the size of business.

Last week I was asked "Is using <company> okay for taking online payments?" and then "what else should I be doing?". Remember we are discussing protection of the SME's own web site, not protecting its employees from using other sites. If I had no information about the business or any existing web security issues, this is what I recommend checking and doing before anything else:

• Obtain regular backup copies of all data that changes (e.g. databases, logs, uploaded files) and store these securely somewhere other than the host servers. This may typically be daily, but the frequency should be selected based on how often data changes and how much data the SME might be prepared to lose in the event of total server failure.
  • check backup data can read and restored periodically
  • don't forget to securely delete data from old backups when they are no longer required
• Use a network firewall in front of the web site to limit public (unauthenticated user) access to those ports necessary to access the web site. If other services are required remotely, use the firewall to limit from where (e.g. IP addresses) these can be used.
  • keep a record of the firewall configuration up-to-date
  • limit who can make changes to the firewall
• Ensure the host servers are fully patched (e.g. operating system, services, applications and supporting code), check all providers for software updates regularly and allow time for installing these.
  • remove or disable all unnecessary services and other software
  • delete old, unused and backup files from the host servers
• Identify all accounts (log in credentials) that provide server access (not just normal web page access), such as used for transferring files, accessing administrative interfaces (e.g. CMS admin, database and server management/configuration control panels) and using remote desktop. Change the passwords. Keep a record of who has access and remove accounts that are no longer required and enable logging for all access using these accounts.
  • restrict what each account can do as much as possible
  • add restrictions to the use of these accounts (e.g. limit access by IP address, require written approval for use, keep account disabled by default)
• Check that every agreement with third parties that are required to operate the web site are in the organisation's own name. These may include the registration of domain names, SSL certificates, hosting contracts, monitoring services, data feeds, affiliate marketing agreements and service providers such as for address look-up, credit checks and making online payments.
  • ensure the third parties have the organisation's official contact details, and not those of an employee or of the site's developers
  • make note of any renewal dates
• Obtain a copy of everything required for the web site including scripts, static files, configuration settings, source code, account details and encryption keys. Keep this updated with changes as they are made.
  • verify who legally owns the source code, designs, database, photographs, etc.
  • check what other licences affect the web site (e.g. use of open source and proprietary software libraries, database use limitations).

Do what you can, when you can. Once those are done, then:

• Verify the web site and all its components (e.g. web widgets and other third party code/content) does not include common web application vulnerabilities that can be exploited by attackers (e.g. SQL injection, cross-site scripting).
  • use resources such as the OWASP Top Ten and WASC Threat Classification
  • build security requirements into future developments
• Check what obligations the organisation is under to protect business and other people's data such as the Data Protection Act, guidance from regulators, trade organisation rules, agreements with customers and other contracts (e.g. PCI DSS via the acquiring bank).
  • impose security standards and obligations on suppliers and partner organisations
  • keep an eye open for changes to business processes that affect data
• Document (even just some short notes) the steps to rebuild the web site somewhere else, and to transfer all the data and business processes to the new site.
  • include configuration details and information about third-party services required
  • think about what else will need to be done if the web site is unavailable (does it matter, if so what exactly is important?)
• Provide information to the web site's users how to help protect themselves and their data.
  • point them to relevant help such as from GetSafeOnline, CardWatch and Think U Know
  • provide easy methods for them to contact the organisation if they think there is a security or privacy problem
• Monitor web site usage behaviour (e.g. click-through rate, session duration, shopping cart abandonment rate, conversion rate), performance (e.g. uptime, response times) and reputation (e.g. malware, phishing, suspicious applications, malicious links) to gather trend data and identify unusual activity.
  • web server logs are a start, but customised logging is better
  • use reputable online tools (some of which are free) to help.

That's just the basics. So, what would be next for an SME? If the web site is a significant sales/engagement channel, the organisation has multiple web sites, is in a more regulated sector or one that is targetted particularly by criminals (e.g. gaming, betting and financial), takes payments or does other electronic commerce, allows users to add their own content or processes data for someone else, the above is just the start. Those SMEs probably need to be more proactive.

This helps to protect the SME's business information, but also helps to protect the web site users and their information. After all, the users are existing and potential customers, clients and citizens.

Oh, the best response I had to someone when I was explaining my work: "You're an anti-hacker than?". Well, I suppose so, but it's not quite how I'd describe it.

Any comments or suggestions?

The weaknesses of personal knowledge questions in password recovery schemes has been discussed previously, but a recent survey indicates you may also need to take into account the user's sex.

The survey may not have scientific rigour, but mentioning it does give me the chance to include this photograph of Alyson Shotz' sculpture Helix:

The survey on colour names shows that colour names are given differently by "girls and guys". I'm not sure too many girls will use "dusty teal", "blush pink" or "dusty lavender" in their "favourite colour" answer, but the listed colour names are certainly worth checking when verifying or demonstrating the strength of these personal knowledge question systems. Oh, as the survey points out, also include mis-spellings (e.g. of "fuchsia") in the testing lists of colour names. But "drop table statements" shouldn't normally be in your fuzzing list—those are for testing SQL injection.

So beware of weak password reset and recovery schemes. Sometimes they are less secure than the equivalent registration and authentication (logging in) processes.

Britain has some snowy and cold weather at the moment causing difficulty for people getting to shops or going on holiday, and web retailers are likely to be doing brisk trade if they can still deliver before Christmas.

E-commerce sites are often associated with HTTPS (a combination of HTTP with the SSL/TLS cryptographic protocol). There was a time when HTTPS was used only where absolutely necessary due to the additional encryption/decryption overhead it placed on a user's browser (client) and the web application (server). But what's the situation today?

N.B. the padlock symbol or green/blue coloured address bar (depending upon the type of certificate in use) indicating the use of a "secure" web server, does not mean your data is safe; it shows the server identity is verified to a certain extent and that data in transit between your web browser and the web server is probably safe from interception, if it is configured correctly on the server, the certificate has not expired or been revoked, and you can ensure the content you see is on the site the address bar says it is. It also says nothing about how the organisation and partners that handle the data once it has been received by the web server—they might forward it by email, allow third parties to have access to the server, print the data and leave it unprotected, etc.

Almost all web sites have some aspects that should only be accessible over HTTPS. Any sort of data entry form is likely to include personal information and therefore HTTPS should be used to at least protect the confidentiality of the information in transit. User registration, authentication (log in) and any pages that contain confidential information would also be included. Previously, many search engines did not index HTTPS addresses, but since its use was mainly restricted to content protected by some type of authentication and authorisation, this was never much of a concern.

But nowadays, search engines are indexing HTTPS content and a few web sites are only available using HTTPS. Is this a configuration worth following? In a discussion Ivan Ristić described the additional benefits of HTTPS (HTTP over SSL):

Issues

So while there are benefits relating to authenticity and integrity, in addition to confidentiality, and dangers to mixing HTTP and HTTPS on the same site due to badly designed authorisation and session management systems, what other issues are there?

Search engines

The most popular search engine robots no longer discriminate whether the content is HTTP or HTTPS, so this is no longer a concern. I am not aware if any adverse effect on search engine optimisation (SEO), other than the effects of changing from HTTP to HTTPS or vice versa which would have to be managed carefully and appropriate permanent redirects set up (also called 301 redirect due to the HTTP response status code of 301 for "moved permanently").

Note that Google, and apparently Yahoo and Microsoft, support the "rel='canonical'" link element and state it can be used for indicating a preference for HTTP vs HTTPS, or vice versa, when pages are available by both. There is also a setting for this choice in Google webmaster tools if you are a site owner. But be careful with allowing both HTTP and HTTPS access to the same page, since this quite often is implemented in a way that adds vulnerabilities to user authentication and session management.

Resources on the server

The server is affected by two aspects—the increased number of requests (see also resources on the client, below) and the overhead of encryption/decryption/building SSL connection. Intermediate proxies should not cache the content and therefore a greater number of requests is to be expected. The additional resources required to serve content using HTTPS are discussed extensively here and in a research paper, i.e. there will be a performance hit, but whether this is a problem depends on your traffic profile, architecture, server utilisation and site's design.

Server side processes

It is possible that any server-side indexing or reporting systems may not support HTTPS and they may need to be updated or configured to work with the different protocol. If you syndicate data to other systems via XML, RSS or web services, these processes will also need to be checked for compatibility.

Traffic management

Network devices that inspect and route internet traffic must be SSL-aware to be able to read and analyse the content. Most modern devices will be able to support this mode of operation.

Client device support

Some devices (e.g. mobile) may not support HTTPS, or HTTPS may not be allowed through firewalls but this is probably less of an issue now. Check if these are issues with your expected users and the devices your site supports.

Address familiarity

Most people will not recognise (or type in) HTTPS addresses and use the common shorthand of the host name (e.g. www.clerkendweller.com) or an alias (e.g. clerkendweller.com) rather than the protocol followed by the full host name. So this would require a redirect from the HTTP address to the HTTPS one, and for many web sites this will be acceptable. For sites of a more sensitive nature, this would have to be handled carefully to protect any session identifiers and still leaves the user potentially vulnerable to a man in the middle (MITM) attack. These are where the redirect is amended and the user taken to a malicious web site instead. If you can rely on users using only the SSL address, perhaps by bookmarking it, you are on safer territory.

Resources on the client

Again there will be a performance hit on the user's client device (e.g. browser of a desktop computer). Much of the time this will not be a problem unless the device already lacks resources (e.g. a mobile device). Then again, due to the lack of caching, more requests will have to be made directly to the server, creating additional lag to download and build content.

Mixed content

Even if all the content from your own site is sent using HTTPS, you may have embedded content such as:

• client-side web analytics
• advertisements
• news feeds
• widgets
• images, videos, scripts and other content hosted elsewhere.

These must also all be provided using HTTPS, otherwise the benefit of being HTTPS-only will be lost and users may see "mixed content" warning. But this can be a problem as much third-party content is not available using HTTPS (SSL), notably including Google AdSense, Amazon Affiliates and YouTube. However, Google Analytics does support SSL.

Conclusions and further reading

An all-HTTPS web site provides additional security benefits, but user acceptance and server constraints need to be considered in the site's design and architecture decision making processes. The partial, or full, use of HTTPS (SSL) in a web site needs to be considered carefully during design and development to ensure weaknesses that could be exploited are not built in, and then verified by thorough testing. If you have a heavily consumer-focused web site or include third-party content, some of the choices may have to be on the side of ease of use rather than with the lowest security risk.

"Whole site SSL" should be a serious consideration for "green field" web sites, especially where user authentication is required for any part of the content and for sites where phishing is a major risk (e.g. gaming, web mail, banking). User knowledge and acceptance may be difficult until we see the likes of major banks or large consumer-orientated sites (Google Mail, Google Docs, Twitter, Facebook) use this configuration and and display a warning/educational message to people who go to the non HTTPS site, rather than a redirect.