Input validation and output encoding are key aspects of web application security. This morning I've just been using a data search on a website and came across this when no results were found:

I imagine "escape(escape())" is some attempt at preventing cross-site scripting (XSS), but implemented correctly. Interestingly, when search results are found there are several "-->" appearing. Another indicator of output encoding problems.

It doesn't have to be difficult - read the OWASP XSS Prevention Cheat Sheet.

Last month I described the ability to track users sessions with browser data. A recent posting on IT Law in Ireland highlighted a series of blog posts elsewhere that give further insight into what is possible.

Well, I just got round to reading them properly. The posts on Freedom to Tinker by David Robinson and Harlan Yu are:

• Identifying John Doe: It might be easier than you think
• What Third Parties Know About John Doe
• The Traceability of an Anonymous Online Comment

The conclusion? It is possible to trace and identify individuals easier than you may think. We are dropping evidence like dead skin cells as we traverse the internet. Fact or fiction? Well the US Defense Advanced Research Projects Agency (DARPA) are taking it seriously with a recent call for research into cyber genetics, cyber anthropology and cyber physiology in its Cyber Genome Program. DARPA hopes to develop advanced methods to fingerprint or identify the origins of a cyber attacks by examining digital artifacts, and presumably other criminal activities utilising computer technology.

Getting a bit more down to earth, web site owners need to consider what information is being gathered and why, ensure this is legal, check that consent is implied or has been explicitly given for the purposes and what monitoring and analysis is performed on the data. It could be easy for system developers to carried away with tracking and tagging. Contracts with third parties should state clearly what the expectations are about the security and privacy of information, to protect web site users (employees, customers, clients, citizens) and the business.

It seems we are spending about ten times as much on infrastructure security as application security.

This is the conclusion of Jeremiah Grossman in his post on Infrastructure vs. Application Security Spending using some broad calculations and estimates from the available information. Have a look at the referenced sources and comments, and keep an eye open for the next web application security spending benchmark report.

What is your organisation spending on these two aspects?

Testing a web site is important, and smoke/regression testing would normally be undertaken each time the web site's code is updated or extended. But what about third party code and data on your web site? I was using a shop's store locator and it wrongly identified the two closest store locations to Farringdon Station in Clerkenwell, EC1M. Both Euston Station NW1 and Canary Wharf E14 are further away than the third result, Cheapside.

Intrigued, I zoomed in on the map and the ordering of the first two results swapped. And the data point for Canary Wharf seems to be located in the centre of the City of London, instead of its actual location downriver on the Isle of Dogs. The data point for the first result wasn't even displayed.

So is this a third party problem, or something else? Well, without investigating further you can't really tell, but it's the concept that matters.

This inaccuracy is worrying for a number of reasons:

• Customers may have difficulty locating shops.
• It undermines customer confidence and therefore trust in the brand.
• It indicates a lack of care in the web site's development and may put off online shoppers.
• It could indicate the presence of a security vulnerability which could be exploited to damage the site, its data or its users.
• The same geo-location code may be used for other internal calculations such as marketing data processing that affect business decisions.

If you are including functionality or data from third parties, you need to know when that system or service is updated. This notification requirement should be built into contracts. In this case, the data being returned may be inaccurate, formatted in an unxpected manner or be exposing a fault in your own processing of the data. Undertaking input validation on the data provided by the third party and output validation on what you are about to send back to the web site user need to test for reasonableness as well as more technical checks. Why not cross-check that the first result's postcode is closer than the second and the third?

It may just be a problem with the company's own data, but that's even more worrying.

This weekend I'm going to a fancy dress party with a theme of film actors and characters, and was looking for Toy Story Buzz Lightyear gear.

One price comparison website listed a Buzz Lightyear toy at £999999.00 which seems rather high. It was either made from something very valuable, or an error.

How do you validate pricing information being added and also when displayed? Who/what can alter pricing, discounts, additional charges, tax rates, etc on your e-commerce web site? What approval processes are in place? Who is accountable? All data should be checked for reasonableness, type, format, characters, length, encoding, etc. Reporting/alerting on changes might also help detect this type of issue.

And, make sure your terms state what happens if a price is "incorrect" and exactly when a contract is formed.

"Voucher Codes: assets or liabilities?" was a question asked on Creative Match recently.

Codes providing discounts during the checkout process on e-commerce sites can be an incentive to attract shoppers and increase their spending but also can affect revenues if they are used by people who would have bought anyway.

Misuse by real shoppers is certainly a concern, but voucher codes can sometimes easily be abused if their implementation, operation and lifecycle are not considered carefully. Unlike in the real world where paper coupons may be difficult to forge and can be cancelled by collecting or marking, online voucher codes can be harder to control and expire.

Plan ahead, don't create vouchers code schemes in a rush.

On Thursday the UK Government's Central Office of Information (COI) is hosting an event about auditing government websites aimed at government agencies (EAs) and non-departmental public bodies (NDPBs) that have a deadline looming in April 2010.

Web site quality and value concerns were raised in a National Audit Office report on Government on the Internet: Progress in Delivering Information and Services Online in published in July 2007 and recommendation made in the Public Accounts Committee (PAC) Sixteenth Report. Along with their other web standards and guidelines, the COI has issued standards relating to costs, usage and quality. Version 1.1 of TG126, November 2009, on measuring website quality describes three requirements for measuring and auditing website usage:

The benefits of web site auditing were described last year by Adam Bailin on the Digigov blog.

It is very encouraging that the COI are developing standards to improve quality and value. Apart from usage measurement and audit, the quality requirements cover the topics of domain names, usability, accessibility, archival, browser testing, web site map, cost monitoring and web site closure (disposal).

But there are some areas that are not represented in these standards. A glance at something like ISO 9126 indicates other important software quality. A starting point would be to monitor some privacy and security metrics.

And of course, I'd like to see some government requiring some standards for security, which unlike privacy, has a much less firm legal guidance and regulation (for privacy these are the Data Protection Act 1998 and the Information Commissioner's Office). The most well-developed standard for web site security verification is the Application Security Verification Standard (ASVS) from the Open Web Application Security Project. It's free to download and use, and perhaps this can be incorporated or referenced by future government standards and other software security assurance programmes.

The next Open Web Application Security Project (OWASP) London meeting is this week.

The London chapter meeting is on Thursday 14 January 2010 in EC1. Everyone is welcome, but you need to register first (free).

There will be talks on Top Ten Deployment Mistakes That Render SSL Useless by Ivan Ristić and Using Selenium to Hold State for Web Application Penetration Testing by Yiannis Pavlosoglou, who recently joined the OWASP Global Industry Committee.

Unfortunately I am unable to attend the meeting but hope to read the presentations afterwards.

It's that time again when organisations want your opinion on how well they've been performed in the previous year. I'm often quite happy to provide feedback, but sometimes prefer this to be anonymous.

Many of these surveys use third party services to display the form and collect the data, but very few have privacy notices or details you would expect/require on a corporate web site (e.g. the company registration number and registered address). But I can't remember one of these where the option for complete anonymity is provided. The links often include a code in the address URL (an argument name with a value) which I suspect identifies the recipient.

I copied the link and removed the value from the argument and was presented with a horrible error message which gives clues as to what the site is doing. Perhaps a database query that can't find NULL as an identifier?

Then I tried it with the argument completely removed.

So very sloppily put together. At least hide this script error message. Better still, inform me why this code is necessary and give options for anonymity. After all, some feedback is better than none.

If you're going to use third party services, ask them what security verification has been undertaken and to what standard. They should be able to provide details of a recent independent audit if they don't allow their customers to verify security themselves.

Britain has some snowy and cold weather at the moment causing difficulty for people getting to shops or going on holiday, and web retailers are likely to be doing brisk trade if they can still deliver before Christmas.

E-commerce sites are often associated with HTTPS (a combination of HTTP with the SSL/TLS cryptographic protocol). There was a time when HTTPS was used only where absolutely necessary due to the additional encryption/decryption overhead it placed on a user's browser (client) and the web application (server). But what's the situation today?

N.B. the padlock symbol or green/blue coloured address bar (depending upon the type of certificate in use) indicating the use of a "secure" web server, does not mean your data is safe; it shows the server identity is verified to a certain extent and that data in transit between your web browser and the web server is probably safe from interception, if it is configured correctly on the server, the certificate has not expired or been revoked, and you can ensure the content you see is on the site the address bar says it is. It also says nothing about how the organisation and partners that handle the data once it has been received by the web server—they might forward it by email, allow third parties to have access to the server, print the data and leave it unprotected, etc.

Almost all web sites have some aspects that should only be accessible over HTTPS. Any sort of data entry form is likely to include personal information and therefore HTTPS should be used to at least protect the confidentiality of the information in transit. User registration, authentication (log in) and any pages that contain confidential information would also be included. Previously, many search engines did not index HTTPS addresses, but since its use was mainly restricted to content protected by some type of authentication and authorisation, this was never much of a concern.

But nowadays, search engines are indexing HTTPS content and a few web sites are only available using HTTPS. Is this a configuration worth following? In a discussion Ivan Ristić described the additional benefits of HTTPS (HTTP over SSL):

Issues

So while there are benefits relating to authenticity and integrity, in addition to confidentiality, and dangers to mixing HTTP and HTTPS on the same site due to badly designed authorisation and session management systems, what other issues are there?

Search engines

The most popular search engine robots no longer discriminate whether the content is HTTP or HTTPS, so this is no longer a concern. I am not aware if any adverse effect on search engine optimisation (SEO), other than the effects of changing from HTTP to HTTPS or vice versa which would have to be managed carefully and appropriate permanent redirects set up (also called 301 redirect due to the HTTP response status code of 301 for "moved permanently").

Note that Google, and apparently Yahoo and Microsoft, support the "rel='canonical'" link element and state it can be used for indicating a preference for HTTP vs HTTPS, or vice versa, when pages are available by both. There is also a setting for this choice in Google webmaster tools if you are a site owner. But be careful with allowing both HTTP and HTTPS access to the same page, since this quite often is implemented in a way that adds vulnerabilities to user authentication and session management.

Resources on the server

The server is affected by two aspects—the increased number of requests (see also resources on the client, below) and the overhead of encryption/decryption/building SSL connection. Intermediate proxies should not cache the content and therefore a greater number of requests is to be expected. The additional resources required to serve content using HTTPS are discussed extensively here and in a research paper, i.e. there will be a performance hit, but whether this is a problem depends on your traffic profile, architecture, server utilisation and site's design.

Server side processes

It is possible that any server-side indexing or reporting systems may not support HTTPS and they may need to be updated or configured to work with the different protocol. If you syndicate data to other systems via XML, RSS or web services, these processes will also need to be checked for compatibility.

Traffic management

Network devices that inspect and route internet traffic must be SSL-aware to be able to read and analyse the content. Most modern devices will be able to support this mode of operation.

Client device support

Some devices (e.g. mobile) may not support HTTPS, or HTTPS may not be allowed through firewalls but this is probably less of an issue now. Check if these are issues with your expected users and the devices your site supports.

Address familiarity

Most people will not recognise (or type in) HTTPS addresses and use the common shorthand of the host name (e.g. www.clerkendweller.com) or an alias (e.g. clerkendweller.com) rather than the protocol followed by the full host name. So this would require a redirect from the HTTP address to the HTTPS one, and for many web sites this will be acceptable. For sites of a more sensitive nature, this would have to be handled carefully to protect any session identifiers and still leaves the user potentially vulnerable to a man in the middle (MITM) attack. These are where the redirect is amended and the user taken to a malicious web site instead. If you can rely on users using only the SSL address, perhaps by bookmarking it, you are on safer territory.

Resources on the client

Again there will be a performance hit on the user's client device (e.g. browser of a desktop computer). Much of the time this will not be a problem unless the device already lacks resources (e.g. a mobile device). Then again, due to the lack of caching, more requests will have to be made directly to the server, creating additional lag to download and build content.

Mixed content

Even if all the content from your own site is sent using HTTPS, you may have embedded content such as:

• client-side web analytics
• advertisements
• news feeds
• widgets
• images, videos, scripts and other content hosted elsewhere.

These must also all be provided using HTTPS, otherwise the benefit of being HTTPS-only will be lost and users may see "mixed content" warning. But this can be a problem as much third-party content is not available using HTTPS (SSL), notably including Google AdSense, Amazon Affiliates and YouTube. However, Google Analytics does support SSL.

Conclusions and further reading

An all-HTTPS web site provides additional security benefits, but user acceptance and server constraints need to be considered in the site's design and architecture decision making processes. The partial, or full, use of HTTPS (SSL) in a web site needs to be considered carefully during design and development to ensure weaknesses that could be exploited are not built in, and then verified by thorough testing. If you have a heavily consumer-focused web site or include third-party content, some of the choices may have to be on the side of ease of use rather than with the lowest security risk.

"Whole site SSL" should be a serious consideration for "green field" web sites, especially where user authentication is required for any part of the content and for sites where phishing is a major risk (e.g. gaming, web mail, banking). User knowledge and acceptance may be difficult until we see the likes of major banks or large consumer-orientated sites (Google Mail, Google Docs, Twitter, Facebook) use this configuration and and display a warning/educational message to people who go to the non HTTPS site, rather than a redirect.