Britain has some snowy and cold weather at the moment causing difficulty for people getting to shops or going on holiday, and web retailers are likely to be doing brisk trade if they can still deliver before Christmas.

E-commerce sites are often associated with HTTPS (a combination of HTTP with the SSL/TLS cryptographic protocol). There was a time when HTTPS was used only where absolutely necessary due to the additional encryption/decryption overhead it placed on a user's browser (client) and the web application (server). But what's the situation today?

N.B. the padlock symbol or green/blue coloured address bar (depending upon the type of certificate in use) indicating the use of a "secure" web server, does not mean your data is safe; it shows the server identity is verified to a certain extent and that data in transit between your web browser and the web server is probably safe from interception, if it is configured correctly on the server, the certificate has not expired or been revoked, and you can ensure the content you see is on the site the address bar says it is. It also says nothing about how the organisation and partners that handle the data once it has been received by the web server—they might forward it by email, allow third parties to have access to the server, print the data and leave it unprotected, etc.

Almost all web sites have some aspects that should only be accessible over HTTPS. Any sort of data entry form is likely to include personal information and therefore HTTPS should be used to at least protect the confidentiality of the information in transit. User registration, authentication (log in) and any pages that contain confidential information would also be included. Previously, many search engines did not index HTTPS addresses, but since its use was mainly restricted to content protected by some type of authentication and authorisation, this was never much of a concern.

But nowadays, search engines are indexing HTTPS content and a few web sites are only available using HTTPS. Is this a configuration worth following? In a discussion Ivan Ristić described the additional benefits of HTTPS (HTTP over SSL):

Issues

So while there are benefits relating to authenticity and integrity, in addition to confidentiality, and dangers to mixing HTTP and HTTPS on the same site due to badly designed authorisation and session management systems, what other issues are there?

Search engines

The most popular search engine robots no longer discriminate whether the content is HTTP or HTTPS, so this is no longer a concern. I am not aware if any adverse effect on search engine optimisation (SEO), other than the effects of changing from HTTP to HTTPS or vice versa which would have to be managed carefully and appropriate permanent redirects set up (also called 301 redirect due to the HTTP response status code of 301 for "moved permanently").

Note that Google, and apparently Yahoo and Microsoft, support the "rel='canonical'" link element and state it can be used for indicating a preference for HTTP vs HTTPS, or vice versa, when pages are available by both. There is also a setting for this choice in Google webmaster tools if you are a site owner. But be careful with allowing both HTTP and HTTPS access to the same page, since this quite often is implemented in a way that adds vulnerabilities to user authentication and session management.

Resources on the server

The server is affected by two aspects—the increased number of requests (see also resources on the client, below) and the overhead of encryption/decryption/building SSL connection. Intermediate proxies should not cache the content and therefore a greater number of requests is to be expected. The additional resources required to serve content using HTTPS are discussed extensively here and in a research paper, i.e. there will be a performance hit, but whether this is a problem depends on your traffic profile, architecture, server utilisation and site's design.

Server side processes

It is possible that any server-side indexing or reporting systems may not support HTTPS and they may need to be updated or configured to work with the different protocol. If you syndicate data to other systems via XML, RSS or web services, these processes will also need to be checked for compatibility.

Traffic management

Network devices that inspect and route internet traffic must be SSL-aware to be able to read and analyse the content. Most modern devices will be able to support this mode of operation.

Client device support

Some devices (e.g. mobile) may not support HTTPS, or HTTPS may not be allowed through firewalls but this is probably less of an issue now. Check if these are issues with your expected users and the devices your site supports.

Address familiarity

Most people will not recognise (or type in) HTTPS addresses and use the common shorthand of the host name (e.g. www.clerkendweller.com) or an alias (e.g. clerkendweller.com) rather than the protocol followed by the full host name. So this would require a redirect from the HTTP address to the HTTPS one, and for many web sites this will be acceptable. For sites of a more sensitive nature, this would have to be handled carefully to protect any session identifiers and still leaves the user potentially vulnerable to a man in the middle (MITM) attack. These are where the redirect is amended and the user taken to a malicious web site instead. If you can rely on users using only the SSL address, perhaps by bookmarking it, you are on safer territory.

Resources on the client

Again there will be a performance hit on the user's client device (e.g. browser of a desktop computer). Much of the time this will not be a problem unless the device already lacks resources (e.g. a mobile device). Then again, due to the lack of caching, more requests will have to be made directly to the server, creating additional lag to download and build content.

Mixed content

Even if all the content from your own site is sent using HTTPS, you may have embedded content such as:

• client-side web analytics
• advertisements
• news feeds
• widgets
• images, videos, scripts and other content hosted elsewhere.

These must also all be provided using HTTPS, otherwise the benefit of being HTTPS-only will be lost and users may see "mixed content" warning. But this can be a problem as much third-party content is not available using HTTPS (SSL), notably including Google AdSense, Amazon Affiliates and YouTube. However, Google Analytics does support SSL.

Conclusions and further reading

An all-HTTPS web site provides additional security benefits, but user acceptance and server constraints need to be considered in the site's design and architecture decision making processes. The partial, or full, use of HTTPS (SSL) in a web site needs to be considered carefully during design and development to ensure weaknesses that could be exploited are not built in, and then verified by thorough testing. If you have a heavily consumer-focused web site or include third-party content, some of the choices may have to be on the side of ease of use rather than with the lowest security risk.

"Whole site SSL" should be a serious consideration for "green field" web sites, especially where user authentication is required for any part of the content and for sites where phishing is a major risk (e.g. gaming, web mail, banking). User knowledge and acceptance may be difficult until we see the likes of major banks or large consumer-orientated sites (Google Mail, Google Docs, Twitter, Facebook) use this configuration and and display a warning/educational message to people who go to the non HTTPS site, rather than a redirect.

Many organisations like to do land grab with domain names by purchasing the same name with different generic top-level domains (e.g. .com, .net, .info), country code top-level domains (e.g. .uk, .es, .fr), and multiple second-level domains (e.g. .co.uk, .org.uk). Then of course there are mis-spellings, similar sounding words, brand names and trademarks.

Well all that leads to complexity, and it's not uncommon for many domains to be aliased to the same site in a way that any of them can be used to access the complete web site.

But it can get especially messy when SSL is enabled on some or all of the site too. Inevitably there end up being certificate warnings. Some organisations should know better. So when I was searching for providers of online and business privacy "seals",

I was very surprised to click on the link to an SSL page which was reported as using an invalid certificate.

Actually the certificate was fine, it just wasn't valid for the .ORG domain. Perhaps they had hoped the wildcard SSL certificate *.trust.com would somehow cater for *.truste.* - no.

Identity theft? Privacy? But apart from these configuration issues, isn't it just very confusing to have many different domains appearing in search engine results? How does this duplicate content affect their search engine ranking? Does it undermine trust in the brand? Should the SSL part of the site be indexed at all? Perhaps. Who makes these decisions? Is it the developers, the person who configured the site or does the business have a viewpoint?

I overheard a (loud) mobile telephone conversation this week in which a marketing manager* was apologising for a problem but they "did not know any of the technicalities". Mmmm, who is accountable? Make it your business to know.

[* Security and technology managers should also understand their organisation's business objectives.]

It's common for access to some web sites to be restricted to users from particular Internet Protocol (IP) addresses. This is usually in addition to some other identification and authentication method. But other IP addresses are often added to this "allow list" and these should not necessarily be trusted in the same way.

In a typical scenario, a web site hosted on the internet that is used to administer another web application might be restricted to the company's own IP addresses. Then the developers say they need to check something on the live site, or another server needs to index the content, or someone wants to work from home for a while, or the site needs to be demonstrated at a client's location. All these additional IP addresses are added to the "allow list". These restrictions may be being applied at a network firewall, traffic management system, at the web server, in the application itself, in intrusion detection systems or in log analytical software, or in many of these. These are difficult to manage and in time there will be many IP addresses that no-one knows why they are allowed unless they are carefully documented, and subject to a fixed time limit when they are confirmed again by an appropriate person or removed. These extra addresses are quite often hard for someone else to guess.

However, there is another area where IP addresses are added to "allow lists", and this is for remote monitoring and testing services. These might be checking uptime, response times, content changes, HTML validation or security testing. The service providers publish the IP addresses of the source systems so that companies can specifically allow access to their web sites. Since the number of these services is relatively small, it's not too difficult to find which one might give access to areas of a web site or web application that the public (and malicious people) should not be able to get to. The particular danger here is that the IP addresses might be excluded from monitoring and logging, and therefore even a diligent web site manager might not realise for example the uptime monitoring service is making unusual, or excessive, requests.

Although it is not likely a malicious person is using this "trusted" address unless routing has been compromised as well, problems can go undetected, from what might seem to be a legitimate source. The IP address may have been typed incorrectly, or worse, the restrictions/exceptions may not have been implemented correctly allowing more addresses to have the privileged access than intended. Not logging a user's session is privileged access.

Allow traffic through, but be very specific what is allowed and monitor what's going on. Review all the exceptions periodically. Be especially careful about anything that bypasses authentication (such as allowing a search engine to crawl restricted-access content) on an otherwise public site.

An article on eConsultancy.com discussing the demise of the tr.im URL shortening service mentioned some information security concerns:

There are good discussions concerning URL shorteners on Joshua Schachter's blog, the Mashable social media guide and the Security Retentive corporate information security blog. These will be a little technical for many people but essentially, apart from the privacy aspect of collecting data, other things might be worth worrying about more (you need to check this!). If you can, create your own aliasing/redirecting system on your own domain. But make sure it can only be used to redirect to a limited number of your own website's URLs.

Interestingly, tr.im is another example where organisations may have been using a service for which they had no contract, service level agreement or rights when the service was withdrawn. Be careful what you reply upon.

Do people understand tab grouping in Internet Explorer version 8 (IE8)? This was a new idea introduced to collect together and identify tabs originating from the same source.

Note this isn't grouping based on the web site (domain name)—it's grouping from which other tab you clicked. The group colour is selected randomly, so yesterday's blue might be today's yellow.

My concern with tab grouping is not the concept, but the use of colour. Not because of the accessibility difficulty that some people may have distinguishing between colours, which is partially addressed by Microsoft in a tab naming convention, but because the colours can lead to confusion about SSL certificates. Take this example with four tabs open, the first tab selected and a current Extended Validation (EV) SSL certificate in use:

1. Bank No X online banking login (domain A / SSL)
2. Bank No X information page (domain B / non-SSL) opened using a link on page 1
3. Bank No Y home page (domain C / non-SSL)
4. Bank No Y business banking login (domain D / SSL) opened using a link on page 2

What does green mean? How about if we have an invalid SSL certificate and the tab group is green:

Confusing? Yes. It can lead to this type of misunderstanding:

Helping people to identify and reject invalid SSL certificates is important—IE8 users are being put at a disadvantage by Microsoft. I'd like to see tab grouping turned off by default for now, and some other indicator of tab groups used instead of distracting colour-coding. Perhaps, since new tabs are always opened adjacent to their 'parent', even something as simple as this mock-up might suffice:

Is this easier to understand? What do you think?

Hardening the underlying server operating system is an important fundamental task to help protect your web applications.

For example, the Payment Card Industry Data Security Standard (PCIDSS) requirement 2.2 states:

Two United States organisations producing guidance in this field are:

• Security Configuration Benchmarks from the Center for Internet Security (CIS)
• Security Content Automation Protocol (SCAP) Checklists from the National Institute of Standards and Technology (NIST) Computer Security Division (CSD)

These are detailed documents and all the recommendations may not be appropriate for your own situation.

Version 3 of the Open Web Application Security Project (OWASP) Testing Guide has been released after a 6-month period of addition, enhancement and review.

The OWASP Testing Guide is an ideal reference for both developers and testers—version 2 was fantastic, and this new version is even better. The testing framework now covers 66 controls and, like in the previous version, each control has a brief summary and is described in detail followed by black box (no additional knowledge) and grey/gray box (partial knowledge) testing methods and examples where appropriate.

The controls and testing methods are fully referenced to provide additional guidance and explanation.

The controls are grouped into ten categories, including new separate categories "Authorization" and "Configuration Management". I'm especially pleased to see the latter broken out on its own, since even a perfectly coded application can have vulnerabilities introduced during deployment and changes to the application.

The OWASP Testing Guide now also includes a "best practice" penetration testing framework and a "low level" penetration testing guide that describes techniques for testing most common web application and web service security issues. More information is available on the Testing Project pages.

Setting up a web server incorrectly can be difficult to change later. Isolating the operating system from web site files and other data using separate partitions or physical devices needs to be done during server commissioning.

It would be usual to have at least three partitions on a typical web server:

• The operating system
• The web site files (scripts and static files such as images and style sheets)
• Server logs

This allows you to restrict permissions - so that if the web site is compromised, it is harder to access the operating system files, and to ensure that logs don't grow excessively and use up all the available space.

Your own data - the database and perhaps other files - should be stored seperately. If possible this should be on other servers, but if not, on a separate partition.

If you allow any user uploaded content, you should also consider storing this on another separate partition, and in any case, never in sub-directories of the web site root. This is to prevent direct access to possibly malicious file content by directly requesting the address in a browser and to ensure the files are stored in an area with limited permissions.

A common way malicious hackers try to obtain information about how your website works is to generate errors and see what is displayed. It is particularly important to stop these giving away information that might help someone break into your web site, but equally you should make sure these pages are not advertising someone else's products and services.

When a web site is developed and then set up on the web server(s), it is possible to define customised error pages for all sorts of unusual events like application errors, internal server errors and, the one most people will recognise, not found. The latter is sent back when a page or other file's address is requested but does not exist. The web server sends a response status code of 404 which means "not found" and the text from whichever document has been set for this. By default many sites will return text which gives away the operating system and web server software:

But some hosting companies rather naughtily hope to gain revenue from people typing the wrong address, following an old link or clicking a dead link on another page. Instead of showing a page from your own web site, or the default web server message, they display an advert for themselves and/or adverts for other web sites. These may neither be what you expect, nor want to be associated with. Here is one from a UK limited company's web site:

Check you own web site by typing an address like:

http://[your host name]/123456doesnotexist

or something similar. Hopefully, you will see a page in the style of your own web site with an apologetic message, and not anything else. If not, speak to your developers (or hosting company) and ask them to "add custom error pages for all possible web server errors" and make sure they are your own design. Also ask them to "ensure errors return the correct HTTP response status codes" - this is especially important for correct indexing by search engines.

Here's an example showing how to do it correctly from the British Library:

If you have more than one domain, host name, or also have an HTTPS address, check them all separately. This advertising could also exist on domains you have purchased, but are not currently using for a site.