Britain has some snowy and cold weather at the moment causing difficulty for people getting to shops or going on holiday, and web retailers are likely to be doing brisk trade if they can still deliver before Christmas.

E-commerce sites are often associated with HTTPS (a combination of HTTP with the SSL/TLS cryptographic protocol). There was a time when HTTPS was used only where absolutely necessary due to the additional encryption/decryption overhead it placed on a user's browser (client) and the web application (server). But what's the situation today?

N.B. the padlock symbol or green/blue coloured address bar (depending upon the type of certificate in use) indicating the use of a "secure" web server, does not mean your data is safe; it shows the server identity is verified to a certain extent and that data in transit between your web browser and the web server is probably safe from interception, if it is configured correctly on the server, the certificate has not expired or been revoked, and you can ensure the content you see is on the site the address bar says it is. It also says nothing about how the organisation and partners that handle the data once it has been received by the web server—they might forward it by email, allow third parties to have access to the server, print the data and leave it unprotected, etc.

Almost all web sites have some aspects that should only be accessible over HTTPS. Any sort of data entry form is likely to include personal information and therefore HTTPS should be used to at least protect the confidentiality of the information in transit. User registration, authentication (log in) and any pages that contain confidential information would also be included. Previously, many search engines did not index HTTPS addresses, but since its use was mainly restricted to content protected by some type of authentication and authorisation, this was never much of a concern.

But nowadays, search engines are indexing HTTPS content and a few web sites are only available using HTTPS. Is this a configuration worth following? In a discussion Ivan Ristić described the additional benefits of HTTPS (HTTP over SSL):

Issues

So while there are benefits relating to authenticity and integrity, in addition to confidentiality, and dangers to mixing HTTP and HTTPS on the same site due to badly designed authorisation and session management systems, what other issues are there?

Search engines

The most popular search engine robots no longer discriminate whether the content is HTTP or HTTPS, so this is no longer a concern. I am not aware if any adverse effect on search engine optimisation (SEO), other than the effects of changing from HTTP to HTTPS or vice versa which would have to be managed carefully and appropriate permanent redirects set up (also called 301 redirect due to the HTTP response status code of 301 for "moved permanently").

Note that Google, and apparently Yahoo and Microsoft, support the "rel='canonical'" link element and state it can be used for indicating a preference for HTTP vs HTTPS, or vice versa, when pages are available by both. There is also a setting for this choice in Google webmaster tools if you are a site owner. But be careful with allowing both HTTP and HTTPS access to the same page, since this quite often is implemented in a way that adds vulnerabilities to user authentication and session management.

Resources on the server

The server is affected by two aspects—the increased number of requests (see also resources on the client, below) and the overhead of encryption/decryption/building SSL connection. Intermediate proxies should not cache the content and therefore a greater number of requests is to be expected. The additional resources required to serve content using HTTPS are discussed extensively here and in a research paper, i.e. there will be a performance hit, but whether this is a problem depends on your traffic profile, architecture, server utilisation and site's design.

Server side processes

It is possible that any server-side indexing or reporting systems may not support HTTPS and they may need to be updated or configured to work with the different protocol. If you syndicate data to other systems via XML, RSS or web services, these processes will also need to be checked for compatibility.

Traffic management

Network devices that inspect and route internet traffic must be SSL-aware to be able to read and analyse the content. Most modern devices will be able to support this mode of operation.

Client device support

Some devices (e.g. mobile) may not support HTTPS, or HTTPS may not be allowed through firewalls but this is probably less of an issue now. Check if these are issues with your expected users and the devices your site supports.

Address familiarity

Most people will not recognise (or type in) HTTPS addresses and use the common shorthand of the host name (e.g. www.clerkendweller.com) or an alias (e.g. clerkendweller.com) rather than the protocol followed by the full host name. So this would require a redirect from the HTTP address to the HTTPS one, and for many web sites this will be acceptable. For sites of a more sensitive nature, this would have to be handled carefully to protect any session identifiers and still leaves the user potentially vulnerable to a man in the middle (MITM) attack. These are where the redirect is amended and the user taken to a malicious web site instead. If you can rely on users using only the SSL address, perhaps by bookmarking it, you are on safer territory.

Resources on the client

Again there will be a performance hit on the user's client device (e.g. browser of a desktop computer). Much of the time this will not be a problem unless the device already lacks resources (e.g. a mobile device). Then again, due to the lack of caching, more requests will have to be made directly to the server, creating additional lag to download and build content.

Mixed content

Even if all the content from your own site is sent using HTTPS, you may have embedded content such as:

• client-side web analytics
• advertisements
• news feeds
• widgets
• images, videos, scripts and other content hosted elsewhere.

These must also all be provided using HTTPS, otherwise the benefit of being HTTPS-only will be lost and users may see "mixed content" warning. But this can be a problem as much third-party content is not available using HTTPS (SSL), notably including Google AdSense, Amazon Affiliates and YouTube. However, Google Analytics does support SSL.

Conclusions and further reading

An all-HTTPS web site provides additional security benefits, but user acceptance and server constraints need to be considered in the site's design and architecture decision making processes. The partial, or full, use of HTTPS (SSL) in a web site needs to be considered carefully during design and development to ensure weaknesses that could be exploited are not built in, and then verified by thorough testing. If you have a heavily consumer-focused web site or include third-party content, some of the choices may have to be on the side of ease of use rather than with the lowest security risk.

"Whole site SSL" should be a serious consideration for "green field" web sites, especially where user authentication is required for any part of the content and for sites where phishing is a major risk (e.g. gaming, web mail, banking). User knowledge and acceptance may be difficult until we see the likes of major banks or large consumer-orientated sites (Google Mail, Google Docs, Twitter, Facebook) use this configuration and and display a warning/educational message to people who go to the non HTTPS site, rather than a redirect.

This weekend the clocks change as we revert from British Summer Time (BST) to Greenwich Mean Time (GMT) at 02:00 BST on Sunday 25 October 2009 and the clocks go back, giving an extra hour.

What does this mean for web site security? Does running 01:00 to 02:00 twice matter? Well some brave web application owners will be disabling their systems like this online bank:

And, I don't think it's just being done as a finale to the current Energy Saving Week. Most people, quite rightly, won't be taking this rather severe step. Another millennium bug anyone? The date/time should be considered rather like other untrusted user input. Most problems will probably fall into the "business logic" category such as:

• Failure of time-based logic where dates are being compared.
• Assumptions of uniqueness in time-stamped output (e.g. by a single-threaded process).
• Running tasks again leading to possible:
  • loss of data due to overwriting
  • duplication of exports or emails
  • creation of inaccuracies in management information.
• Chronological ordering anomalies leading to other faults.

It's not just banks and other financial organisations that may have difficulties.

The time change may expose some other vulnerabilities that only exist at changeover time and/or during the next overlap hour.

• Circumvention of brute force attacks on user authentication mechanisms.
• Increased risk due to extension of a session's validity where local time is recorded.
• Failure in data validation routines for time-related comparisons.
• Incubated vulnerabilities where a time-related aspect causes the attack to be possible.
• Denial of service due to extension of account lock-out.
• Using time as a loop counter.
• Additional errors caused by any of the above leading to information leakage.

Recording the offset of local time to GMT/UTC and synchronisation should certainly be done, but may not resolve the time overlap issues. The effects on long-running "saga" requests might be especially difficult to determine. Time dependencies need to be specified and considered through the development lifecycle. Perhaps the bank is right after all?

Sometimes you read someone else's blog post and think "I wish I had written that".

Well, Jeremiah Grossman has blogged All about Website Password Policies which neatly sums up password policies for typical web sites and web applications.

The suggested policy is not for your online bank, but the sort of web sites most developers have to work on day-to-day. He also has a good brief explanation of how to store passwords in a hash digest form (but also read the associated comments).

I do think, however, that there is never a reason to have passwords stored in plain text—password recovery mechanisms can be built that do not need to send the actual password. But also, I agree with the comment that passwords truncation should be done with care and that allowing users to have longer pass phrases (containing space characters) can be beneficial. Let the user decide on what they are happy with, above the minimum standards.

My own password related posts are Is Password Complexity Too Complex to Implement? and Guessable Usernames and Passwords.

It's common for access to some web sites to be restricted to users from particular Internet Protocol (IP) addresses. This is usually in addition to some other identification and authentication method. But other IP addresses are often added to this "allow list" and these should not necessarily be trusted in the same way.

In a typical scenario, a web site hosted on the internet that is used to administer another web application might be restricted to the company's own IP addresses. Then the developers say they need to check something on the live site, or another server needs to index the content, or someone wants to work from home for a while, or the site needs to be demonstrated at a client's location. All these additional IP addresses are added to the "allow list". These restrictions may be being applied at a network firewall, traffic management system, at the web server, in the application itself, in intrusion detection systems or in log analytical software, or in many of these. These are difficult to manage and in time there will be many IP addresses that no-one knows why they are allowed unless they are carefully documented, and subject to a fixed time limit when they are confirmed again by an appropriate person or removed. These extra addresses are quite often hard for someone else to guess.

However, there is another area where IP addresses are added to "allow lists", and this is for remote monitoring and testing services. These might be checking uptime, response times, content changes, HTML validation or security testing. The service providers publish the IP addresses of the source systems so that companies can specifically allow access to their web sites. Since the number of these services is relatively small, it's not too difficult to find which one might give access to areas of a web site or web application that the public (and malicious people) should not be able to get to. The particular danger here is that the IP addresses might be excluded from monitoring and logging, and therefore even a diligent web site manager might not realise for example the uptime monitoring service is making unusual, or excessive, requests.

Although it is not likely a malicious person is using this "trusted" address unless routing has been compromised as well, problems can go undetected, from what might seem to be a legitimate source. The IP address may have been typed incorrectly, or worse, the restrictions/exceptions may not have been implemented correctly allowing more addresses to have the privileged access than intended. Not logging a user's session is privileged access.

Allow traffic through, but be very specific what is allowed and monitor what's going on. Review all the exceptions periodically. Be especially careful about anything that bypasses authentication (such as allowing a search engine to crawl restricted-access content) on an otherwise public site.

A friend forwarded me a wine offer from his bank. But the email from his bank included additional information meant to verify the authenticity of the sender.

The bank included the last part of his postcode in the body of the message as well as his title and family name. If those were meant to be for verification purposes, why did it suggest he forward the email to someone else?

Yes, the forwarded email has all the verification details embedded in it. No, it's not the username and password, but it's everything someone would need to construct a phishing email that would be very hard to distinguish from a real one.

This has the feel of using a security measure as a marketing gimmick. Mixing marketing emails and those necessary for servicing a bank account is a difficult balance, but this is way off the mark.

Last week I highlighted Jakob Nielsen's advice on Password Masking, which I believe to be misguided.

Nielsen's Alerbox generated a wide-ranging discussion—in particular there are some excellent points of view at Web Security Mailing List, and a comment on this blog from Robert Campbell. I was surprised that "security guru" Bruce Schneier seemed to agree with Nielsen and this has been widely reported such as Usability and Security Gurus Agree That Masked Passwords Should Go.

Bruce Schneier now admits he was "probably" wrong The Pros and Cons of Password Masking. Good, let's hope that becomes as widely reported.

I was surprised to see the latest advice Stop Password Masking from Jakob Nielsen.

Jakob Nielsen's has raised many usability topics in his Alertbox but he is not always correct. Although I used to read his column with an open, somewhat sceptical, mind I gave up some time ago*.

No, password masking isn't just some legacy design artefact. Like other design choices relating to user identification and authentication, these have a significant impact on user trust and data privacy, confidentiality and integrity. It is wrong to suggest that masking should be removed by default. By all means inform users of the risks and let them choose to display the characters being typed, but don't have this status set by default. More-and-more web sites are being accessed away from home, and being overseen by other people or surveillance equipment is commonplace almost everywhere.

On e-commerce sites, the need to log in can often be removed completely, or made non-compulsory. Too often security controls are applied for other reasons, such as to generate information for sales and marketing reports, rather than to ease the purchasing process. For more critical data, the use of authentication mechanisms other than static passwords should be considered.

* I stopped reading Alertbox after Jakob Nielsen became very defensive about his training material only being available on DVD and not VHS tape, as many people had requested. His argument was that DVD players were so cheap, people should upgrade. Yet at the time, he was promoting the idea that web sites would render in all browsers—including old legacy ones.

Update 7th July 2009: Password Masking Update.

Many websites are updated using File Transfer Protocol (FTP). Don't do it.

A discussion thread How Do You Store FTP Login Information For Your Clients? highlighted what common practices are, but almost entirely missed the issues of transfer of login credentials over unencrypted channels, privileged access to the whole of the server, account sharing, password and user management.

It's no surprise that some of the most serious hacks are suspected of being undertaken using compromised FTP accounts.

FTP is not an option. Ask your hosting company or systems staff to disable FTP services and block all traffic to/from your web servers on TCP ports 20 and 21, at your network firewall.

Authentication bypass is when someone can get to restricted-access resources (data, files, functionality, etc) without approval.

As an example, my gym has issued me with a membership card. Each time I visit, I have to run the card through a reader that checks it is still valid, not already used by someone inside the gym and then lets me enter. My photograph is also displayed briefly to the reception staff so they can check if the person with the card is the same as the member.

But my gym has recently introduced some security boxes—small lockable stores for valuable items such as wallets, mobile phones and keys—located in sight of reception staff. To use these, you give your card to reception, they issue you with a key for a box and manually override the entrance gate.

It's possible that in this procedure the are no longer checking the card is valid, or that the stored photograph matches the person with the card. This would allow unauthorised access to the gym by anyone with a found, stolen or borrowed membership card. Similarly, walking through an open fire-escape door would achieve this.

So, why is this an issue for web application security? Many web sites start out with everything being freely available. It's usually not long before user registration, newsletter sign-up or customer-only requirements turn up. These are often added on, without proper consideration of how the authorisation processes will be applied consistently and thoroughly across all resources. It is not uncommon to find a login screen that provides access to, say, file downloads that can be accessed without signing in.

So the first step should be to identify all resources which have to be protected, and what people or groups can access them, under what conditions.

Enforcing complexity is one aspect of a good password policy. But web applications often implement this poorly.

Yesterday I had to change a password on a web application provided by an information security vendor. You'd think they'd have this sorted. It began promisingly:

• the page was only available on a page transmitted over secure HTTP
• the page was only available to currently authenticated users
• there was an explanation of the minimum password complexity requirements
• the form asked for the current password
• the form asked for a new password and provided a visual indicator of the password complexity (length and mix of characters)
• the new password wasn't allowed to be blank(!)
• confirmation of the new password was sought
• password fields were initially all blank and masked as you typed
• the submission was sent by the post method over secure HTTP
• the session was ended (forced log out) and the account locked after multiple failures to change the password correctly

Also, the form had some helpful client-side JavaScript validation—but did not rely this and, correctly, was also undertaken once submitted to the web server.

The client-side validation even included an indicator of password complexity. But I was surprised when my "great" new password was rejected apparently because it did not contain solely ASCII characters:

Well, I can assure you I did use ASCII characters. In this case the client-side and server-side validation are not identical. This suggests that there may be contradictory requirements in the specification or the validation has been implemented incorrectly, and there was inadequate testing and verification undertaken.

This is a usability error, and a potential security vulnerability.

A small issue, but it demonstrates how easy it is to have faults in applications. This may not have an impact on the application's security, but it would be an area a malicious user would explore further to see if the weakness could be exploited.