Two great papers on web site password security were published this month. We are swamped with passwords and every daily activity is increasingly linked with an online version, which requires users to register to obtain some additional benefits. Every organisation, resource, activity and event encourages us to visit their own website and sign-up.

Firstly, in Where Do [Password] Security Policies Come From?, Dinei Florêncio and Cormac Herley of Microsoft Research discuss the password policies of 75 different web sites, in an effort to determine password strength requirements with other aspects such as size of site, assets protected, number of users and frequency of attacks.

The authors' findings suggest that none of these are the key factors, and in fact some of the largest sites, most attacked and with higher-value assets have the weakest password policies. The authors suggest stronger policies exist where organisations are more insulated from the consequences of poor usability, whereas online retailers and sites that rely on advertising revenues have to compete rigorously for users and traffic. The paper also discusses how strong passwords need to be, and how this is affected also by what attack methods you are considering (e.g. online vs. offline brute-force), and whether other security controls are implemented (e.g. account lock-out).

This idea of considering the whole password environment is taken further in The Password Thicket: Technical and Market Failures in Human Authentication on the Web by Joseph Bonneau and Sören Preibusch at the Cambridge University Computing Laboratory, and presented at this year's Economics of Information Security (WEIS 2010). Their study included 150 web sites looking at password implementations. the study looked more broadly at the protective measures used— not just complexity requirements—but whether these were applied consistently across the site's functionality (e.g. registration/enrolment, log-in/authentication, password change, password reset/recovery, log-out), encryption during transmission, storage of passwords in clear text, inclusion of passwords in emails, as well as protection from brute-force attacks.

The authors found that stricter security in one area was often undermined by weaknesses in another, suggesting that a lack of standards is harming security. The paper also discusses economic interpretations, such as how deploying passwords might be being used to justify collection of marketing data, and how password insecurity can be a negative externality.

Sometimes when I'm out socially and people ask what I do, the conversation progresses to concerns about their own web site. They may have a hobby site, run a micro-business or be a manager or director of a small and medium-sized enterprise (SME)—there's all sorts of great entrepreneurial activity going on.

It is very common for SMEs not to have much time or budget for information security, and the available information can be poor or inappropriate (ISSA-UK, under the guidance of their Director of Research David Lacey, is trying to improve this). But what can SMEs do about their web presence—and it is very unusual not to have a web site, whatever the size of business.

Last week I was asked "Is using <company> okay for taking online payments?" and then "what else should I be doing?". Remember we are discussing protection of the SME's own web site, not protecting its employees from using other sites. If I had no information about the business or any existing web security issues, this is what I recommend checking and doing before anything else:

• Obtain regular backup copies of all data that changes (e.g. databases, logs, uploaded files) and store these securely somewhere other than the host servers. This may typically be daily, but the frequency should be selected based on how often data changes and how much data the SME might be prepared to lose in the event of total server failure.
  • check backup data can read and restored periodically
  • don't forget to securely delete data from old backups when they are no longer required
• Use a network firewall in front of the web site to limit public (unauthenticated user) access to those ports necessary to access the web site. If other services are required remotely, use the firewall to limit from where (e.g. IP addresses) these can be used.
  • keep a record of the firewall configuration up-to-date
  • limit who can make changes to the firewall
• Ensure the host servers are fully patched (e.g. operating system, services, applications and supporting code), check all providers for software updates regularly and allow time for installing these.
  • remove or disable all unnecessary services and other software
  • delete old, unused and backup files from the host servers
• Identify all accounts (log in credentials) that provide server access (not just normal web page access), such as used for transferring files, accessing administrative interfaces (e.g. CMS admin, database and server management/configuration control panels) and using remote desktop. Change the passwords. Keep a record of who has access and remove accounts that are no longer required and enable logging for all access using these accounts.
  • restrict what each account can do as much as possible
  • add restrictions to the use of these accounts (e.g. limit access by IP address, require written approval for use, keep account disabled by default)
• Check that every agreement with third parties that are required to operate the web site are in the organisation's own name. These may include the registration of domain names, SSL certificates, hosting contracts, monitoring services, data feeds, affiliate marketing agreements and service providers such as for address look-up, credit checks and making online payments.
  • ensure the third parties have the organisation's official contact details, and not those of an employee or of the site's developers
  • make note of any renewal dates
• Obtain a copy of everything required for the web site including scripts, static files, configuration settings, source code, account details and encryption keys. Keep this updated with changes as they are made.
  • verify who legally owns the source code, designs, database, photographs, etc.
  • check what other licences affect the web site (e.g. use of open source and proprietary software libraries, database use limitations).

Do what you can, when you can. Once those are done, then:

• Verify the web site and all its components (e.g. web widgets and other third party code/content) does not include common web application vulnerabilities that can be exploited by attackers (e.g. SQL injection, cross-site scripting).
  • use resources such as the OWASP Top Ten and WASC Threat Classification
  • build security requirements into future developments
• Check what obligations the organisation is under to protect business and other people's data such as the Data Protection Act, guidance from regulators, trade organisation rules, agreements with customers and other contracts (e.g. PCI DSS via the acquiring bank).
  • impose security standards and obligations on suppliers and partner organisations
  • keep an eye open for changes to business processes that affect data
• Document (even just some short notes) the steps to rebuild the web site somewhere else, and to transfer all the data and business processes to the new site.
  • include configuration details and information about third-party services required
  • think about what else will need to be done if the web site is unavailable (does it matter, if so what exactly is important?)
• Provide information to the web site's users how to help protect themselves and their data.
  • point them to relevant help such as from GetSafeOnline, CardWatch and Think U Know
  • provide easy methods for them to contact the organisation if they think there is a security or privacy problem
• Monitor web site usage behaviour (e.g. click-through rate, session duration, shopping cart abandonment rate, conversion rate), performance (e.g. uptime, response times) and reputation (e.g. malware, phishing, suspicious applications, malicious links) to gather trend data and identify unusual activity.
  • web server logs are a start, but customised logging is better
  • use reputable online tools (some of which are free) to help.

That's just the basics. So, what would be next for an SME? If the web site is a significant sales/engagement channel, the organisation has multiple web sites, is in a more regulated sector or one that is targetted particularly by criminals (e.g. gaming, betting and financial), takes payments or does other electronic commerce, allows users to add their own content or processes data for someone else, the above is just the start. Those SMEs probably need to be more proactive.

This helps to protect the SME's business information, but also helps to protect the web site users and their information. After all, the users are existing and potential customers, clients and citizens.

Oh, the best response I had to someone when I was explaining my work: "You're an anti-hacker than?". Well, I suppose so, but it's not quite how I'd describe it.

Any comments or suggestions?

I've mentioned input validation and knowing your data previously, but a vulnerability came up in a recent project regarding email addresses. People make many different assumptions about what might be a valid email address format.

Don't! As this recent post states, go to the source i.e. RFC 3696. Inform your developers what types and formats they should allow for each input field and how mismatches should be handled—and verify these.

The e-commerce project I had been working on had some client-side validation for the email address field in a check-out registration form and this excluded lots of valid possibilities; it was using the regular expression "\w+([-+.']\w+)*@\w+([-.]\w+)*\.\w+([-.]\w+)*". This might prevent some people becoming customers. Is this classified as a security vulnerability? Normally not, but it could affect data integrity and will affect the availability to some users. But, it is an indicator of possible data validation problems elsewhere. In fact we found the server-side validation for the same form data had different constraints to the client-side (browser) checks, and yes, plenty of other input validation problems. Security problems are often related to revenue problems.

And remember, it's not just Latin characters in domain names you need to worry about now. From last week, you might begin to see unexpected problems with users who have email accounts using domains related to the United Arab Emirates, Saudi Arabia, the Russian Federation and Egypt.

The weaknesses of personal knowledge questions in password recovery schemes has been discussed previously, but a recent survey indicates you may also need to take into account the user's sex.

The survey may not have scientific rigour, but mentioning it does give me the chance to include this photograph of Alyson Shotz' sculpture Helix:

The survey on colour names shows that colour names are given differently by "girls and guys". I'm not sure too many girls will use "dusty teal", "blush pink" or "dusty lavender" in their "favourite colour" answer, but the listed colour names are certainly worth checking when verifying or demonstrating the strength of these personal knowledge question systems. Oh, as the survey points out, also include mis-spellings (e.g. of "fuchsia") in the testing lists of colour names. But "drop table statements" shouldn't normally be in your fuzzing list—those are for testing SQL injection.

So beware of weak password reset and recovery schemes. Sometimes they are less secure than the equivalent registration and authentication (logging in) processes.

Thinking of creating a mobile phone application for your business? A major privacy failure with an iPhone application has been reported on the Zero Day blog and is a useful case study.

All iPhone apps have to be approved by Apple to "protect consumer privacy, safeguard children from inappropriate content, and avoid applications that degrade the core experience of the iPhone". But the application Quip from Addy Mobile Inc provides provided unlimited photo texting with the slogan "Why pay more for MMS? Don't you pay enough for your iPhone already?". Well, many of the application's users are paying for it now.

It seems the images (typically photos) were stored on a publicly-accessible web site, with the only access "control" being a random directory (folder) name five characters long—something that is easily iterated through to find photos and breach their customers' privacy. What makes it worse is that many of these messages and images were also turning up in public search engine results leaking sensitive information.

I was unable to find any privacy notice or privacy policy from the company:

The cached search results indicate the images are being stored using Amazon Web Services (AWS) S3. This is not a cloud computing specific issue. It could just as well be on a web site hosted by the company itself. The Quip web site (http://www.quiptxt.com), Quip message site (http://www.quiptxt.com) and Quip S3 image repository (http://quipimg.s3.amazonaws.com) are all currently offline. The comany issued a statement via Reddit. As the Zero Day blog says, these vulnerabilities would have been trivial to fix and protect the confidentiality of users' message text and photos. Using any of common sense, threat modelling, risk assessment or security verification should have identified the problems. I'm sure lawyers in the US will be circling. Since Apple approved the application, let's hope they are ready in case the lawyers knock on their door as well.

Addy Mobile Inc may itself be unavailable soon.

Almost every sale, citizen enquiry and support request now seems to lead to being asked to complete an online customer survey. Almost without exception, the user experience, privacy and security of these online customer surveys are worse than the service being asked to comment on. Here are a couple of customer surveys I was asked to complete last week.

Problems with using SSL, such as shown above, do occur but more often than not people are asked to submit personal identifiable information and other forms of personal data without the use of SSL. Bad layout, poorly designed questions, missing privacy notices and improper validation are extremely common. Many forms have mis-configured web sites that give away sensitive information about how the site and server are set up when they don't work:

And on Saturday I was given a coupon given at a shop's physical checkout to provide feedback on how they did with the chance of winning an iPod or cash for doing so. Yesterday I typed in the URL from the coupon, entered the required store code, and... that was the end of that marketing exercise:

I didn't time out as the message suggested, unless you have less than 5s to answer one question. Perhaps there is only one custom error page for all server-side errors, or the wrong error page is assigned? Points for hiding internal error messages, but still a failure.

Is 3/3 customer surveys tried in the last fortnight broken just bad luck? Or does it indicate a poor standard of such efforts? One of these is an international consumer brand, another a major UK High Street retailer and the other, a medium-sized business services company. I can't quite remember the the previous customer survey to these three, but I think it may have been a salary/skill survey. That had poorly thought-out questions and although it didn't obviously fail, it did ask me to log in on submission. So I'm not sure if that meant my efforts had been saved or not.

Do all these problems and errors mean the data from other people's forms that were successfully submitted (if any) are less valid? I can imagine management decisions are being made as a result of the survey feedback (if not, why waste everyone's time?). What is the effect on data quality? It could be that some forms fail when a particular answer is selected or left blank—this could be important marketing knowledge, and if no responses include the particular option it may be incorrectly assumed no-one selected it. The management decisions will be being based on poor data.

Perhaps part of the problem is that customer surveys are often managed, operated and hosted by third parties due to the ease of implementation. But "easy" doesn't mean it meets your own organisation's standards, or general good practice. You are still accountable for the web issues and it's your organisation's reputation that will be affected detrimentally.

Good design, privacy and security impact assessments, thorough testing and verification are required like any other other addition to a web site. Analytics should be used to track survey users through forms and this data combined with server logs of access and errors generated by the web server. Prove your marketing data are valid before you use it in business decisions.

Testing a web site is important, and smoke/regression testing would normally be undertaken each time the web site's code is updated or extended. But what about third party code and data on your web site? I was using a shop's store locator and it wrongly identified the two closest store locations to Farringdon Station in Clerkenwell, EC1M. Both Euston Station NW1 and Canary Wharf E14 are further away than the third result, Cheapside.

Intrigued, I zoomed in on the map and the ordering of the first two results swapped. And the data point for Canary Wharf seems to be located in the centre of the City of London, instead of its actual location downriver on the Isle of Dogs. The data point for the first result wasn't even displayed.

So is this a third party problem, or something else? Well, without investigating further you can't really tell, but it's the concept that matters.

This inaccuracy is worrying for a number of reasons:

• Customers may have difficulty locating shops.
• It undermines customer confidence and therefore trust in the brand.
• It indicates a lack of care in the web site's development and may put off online shoppers.
• It could indicate the presence of a security vulnerability which could be exploited to damage the site, its data or its users.
• The same geo-location code may be used for other internal calculations such as marketing data processing that affect business decisions.

If you are including functionality or data from third parties, you need to know when that system or service is updated. This notification requirement should be built into contracts. In this case, the data being returned may be inaccurate, formatted in an unxpected manner or be exposing a fault in your own processing of the data. Undertaking input validation on the data provided by the third party and output validation on what you are about to send back to the web site user need to test for reasonableness as well as more technical checks. Why not cross-check that the first result's postcode is closer than the second and the third?

It may just be a problem with the company's own data, but that's even more worrying.

This weekend I'm going to a fancy dress party with a theme of film actors and characters, and was looking for Toy Story Buzz Lightyear gear.

One price comparison website listed a Buzz Lightyear toy at £999999.00 which seems rather high. It was either made from something very valuable, or an error.

How do you validate pricing information being added and also when displayed? Who/what can alter pricing, discounts, additional charges, tax rates, etc on your e-commerce web site? What approval processes are in place? Who is accountable? All data should be checked for reasonableness, type, format, characters, length, encoding, etc. Reporting/alerting on changes might also help detect this type of issue.

And, make sure your terms state what happens if a price is "incorrect" and exactly when a contract is formed.

"Voucher Codes: assets or liabilities?" was a question asked on Creative Match recently.

Codes providing discounts during the checkout process on e-commerce sites can be an incentive to attract shoppers and increase their spending but also can affect revenues if they are used by people who would have bought anyway.

Misuse by real shoppers is certainly a concern, but voucher codes can sometimes easily be abused if their implementation, operation and lifecycle are not considered carefully. Unlike in the real world where paper coupons may be difficult to forge and can be cancelled by collecting or marking, online voucher codes can be harder to control and expire.

Plan ahead, don't create vouchers code schemes in a rush.

It's common for access to some web sites to be restricted to users from particular Internet Protocol (IP) addresses. This is usually in addition to some other identification and authentication method. But other IP addresses are often added to this "allow list" and these should not necessarily be trusted in the same way.

In a typical scenario, a web site hosted on the internet that is used to administer another web application might be restricted to the company's own IP addresses. Then the developers say they need to check something on the live site, or another server needs to index the content, or someone wants to work from home for a while, or the site needs to be demonstrated at a client's location. All these additional IP addresses are added to the "allow list". These restrictions may be being applied at a network firewall, traffic management system, at the web server, in the application itself, in intrusion detection systems or in log analytical software, or in many of these. These are difficult to manage and in time there will be many IP addresses that no-one knows why they are allowed unless they are carefully documented, and subject to a fixed time limit when they are confirmed again by an appropriate person or removed. These extra addresses are quite often hard for someone else to guess.

However, there is another area where IP addresses are added to "allow lists", and this is for remote monitoring and testing services. These might be checking uptime, response times, content changes, HTML validation or security testing. The service providers publish the IP addresses of the source systems so that companies can specifically allow access to their web sites. Since the number of these services is relatively small, it's not too difficult to find which one might give access to areas of a web site or web application that the public (and malicious people) should not be able to get to. The particular danger here is that the IP addresses might be excluded from monitoring and logging, and therefore even a diligent web site manager might not realise for example the uptime monitoring service is making unusual, or excessive, requests.

Although it is not likely a malicious person is using this "trusted" address unless routing has been compromised as well, problems can go undetected, from what might seem to be a legitimate source. The IP address may have been typed incorrectly, or worse, the restrictions/exceptions may not have been implemented correctly allowing more addresses to have the privileged access than intended. Not logging a user's session is privileged access.

Allow traffic through, but be very specific what is allowed and monitor what's going on. Review all the exceptions periodically. Be especially careful about anything that bypasses authentication (such as allowing a search engine to crawl restricted-access content) on an otherwise public site.