This year I have mentioned web application security programmes, how software vulnerability testing recommended risk-based, application security programmes and generalised results from a survey about web application security programs.

But what are enterprises doing in real life and what are the issues? During the second day of OWASP AppSec Research 2010, Michael Craigue of Dell presented on Secure Application Development for the Enterprise: Practical, Real-World Tips. Although I missed it, people who did attend this track were enthusiastic about it and the video recording has now been published. I watched it last weekend.

Michael described Dell's 10-strong Global Information Security Services group and how it works with 3,000-5,000 developers in internal teams and how their appsec work is built on a published and maintained secure application development standard. Some of the problems encountered at Dell were platform diversity, security expert retention, the need to develop self-help documentation for the low and medium risk projects, lack of good metrics around security awareness training, high overhead of conventional threat modelling and the need to build security into the development lifecyle slowly, and in a business-focused manner.

At Dell, the project risk is calculated from ten factors including data classification, compliance requirements, whether it is externally facing, and the security knowledge of the development team. Interestingly, in the final questions from the audience, Michael mentioned Dell are using Open SAMM to identify gaps, measure how well their security programme is performing and to focus improvement efforts. Even projects that the group does not get involved with directly, are subject to quality checks and audit such as using Control Self Assessments (CSAs), which look for the artifacts required in the self-help documentation, even for low-risk applications.

There is another description of how software assurance practices at Ford in 2009, and recently published on US DHS's best practices web site Build Security In. The Ford programme is quite different. Every application security programme is unique because every organisation's culture, application and acceptance of risk is different.

What is yours like?

Sometimes when I'm out socially and people ask what I do, the conversation progresses to concerns about their own web site. They may have a hobby site, run a micro-business or be a manager or director of a small and medium-sized enterprise (SME)—there's all sorts of great entrepreneurial activity going on.

It is very common for SMEs not to have much time or budget for information security, and the available information can be poor or inappropriate (ISSA-UK, under the guidance of their Director of Research David Lacey, is trying to improve this). But what can SMEs do about their web presence—and it is very unusual not to have a web site, whatever the size of business.

Last week I was asked "Is using <company> okay for taking online payments?" and then "what else should I be doing?". Remember we are discussing protection of the SME's own web site, not protecting its employees from using other sites. If I had no information about the business or any existing web security issues, this is what I recommend checking and doing before anything else:

• Obtain regular backup copies of all data that changes (e.g. databases, logs, uploaded files) and store these securely somewhere other than the host servers. This may typically be daily, but the frequency should be selected based on how often data changes and how much data the SME might be prepared to lose in the event of total server failure.
  • check backup data can read and restored periodically
  • don't forget to securely delete data from old backups when they are no longer required
• Use a network firewall in front of the web site to limit public (unauthenticated user) access to those ports necessary to access the web site. If other services are required remotely, use the firewall to limit from where (e.g. IP addresses) these can be used.
  • keep a record of the firewall configuration up-to-date
  • limit who can make changes to the firewall
• Ensure the host servers are fully patched (e.g. operating system, services, applications and supporting code), check all providers for software updates regularly and allow time for installing these.
  • remove or disable all unnecessary services and other software
  • delete old, unused and backup files from the host servers
• Identify all accounts (log in credentials) that provide server access (not just normal web page access), such as used for transferring files, accessing administrative interfaces (e.g. CMS admin, database and server management/configuration control panels) and using remote desktop. Change the passwords. Keep a record of who has access and remove accounts that are no longer required and enable logging for all access using these accounts.
  • restrict what each account can do as much as possible
  • add restrictions to the use of these accounts (e.g. limit access by IP address, require written approval for use, keep account disabled by default)
• Check that every agreement with third parties that are required to operate the web site are in the organisation's own name. These may include the registration of domain names, SSL certificates, hosting contracts, monitoring services, data feeds, affiliate marketing agreements and service providers such as for address look-up, credit checks and making online payments.
  • ensure the third parties have the organisation's official contact details, and not those of an employee or of the site's developers
  • make note of any renewal dates
• Obtain a copy of everything required for the web site including scripts, static files, configuration settings, source code, account details and encryption keys. Keep this updated with changes as they are made.
  • verify who legally owns the source code, designs, database, photographs, etc.
  • check what other licences affect the web site (e.g. use of open source and proprietary software libraries, database use limitations).

Do what you can, when you can. Once those are done, then:

• Verify the web site and all its components (e.g. web widgets and other third party code/content) does not include common web application vulnerabilities that can be exploited by attackers (e.g. SQL injection, cross-site scripting).
  • use resources such as the OWASP Top Ten and WASC Threat Classification
  • build security requirements into future developments
• Check what obligations the organisation is under to protect business and other people's data such as the Data Protection Act, guidance from regulators, trade organisation rules, agreements with customers and other contracts (e.g. PCI DSS via the acquiring bank).
  • impose security standards and obligations on suppliers and partner organisations
  • keep an eye open for changes to business processes that affect data
• Document (even just some short notes) the steps to rebuild the web site somewhere else, and to transfer all the data and business processes to the new site.
  • include configuration details and information about third-party services required
  • think about what else will need to be done if the web site is unavailable (does it matter, if so what exactly is important?)
• Provide information to the web site's users how to help protect themselves and their data.
  • point them to relevant help such as from GetSafeOnline, CardWatch and Think U Know
  • provide easy methods for them to contact the organisation if they think there is a security or privacy problem
• Monitor web site usage behaviour (e.g. click-through rate, session duration, shopping cart abandonment rate, conversion rate), performance (e.g. uptime, response times) and reputation (e.g. malware, phishing, suspicious applications, malicious links) to gather trend data and identify unusual activity.
  • web server logs are a start, but customised logging is better
  • use reputable online tools (some of which are free) to help.

That's just the basics. So, what would be next for an SME? If the web site is a significant sales/engagement channel, the organisation has multiple web sites, is in a more regulated sector or one that is targetted particularly by criminals (e.g. gaming, betting and financial), takes payments or does other electronic commerce, allows users to add their own content or processes data for someone else, the above is just the start. Those SMEs probably need to be more proactive.

This helps to protect the SME's business information, but also helps to protect the web site users and their information. After all, the users are existing and potential customers, clients and citizens.

Oh, the best response I had to someone when I was explaining my work: "You're an anti-hacker than?". Well, I suppose so, but it's not quite how I'd describe it.

Any comments or suggestions?

Like many other document and files on your web site, PDFs can leak information in more than one way. Apart from the normal content, there is often meta data, information from previous versions and sometimes, links to internal resources.

On Wednesday, the Financial Services Authority (FSA) reported they had acquired a list of 38,000 names, addresses and telephone number that boiler rooms were using to target potential investors in worthless shares. The FSA wrote a letter to all the people listed. They also published the letter's outline format as a PDF. Unfortunately four enabled hyperlinks in the document do not reference the www.fsa.gov.uk website as intended, but instead a file on someone's computer, probably at the FSA.

Oh dear. Nothing too serious this time, but everything that is published should be checked for validity before release, and verified after publication. This should include all information, not just the normal visual content. The FSA should know better. Publishing to print (e.g. the letters) removes some of this additional information where publishing to an electronic format (e.g. PDF) doesn't always. All publishing should follow standard procedures and approvals processes should include checks for additional information.

Embedded data may leak business information (e.g. previous changes, authors' comments, file paths, account names, intellectual property) or personal data (e.g. location data, names), and possibly give malicious users information that will help them exploit the organisation's systems.

Also, some good news for the FSA, it has survived the change of government.

I've mentioned input validation and knowing your data previously, but a vulnerability came up in a recent project regarding email addresses. People make many different assumptions about what might be a valid email address format.

Don't! As this recent post states, go to the source i.e. RFC 3696. Inform your developers what types and formats they should allow for each input field and how mismatches should be handled—and verify these.

The e-commerce project I had been working on had some client-side validation for the email address field in a check-out registration form and this excluded lots of valid possibilities; it was using the regular expression "\w+([-+.']\w+)*@\w+([-.]\w+)*\.\w+([-.]\w+)*". This might prevent some people becoming customers. Is this classified as a security vulnerability? Normally not, but it could affect data integrity and will affect the availability to some users. But, it is an indicator of possible data validation problems elsewhere. In fact we found the server-side validation for the same form data had different constraints to the client-side (browser) checks, and yes, plenty of other input validation problems. Security problems are often related to revenue problems.

And remember, it's not just Latin characters in domain names you need to worry about now. From last week, you might begin to see unexpected problems with users who have email accounts using domains related to the United Arab Emirates, Saudi Arabia, the Russian Federation and Egypt.

An article about the upcoming new regime for the classification and labelling of chemicals reminded me to write about software assurance (i.e. software security) labelling (and since web sites are software). From 1 December 2010, the UN Globally Harmonized System of Classification and Labelling of Chemicals (GHS) comes into force, implemented in Europe by the Regulation (EC) No 1272/2008 of the European Parliament and of the Council of 16 December 2008 on classification, labelling and packaging of substances and mixtures (CLP), amending and repealing Directives 67/548/EEC and 1999/45/EC, and amending Regulation (EC) No 1907/2006.

CLP implications and guidelines are explained by the UK Health and Safety Executive (HSE) but are defined fully in the UN's documentation. The headline chemical labelling indicates the potential damage/harm that can occur, rather than the content/properties of the agent. I like this "impact" approach. Nutritional labelling on the other hand generally tends to focus on ingredients and their properties, but some food labelling is also beginning to attempt to classify low/medium/high fat/saturates/sugars/salt levels, which is more akin to the impact approach.

Jeff Williams (CEO of Aspect Security and Chair of OWASP Foundation) proposed a Software Facts label five years ago at OWASP Appsec Europe. This would be similar to appliance energy usage labels, food nutrition facts label, material safety data sheets or laser safety classes. That idea was taken up by NIST and the Software Assurance Consortium (SwAC) to develop another proposal.

Comment here and here around the same time in 2005 describes some of the challenges. Indeed many more aspects of the software development lifecycle impact upon the creation of secure software. But simplicity is needed in the presentation of such information—perhaps some high-level impact related indicators augmented by the more detailed information for different audiences (e.g. users, operators, administrators, system achitects). SwAC's version seems to be somewhat aimed at software development teams, instead of people in end user organisations, especially those involved with procurement decisions. Whilst some people will want to know the data behind a classification, most businesses and consumers will need something more akin to the CLP headline labels relating to business (or personal) impact as a starting point for their decisions.

• How dangerous is this software?
• How reliable is it?
• How does it affect privacy?
• How does the IT environment affect these?
• How are these affected by changing the default settings?

This a big challenge. Just specifying the privacy practices for a web site can be complex. ENISA's Common Assurance Maturity Model (CAMM) project is trying to define how cloud service providers can be compared to allow users to make informed decisions about the risks. Perhaps that project will develop into some form of labelling scheme, or at least provide ways for typical consumers of the services to determine their own risks as simply as possible.

I don't know the status of the SwAC project but will now make the effort to find out.

Almost every sale, citizen enquiry and support request now seems to lead to being asked to complete an online customer survey. Almost without exception, the user experience, privacy and security of these online customer surveys are worse than the service being asked to comment on. Here are a couple of customer surveys I was asked to complete last week.

Problems with using SSL, such as shown above, do occur but more often than not people are asked to submit personal identifiable information and other forms of personal data without the use of SSL. Bad layout, poorly designed questions, missing privacy notices and improper validation are extremely common. Many forms have mis-configured web sites that give away sensitive information about how the site and server are set up when they don't work:

And on Saturday I was given a coupon given at a shop's physical checkout to provide feedback on how they did with the chance of winning an iPod or cash for doing so. Yesterday I typed in the URL from the coupon, entered the required store code, and... that was the end of that marketing exercise:

I didn't time out as the message suggested, unless you have less than 5s to answer one question. Perhaps there is only one custom error page for all server-side errors, or the wrong error page is assigned? Points for hiding internal error messages, but still a failure.

Is 3/3 customer surveys tried in the last fortnight broken just bad luck? Or does it indicate a poor standard of such efforts? One of these is an international consumer brand, another a major UK High Street retailer and the other, a medium-sized business services company. I can't quite remember the the previous customer survey to these three, but I think it may have been a salary/skill survey. That had poorly thought-out questions and although it didn't obviously fail, it did ask me to log in on submission. So I'm not sure if that meant my efforts had been saved or not.

Do all these problems and errors mean the data from other people's forms that were successfully submitted (if any) are less valid? I can imagine management decisions are being made as a result of the survey feedback (if not, why waste everyone's time?). What is the effect on data quality? It could be that some forms fail when a particular answer is selected or left blank—this could be important marketing knowledge, and if no responses include the particular option it may be incorrectly assumed no-one selected it. The management decisions will be being based on poor data.

Perhaps part of the problem is that customer surveys are often managed, operated and hosted by third parties due to the ease of implementation. But "easy" doesn't mean it meets your own organisation's standards, or general good practice. You are still accountable for the web issues and it's your organisation's reputation that will be affected detrimentally.

Good design, privacy and security impact assessments, thorough testing and verification are required like any other other addition to a web site. Analytics should be used to track survey users through forms and this data combined with server logs of access and errors generated by the web server. Prove your marketing data are valid before you use it in business decisions.

Input validation and output encoding are key aspects of web application security. This morning I've just been using a data search on a website and came across this when no results were found:

I imagine "escape(escape())" is some attempt at preventing cross-site scripting (XSS), but implemented correctly. Interestingly, when search results are found there are several "-->" appearing. Another indicator of output encoding problems.

It doesn't have to be difficult - read the OWASP XSS Prevention Cheat Sheet.

The first report on the state of software security has been published by Veracode who provide a cloud-based application risk management service.

The data presented in State of Software Security Report - The Intractable Problem of Insecure Software are interesting, it relates to both web application (40%) and non-web application (60%) software but spans in-house developed, commercial, open source and outsourced, developed in .NET, C/C++ and Java. But don't get bogged down in the data. I'd recommend that everyone responsible for web development, or who commissions or operates a web site, read the executive summary. I was quite surprised that backdoors (a method of bypassing normal authentication) are still a significant issue: "Open Source projects have ... fewer potential backdoors than commercial or outsourced software". This is why "V13 Malicious Code Search Verification Requirements" appears in the Application Security Verification Standard.

There are seven recommendations, which are in summary:

• Implement a comprehensive, risk-based, application security programme.
• Implement security acceptance criteria for third-party suppliers.
• Test code from outsourced, commercial and open source suppliers as rigorously as internally developed code.
• Verification of C/C++ code must not be ignored and it is likely to be present in many applications.
• Implement specific developer training on security.
• Learn from organisations in higher-risk sectors such as finance and government.
• Ensure security requirements are built into outsourced software development.

Some good advice there. The report provides a fuller description and gives the background to these recommendations.

I guess there must be much more detailed data available to Veracode than is presented in this "Volume 1". Perhaps Volume 2 will look at trends over time, but I'd also like to see:

• Breakdown of root causes (e.g. unvalidated user input).
• Breakdown of how the vulnerabilities had been fixed when the software was re-tested (e.g. parameterized queries implemented).
• What proportion of faulty code was identical — found in more than one application (i.e. copied/duplicated from elsewhere).
• Details of any secure development lifecycle in use by suppliers and their level of maturity with these.

Hopefully this type of aggregated data can be shared under Veracode's terms of service and agreements with individual customers. The software suppliers included in Veracode's analysis are likely to exclude customer organisations who do not have the knowledge or resources to have their code analysed. It would be an interesting research project to test a selection of applications developed by other suppliers to see how they compare.

Testing a web site is important, and smoke/regression testing would normally be undertaken each time the web site's code is updated or extended. But what about third party code and data on your web site? I was using a shop's store locator and it wrongly identified the two closest store locations to Farringdon Station in Clerkenwell, EC1M. Both Euston Station NW1 and Canary Wharf E14 are further away than the third result, Cheapside.

Intrigued, I zoomed in on the map and the ordering of the first two results swapped. And the data point for Canary Wharf seems to be located in the centre of the City of London, instead of its actual location downriver on the Isle of Dogs. The data point for the first result wasn't even displayed.

So is this a third party problem, or something else? Well, without investigating further you can't really tell, but it's the concept that matters.

This inaccuracy is worrying for a number of reasons:

• Customers may have difficulty locating shops.
• It undermines customer confidence and therefore trust in the brand.
• It indicates a lack of care in the web site's development and may put off online shoppers.
• It could indicate the presence of a security vulnerability which could be exploited to damage the site, its data or its users.
• The same geo-location code may be used for other internal calculations such as marketing data processing that affect business decisions.

If you are including functionality or data from third parties, you need to know when that system or service is updated. This notification requirement should be built into contracts. In this case, the data being returned may be inaccurate, formatted in an unxpected manner or be exposing a fault in your own processing of the data. Undertaking input validation on the data provided by the third party and output validation on what you are about to send back to the web site user need to test for reasonableness as well as more technical checks. Why not cross-check that the first result's postcode is closer than the second and the third?

It may just be a problem with the company's own data, but that's even more worrying.

This weekend I'm going to a fancy dress party with a theme of film actors and characters, and was looking for Toy Story Buzz Lightyear gear.

One price comparison website listed a Buzz Lightyear toy at £999999.00 which seems rather high. It was either made from something very valuable, or an error.

How do you validate pricing information being added and also when displayed? Who/what can alter pricing, discounts, additional charges, tax rates, etc on your e-commerce web site? What approval processes are in place? Who is accountable? All data should be checked for reasonableness, type, format, characters, length, encoding, etc. Reporting/alerting on changes might also help detect this type of issue.

And, make sure your terms state what happens if a price is "incorrect" and exactly when a contract is formed.