Input data validation is a fundamental practice that web applications need to get right to protect users, their data, your data and your web site. But free text entry fields in web forms can be problematic - we all make mistakes, misread the instructions, type in the wrong fields, avoid filling in the mandatory items and so on. But how should the web application respond to these?

Users can submit data to web applications in many ways - file uploads, page requests, URL parameters, form parameters, file uploads, request headers, cookies and so on, but it is in conventional forms where we need to be somewhat lenient in how we respond to data validation issues.

Form data must be checked for integrity, business rules and validity of the following:

• Required (whether the field must be submitted, not null)
• Type (e.g. positive integer, text string, date, true/false boolean)
• Length/Range (e.g. numerical limits, number of decimal places, length of text, applicability of a date)
• Format (e.g. DD/MMM/YYYY for a date, international format for telephone numbers, allowed/disallowed characters)
• Character encoding/character set.

HTML forms allow radio buttons, checkboxes and drop down lists where the value(s) meant to be submitted are predetermined. In these cases it is reasonable that some of these failures are likely to be caused by malicious users tampering with values, or some fault in the application itself.

But what about text fields where users can enter any value in free format? I mentioned in Separate the Text from the Code a posting elsewhere about the display of form entry error messages. At what point to you reject the user input completely as malicious?

You may have mechanisms looking for dangerous input such as custom application code, an application filter module, security settings in your web server software or a web application firewall. If a user accidentally or without malice types text that could be construed as malicious because it matches a signature for something like cross site scripting or SQL injection, when they submit the form they could be presented with something much less friendly. This could occur if the control points are set to intervene rather than simply monitor, and so the 'friendly' message will never appear and they will see something else, like these three examples:

So be wary about validation and active blocking. In some cases, user data should be accepted and then sanitised before explaining the problem and possibly re-displaying their data appropriately encoded. The problem might be as simple as putting an item of text in a date field, or using an "incorrect" date format. If the filtering mechanisms are too strict, the usability of the application will be much degraded.

For each item of user input, try to identify:

1. What is allowed to be received by the web application for each field
2. Exact validation rules before the application can use, store or transmit the data value
3. Which control points will be checking and enforcing these
4. The sensitivity of the values
5. How the data will be used subsequently
6. How the data values should be encoded when output.

In some cases nos. 1 and 2 will be the same, but more often they will be different. Build these into your application test cases.

After all, some people live near "Alert Street" and "Script Drive":

For more information on integrity checks, validation and business rules, see data validation from OWASP's Development Guide. I'll discuss client-side validation, whitelisting and blacklisting in future posts.

Last week's Open Web Application Security Project (OWASP) summit in Portugal was a great success. The summit pages will be updated with the presentation materials and working session outcomes over the next few days.

OWASP has the most comprehensive range of information and tools to help development, testing and operation of secure web applications. It's open to everyone and everything is available free of charge. The active contributors to the Summer of Code 2008 Projects were invited to the EU Summit 08 to participate in sharing of information, discussion of issues, development of ideas for solutions and creation of suggested objectives for the organisation next year.

I managed to attend many of the project briefings and the OWASP Documentation Projects, OWASP Testing Guide, OWASP Intra Governmental Affairs, OWASP Live CD and Live DVD, OWASP Certification and OWASP Strategic Planning for 2009 working sessions. I am looking forward to working on providing official OWASP input into draft standards, guidelines and legislation, along with the other people who attended the OWASP Intra Governmental Affairs working session.

Look out for the new version (3) of the OWASP Testing Guide, available within a week. Version 2 was such an impressive piece of work, and it has been completely reviewed and extended.

I'd recommend anyone involved with the specification, development, testing, operation and management of web applications to have a look at OWASP's key resources like the Top 10, Development Guide, Code Review Guide and Testing Guide, view some of the many presentations and go along to local meetings.