A couple of weeks ago I mentioned listings of user agent (browser identifications). But can this data be helpful in validating logged in user sessions?

Session data (relating to the user) should be stored on the server rather than in cookies or locally on the client (browser) and this is often referenced by a unique, difficult to predict, session identifier, usually set as a secure, httpd-only, temporary cookie. It is sometimes better to validate the session identifier is still being used by the same user it was issued to. Apart from checking the session is still valid (exists and has not expired), you can also check that it corresponds to the same IP address, or at least an IP address range. It's also possible to build into this, checks that the following Hypertext Transfer Protocol (HTTP) request headers haven't changed:

• User agent
• Acceptable languages for response
• Acceptable encodings for response

by storing these properties, or a hash of them, along with the session identifier in the web application's database. They should not change. These additional checks make it harder for someone else to impersonate the (authenticated) user.

You might wonder how unique user agent data can be. In an attempt to determine the uniqueness of browser information and whether this constitutes personal data for privacy protection because it identifies them, the Electronic Frontier Foundation (EFF) has released a new online tool called Panopticlick to let you calculate how unique your own web browser fingerprint is. The test identifies the user agent string, HTTP accept headers, browser plug-ins, time zone, screen size, screen colour depth, system fonts, whether cookies are enabled and storage settings. There is a good write-up about the personal data issues on the Tech and Law blog. But can we use this data for session tracking?/p>

I tried three browsers with Panopticlick:

• Firefox 3.6 running several add-ons (FF3.6)
• Opera 10.10 with JavaScript enabled (O10)
• Internet Explorer 8.0.6 (IE8)

which indicated that all three browser's fingerprints were "unique among the 71,823 tested so far" with at least 16.13 bits of identifying information.

The differences appear to be related to how much information could be gleaned from the browser and system, with plug-ins, screen dimensions and system fonts being very unique—the computer has two graphics packages installed and several custom fonts. Making the browser full screen reduced this aspect's uniqueness, but had no overall change in the browser's identifiability.

This means that some of these data could be used to check for impersonation or even to help identify returning site visitors without setting cookies or requiring people to log in, but you would have to be careful because some browsers won't send all of this data e.g. if JavaScript is disabled. For example, with NoScript enabled, FF3.6 reported as only one in 2,433 browsers with the same fingerprint and that it conveyed 11.25 bits of identifying information instead of over 16 bits.

Further reading about the scary things here and here that JavaScript might be able to detect about your computer and network.

The next Open Web Application Security Project (OWASP) London meeting is this week.

The London chapter meeting is on Thursday 14 January 2010 in EC1. Everyone is welcome, but you need to register first (free).

There will be talks on Top Ten Deployment Mistakes That Render SSL Useless by Ivan Ristić and Using Selenium to Hold State for Web Application Penetration Testing by Yiannis Pavlosoglou, who recently joined the OWASP Global Industry Committee.

Unfortunately I am unable to attend the meeting but hope to read the presentations afterwards.

The web browser (user agent) normally sends a string of text to identify itself, but this can be blank or many other things, and can be altered by users.

The Bots vs Browsers (robots versus web browsers) web site monitors the user agent identification strings and helpfully provides access to their valuable data. They have identified over 400,000 different user agent strings to date. You can see 5,000 unidentified that are not obviously any particular search engine robot or browser and could be accidental, mischievous or malicious.

Always treat user agent strings as untrusted, and check their length and content before using the sanitised text to display, or in decision-making logic or when writing it to a file or database. The lists for particular types of devices (e.g. the iPhone) may be useful to remind you of the range of values sent. User agent strings certainly shouldn't be used for any form of user authentication.

Examples of content aggregation often refer to the use of web services and XML data such as RSS feeds. But today's world of web 2.0 in creating more and more data in a wide variety of formats including JSON (JavaScript Object Notation); and web applications are being used as a concentrator to combine these together.

With the growth of layered communications, multiple communication channels such as text, video and audio are merged into one event. If the content is recorded it can be republished via a web site. But what are the specific security risks of this?

Web services and XML data can include invalid or malicious data. The format/schema may be incorrect. But with the increase in layered communications, content from many different devices in many media may need to be aggregated into a single resource; and these often don't have any formal syntactical structure. The data might even include active content such as embedded rich applications.

If these need to be stored and replayed such content at a later date, how might they affect a web page? The content could contain, or link to, malicious content that steals user data such as session cookies, modifies the page's content or installs malware onto user's computers.

• Identify all the data streams.
• Determine their formats and encoding where appropriate.
• Ruthlessly limit what active (script) content is allowed and what ability it has to interact with the parent web site and its domain.
• Analyse the data streams to validate they contain what is intended and scan for malware.
• Sanitise content where applicable.
• Limit file size/length/number of nodes.
• Avoid merging trusted and untrusted content in data fields.
• Encode the output correctly for your own application.
• Monitor activity and look out for unusual events.

And beware embedding rich internet applications (RIAs) such as Adobe Flash or Microsoft Silverlight, which may be doing this aggregation themselves.

After all, you don't want your web site to be a concentrator multiplexing malware.

Look out for the busy schedule of upcoming Open Web Application Security Project (OWASP) events.

This Thursday 5th November, the next OWASP London meeting is taking place. There will be a presentation about using Selenium to hold state for web application penetration testing, and a discussion of the chapter's activities and plans. Advance registration is required, but there is no charge.

Also on 5th November in Rome, there is Italy OWASP Day E-Gov 09 for public administrators, and the day after on the 6th in Milan the Italy OWASP Day IV.

Then next week, AppSec 09 is taking place in Washington DC, United States, with two days of training courses, an OWASP summit for the OWASP board, project leaders, chapter leaders and global committee members. The main conference has 4 agenda tracks about OWASP, tools, Web 2.0 and software development life cycle, and 4 further tracks on the second day encompassing Attack & Defend, Process, Metrics and Compliance.

In December there is the BeNeLux OWASP day in Leuven, Belgium, at which I am speaking about compliance driven vulnerabilities.

If web application security is an area of interest, try to get along to one or more of these, or download the presentations after the events.

This weekend the clocks change as we revert from British Summer Time (BST) to Greenwich Mean Time (GMT) at 02:00 BST on Sunday 25 October 2009 and the clocks go back, giving an extra hour.

What does this mean for web site security? Does running 01:00 to 02:00 twice matter? Well some brave web application owners will be disabling their systems like this online bank:

And, I don't think it's just being done as a finale to the current Energy Saving Week. Most people, quite rightly, won't be taking this rather severe step. Another millennium bug anyone? The date/time should be considered rather like other untrusted user input. Most problems will probably fall into the "business logic" category such as:

• Failure of time-based logic where dates are being compared.
• Assumptions of uniqueness in time-stamped output (e.g. by a single-threaded process).
• Running tasks again leading to possible:
  • loss of data due to overwriting
  • duplication of exports or emails
  • creation of inaccuracies in management information.
• Chronological ordering anomalies leading to other faults.

It's not just banks and other financial organisations that may have difficulties.

The time change may expose some other vulnerabilities that only exist at changeover time and/or during the next overlap hour.

• Circumvention of brute force attacks on user authentication mechanisms.
• Increased risk due to extension of a session's validity where local time is recorded.
• Failure in data validation routines for time-related comparisons.
• Incubated vulnerabilities where a time-related aspect causes the attack to be possible.
• Denial of service due to extension of account lock-out.
• Using time as a loop counter.
• Additional errors caused by any of the above leading to information leakage.

Recording the offset of local time to GMT/UTC and synchronisation should certainly be done, but may not resolve the time overlap issues. The effects on long-running "saga" requests might be especially difficult to determine. Time dependencies need to be specified and considered through the development lifecycle. Perhaps the bank is right after all?

The Web Application Security Consortium (WASC) has published its Web Application Security Statistics Project 2008.

The data has been gathered from anonymous submissions by eight organisations about their web application security assessment projects in 2008. The large number of projects gives a good spread of data, though of course it excludes web sites and web applications that have never been tested.

The owners of web sites and web applications that are not included here will include those validated/tested by others, tested internally and not tested at all. The latter may include simpler, less functional web sites or those where the potential damage is considered to be less than the cost of application security validation/testing. Nevertheless, the data are important for web developers, testers and auditors for many reasons:

• The most common vulnerabilities are listed—they are nothing new and demonstrate where greater effort is often required to standardise code and stamp these out.
• The risk assessment methodology is explained and can be used as a guide for internal assessments.
• High risk vulnerabilities are common (more than 80% have urgent and critical vulnerabilities), even in applications that are likely to be considered more important, and therefore are being tested by independent third parties.
• Organisations that are required to be PCI DSS compliant, may be failing to ensure their web sites meet the requirements.
• Manual white box (full knowledge) testing is needed as well as automated scanning to detect many vulnerabilities.

All the contributors to this project should be thanked for sharing their insights with us.

An article on eConsultancy.com discussing the demise of the tr.im URL shortening service mentioned some information security concerns:

There are good discussions concerning URL shorteners on Joshua Schachter's blog, the Mashable social media guide and the Security Retentive corporate information security blog. These will be a little technical for many people but essentially, apart from the privacy aspect of collecting data, other things might be worth worrying about more (you need to check this!). If you can, create your own aliasing/redirecting system on your own domain. But make sure it can only be used to redirect to a limited number of your own website's URLs.

Interestingly, tr.im is another example where organisations may have been using a service for which they had no contract, service level agreement or rights when the service was withdrawn. Be careful what you reply upon.

Data quality is often neglected. Do you know what data you have, where it came from, what constraints there are on its use and where it all is? But what about its quality?

Last week, Philip Virgo discussed data quality in When IT Meets Politics. He referred to the Audit Commission report on Figures You Can Trust (Briefing on Data Quality in the NHS). Data quality is an ongoing issue and the report summary states:

Interestingly the Audit Commission found the error rate in a key data set for NHS activity varied from 0.3 per cent to 52 per cent.

Philip Virgo discusses the importance of data quality and what information governance is:

Do you know how accurate your own data are?

A Valentine's Day special (yes a day early, I know, but better than a Friday 13th item). What data type should the "sex" attribute be in your database and what values should your form fields have?

No jokes such as the phrases (strings) "yes please" or "no thank you, we're British"... well, I guess a single character for gender of either "M" or "F" is often chosen. You might need to allow null for when this isn't known, or not divulged. Other people might use an alternative coding scheme.

Why not use a standard scheme?

You might be surprised to learn that there is an International Organization for Standardization (ISO) standard ISO/IEC 5218 on "Codes for the Representation of Human Sexes". And interestingly the technical committee have selected a one-character numeric code, with four (yes 4) valid options:

• 0 = Not Known
• 1 = Male
• 2 = Female
• 9 = Not Applicable

The standard points out (see the ISO 5218 entry on Wikipedia) the ordering is not meant to be significant, but UFO watchers and others will be pleased by the spare slots 3-8.

The current four values have also been incorporated as the sex data attribute in the P5 Guidelines for Electronic Text Encoding and Interchange by the Text Encoding Initiative (TEI) who develop and maintain a standard for the representation of texts in digital form. So, for a positive security models where the values passed to and from user input screens and in processing are the same as in the database, the acceptable values (also known as a "white list") is a set of integers: {0,1,2,9}.

The answer is therefore a small positive integer.