Determining Html Element Visibility with WatiN

I recently started using WatiN to write automated Web application tests for the first time. WatiN is a great and rapidly maturing tool with lots of developer support, and I highly recommend it based on my experiences so far. But after writing several tests of dynamic screen elements I realized the tests weren't verifying everything I thought they were verifying.

When you use WatiN "Find" syntax to search for elements and screen text there's no obvious distinction made between hidden and visible elements. The screens I was testing use ASP.NET validation controls to perform client-side validation and include other dynamic elements such as the AJAX.NET ModalPopupExtender. I wanted to verify that these elements were visible and hidden at the appropriate times, and the simplest way of accomplishing this with WatiN was not immediately obvious.

After digging around in the API a bit I came up with a method that's simple, robust and seems to perform reasonably well. As demonstrated in the sample tests below, you can simply walk up the element hierarchy checking whether an element or any of its parents are styled with "display: none".

   1: using System.Text.RegularExpressions;

   2: using NUnit.Framework;

   3: using WatiN.Core;

   4:  

   5: [TestFixture]

   6: public class WatinTest

   7: {

   8:     [Test]

   9:     public void PopupDisplayed()

  10:     {

  11:         var ie = new IE();

  12:         ie.GoTo("http://www.example.com");

  13:  

  14:         Div popup = ie.Div(Find.ById(new Regex("pnlPopup$")));

  15:         Assert.IsTrue(IsDisplayed(popup));

  16:     }

  17:  

  18:     [Test]

  19:     public void ValidatorDisplayed()

  20:     {

  21:         var ie = new IE();

  22:         ie.GoTo("http://www.example.com");

  23:  

  24:         Span validator = ie.Span(Find.ByText("Required field"));

  25:         Assert.IsTrue(IsDisplayed(validator));

  26:     }

  27:  

  28:     public bool IsDisplayed(Element element)

  29:     {

  30:         if (string.Equals(element.Style.Display, "none"))

  31:         {

  32:             return false;

  33:         }

  34:         if (element.Parent != null)

  35:         {

  36:             return IsDisplayed(element.Parent);

  37:         }

  38:         return true;

  39:     }

  40: }

The first test case (PopupDisplayed) searches for the div element representing a modal popup panel by the element id. It uses a regular expression since the ASP.NET control ids for nested controls are unwieldy and variable. In this test we are checking whether the div itself is displayed, but this method would also work for any control contained by the div (such as a submit or text input element).

The second test (ValidatorDisplayed) searches for the span element representing a RequiredFieldValidator by the validation text contained in the span.

Some caveats apply when using this method:

• It works only with embedded CSS styles. If the "display: none" style was applied using a CSS class contained in a style sheet the WatiN Element.Style property would NOT automatically reflect this.
• It doesn't determine true element visibility--only whether the element is displayed. Elements may also be hidden using "visibility: hidden", occluded by other elements with a higher z-index, or moved offscreen using the position style.

As a side note, I highly recommend that you grab the Internet Explorer Developer Toolbar and take some time to understand the WatiN page model before writing many tests.

Confusing Errors Using WCF Transport Security With Client Certificates

While prototyping a WCF service last week I ran into a number of confusing security-related errors on both the client and server. My setup was as follows:

• Windows 2008 Server
• IIS 7
• .NET 3.5
• basicHttpBinding/transport security/client certificates

Most of the errors that can result from this setup have been documented elsewhere. This detailed post by Imaya Kumar does a great job of walking through the overall configuration and explaining the error messages caused by various misconfigurations. However, I did run into one client error that wasn't specifically documented anywhere else:

System.ServiceModel.Security.MessageSecurityException: The HTTP request was forbidden with client authentication scheme 'Anonymous'. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.

This error actually indicates that your client certificate could not be validated by the server. In my case I was using self-generated client and root certificates, and I had inadvertently uninstalled the root certificate from my server.

A related gotcha is that setting a particular clientCertificate.certificateValidationMode for your service is meaningless when using transport security. For example, the "PeerTrust" value in the configuration snippet below is ignored. Client certificates seem always to be validated using the PeerOrChainTrust method when using transport security with the http bindings.

   1: <behaviors>

   2:   <serviceBehaviors>

   3:     <behavior name="TransportBehavior">

   4:       <serviceDebug includeExceptionDetailInFaults="true" httpHelpPageEnabled="false" httpsHelpPageEnabled="true" />

   5:       <serviceMetadata httpsGetEnabled="true" httpGetEnabled="false" />

   6:       <serviceCredentials>

   7:         <clientCertificate>

   8:           <authentication certificateValidationMode="PeerTrust" />

   9:         </clientCertificate>

  10:       </serviceCredentials>

  11:     </behavior>        

  12:   </serviceBehaviors>

  13: </behaviors>

New Simple Savant Release

I've just released Simple Savant v0.2 at CodePlex. This version brings the library up to date with the latest Amazon SimpleDB features and also completes the baseline feature set. (See this post for an introduction to Simple Savant.) The new release adds support for:

• Partial object operations
• Scalar selects including support for Amazon's new select count(*) operator
• Amazon's new BatchPutAttributes operation
• Multi-valued command parameters for use with "in" clauses

This release also includes a couple of breaking changes to reduce the potential for ambiguous or unsafe behavior in certain circumstances:

• Select operations now return all available results by default. Previously Savant returned just the first page of results by default.
• Savant now only limits the number of requests sent to SimpleDB on select operations rather than attempting to return a precise number of results. This seemed the best approach considering the potential confusion when using Savant with selects containing the "limit" keyword. Use SelectCommand.MaxResultPages to limit the number of result pages requested from SimpleDB in a single call.
• DeleteAttribute requests are no longer used to delete null item properties on put operations. This not only degraded performance but could also lead to inconsistent data states. There are now two configuration options available for null property handling. Null item properties can either be ignored--meaning you become responsible for explicitly deleting attributes for null properties on put operations OR Savant can manage this for you by storing null properties as a single null character (\0) in SimpleDB.

Partial Object Operations

Partial object operations are useful when you need to put, get, select, or delete a small number of item attributes. For example, if we were working with a Person domain and needed to update just the birth date and email address for a large number of people we could do so with the code below. The example first puts new values for the two attributes and then gets just the birth date.

   1: PropertyValues values1 = new PropertyValues(person.Id);

   2: values1["BirthDate"] = new DateTime(2000, 1, 1);

   3: values1["EmailAddress"] = "mike@example.com";

   4: savant.PutAttributes<PersonItem>(values1);

   5:  

   6: PropertyValues values2 = Savant.GetAttributes<PersonItem>(person.Id, "BirthDate");

Scalar Selects

Scalar selects not only provide support for count(*) queries, but also make it easy to read a single scalar value with a minimal amount of code. In the example below we're selecting just the birth date for a person with a specific email address. If this method is used with a select query that returns extra items or attributes they will be silently discarded.

   1: DateTime? birthDate = (DateTime?)

   2:     savant.SelectScalar<PersonItem>("select BirthDate from Person where EmailAddress = @EmailAddress",

   3:                                     new CommandParameter("EmailAddress", "mike@example.com"));

BatchPutAttributes

Amazon's new BatchPutAttributes operation supports transactional puts of attributes for multiple items in the same domain. This is used automatically when you invoke SimpleSavant.Put() with multiple item parameters. For example, we could ensure that Tom, Dick, and Harry are either all in or all out of SimpleDB with the code below.

   1: PersonItem tom = new PersonItem();

   2: PersonItem dick = new PersonItem();

   3: PersonItem harry = new PersonItem();

   4: savant.Put(tom, dick, harry);

Multi-valued Command Parameters

Finally, it's now possible to provide multiple command parameter values for use with "in" clauses. The parameter values are expanded to a comma-delimited list for execution against SimpleDB. In the code snippet below we're using this feature to count the number of people in our domain whose height is denoted in inches or meters.

   1: CommandParameter parameter = new CommandParameter("HeightUnit", null);

   2: parameter.Values = new List<object> {HeightUnit.Inches, HeightUnit.Meters};

   3: int count = (int) savant.SelectScalar<PersonItem>("select count(*) from Person where HeightUnit in (@HeightUnit)",

   4:                                     parameter);

Simple Savant: .NET Object-Persistance Framework for Amazon SimpleDB

I'm building an application that stores all structured data using Amazon's SimpleDB service. When I started creating the overall architecture I searched for recommendations on designing applications specifically for SimpleDB or similar services. I didn't find many tips, but I did find lots of complaints about the disadvantages of SimpleDB when compared to mature RDBS products. I also discovered the available .NET interfaces to SimpleDB were fairly low-level and didn't put much effort into overcoming these inherent deficiencies.

So I put together a list of the higher-level features that would simplify building an application with SimpleDB and built many of these features on top of the Amazon C# Library for SimpleDB. The result is the Simple Savant .NET library (written in C#) which I've open-sourced at CodePlex.

So what are the biggest hurdles when designing for SimpleDB vs an RDBMS? I came up with the following list:

• No transactions.
• No numeric or date/time types. All attributes are stored as text strings so special formatting is required to support sorting and searching.
• Arbitrary truncation of query results.
• Eventual consistency. Item modifications are not guaranteed to be immediately visible on successive requests.
• No full-text searching.
• Item attributes are limited to 1024 characters.

(You could throw in additional deficiencies on the administration/reporting side, but my focus here is on application design.)

The first release of Simple Savant addresses several of these fundamental issues and dramatically reduces the level of effort required to work with SimpleDB. Features include:

• Mapping object properties to SimpleDB attributes.
• Formatting of basic .NET data types to support lexicographical sorts and searches.
• Support for ADO.NET-style parameterized select operations (including formatting and escaping of parameter values)
• Unlimited select results in a single call.
• Transparent caching on Get and Put operations to mitigate the effects of SimpleDB's eventual consistency model.
• Automatic domain creation

Using Simple Savant

Let's start by designing a class to hold information about a person:

   1: using System;

   2:  

   3: namespace Coditate.Savant.ConsoleSample

   4: {

   5:     [DomainName("Person")]

   6:     public class PersonItem

   7:     {

   8:         [ItemName]

   9:         public Guid Id { get; set; }

  10:  

  11:         public string FirstName { get; set; }

  12:  

  13:         public string LastName { get; set; }

  14:  

  15:         public string EmailAddress { get; set; }

  16:  

  17:         public DateTime BirthDate { get; set; }

  18:  

  19:         public float Height { get; set; }

  20:  

  21:         public float Weight { get; set; }

  22:  

  23:         public WeightUnit WeightUnit { get; set; }

  24:  

  25:         public HeightUnit HeightUnit { get; set; }

  26:  

  27:         public override string ToString()

  28:         {

  29:             return string.Format("\n\r\tName: \t\t{0}, {1}\n\r\tEmailAddress: \t{2}\n\r\tId: \t\t{3}", LastName,

  30:                                  FirstName,

  31:                                  EmailAddress, Id);

  32:         }

  33:     }

  34:  

  35:     public enum WeightUnit

  36:     {

  37:         Pounds,

  38:         Kilograms

  39:     }

  40:  

  41:     public enum HeightUnit

  42:     {

  43:         Inches,

  44:         Meters

  45:     }

  46: }

This is just about the simplest possible class we could use with Simple Savant. The ItemName attribute attached to PersonItem.Id is the only customization required for storing person instances in SimpleDB. It tells Simple Savant which property to use as the SimpleDB item name for Get, Put, and Delete operations.

The class-level DomainName attribute is optional. It lets us customize the SimpleDB domain where instances are stored. By default the class name is used for the domain name. Thus if we removed the DomainName attribute person instances would be stored in the "PersonItem" domain, but with the attribute they will be stored in the "Person" domain.

Next let's populate a PersonItem and put it in SimpleDB:

   1: PersonItem person = new PersonItem

   2:     {

   3:         BirthDate = new DateTime(1972, 1, 15),

   4:         EmailAddress = "bob@example.com",

   5:         FirstName = "Bob",

   6:         Height = 72.5f,

   7:         HeightUnit = HeightUnit.Inches,

   8:         Id = Guid.NewGuid(),

   9:         LastName = "Smith",

  10:         Weight = 200,

  11:         WeightUnit = WeightUnit.Pounds

  12:     };

  13:  

  14: string awsAccessKeyId = "xxxxxxx";

  15: string awsSecretAccessKey = "xxxxxxx";

  16: SimpleSavant savant = new SimpleSavant(awsAccessKeyId, awsSecretAccessKey);

  17:  

  18: savant.Put(person);

Once we've populated our person object it takes just a few lines of code to configure Simple Savant and send our person to SimpleDB!

Here's what we'd find in SimpleDB after executing this code:

ItemName	EmailAddress	Height	Weight	HeightUnit	WeightUnit	BirthDate	FirstName	LastName
c9748367-c929-408d-b7cf-60b3677717cf	bob@example.com	1072.5	1200	Inches	Pounds	1972-01-15T00:00:00.000-05:00	Bob	Smith

By default property names are used for SimpleDB attribute names, but you can customize them using the AttributeName attribute. Other points of note:

• Dates are formatted using the ISO 8601 standard to support lexicographical ordering.
• Unsigned numeric types are zero-padded to support lexicographical ordering.
• Signed numeric types are offset to support lexicographical ordering when positive and negative values are mixed together. The default offset value is 10 raised to the nth power, where n is the maximum number of whole digits supported by the type. This is why our height value of 72.5 is stored as 1072.5 and our weight value of 200 is stored as 1200. (Float or Single precision values are formatted with three whole and four decimal digits by default.) 
• All properties are mapped to SimpleDB attributes by default. You can customize this behavior using the SavantInclude and SavantExclude attributes.
• Property formatting can be customized using the CustomFormat and NumberFormat attributes. For example, we could easily customize  PersonItem.Weight to be formatted with five whole digits and two decimal digits to increase the maximum possible weight value from 999 to 99,999.

Getting our person back from SimpleDB takes just a couple more lines of code:

   1: Guid personId = person.Id;

   2: PersonItem person2 = savant.Get<PersonItem>(personId);

If we added many people to our person domain and needed to select them all we could do so like this:

   1: IList<PersonItem> allPeople = savant.Select<PersonItem>("select * from Person");

Note that this query would return all items in the domain. Simple Savant requests all available pages of results from SimpleDB unless you explicitly limit results using properties on SelectCommand. See the API documentation for more details on this.

Finally, to run a more sophisticated range query finding all people born during the 1980s we can use a parameterized command:

   1: SelectCommand<PersonItem> command = new SelectCommand<PersonItem>("select * from Person where BirthDate between @StartDate and @EndDate");

   2: command.AddParameter(new CommandParameter("StartDate", "BirthDate", new DateTime(1980, 1, 1)));

   3: command.AddParameter(new CommandParameter("EndDate", "BirthDate", new DateTime(1989, 12, 31)));

   4:  

   5: SelectResults<PersonItem> eightiesPeople = savant.Select(command);

The same formatting rules are applied to select parameters as when performing Get and Put operations on the Person domain. In other words, if we defined custom formatting behavior for PersonItem.BirthDate, the same formatting rules would be used for select operations involving the BirthDate attribute--and the query above would still just work.

Hopefully this post will help you get started using Simple Savant. The CodePlex release includes sample code and full API documentation that should keep you going.

UPDATE: Here is a brief tutorial on the new features in Simple Savant v0.2.

Version of System.Web.HttpUtility for .NET Client Profile

The .NET Framework Client Profile released with .NET 3.5 SP1 defines a stripped-down version of the .NET Framework for distribution with rich-client applications. This is a valuable feature for developers who need to keep their software distributions small and convenient to install. (For more information, see my previous post comparing bootstrap install times for various .NET framework versions.)

The Client Profile originally included the System.Web assembly, but this was removed from the final service pack release. This is unfortunate because System.Web includes several utility operations that are useful in rich client applications. One of these is URL encoding and decoding. It would be easy enough to code up a replacement function, but why bother? Instead I've decompiled the latest version of System.Web.HttpUtility (v2.0.50727) and stripped out the functions that depend on System.Web. All of the URL encoding methods are intact. The code is below--use at your own risk.

   1: using System.Text;

   2:  

   3: namespace System.Web

   4: {

   5:     public sealed class HttpUtility

   6:     {

   7:         private static int HexToInt(char h)

   8:         {

   9:             if ((h >= '0') && (h <= '9'))

  10:             {

  11:                 return (h - '0');

  12:             }

  13:             if ((h >= 'a') && (h <= 'f'))

  14:             {

  15:                 return ((h - 'a') + 10);

  16:             }

  17:             if ((h >= 'A') && (h <= 'F'))

  18:             {

  19:                 return ((h - 'A') + 10);

  20:             }

  21:             return -1;

  22:         }

  23:  

  24:         internal static char IntToHex(int n)

  25:         {

  26:             if (n <= 9)

  27:             {

  28:                 return (char) (n + 0x30);

  29:             }

  30:             return (char) ((n - 10) + 0x61);

  31:         }

  32:  

  33:         private static bool IsNonAsciiByte(byte b)

  34:         {

  35:             if (b < 0x7f)

  36:             {

  37:                 return (b < 0x20);

  38:             }

  39:             return true;

  40:         }

  41:  

  42:         internal static bool IsSafe(char ch)

  43:         {

  44:             if ((((ch >= 'a') && (ch <= 'z')) || ((ch >= 'A') && (ch <= 'Z'))) || ((ch >= '0') && (ch <= '9')))

  45:             {

  46:                 return true;

  47:             }

  48:             switch (ch)

  49:             {

  50:                 case '\'':

  51:                 case '(':

  52:                 case ')':

  53:                 case '*':

  54:                 case '-':

  55:                 case '.':

  56:                 case '_':

  57:                 case '!':

  58:                     return true;

  59:             }

  60:             return false;

  61:         }

  62:  

  63:         public static string UrlDecode(string str)

  64:         {

  65:             if (str == null)

  66:             {

  67:                 return null;

  68:             }

  69:             return UrlDecode(str, Encoding.UTF8);

  70:         }

  71:  

  72:         public static string UrlDecode(byte[] bytes, Encoding e)

  73:         {

  74:             if (bytes == null)

  75:             {

  76:                 return null;

  77:             }

  78:             return UrlDecode(bytes, 0, bytes.Length, e);

  79:         }

  80:  

  81:         public static string UrlDecode(string str, Encoding e)

  82:         {

  83:             if (str == null)

  84:             {

  85:                 return null;

  86:             }

  87:             return UrlDecodeStringFromStringInternal(str, e);

  88:         }

  89:  

  90:         public static string UrlDecode(byte[] bytes, int offset, int count, Encoding e)

  91:         {

  92:             if ((bytes == null) && (count == 0))

  93:             {

  94:                 return null;

  95:             }

  96:             if (bytes == null)

  97:             {

  98:                 throw new ArgumentNullException("bytes");

  99:             }

 100:             if ((offset < 0) || (offset > bytes.Length))

 101:             {

 102:                 throw new ArgumentOutOfRangeException("offset");

 103:             }

 104:             if ((count < 0) || ((offset + count) > bytes.Length))

 105:             {

 106:                 throw new ArgumentOutOfRangeException("count");

 107:             }

 108:             return UrlDecodeStringFromBytesInternal(bytes, offset, count, e);

 109:         }

 110:  

 111:         private static byte[] UrlDecodeBytesFromBytesInternal(byte[] buf, int offset, int count)

 112:         {

 113:             int length = 0;

 114:             byte[] sourceArray = new byte[count];

 115:             for (int i = 0; i < count; i++)

 116:             {

 117:                 int index = offset + i;

 118:                 byte num4 = buf[index];

 119:                 if (num4 == 0x2b)

 120:                 {

 121:                     num4 = 0x20;

 122:                 }

 123:                 else if ((num4 == 0x25) && (i < (count - 2)))

 124:                 {

 125:                     int num5 = HexToInt((char) buf[index + 1]);

 126:                     int num6 = HexToInt((char) buf[index + 2]);

 127:                     if ((num5 >= 0) && (num6 >= 0))

 128:                     {

 129:                         num4 = (byte) ((num5 << 4) | num6);

 130:                         i += 2;

 131:                     }

 132:                 }

 133:                 sourceArray[length++] = num4;

 134:             }

 135:             if (length < sourceArray.Length)

 136:             {

 137:                 byte[] destinationArray = new byte[length];

 138:                 Array.Copy(sourceArray, destinationArray, length);

 139:                 sourceArray = destinationArray;

 140:             }

 141:             return sourceArray;

 142:         }

 143:  

 144:         private static string UrlDecodeStringFromBytesInternal(byte[] buf, int offset, int count, Encoding e)

 145:         {

 146:             UrlDecoder decoder = new UrlDecoder(count, e);

 147:             for (int i = 0; i < count; i++)

 148:             {

 149:                 int index = offset + i;

 150:                 byte b = buf[index];

 151:                 if (b == 0x2b)

 152:                 {

 153:                     b = 0x20;

 154:                 }

 155:                 else if ((b == 0x25) && (i < (count - 2)))

 156:                 {

 157:                     if ((buf[index + 1] == 0x75) && (i < (count - 5)))

 158:                     {

 159:                         int num4 = HexToInt((char) buf[index + 2]);

 160:                         int num5 = HexToInt((char) buf[index + 3]);

 161:                         int num6 = HexToInt((char) buf[index + 4]);

 162:                         int num7 = HexToInt((char) buf[index + 5]);

 163:                         if (((num4 < 0) || (num5 < 0)) || ((num6 < 0) || (num7 < 0)))

 164:                         {

 165:                             goto Label_00DA;

 166:                         }

 167:                         char ch = (char) ((((num4 << 12) | (num5 << 8)) | (num6 << 4)) | num7);

 168:                         i += 5;

 169:                         decoder.AddChar(ch);

 170:                         continue;

 171:                     }

 172:                     int num8 = HexToInt((char) buf[index + 1]);

 173:                     int num9 = HexToInt((char) buf[index + 2]);

 174:                     if ((num8 >= 0) && (num9 >= 0))

 175:                     {

 176:                         b = (byte) ((num8 << 4) | num9);

 177:                         i += 2;

 178:                     }

 179:                 }

 180:                 Label_00DA:

 181:                 decoder.AddByte(b);

 182:             }

 183:             return decoder.GetString();

 184:         }

 185:  

 186:         private static string UrlDecodeStringFromStringInternal(string s, Encoding e)

 187:         {

 188:             int length = s.Length;

 189:             UrlDecoder decoder = new UrlDecoder(length, e);

 190:             for (int i = 0; i < length; i++)

 191:             {

 192:                 char ch = s[i];

 193:                 if (ch == '+')

 194:                 {

 195:                     ch = ' ';

 196:                 }

 197:                 else if ((ch == '%') && (i < (length - 2)))

 198:                 {

 199:                     if ((s[i + 1] == 'u') && (i < (length - 5)))

 200:                     {

 201:                         int num3 = HexToInt(s[i + 2]);

 202:                         int num4 = HexToInt(s[i + 3]);

 203:                         int num5 = HexToInt(s[i + 4]);

 204:                         int num6 = HexToInt(s[i + 5]);

 205:                         if (((num3 < 0) || (num4 < 0)) || ((num5 < 0) || (num6 < 0)))

 206:                         {

 207:                             goto Label_0106;

 208:                         }

 209:                         ch = (char) ((((num3 << 12) | (num4 << 8)) | (num5 << 4)) | num6);

 210:                         i += 5;

 211:                         decoder.AddChar(ch);

 212:                         continue;

 213:                     }

 214:                     int num7 = HexToInt(s[i + 1]);

 215:                     int num8 = HexToInt(s[i + 2]);

 216:                     if ((num7 >= 0) && (num8 >= 0))

 217:                     {

 218:                         byte b = (byte) ((num7 << 4) | num8);

 219:                         i += 2;

 220:                         decoder.AddByte(b);

 221:                         continue;

 222:                     }

 223:                 }

 224:                 Label_0106:

 225:                 if ((ch & 0xff80) == 0)

 226:                 {

 227:                     decoder.AddByte((byte) ch);

 228:                 }

 229:                 else

 230:                 {

 231:                     decoder.AddChar(ch);

 232:                 }

 233:             }

 234:             return decoder.GetString();

 235:         }

 236:  

 237:         public static byte[] UrlDecodeToBytes(byte[] bytes)

 238:         {

 239:             if (bytes == null)

 240:             {

 241:                 return null;

 242:             }

 243:             return UrlDecodeToBytes(bytes, 0, (bytes != null) ? bytes.Length : 0);

 244:         }

 245:  

 246:         public static byte[] UrlDecodeToBytes(string str)

 247:         {

 248:             if (str == null)

 249:             {

 250:                 return null;

 251:             }

 252:             return UrlDecodeToBytes(str, Encoding.UTF8);

 253:         }

 254:  

 255:         public static byte[] UrlDecodeToBytes(string str, Encoding e)

 256:         {

 257:             if (str == null)

 258:             {

 259:                 return null;

 260:             }

 261:             return UrlDecodeToBytes(e.GetBytes(str));

 262:         }

 263:  

 264:         public static byte[] UrlDecodeToBytes(byte[] bytes, int offset, int count)

 265:         {

 266:             if ((bytes == null) && (count == 0))

 267:             {

 268:                 return null;

 269:             }

 270:             if (bytes == null)

 271:             {

 272:                 throw new ArgumentNullException("bytes");

 273:             }

 274:             if ((offset < 0) || (offset > bytes.Length))

 275:             {

 276:                 throw new ArgumentOutOfRangeException("offset");

 277:             }

 278:             if ((count < 0) || ((offset + count) > bytes.Length))

 279:             {

 280:                 throw new ArgumentOutOfRangeException("count");

 281:             }

 282:             return UrlDecodeBytesFromBytesInternal(bytes, offset, count);

 283:         }

 284:  

 285:         public static string UrlEncode(byte[] bytes)

 286:         {

 287:             if (bytes == null)

 288:             {

 289:                 return null;

 290:             }

 291:             return Encoding.ASCII.GetString(UrlEncodeToBytes(bytes));

 292:         }

 293:  

 294:         public static string UrlEncode(string str)

 295:         {

 296:             if (str == null)

 297:             {

 298:                 return null;

 299:             }

 300:             return UrlEncode(str, Encoding.UTF8);

 301:         }

 302:  

 303:         public static string UrlEncode(string str, Encoding e)

 304:         {

 305:             if (str == null)

 306:             {

 307:                 return null;

 308:             }

 309:             return Encoding.ASCII.GetString(UrlEncodeToBytes(str, e));

 310:         }

 311:  

 312:         public static string UrlEncode(byte[] bytes, int offset, int count)

 313:         {

 314:             if (bytes == null)

 315:             {

 316:                 return null;

 317:             }

 318:             return Encoding.ASCII.GetString(UrlEncodeToBytes(bytes, offset, count));

 319:         }

 320:  

 321:         private static byte[] UrlEncodeBytesToBytesInternal(byte[] bytes, int offset, int count,

 322:                                                             bool alwaysCreateReturnValue)

 323:         {

 324:             int num = 0;

 325:             int num2 = 0;

 326:             for (int i = 0; i < count; i++)

 327:             {

 328:                 char ch = (char) bytes[offset + i];

 329:                 if (ch == ' ')

 330:                 {

 331:                     num++;

 332:                 }

 333:                 else if (!IsSafe(ch))

 334:                 {

 335:                     num2++;

 336:                 }

 337:             }

 338:             if ((!alwaysCreateReturnValue && (num == 0)) && (num2 == 0))

 339:             {

 340:                 return bytes;

 341:             }

 342:             byte[] buffer = new byte[count + (num2*2)];

 343:             int num4 = 0;

 344:             for (int j = 0; j < count; j++)

 345:             {

 346:                 byte num6 = bytes[offset + j];

 347:                 char ch2 = (char) num6;

 348:                 if (IsSafe(ch2))

 349:                 {

 350:                     buffer[num4++] = num6;

 351:                 }

 352:                 else if (ch2 == ' ')

 353:                 {

 354:                     buffer[num4++] = 0x2b;

 355:                 }

 356:                 else

 357:                 {

 358:                     buffer[num4++] = 0x25;

 359:                     buffer[num4++] = (byte) IntToHex((num6 >> 4) & 15);

 360:                     buffer[num4++] = (byte) IntToHex(num6 & 15);

 361:                 }

 362:             }

 363:             return buffer;

 364:         }

 365:  

 366:         private static byte[] UrlEncodeBytesToBytesInternalNonAscii(byte[] bytes, int offset, int count,

 367:                                                                     bool alwaysCreateReturnValue)

 368:         {

 369:             int num = 0;

 370:             for (int i = 0; i < count; i++)

 371:             {

 372:                 if (IsNonAsciiByte(bytes[offset + i]))

 373:                 {

 374:                     num++;

 375:                 }

 376:             }

 377:             if (!alwaysCreateReturnValue && (num == 0))

 378:             {

 379:                 return bytes;

 380:             }

 381:             byte[] buffer = new byte[count + (num*2)];

 382:             int num3 = 0;

 383:             for (int j = 0; j < count; j++)

 384:             {

 385:                 byte b = bytes[offset + j];

 386:                 if (IsNonAsciiByte(b))

 387:                 {

 388:                     buffer[num3++] = 0x25;

 389:                     buffer[num3++] = (byte) IntToHex((b >> 4) & 15);

 390:                     buffer[num3++] = (byte) IntToHex(b & 15);

 391:                 }

 392:                 else

 393:                 {

 394:                     buffer[num3++] = b;

 395:                 }

 396:             }

 397:             return buffer;

 398:         }

 399:  

 400:         internal static string UrlEncodeNonAscii(string str, Encoding e)

 401:         {

 402:             if (string.IsNullOrEmpty(str))

 403:             {

 404:                 return str;

 405:             }

 406:             if (e == null)

 407:             {

 408:                 e = Encoding.UTF8;

 409:             }

 410:             byte[] bytes = e.GetBytes(str);

 411:             bytes = UrlEncodeBytesToBytesInternalNonAscii(bytes, 0, bytes.Length, false);

 412:             return Encoding.ASCII.GetString(bytes);

 413:         }

 414:  

 415:         internal static string UrlEncodeSpaces(string str)

 416:         {

 417:             if ((str != null) && (str.IndexOf(' ') >= 0))

 418:             {

 419:                 str = str.Replace(" ", "%20");

 420:             }

 421:             return str;

 422:         }

 423:  

 424:         public static byte[] UrlEncodeToBytes(string str)

 425:         {

 426:             if (str == null)

 427:             {

 428:                 return null;

 429:             }

 430:             return UrlEncodeToBytes(str, Encoding.UTF8);

 431:         }

 432:  

 433:         public static byte[] UrlEncodeToBytes(byte[] bytes)

 434:         {

 435:             if (bytes == null)

 436:             {

 437:                 return null;

 438:             }

 439:             return UrlEncodeToBytes(bytes, 0, bytes.Length);

 440:         }

 441:  

 442:         public static byte[] UrlEncodeToBytes(string str, Encoding e)

 443:         {

 444:             if (str == null)

 445:             {

 446:                 return null;

 447:             }

 448:             byte[] bytes = e.GetBytes(str);

 449:             return UrlEncodeBytesToBytesInternal(bytes, 0, bytes.Length, false);

 450:         }

 451:  

 452:         public static byte[] UrlEncodeToBytes(byte[] bytes, int offset, int count)

 453:         {

 454:             if ((bytes == null) && (count == 0))

 455:             {

 456:                 return null;

 457:             }

 458:             if (bytes == null)

 459:             {

 460:                 throw new ArgumentNullException("bytes");

 461:             }

 462:             if ((offset < 0) || (offset > bytes.Length))

 463:             {

 464:                 throw new ArgumentOutOfRangeException("offset");

 465:             }

 466:             if ((count < 0) || ((offset + count) > bytes.Length))

 467:             {

 468:                 throw new ArgumentOutOfRangeException("count");

 469:             }

 470:             return UrlEncodeBytesToBytesInternal(bytes, offset, count, true);

 471:         }

 472:  

 473:         public static string UrlEncodeUnicode(string str)

 474:         {

 475:             if (str == null)

 476:             {

 477:                 return null;

 478:             }

 479:             return UrlEncodeUnicodeStringToStringInternal(str, false);

 480:         }

 481:  

 482:         private static string UrlEncodeUnicodeStringToStringInternal(string s, bool ignoreAscii)

 483:         {

 484:             int length = s.Length;

 485:             StringBuilder builder = new StringBuilder(length);

 486:             for (int i = 0; i < length; i++)

 487:             {

 488:                 char ch = s[i];

 489:                 if ((ch & 0xff80) == 0)

 490:                 {

 491:                     if (ignoreAscii || IsSafe(ch))

 492:                     {

 493:                         builder.Append(ch);

 494:                     }

 495:                     else if (ch == ' ')

 496:                     {

 497:                         builder.Append('+');

 498:                     }

 499:                     else

 500:                     {

 501:                         builder.Append('%');

 502:                         builder.Append(IntToHex((ch >> 4) & '\x000f'));

 503:                         builder.Append(IntToHex(ch & '\x000f'));

 504:                     }

 505:                 }

 506:                 else

 507:                 {

 508:                     builder.Append("%u");

 509:                     builder.Append(IntToHex((ch >> 12) & '\x000f'));

 510:                     builder.Append(IntToHex((ch >> 8) & '\x000f'));

 511:                     builder.Append(IntToHex((ch >> 4) & '\x000f'));

 512:                     builder.Append(IntToHex(ch & '\x000f'));

 513:                 }

 514:             }

 515:             return builder.ToString();

 516:         }

 517:  

 518:         public static byte[] UrlEncodeUnicodeToBytes(string str)

 519:         {

 520:             if (str == null)

 521:             {

 522:                 return null;

 523:             }

 524:             return Encoding.ASCII.GetBytes(UrlEncodeUnicode(str));

 525:         }

 526:  

 527:         public static string UrlPathEncode(string str)

 528:         {

 529:             if (str == null)

 530:             {

 531:                 return null;

 532:             }

 533:             int index = str.IndexOf('?');

 534:             if (index >= 0)

 535:             {

 536:                 return (UrlPathEncode(str.Substring(0, index)) + str.Substring(index));

 537:             }

 538:             return UrlEncodeSpaces(UrlEncodeNonAscii(str, Encoding.UTF8));

 539:         }

 540:  

 541:         // Nested Types

 542:         private class UrlDecoder

 543:         {

 544:             // Fields

 545:             private int _bufferSize;

 546:             private byte[] _byteBuffer;

 547:             private char[] _charBuffer;

 548:             private Encoding _encoding;

 549:             private int _numBytes;

 550:             private int _numChars;

 551:  

 552:             // Methods

 553:             internal UrlDecoder(int bufferSize, Encoding encoding)

 554:             {

 555:                 _bufferSize = bufferSize;

 556:                 _encoding = encoding;

 557:                 _charBuffer = new char[bufferSize];

 558:             }

 559:  

 560:             internal void AddByte(byte b)

 561:             {

 562:                 if (_byteBuffer == null)

 563:                 {

 564:                     _byteBuffer = new byte[_bufferSize];

 565:                 }

 566:                 _byteBuffer[_numBytes++] = b;

 567:             }

 568:  

 569:             internal void AddChar(char ch)

 570:             {

 571:                 if (_numBytes > 0)

 572:                 {

 573:                     FlushBytes();

 574:                 }

 575:                 _charBuffer[_numChars++] = ch;

 576:             }

 577:  

 578:             private void FlushBytes()

 579:             {

 580:                 if (_numBytes > 0)

 581:                 {

 582:                     _numChars += _encoding.GetChars(_byteBuffer, 0, _numBytes, _charBuffer, _numChars);

 583:                     _numBytes = 0;

 584:                 }

 585:             }

 586:  

 587:             internal string GetString()

 588:             {

 589:                 if (_numBytes > 0)

 590:                 {

 591:                     FlushBytes();

 592:                 }

 593:                 if (_numChars > 0)

 594:                 {

 595:                     return new string(_charBuffer, 0, _numChars);

 596:                 }

 597:                 return string.Empty;

 598:             }

 599:         }

 600:     }

 601: }

Note: The source for HttpUtility is also available through the .NET Framework source release, but I actually found it easier to get a clean copy by decompiling.

The code above was formatted using Leo Vildosola's Code Snippet plugin for Windows Live Writer, which I just started using with this post.