Bad Experiences with StackOverflow

If you build software you're probably familiar with StackOverflow, the 2-year-old, collaboratively edited question-and-answer site for programmers. SO was not a new idea, but it was well-executed and quickly gained a a large number of users due to the popular blogs of its founders.

I haven't been a particularly active SO user, and my "reputation" peaked at a little over 400 points after one question and 20+ answers on the site. But what's been striking is how many negative experiences I've had in my relatively limited usage of SO.

My first and only question was quickly deleted by another user who said it was "too localized, outside the scope of [StackOverflow]...and argumentative" (my question concerned an esoteric provision of employment agreements and state intellectual property laws). I was a powerless newbie with no reputation so I had little recourse at that point. Fortunately at least one other user disagreed, and my question was eventually restored and even added to the SO community wiki.

The next negative experience occurred after I downvoted an answer by Mitch Wheat of Perth, Australia. As per the SO recommended protocol I added a comment explaining why I downvoted his answer. Normally you can't see who downvotes your answers--except when they add comments explaining their votes! Mitch used this information to rapidly downvote several of my answers in retaliation. (I know this because he couldn't restrain from also adding sarcastic comments of his own. But he must have agreed with me because he proceeded to delete his own answer which I had downvoted!)

My latest experience with SO has left such a sour taste that I've decided not to waste my time contributing answers to the site. Instead I'll stick to blog posts. Here's what happened: About half my answers on SO have something to do with Amazon SimpleDb. They will sometimes include a reference to my Simple Savant open-source library--but only when pertinent to the question. Sometimes I'll phrase the answer in a way that indicates I'm the creator of Savant, and 2 or 3 times I mentioned that Savant has a feature that might help solve the questioner's problem without explaining that I created Savant. Explicitly stating that I created Savant in the answer has always been something of an afterthought because

1. I use my real name on both SO and Codeplex, making the connection blindingly obvious.
2. Savant is free, open-source software from which I have not directly or indirectly earned one penny.
3. These haven't been cheerleading Savant-is-the-greatest-thing-since-Charles-Babbage-type answers, merely plain statements of fact.

Unfortunately some poor, benighted SO super-user 5 poor, benighted SO users decided the answers which don't explicitly disclose my connection to Simple Savant are unacceptable. These answers were flagged as spam and deleted, and my SO account was nuked back to 1 reputation point. Here's an example of one of my answers that was deleted:

Question: Is there a production grade SimpleDB library, preferable built in C#. If not, May I use the VB.NET library on a C# project as a reference?

My Answer: You should check out Simple Savant. It builds on the Amazon library to provide many additional features and is used in production or beta sites by quite a few folks.

Apparently an answer like the following would have been just fine:

You should check out Simple Savant. It builds on the Amazon library to provide many additional features and is used in production or beta sites by quite a few folks.

FULL DISCLOSURE: I'm the creator of Simple Savant. You should be aware that I may not have your best interests in mind. I may have spent several hundred hours of uncompensated time building and documenting this open-source project solely to lure unsuspecting application developers like yourself into an unhealthy, asymmetric relationship where you get valuable, free software and I get...er, warm fuzzy feelings because people are using my library. Also be aware that I strive to provide the highest quality of support to users of the Savant library. You may find yourself startled--nay--alarumed even by the rapidity with which answers to your questions stream across the Savant discussion forum. Consider yourself warned.

Of course, the delicious irony here is that the spam-offended SO user users who nuked my account only knew that I was connected to Simple Savant because I make that obvious by using my real name in both places!

UPDATE: Jeff Atwood explains in the comments that 5 StackOverflow users must flag a post as spam before it's deleted. If anything this makes the problem seem even worse. It's also worth noting that the posts marked as spam were all 7+ months old. I never received any warning or indication of a problem until I visited SO one day and found my reputation blanked.

UPDATE 2: After some more back and forth, the folks at StackOverflow said they were willing to chalk this up to a mistake and (mostly) restored my lost reputation.

Simple Savant Passes 2000 Downloads

Simple Savant, my .NET library for Amazon's SimpleDB service, passed 2000 downloads last week!

That's not a huge number compared to the most popular open-source projects. But considering that SimpleDB is still in beta and something of a niche product and that .NET apps built on SimpleDB are a niche within the niche, it's not too bad!

Also nice: So far Savant has been rated a 5-star project (the highest) by everyone who's reviewed it!

Handling Login Redirects When Mixing ASP.NET MVC 2 and WebForms

I've been migrating parts of GridRoom over to the ASP.NET MVC 2 framework as a precursor to implementing more advanced Ajax support (sans WebForms and ASP.NET AJAX).

One of the challenges to making WebForms and MVC coexist is handling redirects to the login page when making Ajax requests. This can happen when a user's session times out or if they log out in another browser window.

When this occurs the server returns a 302 temporary redirect to the login page, but the redirect status code is not exposed via the JavaScript XMLHttpRequest object. Instead, you get back a 200 OK response that happens to include the entire login page rather than the snippet of Html or Json payload you were expecting.

The simplest way I've found to handle this scenario (while retaining the existing WebForms login page) is to check for the presence of the ASP.NET MVC version header in every Ajax response. If the header doesn't exist the current document is simply reloaded to force a top-level redirect to the login page.

To make this work with jQuery, simply register a callback with the jQuery.ajaxSuccess() event and check for the "X-AspNetMvc-Version" response header:

   1: $(document).ajaxSuccess(function(event, XMLHttpRequest, ajaxOptions) {

   2:    // if request returns non MVC page reload because this means the user 

   3:    // session has expired

   4:    var mvcHeaderName = "X-AspNetMvc-Version";

   5:    var mvcHeaderValue = XMLHttpRequest.getResponseHeader(mvcHeaderName);

   6:  

   7:    if (!mvcHeaderValue) {

   8:        location.reload();

   9:    }

  10: );

The page reload may cause some JavaScript errors (depending on what you're doing with the Ajax response), but in most cases where debugging is off the user will never see these.

Preventing Multiple Form Submits In ASP.NET

Last year when I started working on the GridRoom Web site I spent some time researching the current best-practices for preventing multiple form submits in ASP.NET. (It was the first site I'd built from scratch using .NET, and I wasn't particularly fond of the architecture of existing .NET sites I had worked on.)

There are many options for preventing multiple submits, some of them quite complex and convoluted. But for most sites where the goal is simply to prevent users from accidentally clicking a submit button more than once the solution is very simple. Just override the OnLoad() method of your master page like this:

   1: protected override void OnLoad(EventArgs e)

   2: {

   3:     base.OnLoad(e);

   4:  

   5:     // prevents form from being submitted multiple times in MOST cases

   6:     // programatic client-side calls to __doPostBack() can bypass this

   7:     Page.ClientScript.RegisterOnSubmitStatement(GetType(), "ServerForm",

   8:                                                 "if (this.submitted) return false; this.submitted = true; return true;");

   9: }

This registers a script block with the client-side form's OnSubmit event that prevents the form from being submitted multiple times, regardless of how many times the user clicks (or how many submit buttons are on the form).

This solution does not prevent users from intentionally submitting the same form multiple times. (This could be done programmatically, by refreshing the post-submit page, or by hitting the back button and resubmitting.)

You'll need to handle intentional re-submits on a case-by-case basis in your business logic. But if you're looking for a very simple solution that simply prevents inexperienced users from being confused by unexpected errors or duplicate data this will do the trick.

Restore Your Missing Blogger Comments

I moved this blog to a custom domain about two months ago. Blogger makes the process very simple if you already own your own domain. I was up and running at blog.coditate.com after just a few clicks in the Blogger dashboard, and adding a new CNAME record for this domain.

A short time later I realized all the comments were missing from this blog. Apparently this has been quite a problem for months and based on the hundreds of unanswered complaints in the Blogger help forum, Google hasn't been very responsive or helpful in explaining why this happens or how to fix it.

Following the advice on this thread, I switched back to my old blogger-hosted address and the old comments reappeared. However, all new comments added since the domain switch had now disappeared. I switched back to my custom domain. Now both the old and new comments were gone!

Finally, after a couple more switches between my custom domain and blogger-address, all comments--both old and new--reappeared at once and seem to be sticking around. But the unreliability and lack of support from Google are not exactly confidence inspiring. In fact, it's downright scary when I think about how much of my important data is spread across Google's free services.

Forcing Firefox to Cache Secure Silverlight Video Content

The latest GridRoom release moves all video playback to the Web using a Silverlight-based video player. During my testing I had some problems getting Firefox caching to work properly with video content delivered via https.

At first I tried to deliver all video content over http, not https. But since the secure areas of the GridRoom site are accessed via https I was forced by Silverlight security restrictions to also deliver video content via https for now. (The Silverlight MediaElement does not support cross-scheme access--or mixed http/https content delivery).

During my initial testing with https Firefox seemed to cache video intermittently. Sometimes video was loaded from the cache and sometimes it was streamed again from the Web site, even during the space of a few minutes. Firefox provides an easy way to inspect cache contents using the "about:cache" URL, and after some poking around I noticed that https-delivered video was only held in the memory cache, while http-delivered video was stored in the disk cache. Caching appeared intermittent because my large video files were randomly evicting each other from the relatively small memory cache as I clicked around.

By default Firefox 3 does not use the disk cache for content delivered over https. Users may force this content to be disk-cached by setting the hidden browser.cache.disk_cache_ssl option to true, but this isn't something developers can depend on. The solution is to set a "Cache-Control: public" header for whatever content you want to be disk-cached. This makes Firefox behave like Internet Explorer, which disk-caches https-delivered content by default.

Be careful when using this workaround. The "Cache-Control: public" header tells any caching proxy servers between your server and the client that it's OK to cache your data for use by anyone!

All About Model Releases

Since I'm working on a product for managing sports video I occasionally need to use sports media for marketing, demos, tutorials, and other purposes. We have lots of photos and videos of our own kids--especially since my wife is quite the avid amateur photographer. But these sometimes include other folk's children as well as ours.

Not wanting to follow in the footsteps of Virgin Mobile, whose marketers were once sued after grabbing 15-year old Allison Chang's picture from Flickr for use in an Australian billboard campaign, it seemed a good idea to understand the legal requirements for obtaining parental permission before digging into our personal stock.

The best layman's resource I've found on the topic is this 15,000 word treatise compiled by photographer Dan Heller. If that seems like more than you ever wanted to know about model releases, you may prefer the shorter primer. Especially interesting are his reasons why you shouldn't consult a lawyer for advice.

How can it possibly take 15,000 words to explain the ins and outs of model releases? Well, it covers arcane distinctions such as whether I'm also violating Chang's rights by including a picture of the Virgin billboard on this blog. (No, because I'm using the picture for editorial commentary, rather than commercial purposes. And it doesn't matter that this blog includes advertisements or is associated with a software company.)

It turns out that model releases are required (for commercial uses only) when both of the following conditions apply:

• Subjects in the photo or video are individually recognizable
• These recognizable subjects appear to be endorsing a specific product, company or belief

Virgin's use of Chang's photo seems problematic because she is recognizable, appears to be endorsing a product, and possibly because her picture was used in a derogatory way. However, the Chang family's lawsuit was dismissed because the US court lacked jurisdiction over the Australian-based Virgin subsidiary.

For my purposes, I've concluded that using our personal stock of sports photos and videos is fine in most cases where the focus of attention is our own children and other kids are in the background and/or unrecognizable.

When Competitors Are Using Your Code...Does That Mean You're Being Too Transparent?

Openness and transparency are all the rage in startups these days. Don't hide the fact you're small. Be accessible to customers. Blog about your ideas and plans. Openly dissect your successes and failures. Share your work.

About 2 years ago I posted a video scene detection algorithm written in C# on this blog, while I was still just tinkering with the ideas behind GridRoom (and before I had decided to make a business of it).

Last night I was contacted by a developer who works for Agile Sports, makers of a high-end competitor to GridRoom. Their product is used by a number of NFL and Division I college football teams. The bottom end of their market is probably the largest high schools, while that's the top end of my intended market for GridRoom.

Apparently they've been using my scene-detection code with great success, but he had a couple of questions about how it worked. I'm pretty sure he didn't realized he was asking a competitor for help.

Simple Savant v0.5 Released

Simple Savant v0.5 is now available at CodePlex. This release includes the following features:

• Reliable cross-domain write support. Reliable-writes guarantee that you will never suffer partial data loss when writing associated data to multiple domains. System failure during a reliable-write will result in all or none of your data being stored in SimpleDB. This is accomplished using a 2-phased update. In the first phase your data is transactionally stored in a single domain. In the second phase your data is propagated to the final destination domains. If the initial propagation attempt is interrupted your data may remain in an inconsistent state for a short time. Other key points:
  • Cross-domain puts, batch-puts, and deletes are supported
  • Maximum of 25 sub-operations may be included in a single reliable-write operation
  • Conditional puts and deletes are not supported
  • Reliable-write adds about 150% overhead to each sub-operation. However, parallel propagation of sub-operations allows the overall write to complete with very little additional latency.
  • Propagation of sub-operations can fail permanently if SimpleDB limits are exceeded. This condition will be very obvious and requires manual intervention to correct.
• Support for custom constraint/validation logic that's invoked automatically whenever you get, select, put, or delete an item.
• Expanded and improved support for asynchronous operations
• Logging supported by the .NET Commons Logging API

The Codeplex documentation wiki will be updated shortly with more details. You may also want to check the release notes for a complete list of bug fixes and minor enhancements.

Simple Savant v0.4 Released

I've just released Simple Savant v0.4 at CodePlex. The new release includes the following features:

• Full-text searches of SimpleDB data using Lucene.NET 2.4
• Attribute spanning and compression to allow the storage of large property values in SimpleDB (attribute values are normally limited to 1024 bytes)
• Item versioning, including support for SimpleDB's new conditional puts to prevent updates with stale data
• Global and locally-scoped support for SimpleDB's new consistent reads

Rather than putting a lot of time into a blog post about the new features I've added documentation at the Simple Savant documentation wiki. You may also want to check the release notes for a complete list of bug fixes and minor enhancements.

Amazon SimpleDB Book Now Available from Packt Publishing

As a result of my work on Simple Savant I was asked to serve as technical reviewer of a new SimpleDB book by YLastic CTO Prabhakar Chaganti.  Amazon SimpleDB Developer Guide is now available from Packt Publishing. It's a great resource for folks getting started with SimpleDB and includes numerous code examples in Java, PHP, and Python.

Anatomy of a Failed PR Stunt

If you're an NFL fan you may already have seen the picture below. This is the story behind the Fire Snyder stadium sign--a self-indulgent PR stunt to promote GridRoom that was reported by dozens of blogs and newspapers but was ultimately a waste of time.

For over a decade the Washington Redskins have been run by a comically dysfunctional owner: Daniel Snyder. This in itself is not remarkable. Other sports team owners are regularly mocked for their parsimony, pomposity, and generally psychotic management decisions. Al Davis, George Steinbrenner, Peter Angelos--if you're a sports fan you know the list.

There are two remarkable things about Washington's dysfunction. First, the owner is both young and insane. Not second-generation-wealth young and insane or techno-wiz-kid-billionaire-with-bad-social-skills young and insane, but I-started-a-marketing-outsourcing-company-and-took-it-public young and insane. Who becomes a billionaire at 32 by starting a marketing outsourcing company? Clearly insane and probably unethical. No question about it.

Becoming an insane billionaire at 80, like Raiders' owner Al Davis, is a fairly normal thing. You're a young business phenom. You build your empire. You surround yourself with sycophants, ex-wives, and inheritance-hungry relatives. You become paranoid. You slip into pure crazy. There's a natural progression here that should not be violated. It takes a special man to become an old, crazy billionaire by 35. Daniel Snyder is that man.

The second remarkable thing about the Redskins is that, despite their dysfunction, despite losing more games than 21 other teams over the past decade, they are still the second-most valuable sports franchise in the world ($1.55 billion). They have the largest stadium in the NFL (92,000 seats), charge some of the highest ticket prices, yet have sold every available ticket since 1968, and also have the NFL's longest waiting-list for season tickets (155,000). The Redskins earned $90 million in profit last year, while the median NFL team earned just $26 million.

There are probably other factors contributing to the team's financial success, such as Federal lobbyists' entertainment slush-funds and multitudes of area transplants who support visiting teams, but it's fair to say that Washington has a deeply invested fan base.

This year fans may have reached their breaking point. It started when head coach Jim Zorn was stripped of his play-calling responsibilities, just 6 games into his second season. Snyder has fired and undermined several previous coaches in the same knee-jerk manner, and this time fan anger over the team's losing ways was directed primarily at the owner.

Then the team suddenly banned all fan-created signs at the stadium, perhaps worried about the potential embarrassment of anti-Snyder signs at an upcoming nationally-televised game. Predictably, the sign ban was implemented with heavy-handed boorishness. When local media reported that innocuous signs like the one pictured had been seized and tossed into the garbage by stadium security, anti-Snyder furor roared to new heights.

Now, I'm sure you're frantic at this point to hear exactly who was calling plays after Zorn lost the job. Well, that gets complicated. Cerrato hired an outside consultant to call plays for Zorn, but the outside consultant didn't really know much about the team so he only called some of the plays, and he relayed them through other coaches (not Zorn, he could only listen), and the other coaches chimed in occasionally with additional information on certain plays since the consultant didn't really know what he was doing. Zorn claimed he could still call plays during the 2 minute drill. And it was never clear what would happen if Zorn stopped "just listening" and overruled the consultant.

But the exact play-calling flow is beside the point. Something big happened here. Do you see it? Snyder missed a prime revenue opportunity. This is the man whose first mandate as owner was to move training camp back to the Washington area so he could sell tickets--and charge $10 parking fees--for fans to watch practice.

Once you've given up on the season and publicly emasculated your head coach you don't pay someone else to call the plays. You market the play-calling opportunity to your "whales" and let them pay you to call the plays! There are dozens of millionaire football fans who will never crack the exclusive NFL owners club. These folks would pay through the nose to call just one play--let alone all the plays for a drive or quarter. The possibilities are endless: you could have call-till-you score packages; 4th quarter blowout specials; design-and-call combo packages, etc.

This might even improve on-field results. Can you imagine opposing coaches game-planning for the irrational sequences that would result? But Snyder missed this one, and I almost feel bad for him. There's nothing sadder than an over-the-hill maestro, still flailing away before the orchestra when he can't keep up with the music.

Just as fan anger over the sign ban peaked I wrapped up the last release of GridRoom planned for 2009 and had some free time on my hands. I had an idea for turning the stadium itself into an anti-Snyder sign by having people wear color-coded shirts to form giant letters. I first discarded the idea as a waste of time, and then decided I could justify the effort as a long-shot publicity stunt if I mixed in some GridRoom advertising. Normally something like this would be impossible to coordinate, but it seemed like the perfect confluence of circumstances to give it a try. (Plus all the other anti-Snyder protests in the works were boring and lame.)

To get started I searched for high-resolution photos of the empty stadium, stadium seating charts, and tickets for sale on StubHub and other sites. I even managed to dig up a 10-year old ticket stub from the last game my wife and I attended at FedEx field before moving to Atlanta. With these resources in hand I figured out pretty quickly that each of the upper deck sideline sections forms a near-perfect grid. These rectangular sections would make it easy to test the concept without wasting much time. I whipped up some seating charts, set up the Fire Snyder Sign blog, emailed links to a few friends and bloggers, and posted links on Facebook.

The idea was an immediate hit. It was facebooked, tweeted, emailed, and posted to sports message boards by hundreds of people. It was linked by the Washington Post, ESPN, the Washington Examiner, DeadSpin, the LA Times, and dozens of smaller bloggers and newspapers. In just two weeks the blog was viewed by over 35,000 visitors. It was parodied by eTrueSports, and people even posted ticket ads on Craig's List mentioning the sign. Ticket holders asked how they could help, and some even planned to move into the sign sections to participate. I met Dan Steinberg of the Washington Post when the Redskins and Falcons played in Atlanta, and received interview requests from Bloomberg News. So in one sense the stunt was a moderate success, even though the sign ultimately fizzled.

But the stunt was a complete failure in terms of GridRoom publicity. A $25 Ad-Sense purchase could have replicated the traffic going to GridRoom. If anything this experience drove home how cost-effective Ad-Sense advertising is for advertisers--and how murderous Ad-Sense rates are for content producers. To date, 48,288 page views on the Fire Snyder Sign blog have generated just $17.68 in Ad-Sense revenues!

 

Simple Savant v0.3 Released

I've just released Simple Savant v0.3 at CodePlex. It's been a number of months since the last release so this one includes a number of bug fixes as well as a several significant new features:

• Typeless operations - All existing functionality has been exposed through methods not tied to .NET item types.
• Asynchronous operations - All existing operations may now be invoked asynchronously.
• Default property formatters are exposed for global replacement or reuse.

Each of these features is described in more detail below. See the release notes for a complete list of bug fixes and minor enhancements.

Typeless Operations

Most methods in the original Savant API required a generic type parameter. This made sense since object mappings could only be defined using attributes applied to your .NET types. This also allowed strong-typing of method parameters and return values. Here's an example:

   1: Guid personId = Guid.NewGuid();

   2: PersonItem person = Savant.Get<PersonItem>(personId);

   3: DateTime birthDate = person.BirthDate;

The biggest drawback of this approach is compile-time dependence on the generic parameter, which makes it difficult to invoke Savant methods from tools and infrastructure code that cannot explicitly reference item types in advance.

With Savant v0.3 we can rewrite the code above without referencing the PersonItem type:

   1: Guid personId = Guid.NewGuid();

   2: AttributeMapping itemNameMapping = AttributeMapping.Create("Id", typeof (Guid));

   3: ItemMapping mapping = ItemMapping.Create("Person", itemNameMapping);

   4:  

   5: AttributeMapping dateMapping = AttributeMapping.Create("BirthDate", typeof (DateTime));

   6: mapping.AttributeMappings.Add(dateMapping);

   7:  

   8: string[] propertyNames = mapping.AttributeMappings.Select(a => a.PropertyName).ToArray();

   9: PropertyValues values = Savant.GetAttributes(mapping, personId, propertyNames);

  10: DateTime birthDate = (DateTime)values["BirthDate"];

There is one significant difference between these examples. In the first example we're retrieving all attribute values mapped for the PersonItem type. In the second example we're retrieving only the "BirthDate" attribute. This makes perfect sense since that's the only attribute mapping added to our item mapping.

You may sometimes want to invoke the typeless methods while still working with .NET types. The ItemMapping and PropertyValues classes include helper methods to reduce the code required for this. The example below demonstrates these methods by building a complete mapping from the PersonItem type and then copying the returned values into a new instance of PersonItem:

   1: Guid personId = Guid.NewGuid();

   2: ItemMapping mapping = ItemMapping.Create(typeof(PersonItem));

   3:  

   4: PropertyValues values = Savant.GetAttributes(mapping, personId);

   5:  

   6: PersonItem person = (PersonItem)PropertyValues.CreateItem(typeof(PersonItem), values);

   7: DateTime birthDate = person.BirthDate;

Other benefits of dynamic mappings and typeless operations include the following:

• Data entity types can be decoupled from the Savant assembly if desired. All the functionality available through Savant mapping attributes can be replicated with dynamically constructed mappings.
• Data partitioning is now much easier with Savant. To use the same mapping across partitioned domains, simply change ItemMapping.DomainName value before invoking the typeless get, put, or select methods on Savant.

Asynchronous Operations

Any Savant method can now be invoked asynchronously using new extension methods that support lambda expressions. You'll need to import the Coditate.Savant.Async namespace to use these methods. Here's an example:

   1: using Coditate.Savant.Async;

   2: ...

   3: Guid personId = Guid.NewGuid();

   4: AsyncFunction<PersonItem> function = Savant.ExecuteAsync(s => s.Get<PersonItem>(personId));

   5:  

   6: // do something else useful

   7: PersonItem person = function.EndInvoke();

   8: DateTime birthDate = person.BirthDate;

Global Formatter Replacement

Savant has supported custom property formatters since the first release. But these could only be specified for individual properties using mapping attributes. You couldn't, say, replace the formatter used for all DateTime properties without applying a CustomFormatAttribute to every DateTime property defined by your application. Nor could you access the installed formatters for use in other contexts.

Both of these are possible now that the formatter registry is exposed on SavantConfig. Here's how to get and use a default formatter:

   1: PropertyFormatter formatter = Savant.Config.Formatter;

   2: ITypeFormatter dateFormatter = formatter.GetFormatter(typeof (DateTime));

   3: string date = dateFormatter.ToString(DateTime.Now);

And here's how to replace one of the default formatters globally:

   1: PropertyFormatter formatter = Savant.Config.Formatter;

   2: ITypeFormatter dateFormatter = new MyCustomDateFormatter();

   3: formatter.SetFormatter(typeof(DateTime), dateFormatter);