Exprimental economics helps solve complex business problems

How do you predict demand from your distributors? Would you try demand simulation, predictive analytics, or a complex mathematical model? Try experimental economics. Wired points us to a story (found via Techdirt) of Kay-Yut Chen who is an experimental economist at HP solving the complex demand forecast problems.

One of Chen's recent projects involved finding a way for H.P. to more accurately predict demand from its nine distributors, who collectively sell as much as $3 billion worth of H.P.'s products. The problem? Its distributors' forecasts for demand were frequently off by as much as 100 percent, wreaking havoc on H.P.'s production planning.

Chen's solution to the planning problem, which H.P. intends to test soon with one distributor, was to develop an incentive system that rewarded distributors for sticking to their forecasts by turning those forecasts into purchase commitments. In the lab, the overlap between distributors' forecasts and their actual orders using this system increased to as high as 80 percent. "That's pretty astonishing given that the underlying demand is completely random," Chen says.

The human beings are terrible at making rational decisions and the complex problems such as demand forecast cannot really be solved by complex modeling algorithms or predictive analytics. Applying the economics of incentives to such problems is likely to yield better results. Freakonomics explains the creative use of economics of incentives in great depth. Dan Ariely writes in Predictably Irrational about people predictably making irrational decisions and how it breaks the rules of traditional economics and free markets that are purely based on demand and supply ignoring the human irrationality.

There is a lesson for an enterprise software vendor to design human-centric software that supports human beings in complex decision management process. Good news is that I do see the enterprise software converging towards social computing. Topics such as security that have been considered highly technical are being examined with a human behavior lens ranging from cognitive psychology to anthropology of religion.

I would welcome a range of tools that could help experimental economics gain popularity and dominance in the mainstream business. For instance behavior-based AB testing can be set up in a lab to test out hypothesis based on experimental economics and the results of the experiment could be directly fed to a tool that reconfigures an application or a website in real-time.

SaaS platform pitfalls and strategy - Part 2

In part 1, I discussed my views on the top 10 mistakes that vendors make while designing a SaaS platform as described in the post at GigaOM. This post, the part 2, has my strategic recommendations to SaaS vendors on some of the important topics that are typically excluded from the overall platform strategy.

Don't simply reduce TCO, increase ROI: According to an enterprise customer survey carried out by McKinsey and SandHill this year, the buying centers for SaaS are expected to shift towards the business with less and less IT involvement. A SaaS vendor should design a platform that not only responds to the changing and evolving business needs of a customer but can also adapt to changing macro-economic climate to server customer better. Similarly a vendor should carve out a Go To Market strategy targeting the businesses to demonstrate increased ROI and not necessarily just reduced TCO even if they are used selling a highly technical component to IT.

The Long Tail: SaaS approach enables a vendor to up-sell a solution to existing customers that is just a click-away and does not require any implementation efforts. A vendor should design a platform that can identify the customer's ongoing needs based on the current information consumption, usage, and challenges and tap into a recommendation engine to up-sell them. A well-designed platform should allow vendors to keep upgrade simple, customers happy, and users delighted.

Hybrid deployment: The world is not black and white for the customers; the deployment landscape is almost never SaaS only or on-premise only. The customers almost always end up with a hybrid approach. A SaaS platform should support the integration scenarios that spans across SaaS to on-premise. This is easier said than done but if done correctly SaaS can start replacing many on-premise applications by providing superior (non)ownership experience. A typical integration scenario could be a recruitment process that an applicant begins outside of a firewall on a SaaS application and the process gradually moves that information into an enterprise application behind the firewall to complete the new hire workflow and provision an employee into the system. Another scenario could be to process a lead-to-order on SaaS and order-to-cash on on-premise.

Ability to connect to other platforms: It would be a dire mistake to assume standalone existence of any platform. Any and all platforms should have open, flexible, and high performance interfaces to connect to other platforms. Traditionally the other platforms included standard enterprise software platforms but now there is a proliferation in the social network platforms and a successful SaaS player would be the one who can tap into such organically growing social networking platforms. The participants of these platforms are the connectors for an organization that could speed up cross-organizational SaaS adoption across silos that have been traditional on-premise consumers.

Built for change: Rarely a platform is designed that can predict the technical, functional, and business impact when a new feature is included or an existing feature is discarded. Take internationalization (i18n) as an example. The challenges associated to support i18n are not necessarily the resources or money required to translate the content into many languages (Facebook crowdsourced it) but to design platform capabilities that can manage content in multiple languages efficiently. Many platform vendors make a conscious choice (rightfully so) not to support i18n in early versions of the platform. However rarely an architect designs the current platform that can be changed predictably in the future to include a feature that was omitted. The design of a platform for current requirements and a design for the future requirements are not mutually exclusive and a good architect should be able to draw a continuum that has change predictability.

Virtualize everything: Virtualization can insulate a platform from ever-changing delivery options and allow vendors to focus on the core to deliver value to the applications built on the platform. A platform should not be married to a specific deployment option. For instance a vendor should be able to take the platform off Amazon's cloud and put it on a different cluster without significant efforts and disruptions. The trends such as cloud computing have not yet hit the point of inflection and the deployment options will keep changing and the vendors should pay close attention to the maturity curve and hype cycle and make intelligent choices that are based on calculated risk.

Vendors should also virtualize the core components of the platform such as multi-tenancy and not just limit their virtualization efforts to the deployment options. Multi-tenancy can be designed in many different ways at each layer such as partitioning the database, shared-nothing clusters etc. The risks and benefits of these approaches to achieve non-functional characteristics such as scalability, performance, isolation etc. change over a period of time. Virtualizing the multi-tenancy allows a vendor to manage the implementation, deployment, and management of a platform independent of constantly moving building components and hence guarantee the non-functional characteristics.

Don't bypass IT: Instead make friends with them and empower them to server users better. Even if IT may not influence many SaaS purchase decisions IT is politically well-connected and powerful organization that can help vendors in many ways. Give IT what they really want in a platform such as security, standardization, and easy administration and make them mavens of your products and platform.

Platform for participation: Opening up a platform to the ecosystem should not be an afterthought instead it should be a core strategy to platform development and consumption. In early years eBay charged the developers to use their API and that inhibited the growth which later forced eBay to make it free and that decision helped eBay grow exponentially. I would even suggest to open source few components of the platform and also allow developers to use the platform the way they want without SaaS being the only deployment option.

Platform Agnostic: The programming languages, hardware and deployment options, and UI frameworks have been changing every few years. A true SaaS platform should be agnostic to these building components and provide many upstream and downstream alternatives to build applications and serve customers. This may sound obvious but vendors do fall into "cool technology" trap and that devalues the platform over a period of time due to inflexibility to adopt to changing technology landscape

Make to think and think to make - Design thinking helps a start-up radio show compete with NPR's Morning Edition

The upstart radio show Takeaway's producers worked with the d.school at Stanford to apply design thinking approach to their show that competes with NPR's Morning Edition. It is quite an interesting story about how a legacy media industry can discard a traditional approach and embrace design thinking to rapidly iterate on the design of a radio show.

"A three-day crash course taught the producers the basic steps of d.school innovation: observe, brainstorm, prototype, and implement; repeat as necessary."

"The program's central idea is a daily question that audiences are asked to riff upon, either by calling in or by emailing. Their responses are then woven into the rest of the show's programming."

Not spelled out in so many words in the story but this is a good example of user-centered and participatory design with a crowdsourcing twist to it.

"But recognizing shortcomings and criticism and iterating quickly is one of the design process's core principles. The students in a d.school course called Design + Media, who are using the show as a class project, are helping producers generate ideas and track online response. For example, they're following Twitter streams to find out which questions and other parts of the broadcast are producing the strongest reactions."

Once again this story reinforces that design is an ongoing process and design thinking is not about talking but making and generating more ideas while making to change what you just made.

SaaS platform - design and architecture pitfalls - Part 1

I cannot overemphasize how critical it is to get the SaaS platform design right upfront. GigaOM has a post that describes the top 10 mistakes that vendors make while designing a SaaS platform. I would argue that many of these mistakes are not specific to a SaaS platform but any platform. I agree with most of the mistakes and recommendations, however I have quite the opposite thoughts about the rest. I also took an opportunity to think about some of the design and architectural must have characteristics of a SaaS platform that I will describe in the part 2 of this post.

1) Failing to design for rollback

"...you can only make one tweak to your current process, make it so that you can always roll back any code changes..."

This is a universal truth for any design decision for a platform irrespective of the delivery model, SaaS or on-premise. eBay makes it a good case study to understand the code change management process called "trains" that can track down code in a production system for a specific defect and can roll back only those changes. A philosophical mantra for the architects and developers would be not to make any decisions that are irreversible. Framing it positively prototype as fast as you can, fail early and often, and don't go for a big bang design that you cannot reverse. Eventually the cumulative efforts would lead you to a sound and sustainable design.

2) Confusing product release with product success

"...Do you have “release” parties? Don’t — you are sending your team the wrong message. A release has little to do with creating shareholder value..."

I would not go to the extreme of celebrating only customer success and not release milestones. Product development folks do work hard towards a release and a celebration is a sense of accomplishment and a motivational factor that has indirect shareholder value. I would instead suggest a cross-functional celebration. Invite the sales and marketing people to the release party. This helps create empathy for the people in the field that developers and architects never or rarely meet and this could also be an opportunity for the people in the field to mingle, discuss, and channel customer's perspective. Similarly include non-field people while celebrating field success. This helps developers, architects, and product managers understand their impact on the business and an opportunity to get to know who actually bought and started using their products.

5) Scaling through third parties

"....If you’re a hyper-growth SaaS site, you don’t want to be locked into a vendor for your future business viability..."

I would argue otherwise. A SaaS vendor or any other platform vendor should really focus on their core competencies and rely on third parties for everything that is non-core.

"Define how your platform scales through your efforts, not through the systems that a third-party vendor provides."

This is partially true. SaaS vendors do want to use Linux, Apache, or JBoss and still be able to describe the scalability of a platform in the context of these external components (that are open source in this case). The partial truth is you still can use the right components the wrong way and not scale. My recommendation to a platform vendor would be to be open and tell their customers why and how they are using the third party components and how it helps them (the vendor) to focus on their core and hence helps customers get the best out of their platform. A platform vendor should share the best practices and gather feedback from customers and peers to improve their own processes and platform and pass it on to third parties to improve their components.

6) Relying on QA to find your mistakes:

"QA is a risk mitigation function and it should be treated as such"

The QA function has always been underrated and misunderstood. QA's role extends way beyond risk mitigation. You can only fix defects that you can find and yes I agree that mathematically it is impossible to find all the defects. That's exactly why we need QA people. The smart and well-trained QA people think differently and find defects that developers would have never imagined. The QA people don't have any code affinity and selection bias and hence they can test for all kinds of conditions that otherwise would have been missed out. Though I do agree that the developers should put themselves in the shoes of the QA people and make sure that they rigorously test their code, run automated unit tests, and code coverage tools and not just rely on QA people to find defects.

8) Not taking into account the multiplicative effect of failure:

"Eliminate synchronous calls wherever possible and create fault-isolative architectures to help you identify problems quickly."

No synchronous calls and swimlane architecture are great concepts but a vendor should really focus on automated recovery and self-healing and not just failure detection. A failure detection could help vendor isolate a problem and help mitigate the overall impact of that failure on the system but for a competitive SaaS vendor that's not good enough. Lowering MTBF is certainly important but lowering MDT (Mean down time) is even more important. A vendor should design a platform based on some of the autonomic computing fundamentals.

10) Not having a business continuity/disaster recovery plan:

"Even worse is not having a disaster recovery plan, which outlines how you will restore your site in the event a disaster shuts down a critical piece of your infrastructure, such as your collocation facility or connectivity provider."

Having a disaster plan is like posting a sign by an elevator instructing people not to use it when there is a fire. Any disaster recovery plan is, well, just a plan unless it is regularly tested, evaluated, and refined. Fire drills and post-drill debriefs are a must-have.

I will describe some of the design and architectural must have characteristics of a SaaS platform in the part 2 of this post.