5 Techniques To Deal With Spam: Open Letter To Twitter

I love Twitter, but lately, I am getting annoyed by Twitter spam and I'm not the only one. I don't want Twitter spam to become email spam. I don't want to whine about that either, so I spent some time thinking about what Twitter could do to deal with spam. Consider this an open letter to Twitter.

Facebook's privacy settings are the new programming a VCR. Google has been criticized a lot about profiting from content farms. I believe that all the major players are playing a catch-up game. A lot of people have stared to complain about LinkedIn spam as well. Quora went into different direction — where they started out with a strict upfront policy regarding who can join Quora, ask questions, answer questions etc. — to maintain the quality of their service. Strict upfront policy hampers new user acquisition and adoption but could ensure better quality where as liberal policy accelerates the user acquisition with a risk of service being abused. I do believe that there's a middle ground that these services could thrive for by implementing clever policies.

Here are five techniques that Twitter could use to deal with spam:

1. Rely on weighted rank based on past performance: I ran a highly unscientific experiment. I kept a record of all the accounts that I reported as spammers on Twitter in the last few days. I went back every few minutes, after reporting an account, to see whether that account was suspended. It took Twitter some time before that happened. Every single account that I have reported so far has been suspended. I don't think Twitter is using that knowledge. If it did, my subsequent actions would have resulted into quicker suspension. Learn from Craigslist. Craig Newmark will tell you all about community-based flagging. Instrument the system to rely on reputation of power users — who are savvy enough to detect spam — to suspend a spammer's account. If it turns out that it's not a spam, give an opportunity to the account owner to appeal. Spammers don't waste time arguing; they simply move on.

2. Expand categories to match how people consume: Create a separate "unsolicited" category to receive mentions and replies from people whom you don't follow. This could be a separate window in a Twitter client that replaces the current "replies and mentions" window. Require Captcha for direct replies (and not mentions) for the conversations where both the accounts don't follow each other to stop automated spam. Everything else, including real spam, goes into "mentions", which is now a new category, that can be consumed in a separate window leaving the "replies" window clean.

3. Remove spam tweets from the stream: Many users don't consume their mentions or replies in real-time or even in near real-time. Mark the tweets spam once you suspend the account and require the Twitter clients to remove them from users' stream in real time. No API restrictions and no throttling. If you do it right and spam gets detected within a few seconds, the account can be suspended in no time, and the tweets are removed even before the most users would even see them. Emails can't be recalled, the tweets can be, if Twitter wants it to. Let's do it.

4. Focus on new accounts: Set a reasonably low limit on number of tweets per hour on a new account. A first-time genuine Twitter user doesn't go from 0-100 in a day, but a new spammer certainly would. Focus energy on new accounts; spammers don't wait for a few weeks or months to start spamming. The current "verified" account feature is a black magic. Open it up to all the people and use standard means such as cell phone, credit cards, and other identities to verify their Twitter accounts. These accounts enjoy the benefit of doubt - an upfront requirement of multiple signals before their accounts are suspended. Spammers don't want to verify themselves.

5. Find and fight bots with bots: There are a bunch of bots out there that look for the words such as iPad, iPhone, and XBOX in your tweets and then they spam you. Twitter can crate their own bots to tweet these words to catch these spam bots and more importantly harvest the links that they are tweeting to detect other spammers. Twitter's own bots would obviously be far more intelligent than the spam bots since they would have access to a lot more information that the spam bots don't.

The spammers do catch up, but if Twitter spends a little time and energy, they can stay ahead in this game. They can even lead the pact of social media companies on how to deal with spam.

Update: As soon as I published this post, I tweeted it and copied Del Harvey on it. She immediately responded to the post. You can read her response here. I really appreciate Twitter responding to this. My take on the response is that they seem to understand what the issues are and how they might solve them, but they haven't fully managed to execute on it, so far. I don't agree with their feedback on issue #2 calling it non-safety. Users see Twitter as one integral product where spam is very much part of it. Personally, I don't think of spam as a security issue for me. It's just plain annoyance. Executing on these ideas will matter the most. Let's hope Twitter gets behind this with full momentum and it doesn't become a "project".

Social Shaming

An interaction designer, Joshua Kaufman, had his MacBook stolen a few days back. He is a smart dude. He had installed an app called Hidden on his MacBook before it was stolen. He tracked down the thief and asked the Oakland PD to catch him. They said no. He was frustrated, obviously. He published all the details regarding the theft including the picture of the guy who stole his MacBook on his blog. This story went viral on Twitter and Facebook and made it almost impossible for the cops to ignore it. Oakland PD found the guy and arrested him. Since then the story has been picked up by many major media outlets and became sort of a sensation.

Social shaming works.

There's a fine line between peer pressure and social shaming. Many car dealerships in the US have a whiteboard that tracks which sales reps sold how many cars. They also ring a bell every time someone sells a car. It's a cheesy thing to do, but it sends a clear message to other people to be more aggressive; it's indeed a form of peer pressure. It's also an efficient technique to motivate the kids.

In fact, it's one of the most important gamification elements.

Public shaming has been used in many different ways e.g. send an email out to all the sales people with a list of people highlighted in red that haven't updated the CRM system. I know of a company that had a practice in place to publicly give a "D'oh! award" to a developer who broke the nightly build. Social shaming is essentially public shaming using social media. During my discussion with many enterprise social software vendors, analysts, and thought leaders I have repeatedly argued that changing end users' behavior is less likely to succeed unless there's a significant upside for the end users. What is more likely to work is codifying the real life end user behavior in the software that they use. Social shaming is one of those. One of the ways to achieve this could be by designing software that promotes radical transparency, signals one's successes to the other, and nudges them to excel without embarrassing them.

Disruptive Cloud Start-Ups - Part 2: AppDirect

Check out the first post of this series on NimbusDB, if you haven't already seen it. This post is about AppDirect. I met with Nicolas Desmarais, a co-founder and the CEO of AppDirect and had a long discussion regarding their current solutions and future strategy. AppDirect is an app store for small businesses. The developers can integrate their applications with AppDirect and AppDirect manages the experience of selling, provisioning, and billing with a 70-30 revenue split with the developers. They also have a white label app store solution that they sell to large customers such as ISPs who can sell these same applications to their customers.

Let's get the things out of the way that I didn't like about them.

The downside:

The target market that comprises of small businesses is extremely difficult to reach to and to market to. This gets even more difficult when the company trying to market is a young start-up and the customers are "S" in SMB. These customers have very different kind of requirements. They look for simple solutions that are not very expensive and have predictable SLA with a clear local support model and not the ones that come with enterprise grade features such as end-to-end integration, single sign on etc. Intuit has owned this channel for a while via Quickbooks and their SMB marketplace (the partner platform) is a great example of selling go-to-market services to other ISVs. AppDirect will have to work much harder if they want to work this channel.

So, why do I think they are disruptive?

The upside:

AppDirect is platform-agnostic. The developers can write applications in any language and run it on any platform as long as they integrate with AppDirect's end points (the APIs). The ISVs or PaaS providers have traditionally locked developers into their platform. That lock-in now goes away.

Even though the telcos are not the most innovative companies, they are laggards with a pile of cash, a ton of customers, and good margins. I believe that telcos can be great enterprise software vendors for SMB. Instead of spending money on the marketing efforts, if AppDirect can convince the Telcos and ISPs to bundle their white label solution, it's a win-win situation. This business alone can make them profitable. What you need is a small number of large customers. Long tail can always be an added bonus.

The team is talented and they have got a good product with some early customers. If they can execute on their vision and pivot as necessary, they're on to something,

Check out their slides and presentation:

Disruptive Cloud Start-Ups - Part 1: NimbusDB

Being at Under The Radar (UTR), watching disruptive companies present and network with entrepreneurs, thought leaders, and venture capitalists is an annual tradition that I don't miss. I have blogged about disruptive start-ups that I saw in the previous years. The biggest exit out of UTR, that I have witnessed so far, is Salesforce.com's $212 million acquisition of Heroku. This post is about one of the disruptive start-ups that I saw at UTR this year - NimbusDB.

I met with Barry Morris, the CEO and Founder of NimbusDB at a reception the night before. I had long conversation with him around the issues with legacy databases, NoSQL, and of course NimbusDB. I must say that, after long time, I have seen a company applying all the right design principles to solve a chronic problem - how can you make SQL databases scale so that they don't suck.

One of the main issues with the legacy relational databases is that they were never designed to scale out to begin with. A range of NoSQL solutions addressed the scale-out issue, but the biggest problem with a NoSQL is that NoSQL is not SQL. This is why I was excited when I saw what NimbusDB has to offer: it's a SQL database at the surface but has radically modern architecture underneath that leverages MapReduce to divide and conquer queries, BitTorrent for messaging, and Dynamo for persistence.

NimbusDB's architecture isolates transactions from storage and uses asynchronous messaging across nodes - a non-blocking atom commit protocol - to gain horizontal scalability. At the application layer, it supports the "most" of SQL 99 features and doesn't require the developers to re-learn or re-code. The architecture doesn't involve any kind of sharding and the nodes can scale on any commodity machine on a variety of operating systems. This eliminates an explicit need of a separate hot back-up since any and all nodes serve as a live database in any zone. This makes NimbusDB an always live system, which also solves a major problem with traditional relational databases - high availability. It's an insert only database and it versions every single atom/record. That's how it achieves MVCC as well. The data is compressed on a disk and is accessed from an in-memory node.

I asked Barry about using NimbusDB as an analytic database and he said that the database is currently not optimized for analytic queries, but he does not see why it can't be tuned and configured as an analytic database since the inherent architecture doesn't really have to change. Though, during his pitch, he did mention that NimbusDB may have challenges with heavy reads and heavy writes. I personally believe that solving a problem of analytic query on large volume of data is a much bigger challenge in the cloud due to the inherent distributed nature of the cloud. Similarly, building a heavy-insert system is equally difficult. However, most systems fit somewhere in between. This could be a great target market for NimbusDB.

I haven't played around with the database, but I do intend to do so. On a cursory look, it seems to defy the CAP theorem. Barry seems to disagree with me. The founders of NimbusDB have great backgrounds. Barry was the CEO of IONA and Streambase and has extensive experience in building and leading technology companies. If NimbusDB can execute based on the principles it is designed on, this will be a huge breakthrough.

As a general trend, I see a clear transition, where people finally agree that SQL is still a preferred interface, but the key is to rethink the underlying architecture.

Update: After I published the post, Benjamin Block raised concerns around NimbusDB not getting the CAP theorem. As I mentioned in the post, I also had the same concern, but I would give them benefit of doubt for now and watch the feedback as the product goes into beta.

Check out their slides and the presentation:

Slides:

Presentation:

Gamification Of Enterprise Applications

Gamification is a hot topic for consumer applications. It is changing the way the companies, especially the start-ups, design their applications. The primary drivers behind revenue and valuation of consumer software companies are number of users, traffic (unique views), and engagement (average time spent + conversion). This is why gamification is critical to consumer applications since it is an effort to increase the adoption of an application amongst the users and maintain the stickiness so that the users keep coming back and enjoy using the application.

This isn't true for enterprise applications at all.

For consumer applications, the end user and the buyer (if they pay to use) are the same. e.g. Amazon, eBay, Google, Facebook, LinkedIn etc. For enterprise applications, the end user is not the buyer. The buyers of enterprise applications write a check but don't use the applications, and even worse, the end users have a little or no influence on what gets bought. The on-premise ISVs don't directly benefit from user adoption, once the software is sold. This is also true for cloud or SaaS solutions except that there is no shelfware in SaaS. I would argue that the enterprise ISVs, on-premise as well as SaaS, would in fact benefit, in short term, from reduced user adoption since they would save money by supporting fewer users and reduced activity. Obviously, this is a very short-sighted and myopic view. I hope that the enterprise ISVs don't actually think that way since broader user adoption and deeper engagement are certainly important for longer term growth that allows the ISVs to build brand loyalty, develop stronger customer base, and gain an opportunity to up-sell and cross-sell.

The fundamental reason behind poor adoption of the enterprise applications is that they are simply not easy-to-use and they almost always come in the way to get the actual work done. In many cases, they are designed to be orthogonal to the actual business process that it is supposed to help an end user with. Also, in most cases, these applications are designed top-down to serve the needs of senior management and not the real needs of end users e.g. a CRM system that helps management to run pipeline reports but doesn't help a rep to be more efficient and agile. In cases where broader adoption for enterprise applications is required, it is typically achieved via a top-down mandate e.g. annoying reminder emails to fill out time sheets. The end users don't see themselves as a clear beneficiary of these applications.

Simply put, the approach to gain user adoption for consumer applications is a "carrot" and for the enterprise applications it is a "stick". But, it doesn't have to be that way. There's a significant potential to apply gamification elements to increase the end user engagement for the enterprise applications, make them sticky and fun to use, and make it a win-win situation for the buyers as well as the end users.

Cater to perpetual intermediaries:

Have you ever played Angry Birds? If not, I would highly encourage you to do so. It serves the category of people known as "casual gamers". These games have pretty much zero adoption barrier for a novice, but when you get serious, there are enough challenges in the game as you progress to keep you entertained and bring you back. The equivalent of casual gamers in the enterprise applications are known as "perpetual intermediaries". They don't want to become power users, but they don't want to stay beginners as well. The tool should have zero barrier for a first time user and should have affordances that encourages users to explore and learn more. Microsoft has done a pheneomenal job with Word and Excel. They are extremely easy to use for a person who has never used these tools before and they provide further discovery via contextual menus and reassurance via drop-down menus (and ribbon in later versions) in the journey of becoming a perpetual intermediary. That's exactly how I expect all the enterprise applications should behave.

Let users leverage serendipity:

One of the early features of Google Apps that I really liked: when user logs into Google Apps for a specific domain, she can see other people in the same domain (same company) who are also using Google Apps. This was not a task that someone explicitly wanted to accomplish, but sheer serendipity allowed them to discover other people and eventually helped collaborate with them. If there's an element of surprise in any app, that experience typically leaves positive impact on a user. How many times did you run into someone at a cafetaria or in a hallway and found that short and tacit conversation extremely valuable? The ISVs should thrive to create this experience in their applications. Foursquare's feature to let users know who else is at a venue, Facebook Places' push notification to notify when friends check-in at a place close by, and certain activity feeds that passively push information to users are all examples that leverages serendipity.

Design for teams over individuals:

The gamification elements for consumer applications target individuals, but that's not how corporations are run. In these corporations, the work gets done by a team and not by individuals — it's a team sport. It's the team and not the individuals that wins and loses. Also, for the most consumer applications, the individuals don't compete with other individuals on aspects beyond the application. The employees in a corporation aren't necessarily known for healthy competition and the gamification rewards might aggravate the existing rivalry. The badges are a digital reward, an accomplishment of some kind. Consumer companies are still struggling to take the badges beyond the reputation. I clearly see an opportunity to link the reputation, gained through some kind of contribution, to an economic reward. I know of a case where a manager had set aside 20% team bonus based on contribution to a group WIki as means to open up information and help others. It did work. However, I would be careful in setting up these kind of systems. The reward model, if not applied correctly, could backfire. But, on the other hand, it's a gamification element that holds significant potential. It'a dagger, use it carefully.

Balance simplicity and productivity:

Simplicity is one of the simplest (no pun intended) yet the most ignored and least understood gamification element. As I mentioned above, the systems that are designed for the perpetual intermediaries should be simple to get started. These systems could potentially get far more complex as you explore more and more features. But, there's another class of systems that people only occasionally use e.g. leave request, annual goal setting etc. It's far more important to keep these systems simple at all levels. Imagine the experience of going from one carnival stall to another and play all the games. You need very little or no instructions. These games at carnival are derived from a few basic games with a few twists, but these twists do not require people to go through a steep learning curve. That's how the applications that people rarely use should be designed; it should use the affordances and principles that the users have witnessed and experienced some place else and it should be broken down like carnival stalls to make the journey easy and fun.

The serious gamers prefer power over simplicity. They like to use shortcuts and a zillion combinations of all the keys on their consoles to get moving quickly inside the game. This is exactly the behavior of the power users of enterprise applications. An Accounts Receivables (AR) interface should not force an AR clerk to learn how to create an invoice every time she opens the application. She has learned the ropes and she expects to be productive and she wants to be faster and better than others. The tools should provide enough "power" features to such users to make them successful.

Photo credit: ccarlstead

Taking The Quotes Out Of "Design Thinking"

Bruce Nussbaum, a design thinking thought leader and a professor of Innovation and Design at Parsons The New School of Design, recently wrote that Design Thinking Is A Failed Experiment.

He claims that:

"Design Thinking has given the design profession and society at large all the benefits it has to offer and is beginning to ossify and actually do harm."

Rubbish.

I would argue otherwise. Design thinking is not a catchphrase anymore, and that perhaps is an issue for someone like Bruce who wants to invent a new catchphrase to sell his book. When I tweeted his post, Enric Gili - a friend, co-worker, and a design thinker whom I respect - had to say this:

I couldn't agree anymore. I have learned, practiced, and taught design thinking, for living. I have worked with folks from IDEO, closely, very closely. I have mentored students at Stanford d.school and I live and breathe design thinking. I don't think of it as a method that goes out of fashion. For me, it's a religion, a set of values, and an approach that I apply to all things that I do on daily basis.

I have taken the quotes out of "design thinking".

Just as I don't get excited by the rounded corners and gradients of Web 2.0 I don't think of design thinking as voodoo dolls. To some, this appears to be a failure of design thinking. Design thinking has gone mainstream; it is not dead. What is dead is a belief that it's a process framework that can fix anything and can even cook dinner for you. Design thinking is an approach that codifies a set of values. Design thinking is not an innate skill. It can be taught, gained, and practiced.

"I place CQ within the intellectual space of gaming, scenario planning, systems thinking and, of course, design thinking. It is a sociological approach in which creativity emerges from group activity, not a psychological approach of development stages and individual genius."

Design thinking is ambidextrous; it advocates abductive as well as deductive thinking. The "design" in design thinking is an integrative discipline. As my boss used to tell me, you can't have Ph.D in design. Unless you're a smartypants clever clogs, it doesn't make sense. If CQ is a sociological-only approach, it fundamentally defies the inclusive and integrative values of design, which is a vital driver for creativity.

"It’s 2020 and my godchild Zoe is applying to Stanford, Cambridge, and Tsinghua universities. The admissions offices in each of these top schools asks for proof of literacies in math, literature, and creativity. They check her SAT scores, her essays, her IQ, and her CQ."

It's 2020 and IDEO has gone out of business and so is d.school. I am applying for a new job and they measure my CQ. I miserably fail at this CQ thing, perhaps. Do I care? Absolutely not. I have got my design thinking value system that may not be catchy to sell a book, but good enough to get my job done, spectacularly.

Creative Quotient? Give me a break.

It's 1999 Again: The Bubble 2.0 And Talent Wars Of The Silicon Valley

I have been living in the Silicon Valley for a while, and sure enough I haven't forgotten the dot com days. A few days back, on my way to the San Francisco airport, I saw a billboard by aol advertising that they are cool (again!). I also observed that parking lots alongside 101 weren't that empty. I told myself "man! this does feel like 1999".

The smart people - entrepreneurs, VCs, and analysts - that I talk to, tell me that we're in a bubble. They call it Bubble 2.0. Perhaps, they're right. The company valuations are through the roof. Facebook is valued around $75 billion and Color, on the launch day, had $40 million in the bank. The angel, super angel, and incubator investment deal flow is bringing all the talent to the Valley and all these young smart entrepreneurs are working on some of the coolest things that I have ever seen. But, there's a talent side that I am worried about. What this influx of easy venture capital has ensued is companies waging talent wars. For companies such as Google, attracting and retaining talent has become very difficult. Facebook and Twitter are new Google and Quora is new Facebook. The talent acquisitions that worked in the past, such as Facebook acquiring Friendfeed, have started to fall apart since the founders realized that serial entrepreneurship is a much better option that allows them to control their destiny against trusting someone else's innovation engine.

I like the creative ways in which the start-ups try to attract the talent. When Google launched a sting operation against bing, they took the honeypot keyword "hiybbprqag" used in the sting operation to register the domain http://www.hiybbprqag.com and redirected it to the Google Jobs page. They received a few thousand resumes that week. I am seeing more and more creative techniques that the companies use to attract talent. The value proposition for a killer designer or a super-geek programmer to work for you has to extend beyond the basics in the valley. This is especially true under current circumstances where there is a stunningly short supply of designers and developers in the Valley.

The talent war is for real. It's easy to get money and get started on an idea, but a real success requires a great team composition that is not easy to achieve. But, that's the reality of the start-up world and we should recognize that the people are even more important than ever before. If you think retaining talent was hard, gaining talent is much harder. I also foresee that these new millionaires will most likely angel invest their money into new start-ups. This floodgate will result into more start-ups competing for talent and possibly with the marketing budget of the incumbents. But, then, if we believe, it's a bubble, it gotta burst one day, and when that happens, it won't be pretty.