Crowd Computing and 6 steps to make it work for you

http://codehandyman.com/2009/03/crowd-computing-and-6-steps-to-make-it.html

I have a lot of people coming to me for advice on their websites. Here is my personal philosophy on the topic and what drives my advice to pretty much everyone these days.

First of all, websites as we generally think of them, are dying or dead. The idea of a static page with some sharp graphics, perhaps a fancy flash logo animation etc. has largely become irrelevant to most people in the past few years.

I like to correct people and say web presence instead of website. Web presence means just that, your presence on the web, in all forms. Your emails, your facebook pages, profiles and groups, your blogs, your pages on other social networks like ning.com, linkedin, twitter, second life, myspace, wikipedia, and your AIM/MSN/Yahoo chat handles. That list seems to be getting longer and longer, but essentially its all just where you're at on the web.

It may seem intimidating to have all these things out there and feel like you HAVE to be on all of them. The good news is, you dont. You only have to participate where you feel most comfortable, and where you get the most "bang for your buck". Buck in this case translates to time and effort.

I watched from the outside for 10 years this blogging and social networking phenomenon, really staying out of it and focussing on my job of building dedicated flash applications for online training. I stayed out because what I saw was millions of people with what I would characterize as a very shallow web "presence" in a wide range of media. I would call it spamminating, or just dipping your toe in a lot of different pools. That sort of thing didnt mesh well with my personality. I like to get into something and go very deep, explore it to as far as its limits, and master it, THEN move on to the next thing.

As the technology evolved, I did too. With the advent of more and more cross linking between them, badges, open application architectures, and flexibility, I saw a new opportunity to dive deep. With all the cross linking done automatically by things like badges, you can now write once, publish everywhere. That exicites me.

I read this line in a book called What Would Google Do by Jeff Jarvis that talked about how linked in disseminates the hundreds of millions of photos by using algorithms that monitor the cross promotion and cross talk between members about photos, and basically could use the wisdom of the crowd(crowd computing?) to determine interestingness.

Now, like the new ad companies, my audience gets my content not by how good my SEO skills are, or my raw willingness and lack of social life to post to thousands of networks all day, rather...and this is important so I'll bold it...my content automatically goes to people who are interested in what I have to say.

This is game changing. I've always thought that the internet was the dawn of a collected conciousness, but now with vastly more intelligent search thanks to google and the like, everything we say can and will be said to the people who need to hear it.

It effectively turns everyone's brains into one giant brain, with computers as the connectors between brains.

For now, we have to post our thoughts in writing to things like blogger, but eventually I think thanks to voice recognition, our every thought, our musings, will bring us kindred spirits to develop those thoughts either into new thoughts, learning journeys, personal catharsis, and evolution.

Crowds have always been feared by people, as angry mobs with pitchforks, but in What Would Google Do, Jeff Jarvis talks about the wisdom of the crowd. If properly analyzed and understood, a crowd is vastly intelligent, it is the sum of all its parts. If you take an individual neuron, its not capable of much, but put them all together and you have the most complex thing in the known universe, the human brain.

For the first time in history, sites like Flickr and Google have begun to speak to the crowd in its language (or even give it a language for it to articulate itself) And its talking. It has a lot to say. Democratic ideals have always tried to make this a reality, but voting never really captured the true meaning of what the crowd wanted, only grunts and warbles. With AI and computing however, the crowd now can finally talk with unprecidented precision.

What will we say to the crowd? More specifically what will you say to the crowd? Here is where we get out of the clouds and return to earth with actionable items:

1. Make sure you have and control your personal brand on as many social networks as you can find. This means own your own domain www.myname.com if you can. Own your company name www.mycompany.com, myproduct.com etc.

2. Create a facebook fan page for your product or service. Get everyone you can to join. If there already is one, become an active and leading participant. Always Give Value in everything you do.

3. Create and own your facebook profile, before someone else does. Facebook is used by 150,000,000 people as of this post, and by default is quickly becoming the defacto identity source. If you dont own your facebook profile, you are wide open to someone else taking it for you. And that is still legal.

4. Create your profile on everything else. Ning.com, Linkedin.com, blogger.com, myspace.com. All these can be more or less empty but just grab and own them.

5. Now that you've experimented with all these things and learned all these new skills... :) Pick which one you like the most and dive deep. In linked in, get all your friends in there, find new contacts, find opportunities, recommend people you respect. In facebook, post funny and entertaining things to your status, make people smile daily. Tell people things they can use, share your ideas, your concerns, your hopes.

6. Watch the comments roll in, tweak what you are doing based on them. If one thing you do gets no fans or comments, but another thing does, do more of the other thing.

That is how you can, right now, capture the wisdom of the crowd.

Good Luck!

CLOUD COMPUTING, MEET CROWD COMPUTING

If you stay up past 10 on any given night, chances are you will eventually run across a TV infomercial promising riches beyond your wildest dreams. You'll be able to stay at home, it says, making thousands of dollars a day; in fact, you'll be able to buy a new home, like Susie did, just from your part-time, stay-at-home job.

Sure.It's really a shame, too, because the sheer workforce potential of stay-at-home moms, college kids and retired baby boomers is overwhelming.

But let's not throw the crazy fox out with the bath water. There are plenty of computer-centric, home-based jobs available that don't promise a new Ferrari and a live-in housekeeper. These more down-to-earth—and significantly less promoted—jobs also happen to be legitimate.

Take, for example, Amazon's Mechanical Turk. Based on the simple idea that computer's can't do everything (yet), Amazon is attempting to tap in to the highly lucrative market of idle human intelligence.

The premise is that some data manipulation requires human intervention. A good example would be a business looking to expand its email database. While it has thousands of records on customers it has collected over the years, no one ever thought to ask for email addresses. With Mechanical Turk, the process is quite trivial: the business creates a HIT, or Human Intelligence Task, that is uploaded with the data file to Mechanical Turks' servers along with the amount the business is willing to pay per email address found.

Magically, a global workforce springs to life, wrangling email addresses. For each HIT that is completed, a worker is paid a commission, typically in the 5-50 cent range, depending on the difficulty of finding the answer and the level of skill involved.

And, within a set time frame, Mechanical Turk sends the company new customer records, complete with email addresses. And, of course, a bill. Meanwhile, someone gets a check. The process is similar for tagging photos on online commerce sites to improve searchability; screening comments and reviews; and keeping Facebook and Twitter accounts appearing active.

It?s a little bit of money per job, but with a potentially huge crowd benefitting from, an completing, those jobs. With all the talk about cloud computing these days, it's nice to see a service tapping into the growing potential of the crowd.

For more information about HITs and Amazon's Mechanical Turk, visit their website at www.mturk.com .

IBM and SAP introduce… Crowd Computing?

For those of you who missed this announcement IBM Research and SAP Demonstrate New Cloud Technology: Real-Time Application Mobility

In this innovative merger of company values and large scale product offerings IBM and SAP are coming to market with a very clear Crowd computing solution.

In this technology demonstration, IBM and SAP show how users can run enterprise applications in the cloud, in particular demonstrating the migration of workloads across physical servers and across data centers. This demonstration is another instance of IBM working with partners across the IT industry to gain insights about creating and configuring workloads, and help companies move to the clouds as smoothly as possible.

This opens the door to a ‘if the big guys can do it, so can we’ methodology and mentality which means the Cloud will not only get to be more innovative faster, but it definitely looks like we can be looking forward to a pretty aggressive and crowded set of Cloud offerings in the Enterprise space.

In this demonstration, the migration of SAP workloads across the cloud is supported by IBM’s POWER6 systems, which enable users to run separate applications on different virtual machines, called logical partitions, on the same physical server. The IBM POWER6 system’s Live Partition Mobility capability further allows for the movement of a partition from one POWER6-based server to another POWER6-based server in the data center with no application downtime, resulting in better system utilization, improved application availability, and energy savings.

Application mobility, often one of the top of mind discussions around ‘Where does my app go” and “is it portable” is becoming more of a reality and less of a concern with the big as well as the small players.

 

Don’t think that the big blue getting involved will in anyway squelch the opportunity in this exploding market.  Infact quite the opposite.  IBM’s steps here today have likely increased the opportunity 100fold and that’s only touching on the surface of the offerings which so far are present in this otherwise very immature market.

…Are you ready for Crowd computing? Your enterprise will be

 

=========================================================

Is Crowd Computing the Future?

With Microsoft's announcement at PDC this fall and with the continued growth of Amazon's EC2 service and Google's AppEngine service, the industry seems to have people's heads up in the clouds. With this shift of focus, though, comes a myriad of questions about reliability, security, and portability. Potential customers of the cloud want to know that it can indeed be depended on. Executives want to know that the security of data in the cloud will not be compromised. Software engineers want to know that if a certain provider evaporates into thin air, minimal effort will be required to move deployed assets and keep mission critical apps moving.

With so many questions about elastic hosted services, and an as of yet unclear track record for the same, I cannot help but wonder if the cloud computing model will really take hold, or if it will just be a bridge to an even more impressive generation of computing architectures to follow. Maybe it will be both. This discussion then begs the question -- of what that generation will look like that does follow.

Nearly 10 years ago, a program was created that would compel sci-fi geeks, amateur astronomers, scientists, programmers, and scholars to change their screensaver. SETI@home launched in 1999 and over the next 9 years would bring grid computing into the living rooms and dorm rooms of over 5 million people. The original software was an app and screen saver that would use idle computer time to drive the search for extraterrestrial intelligence. It harnessed the untapped power of millions of computers with unrealized potential. It was built as an experiment, to break free of the constraints imposed by a supercomputer. Even hosted clusters have their limits, and some problems go beyond those limits.

With cloud computing the sky is the limit, but what if this world is not enough? What if a single company's data centers won't cut it? What if you want to maintain your data center, while still being able to tap additional resources on demand? What if you wanted to maximize and monetize under-utilized computational resources, instead of just writing them off as depreciating assets each year?

That seemed to be the aim of now defunct CPUShare. It offered users the opportunity to sell their idle CPU time to people who needed computational resources. What if the spirit of this project was matched with the vision of Windows Azure, or the ease of entry of Amazon's EC2. What if it added storage into the mix, RAM, and even bandwidth? What if each of these was currency in a new economy? This new economy would not be comprised of just one company's slice of the cloud; it would be the whole thing.

Crowd sourcing CPU hours might very well be the future, or it may be a pipe dream that will never be possible. It has the same questions of reliability, security, and portability, and it brings with it the question of control. The way the industry deals with the questions about cloud computing today, could very well pave the way for crowd computing to be the driving force behind Web 4.0 and beyond.

Better and Better: The Myth of Inevitable Progress | Foreign Affairs

Crowd computing

                                  Crowd computing is an overarching term which defines the plethora of human interaction tools that enable idea sharing, non-hierarchical decision making and the full utilization of the world’s mind space. Examples of these tools (many falling under the Web2.0 umbrella) include collaboration packages, information sharing software, such as Microsoft’s SharePoint, wikis, blogs, alerting systems, social networks, SMS, MMS, Twitter, Flicker, and even mashups. Business and society in general increasingly rely on the combined intelligence, knowledge, and life experiences of the “crowd” to improve processes, make decisions, identify solutions to complex problems and monitor changes in consumer taste.  An early example of crowd computing was the discovery of a gold deposit location at the Moribund Red Lake Mine in Northern Ontario. Using all available data, the company, Goldcorp, Inc. had been unable to identify the location of new deposits on their land. In desperation, the CEO put all relevant geological data on the web and created a contest, open to anyone in the world. An obscure firm in Australia used their software and algorithms to crack the puzzle. As a result, the company found an additional 8 million ounces of gold at the mine. The only cost was the nominal prize money awarded.

The Future Is Big Data in the Cloud

While when it comes to cloud computing, no one has entirely sorted out what’s hype and what isn’t, nor exactly how it will be used by the enterprise, what is becoming increasingly clear is that Big Data is the future of IT. To that end, tackling Big Data will determine the winners and losers in the next wave of cloud computing innovation.

Data is everywhere (be it from users, applications or machines) and as we get propelled into the “Exabyte Era” (PDF), is growing exponentially; no vertical or industry is being spared. The result is that IT organizations everywhere are being forced to grapple with storing, managing and extracting value from every piece of it -– as cheaply as possible. And so the race to cloud computing has begun.

This isn’t the first time IT architectures have been reinvented in order to remain competitive. The shift from mainframe to client-server was fueled by disruptive innovation in computing horsepower that enabled distributed microprocessing environments. The subsequent shift to web applications/web services during the last decade was enabled by the open networking of applications and services through the Internet buildout. While cloud computing will leverage these prior waves of technology –- computing and networking –- it will also embrace deep innovations in storage/data management to tackle Big Data.

A Big Data stack
But as with prior data center platform shifts, a new “stack” (like mainframe and OSI) will also need to emerge before cloud computing will be broadly embraced by the enterprise. Basic platform capabilities, such as security, access control, application management, virtualization, systems management, provisioning, availability, etc. will have to be standard before IT organizations are able to adopt the cloud completely. In particular, this new cloud framework needs the ability to process data in increasingly real-time and greater orders of magnitude -– and do it at a fraction of what it would typically cost -– by leveraging commodity servers for storage and computing. Maybe cloud computing is all about creating a new “Big Data stack.”

In many ways, this cloud stack has already been implemented, albeit in primitive form, at large-scale Internet data centers, which quickly encountered the scaling limitations of traditional SQL databases as the volume of data exploded. Instead, high-performance, scalable/distributed, object-orientated data stores are being developed internally and implemented at scale. At first, many solved this problem by sharding vast MySQL instances, in essence using them more as data stores than true relational databases (no complex table joins, etc.). As Internet data centers scaled, however, sharding MySQL obviously didn’t.

The rise of DNRDBMS
In response to this, large web properties have been building their own so-called “NoSQL” databases, also known as distributed, non-relational database systems (DNRDBMS). But while it can seem like a different version sprouts up every day, they can largely be categorized into two flavors: One, distributed key value stores, such as Dynamo (Amazon) and Voldemort (LinkedIn); and two, distributed column stores such as Big Table (Google), Cassandra (Facebook), HBase (Yahoo/Hadoop) and Hypertable (Zvents).

These projects are in various stages of deployment and adoption (it is early days, to be sure), but promise to deliver a “cloud-scale” data layer on which applications can be built quickly and elastically, all while having aspects of the reliability/availability of traditional databases. One facet that is common across these myriad of NoSQL databases is a data caching layer, essentially a high-performance, distributed memory caching system that can accelerate web applications by avoiding continual database hits. Memcached’s (disclosure: Accel is an investor in Northscale, parent company of Memcached) broad distribution (which is behind pretty much every Web 2.0 application) has become this de facto layer and is now accepted as a “standard” tier in data centers.

Managing non-transactional data has become even more daunting. From log files to clickstream data to web indexing, Internet data centers are collecting massive volumes of data that need to be processed cheaply in order to drive monetization value. One solution that was been deployed by some of the largest web properties (Yahoo, LinkedIn, Facebook, etc.) for massive parallel computation and distributed file systems in a cloud environment is Hadoop (disclosure: Accel is an investor in Cloudera, the company behind Hadoop). In many cases, Hadoop essentially provides an intelligent primary storage and compute layer for the NoSQL databases. Although the framework has roots in Internet data centers, Hadoop is quickly penetrating broader enterprise use cases, as the diverse set of participants at the recent Hadoop World NYC event made clear.

As this cloud stack hardens, new applications and services –- previously unthinkable -– will come to light, in all shapes and sizes. But the one thing they will all have in common is Big Data.

Ping Li is a partner with Accel.

Announcing support for PHP and other languages on Windows Azure

The Azure Services Platform team is delivering on its commitment to providing an interoperable, comprehensive and flexible cloud platform.

Windows Azure

At MIX09, the Windows Azure team is updating its CTP to include feature updates which will allow developers to take advantage of:

• FastCGI: allows developers to deploy and run web applications written with 3rd party programming languages such as PHP. This provides developers using non-Microsoft languages the ability to take advantage of scalability on Windows Azure. (Read more here: Using 3rd Party Programming Languages via FastCGI)
• .NET Full Trust: provides developers with a level of flexibility in Windows Azure that removes limitations on .NET Libraries which require full trust (including .NET Services) .NET Full Trust, via spawning process and p/invoke, also allows developers to utilize existing investments in native code or legacy components that they will now be able to invoke on Windows Azure. (Read more here: .NET Full Trust)
• Geolocation: provides developers with the ability to specify a location for their applications and data to build responsive services with lower network latency as well as the capability to meet location-based regulatory and legal requirements. This feature will be available a few weeks after MIX 2009. (Read more here: Geo Location Enables Developers To Choose Data Centers and Group Applications & Storage)

A new version of the developer SDK and Tools for Visual Studio will be available for download to enable developers to take advantage of the new features. The SDK update will include:

• Managed Full Trust support (including Native Code support via P/Invoke and spawning native code processes)
• Support for FastCGI applications.
• Support for rewrite rules via the URL Rewrite Module. Creates URLs so developers can lead users to shorter, search engine friendly, and easier to remember URLs.
• Support for SQL Server as the data store for Development Storage – move from SQL Express to full SQL Server for backend developer store.

In addition to supporting the latest Windows Azure SDK, the Tools for Visual Studio will offer:

• Native debugging of roles called via PInvoke running on the Development Fabric
• FastCGI starter template
• Chained install of both the Tools and SDK (one install)
• Update notification for newer releases

To summarize, this is what Windows Azure entails as of today:

Computation Services

• Ability to run Microsoft ASP.NET Web applications or .NET code in the cloud
• Service hosting environment that includes Internet Information Services (IIS) 7.0 and Microsoft .NET Framework 3.5 SP1
• Security supported by flexible Code Access Security policies
• Small runtime API that supports logging and local scratch storage
• Web portal that helps you deploy, scale, and upgrade your services quickly and easily
• FastCGI, a protocol for interfacing applications to web servers, which will allow customers to deploy and run web applications written with non-Microsoft programming languages such as PHP (Developers will be responsible for including the relevant runtime libraries for these languages when deploying applications.)
• .NET Full Trust to allow usage of additional .NET features such as Windows Communication Foundation (WCF).
• From Full Trust .NET, developers can call into unmanaged DLLs

Simple data storage services

• Blobs, tables, and queues hosted in the cloud, close to your computation
• Authenticated access and triple replication to help keep your data safe
• Easy access to data with simple REST interfaces, available remotely and from the data center

Development Tools

• Complete offline development environment, including computation and storage services
• Complete command-line SDK tools and samples
• Visual Studio add-in that enables local debugging
• New SDK Download: a new version of the Windows Azure SDK will be available for download at a time to coincide with the MIX09 conference, which will enable developers to take advantage of the new features offered by Windows Azure, as well as an update to the Visual Studio add-ins.

The SDK and the Tools for VS addin is now available for download at http://www.microsoft.com/azure/sdk.mspx.

SQL Data Services

You may recall seeing this when Azure was first announced:

The SQL Data Services (SDS) team (the 3rd block above Windows Azure in the image above) publicly shared the evolving capabilities in SDS to provide customers with the ability to utilize a RDBMS data model in a cloud-based environment supporting Transact-SQL (T-SQL) over TDS (Tabular Data Stream) protocol (read more here: What’s Next for SQL Data Services…) SQL Data Services is on track to deliver a public CTP mid-calendar year 2009 and be commercially available in the second half of calendar year 2009. At MIX, the SDS team has announced the evolution of SDS capabilities to provide traditional relational database service with T-SQL compatibility over protocols that support data access APIs such as ADO.NET, ODBC and OLE DB.

Can Cloud Computing Ever Truly Be Sustainable?

Data centers aren't exactly known for their sustainability--the power hogs are responsible for 1.5% of all power use in the United States. But as cloud computing, the IT golden child that uses mega-data centers to store information, becomes more popular, so do data centers. Without energy efficiency measures, data center consumption will total $7.4 billion annually by 2011 (compared to $4.5 billion today). So what can be done to save cloud computing without bleeding cash and energy?

Companies like IBM, Google, and HP already have made strides in cutting data center energy use. IBM, for example, recently broke ground on a Syracuse,New York-based data center that will use 50% less energy than traditional facilities. The $12.4 million data center contains an on-site electrical co-generation system with gas-fueled microturbine engines as well as water-cooled server racks. Sensors direct workloads to optimal servers. Similarly, Google recently revealed that its data centers use half as much energy as standard data centers thanks to cooling towers that evaporate excess heat and recycle cooled-down water back into the facility

At HP, cloud computing is part of the company's vision of "everything as a service." That means the company is relying on the cloud for everything from on-demand book printing to online picture and video storage. It also means that the company is invested in keeping data center costs down. To that end, HP sells its HP POD, a veritable data center in a box that is 50% more energy-efficient that most data center buildouts.

And soon enough, we'll even have an army of dedicated green data center managers. Metropolitan Community College in Omaha, Nebraska recently announced the country's first degree in green data center management. At the end of the program, students are given the chance to work in the on-campus Information Technology Data Center.

Cloud computing in and of itself isn't particularly sustainable--no one has figured out how to maintain a net-zero energy data center--but it won't go away any time soon either. The best we can hope for is to minimize its impact with energy efficiency measures. Fortunately, it's in the best interest of anyone running a data center to cut down on power costs.

Safety in the Cloud(s): 'Vaporizing' the Web Application Firewall to Secure Cloud Computing

Cloud computing was not designed for security, although organizations such as Cloud Security Alliance (CSA) and Open Web Application Security Project (OWASP) are making great strides in helping the industry solve the myriad security problems confronting cloud computing. The benchmark guidelines established by the CSA in the document, Guidance for Critical Areas of Focus in Cloud Computing, are a great first step. This article is intended to pick up where the CSA guide left off in terms of defining what a distributed Web application firewall (dWAF) should look like in order to meet the standards set within the CSA document.

In order to accurately outline how a dWAF is possible while maintaining all the benefits of a completely virtualized environment – reduced IT overhead, flexible footprint management, virtually unlimited scalability – a brief overview of cloud technology is needed. Far more than simply maximizing current hardware resources to benefit from unused CPU power, today there are three main technologies available in a cloud that provide the backbone for real productivity gains and compelling business services for companies that don’t want to invest in the hardware scaling burdens common today.

Software as a service (SaaS) offers users virtualized software through a thin client, usually any standard Web browser. The benefit for users is access to software without any of the headaches of owning the programs – scaling and resources are taking care of, and patching and upgrades are managed.

Platform as a service (PaaS) provides users with virtual databases, storage and programming languages with which custom applications can be built. This service provides nearly unlimited resources behind the platform and allows customers to scale throughout the lifetime of the application. It is an effective solution for companies ranging from the very small to those serving millions of customers. The customer does not worry about the infrastructure needed to run the services and is billed in per usage model.

Infrastructure as a service (IaaS) allows users access to virtually unlimited resources to build and manage their own virtual network. Customers can commission and decommission virtual resources depending on their need. The most obvious benefit is that there is no end-of-life for hardware anymore for the customers. The providers move them according to their service level from hardware to hardware without any downtime.

The common user benefit of services available through a cloud is access to key resources via the Internet, which provides an incredible degree of scaling without the need to invest in expensive hardware infrastructure.

Cloud Applications Are Highly Exposed to Threats

Accessing cloud technologies requires a thin client, and the world’s most commonly used thin client for this purpose is a Web browser. This means the vast majority of all applications on the Internet have some kind of Web and/or application server on which the business logic is implemented. Currently, most of the money spent on security goes into firewalls and antivirus solutions, but in the last 10 years the typical target for attacks has shifted from the network layer to the application layer because the operating systems and services available to the general public were cut down. As a result, it is now easier to target the application logic or framework of an application than the actual server behind the hardened network perimeter. Applications are mostly developed by the businesses themselves and not every developer considers security the highest priority, which leads to a wide variety of problems.

The IBM X-Force® 2008 Annual Report highlights that Web application vulnerabilities are the Achilles’ heel for corporate IT security. The impact of not being able to secure these vulnerabilities is far reaching.

 

 

 

 

Further, attack vectors increase exponentially in correlation with the mainstream adoption of cloud computing. Their increase is dictated by hosting and delivering infrastructure, platform and software. Establishing a comprehensive patch management system is the common solution offered by most in the industry, however, in practice this approach has proved very difficult and costly. Typical Web applications are built on open source components, by third parties, who rely on Web frameworks. This approach has the obvious benefits of interoperability and shortened development time, however, patching becomes exponentially more difficult. A flaw in one piece of open source code must be patched for each instance it is used throughout each application in which it is used. In a cloud setting, this becomes a very large issue.

 Applications developed specifically for a cloud are often very complex, designed for access speed, scalability and flexibility for third-party development through an open API. For example, Salesforce.com, Google Docs, MySpace, Facebook and Twitter, are all prime examples. These ‘as a Service’ applications are developed two ways today: by moving on-premise applications to a cloud, and by developing and operating applications directly in a cloud.

Applications that are forced out of the internal company network and into a cloud carry the risks of exposing protected software to Web threats it was not designed to combat. Common security threats include injection attacks, cross site scripting or cross site request forgery.

There are a variety of services available for developing in a cloud, such as MS Azure Services, Google App Engine or Amazon EC2. There are many security challenges involved in developing Web applications in a cloud. For example, parameter validation, session management and access control are 'hotspots' for attackers. Developers not trained in those three fields of application development will most definitely create/develop applications that have security problems.

 

 

Why a Traditional Web Application Firewall Will Not Work

 

In a cloud, the infrastructure and the services are shared between customers, meaning one set of hardware is used by many business, organizations and even individuals. Each of these cloud operator customers adds a unique layer of policy settings, use cases and administrative enforcement requirements. For the cloud or service provider, security quickly becomes very complex. The average provider may have 10,000 customers subscribing to its service, each with varied policy settings for individual divisions within the company. The service provider now has to manage an nth degree of application filter settings.

Currently, Web application firewalls (WAF) and other security solutions are restricted to hardware appliances, which creates a serious bottleneck for cloud service providers. Dedicated hardware boxes simply don't allow for reasonably scalable levels of multiple administrators’ duties within a box’s singular security policy mechanism. Ironically, in addition to the traditional network hardware, cloud service providers are forced to have a rack full of dedicated WAF machines – one per customer – that take up space and eat up resources. Security becomes counter to the efficiency promises of a fully virtualized environment. This cost is passed on to customers, increasing adoption barriers to mainstream cloud computing.

In an ideal world, applications would be designed from the ground up to meet the rigors of a virtualized world, integrating security measures directly into the applications and thus solving a core problem with current cloud computing. Until the industry reaches this ideal), traditional Web application firewall boxes are preventing the industry from reaching the full potential of a cloud computing.

Defining the Distributed Web Application Firewall (dWAF) for Cloud Protection

Web application security in a cloud has to be scalable, flexible, virtual and easy to manage.

A WAF must escape hardware limitations and be able to dynamically scale across CPU, computer, server rack and data center boundaries, customized to the demands of individual customers. Resource consumption of this new distributed WAF must be minimal and remain tied to detection/prevention use instances rather than consuming increasingly high levels of CPU resources. Clouds come in all sizes and shapes, so WAFs must as well.

The dWAF must be able to live in a wide variety of components to be effective without adding undue complexity for cloud service providers. Today’s providers are using a variety of traditional and virtual technologies to operate their clouds, so the ideal dWAF should accommodate this mixed environment and be available as a virtual software appliance, a plug-in, SaaS or be able to integrate with existing hardware. Flexibility with minimal disruption to the existing network is central.

A Web-based user interface must allow customers to easily administrate their applications. Configuration should be based on the applications under protection, not defined by a singular host, allowing far more granular settings for each application. Ruleset configuration must be supported by setup wizards. Statistics, logging and reporting has to be intuitive and easy to use and must also integrate seamlessly into other systems. Most importantly for a dWAF, multi-administrator privileges must be made available and flexible enough to effectively manage widely divergent policy enforcement schemes. Cloud providers should look for a set of core protections.

Detection and Protection

Foundational security using black, white and grey listings for application requests and responses must be possible. To make sure pre-set policy enforcements are not activated or deactivated without approval from an administrator, deployment and policy refinement through establishing rulesets must be possible in a shadow monitoring or detection-only mode. Once the shadow monitoring ruleset is stable, only then should it be allowed to deploy in an enforcement mode on the dWAF. This allows complete transparency for the administrator into the real-world effect of this ruleset, while at the same time allowing layered rulesets to be tested without compromising existing policy enforcement. Avoiding false positives and relaxed established defenses is essential for a real-world, usable dWAF in a cloud.

Automated learning and ruleset suggestions based on intelligent algorithms or recommendations from a static source code analyzer or Web vulnerability scanner are also desirable from a manageability view. Again, this only holds true if the administrator retains full control over activation/deactivation of each ruleset. Without this control, wanted traffic may become blocked and policy settings would become compromised.

Application Shielding

Pro-active security functions are highly recommended to reinforce any application in a cloud. Detection is simply not enough for today’s Web application security. Features like transparent secure session management, URL encryption and form-field virtualization will provide strong deterrence to attack, while saving application development and deployment time. These features are effective because session management, URL encryption and form-field virtualization is done at the dWAF level and not in the application itself.

An authentication framework support that enables businesses to consolidate their applications under one management schema is also desirable for a dWAF. This enables users to handle the authentication in front of their applications rather than behind, which adds another perimeter of security. A consolidation of all applications with dedicated rights-management ability is also a strong usability function that will make an administrator’s life easier.

Integration with Existing Technology

Avoiding vendor lock-in is a common best practice for both networking and application security. Any technology that is added to an infrastructure, platform or application itself must connect as seamlessly as possible with existing technology. Security is all about layering technologies to create the best possible protection, so a dWAF must communicate freely between a security incident and the event management system (SIEMs).

The Cloud Hanging Over Skype

Early this week, eBay announced that after four years of owning Skype, the popular, and free, online phone service, it had sold the company to an investor group for around $2 billion. The investors included the Silicon Valley private equity firm Silver Lake Partners; Marc Andreessen’s new venture capital firm, Andreessen Horowitz; a London firm called Index Ventures; and the Canada Pension Plan Investment Board. Under the terms of the deal, eBay will retain a 35 percent stake in Skype, giving it a valuation of $2.75 billion.

Many people on Wall Street — and a number of telecommunications experts I spoke to this week — were stunned by the price Skype sold for, and not just because we’re in the middle of a recession. In 2005, when eBay bought Skype from its founders, Janus Friis and Niklas Zennstrom, it paid $3.1 billion. But the company had performed so poorly that by the fall of 2007, eBay had been forced to take a $1.1 billion write-down.

Around that same time, Mr. Zennstrom, whose relationship with eBay management had turned acrimonious, stepped down as Skype’s chief executive. (Mr. Friis had already left the company.) Although Skype’s performance has improved since the installation of a new chief executive last year, it was no secret that eBay was trying to unload it. Many potential buyers had walked away, believing that eBay simply wanted too much.

There is another reason that the Skype deal has raised eyebrows, however. Not long after Mr. Friis and Mr. Zennstrom left the company, they became embroiled in a dispute with eBay that has turned into a very nasty lawsuit.

It turns out that in selling Skype to eBay, Mr. Friis and Mr. Zennstrom retained control of a key part of the Skype technology, which they licensed to eBay. Although the details are under seal in a London court, the Skype founders’ essential complaint is that eBay tampered with their software, and in doing so, violated the terms of the licensing agreement. They were demanding that Skype be forced to stop using the technology, which, for all intents and purposes, would mean shutting down Skype itself. The case is set for trial in 2010.

Companies are sued all the time, of course. But this lawsuit feels different; to put it bluntly, it feels more dangerous than the typical lawsuit aimed at a corporation. In a court hearing in London last June, eBay’s lawyer told the court that if Mr. Friis and Mr. Zennstrom won the case, the result would be “devastating.”

In its financial documents, eBay says that it is “confident” of its legal position. But it also acknowledges that an “adverse result” could mean that the “continued operation of Skype’s business as currently conducted would likely not be possible.” That is hardly your typical corporate boilerplate. Indeed, after that court hearing in June, a telecom analyst named Jayanth Angl told Bloomberg that “if eBay can’t reach an agreement over that piece of technology, that could certainly turn the Skype acquisition into a debacle.”

And so, the mystery of the Skype deal: why were the winning bidders willing to pay so a high price for a company whose very existence could be threatened by this lawsuit? One possibility is that they have nerves of steel. The other is that they know something nobody else does.

•

Skype was not Mr. Friis’s and Mr. Zennstrom’s first company. No, that was the infamous Kazaa, a peer-to-peer company that the two men founded in 1999, not long after Napster showed the world exactly how easy it was to steal copyrighted music using peer-to-peer computing. By 2001, the recording industry, having routed Napster, turned its sights on Kazaa.

Going after Kazaa was tougher because it was located somewhere in Northern Europe, outside the purview of United States law enforcement. (No one knew exactly where.) The Kazaa founders moved periodically to keep the recording industry from being able to subpoena them, and for years, they stayed away from the United States for the same reason. But the recording industry kept up the pressure, and as their legal costs mounted, Mr. Friis and Mr. Zennstrom finally decided to get rid of the company and move on.

Former Skype executives will tell you that the Kazaa experience did a lot to shape Mr. Friis’s and Mr. Zennstrom’s approach to business. It made them extremely secretive. They almost never talk to the press. (They didn’t speak to me for this column.) And it also made them extremely protective of the technology they created. Which is why, long before they sold Kazaa, they moved their peer-to-peer software into a new company, called JoltID.

In 2003, when they started Skype, that same technology that had powered Kazaa became an important part of the Skype code; it was the means by which computer users connected to each other and created a larger network. (VoIP — voice over Internet protocol — was the means by which they spoke to each other online.) But Skype never owned the technology; JoltID did.

Why eBay was willing to go along with such an arrangement when it bought Skype two years later will forever be a puzzle. But so long as the two men remained part of the eBay “family,” it didn’t matter much. Any changes to the peer-to-peer code were ones they approved.

When the deal went sour, however, and the founders left eBay, that all changed. And when eBay continued to tinker with the code — something eBay contends it has a right to do under the license — they entered into negotiations that went nowhere. Finally, by March of 2009, the two sides had sued each other.

At the same time, the founders, together with some big private equity firms, including Elevation Partners in Silicon Valley (yes, the Bono firm), and General Atlantic in New York, were trying to buy back Skype. It was, after all, their one big success. (Their third start-up, Joost.com, has gone nowhere.)

It is hard to know precisely what happened next. EBay claims that all the bidders were treated the same, and that the losers simply didn’t put up as much money as the winner. But according to supporters of the Skype founders, their investing consortium made three serious efforts, over the course of a year, to bid for the company. Every time, they say, they were stiff-armed by eBay’s investment bankers. About a month ago, they wrote a letter to eBay protesting their inability to get a hearing for their proposals.

And maybe the Skype founders did try to buy back the company on the cheap. The sense I got, however, is that the founders would have been willing to come up with a price that suited eBay — if they had been able to enter into negotiations. What is clear is that the bad blood that had developed between eBay and the founders was infecting the potential negotiations over a buyback of the company. (EBay denies this.)

And then, a few months ago, out of the blue, came the $2 billion bid from the Silver Lake consortium. One way it has dealt with the litigation risk is by persuading eBay to assume 50 percent of any losses resulting from the lawsuit. But that still doesn’t mitigate against the possibility that the founders could win the lawsuit — and put their creation, Skype, out of business.

So why were they willing to bid so high? One theory is that the Silver Lake people think they can win in court. Indeed, if by next summer the two sides are still arguing in court, we’ll know that is the answer to the mystery. That is the “nerves of steel” theory.

But how likely is that? In this environment, big-time private equity firms don’t commit $2 billion if there is a serious possibility the company they’ve just bought might be put out of business. As it happens, not long before Index Ventures became interested in Skype, it brought on board a man named Michelangelo A. Volpi, a highly respected former Cisco executive who — hmmm — once sat on the Skype board. In fact, he was so well liked by the Skype founders that they hired him to run Joost. Wouldn’t you know it? Joost uses the same peer-to-peer technology as Skype and Kazaa.

Mr. Volpi told me that not long after he arrived at Index Ventures, he discussed the possibility of making a run at Skype — and he and another Index Ventures partner, Danny Rimer, in turn rounded up Silver Lake and Mr. Andreessen, who — hmmm — sits on the eBay board. (As soon as he got involved with the bid, Mr. Andreessen recused himself from any board discussion about the Skype sale.) In the end, Mr. Andreessen committed $50 million to the deal — a very large percentage of his $300 million venture fund.

So another theory: because of his friendship with the Skype founders, Mr. Volpi believes he’ll be able to settle the lawsuit. Rich Tehrani, the president of TMC, a telecom publishing company, told me that he had just come from a conference where rumors were rife that the Silver Lake consortium had already cut a side deal with the Skype founders. (All the parties deny this.)

The third possibility is that Mr. Andreessen and the others have figured out a technology “workaround” so they no longer have to rely on the JoltID technology, something eBay had already begun working on. But almost everyone I spoke to said such a workaround would be, at best, difficult and expensive — and could cause such severe disruption to Skype’s business that it might never recover.

It is, alas, unsatisfying to delve into a mystery like this and not be able to solve it. But over time, it will become clear. Either the case will linger, and we’ll know that Silver Lake, Andreessen et al. do indeed have nerves of steel.

Or it will quickly go away, which will provide an answer of a less seemly sort. The mission of Skype, after all, is to shrink the world and bring people together.