An emerging trend in IT (or perhaps fad) that seems to have one of the following definitions. * An approach to IT provisioning and management that takes advantage of techniques and an approaches to building large-scale web applications over the past 10 years (e.g. Google, Amazon, etc.). * PublicClouds: Information and/or computing services available via the Internet at various levels of abstraction, from Infrastructure like RelationalDatabases to VirtualizedHardware like AmazonEc2, or a BusinessApplication like SalesforceDotCom. * PrivateClouds: Managed information and/or computing services available inside an organization, at various desirable levels of abstraction, with some level of automated provisioning and configuration management. (e.g. SLAs for Applicaitons, BusinessInformationServices, etc.) * The ability to integrate between public & private clouds as appropriate. * A recognition that the WebArchitecture is the common denominator for information exchange (not WebServices, not CommonObjectRequestBrokerArchitecture, not SimpleNetworkManagementProtocol, etc.) A list of CloudComputing Providers: http://www.johnmwillis.com/cloud-computing/cloud-vendors-a-to-z-revised/ Google Group: http://groups.google.com/group/cloud-computing CloudComputing is DistributedComputing, with a focus on external compute-service providers (?) ---- '''Major CloudComputing platforms/vendors:''' * AmazonWebServices: ** http://aws.amazon.com/what-is-aws/ ** http://itmanagement.earthweb.com/features/article.php/12297_3798591_4/Five-Companies-Shaping-Cloud-Computing-Who-Wins.htm * See also: AmazonElasticComputeCloud (EC2), AmazonSimpleStorageService (S3) & AmazonSimpleDb * GoogleAppEngine: ** http://code.google.com/appengine/ in the PythonLanguage ** http://itmanagement.earthweb.com/features/article.php/12297_3798591_5/Five-Companies-Shaping-Cloud-Computing-Who-Wins.htm * InternationalBusinessMachines: ** http://www-03.ibm.com/press/us/en/pressrelease/22613.wss * ''[Press releases but not product pages or web sites. Doesn't sound very promising ... except maybe for internal work by their legendary sales force.]'' ** http://itmanagement.earthweb.com/features/article.php/12297_3798591_6/Five-Companies-Shaping-Cloud-Computing-Who-Wins.htm * MicrosoftAzure: ** http://www.microsoft.com/azure/ ** http://blogs.zdnet.com/microsoft/?p=1671 ** http://itmanagement.earthweb.com/features/article.php/12297_3798591_2/Five-Companies-Shaping-Cloud-Computing-Who-Wins.htm * Microsoft Office 365: ** http://www.microsoft.com/singapore/cloud/Productivity/Office365/Default.aspx * SalesForce application development platform: ** "Force.com" PlatformAsaService: http://www.salesforce.com/platform/ ** http://itmanagement.earthweb.com/features/article.php/12297_3798591_3/Five-Companies-Shaping-Cloud-Computing-Who-Wins.htm * ''(Chart from http://itmanagement.earthweb.com/features/article.php/3798591/Five-Companies-Shaping-Cloud-Computing-Who-Wins.htm article.)'' http://itmanagement.earthweb.com/img/2009/01/cloud-computing-chart.jpg ---- '''Blogs:''' * AllThingsDistributedBlog: http://www.allthingsdistributed.com/ ---- ---- '''Discussion:''' (My initial comments are derived from a post I recently made to the CloudComputing Google Groups list) Arguably the first reason CloudComputing seems to be a trend with legs is the release of NicholasCarr 's recent book, TheBigSwitch. There's a trend afoot to look at IT as something that will become a lot like utilities (e.g. water, electricity, etc.). So, why not call it UtilityComputing? Well, that term has a fair amount of baggage. It seems to have been used to describe everything from plain old virtualization, to "IT as a Service Provider" within an organization, to blade array provisioning, etc. It often seems to "stop" at the OS level and doesn't include higher level elements, like applications, frameworks, etc. that are also clearly important to some audiences. GridComputing is another contender for the concept, but tends to be associated with either a marketecture for easy-to-manage clusters (i.e. OracleDatabase 10g / 11g). Alternatively, it brings to mind the world of BatchProcessing, ScientificComputation, the OpenGridSoftwareAlliance, GlobusToolkit, etc. - a siloed world that doesn't seem at all familiar to those hosting large databases or web farms today. The reason for a new buzzword seems to all come down to a simple question: * Why are online services so easy to use and cheap to access, and my internal IT services so hard to use and expensive to maintain? The traditional enterprisey arguments are falling like pins: * First, it was "scalability", but Amazon and Google are now bigger than most IT shops. ** What is important is not how much resources Amazon and Google have, the important is how much of that they give to you. * Then it was "security", but as far as I can tell, Amazon's information assurance is top notch. ** Security is not only about being "safe from crackers". How about some change in the law of the country hosting cloud. I see a lot of "CloudEmbargos" in the future. ** ''Therefore, to address the "CloudEmbargos" issue, for CloudComputing services to be really successful they'll have to be multi-national.'' *** ''They had to be not only multi-national, but mirrored-multinational (That is, for country X to feel save storing its information in CloudServiceProvider Y, Y has to have independent installations mirroring data for X in to countries that are not allies... or perhaps a protocol for mirroring between different CloudServiceProvider in different countries will needs to be developed, so that competition encourages them to keep their copy of customers data safe (if they don't, the customer should be able to easily stop using any of the providers in an instant) ... or the easy solution... every government must use its own Cloud, particulars on the other hand can choose between the national Clouds or the foreign ones, and if they are wise they will choose those that support some kind of multi-nacional mirroring'' * Then it was "integration", but most IT departments spend reams of money to do this poorly * Then it was "reliability", which is ever a problem but the largest sites seems to do quite well ** Yes, but they have it as kind of local resource... the question where is will you get access to you info it the moment you need it now that it is in the cloud?, for example, in the middle of natural disaster that disrupts long distance communications? * Then it was "manageability", but as most of us know, this is a huge cluster-mess in most IT shops; InformationTechnologyInfrastructureLibrary and the like are more dreams of paradise to keep ops people sane than a practical reality. This is not to say that there's nothing to learn from traditional Enterprise IT. But, having said that, we now have, for the first time since the 70's, an environment (the consumer web) that's evolved under a different culture & mindset than business systems but has required '''the same or greater''' levels of "-ilities" in the largest sites. One could say it's analogous to the shift from military computing to mainstream business computing in the 60's. Cloud computing seems to be an answer that there's a better way to organize this stuff, and some technical "special sauce" that we haven't been using thus far (for a variety of reasons, mostly due to FUD (FearUncertaintyDoubt), inertia or politics). Time will tell if as to how real this is -- in a way it's tackling the same ball of mud that ServiceOrientedArchitecture did, just from a different angle and a much more tangible level of granularity and value. In a nutshell, for an IT audience, cloud computing is a rethink of how IT Management & Provisioning should work given what we've learned over the past 10+ years growing the Web. Anyway, that's my first stab at this discussion. Perhaps it's a load of crock and we can all go home. I think there may be something here, though... But, note, my track record isn't great (I did start the WebServices page eight years ago, and look at the mess that's got us into ;-) --StuartCharlton ---- ''The interesting thing about cloud computing is that we've redefined cloud computing to include everything that we already do. I can't think of anything that isn't cloud computing with all of these announcements. The computer industry is the only industry that is more fashion-driven than women's fashion. Maybe I'm an idiot, but I have no idea what anyone is talking about. What is it? It's complete gibberish. It's insane. When is this idiocy going to stop?''. LarryEllison speaking at OracleOpenWorld I wonder... will there be a CloudBubble? Right now a lot of people think the Cloud is the silver bullet that will solve all IT related problems... in 10 years... will people remember this as the nice time before the disaster? Wars in the past have been about access to natural resources (Oil for example) I wonder... will we have a war related to access to computing resources in the future? CloudEmbargos certainly could lead to that... ---- I really like the idea that one can "grab some (virtual) hardware" and get something up and running (or add resources to an existing CloudComputing system) very quickly and easily. Horror pre-CloudComputing Story: I joined a project (quite some years ago) to develop server application software (in C++ on Unix) for client computers (running VB) using a custom middleware layer. It was a RapidApplicationDevelopment environment with an ambitious schedule and an intention to produce and demonstrate prototypes quickly. But we quickly discovered that the corporate I.T. infrastructure group had a '''''six month''''' back log for servicing requests for new development servers. ''(And, of course, '''for some reason''', they had a corporate policy that development projects couldn't share development Unix servers.)'' The only thing that "saved" us was that Windows was sufficiently Posix-compliant that with a few fixes I was able to get the Unix server software running on my Windows development box. I see a promising future for CloudComputing to provide development projects quick and easy access to server infrastructure resources that they need: Just charge it to a credit card and expense it to the project. This could be a great way to fire up a ContinuousIntegration build server, and some test servers, and maybe even large numbers of client test scripts for volume testing. Should this happen, the human tendency for inertia could work in CloudComputing's favor. -- JeffGrigg ---- The reason for a new buzzword seems to all come down to a simple question: * Why are online services so easy to use and cheap to access, and my internal IT services so hard to use and expensive to maintain? ''Maybe because we are not looking close enough(like those guys that say MacOsX is more secure just because less people is attacking it)? I mean, if my free email account stops working for 1 week I do not go crazy, and if my favorite search engine stops for 1 week I can easily switch to my second option... but now imagine that your payroll system is running in the cloud, and you, by any reason loose contact with it for 1 week... and you need to pay this month tomorrow... or imagine you are a multinational fast food chain and your point of sale system is running in the cloud... and it fails you for 1 week, all you operations stoped for 7 days... (Yes you may argue that has never happened, but the thing is if it had or if it does, as things are right now you do not really care, but when all your critical information is up there,this kind of things will start worrying you). And remember, failing is not only being unable to connect, failing could be inaccuracy... if I still can search in my favorite search engine, and it has a bug that prevents me from finding some very funny page, it is not a big deal, if you cloud based payroll overpays or underpays your employees 15%, you are in for really big trouble. Imagine, for example a problem like this: http://en.wikinews.org/wiki/Google_bug_brands_every_page_as_harmful but linked to something mission critical in you enterprise... now imagine 25% of the enterprises of a country affected by the same bug because they all run their systems on the same CloudProvider... great start for an economical disaster don't you agree?'' [Are cloud-based services more or less likely to fail than internal IT infrastructure? As I'm writing this, almost all of my workplace's IT infrastructure is unavailable due to an air-conditioning failure in one of the server rooms. At the very least, the (presumably) distributed nature of cloud-based services would (presumably) prevent that category of failures. As for accuracy, I don't see why an external payroll service would be any more likely to be inaccurate than an internal one. Having worked as a Canadian payroll consultant in a past life, I know both are possible, and in my experience, equally likely given equal expertise.] ''Yes, but when the private payroll system of your enterprise fails, only your enterprise is in trouble, when the cloud based payroll system fails, maybe 25% or the enterprises in the world could be in trouble, the impact of failure is much stronger with centralized resources.'' Certainly, I know that '''I'm''' concerned about reliability issues: Not only could my vendor's cloud have problems that are out of my control, but long-distance high-bandwidth communication becomes a much more critical business issue than it is now: Today, when we lose Internet access, people are annoyed that they can't get to information they need to be more highly productive. But when Internet access to any of one's CloudComputing partners fails, it could break a company's ability to take orders, manage inventory, ship orders, service customers, etc. Internet access becomes much more business critical. Now maybe this is inevitable. And maybe it's a good thing. But I think I'd like to see more planning and investment to ensure that things work well, and less learning from experience after they fail. ;-> -- JeffGrigg ''But, are cloud-based services really distributed? are we not aiming for centralization by sending all our data to CloudProviders? right now we are truly distributed, the internal network of the company where I work works even if there is a failure in the InternetServiceProvider and nobody can access InterNet. Sure it fails more often than the network of most CloudProviders, but failures of internal networks are distributed somewhat randomly between different companies (I imagine it like the lights of a Christmas tree , some of them go out, but never at the same time), but if CloudComputing becomes really popular without good multi provider mirroring technology, we could witness a CloudBlackout that could collapse the economy of the affected enteprises/countries.'' ---- Wow, all this reminds me of what MulticsOs was originally invented for. My, oh my, how history repeats itself. ''Yes, right now, the cloud is great, everybody wants to be on it, it is is the silver bullet, everybody wants to have their systems built with HTML and JavaScript and hosted by a big CloudProvider far far away... in... 5 years? we might see the mechanism come full circle and the market very exited with the PC ... but PC will not stand for PersonalComputer, but to PersonalCloud (or perhaps something more innovative like PRD (PersonalRainDrop)) ;-): The greatest idea ever, "networks and computers with software that works even if not connected to other networks! revolutionary! Each PersonalRainDrop is so powerful that can do the job a stadium filled with a traditional cloud could do in the past... where have I read that before?..."'' ------- '''Versioning''' could be a major hindrance to this. One group of people may have their app tuned to version 2.1 of product X, another version 2.2, another version 3.1, etc. You cannot just force the 2.1 people to convert to to the latest. ISP's get a lot of flack when they force people to upgrade because they don't want to support say old versions of PHP anymore. If the cloud manager has to maintain 20 version of product X, then the advantage of clouding is greatly diminished. (100% backward compatibility is fool's gold.) A similar issue applies to custom configurations. These "services" are too complex to be considered a commodity, and '''hardware is relatively cheap''' compared to the effort of constant migrating and having to live with generic defaults. Centralized hosting of custom-tuned servers may be a better sell at this stage. In other words, "here's my server, you watch it for me." If the effort of forced migrations comes to 2 week's of labor a year, and we cost labor at $60 an hour (wages plus overhead), then that 2 weeks costs about 5 grand (USD), the cost of medium server. --top ''Versioning is likely to be a negligible issue, since current research and development efforts are well aware of the problem. Popular cloud computing platforms (if such things come to exist) will deploy updated components invisibly and automatically. It won't eliminate versioning problems, obviously -- there will inevitably be ancient nodes on the network that lack the capability to run updated components -- but it will, hopefully, eliminate the need to manually deal with versioning issues.'' * "invisibly and automatically" almost strikes me as an AlarmBellPhrase. Anyhow, I was thinking more about custom or domain-specific software that depends on certain versions more so than the effort of installing vendor upgrades. Upgrades often break existing software, and some vendors don't seem to care much. For example, suppose you have a domain-specific piece of software X from a dead company, and X depends on Microsoft Foo 4.0 as a component/service. Well after MS releases version 8.0 of Foo, the cloud hoster decides to stop supporting Foo 4.0 and announces they will be removing it to avoid version conflicts. Thus, you can no longer use X, at least not on the cloud. However, I can see the concept working for commodity software and services, such as email, word-processing, etc. However, the commodity stuff can also be packaged to be easier to install on local data centers. Thus, it's kind of a catch-22. Server managers who understand what special software is used for and why seem better able to baby-sit it. A big generic data center operator won't be able to provide that. {Versioning is simplified greatly by moving the "app" to the cloud. For example, Google can, indeed, force you to upgrade to version 2.2 of their google-mail web applications, and do so even more easily for dependencies such as spam detector libraries and such. But there will always be some BackwardsCompatibility concerns for protocols and languages. There are no 100% satisfactory solutions there, but one can design both languages and protocols with an eye towards forward compatibility - a bit of BigDesignUpFront where YouAreGonnaNeedIt has been proven by long history.} {As far as the "centralized hosting" mentioned by Top: I agree that management at least must be centralized. Any CloudComputing service will need to present a 'centralized' view for management of the services maintained on the 'cloud'. It's a GoodThing, though, if the services themselves can be distributed: a well-designed CloudComputing service can achieve scalable performance in the face of the SlashDot floods, automated redundancy, region-local computations, and so on. This means LocationTransparency, rather than centralization, should be the default.} Maybe we should make a distinction between a resource cloud and a service cloud. Google-mail is an example of a service. But I'm also thinking about resources like disk space and networks. ------- What happens if your cloud company goes '''bankrupt or flakes or lies about backups'''? Many of us have been burned by the TheAdjunct fiasco where a year's worth of material went to the heavenly bit bucket forever, for example. You may have a backup copy of the files (if you prepared), but when you slap a custom application that requires language version Q and database version R into a new environment, they may likely not have version Q of the language and version R of the database and some tweaking will have to be done, all the while our app is down. Finding a service provider with the same combinations of your platform stack versions is just not likely. My example was a 2-factor stack, but there are apps with many more factors. It's almost comparable to finding a matching resume of all the tools your company uses when hiring. It rarely happens and training time is required. Plus, there are probably some configuration differences. More standardization would have to take place for this to work sufficient to trust. Perhaps the "stacks" should be standardized, such as grouping a given '''version set''' of Php and MySql and Apache together as PMA Stack 1.0, Stack 2.0, etc. But this would take industry agreement on the version set and take well-coordinated security patching that may go outside of (not match up with) each product by itself. The industry buy-in would have to reach a sufficient user-base size threshold before there is enough incentive for such multi-product coordination. It could reach such maybe if a handful of big players such as IBM and Google together defined the stack set and supported an OSS patching "authority" and pattern guidelines. There is a '''disincentive for hosting/cloud companies to make it easy to leave them''', hindering plug-compatible services. One could perhaps prepare for the above by hiring two cloud providers (if they find a version stack match) and every month test the backup on the "spare" vendor, but the effort is similar to an in-house solution of the same such that the management of reliability approaches the savings of the cloud. In some ways this all smacks of the idealism of the ThinClient fad once triggered by Sun. The standardization needed for ThinClient to work couldn't keep up with the pace of user interface changes. --top What would a '''cloud-friendly''' environment look like? A development tool that packaged the components into a single product may narrow down the version variations because one is targeting or shopping for a single specific version instead of a stack combo of versions. AllaireColdFusion ("CF" for short) has something pretty close to that. Only the database is separate. Some complain about Cold Fusion language(s) for various reasons, but the company did give environment and configuration packaging a lot of thought and most of it is strait-forward and covers a lot of ground, including scheduled scripts. (Newer versions come with built-in database, but I'm not sure it's powerful enough for all but small applications. Also, one can use its JavaScript-like language if they don't like the tag-based variation.) -t ''What do you mean by "cloud"? Depending on the context, it could mean "the Internet", "your data-center", "our data-center", "their data-centre", "a cluster of closely-linked computational devices", or "something with computers that I don't know enough about to be more specific about."'' Hosted internal, biz-to-biz, and/or Internet applications and file "servers". Basically it's "web hosting services", but with an emphasis on commoditizing services or tools to make them more hosting-vendor-swappable so that one can move them from one "cloud" to another. (At least that's my working description of "cloud" for this topic.) I'm defining it by commodity level, not so much hardware type. If it's commoditized well enough, the hardware either does not matter or is a minor issue, to be scaled up as needed upon request and payment. This does not mean that all applications magically scale up in terms of volume, speed, and/or redundancy count; for each app has to consider its max scalability during design. But at least the designers can '''design around commodity cloud standards''' rather than worry about whether they have to target IBM server 123-XYZ.23918918 or Oracle Server 765-BBB.492381278 with Microsoft Windows 8.492382 and PHP version 5.4838278393, etc. With CF, one pretty much only had to worry about which CF version as linear list: not combos of language versions, server versions, and (hopefully) database versions. It's not realistic for multiple vendors to support all combo's of such stacks. If you want hosting vendor swappability, then '''bundling the environment/stack''' into a single package "line" will probably be necessary. -t --------- I am interested in super-clouds, i.e. where resources can float between different cloud service providers based on whichever is cheaper - modulo inertia and security policy. CloudComputing becomes a fungible utility. Obviously you could not write code for a specific provider - instead, you'd need to write to some higher-level abstraction that can be compiled and distributed to many providers. This would greatly reduce economic risks, and a fair portion of the cloud could be provided by an ad-hoc network of open devices (e.g. your iPhone could dedicate some storage to serve as a provider of the cloud, not just a client). --------------- '''Shifting the Burden to Apps''' Perhaps if the services were simplified, application clouds could avoid or reduce the kind of combo versioning headaches mentioned above. For example, databases (DBMS) could be simplified and use a simpler query language than SQL (broken into simpler primitives). This would push more of the burden of specifics into the cloud-user's application. For example, security (roles, users, etc.) would be removed from the DBMS as part of the simplification process. However, the cloud renter's application would then have to implement security itself or find a code library for it rather than use a DBMS's security engine. The disadvantage is that the app has to reinvent and take care of more issues than it did before. The advantage is that it's easier to swap cloud vendors. -t ----------------- To get away from the "stack combo" problem, perhaps cloud vendors could simply '''host virtual servers'''. The customer manages whats on such servers such as which language version, database version etc. The "cloud" vendors would then essentially be rented chip-farms and not software service providers. But, managing the stack and the OS is probably 2/3 of typical maintenance cost, not the hardware, such that one is not saving much and might as well physically host the boxes themselves. ''Sure. It's quite popular, and is called VPS (virtual private server) hosting. Managing the stack is inexpensive -- when a host dies, you yank it from the rack, plug in another, and configure the bare metal (i.e., a blank server) back to production using cloud orchestration tools. Upgrading uses the same tools to re-image the bare metal.'' ------- See also: MergingFilesAndDatabase ------- CategoryDistributed