By banding together, the trio of computer industry titans hopes to foster collaboration among vendors, universities and government agencies around cloud computing, whose progress is hampered by "financial and logistical barriers," the companies said Tuesday.
Intel, Yahoo and HP are forming the Cloud Computing Test Bed, which they describe as a global, multi-data center, open-source effort designed to promote research on software, data center management and hardware for large-scale, Internet-hosted computing.
Partners in the initiative include the Infocomm Development Authority of Singapore, the University of Illinois at Urbana-Champaign, the National Science Foundation and the Karlsruhe Institute of Technology in Germany
These partners, HP Labs, Intel Research and Yahoo will host six "centers of excellence," each of which will have a cloud computing infrastructure mostly based on HP hardware and Intel processors. The centers will have 1,000 to 4,000 processor cores and are expected to be up and running later this year for selected researchers from around the globe.
Non-founding members will be invited to participate in the test bed by the end of the year, said John Manley, director of the Automated Infrastructure Lab with HP Labs in Bristol, England, in an interview.
Manley said the research could spawn new topics for researchers to study, such as how to ensure services keep running when equipment fails, known as fault tolerance.
"It might well be that some of the fault tolerance mechanisms we know nowadays have a hard time scaling," Manley said.
When companies begin using services hosted elsewhere for their processing, security is a huge concern. Those services must be created in a secure partition that is isolated from others. The goal is to be able to provision those services quickly in their own partition or cell, Manley said.
Another concern is the management of virtual machines when several operating systems are running on one piece of hardware. The configuration can save money on infrastructure but can be more complex to manage.
Much of intellectual property that comes out of the research will be shared. "The intention is to be very open about the results," Manley said. "This is being set up as an open collaborative framework."
"Cloud computing is of critical importance to the industry and Intel is very focused on that," Andrew Chien, vice president of Intel's Corporate Technology Group and director of Intel Research, said during a press conference on Tuesday.
The project will allow universities to conduct research on large-scale environments and re-create cloud computing scenarios from the real world. It's very important to allow academic researchers to perform "empirical work at scale," as opposed to limiting them to simply theoretical explorations and small "test tube" studies, Chien said.
"The revolution of information technology computing has just begun, and the excitement around cloud [computing] is a great example of that," Chien said.
CIOs and IT buyers in general will likely see concrete benefits in a few years, when the project's research efforts yield technological breakthroughs for faster and more powerful cloud computing products and services at lower costs, said Gartner analyst Martin Reynolds in an interview.
The industry in general will also benefit from letting computer-science university students work with large-scale cloud infrastructures, since they will carry this experience to their future jobs, he said. "I think this project will be successful in attaining its goals," Reynolds said.
The centers will run the open-source, distributed computing Apache Hadoop project from the Apache Software Foundation and other open-source, distributed computing software such as Pig, a parallel programming language developed by Yahoo Research.
In a reorganization announced in June, Yahoo announced the creation of a Cloud Computing & Data Infrastructure Group.
Prabhakar Raghavan, head of Yahoo's research unit, said that for his company, this effort builds on its M45 supercomputing data center for cloud computing research, which Carnegie Mellon University has already used. Yahoo believes academia will play a crucial part in developing the "next generation" of cloud infrastructure and Web applications, he said.
Meanwhile, HP Labs will use the centers to do research on "intelligent infrastructure and dynamic cloud services," an area the company has identified as a priority, the companies said.
Intel researchers will in turn focus on extending the company's knowledge and development of processors, chipsets and other technologies for cloud computing data centers.
