4th Generation Data Centers: Containerized Data Centers
April 26,2020
Prepared By: Mark Rauchwarter
The 4th generation of data centers is emerging, bringing with them a radical redesign from their predecessors. Self-contained containers now allow for modularity and contain the necessary core components that allow this new design to function. This paper discusses the advancements in data center management and the changes in technology and business goals that have driven the evolution of the data center into its 4th generation.
It will also take a closer look into containerized data center solutions and how their integration into the data center environment provides businesses with improved value.
1st Generation
The first generation of data center design lasted for many years and remained relatively similar to the original purpose-built computer room in the mid-1900s. According to Manos, “these facilities focused more around uptime, reliability and redundancy.
Big infrastructure was held accountable to solve all potential environmental shortfalls. ” (Manos, 2008) For many data centers, these are still their primary operational design and continue to function as 1st generation data centers today.
2nd Generation
Second generation data centers started to emerge when it was realized that first generation data centers were quickly becoming outdated and were unable to keep up with the demands of what was happening with technology, the environment, and the marketplace.
These facilities looked closer at, and incorporated, options to improve energy efficiency, sustainability, and the total cost of operations. These facilities looked more at their overall costs, rather than just initial upfront costs at the time of their creation.
3rd Generation
Third generation data centers have just recently been brought into operation, and for some, continue to be designed and built instead of opting for the fourth generation design. This third generation focuses on improving operating costs by focusing even more closely on energy efficiency and by starting to incorporate modularity into the designs.
Modularity allows for data centers to more easily scale to match the market and needs of the business, by making smaller increases or decreases to the infrastructure of the data center.
4th Generation
Data Center Design As data centers continue along their evolutionary path, the industry is now starting to embrace what is being called the fourth generation of data center design. This generational advancement further builds on its predecessors by continuing to focus on the core aspects that ultimately lead to reducing overall capital expenditures, and improving cost efficiencies.
This includes further pushing the envelope of modularity allowing for increased scalability, energy efficiency, and reducing operational and maintenance requirements. Manos summarizes the promise of fourth generation design as “a highly modular, scalable, efficient, just-in-time data center capacity program that can be delivered anywhere in the world very quickly and cheaply, while allowing for continued growth as required. ” (Manos, 2008) Modularity Modularity is one of the key themes the fourth generation of data center focuses on.
Modularity allows data centers to more easily scale to meet the needs of the business. By having a modular data center, costs can be more easily managed over the lifetime of the data center and can respond to fluctuations in the market and computing requirements since modular units can be added or removed more easily and cost effectively than building traditional data center structures. “Modular data centers reached prominence in 2007, [and] are now offered by many suppliers, and being deployed by ‘utility scale’ data center operators such as Microsoft and Google and in smaller installations throughout the industry. (Coles, 2011) Information Technology (IT) infrastructure manufacturers have responded to this by providing a variety of self-contained modular units, sometimes housed in shipping containers. These self-contained units can further improve costs by incorporating cooling and backup power systems appropriately scaled for the unit, no longer requiring a significant centrally located infrastructure outside of network connectivity to the modular units and primary power connections. Some data centers currently in development have fully grasped the concept of modularity in developing their fourth generation design, such as Microsoft.
In Microsoft’s design, they will take the flexibility that containerized servers provide and apply it across the entire facility, constructing it entirely “of modular units of prefabricated mechanical, electrical, security components, etc. , in addition to containerized servers. ” (Manos, 2008) This means that no building is required to be constructed to house these modular units; instead they will sit outside and connect directly to the core infrastructure not already contained within the modular units.
Energy Efficiency Another key component to fourth generation data center design is an ongoing focus on improving energy efficiencies within the data center. As energy costs continue to rise across the globe, more companies and industries are looking for ways to reduce those costs. Additionally, with a greater focus on green initiatives, such as reducing carbon emissions, the data center has become a critical focal point because it is one of the primary areas where a significant amount of power is used.
According to IBM, “with the cost of energy on the increase, we estimate that data centers use 30 to 80 times the energy of an office building per square foot (or square meter). This means that energy can represent 70 percent of the operational cost of a data center. ” (IBM, 2011) Cooling Systems Cooling system requirements go hand-in-hand with improving energy efficiency within the data center. Improved cooling techniques and methodologies help to further reduce energy costs.
Several data center facilities have embraced this concept by allowing the IT equipment to run warmer than past generational data centers. The latest technology has been built in such a way that allows equipment to run at an increased temperature without doing significant damage to the components or producing performance issues. This translates into reduced operational costs from the cooling systems by allowing the area to remain at a warmer temperature, therefore not requiring the cooling units to run as long.
In some cases, if the data center is located in a temperate or cooler climate zone, outside air can be used alone without any supplemental cooling system. This allows for the greatest energy efficiency as measured by PUE (Power Utilization Effectiveness – defined as Total Facility Power / IT Power). (Coles, 2011) Containerized Infrastructure Solutions Self-contained modular data center units have made their way to the marketplace within the past 5 to 10 years. These units vary fairly significantly by what components and options are contained within, depending on what a customer’s needs may be.
These modular units can be as simple as housing only racks of severs, requiring outside connections for power, backup power, cooling/ventilation, and network connectivity, to as complex as containing all necessary components including backup power and cooling, and only requiring connections to primary power and WAN (Wide Area Network) connectivity. Modularity Figure [ 1 ] – A Chicago Area Data Center Featuring Stacked Container Units (Coles, 2011) Figure [ 1 ] – A Chicago Area Data Center Featuring Stacked Container Units (Coles, 2011) The containerized solutions come in a variety of sizes.
Some are housed in 10, 20, or 40 foot shipping containers, while others are purpose built but still allow for the modularity on the scale of a shipping container size. These prefabricated structures help data centers and businesses more easily manage costs by having a definitive cost associated with its initial purchase, and calculable ongoing operational costs. Many of these units are made to sit directly outside and therefore only require the necessary external connections for power, network connectivity, and possibly cooling if not self-contained.
As such, no building or raised floor is required with this setup. The self-contained, prefabricated units also allow for decreased time of implementation from their purchase to their operational state. While a standard brick-and-mortar facility may take months, if not years, to become operational, these units can be shipped and operational in a matter of weeks to months. This reduces, drastically in some cases, overall initial implementation and build costs. According to a study conducted by Hewlett-Packard (HP), a typical brick-and-mortar data center was constructed for a total cost of $58,000,000.
The equivalent modular data center through the HP Flexible DC program was only $28,000,000, resulting in a 55% reduction in initial capital development costs. (HP, 2010) Server Configuration Within these containerized solutions, a range of server configurations are available. These configurations range from several hundred servers to several thousand servers, depending on the needs of the customer or the deployment scenario. Server types also range in size from full-rack options to half-rack options and blade server designs. Figure [ 2 ] – HP’s POD Unit Featuring a Single Row of IT Rack Space.
Cooling Design Uses Overhead Water-Cooled Coils (Coles, 2011) Figure [ 2 ] – HP’s POD Unit Featuring a Single Row of IT Rack Space. Cooling Design Uses Overhead Water-Cooled Coils (Coles, 2011) An inherent issue with these containerized solutions, however, is their lack of space. This can make maintenance and any sort of physical configuration changes difficult for technicians. Some approaches to combat this issue are to instead utilize only the systems that continue to work properly, and take all malfunctioning systems offline until the container is replaced.
A decision to proceed with that approach needs to be fully discussed and the calculated on which option may yield the most risk for additional costs. Networking Figure [ 3 ] – Oracle Sun Modular Data Center, Featuring a Unique IT Rack Layout (Coles, 2011) Figure [ 3 ] – Oracle Sun Modular Data Center, Featuring a Unique IT Rack Layout (Coles, 2011) In nearly all of the containerized solutions, networking is included as part of the prefabricated unit. A primary reason for this is because cable management can become an issue in such a confined space.
Cables and wiring need to be properly managed in order to make the containers an efficient space. Standard networking equipment can be used, but is often opted for the fastest available, either utilizing fiber or 10 Gigabit connection options for optimum data center performance. Power There are multiple power options when it comes to containerized solutions. Depending on the size of the unit, this can dictate the size and connection requirements for the container. Units can be configured for a hard-wired stationary installation or for a quick-disconnect installation, depending on what may be most appropriate or supported by the vendor.
As well, some units include incorporated backup power solutions. Uninterruptable Power Supplies (UPS) can be included at the rack level “that can provide sufficient hold-up time for an automatic switch-over from the primary to secondary power source. ” (SGI, 2011) The UPS can also be integrated into the entire container. The power distribution systems used when a UPS is incorporated into the container can help further improve energy efficiencies by reducing the amount of transformation from AC to DC power and removing components between the utility and the server. HP, 2010) This can be done by removing the power supplies from the individual servers and connecting DC current directly from the UPS to the servers and network equipment. Cooling Containerized data center units have options to include or exclude cooling from the container. When opting to not include cooling in the self-contained unit, an outside source is required. Typically this is accomplished through an external hookup, or an attachable, modular cooling system, sometimes provided by the same vendor.
What seem to be most prominent in the market, however, are the units that include the necessary cooling systems. These systems come in a variety of standard cooling options, including utilizing a typical hot and cold aisle configuration, or can utilize water and/or refrigerant cooling attached directly to the racking within the enclosure. Each option represents a different level of energy efficiency. An analysis was completed by M. Bramfitt & H. Coles that paired with different vendors that provided cooling system options for their containerized units. Figure 4 displays a breakdown of the results.
From the results, notice that utilizing a Free Air system, which utilizes only outside air to help cool the container, without any other supplemental cooling options, provided the best PUE. (Coles, 2011) Figure [ 4 ] In the example of Microsoft’s fourth generation data center previously mentioned, their facility will utilize either an air-side economizer or a water-side economizer. This allows Microsoft to be flexible with the usage of their facility and determine whether certain containers are more critical than another and if one version of cooling is better suited for the situation over another.
These options also provide the lowest PUE values, resulting in the greatest efficiencies. Procurement Considerations Purchasing a modular data center unit requires having a complete understanding of the environment and the intended future goals of the data center. To many, a containerized unit is a significant capital expenditure and the wide variety of options in the market may make the decision on which unit to buy, difficult. Bramfitt & Coles provide a comprehensive list of items that should be documented and considered before and during the vendor selection process: * Before engaging with a vendor, document the following: Required initial and future rack space * Required initial, average and future maximum IT power per rack in each module if multiple modules will be needed. Detailed list of IT equipment for each rack desired. * Allowable maximum IT equipment inlet air temperature * Minimum and maximum IT equipment cooling air exit temperature * IT equipment air flow type (e. g. side entry or front to back) * Required power connections per rack – i. e. single or redundant power input * Select Cooling Technology * Tower chilled water Water-cooled chiller combined with tower chilled water * Air-cooled chiller * DX compressor cooling * Review the following topics with potential modular and IT equipment suppliers * For cooling requiring chilled water: highest possible chilled water temperature that will meet the IT air temperature inlet requirements * Humidity controls and requirements * Energy for IT equipment cooling air circulation provided by the modular system. This should be in the range of 2 to 4 percent of the IT power. * Cooling fluid pumping energy * Modular enclosure heat insulation specifications Removal of cooling fans inside IT equipment * IT equipment DC power * Part load energy efficiency and controls * Additional Requirements * Service Availability (some cooling designs may contain difficult to obtain materials needed for repairs, e. g. fans or specialized coolants, or require specialized tools or specific training to make repairs). For example it may be more convenient to make timely interim repairs to cooling systems using only water, by on-site personnel, compared to systems that use refrigerant that need specialists trained in the use of refrigerants. Service access to the unit (clearances to accommodate connections, door openings, weather protection, IT equipment installation, etc. ) * Site security, including lighting, fencing, and access control * Proximity to existing power and chilled water distribution (if used) systems * Backup power equipment, if desired (including generators, switchgear, and fuel supply) * Control and reporting systems for the modular units should be compatible with existing building management systems.
Check if software is available for monitoring energy use or calculating energy efficiency metrics. * Smoke detection and fire suppression systems, and interconnection to existing systems if present * Site access for delivery and locating, maximum allowable weight, space to operate crane for unloading * Module orientation relative to prevailing wind direction may be a consideration for units equipped with air-side economizers (outside-air cooling) to realize maximum energy efficiency performance. Pad requirements (modular/container units often require significant foundation/pad upgrades) * Condensation management (local officials may not allow connection of condensate drains directly to sanitary sewer systems without a permit) * Emergency power-off functionality may be required by fire protection services * Authorities having local jurisdiction should be consulted to determine agency requirements, applicable building and safety codes, and taxation treatment if applicable. (Coles, 2011) Additional Considerations
While containerized data centers are shaping up to be the future of the data center, there are some significant concerns that should be considered before deciding if a containerized data center is the proper solution for the situation. John Edwards states what he believes to be the primary Cons of a modular data center: * Durability – The jury’s still out on how well pod and pre-fab modular data centers will withstand the ravages of time and weather. * Service availability – Provisioning utility and network resources to pods and pre-fabs placed in remote locations can be difficult and expensive. Lack of space to work in – Most modular facilities, particularly pods, are designed to accommodate equipment, not people. * Vendor lock-in – Many modular data center offerings require adopters to commit to a vendor’s hardware and/or support offerings. * Security – An isolated pod or pre-fab may be easier to break into or vandalize than an ordinary building. (Edwards, 2011) A modular design may not always be the best fit. While the benefits are fairly clear when building a new, large data center, some organizations may be better suited with a more traditional data center.
Similarly, Edwards writes, “an existing facility – already bought and paid for – may be able to be more densely populated, making it a better financial bet than adding a modular unit. ” (Edwards, 2011) The Future of Data Centers The concepts of the fourth generation data center have the opportunity to revolutionize the way people think about what a data center is. The idea of a large, cold room with racks of servers, sitting on a raised floor may soon be a thing of the past. The idea of modularity, which allows for greater scalability, has piqued the interests of the industry.
With the improvements in broadband technologies and the growth of Internet backbones, the doors have been opened for where data centers can be placed and how they can be utilized. Virtualization of servers was just the first step in the virtualization of data centers. As virtualization technologies continue to advance and develop, industry analysts predict the entire data center may become virtualized. “In a virtual environment, time-honored techniques like passive standby systems are a thing of the past because administrators will have the power of near-instant provisioning of virtual machines should one physical system go down.
So even though two machines are in full active mode, one can always handle the workload of the other in case of failover, so the enterprise gains a more efficient use of its infrastructure. ” (Cole, 2009) The Cloud is also playing an important role in the future of data centers. Cloud systems will pair well with modular, containerized solutions by allowing cloud service providers to easily scale up and down as demand increases or decreases.
Cloud technologies will advance, allowing containerized systems to automatically communicate with each other, provision, de-provision and move data and resources around the globe to be most appropriately placed depending on demand. There is the opportunity that data center resource usage will reach a true utility level of functionality, and businesses will be able to tap into those resources as needed. Orchestration software is on the path of performing a much larger role in the data center, taking care of all the documentable, repeatable processes otherwise done by humans, resulting in further cost savings and reduced time to resolution.
The potential advent of programs that can learn processes and develop new processes (artificial intelligence – AI) is being researched and developed, further paving the way for more automation. (CloudBzz, 2011) Conclusion Modular and containerized data centers are instrumental in the key design aspects of the fourth generation data center. With an increased focus on ease of scalability, reducing overall capital expenses as well as operating costs, further improving energy efficiency, and supporting green initiatives, the fourth generation data center is well aligned to achieve those goals.
As technology has progressed over the past several decades, the demand on technology from businesses has been ever increasing. While it can never be certain just what will happen in the future, it can be assumed that the future data center will continue its evolution process, becoming ever more cost effective, automated, and capability rich. Bibliography The New Future of Data Centers. (2010, April/May). Focus DatacenterDynamics, pp. 19-20. What Will a Next Generation Data Center Look Like? (2010, June 2). Retrieved October 19, 2011, from IO Data Centers: http://www. odatacenters/com/blog/tag/nxt-generation-data-center/ CloudBzz. (2011, January 3). A Vision of the Future Cloud Data Center. Retrieved October 19, 2011, from CloudBzz: http://www. cloudbzz. com/a-vision-of-the-future-cloud-data-cente/ Cole, A. (2009, May 4). The Three Factors Shaping the Future of the Data Center. Retrieved October 19, 2011, from IT Business Edge: http://www. itbusinessedge. com/cm/community/features/articles/blog/the-three-factors-shaping-the-future-of-the-data-center/? cs=32327 Coles, M. B. (2011).
Modular/Container Data Centers Procurement Guide: Optimizing for Energy Efficiency and Quick Deployment. Lawrence Berkleley National Laboratory. Edwards, J. (2011, March 28). Data Centers: Make Mine Modular. Retrieved October 19, 2011, from ComputerWorld: http://www. computerworld. com/s/article/9215003/Data_centers_Make_mine_modular Hoover, J. N. (2008, March 1). Data Center Best Practices. Retrieved October 19, 2011, from InformationWeek: http://www. informationweek. com/news/206900660 HP. (2010). HP Flexible Data Center – A New Approach to Industrialized IT. Hewlett-Packard.
HP. (2011). Manifest: The Best Practices for Data Center Planning. Hewlett-Packard. IBM. (2011). Enterprise modular data center: Increase flexibility while reducing costs. Somers: IBM Global Services. Manos, M. (2008, December 2). Our Vision of Generation 4 Modular Data Centers – One way of Getting it just right… Retrieved October 19, 2011, from LooseBolts: http://loosebolts. wordpress. com/2008/12/02/our-vision-for-generation-r-modular-data-centers-one-way-of-getting-it-just-right/ SGI. (2011). Next Generation Data Center Infrastructure: ICE Cube Modular Data Center. SGI.
