Site reliability engineering

Site reliability engineering

Site reliability engineering (SRE) is a discipline in the field of software engineering and IT infrastructure support that monitors and improves the availability and performance of deployed software systems and large software services (which are expected to deliver reliable response times across events such as new software deployments, hardware failures, and cybersecurity attacks). There is typically a focus on automation and an infrastructure as code methodology. SRE uses elements of software engineering, IT infrastructure, web development, and operations to assist with reliability. It is similar to DevOps as they both aim to improve the reliability and availability of deployed software systems. == History == Site Reliability Engineering originated at Google with Benjamin Treynor Sloss, who founded SRE team in 2003. The concept expanded within the software development industry, leading various companies to employ site reliability engineers. By March 2016, Google had more than 1,000 site reliability engineers on staff. Dedicated SRE teams are common at larger web development companies. In middle-sized and smaller companies, DevOps teams sometimes perform SRE, as well. Organizations that have adopted the concept include Airbnb, Dropbox, IBM, LinkedIn, Netflix, and Wikimedia. == Definition == Site reliability engineers (SREs) are responsible for a combination of system availability, latency, performance, efficiency, change management, monitoring, emergency response, and capacity planning. SREs often have backgrounds in software engineering, systems engineering, and/or system administration. The focuses of SRE include automation, system design, and improvements to system resilience. SRE is considered a specific implementation of DevOps; focusing specifically on building reliable systems, whereas DevOps covers a broader scope of operations. Despite having different focuses, some companies have rebranded their operations teams to SRE teams. == Principles and practices == Common definitions of the practices include (but are not limited to): Automation of repetitive tasks for cost-effectiveness. Defining reliability goals to prevent endless effort. Design of systems with a goal to reduce risks to availability, latency, and efficiency. Observability, the ability to ask arbitrary questions about a system without having to know ahead of time what to ask. Common definitions of the principles include (but are not limited to): Toil management, the implementation of the first principle outlined above. Defining and measuring reliability goals—SLIs, SLOs, and error budgets. Non-Abstract Large Scale Systems Design (NALSD) with a focus on reliability. Designing for and implementing observability. Defining, testing, and running an incident management process. Capacity planning. Change and release management, including CI/CD. Chaos engineering. == Deployment == SRE teams collaborate with other departments within organizations to guide the implementation of the mentioned principles. Below is an overview of common practices: === Kitchen Sink === Kitchen Sink refers to the expansive and often unbounded scope of services and workflows that SRE teams oversee. Unlike traditional roles with clearly defined boundaries, SREs are tasked with various responsibilities, including system performance optimization, incident management, and automation. This approach allows SREs to address multiple challenges, ensuring that systems run efficiently and evolve in response to changing demands and complexities. === Infrastructure === Infrastructure SRE teams focus on maintaining and improving the reliability of systems that support other teams' workflows. While they sometimes collaborate with platform engineering teams, their primary responsibility is ensuring up-time, performance, and efficiency. Platform teams, on the other hand, primarily develop the software and systems used across the organization. While reliability is a goal for both, platform teams prioritize creating and maintaining the tools and services used by internal stakeholders, whereas Infrastructure SRE teams are tasked with ensuring those systems run smoothly and meet reliability standards. === Tools === SRE teams utilize a variety of tools with the aim of measuring, maintaining, and enhancing system reliability. These tools play a role in monitoring performance, identifying issues, and facilitating proactive maintenance. For instance, Nagios Core is commonly employed for system monitoring and alerting, while Prometheus (software) is frequently used for collecting and querying metrics in cloud-native environments. === Product or Application === SRE teams dedicated to specific products or applications are common in large organizations. These teams are responsible for ensuring the reliability, scalability, and performance of key services. In larger companies, it's typical to have multiple SRE teams, each focusing on different products or applications, ensuring that each area receives specialized attention to meet performance and availability targets. === Embedded === In an embedded model, individual SREs or small SRE pairs are integrated within software engineering teams. These SREs collaborate with developers, applying core SRE principles—such as automation, monitoring, and incident response—directly to the software development lifecycle. This approach aims to enhance reliability, performance, and collaboration between SREs and developers. === Consulting === Consulting SRE teams specialize in advising organizations on the implementation of SRE principles and practices. Typically composed of seasoned SREs with a history across various implementations, these teams provide insights and guidance for specific organizational needs. When working directly with clients, these SREs are often referred to as 'Customer Reliability Engineers.' In large organizations that have adopted SRE, a hybrid model is common. This model includes various implementations, such as multiple Product/Application SRE teams dedicated to addressing the specific reliability needs of different products. An Infrastructure SRE team may collaborate with a Platform engineering group to achieve shared reliability goals for a unified platform that supports all products and applications. == Industry == Since 2014, the USENIX organization has hosted the annual SREcon conference, bringing together site reliability engineers from various industries. This conference is a platform for professionals to share knowledge, explore effective practices, and discuss trends in site reliability engineering.

Carrier cloud

In cloud computing, a carrier cloud is a class of cloud that integrates wide area networks (WAN) and other attributes of communications service providers’ carrier-grade networks to enable the deployment of highly-complex applications in the cloud. In contrast, classic cloud computing focuses on the data center and does not address the network connecting data centers and cloud users. This may result in unpredictable response times and security issues when business-critical data are transferred over the Internet. == History == The advent of virtualization technology, cost-effective computing hardware, and ubiquitous Internet connectivity have enabled the first wave of cloud services starting in the early years of the 21st century. But many businesses and other organizations hesitated to move to more demanding applications, from on-premises dedicated hardware to private or public clouds. As a response, communications service providers started in the 2010/2011 time frame to develop carrier clouds that address perceived weaknesses in existing cloud services. Cited weaknesses vary but often include possible downtime, security issues, high cost of custom software and data transfer, inflexibility of some cloud apps, poor customer and nonfulfillment of service level agreements (SLAs). == Characteristics == To enable the deployment of time-sensitive and business critical applications in the cloud, the carrier cloud is designed to match or even exceed the characteristics of on-premises deployments. Therefore, the carrier cloud is characterized by some or all of the following items: Configurable, elastic network performance: Typical cloud computing solutions use the best effort of the public Internet to connect cloud users and data centers. This approach provides instant connectivity but does not offer control over network capacities, latencies, and jitter. Carrier clouds address these gaps with content delivery networks and/or dedicated virtual private networks (VPN) at OSI layers 1 (optical wavelengths), 2 (data link layer), and 3 (network layer). These VPNs can be configured to offer the desired performance parameters and exhibit the same type of elasticity for the network that regular clouds provide for servers and storage. To achieve the requested performance parameters, such as low latency, cloud applications can be (automatically) allocated to distributed data centers that are close enough to the cloud users. Automatic resource placement: For a cloud with multiple data centers, information about both the data center and the connecting network is relevant for a decision of where to place cloud images and storage volumes. For this decision, carrier clouds can obtain relevant information about the network, e.g., using the Application-Layer Traffic Optimization (ALTO) protocol. High level of security and governance: Cloud application providers are subject to general and domain specific security, privacy, and governance requirements and regulations, such as the European Data Protection Directive and the U.S. Health Insurance Portability and Accountability Act. For added security, the wide area network of the carrier cloud can provide segregated encrypted or unencrypted network links that are not accessible from the general Internet. At the data center, the carrier cloud provides e.g. virtual private servers, management processes, logs, and documentation to fulfill security and governance rules. Location control: Fundamentally, cloud users should not be concerned with the geographic location of their cloud resources. However, privacy and other regulations may mandate that certain types of data must not be sent outside a national jurisdiction or other geographical region. Open APIs: Carrier clouds provide graphical user interfaces and Web application programming interfaces that allow cloud application providers to set up, manage, and monitor both, the data center and the WAN, of their cloud services. == Architecture == Carrier clouds encompass data centers at different network tiers and wide area networks that connect multiple data centers to each other as well as to the cloud users. Links between data centers are used for failover, overflow, backup, and geographic diversity. Carrier clouds can be set up as public, private, or hybrid clouds. The carrier cloud federates these cloud entities by using a single management system to orchestrate, manage, and monitor data center and network resources as a single system.

Coda (document editor)

Coda is a cloud-based multi-user document editor. == Features == Coda is a document editor that provides features from spreadsheets, presentation documents, word processor files, and apps. Possible uses for Coda documents include using them as a wiki, database, or project management tool. Coda has built a formula system, much like spreadsheets commonly have, but in Coda documents, formulas can be used anywhere within the document, and can link to things that aren't just cells, including other documents, calendars or graphs. Coda also has the ability to integrate with custom third-party services, and has automations. It has offered $1 million in grants for developers that create such integrations. == Development == Coda Project, Inc. was founded by Shishir Mehrotra and Alex DeNeui in June 2014. Having met at MIT, they developed the project mostly privately before announcing a public beta in October 2017. The company was named Coda, which is an anadrome for “a doc”. Coda raised $60 million in venture capital funding over two rounds by 2017. The Coda software came out of beta in February 2019. Version 1.0 had an improved user interface, new features for folders and workspaces, and permission levels for accessing files. Coda raised another $80 million in 2020, and $100 million in 2021. The 2021 funding brought Coda's valuation to $1.4 billion, making it a unicorn. In December 2024, Coda was acquired by Grammarly in an all-stock deal for an undisclosed amount. In October 2025, Grammarly rebranded as Superhuman, incorporating Coda as a core product within the new Superhuman productivity suite alongside Grammarly's writing tools, Superhuman Mail, and a new AI assistant called Superhuman Go.

Cloud-to-cloud integration

Cloud-to-Cloud Integration ( C2I ) allows users to connect disparate cloud computing platforms. While Paas (Platform as a service) and Saas (Software as a service) continue to gain momentum, different vendors have different implementations for cloud computing, e.g. Database, REST, SOAP API. Another name for Cloud-to-Cloud Integration is Cloud-Surfing. See also Cloud-based integration

Fabric computing

Fabric computing or unified computing involves constructing a computing fabric consisting of interconnected nodes that look like a weave or a fabric when seen collectively from a distance. Usually the phrase refers to a consolidated high-performance computing system consisting of loosely coupled storage, networking and parallel processing functions linked by high bandwidth interconnects (such as 10 Gigabit Ethernet and InfiniBand) but the term has also been used to describe platforms such as the Azure Services Platform and grid computing in general (where the common theme is interconnected nodes that appear as a single logical unit). The fundamental components of fabrics are "nodes" (processor(s), memory, and/or peripherals) and "links" (functional connections between nodes). While the term "fabric" has also been used in association with storage area networks and with switched fabric networking, the introduction of compute resources provides a complete "unified" computing system. Other terms used to describe such fabrics include "unified fabric", "data center fabric" and "unified data center fabric". Ian Foster, director of the Computation Institute at the Argonne National Laboratory and University of Chicago suggested in 2007 that grid computing "fabrics" were "poised to become the underpinning for next-generation enterprise IT architectures and be used by a much greater part of many organizations". == History == While the term has been in use since the mid to late 1990s the growth of cloud computing and Cisco's evangelism of unified data center fabrics followed by unified computing (an evolutionary data center architecture whereby blade servers are integrated or unified with supporting network and storage infrastructure) starting March 2009 has renewed interest in the technology. There have been mixed reactions to Cisco's architecture, particularly from rivals who claim that these proprietary systems will lock out other vendors. Analysts claim that this "ambitious new direction" is "a big risk" as companies such as IBM and HP who have previously partnered with Cisco on data center projects (accounting for $2–3bn of Cisco's annual revenue) are now competing with them. In 2007, Wombat Financial Software launched the "Wombat Data Fabric," the first commercial off-the-shelf software platform providing high performance / low-latency RDMA-based messaging across an Infiniband switch. == Key characteristics == The main advantages of fabrics are that massive concurrent processing combined with a huge, tightly coupled address space makes it possible to solve huge computing problems (such as those presented by delivery of cloud computing services); and that they are both scalable and able to be dynamically reconfigured. Challenges include a non-linearly degrading performance curve, whereby adding resources does not linearly increase performance which is a common problem with parallel computing and maintaining security. == Companies == As of 2015 companies offering unified or fabric computing systems include Avaya, Brocade, Cisco, Dell, Egenera, HPE, IBM, Liquid Computing Corporation, TIBCO, Unisys, and Xsigo Systems.

C3D Toolkit

C3D Toolkit is a proprietary cross-platform geometric modeling kit software developed by Russian C3D Labs (previously part of ASCON Group). It's written in C++ . It can be licensed by other companies for use in their 3D computer graphics software products. The most widely known software in which C3D Toolkit is typically used are computer aided design (CAD), computer-aided manufacturing (CAM), and computer-aided engineering (CAE) systems. C3D Toolkit provides routines for 3D modeling, 3D constraint solving, polygonal mesh-to-B-rep conversion, 3D visualization, and 3D file conversions etc. == History == Nikolai Golovanov is a graduate of the Mechanical Engineering department of Bauman Moscow State Technical University as a designer of space launch vehicles. Upon his graduation, he began with the Kolomna Engineering Design bureau, which at the time employed the future founders of ASCON, Alexander Golikov and Tatiana Yankina. While at the bureau, Dr Golovanov developed software for analyzing the strength and stability of shell structures. In 1989, Alexander Golikov and Tatiana Yankina left Kolomna to start up ASCON as a private company. Although they began with just an electronic drawing board, even then they were already conceiving the idea of three-dimensional parametric modeling. This radical concept eventually changed flat drawings into three-dimensional models. The ASCON founders shared their ideas with Nikolai Golovanov, and in 1996 he moved to take up his current position with ASCON. As of 2012 he was involved in developing algorithms for C3D Toolkit. In 2012 the earliest version of the C3D Modeller kernel was extracted from KOMPAS-3D CAD. It was later adopted to a range of different platforms and advertised as a separate product. == Overview == It incorporates five modules: C3D Modeler constructs geometric models, generates flat projections of models, performs triangulations, calculates the inertial characteristics of models, and determines whether collisions occur between the elements of models; C3D Modeler for ODA enables advanced 3D modeling operations through the ODA's standard "OdDb3DSolid" API from the Open Design Alliance; C3D Solver makes connections between the elements of geometric models, and considers the geometric constraints of models being edited; C3D B-Shaper converts polygonal models to boundary representation (B-rep) bodies; C3D Vision controls the quality of rendering for 3D models using mathematical apparatus and software, and the workstation hardware; C3D Converter reads and writes geometric models in a variety of standard exchange formats. == Features == == Development == == Applications == Since 2013 - the date the company started issuing a license for the toolkit -, several companies have adopted C3D software components for their products, users include: Recently, C3D Modeler has been adapted to ODA Platform. In April 2017, C3D Viewer was launched for end users. The application allows to read 3D models in common formats and write it to the C3D file format. Free version is available.

Software construction

Software construction is the process of creating working software via coding and integration. The process includes unit and integration testing although does not include higher level testing such as system testing. Construction is an aspect of the software development lifecycle and is integrated in the various software development process models with varying focus on construction as an activity separate from other activities. In the waterfall model, a software development effort consists of sequential phases including requirements analysis, design, and planning which are prerequisites for starting construction. In an iterative model such as scrum, evolutionary prototyping, or extreme programming, construction as an activity that occurs concurrently or overlapping other activities. Construction planning may include defining the order in which components are created and integrated, the software quality management processes, and the allocation of tasks to teams and developers. To facilitate project management, numerous construction aspects can be measured; these include the amount of code developed, modified, reused, and destroyed, code complexity, code inspection statistics, faults-fixed and faults-found rates, and effort expended. These measurements can be useful for aspects such as ensuring quality and improving the process. == Activities == Construction includes many activities. === Coding === The following are a few of the key aspects of the coding activity: Naming Choice of name for each identifier. One study showed that the effort required to debug a program is minimized when variable names are between 10 and 16 characters. Logic Organization into statements and routines Highly cohesive routines proved to be less error prone than routines with lower cohesion. A study of 450 routines found that 50 percent of the highly cohesive routines were fault free compared to only 18 percent of routines with low cohesion. Another study of a different 450 routines found that routines with the highest coupling-to-cohesion ratios had 7 times as many errors as those with the lowest coupling-to-cohesion ratios and were 20 times as costly to fix. Although studies showed inconclusive results regarding the correlation between routine sizes and the rate of errors in them, but one study found that routines with fewer than 143 lines of code were 2.4 times less expensive to fix than larger routines. Another study showed that the code needed to be changed least when routines averaged 100 to 150 lines of code. Another study found that structural complexity and amount of data in a routine were correlated with errors regardless of its size. Interfaces between routines are some of the most error-prone areas of a program. One study showed that 39 percent of all errors were errors in communication between routines. Unused parameters are correlated with an increased error rate. In one study, only 17 to 29 percent of routines with more than one unreferenced variable had no errors, compared to 46 percent in routines with no unused variables. The number of parameters of a routine should be 7 at maximum as research has found that people generally cannot keep track of more than about seven chunks of information at once. One experiment showed that designs which access arrays sequentially, rather than randomly, result in fewer variables and fewer variable references. One experiment found that loops-with-exit are more comprehensible than other kinds of loops. Regarding the level of nesting in loops and conditionals, studies have shown that programmers have difficulty comprehending more than three levels of nesting. Control flow complexity has been shown to correlate with low reliability and frequent errors. Modularity Structuring and refactoring the code into classes, packages and other structures. When considering containment, the maximum number of data members in a class shouldn't exceed 7±2. Research has shown that this number is the number of discrete items a person can remember while performing other tasks. When considering inheritance, the number of levels in the inheritance tree should be limited. Deep inheritance trees have been found to be significantly associated with increased fault rates. When considering the number of routines in a class, it should be kept as small as possible. A study on C++ programs has found an association between the number of routines and the number of faults. A study by NASA showed that the putting the code into well-factored classes can double the code reusability compared to the code developed using functional design. Error handling Encoding logic to handle both planned and unplanned errors and exceptions. Resource management Managing computational resource use via exclusion mechanisms and discipline in accessing serially reusable resources, including threads or database locks. Security Prevention of code-level security breaches such as buffer overrun and array index overflow. Optimization Optimization while avoiding premature optimization. Documentation Both embedded in the code as comments and as external documents. === Integration === Integration is about combining separately constructed parts. Concerns include planning the sequence in which components will be integrated, creating scaffolding to support interim versions of the software, determining the degree of testing and quality work performed on components before they are integrated, and determining points in the project at which interim versions are tested. === Testing === Testing can reduce the time between when faulty logic is inserted in the code and when it is detected. In some cases, testing is performed after code has been written, but in test-first programming, test cases are created before code is written. Construction includes at least two forms of testing, often performed by the developer who wrote the code: unit testing and integration testing. === Reuse === Software reuse entails more than creating and using libraries. It requires formalizing the practice of reuse by integrating reuse processes and activities into the software life cycle. The tasks related to reuse in software construction during coding and testing may include: selection of the reusable code, evaluation of code or test re-usability, reporting reuse metrics. === Quality assurance === Techniques for ensuring quality as software is constructed include: Testing One study found that the average defect detection rates of Unit testing and integration testing are 30% and 35% respectively. Software inspection With respect to software inspection, one study found that the average defect detection rate of formal code inspections is 60%. Regarding the cost of finding defects, a study found that code reading detected 80% more faults per hour than testing. Another study shown that it costs six times more to detect design defects by using testing than by using inspections. A study by IBM showed that only 3.5 hours were needed to find a defect through code inspections versus 15–25 hours through testing. Microsoft has found that it takes 3 hours to find and fix a defect by using code inspections and 12 hours to find and fix a defect by using testing. In a 700 thousand lines program, it was reported that code reviews were several times as cost-effective as testing. Studies found that inspections result in 20% - 30% fewer defects per 1000 lines of code than less formal review practices and that they increase productivity by about 20%. Formal inspections will usually take 10% - 15% of the project budget and will reduce overall project cost. Researchers found that having more than 2 - 3 reviewers on a formal inspection doesn't increase the number of defects found, although the results seem to vary depending on the kind of material being inspected. Technical review With respect to technical review, one study found that the average defect detection rates of informal code reviews and desk checking are 25% and 40% respectively. Walkthroughs were found to have a defect detection rate of 20% - 40%, but were found also to be expensive especially when project pressures increase. Code reading was found by NASA to detect 3.3 defects per hour of effort versus 1.8 defects per hour for testing. It also finds 20% - 60% more errors over the life of the project than different kinds of testing. A study of 13 reviews about review meetings, found that 90% of the defects were found in preparation for the review meeting while only around 10% were found during the meeting. Static analysis With respect to Static analysis (IEEE1028), studies have shown that a combination of these techniques needs to be used to achieve a high defect detection rate. Other studies showed that different people tend to find different defects. One study found that the extreme programming practices of pair programming, desk checking, unit testing, integration testing, and regression testing can achieve a 90% defect detection rate. An experiment involving exper