A motion picture film scanner is a device used in digital filmmaking to scan original film for storage as high-resolution digital intermediate files. A film scanner scans original film stock: negative or positive print or reversal/IP. Units may scan gauges from 8 mm to 70 mm (8 mm, Super 8, 9.5 mm, 16 mm, Super 16, 35 mm, Super 35, 65 mm and 70 mm) with very high resolution scanning at 2K, 4K, 8K, or 16K resolutions. (2K is approximately 2048×1080 pixels and 4K is approximately 4096×2160 pixels). Some models of film scanner are intermittent pull-down film scanners which scan each frame individually, locked down in a pin-registered film gate, taking roughly a second per frame. Continuous-scan film scanners, where the film frames are scanned as the film is continuously moved past the imaging pick up device, are typically evolved from earlier telecine mechanisms, and can act as such at lower resolutions. The scanner scans the film frames into a file sequence (using high-end computer data storage devices), whose single file contains a digital scan of each still frame; the preferred image file format used as output are usually Cineon, DPX or TIFF, because they can store color information as raw data, preserving the optical characteristics of the film stock. These systems take a lot of storage area network (SAN) disk space. The files can be played back one after each other on high-end workstation non-linear editing system (NLE) or a virtual telecine systems. The playback is at the normal rate of 24 frames per second (or original projection frame rate of: 25, 30 or other speeds). Each year hard disks get larger and are able to hold more hours of movies on SAN systems. The challenge is to archive this massive amount of data on to data storage devices. The scanned footage is edited and composited on work stations then mastered back on film, see film-out and digital intermediate. Scanned film frames may also be used in digital film restoration. The film may also be projected directly on a digital projector in the theater. The data film files may be converted to SDTV (NTSC or PAL) video TV systems. Film recorders are the opposite of film scanners, copying content from a computer system onto film stock, for preservation or for display using film projectors. Telecines are similar to film scanners. == Imaging device == The front end of a motion picture film scanner is similar to a telecine. The imaging system may be either a charge-coupled device (CCD), a complementary metal–oxide–semiconductor (CMOS) or photomultipliers imaging pick up. A lamp is used as the light source in a CCD imaging front end. The CCDs convert the light to the video signals. In a cathode-ray tube (CRT) imaging system the CRT (also called a Flying spot tube) is used as the light source and part of the scanning system. Photomultipliers or avalanche photodiodes are used to convert the light to electrical video signals. A prism and/or dichroic mirrors or color filters are used to separate the light into the three: red, green and blue, imaging pick up devices. == Image processing == The three color signals (RGB) are electronically processed and color graded. A 3D look up table (3D LUT) is usually applied to the RGB values before it is coded into the DPX output files. The DPX files are usually made output through a network port cable or an optical fiber port: HIPPI, Fibre Channel or newer systems like gigabit Ethernet. A computer then stores the files on to hard drives of a storage area network for later processing and use. Modern motion picture film scanners many have an option for an infrared CCD channel for dirt mapping, that can be used to automatically or in post manually remove dirt-dust spots on the film. The IR camera channel can be used with IR dirt and scratch removal system or made output on a four IR channel for downstream dirt and dirt and scratch removal systems. Popular downstream dirt and dirt and scratch removal systems are PF Clean and Digital ICE. HDR or high dynamic range is a new system, using a compositing and tone-mapping of images to extend the dynamic range beyond the native capability of the capturing device. This may be done by using a triple exposure for the film and then compositing the three back together. Compositing can be done in a workstation in none real time or in the scanner in real time. == Models == Bold indicates a currently produced model Single frame intermittent pull-down: ARRI - Arriscan Cintel - diTTo Filmlight - Northlight 1 (up to 6K, 16mm to VistaVision), Northlight 2 (up to 8K, 16mm to VistaVision) Imagica scanner, single frame intermittent scanner. Kodak - Cineon, the first system designed for DI work, included a scanner, tapes drives, workstations and a film recorder. Lasergraphics Director 13.5K, 8mm to 70mm, IMAX & VistaVision) Continuous motion scanning: Arri - ARRISCAN XT (up to 6K, S35 down to 16mm) Cintel's C-Reality/DSX and ITK - Millennium/dataMill. Under ownership of Blackmagic Design, the Cintel Scanner was released, with the current 3rd generation capable of up to 4K scans at 30 fps. DFT - Spirit Classic (up to 2K), Spirit 4k/2k/HD (up to 4K), POLAR HQ (up to 8K, 8mm to S35), OXScan 14K (up to 14K, 16mm to 70mm), Scanity HDR (up to 4K, 16mm to S35) Filmfabriek - HDS+ (up to 4k), Pictor Pro (up to 2.7K), Pictor (up to 1080p). Filmfabriek scanners can only scan 17.5mm or smaller film formats. GE4 - Golden Eye Four - Filmscanner, 38 Mega Pixel camera. LED light source and continuous film transport using Capstan. From Digital Vision. Lasergraphics ScanStation (6.5K, 8mm to 70mm, IMAX & VistaVision) Lasergraphics Archivist (up to 5K) MWA Nova Vario series with patented laser-based, sprocket and claw free transport for 16/35mm for realtime (24/25fps) scanning with sensors for either 2K+ 2236 x 1752, or 2.5K+ HDR High Dynamic Range at 2560 x 2160, direct optical and magnetic sound on and 16 and 35mm. MWA Nova Choice 2K+ patented laser-based, sprocket and claw free transport for 8/Super8, 9.5mm, 16mm realtime (24/25fps) scanning w at 2K+, 2236 x 1752 with direct optical and magnetic sound on 16mm, magnetic from main and balance stripes on 8, Super8. Faster than real time scanning at lower resolution. P+S Technik - SteadyFrame Universal Format Film Scanner Walde - FilmStar 4K UHD 2K @ 25fps, 4K UHD @ 6fps. 35mm/16mm/8mm archive quality, continuous motion capstan driven.
Inductive programming
Inductive programming (IP) is a special area of automatic programming, covering research from artificial intelligence and programming, which addresses learning of typically declarative (logic or functional) and often recursive programs from incomplete specifications, such as input/output examples or constraints. Depending on the programming language used, there are several kinds of inductive programming. Inductive functional programming, which uses functional programming languages such as Lisp or Haskell, and most especially inductive logic programming, which uses logic programming languages such as Prolog and other logical representations such as description logics, have been more prominent, but other (programming) language paradigms have also been used, such as constraint programming or probabilistic programming. == Definition == Inductive programming incorporates all approaches which are concerned with learning programs or algorithms from incomplete (formal) specifications. Possible inputs in an IP system are a set of training inputs and corresponding outputs or an output evaluation function, describing the desired behavior of the intended program, traces or action sequences which describe the process of calculating specific outputs, constraints for the program to be induced concerning its time efficiency or its complexity, various kinds of background knowledge such as standard data types, predefined functions to be used, program schemes or templates describing the data flow of the intended program, heuristics for guiding the search for a solution or other biases. Output of an IP system is a program in some arbitrary programming language containing conditionals and loop or recursive control structures, or any other kind of Turing-complete representation language. In many applications the output program must be correct with respect to the examples and partial specification, and this leads to the consideration of inductive programming as a special area inside automatic programming or program synthesis, usually opposed to 'deductive' program synthesis, where the specification is usually complete. In other cases, inductive programming is seen as a more general area where any declarative programming or representation language can be used and we may even have some degree of error in the examples, as in general machine learning, the more specific area of structure mining or the area of symbolic artificial intelligence. A distinctive feature is the number of examples or partial specification needed. Typically, inductive programming techniques can learn from just a few examples. The diversity of inductive programming usually comes from the applications and the languages that are used: apart from logic programming and functional programming, other programming paradigms and representation languages have been used or suggested in inductive programming, such as functional logic programming, constraint programming, probabilistic programming, abductive logic programming, modal logic, action languages, agent languages and many types of imperative languages. == History == The early works of Plotkin, and his "relative least general generalization (rlgg)", had an enormous impact in inductive logic programming. There were some encouraging results on learning recursive Prolog programs such as quicksort from examples together with suitable background knowledge, for example with GOLEM. However, after initial success, the community got disappointed by limited progress about the induction of recursive programs with ILP less and less focusing on recursive programs and leaning more and more towards a machine learning setting with applications in relational data mining and knowledge discovery. In parallel to work in ILP, Koza proposed genetic programming in the early 1990s as a generate-and-test based approach to learning programs. The idea of genetic programming was further developed into the inductive programming system ADATE and the systematic-search-based system MagicHaskeller. Here again, functional programs are learned from sets of positive examples together with an output evaluation (fitness) function which specifies the desired input/output behavior of the program to be learned. The early work in grammar induction (also known as grammatical inference) is related to inductive programming, as rewriting systems or logic programs can be used to represent production rules. In fact, early works in inductive inference considered grammar induction and Lisp program inference as basically the same problem. The results in terms of learnability were related to classical concepts, such as identification-in-the-limit, as introduced in the seminal work of Gold. More recently, the language learning problem was addressed by the inductive programming community. In the recent years, the classical approaches have been resumed and advanced with great success. Therefore, the synthesis problem has been reformulated on the background of constructor-based term rewriting systems taking into account modern techniques of functional programming, as well as moderate use of search-based strategies and usage of background knowledge as well as automatic invention of subprograms. Many new and successful applications have recently appeared beyond program synthesis, most especially in the area of data manipulation, programming by example and cognitive modelling (see below). Other ideas have also been explored with the common characteristic of using declarative languages for the representation of hypotheses. For instance, the use of higher-order features, schemes or structured distances have been advocated for a better handling of recursive data types and structures; abstraction has also been explored as a more powerful approach to cumulative learning and function invention. One powerful paradigm that has been recently used for the representation of hypotheses in inductive programming (generally in the form of generative models) is probabilistic programming (and related paradigms, such as stochastic logic programs and Bayesian logic programming). == Application areas == The first workshop on Approaches and Applications of Inductive Programming (AAIP) Archived 2016-03-03 at the Wayback Machine held in conjunction with ICML 2005 identified all applications where "learning of programs or recursive rules are called for, [...] first in the domain of software engineering where structural learning, software assistants and software agents can help to relieve programmers from routine tasks, give programming support for end users, or support of novice programmers and programming tutor systems. Further areas of application are language learning, learning recursive control rules for AI-planning, learning recursive concepts in web-mining or for data-format transformations". Since then, these and many other areas have shown to be successful application niches for inductive programming, such as end-user programming, the related areas of programming by example and programming by demonstration, and intelligent tutoring systems. Other areas where inductive inference has been recently applied are knowledge acquisition, artificial general intelligence, reinforcement learning and theory evaluation, and cognitive science in general. There may also be prospective applications in intelligent agents, games, robotics, personalisation, ambient intelligence and human interfaces.
Content-based image retrieval
Content-based image retrieval, also known as query by image content (QBIC) and content-based visual information retrieval (CBVIR), is the application of computer vision techniques to the image retrieval problem, that is, the problem of searching for digital images in large databases (see this survey for a scientific overview of the CBIR field). Content-based image retrieval is opposed to traditional concept-based approaches (see Concept-based image indexing). "Content-based" means that the search analyzes the contents of the image rather than the metadata such as keywords, tags, or descriptions associated with the image. The term "content" in this context might refer to colors, shapes, textures, or any other information that can be derived from the image itself. CBIR is desirable because searches that rely purely on metadata are dependent on annotation quality and completeness. == Comparison with metadata searching == An image meta search requires humans to have manually annotated images by entering keywords or metadata in a large database, which can be time-consuming and may not capture the keywords desired to describe the image. The evaluation of the effectiveness of keyword image search is subjective and has not been well-defined. In the same regard, CBIR systems have similar challenges in defining success. "Keywords also limit the scope of queries to the set of predetermined criteria." and, "having been set up" are less reliable than using the content itself. == History == The term "content-based image retrieval" seems to have originated in 1992 when it was used by Japanese Electrotechnical Laboratory engineer Toshikazu Kato to describe experiments into automatic retrieval of images from a database, based on the colors and shapes present. Since then, the term has been used to describe the process of retrieving desired images from a large collection on the basis of syntactical image features. The techniques, tools, and algorithms that are used originate from fields such as statistics, pattern recognition, signal processing, and computer vision. === QBIC - Query By Image Content === The earliest commercial CBIR system was developed by IBM and was called QBIC (Query By Image Content). Recent network- and graph-based approaches have presented a simple and attractive alternative to existing methods. While the storing of multiple images as part of a single entity preceded the term BLOB (Binary Large OBject), the ability to fully search by content, rather than by description, had to await IBM's QBIC. === VisualRank === == Technical progress == The interest in CBIR has grown because of the limitations inherent in metadata-based systems, as well as the large range of possible uses for efficient image retrieval. Textual information about images can be easily searched using existing technology, but this requires humans to manually describe each image in the database. This can be impractical for very large databases or for images that are generated automatically, e.g. those from surveillance cameras. It is also possible to miss images that use different synonyms in their descriptions. Systems based on categorizing images in semantic classes like "cat" as a subclass of "animal" can avoid the miscategorization problem, but will require more effort by a user to find images that might be "cats", but are only classified as an "animal". Many standards have been developed to categorize images, but all still face scaling and miscategorization issues. Initial CBIR systems were developed to search databases based on image color, texture, and shape properties. After these systems were developed, the need for user-friendly interfaces became apparent. Therefore, efforts in the CBIR field started to include human-centered design that tried to meet the needs of the user performing the search. This typically means inclusion of: query methods that may allow descriptive semantics, queries that may involve user feedback, systems that may include machine learning, and systems that may understand user satisfaction levels. == Techniques == Many CBIR systems have been developed, but as of 2006, the problem of retrieving images on the basis of their pixel content remains largely unsolved. Different query techniques and implementations of CBIR make use of different types of user queries. === Query By Example === QBE (Query By Example) is a query technique that involves providing the CBIR system with an example image that it will then base its search upon. The underlying search algorithms may vary depending on the application, but result images should all share common elements with the provided example. Options for providing example images to the system include: A preexisting image may be supplied by the user or chosen from a random set. The user draws a rough approximation of the image they are looking for, for example with blobs of color or general shapes. This query technique removes the difficulties that can arise when trying to describe images with words. === Semantic retrieval === Semantic retrieval starts with a user making a request like "find pictures of Abraham Lincoln". This type of open-ended task is very difficult for computers to perform - Lincoln may not always be facing the camera or in the same pose. Many CBIR systems therefore generally make use of lower-level features like texture, color, and shape. These features are either used in combination with interfaces that allow easier input of the criteria or with databases that have already been trained to match features (such as faces, fingerprints, or shape matching). However, in general, image retrieval requires human feedback in order to identify higher-level concepts. === Relevance feedback (human interaction) === Combining CBIR search techniques available with the wide range of potential users and their intent can be a difficult task. An aspect of making CBIR successful relies entirely on the ability to understand the user intent. CBIR systems can make use of relevance feedback, where the user progressively refines the search results by marking images in the results as "relevant", "not relevant", or "neutral" to the search query, then repeating the search with the new information. Examples of this type of interface have been developed. === Iterative/machine learning === Machine learning and application of iterative techniques are becoming more common in CBIR. === Other query methods === Other query methods include browsing for example images, navigating customized/hierarchical categories, querying by image region (rather than the entire image), querying by multiple example images, querying by visual sketch, querying by direct specification of image features, and multimodal queries (e.g. combining touch, voice, etc.) == Content comparison using image distance measures == The most common method for comparing two images in content-based image retrieval (typically an example image and an image from the database) is using an image distance measure. An image distance measure compares the similarity of two images in various dimensions such as color, texture, shape, and others. For example, a distance of 0 signifies an exact match with the query, with respect to the dimensions that were considered. As one may intuitively gather, a value greater than 0 indicates various degrees of similarities between the images. Search results then can be sorted based on their distance to the queried image. Many measures of image distance (Similarity Models) have been developed. === Color === Computing distance measures based on color similarity is achieved by computing a color histogram for each image that identifies the proportion of pixels within an image holding specific values. Examining images based on the colors they contain is one of the most widely used techniques because it can be completed without regard to image size or orientation. However, research has also attempted to segment color proportion by region and by spatial relationship among several color regions. === Texture === Texture measures look for visual patterns in images and how they are spatially defined. Textures are represented by texels which are then placed into a number of sets, depending on how many textures are detected in the image. These sets not only define the texture, but also where in the image the texture is located. Texture is a difficult concept to represent. The identification of specific textures in an image is achieved primarily by modeling texture as a two-dimensional gray level variation. The relative brightness of pairs of pixels is computed such that degree of contrast, regularity, coarseness and directionality may be estimated. The problem is in identifying patterns of co-pixel variation and associating them with particular classes of textures such as silky, or rough. Other methods of classifying textures include: Co-occurrence matrix Laws texture energy Wavelet transform Orthogonal transforms (discrete Chebyshev moments) =
DialogOS
DialogOS is a graphical programming environment to design computer system which can converse through voice with the user. Dialogs are clicked together in a Flowchart. DialogOS includes bindings to control Lego Mindstorms robots by voice and has bindings to SQL databases, as well as a generic plugin architecture to integrate with other types of backends. DialogOS is used in computer science courses in schools and universities to teach programming and to introduce beginners in the basic principles of human/computer interaction and dialog design. It has also been used in research systems. DialogOS was initially developed commercially by CLT Sprachtechnologie GmbH until its liquidation in 2017. The rights were then acquired by Saarland University and the software was released as open-source. == Bindings to Lego Mindstorms NXT == DialogOS can control the LEGO Mindstorms NXT Series. It uses sensor-nodes to obtain values for the following sensors: noise sensor ultrasonic sensor touch sensor luminosity sensor
2024 Abu Dhabi Autonomous Racing League
On 27 April 2024, the inaugural race of the Abu Dhabi Autonomous Racing League was held at the Yas Marina Circuit in Abu Dhabi. The race, originally scheduled to last eight laps, was ultimately shortened to six laps due to various complications, including subpar performance. It involved four self-driving race cars, only two of which – German cars Hailey and Constructor AI – finished the race; the other two did not finish. == Background == === Abu Dhabi Autonomous Racing League (A2RL) === The A2RL is an autonomous racing championship based in Abu Dhabi and organized by ASPIRE, part of the Advanced Technology Research Council. It is one of two active autonomous car racing championships, the second being the US-based Indy Autonomous Challenge. Unlike the IAC, which primarily focuses on time trials, simulated races, and challenges for teams, the A2RL's car races are closer to a standard grand prix formula race format. Both use Dallara-supplied racecars; the IAC uses the AV-24 chassis derived from Indy NXT's IL-15, while the A2RL chassis is designated EAV-24 and is derived from the SF-23 chassis used in Japanese Super Formula races. === Entrants === In total, eight teams were part of the A2RL in 2024, but only four would compete in the race proper. The list of teams in 2024 is: Fly Eagle (China/UAE) Code19 Racing (United States) Constructor University (Germany) Kinetiz (Singapore/UAE) Humda Lab (Hungary) PoliMove (Italy) Unimore (Italy) Technical University of Munich (Germany) Most teams come from universities and many, such as PoliMove and TUM, already have experience with autonomous racing, primarily from competing in the IAC. All teams had two months to code and test their AIs. Unlike most international open-wheel racing tournaments, such as Formula 1 or Formula E, no free practice sessions were undertaken. === TII Pre-race demonstration === Prior to the race itself, a mock 1v1 duel between former F1 driver Danill Kvyat and a self-driving car from the non-competing TII Racing team took place; the autonomous car was green and had number 01, while Kvyat's car was red and had number 00. Kvyat spent most of the duel in the pits. Kvyat himself said: "I'm not racing autonomous cars here. It won't be a flat-out race". == Qualifying == === Qualifying report === As only four of the eight entrants would compete in the main event, qualifying time trials were held to determine the four main race competitors, as well as their positions in the grid. Only the cars with the four best lap times over three time trial sessions held on Friday and Saturday would qualify. Multiple errors and setbacks occurred during qualifying. In the first session, Maveric AI, Code19's car, left the track and stopped just after turn 14 due to connectivity issues. Fly Eagle's car, Feiying, had multiple upsets; at one point, Feiying ran into localization issues and began swerving left and right before stopping just before turn 10. Later, Feiying swerved again and nearly hit the wall at the back straight, near the support pits, due to further localization issues. Sparkz, the Kinetiz team's car, swerved and crashed into the wall near yacht berths 51-56 after turn 11, damaging the front right wheel's axle and partially detaching the forward wings. Sparkz would be the only car to not have a set time at the end of the time trials. PoliMove car Eva braked hard without warning at the straight, the LED status indicator turning off, suggesting the AI computer had a system crash or shut itself down. After the sun went down, during the second session, Hailey, the car from the TUM team, went off-track after turn 9 and stopped, its status indicator flashing red, meaning Hailey's AI disengaged itself. Eva had further issues, once again braking hard and spinning out into turn 1. Later, the same thing happened to Feiying; it later swerved left and right and stopped due to further localization issues. The morning after, during the third and final session, Hailey went off-track after turn 5, and were unable to regain the pole position. === Qualifying classification === == Attack/Defend challenge == === Attack/Defend challenge report === In this part of the event, cars would be put on a series of 1v1 duels to see how well they could defend their position or attack to gain one higher. During one such duel, an incident occurred where Hailey rear-ended Eva, sending both off the track and prematurely ending the duel. The challenge was otherwise uneventful. === Attack/Defend challenge results === == Main race == === Race report === Eventually, at around 20:30 Gulf Standard Time on the night of 27 April, the main event (termed the "Grand Final" on-stream) would begin. The starting order was Eva first, Gianna second, Hailey third, and Constructor AI last. The race began with a rolling start. As a safety measure, the first two laps were conducted under virtual safety car (VSC) to make sure the cars stayed together, making them de facto formation laps, even if they counted towards race distance. However, Hailey ended up stopping at the final turn and strayed too far from the cars ahead, and as a result, the VSC conditions were extended for another lap. According to the livestream's on-screen graphics, Hailey was upwards of one minute and 22.3 seconds behind Gianna after the former started moving again. On lap 4, halfway through the planned race, and with Hailey more than 30 seconds behind Gianna, the VSC was lifted, and the green flag finally dropped. At first, the two Italian cars were leading the pack, Eva was the race leader with Gianna 3.2 seconds behind, however, as it entered the chicane, Eva hit the brakes and spun out, with Gianna briefly stopping as it passed Eva. Eva's spin automatically triggered a full-course yellow flag. Normally, under yellow flag conditions, overtaking is not permitted, but with Eva stopped and being moved off the track, it was theoretically permitted to overtake Eva. However, presumably due to an oversight in the AI's code, the cars assumed overtaking Eva, despite being off the track, was not permitted. As a result, both Gianna and Constructor AI stopped as they did not want to overtake Eva due to the yellow flag, with Hailey following suit as it approached. Constructor AI's status indicator was solid red, suggesting the AI had disengaged; however, Gianna's status indicator remained solid purple, showing the AI was still in control. Eva's status indicator was also solid purple, but was soon flashing green, suggesting the AI had disengaged but was ready to take control again. With all cars stalled, and Eva being off the track, the race was effectively red-flagged and suspended. Hailey, Gianna, and Constructor AI drove themselves back to their team's pits; Eva did not, it was towed to the main pits on a flatbed truck. Constructor was the first to arrive at the pits, followed by Gianna and Hailey, in that order. This incident, combined with loss of internet connection, led to Eva retiring - it did not finish the race. Eventually, it was decided to resume the race. With Eva retired, the restart order was Gianna first, Hailey second, and Constructor AI third. The race was also shortened - from eight laps to six. With lap 5 under full-course yellow, this meant all three remaining teams would effectively restart the race on the sixth and final lap. The trio left the pits at 22:25 Gulf Standard Time, and the race resumed two minutes later. At first, Gianna was winning with Hailey 2.6 seconds behind, but then Gianna stopped on turn 5, giving Hailey the lead. Constructor AI also overtook Gianna, but not without briefly stopping. Gianna remained stopped, its status indicator solid red - it did not finish either. With both Italian teams out of the picture, Hailey finished first and won A2RL 2024, with Constructor AI finishing second, 27.2 seconds behind. === Final race classification ===
Site reliability engineering
Site reliability engineering (SRE) is a discipline in the field of software engineering and IT infrastructure support that monitors and improves the availability and performance of deployed software systems and large software services (which are expected to deliver reliable response times across events such as new software deployments, hardware failures, and cybersecurity attacks). There is typically a focus on automation and an infrastructure as code methodology. SRE uses elements of software engineering, IT infrastructure, web development, and operations to assist with reliability. It is similar to DevOps as they both aim to improve the reliability and availability of deployed software systems. == History == Site Reliability Engineering originated at Google with Benjamin Treynor Sloss, who founded SRE team in 2003. The concept expanded within the software development industry, leading various companies to employ site reliability engineers. By March 2016, Google had more than 1,000 site reliability engineers on staff. Dedicated SRE teams are common at larger web development companies. In middle-sized and smaller companies, DevOps teams sometimes perform SRE, as well. Organizations that have adopted the concept include Airbnb, Dropbox, IBM, LinkedIn, Netflix, and Wikimedia. == Definition == Site reliability engineers (SREs) are responsible for a combination of system availability, latency, performance, efficiency, change management, monitoring, emergency response, and capacity planning. SREs often have backgrounds in software engineering, systems engineering, and/or system administration. The focuses of SRE include automation, system design, and improvements to system resilience. SRE is considered a specific implementation of DevOps; focusing specifically on building reliable systems, whereas DevOps covers a broader scope of operations. Despite having different focuses, some companies have rebranded their operations teams to SRE teams. == Principles and practices == Common definitions of the practices include (but are not limited to): Automation of repetitive tasks for cost-effectiveness. Defining reliability goals to prevent endless effort. Design of systems with a goal to reduce risks to availability, latency, and efficiency. Observability, the ability to ask arbitrary questions about a system without having to know ahead of time what to ask. Common definitions of the principles include (but are not limited to): Toil management, the implementation of the first principle outlined above. Defining and measuring reliability goals—SLIs, SLOs, and error budgets. Non-Abstract Large Scale Systems Design (NALSD) with a focus on reliability. Designing for and implementing observability. Defining, testing, and running an incident management process. Capacity planning. Change and release management, including CI/CD. Chaos engineering. == Deployment == SRE teams collaborate with other departments within organizations to guide the implementation of the mentioned principles. Below is an overview of common practices: === Kitchen Sink === Kitchen Sink refers to the expansive and often unbounded scope of services and workflows that SRE teams oversee. Unlike traditional roles with clearly defined boundaries, SREs are tasked with various responsibilities, including system performance optimization, incident management, and automation. This approach allows SREs to address multiple challenges, ensuring that systems run efficiently and evolve in response to changing demands and complexities. === Infrastructure === Infrastructure SRE teams focus on maintaining and improving the reliability of systems that support other teams' workflows. While they sometimes collaborate with platform engineering teams, their primary responsibility is ensuring up-time, performance, and efficiency. Platform teams, on the other hand, primarily develop the software and systems used across the organization. While reliability is a goal for both, platform teams prioritize creating and maintaining the tools and services used by internal stakeholders, whereas Infrastructure SRE teams are tasked with ensuring those systems run smoothly and meet reliability standards. === Tools === SRE teams utilize a variety of tools with the aim of measuring, maintaining, and enhancing system reliability. These tools play a role in monitoring performance, identifying issues, and facilitating proactive maintenance. For instance, Nagios Core is commonly employed for system monitoring and alerting, while Prometheus (software) is frequently used for collecting and querying metrics in cloud-native environments. === Product or Application === SRE teams dedicated to specific products or applications are common in large organizations. These teams are responsible for ensuring the reliability, scalability, and performance of key services. In larger companies, it's typical to have multiple SRE teams, each focusing on different products or applications, ensuring that each area receives specialized attention to meet performance and availability targets. === Embedded === In an embedded model, individual SREs or small SRE pairs are integrated within software engineering teams. These SREs collaborate with developers, applying core SRE principles—such as automation, monitoring, and incident response—directly to the software development lifecycle. This approach aims to enhance reliability, performance, and collaboration between SREs and developers. === Consulting === Consulting SRE teams specialize in advising organizations on the implementation of SRE principles and practices. Typically composed of seasoned SREs with a history across various implementations, these teams provide insights and guidance for specific organizational needs. When working directly with clients, these SREs are often referred to as 'Customer Reliability Engineers.' In large organizations that have adopted SRE, a hybrid model is common. This model includes various implementations, such as multiple Product/Application SRE teams dedicated to addressing the specific reliability needs of different products. An Infrastructure SRE team may collaborate with a Platform engineering group to achieve shared reliability goals for a unified platform that supports all products and applications. == Industry == Since 2014, the USENIX organization has hosted the annual SREcon conference, bringing together site reliability engineers from various industries. This conference is a platform for professionals to share knowledge, explore effective practices, and discuss trends in site reliability engineering.
AI Snake Oil
AI Snake Oil: What Artificial Intelligence Can Do, What It Can't, and How to Tell the Difference is a 2024 non-fiction book written by scholars Arvind Narayanan and Sayash Kapoor. It is a critique of the tech industry's overly inflated promises and capabilities of artificial intelligence (AI) as well as a debunking of the flawed science fueling AI hype while attempting to outline both the potential positives and negatives that come with different modes of the technology. == Contents == === Publication === The book was published in September 2024 by the Princeton University Press. AI Snake Oil consists of 360 pages and features eight chapters, and sections for acknowledgements, references, and an index. An updated edition with a new preface and epilogue by the authors was published in September 2025. The authors use the term "AI snake oil" derived from the U.S. idiom for a fraudulent remedy, to describe overhyped AI systems. === Chapter one: Introduction === Narayanan and Kapoor argue that many individuals do not yet have the literacy to detect functioning aspects of AI compared to potential snake oil, which they identify as "AI that does not and cannot work as advertised". Some of the major examples utilized by the authors include Allstate's 2013 use of predictive AI, as well as the concern surrounding actors and AI attempting to replicate or use their likeness. Important discussions regarding discrimination are brought up and explored in the first chapter, including the false arrests of six Black individuals due to errors with AI facial recognition tools. The chapter concludes with a comparison to the Industrial Revolution, where Narayanan and Kapoor highlight the extensive human labour that is necessary for artificial intelligence technologies to function. === Chapter two: How Predictive AI Goes Wrong === Chapter two focuses on predictive artificial intelligence, and criticizes the overestimation of the capabilities of the technology. === Chapter three: Why Can't AI Predict the Future? === Chapter three works to inform the reader about the history of early computational prediction attempts, with examples from companies like Simulatics. === Chapter four: The Long Road to Generative AI === The fourth chapter goes in more in-depth in explorations of generative AI. Generative AI software examples include ChatGPT, Midjourney, and DALL-E. The section begins with a positive example of generative AI. As the chapter progresses, the authors begin to provide examples of harm produced by generative AI, including the suicide of a Belgian man after connecting with Chai, a generative chatbot. Issues of deepfakes and preservation of artistic property are also discussed. The use of generative AI to create non-consensual pornographic deepfake content is discussed in relation to female celebrities. === Chapter five: Is Advanced AI an Existential Threat? === The fifth chapter draws attention the AGI, or Artificial General Intelligence. The authors describe AGI as "AI that can perform most or economically relevant tasks as effectively as any human". They summarize that many contributors to the field of artificial intelligence believe AGI to be an impending threat that demands attention. However, they argue that the perceived threat of AGI would only exist if the technology continually functioned reliably. In order to better illustrate the hype surrounding AGI, Narayanan and Kapoor use the Ladder of Generality, which is described as a visual tool in which "each rung represents a way of computing that is more flexible, and more general, than the previous one". They note that we are not yet aware of the next rungs on the ladder, or if the ladder will eventually result in a dead end. The rungs that have been identified so far are as follows: (0, or floor) special purpose hardware, (1) programmable computers, (2) stored program computers, (3) machine learning, (4) deep learning, (5) pretrained models, and, finally, (6) instruction-tuned models. The potential for future rungs and what those rungs might be are currently undetermined. The chapter also discusses the ELIZA effect, which Lawrence Switzky discusses in his article "ELIZA Effects". Switzky attributes the coined term ELIZA Effect to Sherry Turke, who defined it as "our more general tendency to treat responsive computer programs as more intelligent than they really are". === Chapter six: Why Can't AI Fix Social Media? === The sixth chapter focuses on content moderation, why it is important, and how it has been and could be affected by artificial automation. The first issue raised in regard to AI-driven content moderation is the inability for computers and machines to understand context and nuance, resulting in potential for discriminatory moderation and shadow banning. While they note that there are issues with automating content moderation, Narayanan and Kapoor also highlight the psychological impact on human content moderators and their labour. They indicate the hidden labour behind moderation, which is often outsourced to less developed countries, where labourers sort through potentially traumatizing content for pay. However, the discussion focuses more heavily on why automated moderation can be problematic, including discriminatory algorithms and lack of nuance. To balance their argument, issues of discrimination and bias are also discussed in relation the human content moderators. To automate moderation, there are two types of AI used, which are fingerprint matching and machine learning. === Chapter seven: Why Do Myths about AI Persist? === The seventh chapter outlines possible factors that contribute to hype surrounding AI. Narayanan and Kapoor explain how companies often promote their new AI models without properly disclosing how the model works, and what it is learning from. They attribute hype to several different groups, including journalists, researchers, and companies. They explain the impact of companies and the misplaced hype that they spread can be attributed to greed and a desire to grow corporate funds. For journalists, one of the stated sources of hype, they argue that news media has a tendency to prioritize financial incentives over validity and quality of writing. As well, Narayanan and Kapoor point out the emergence of company statement regurgitation in news media, leading to clickbait. Hype from researchers is potentially linked to lack of reproducibility in studies as well as leakage, which occurs when AI models are tested on their training data. === Chapter eight: Where do we go from here? === The final chapter, chapter eight, turns its attention to the future. The authors express their ideas and predictions for how the technology will evolve and be utilized in the upcoming years. == Authors == Author Narayanan is a computer science professor at Princeton University. Kapoor is a doctoral candidate at the same university, and both scholars are located at the Center for Information Technology at Princeton. In 2023, Narayanan and Kapoor appeared on the TIME100 Artificial Intelligence list, which features influential figures in the field. == Reception == Nature, a science and technology peer-reviewed journal, released an article highlighting the top "10 essential reads from the past year", listing Arvind Narayanan and Sayash Kapoor's AI Snake Oil. The article states the that text is "one of the best on this controversial subject". Elizabeth Quill, in her review of the text in Science News, writes that the authors "squarely achieve their stated goal: to empower people to distinguish AI that works well from AI snake oil". Joshua Rothman of The New Yorker writes that "compared with many technologists, Narayanan, Kapoor, and Vallor [Shannon Vallor, University of Edinburgh], are deeply skeptical about today's A.I. technology and what it can achieve. Perhaps they shouldn't be". Rothman argues, following an interview with prominent computer scientist Geoffrey Hinton of University of Toronto, that the potential for AI to replicate complexity is already here and continues to be heavily funded, enhancing the prospective capabilities of the technology. However, he does praise the author's ability to address questions regarding the existential human experience. Alexya Martinez discusses the text in a book review for Journalism and Mass Communication Quarterly, critiquing AI Snake Oil for its extensive focus on the West. Martinez writes that Narayanan and Kapoor "do not fully explore how AI impacts other countries", and suggests more focus on countries outside of the United States to enhance their argument.