Julia Hirschberg is an American computer scientist noted for her research on computational linguistics and natural language processing. She received her first PhD in history from the University of Michigan and the second from the University of Pennsylvania in computer science doing research in Natural Language Processing. She worked at Bell Labs and AT&T Bell Labs from 1985 to 2002 and from 2002 at Columbia University where she is currently the Percy K. and Vida L. W. Hudson Professor of Computer Science. == Biography == Julia Linn Bell Hirschberg received her first Ph.D. degree in history (16th-century Mexico) from University of Michigan in 1976. She served on the History faculty of Smith College from 1974 to 1982. She subsequently shifted to Computer Science studies, receiving her M.S. in Computer and Information Science from University of Pennsylvania in 1982 and a Ph.D. in Computer and Information Science from University of Pennsylvania in 1985. Upon graduation from University of Pennsylvania in 1985, Hirschberg joined AT&T Bell Labs as a Member of Technical staff in the Linguistics Research Department, where she worked on improving prosody assignment for Text-to-Speech Synthesis (TTS) in the Bell Labs TTS system. She was promoted to Department Head in 1994 when she created a new Human Computer Interface Research Lab. She and her department remained at Bell Labs until 1996 when they moved to AT&T Labs Research as part of a corporate reorganization. In 2002, she joined the Columbia University faculty as a professor in the Department of Computer Science. She served as Chair of the Computer Science Department from 2012 to 2018. She still leads classes at Columbia in speech and natural language research and supervises PhD students and a large number of research project students. == Research == Hirschberg's research has included prosody, discourse structure, conversational implicature, text-to-speech synthesis, speech summarization, spoken dialogue systems, emotional speech, deceptive speech, charismatic speech, entrainment, empathetic speech and code-switching. Hirschberg was among the first to combine Natural Language Processing (NLP) approaches to discourse and dialogue with speech research. She pioneered techniques in text analysis for prosody assignment in Text-to-Speech synthesis at Bell laboratories in the 1980s and 1990s, developing corpus-based statistical models based upon syntactic and discourse information which are in general use today in TTS systems. With Janet Pierrehumbert, she developed a theoretical model of intonational meaning. She was a leader in the development of the ToBI conventions for intonational description, which have been extended to numerous languages and which today are the most widely used standard for intonational annotation. Hirschberg has been a pioneer together with Gregory Ward in much experimental work on intonational sources of language meaning and how these interact with pragmatic phenomena, particularly on the meaning of accent (intonational prominent) items and the meaning of intonational contours. She also has innovated in numerous other areas involving prosody and meaning, including the role of grammatical function and surface position in pitch accent location, the use of prosody in disambiguating cue phrases (discourse markers) with Diane Litman, the role of prosody in disambiguation in English, Italian, and Spanish with Cinzia Avesani and Pilar Prieto, and the automatic identification of speech recognition errors using prosodic information, At AT&T Labs she worked with Fernando Pereira, Steve Whittaker, and others on speech search and developing new interfaces for speech navigation. At Columbia, she and her students have continued and extended research on spoken dialogue systems (automatically detecting speech recognition errors and inappropriate system queries, modeling turn-taking behavior, dialogue entrainment, modeling and generating clarification dialogues); on the automatic classification of trust, charisma, deception and emotion from speech; on speech summarization; prosody translation, hedging behavior in text and speech, text-to-speech synthesis, and speech search in low resource languages. She also holds several patents in TTS and in speech search. Corpora she and collaborators have collected include the Boston Directions Corpus, the Columbia SRI Colorado Deception Corpus, and the Columbia Games Corpus. She has served on numerous technical boards and editorial committees. She has served as a member of the Computing Research Association's (CRA) Board of Directors and as co-chair of CRA-W. She is also noted for her leadership in broadening participation in computing. == Awards == Hirschberg's notable honors and awards include: Elected as a member of the National Academy of Artificial Intelligence Academy of Sciences and recipient of the NAAI Artificial Intelligence Exploration Award, 2025 Elected as a Fellow of Asia-Pacific Artificial Intelligence Association (AAIA), 2024. 2020 ISCA Special Service Medal Honorary Doctorate (eredoctoraat) from Tilburg University, Netherlands, 2018. American Academy of Arts and Sciences, 2018. IEEE Fellow, 2017 National Academy of Engineering, 2017 ACM Fellow in 2015 Elected member, American Philosophical Society, 2014. Honorary member, Association for Laboratory Phonology, 2014. Association for Computational Linguistics (ACL) (Founding) Fellow, 2011. International Speech Communication Association (ISCA) Medal for Scientific Achievement, 2011. IEEE James L. Flanagan Speech and Audio Processing Award, 2011. Honorary Doctorate (Hedersdoktorer), KTH (Royal Institute of Technology) Stockholm, Sweden, 2007. AAAI Fellow, 1994. == Publications == A social history of Puebla de Los Ángeles, 1531-60, 1976 Empirical studies on the disambiguation of cue phrases, 1991 Prosody and conversation, 1998 Most recent publications and other information, https://www.cs.columbia.edu/speech/.
Automatic summarization
Automatic summarization is the process of shortening a set of data computationally, to create a subset (a summary) that represents the most important or relevant information within the original content. Artificial intelligence (AI) algorithms are commonly developed and employed to achieve this, specialized for different types of data. Text summarization is usually implemented by natural language processing methods, designed to locate the most informative sentences in a given document. On the other hand, visual content can be summarized using computer vision algorithms. Image summarization is the subject of ongoing research; existing approaches typically attempt to display the most representative images from a given image collection, or generate a video that only includes the most important content from the entire collection. Video summarization algorithms identify and extract from the original video content the most important frames (key-frames), and/or the most important video segments (key-shots), normally in a temporally ordered fashion. Video summaries simply retain a carefully selected subset of the original video frames and, therefore, are not identical to the output of video synopsis algorithms, where new video frames are being synthesized based on the original video content. == Commercial products == In 2022 Google Docs released an automatic summarization feature. == Approaches == There are two general approaches to automatic summarization: extraction and abstraction. === Extraction-based summarization === Here, content is extracted from the original data, but the extracted content is not modified in any way. Examples of extracted content include key-phrases that can be used to "tag" or index a text document, or key sentences (including headings) that collectively comprise an abstract, and representative images or video segments, as stated above. For text, extraction is analogous to the process of skimming, where the summary (if available), headings and subheadings, figures, the first and last paragraphs of a section, and optionally the first and last sentences in a paragraph are read before one chooses to read the entire document in detail. Other examples of extraction that include key sequences of text in terms of clinical relevance (including patient/problem, intervention, and outcome). === Abstractive-based summarization === Abstractive summarization methods generate new text that did not exist in the original text. This has been applied mainly for text. Abstractive methods build an internal semantic representation of the original content (often called a language model), and then use this representation to create a summary that is closer to what a human might express. Abstraction may transform the extracted content by paraphrasing sections of the source document, to condense a text more strongly than extraction. Such transformation, however, is computationally much more challenging than extraction, involving both natural language processing and often a deep understanding of the domain of the original text in cases where the original document relates to a special field of knowledge. "Paraphrasing" is even more difficult to apply to images and videos, which is why most summarization systems are extractive. === Aided summarization === Approaches aimed at higher summarization quality rely on combined software and human effort. In Machine Aided Human Summarization, extractive techniques highlight candidate passages for inclusion (to which the human adds or removes text). In Human Aided Machine Summarization, a human post-processes software output, in the same way that one edits the output of automatic translation by Google Translate. == Applications and systems for summarization == There are broadly two types of extractive summarization tasks depending on what the summarization program focuses on. The first is generic summarization, which focuses on obtaining a generic summary or abstract of the collection (whether documents, or sets of images, or videos, news stories etc.). The second is query relevant summarization, sometimes called query-based summarization, which summarizes objects specific to a query. Summarization systems are able to create both query relevant text summaries and generic machine-generated summaries depending on what the user needs. An example of a summarization problem is document summarization, which attempts to automatically produce an abstract from a given document. Sometimes one might be interested in generating a summary from a single source document, while others can use multiple source documents (for example, a cluster of articles on the same topic). This problem is called multi-document summarization. A related application is summarizing news articles. Imagine a system, which automatically pulls together news articles on a given topic (from the web), and concisely represents the latest news as a summary. Image collection summarization is another application example of automatic summarization. It consists in selecting a representative set of images from a larger set of images. A summary in this context is useful to show the most representative images of results in an image collection exploration system. Video summarization is a related domain, where the system automatically creates a trailer of a long video. This also has applications in consumer or personal videos, where one might want to skip the boring or repetitive actions. Similarly, in surveillance videos, one would want to extract important and suspicious activity, while ignoring all the boring and redundant frames captured. At a very high level, summarization algorithms try to find subsets of objects (like set of sentences, or a set of images), which cover information of the entire set. This is also called the core-set. These algorithms model notions like diversity, coverage, information and representativeness of the summary. Query based summarization techniques, additionally model for relevance of the summary with the query. Some techniques and algorithms which naturally model summarization problems are TextRank and PageRank, Submodular set function, Determinantal point process, maximal marginal relevance (MMR) etc. === Keyphrase extraction === The task is the following. You are given a piece of text, such as a journal article, and you must produce a list of keywords or key[phrase]s that capture the primary topics discussed in the text. In the case of research articles, many authors provide manually assigned keywords, but most text lacks pre-existing keyphrases. For example, news articles rarely have keyphrases attached, but it would be useful to be able to automatically do so for a number of applications discussed below. Consider the example text from a news article: "The Army Corps of Engineers, rushing to meet President Bush's promise to protect New Orleans by the start of the 2006 hurricane season, installed defective flood-control pumps last year despite warnings from its own expert that the equipment would fail during a storm, according to documents obtained by The Associated Press". A keyphrase extractor might select "Army Corps of Engineers", "President Bush", "New Orleans", and "defective flood-control pumps" as keyphrases. These are pulled directly from the text. In contrast, an abstractive keyphrase system would somehow internalize the content and generate keyphrases that do not appear in the text, but more closely resemble what a human might produce, such as "political negligence" or "inadequate protection from floods". Abstraction requires a deep understanding of the text, which makes it difficult for a computer system. Keyphrases have many applications. They can enable document browsing by providing a short summary, improve information retrieval (if documents have keyphrases assigned, a user could search by keyphrase to produce more reliable hits than a full-text search), and be employed in generating index entries for a large text corpus. Depending on the different literature and the definition of key terms, words or phrases, keyword extraction is a highly related theme. ==== Supervised learning approaches ==== Beginning with the work of Turney, many researchers have approached keyphrase extraction as a supervised machine learning problem. Given a document, we construct an example for each unigram, bigram, and trigram found in the text (though other text units are also possible, as discussed below). We then compute various features describing each example (e.g., does the phrase begin with an upper-case letter?). We assume there are known keyphrases available for a set of training documents. Using the known keyphrases, we can assign positive or negative labels to the examples. Then we learn a classifier that can discriminate between positive and negative examples as a function of the features. Some classifiers make a binary classification for a test example, while others assign a probability of being a keyphrase. For ins
Juergen Pirner
Juergen Pirner (born 1956) is the German creator of Jabberwock, a chatterbot that won the 2003 Loebner prize. Pirner created Jabberwock modelling the Jabberwocky from Lewis Carroll's poem of the same name. Initially, Jabberwock would just give rude or fantasy-related answers; but over the years, Pirner has programmed better responses into it. As of 2007 he has taught it 2.7 million responses. Pirner lives in Hamburg, Germany.
Simultaneous localization and mapping
Simultaneous localization and mapping (SLAM) is a process where a computer constructs or updates a map of an unknown environment while simultaneously keeping track of an entity's location within it. While this initially appears to be a chicken or the egg problem, there are several algorithms known to solve it in, at least approximately, tractable time for certain environments. Popular approximate solution methods include the particle filter, extended Kalman filter, covariance intersection, and GraphSLAM. SLAM algorithms are based on concepts in computational geometry and computer vision, and are used in robot navigation, robotic mapping and odometry for virtual reality or augmented reality. SLAM algorithms are tailored to the available resources and are not aimed at perfection but at operational compliance. Published approaches are employed in self-driving cars, unmanned aerial vehicles, autonomous underwater vehicles, planetary rovers, newer domestic robots and even inside the human body. == Mathematical description of the problem == Given a series of controls u t {\displaystyle u_{t}} and sensor observations o t {\displaystyle o_{t}} over discrete time steps t {\displaystyle t} , the SLAM problem is to compute an estimate of the agent's state x t {\displaystyle x_{t}} and a map of the environment m t {\displaystyle m_{t}} . All quantities are usually probabilistic, so the objective is to compute P ( m t + 1 , x t + 1 | o 1 : t + 1 , u 1 : t ) {\displaystyle P(m_{t+1},x_{t+1}|o_{1:t+1},u_{1:t})} Applying Bayes' rule gives a framework for sequentially updating the location posteriors, given a map and a transition function P ( x t | x t − 1 ) {\displaystyle P(x_{t}|x_{t-1})} , P ( x t | o 1 : t , u 1 : t , m t ) = ∑ m t − 1 P ( o t | x t , m t , u 1 : t ) ∑ x t − 1 P ( x t | x t − 1 ) P ( x t − 1 | m t , o 1 : t − 1 , u 1 : t ) / Z {\displaystyle P(x_{t}|o_{1:t},u_{1:t},m_{t})=\sum _{m_{t-1}}P(o_{t}|x_{t},m_{t},u_{1:t})\sum _{x_{t-1}}P(x_{t}|x_{t-1})P(x_{t-1}|m_{t},o_{1:t-1},u_{1:t})/Z} where Z {\displaystyle Z} is the normalization constant, which ensures all the probabilities sum up to 1. Similarly the map can be updated sequentially by P ( m t | x t , o 1 : t , u 1 : t ) = ∑ x t ∑ m t P ( m t | x t , m t − 1 , o t , u 1 : t ) P ( m t − 1 , x t | o 1 : t − 1 , m t − 1 , u 1 : t ) {\displaystyle P(m_{t}|x_{t},o_{1:t},u_{1:t})=\sum _{x_{t}}\sum _{m_{t}}P(m_{t}|x_{t},m_{t-1},o_{t},u_{1:t})P(m_{t-1},x_{t}|o_{1:t-1},m_{t-1},u_{1:t})} Like many inference problems, the solutions to inferring the two variables together can be found, to a local optimum solution, by alternating updates of the two beliefs in a form of an expectation–maximization algorithm. == Algorithms == Statistical techniques used to approximate the above equations include Kalman filters and particle filters (the algorithm behind Monte Carlo Localization). They provide an estimation of the posterior probability distribution for the pose of the robot and for the parameters of the map. Methods which conservatively approximate the above model using covariance intersection are able to avoid reliance on statistical independence assumptions to reduce algorithmic complexity for large-scale applications. Other approximation methods achieve improved computational efficiency by using simple bounded-region representations of uncertainty. Set-membership techniques are mainly based on interval constraint propagation. They provide a set which encloses the pose of the robot and a set approximation of the map. Bundle adjustment, and more generally maximum a posteriori estimation (MAP), is another popular technique for SLAM using image data, which jointly estimates poses and landmark positions, increasing map fidelity, and is used in commercialized SLAM systems such as Google's ARCore which replaces their prior augmented reality computing platform named Tango, formerly Project Tango. MAP estimators compute the most likely explanation of the robot poses and the map given the sensor data, rather than trying to estimate the entire posterior probability. New SLAM algorithms remain an active research area, and are often driven by differing requirements and assumptions about the types of maps, sensors and models as detailed below. Many SLAM systems can be viewed as combinations of choices from each of these aspects. === Mapping === Topological maps are a method of environment representation which capture the connectivity (i.e., topology) of the environment rather than creating a geometrically accurate map. Topological SLAM approaches have been used to enforce global consistency in metric SLAM algorithms. In contrast, grid maps use arrays (typically square or hexagonal) of discretized cells to represent a topological world, and make inferences about which cells are occupied. Typically the cells are assumed to be statistically independent to simplify computation. Under such assumption, P ( m t | x t , m t − 1 , o t ) {\displaystyle P(m_{t}|x_{t},m_{t-1},o_{t})} are set to 1 if the new map's cells are consistent with the observation o t {\displaystyle o_{t}} at location x t {\displaystyle x_{t}} and 0 if inconsistent. Modern self driving cars mostly simplify the mapping problem to almost nothing, by making extensive use of highly detailed map data collected in advance. This can include map annotations to the level of marking locations of individual white line segments and curbs on the road. Location-tagged visual data such as Google's StreetView may also be used as part of maps. Essentially such systems simplify the SLAM problem to a simpler localization only task, perhaps allowing for moving objects such as cars and people only to be updated in the map at runtime. === Sensing === SLAM will always use several different types of sensors, and the powers and limits of various sensor types have been a major driver of new algorithms. Statistical independence is the mandatory requirement to cope with metric bias and with noise in measurements. Different types of sensors give rise to different SLAM algorithms which assumptions are most appropriate to the sensors. At one extreme, laser scans or visual features provide details of many points within an area, sometimes rendering SLAM inference unnecessary because shapes in these point clouds can be easily and unambiguously aligned at each step via image registration. At the opposite extreme, tactile sensors are extremely sparse as they contain only information about points very close to the agent, so they require strong prior models to compensate in purely tactile SLAM. Most practical SLAM tasks fall somewhere between these visual and tactile extremes. Sensor models divide broadly into landmark-based and raw-data approaches. Landmarks are uniquely identifiable objects in the world which location can be estimated by a sensor, such as Wi-Fi access points or radio beacons. Raw-data approaches make no assumption that landmarks can be identified, and instead model P ( o t | x t ) {\displaystyle P(o_{t}|x_{t})} directly as a function of the location. Optical sensors may be one-dimensional (single beam) or 2D- (sweeping) laser rangefinders, 3D high definition light detection and ranging (lidar), 3D flash lidar, 2D or 3D sonar sensors, and one or more 2D cameras. Since the invention of local features, such as SIFT, there has been intense research into visual SLAM (VSLAM) using primarily visual (camera) sensors, because of the increasing ubiquity of cameras such as those in mobile devices. Follow up research includes. Both visual and lidar sensors are informative enough to allow for landmark extraction in many cases. Other recent forms of SLAM include tactile SLAM (sensing by local touch only), radar SLAM, acoustic SLAM, and Wi-Fi-SLAM (sensing by strengths of nearby Wi-Fi access points). Recent approaches apply quasi-optical wireless ranging for multi-lateration (real-time locating system (RTLS)) or multi-angulation in conjunction with SLAM as a tribute to erratic wireless measures. A kind of SLAM for human pedestrians uses a shoe mounted inertial measurement unit as the main sensor and relies on the fact that pedestrians are able to avoid walls to automatically build floor plans of buildings by an indoor positioning system. For some outdoor applications, the need for SLAM has been almost entirely removed due to high precision differential GPS sensors. From a SLAM perspective, these may be viewed as location sensors which likelihoods are so sharp that they completely dominate the inference. However, GPS sensors may occasionally decline or go down entirely, e.g. during times of military conflict, which are of particular interest to some robotics applications. === Kinematics modeling === The P ( x t | x t − 1 ) {\displaystyle P(x_{t}|x_{t-1})} term represents the kinematics of the model, which usually include information about action commands given to a robot. As a part of the model, the kinematics of the robot is included, to improve estimates of sensing under con
Sasha Stiles
Sasha Stiles (born 1980) is an American artist and poet. After discovering natural language processing, she created the 2021 poetry collection Technelegy through an eponymous AI model, before presenting the 2025–2026 installation A Living Poem at the Museum of Modern Art. In addition to artificial intelligence, binary code and non-fungible tokens have been important aspects of her work. == Biography == Stiles was born in 1980 in Pasadena, California, to documentary filmmaker parents whose work includes Cosmos: A Personal Voyage. She was interested in science fiction during her youth, particularly how they addressed human-machine collaboration and posthumanism. She graduated magna cum laude from Harvard University with a Bachelor of Arts in 2002 and she graduated with high honors from the University of Oxford with a Master of Studies in 2004. Originally, Stiles's poetry focused on technology. In 2017, she discovered natural language processing, piquing her interest in its ability to process thoughts and words comparably to its human counterparts. Despite lacking a technological background, she managed to channel people like Gwern Branwen, Ross Goodwin, and Allison Parrish as inspirations for her AI work, and in 2019, she started training an AI model named Technelegy. In 2021, Black Spring Press published her poetry collection Technelegy, where she combines AI-generated content produced by the titular AI model with her own traditionally-created work; the AI-generated content was produced by processing Stiles's own poetry onto GPT-2 and GPT-3. She and Technelegy later co-created A Living Poem, which ran at the Museum of Modern Art's Hyundai Card Digital Wall from September 2025 to March 2026. Stiles also has used non-fungible tokens as a platform for her poetry, having been inspired to go into blockchain by her experiences working with a metaverse exhibition curated by Jess Conatser. She has used Christie's and SuperRare to sell several of her poems as tokenized real-world assets, including Daughter of E.V.E. (Ex-Vivo Uterine Environment), a 2021 single-channel video using freeze-frame shots to hide poetry. In 2021, she co-founded TheVerseVerse (stylized as theVERSEverse), a non-fungible token gallery specializing in poetry. She later created Four Core Texts: Humanifesto and Other Poems, involving four NFT videos of poetry written in looping handwriting and powered by Technelegy. Stiles uses binary code as an inspiration for her work, citing in part its "quite antagonistic system of a binary 'EITHER / OR'", which she connected to several dichotomies pitting humanity and the present against technology and the future. In 2018, she started Analog Binary Code, where she creates sculptures by arranging objects in binary code ciphers. She also created Cursive Binary, where she combines binary with cursive handwriting, after writing zeros and ones on a steamed wall while showering. Stiles and the robot BINA48 co-created the 2020 ArtYard exhibition A Valentine for the Future. She was part of the 2021 group exhibition Computational Poetics at the Beall Center for Art and Technology. From February 24 to March 18, 2023, she held her solo show Binary Odes (stylized as B1NARY 0DES) at Annka Kultys Gallery. By 2024, her work had appeared in places such as Gucci storefronts and Times Square billboards. She designed Words Beyond Words, the official poster for Art Basel in Basel 2025. Stiles is based in Milford, New Jersey, where she lives with her husband, musician Kris Bones. She has also lived in Jersey City and Bucks County, Pennsylvania. She is Kalmyk-American on her mother's side, and she has also announced plans to create a version of Technelegy in her ancestral language Kalmyk.
Kernel (image processing)
In image processing, a kernel, convolution matrix, or mask is a small matrix used for blurring, sharpening, embossing, edge detection, and more. This is accomplished by doing a convolution between the kernel and an image. Or more simply, when each pixel in the output image is a function of the nearby pixels (including itself) in the input image, the kernel is that function. == Details == The general expression of a convolution is g x , y = ω ∗ f x , y = ∑ i = − a a ∑ j = − b b ω i , j f x − i , y − j , {\displaystyle g_{x,y}=\omega f_{x,y}=\sum _{i=-a}^{a}{\sum _{j=-b}^{b}{\omega _{i,j}f_{x-i,y-j}}},} where g ( x , y ) {\displaystyle g(x,y)} is the filtered image, f ( x , y ) {\displaystyle f(x,y)} is the original image, ω {\displaystyle \omega } is the filter kernel. Every element of the filter kernel is considered by − a ≤ i ≤ a {\displaystyle -a\leq i\leq a} and − b ≤ j ≤ b {\displaystyle -b\leq j\leq b} . Depending on the element values, a kernel can cause a wide range of effects: The above are just a few examples of effects achievable by convolving kernels and images. === Origin === The origin is the position of the kernel which is above (conceptually) the current output pixel. This could be outside of the actual kernel, though usually it corresponds to one of the kernel elements. For a symmetric kernel, the origin is usually the center element. == Convolution == Convolution is the process of adding each element of the image to its local neighbors, weighted by the kernel. This is related to a form of mathematical convolution. The matrix operation being performed—convolution—is not traditional matrix multiplication, despite being similarly denoted by . For example, if we have two three-by-three matrices, the first a kernel, and the second an image piece, convolution is the process of flipping both the rows and columns of the kernel and multiplying locally similar entries and summing. The element at coordinates [2, 2] (that is, the central element) of the resulting image would be a weighted combination of all the entries of the image matrix, with weights given by the kernel: ( [ a b c d e f g h i ] ∗ [ 1 2 3 4 5 6 7 8 9 ] ) [ 2 , 2 ] = {\displaystyle \left({\begin{bmatrix}a&b&c\\d&e&f\\g&h&i\end{bmatrix}}{\begin{bmatrix}1&2&3\\4&5&6\\7&8&9\end{bmatrix}}\right)[2,2]=} ( i ⋅ 1 ) + ( h ⋅ 2 ) + ( g ⋅ 3 ) + ( f ⋅ 4 ) + ( e ⋅ 5 ) + ( d ⋅ 6 ) + ( c ⋅ 7 ) + ( b ⋅ 8 ) + ( a ⋅ 9 ) . {\displaystyle (i\cdot 1)+(h\cdot 2)+(g\cdot 3)+(f\cdot 4)+(e\cdot 5)+(d\cdot 6)+(c\cdot 7)+(b\cdot 8)+(a\cdot 9).} The other entries would be similarly weighted, where we position the center of the kernel on each of the boundary points of the image, and compute a weighted sum. The values of a given pixel in the output image are calculated by multiplying each kernel value by the corresponding input image pixel values. This can be described algorithmically with the following pseudo-code: for each image row in input image: for each pixel in image row: set accumulator to zero for each kernel row in kernel: for each element in kernel row: if element position corresponding to pixel position then multiply element value corresponding to pixel value add result to accumulator endif set output image pixel to accumulator corresponding input image pixels are found relative to the kernel's origin. If the kernel is symmetric then place the center (origin) of the kernel on the current pixel. The kernel will overlap the neighboring pixels around the origin. Each kernel element should be multiplied with the pixel value it overlaps with and all of the obtained values should be summed. This resultant sum will be the new value for the current pixel currently overlapped with the center of the kernel. If the kernel is not symmetric, it has to be flipped both around its horizontal and vertical axis before calculating the convolution as above. The general form for matrix convolution is [ x 11 x 12 ⋯ x 1 n x 21 x 22 ⋯ x 2 n ⋮ ⋮ ⋱ ⋮ x m 1 x m 2 ⋯ x m n ] ∗ [ y 11 y 12 ⋯ y 1 n y 21 y 22 ⋯ y 2 n ⋮ ⋮ ⋱ ⋮ y m 1 y m 2 ⋯ y m n ] = ∑ i = 0 m − 1 ∑ j = 0 n − 1 x ( m − i ) ( n − j ) y ( 1 + i ) ( 1 + j ) {\displaystyle {\begin{bmatrix}x_{11}&x_{12}&\cdots &x_{1n}\\x_{21}&x_{22}&\cdots &x_{2n}\\\vdots &\vdots &\ddots &\vdots \\x_{m1}&x_{m2}&\cdots &x_{mn}\\\end{bmatrix}}{\begin{bmatrix}y_{11}&y_{12}&\cdots &y_{1n}\\y_{21}&y_{22}&\cdots &y_{2n}\\\vdots &\vdots &\ddots &\vdots \\y_{m1}&y_{m2}&\cdots &y_{mn}\\\end{bmatrix}}=\sum _{i=0}^{m-1}\sum _{j=0}^{n-1}x_{(m-i)(n-j)}y_{(1+i)(1+j)}} === Edge handling === Kernel convolution usually requires values from pixels outside of the image boundaries. There are a variety of methods for handling image edges. Extend The nearest border pixels are conceptually extended as far as necessary to provide values for the convolution. Corner pixels are extended in 90° wedges. Other edge pixels are extended in lines. Wrap The image is conceptually wrapped (or tiled) and values are taken from the opposite edge or corner. Mirror The image is conceptually mirrored at the edges. For example, attempting to read a pixel 3 units outside an edge reads one 3 units inside the edge instead. Crop / Avoid overlap Any pixel in the output image which would require values from beyond the edge is skipped. This method can result in the output image being slightly smaller, with the edges having been cropped. Move kernel so that values from outside of image is never required. Machine learning mainly uses this approach. Example: Kernel size 10x10, image size 32x32, result image is 23x23. Kernel Crop Any pixel in the kernel that extends past the input image isn't used and the normalizing is adjusted to compensate. Constant Use constant value for pixels outside of image. Usually black or sometimes gray is used. Generally this depends on application. === Normalization === Normalization is defined as the division of each element in the kernel by the sum of all kernel elements, so that the sum of the elements of a normalized kernel is unity. This will ensure the average pixel in the modified image is as bright as the average pixel in the original image. === Optimization === Fast convolution algorithms include: separable convolution ==== Separable convolution ==== 2D convolution with an M × N kernel requires M × N multiplications for each sample (pixel). If the kernel is separable, then the computation can be reduced to M + N multiplications. Using separable convolutions can significantly decrease the computation by doing 1D convolution twice instead of one 2D convolution. === Implementation === Here a concrete convolution implementation done with the GLSL shading language :
John Schulman
John Schulman (born 1987 or 1988) is an American artificial intelligence researcher and co-founder of OpenAI. In August 2024, he announced he would be joining Anthropic. In February 2025, he announced he was leaving to join Thinking Machines Lab, where he is chief scientist. == Early life and education == Schulman had an interest in science and math from a young age. He enjoyed science fiction, especially the work of Isaac Asimov. When he was in seventh grade, he became deeply interested in the television program BattleBots, which featured combat between remote-controlled robots. In what he said was his first self-directed study, he read extensively in subject areas that would help him design a superior robot, but the robot he and his friends worked on was never built. He attended Great Neck South High School. He was a member of the US Physics olympiad Team in 2005. In 2010, he graduated from Caltech with a degree in physics. He has a PhD in electrical engineering and computer sciences from the University of California, Berkeley, where he was advised by Pieter Abbeel. == Career == In December 2015, shortly before finishing his PhD, Schulman co-founded OpenAI with Sam Altman, Elon Musk, Ilya Sutskever, Greg Brockman, Trevor Blackwell, Vicki Cheung, Andrej Karpathy, Durk Kingma, Pamela Vagata, and Wojciech Zaremba, with Sam Altman and Elon Musk as the co-chairs. There, he led the reinforcement learning team that created ChatGPT. He has been referred to as the "architect" of ChatGPT. In August 2024, Schulman announced he would be joining Anthropic. He stated his move was to allow him to deepen his focus on AI alignment and return to more hands-on technical work. In February 2025, he announced he was leaving to join Thinking Machines Lab, where he is chief scientist. == Awards and honors == In 2025, Schulman received the Mark Bingham Award for Excellence in Achievement by Young Alumni from his alma mater, UC Berkeley.