AI Coding Interview Questions

AI Coding Interview Questions — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

Adrozek

Adrozek is malware that injects fake ads into online search results. Microsoft announced the malware threat on 10 December 2020, and noted that many different browsers are affected, including Google Chrome, Microsoft Edge, Mozilla Firefox and Yandex Browser. The malware was first detected in May 2020 and, at its peak in August 2020, controlled over 30,000 devices a day. But during the December 2020 announcement, Microsoft claimed "hundreds of thousands" of infected devices worldwide between May and September 2020. According to Microsoft, if not detected and blocked, Adrozek adds browser extensions, modifies a specific DLL per target browser, and changes browser settings to insert additional, unauthorized ads into web pages, often on top of legitimate ads from search engines. For each user tricked into clicking on the fake ads, the scammers earn affiliate advertising dollars. The malware has been observed to extract device data and, in some cases, steal credentials, sending them to remote servers. Users may unintentionally install the malware because of a drive-by download, by visiting a tampered website, opening an e-mail attachment, or clicking on a deceptive link or a deceptive pop-up window. The main malware program is downloaded to the “Programs Files” folder using file names such as Audiolava.exe, QuickAudio.exe, and converter.exe. According to PC Magazine, a good way to avoid, or mitigate, infection by Adrozek is to keep browser and related software programs up to date.
Read more →
G.9972

G.9972 (also known as G.cx) is a Recommendation developed by ITU-T that specifies a coexistence mechanism for networking transceivers capable of operating over electrical power line wiring. It allows G.hn devices to coexist with other devices implementing G.9972 and operating on the same power line wiring. G.9972 received consent during the meeting of ITU-T Study Group 15, on October 9, 2009, and final approval on June 11, 2010. G.9972 specifies two mechanisms for coexistence between G.hn home networks and broadband over power lines (BPL) Internet access networks: Frequency-division multiplexing (FDM), in which the available spectrum is divided into two parts: frequencies below 10 or 14 MHz (specific value can be selected by the access network) are reserved for the access network, while frequencies above them are reserved for the in-home network. Time-division multiplexing (TDM), in which the available channel time is split equally between both networks. 50% of time slots are allocated for the access network, and 50% are allocated to the in-home network.
Read more →
Open Data-Link Interface

The Open Data-Link Interface (ODI) is an application programming interface (API) for network interface controllers (NICs) developed by Apple and Novell. The API serves the same function as Microsoft and 3COM's Network Driver Interface Specification (NDIS). Originally, ODI was written for NetWare and Macintosh environments. Like NDIS, ODI provides rules that establish a vendor-neutral interface between the protocol stack and the adapter driver. It resides in Layer 2, the Data Link layer, of the OSI model. This interface also enables one or more network drivers to support one or more protocol stacks.
Read more →
Social media use in the fashion industry

Social media in the fashion industry refers to the use of social media platforms by fashion designers and users to promote and participate in trends. Over the past several decades, the development of social media has increased along with its usage by consumers. The COVID-19 pandemic was a sharp turn of reliance on the virtual sphere for the industry and consumers alike. Social media has created new channels of advertising for fashion houses to reach their target markets. Since its surge in 2009, luxury fashion brands have used social media to build interactions between the brand and its customers to increase awareness and engagement. The emergence of influencers on social media has created a new way of advertising and maintaining customer relationships in the fashion industry. Numerous social media platforms are used to promote fashion trends, with Instagram and TikTok being the most popular among Generation Y and Z. The overall impact of social media in the fashion industry included the creation of online communities, direct communication between industry leaders and consumers, and criticized ideals that are promoted by the industry through social media. == Background == In 2003, at the beginning of social media development, MySpace was founded as a “social networking service.” It allowed people to create a profile, connect with other people, and post videos, pictures, and songs. As MySpace grew in popularity, it attracted interest from companies wishing to promote their brands on the social platform. MySpace is most well known for exposing musicians and artists who made it big in the industry, and companies wanted to capitalize on their popularity by making brand deals. One of MySpace's deals was with Chevrolet, putting on a ‘secret show’. They had a ‘secret’ list of 10 top artists on MySpace, and many artists posted about the show on their accounts. Another brand deal was with Gucci promoting their “Gucci Synch Watch”, which was very successful as Gucci tapped into the youthful audience on MySpace and advertised a sleek, simple, trendy unisex watch. In 2005, YouTube was released and remains one of the most popular social media platforms today. YouTube allows users to upload videos and is free to anyone with access to the internet. It grew in popularity offering a range of videos: vlogs, cooking, health and diet videos, step-by-step tutorials, tutoring help, and more. Much like MySpace, users create accounts and can build a following, often referring to themselves as ‘YouTubers.’ When YouTube grew in popularity, it piqued the interest of brands wanting to partner with YouTube and individual YouTubers. Some brand deals were made by having ads at the beginning of each video, and the YouTuber would make a profit from each view they receive. Some deals are made by individual YouTubers thanking the brand in videos and promoting the brand's products. More recently, YouTube has delved into fashion. While there were always YouTube channels for Vogue and other fashion companies, popular YouTubers have been invited to different fashion shows and have filmed experiences there. Brands are able to target individual YouTubers based on their followers and the target audiences. In 2010, Instagram was launched, which enlarged the scope of fashion advertising. Instagram allows people to post pictures and short videos with the ability to tag different accounts. For brand deals, companies can simply be tagged in a picture instead of creating ads or lines for a user to say. In each picture, users can tag the brands of clothing they were wearing, making it very easy to promote brands. Additionally, Instagram could display ads on users' feed based on other posts the users liked, which used by fashion companies to target their potential customers. Users also use Instagram to promote fashion when they get invited to fashion events. For example, they can take a picture at the event and post it to their Instagram and put their location at the venue and tag the company. During the beginning of the COVID-19 pandemic, companies relied more on social media to keep their public virtually engaged. Fashion companies had virtual fashion shows, creating videos and content about their designs. As social media expands and new platforms come into existence, new ways of advertising are projected to be created. == Uses == === Advertising === Social media is a popular use of advertisement in the fashion industry. Information sharing has expanded due to the growth of social media platforms, which impacts social consumer involvement with fashion brands. Fashion companies use social media platforms to reach customers on emotional levels and stoke engagement with brand images and messages. Researchers in the United Kingdom have demonstrated that engaging with customers with social media messages that express social passion, social tendency, and personal warmth can boost social engagement with fashion brands. In social spheres, fashion is a method for individuals to represent their distinction through clothing. Some people who desire to socially influence others through their fashion and style now have the possibility thanks to social media in the fashion sector. Customers who want to purchase fashion brands frequently follow fashion authorities on social media and heed their recommendations for purchasing fashion products. === Influencers === Companies leveraged celebrities' fame and social standing to advertise their brands, as Tommy Hilfiger did when incorporating social media into their marketing strategy, making Gigi Hadid, who has 15.5 million Instagram followers as of 2016, a brand ambassador. Though recent developments in social media platforms have led to an increase in the awareness of influencers. Influencer marketing has emerged as a fast expanding marketing strategy in various industries as a result of the unheard-of increase in the number of social media influencers' followers. Recently, influencer marketing has received significant attention in the fashion industry. Research shows that influencer marketing may provide a rate of influence that is 11x times greater than that of other conventional advertising channels. Fashion consumers, specifically those in generations Y and Z, may be more influenced by influencers in the context of the fashion industries as they often view them as friends and personal assistants. Fashion influencer marketing on social media platforms have led fashion consumption on social sopping services. One of these social fashion services is LTK (LIKEtoKNOW.it before 2021) where everyday consumers can find and purchase clothing worn by social media fashion influencers (also known as SMFIs). Launched in 2014, LTK has gained a massive following on Instagram (over 3 million) and has 1.3 million registered users on their mobile application. Utilizing SMFIs has led to massive sales within the fashion industry, 80% of visitors of Nordstrom's mobile platform are referred by influencers. Social media fashion influencers try new fashion products, adopt fashion trends and have power in what their audience purchases. Social media fashion influencers gain a following though promoting fashion products, and posting about their lavish lifestyles attained through their higher socioeconomic status. The attractive lifestyles of the influencers influence their followers to mimic their luxurious lifestyle and are allowed to consume the same products through social shopping services. In addition to brands themselves having direct access to social media users, many content creators have great influence over consumers. "Influencers" across all social media platforms have great power when it comes to where people shop and what they purchase. Influencer marketing has become one of the most effective marketing strategies for many fashion brands. These brand deals and creator partnerships are targeted towards Millennial and Gen Z consumers, specifically on Instagram and TikTok, and 74% of consumers have made a purchase simply because an influencer they follow had recommended it. === Trends === The connection between social media and fashion has become common. Influencer marketing has emerged as a necessity and crucial component of advertising. 85% of American businesses are presently using influencer marketing as part of their marketing plan. Wearing fashion brands is a method to show oneself at social gatherings. Through their clothing, people try to demonstrate how distinct they are. Some people who really desire to socially influence others through their fashion and style now have the possibility thanks to social media in the fashion sector. Customers who want to purchase fashion brands frequently follow fashion authorities on social media and heed their recommendations for purchasing fashion products. In January 2021, the Italian fashion house Bottega Veneta deleted all its social media accounts "to lean much more on its ambassadors and fans" to spread the com
Read more →
Collabora Online

Collabora Online (often abbreviated as COOL) is an open-source online office suite developed by Collabora, based on LibreOffice Online, the web-based edition of the LibreOffice office suite. It enables real-time collaborative editing of documents, spreadsheets, presentations, and vector graphics in a web browser. Optional applications are available for offline use on Android, ChromeOS, iOS, iPadOS, Linux distributions, macOS, and Windows. It supports the OpenDocument format and is compatible with other major formats, including those used by Microsoft Office. The Document Foundation (TDF), the nonprofit organization behind LibreOffice, states that a majority of the LibreOffice software development is done by its partners like Collabora. Collabora Online is an open-source alternative to proprietary cloud office platforms such as Google Workspace and Microsoft 365. Unlike these services, it can be self-hosted or hosted by third-party providers. The platform is marketed particularly toward enterprises and public institutions seeking greater digital sovereignty and independence from U.S.-based "big tech" companies. Collabora also develops Collabora Office, a standalone desktop and mobile app suite based on LibreOffice. Although Collabora Online has increasingly taken on a central role, both products may be used in parallel, similar to Microsoft Office and Microsoft 365. In November 2025, Collabora released Collabora Office Desktop and renamed the previous product Collabora Office Classic. The new product shares code with Collabora Online and brings the same user interface to the desktop on Linux, Windows and MacOS. A separate version, the Collabora Online Development Edition (CODE), is offered free of charge and is recommended for individuals, small teams, and developers. CODE provides early access to new features and serves as a testing and development platform for open-source community contributors. As TDF does not offer a free version of LibreOffice Online, CODE represents the primary freely available option for organizations and individuals interested in deploying LibreOffice in a web-based, collaborative setting. == Applications == Collabora Online includes several applications for document editing, available through the web-based interface and optional desktop and mobile apps: Collabora Writer – A word processor based on LibreOffice Writer, comparable to Microsoft Word and Google Docs. It supports WYSIWYG editing, styles, formatting tools, comment threads, and change tracking. Collabora Calc – A spreadsheet editor based on LibreOffice Calc, similar to Microsoft Excel and Google Sheets. Features include pivot tables, formulas, data validation, conditional formatting, advanced sorting and filtering, charts, and support for up to 16,000 columns. Compatible with some macros written in VBA. Collabora Impress – A presentation program based on LibreOffice Impress, comparable to Microsoft PowerPoint and Google Slides. It supports master slides, transitions, speaker notes, and multimedia elements. Collabora Draw is not a separate application, most of the functionality of the Draw application is now integrated in Writer and Impress – vector graphics editor based on LibreOffice Draw, comparable to Microsoft Visio and Google Drawings. == Features == Collabora Online can be accessed from modern web browsers without the need for plug-ins or add-ons. It supports real-time collaborative editing of word processing documents, spreadsheets, presentations, and vector graphics. Collaboration features include commenting, version tracking with document comparison and restoration, and integration with communication tools such as chat or video calls. These functions are often enabled through integration with enterprise open-source cloud platforms like Nextcloud, ownCloud, Seafile, EGroupware, GroupOffice and others. Collabora Online can also be embedded or integrated into a variety of third-party applications. Although client apps are not required to use the web-based suite, optional applications are available for offline use on Android, ChromeOS, iOS, iPadOS, Linux distributions, macOS, and Windows. These apps share the same LibreOffice-based core as the server version, ensuring document compatibility across platforms. Development of the LibreOffice core benefits both the online server and the client applications simultaneously. The mobile apps offer touch-optimized interfaces that adapt to different screen sizes and can be used offline, with optional integration into cloud storage services. Collabora Online supports OpenDocument formats (ODF; .odt, .odp, .ods, .odg) in accordance with ISO/IEC 26300. It is also compatible with Microsoft Office formats, including Office Open XML (.docx, .pptx, .xlsx) and legacy binary formats (.doc, .ppt, .xls). Additional supported formats include PDF, PNG, CSV, TSV, RTF, EPUB, and others. The suite can import a range of formats supported by LibreOffice, including Microsoft Visio and Publisher files, Apple Keynote, Numbers, and Pages files, as well as legacy formats used by Lotus 1-2-3, Microsoft Works, and Quattro Pro. The core of Collabora Online is written in C++ and utilizes LibreOfficeKit, a programming interface that enables reuse of much of LibreOffice's existing code for document saving, loading, and rendering. Collabora Online operates on the principle that documents remain on the server, with users viewing tile-rendered images of the document and sending their edits back to the server. The user interface is implemented in JavaScript. For file access and authentication with file hosting services, Collabora Online uses Microsoft's WOPI protocol, allowing compatibility with any service supporting Microsoft 365 integration. == Server == The server component can be self-hosted or deployed through third-party enterprise open-source cloud platforms, allowing organizations to maintain control over data and infrastructure. It is available for various Linux distributions and as a Docker image. The server enables features such as in-browser document editing, file synchronization, and real-time communication. These third-party cloud platforms typically offer additional functionality comparable to services such as Dropbox, Google Workspace, Microsoft 365, or Zoom, including file sharing, calendars, email, contacts, chat, and video conferencing. Collabora Online can be integrated into these applications, as well as with other services such as learning management systems and enterprise content platforms, through open APIs and an SDK. == Reception == Various online and print publications have discussed Collabora Online. In December 2016 the technology website Softpedia mentioned the availability of collaborative editing in version 2.0 and the integration with ownCloud, Nextcloud, and other file synchronization and sharing solutions. In June 2020, ZDNET reported that Collabora Online would be included as the standard office suite in Nextcloud version 19, noting that direct document editing was added to the native video conferencing software Talk. The technology blog OMG! Ubuntu! covered the release of Collabora's Android and iOS apps, emphasizing their offline functionality. In September 2020, Linux Magazine compared Collabora Online with OnlyOffice, noting the flexibility and platform independence of both tools and highlighting Collabora's extensive feature set derived from LibreOffice. === Digital sovereignty === Collabora Online's open-source design and support for self-hosting have made it notable in discussions about digital sovereignty—the ability of users and organizations to control their own data. This is particularly relevant in Europe, where concerns about dependence on U.S.-based "big tech" companies and data privacy have grown in recent years. On 10th June 2025, Microsoft executives under oath in the French Senate admitted that they cannot guarantee data sovereignty and would be compelled to pass French (and by implication the wider European Union) information to the US administration if requested via a warrant or subpoena. The Cloud Act is a law that gives the US government authority to obtain digital data held by US-based tech corporations, irrespective of whether that data is stored on servers at home or on foreign soil. A 2020 briefing by the European Parliament highlighted risks associated with reliance on major technology companies that collect and exploit user data. Legal decisions such as the Schrems II ruling have further underscored these concerns. Several European government agencies have adopted private cloud solutions using Collabora Online and related platforms to enhance data security and maintain control over sensitive information. == History == The former LibreOffice development team from SUSE joined Collabora in September 2013, forming the subsidiary Collabora Productivity. In 2015 Collabora and IceWarp announced the development of an enterprise-ready version of LibreOffice Online to compete wi
Read more →
Brooklyn Bridge (software)

The Brooklyn Bridge from White Crane Systems was a data transfer enabler. Although it came with some hardware, it was the software which was the basis of the product. It also could transform the data's format. == Overview == The New York Times described its category as being among "communications packages used to transfer files." In an era of 300 baud, Brooklyn Bridge operated at "115,200 baud" so that a transfer which "at 300 baud took 4 minutes and 36 seconds" only needed 5 seconds. Unlike some communications packages, this one retains the original version-date, so as not to alarm people when they seem to have what looks like an update, when it's not. == Description == Once the software is installed, users comfortable with typing the word "COPY" can do so as readily as they sneakernet. An earlier review described it as "less cumbersome than conventional communications software" The use of neither specialized hardware nor specialized software is ideal in an era when this can be done using online or other "outside" services.
Read more →
Cambridge Analytica

Cambridge Analytica Ltd. (CA), previously known as SCL USA, was a British political consulting firm that came to prominence through the Facebook–Cambridge Analytica data scandal. It was founded in 2013, as a subsidiary of the private intelligence company and self-described "global election management agency" SCL Group by long-time SCL executives Nigel Oakes, Alexander Nix and Alexander Oakes, with Nix as CEO. Cambridge Analytica was hired by a variety of political actors, including the Trinidadian government in 2010 and the 2016 presidential campaigns of Ted Cruz and Donald Trump. The firm maintained offices in London, New York City, and Washington, D.C. The company closed operations in 2018 due to backlash from the scandal, although firms related to both Cambridge Analytica and its parent firm SCL still exist. == History == Cambridge Analytica was founded in 2013 as a subsidiary of the private intelligence company SCL Group, which describes itself as providing "data, analytics and strategy to governments and military organisations worldwide". The company was part of "an international web of companies" headed by the London-based SCL Group. Cambridge Analytica (SCL USA) was incorporated in January 2013 with its registered office being in Westferry Circus, London and consisting of just one staff member, director and CEO Alexander Nix (also appointed in January 2015). Nix was also the director of nine similar companies sharing the same registered offices in London, including Firecrest technologies, Emerdata and six SCL Group companies including "SCL elections limited". Nigel Oakes, known as the former boyfriend of Lady Helen Windsor, had founded the predecessor SCL Group in the 1990s, and in 2005 Oakes established SCL Group together with his brother Alexander Oakes and Alexander Nix; SCL Group was the parent company of Cambridge Analytica. Former Conservative minister and MP Sir Geoffrey Pattie was the founding chairman of SCL; Lord Ivar Mountbatten also joined Oakes as a director of the company. As a result of the Facebook–Cambridge Analytica data scandal, Nix was removed as CEO and replaced by Julian Wheatland before the company closed. Several of the company's executives were Old Etonians. The company's owners included several of the Conservative Party's largest donors such as billionaire Vincent Tchenguiz, former British Conservative minister Jonathan Marland, Baron Marland and the family of American hedge fund manager Robert Mercer. The company combined misappropriation of digital assets, data mining, data brokerage, and data analysis with strategic communication during electoral processes. While its parent SCL had focused on influencing elections in developing countries since the 1990s, Cambridge Analytica focused more on the western world, including the United Kingdom and the United States; CEO Alexander Nix has said CA was involved in 44 U.S. political races in 2014. In 2015, CA performed data analysis services for Ted Cruz's presidential campaign. In 2016, CA worked for Donald Trump's presidential campaign as well as for Leave.EU (one of the organisations campaigning in the United Kingdom's referendum on European Union membership). CA's role in those campaigns has been controversial and is the subject of ongoing inquiries in both countries. Political scientists question CA's claims about the effectiveness of its methods of targeting voters. == Data scandal == In March 2018, media outlets broke news of Cambridge Analytica's business practices. The New York Times and The Observer reported that the company had acquired and used personal data about Facebook users from an external researcher who had told Facebook he was collecting it for academic purposes. Shortly afterwards, Channel 4 News aired undercover investigative videos showing Nix boasting about using prostitutes, bribery sting operations, and honey traps to discredit politicians on whom it had conducted opposition research, and saying that the company "ran all of (Donald Trump's) digital campaign". In response to the media reports, the Information Commissioner's Office (ICO) of the UK pursued a warrant to search the company's servers. Facebook banned Cambridge Analytica from advertising on its platform, saying that it had been deceived. On 23 March 2018, the British High Court granted the ICO a warrant to search Cambridge Analytica's London offices. As a result, Nix was suspended as CEO, and replaced by Julian Wheatland. The personal data of up to 87 million Facebook users were acquired via the 270,000 Facebook users who used a Facebook app created by Aleksandr Kogan called "This Is Your Digital Life". This was a personality profiling app and asked simple personality questions similar to other Facebook quizzes. Kogan was a scientist and psychologist, also being an employed lecturer for the University of Cambridge from 2012 to 2018. Alexander Nix claimed they had close to five thousand data points on each person who participated. They also gathered information through other data brokers ending with them acquiring millions of data points from American citizens. Kogan's app exploited a feature of Facebook's Graph API (version 1.0), which permitted any third-party app to access not only the app user's data, but also the full profile data of all of that user's Facebook friends, without those friends' knowledge or consent. This platform-wide design was available to all developers and was used by tens of thousands of apps; Facebook CEO Mark Zuckerberg later told the House Energy and Commerce Committee that the company was auditing "tens of thousands" of apps that had had access to large amounts of user data. Because the average Facebook user at the time had approximately 300 friends, the 270,000 users who installed Kogan's app yielded data on up to 87 million people. Facebook deprecated the friends-data API in April 2014 and shut it down entirely in April 2015, but data already collected by apps remained in developers' possession. Kogan passed this data to Cambridge Analytica, breaching Facebook's terms of service. On 1 May 2018, Cambridge Analytica and its parent company SCL filed for insolvency proceedings and closed operations. Alexander Tayler, a former director for Cambridge Analytica, was appointed director of Emerdata on 28 March 2018. Rebekah Mercer, Jennifer Mercer, Alexander Nix and Johnson Chun Shun Ko, who has links to American businessman Erik Prince, are in leadership positions at Emerdata. The Russo brothers are producing an upcoming film on Cambridge Analytica. In 2019 the Federal Trade Commission filed an administrative complaint against Cambridge Analytica for misuse of data. In 2020, the British Information Commissioner's Office closed a three-year inquiry into the company, concluded that Cambridge Analytica was "not involved" in the 2016 Brexit referendum and found no additional evidence for Russia's alleged interference during the campaign. US sensitive polling and election data, however, were passed to Russian Intelligence via a Cambridge Analytica contractor Sam Patten, Trump campaign manager Paul Manafort, and Russian agent Konstantin Kilimnik, who was indicted during the affair. Publicly, parent company SCL Group called itself a "global election management agency", Politico reported it was known for involvement "in military disinformation campaigns to social media branding and voter targeting". SCL gained work on a large number of campaigns for the US and UK governments' war on terror advancing their model of behavioral conflict during the 2000s. SCL's involvement in the political world has been primarily in the developing world where it has been used by the military and politicians to study and manipulate public opinion and political will. Slate writer Sharon Weinberger compared one of SCL's hypothetical test scenarios to fomenting a coup. Among the investors in Cambridge Analytica were some of the Conservative Party's largest donors such as billionaire Vincent Tchenguiz, former Conservative minister Jonathan Marland, Baron Marland, Roger Gabb, the family of American hedge fund manager Robert Mercer, and Steve Bannon. A minimum of 15 million dollars has been invested into the company by Mercer, according to The New York Times. Bannon's stake in the company was estimated at 1 to 5 million dollars, but he divested his holdings in April 2017 as required by his role as White House Chief Strategist. In March 2018, Jennifer Mercer and Rebekah Mercer became directors of Emerdata limited. In March 2018 it became public by Christopher Wylie, that Cambridge Analytica's first activities were founded on a data set, which its parent company SCL bought 2014 from a company named Global Science Research founded by Aleksandr Kogan and his team present across the world who worked as a psychologist at Cambridge. During Boris Johnson's tenure as foreign secretary, the Foreign Office sought advice from Cambridge Analytica and Boris Johnson had a meeting with Alexander N
Read more →
Social media mining

Social media mining is the process of obtaining data from user-generated content on social media in order to extract actionable patterns, form conclusions about users, and act upon the information. Mining supports targeting advertising to users or academic research. The term is an analogy to the process of mining for minerals. Mining companies sift through raw ore to find the valuable minerals; likewise, social media mining sifts through social media data in order to discern patterns and trends about matters such as social media usage, online behaviour, content sharing, connections between individuals, buying behaviour. These patterns and trends are of interest to companies, governments and not-for-profit organizations, as such organizations can use the analyses for tasks such as design strategies, introduce programs, products, processes or services. Social media mining uses concepts from computer science, data mining, machine learning, and statistics. Mining is based on social network analysis, network science, sociology, ethnography, optimization and mathematics. It attempts to formally represent, measure and model patterns from social media data. In the 2010s, major corporations, governments and not-for-profit organizations began mining to learn about customers, clients and others. Platforms such as Google, Facebook (partnered with Datalogix and BlueKai) conduct mining to target users with advertising. Scientists and machine learning researchers extract insights and design product features. Users may not understand how platforms use their data. Users tend to click through Terms of Use agreements without reading them, leading to ethical questions about whether platforms adequately protect users' privacy. During the 2016 United States presidential election, Facebook allowed Cambridge Analytica, a political consulting firm linked to the Trump campaign, to analyze the data of an estimated 87 million Facebook users to profile voters, creating controversy when this was revealed. == Background == As defined by Kaplan and Haenlein, social media is the "group of internet-based applications that build on the ideological and technological foundations of Web 2.0, and that allow the creation and exchange of user-generated content." There are many categories of social media including, but not limited to, social networking (Facebook or LinkedIn), microblogging (Twitter), photo sharing (Flickr, Instagram, Photobucket, or Picasa), news aggregation (Google Reader, StumbleUpon, or Feedburner), video sharing (YouTube, MetaCafe), livecasting (Ustream or Twitch), virtual worlds (Kaneva), social gaming (World of Warcraft), social search (Google, Bing, or Ask.com), and instant messaging (Google Talk, Skype, or Yahoo! messenger). The first social media website was introduced by GeoCities in 1994. It enabled users to create their own homepages without having a sophisticated knowledge of HTML coding. The first social networking site, SixDegrees.com, was introduced in 1997. Since then, many other social media sites have been introduced, each providing service to millions of people. These individuals form a virtual world in which individuals (social atoms), entities (content, sites, etc.) and interactions (between individuals, between entities, between individuals and entities) coexist. Social norms and human behavior govern this virtual world. By understanding these social norms and models of human behavior and combining them with the observations and measurements of this virtual world, one can systematically analyze and mine social media. Social media mining is the process of representing, analyzing, and extracting meaningful patterns from data in social media, resulting from social interactions. It is an interdisciplinary field encompassing techniques from computer science, data mining, machine learning, social network analysis, network science, sociology, ethnography, statistics, optimization, and mathematics. Social media mining faces grand challenges such as the big data paradox, obtaining sufficient samples, the noise removal fallacy, and evaluation dilemma. Social media mining represents the virtual world of social media in a computable way, measures it, and designs models that can help us understand its interactions. In addition, social media mining provides necessary tools to mine this world for interesting patterns, analyze information diffusion, study influence and homophily, provide effective recommendations, and analyze novel social behavior in social media. == Uses == Social media mining is used across several industries including business development, social science research, health services, and educational purposes. Once the data received goes through social media analytics, it can then be applied to these various fields. Often, companies use the patterns of connectivity that pervade social networks, such as assortativity—the social similarity between users that are induced by influence, homophily, and reciprocity and transitivity. These forces are then measured via statistical analysis of the nodes and connections between these nodes. Social analytics also uses sentiment analysis, because social media users often relay positive or negative sentiment in their posts. This provides important social information about users' emotions on specific topics. These three patterns have several uses beyond pure analysis. For example, influence can be used to determine the most influential user in a particular network. Companies would be interested in this information in order to decide who they may hire for influencer marketing. These influencers are determined by recognition, activity generation, and novelty—three requirements that can be measured through the data mined from these sites. Analysts also value measures of homophily: the tendency of two similar individuals to become friends. Users have begun to rely on information of other users' opinions in order to understand diverse subject matter. These analyses can also help create recommendations for individuals in a tailored capacity. By measuring influence and homophily, online and offline companies are able to suggest specific products for individuals consumers, and groups of consumers. Social media networks can use this information themselves to suggest to their users possible friends to add, pages to follow, and accounts to interact with. == Perception == Modern social media mining is a controversial practice that has led to exponential gains in user growth for tech giants such as Facebook, Inc., Twitter, and Google. Companies such as these, considered "Big Tech" are companies that build algorithms that take advantage of user input to understand their preferences, and keep them on the platform as much as possible. These inputs, that can be as simple as time spent on a given screen, provide the data being mined, and lead to companies profiting heavily from using that data to capitalize on extremely accurate predictions about user behavior. The growth of platforms accelerated rapidly once these strategies were put in place; Most of the largest platforms now average over 1 billion active users per month as of 2021. It has been claimed by a multitude of anti-algorithm personalities, like Tristan Harris or Chamath Palihapitiya, that certain companies (specifically Facebook) valued growth above all else, and ignored potential negative impacts from these growth engineering tactics. At the same time, users have now created their own data arbitrages with the help of their own data, through content monetization and becoming influencers. Users typically have access to a varied set of analytics specific to people that interact with them on social media, and can use these as building blocks for their own targeting and growth strategies through ads and posts that cater to their audiences. Influencers also commonly promote products and services for established brands, creating one of the largest digital industries: Influencer marketing. Instagram, Facebook, Twitter, YouTube, Google, and others have long given access to platform analytics, and allowed third parties to access that information as well, at times unbeknownst to even the user whose data is being viewed/bought. == Research == === Research areas === Social media event detection – Social networks enable users to freely communicate with each other and share their recent news, ongoing activities or views about different topics. As a result, they can be seen as a potentially viable source of information to understand the current emerging topics/events. Public health monitoring and surveillance - Using large-scale analysis of social media to study large cohorts of patients and the general public, e.g. to obtain early warning signals of drug-drug interactions and adverse drug reactions, or understand human reproduction and sexual interest. Community structure (Community Detection/Evolution/Evaluation) – Identifying communities on social networks, how t
Read more →
System appreciation

System appreciation is an activity often included in the maintenance phase of software engineering projects. Key deliverables from this phase include documentation that describes what the system does in terms of its functional features, and how it achieves those features in terms of its architecture and design. Software architecture recovery is often the first step within System appreciation.
Read more →
Data analysis

Data analysis is the process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusions, and supporting decision-making. Data analysis has multiple facets and approaches, encompassing diverse techniques under a variety of names, and is used in different business, science, and social science domains. In today's business world, data analysis plays an important role in making decisions more scientific and helping businesses operate more effectively. It is widely used in fields such as business analytics, healthcare, and artificial intelligence to extract meaningful insights from data. Data mining is a particular data analysis technique that focuses on statistical modeling and knowledge discovery for predictive rather than purely descriptive purposes, while business intelligence covers data analysis that relies heavily on aggregation, focusing mainly on business information. In statistical applications, data analysis can be divided into descriptive statistics, exploratory data analysis (EDA), and confirmatory data analysis (CDA). EDA focuses on discovering new features in the data, while CDA focuses on confirming or falsifying existing hypotheses. Predictive analytics focuses on the application of statistical models for predictive forecasting or classification, while text analytics applies statistical, linguistic, and structural techniques to extract and classify information from textual sources, a variety of unstructured data. All of the above are varieties of data analysis. == Data analysis process == Data analysis is a process for obtaining raw data, and subsequently converting it into information useful for decision-making by users. Statistician John Tukey, defined data analysis in 1961, as:"Procedures for analyzing data, techniques for interpreting the results of such procedures, ways of planning the gathering of data to make its analysis easier, more precise or more accurate, and all the machinery and results of (mathematical) statistics which apply to analyzing data." There are several phases, and they are iterative, in that feedback from later phases may result in additional work in earlier phases. === Data requirements === The data is necessary as inputs to the analysis, which is specified based upon the requirements of those directing the analytics (or customers, who will use the finished product of the analysis). The general type of entity upon which the data will be collected is referred to as an experimental unit (e.g., a person or population of people). Specific variables regarding a population (e.g., age and income) may be specified and obtained. Data may be numerical or categorical (i.e., a text label for numbers). === Data collection === Data may be collected from a variety of sources. A list of data sources are available for study & research. The requirements may be communicated by analysts to custodians of the data; such as, Information Technology personnel within an organization. Data collection or data gathering is the process of gathering and measuring information on targeted variables in an established system, which then enables one to answer relevant questions and evaluate outcomes. The data may also be collected from sensors in the environment, including traffic cameras, satellites, recording devices, etc. It may also be obtained through interviews, downloads from online sources, or reading documentation. === Data processing === Data integration is a precursor to data analysis: Data, when initially obtained, must be processed or organized for analysis. For instance, this may involve placing data into rows and columns in a table format (known as structured data) for further analysis, often through the use of spreadsheet (e.g. Excel) or statistical software. === Data cleaning === Once processed and organized, the data may be incomplete, contain duplicates, or contain errors. The need for data cleaning will arise from problems in the way that the data is entered and stored. Data cleaning is the process of preventing and correcting these errors. Common tasks include record matching, identifying inaccuracy of data, overall quality of existing data, deduplication, and column segmentation. Such data problems can also be identified through a variety of analytical techniques. For example; with financial information, the totals for particular variables may be compared against separately published numbers that are believed to be reliable. Unusual amounts, above or below predetermined thresholds, may also be reviewed. There are several types of data cleaning that are dependent upon the type of data in the set; this could be phone numbers, email addresses, employers, or other values. Quantitative data methods for outlier detection can be used to get rid of data that appears to have a higher likelihood of being input incorrectly. Text data spell checkers can be used to lessen the amount of mistyped words. However, it is harder to tell if the words are contextually (i.e., semantically and idiomatically) correct. === Exploratory data analysis === Once the datasets are cleaned, they can then begin to be analyzed using exploratory data analysis. The process of data exploration may result in additional data cleaning or additional requests for data; thus, the initialization of the iterative phases mentioned above. Descriptive statistics, such as the average, median, and standard deviation, are often used to broadly characterize the data. Data visualization is also used, in which the analyst is able to examine the data in a graphical format in order to obtain additional insights about messages within the data. === Modeling and algorithms === Mathematical formulas or mathematical models (supported by algorithms) may be applied to the data in order to identify relationships among the variables; for example, checking for correlation and by determining whether or not there is the presence of causality. In general terms, models may be developed to evaluate a specific variable based on other variable(s) contained within the dataset, with some residual error depending on the implemented model's accuracy (e.g., Data = Model + Error). Inferential statistics utilizes techniques that measure the relationships between particular variables. For example, regression analysis may be used to model whether a change in advertising (independent variable X), provides an explanation for the variation in sales (dependent variable Y), i.e. is Y a function of X? This can be described as (Y = aX + b + error), where the model is designed such that (a) and (b) minimize the error when the model predicts Y for a given range of values of X. === Data product === A data product is a computer application that takes data inputs and generates outputs, feeding them back into the environment. It may be based on a model or algorithm. For instance, an application that analyzes data about customer purchase history, and uses the results to recommend other purchases the customer might enjoy. === Communication === Once data is analyzed, it may be presented in many formats to the users of the analysis to support their requirements. The users may have feedback, which results in additional analysis. When determining how to communicate the results, the analyst may consider implementing a variety of data visualization techniques to help communicate the message more clearly and efficiently to the audience. Data visualization uses information displays (graphics such as, tables and charts) to help communicate key messages contained in the data. Tables are a valuable tool by enabling the ability of a user to query and focus on specific numbers; while charts (e.g., bar charts or line charts), may help explain the quantitative messages contained in the data. == Quantitative messages == Stephen Few described eight types of quantitative messages that users may attempt to communicate from a set of data, including the associated graphs. Time-series: A single variable is captured over a period of time, such as the unemployment rate over a 10-year period. A line chart may be used to demonstrate the trend. Ranking: Categorical subdivisions are ranked in ascending or descending order, such as a ranking of sales performance (the measure) by salespersons (the category, with each salesperson a categorical subdivision) during a single period. A bar chart may be used to show the comparison across the salespersons. Part-to-whole: Categorical subdivisions are measured as a ratio to the whole (i.e., a percentage out of 100%). A pie chart or bar chart can show the comparison of ratios, such as the market share represented by competitors in a market. Deviation: Categorical subdivisions are compared against a reference, such as a comparison of actual vs. budget expenses for several departments of a business for a given time period. A bar chart can show the comparison of the actual versus the reference amount. Frequency distribution:
Read more →
Control break

In computer programming, a control break is a change in the value of one of the keys on which a file is sorted, which requires some extra processing. For example, with an input file sorted by post code, the number of items found in each postal district might need to be printed on a report, and a heading shown for the next district. Quite often there is a hierarchy of nested control breaks in a program, such as streets within districts within areas, with the need for a grand total at the end. Structured programming techniques have been developed to ensure correct processing of control breaks in languages such as COBOL and to ensure that conditions such as empty input files and sequence errors are handled properly. With fourth-generation languages such as SQL, the programming language should handle most of the details of control breaks automatically.
Read more →
Social Media (Age-Restricted Users) Bill

The Social Media (Age-Restricted Users) Bill is a member's bill by National Party Member of Parliament Catherine Wedd that seeks to ban children under the age of 16 years from accessing social media by forcing social media companies to implement age verification measures. It is modelled after the Australian government's Online Safety Amendment. In mid October 2025, the New Zealand Parliament confirmed plans to introduce the social media age restriction bill. == Background == In late November 2024, the Albanese government of Australia, with support from the opposition Coalition parties, passed the Online Safety Amendment creating a world-first age verification regime targeting social media platforms operating in the country. The ban targets several social media platforms including Facebook, Instagram, Kick, Reddit, Snapchat, Threads, TikTok, Twitch, X (formerly Twitter) and YouTube. These platforms were required to implement age verification systems and to remove under-age users by 10 December 2025, when the law change came into effect. == Draft provisions == The draft Social Media (Age-Restricted Users) Bill defines social media platforms as electronic platforms that enable social media interactions between two or more end-users, facilitates communication between multiple end-users and allows users to post content on the platform. The proposed bill requires social media companies to take action to prevent users under the age of 16 from creating accounts on their platforms. It also creates a framework for courts to impose fines on platforms that fail to take reasonable steps to prevent underaged users from accessing the platform. == Legislative history == === Draft legislation === On 6 May 2025, Wedd announced a private member's bill called the "Social Media (Age-Restricted Users) Bill" that would bar access to social media platforms for people under the age of 16 years. She said that she was motivated as the mother of four children to support families, parents and teachers' efforts to manage their children's online exposure and the passage of the Australian Online Safety Amendment legislation in December 2024. Since National's coalition partner ACT New Zealand had refused to support the bill, the Sixth National Government announce it as a member's bill rather than a government bill. Prime Minister Christopher Luxon has confirmed that National would seek cross-party support for the legislation. ACT MP and the Minister of Internal Affairs Brooke van Velden said that the Government would watch the implementation of the Australian social media age restriction policy. In October 2025, Wedd's bill was drawn from the parliamentary ballot. In addition, Labour Reuben Davidson drafted a similar member's bill that would hold social media providers responsible for restricting "harmful content" and imposed NZ$50,000 fines for non-compliance. In November 2025, Luxon reiterated his support for social media age restriction legislation and said the New Zealand government would introduce a bill in 2026 before the 2026 New Zealand general election. He also confirmed that Education Minister Erica Stanford was leading an investigation into what lessons could be learnt from the Australian legislation. At the request of ACT MP Parmjeet Parmar, Parliament's Education and Workforce Committee held an inquiry into a proposed social media ban in early October 2025. The committee was led by National MP Carl Bates and received 430 submissions from 400 groups and individuals. The committee also heard from 87 in-person submissions. On 10 December 2025, the committee made 12 recommendations including restricting social media access to persons under the age of 16, re-evaluating existing legislation such as the Films, Videos, and Publications Classification Act and the Harmful Digital Communications Act 2015, and regulating online platforms and Internet service providers. The ACT party released a dissenting view disagreeing with the need for a law restricting social media access to under-16 year olds. In mid-May 2026, the Government confirmed that work on the proposed bill to ban under-16 year olds from social media had been paused. The New Zealand Parliament held a debate on the proposed bill on 13 May following a select committee inquiry into the harms caused by social media platforms. While the opposition Labour Party has agreed to support the member's bill, the ACT and Green parties opposed the proposed bill on the grounds that the rules were easy to circumvent, that at-risk groups could become more isolated, and that social media also harmed other age groups. == Responses == === Academia and civil society === In late July 2025, the New Zealand Council for Civil Liberties (NZCCL) expressed concern that the proposed social media age restriction could infringe upon the New Zealand Bill of Rights Act 1990, the Privacy Act 2020 and the United Nations' Convention on the Rights of the Child. The NZCCL also questioned the practicality of age verification software, a social media age limit and whether it would fulfil its stated goal of combating online harm. In August 2025, University of Auckland criminologist and senior lecturer Claire Meehan expressed concern that the social media age restriction legislation would cut children from their friendship and support networks. She also said that children and young people were digital natives who could use VPNs to circumvent the ban. Similar sentiments were echoed by Victoria University of Wellington media and communications lecturer Alex Beattie and "Ocean Today" Instagram social media influencer "Charlie." In October 2025, New Zealand Initiative representative Dr Eric Crampton expressed concern that a social media age restriction would involve the introduction of digital IDs. He argued that a new law was unnecessary and said that parents could limit their children's exposure to social media via Google's Family Link and Apple's equivalent. Similarly, Institute of Economic Affairs public policy fellow Matthew Lesh and the British Free Speech Union expressed concerns that young people could use VPNs to circumvent a social media ban, citing the spike in VPN usage in the United Kingdom following the passage of the Online Safety Act 2023. The advocacy group B416's co-chair Anna Curzon advocated for a social media ban on underage users, stating that social media apps "are made to be addictive" and made it difficult for parents to relate with their children. In late November 2025, B416's co-founder Anna Mowbray expressed support for the Government's social media age restriction bill but expressed disappointment that Luxon had not timed his announcement with the launch of the group's campaign. Generation-Z Aotearoa co-founder Lola Fisher has called on the New Zealand Government to consult with young people on the development of the legislation. === Government agencies and departments === In early October 2025, Privacy Commissioner Michael Webster expressed concern that social media platforms requiring users to prove their age via digital IDs could raise privacy concerns. Webster suggested that age verification systems could relay on various documents including passports. He said that age estimation technologies had high error rates and that age inference technologies relied on data mining. === Political parties === In early May 2025, the National Party government expressed support for a social media age restriction legislation. By contrast, its coalition partner ACT has opposed such legislation. ACT leader David Seymour described the ban as hasty and unworkable since it did not involve parents. Meanwhile, New Zealand First leader Winston Peters expressed support for a social media age restriction but said the bill should be subject to a select committee inquiry. The opposition Labour Party leader Chris Hipkins has expressed interest in a social media age restriction legislation but emphasised the need for consensus. Meanwhile, Green Party co-leader Chlöe Swarbrick said she wanted to learn more about the bill but described it as simplistic. Fellow Greens co-leader Marama Davidson said that the proposed bill would punish children and young people for the harm caused by big tech platforms. === Tech companies === In early October 2025, representatives of TikTok and Meta Platforms cautioned against proposed social media ban on under-16 years olds. During a one-day parliamentary inquiry, Ella Woods-Joyce, TikTok's public policy lead for Australia and New Zealand, and Mia Garlick, Meta's regional director of policy, expressed concern that the social media age restriction could send children and young people to less regulated online spaces. Woods-Joyce highlighted TikTok's policy of closing down accounts belonging to users under the age of 13 years while Garlick highlighted Meta's policy of placing users under the age of 16 in private accounts by default. In early February 2026 Meta's vice president and global head of safety, Antigone Da
Read more →
Galaksija BASIC

Galaksija BASIC was the BASIC interpreter of the Galaksija build-it-yourself home computer from Yugoslavia. While being partially based on code taken from TRS-80 Level 1 BASIC, which the creator believed to have been a Microsoft BASIC, the extensive modifications of Galaksija BASIC—such as to include rudimentary array support, video generation code (as the CPU itself did it in absence of dedicated video circuitry) and generally improvements to the programming language—is said to have left not much more than flow-control and floating point code remaining from the original. The core implementation of the interpreter was fully contained in the 4 KiB ROM "A" or "1". The computer's original mainboard had a reserved slot for an extension ROM "B" or "2" that added more commands and features such as a built-in Zilog Z80 assembler. == ROM "A"/"1" symbols and keywords == The core implementation, in ROM "A" or "1", contained 3 special symbols and 32 keywords: ! begins a comment (equivalent of standard BASIC REM command) # Equivalent of standard BASIC DATA statement & prefix for hex numbers ARR$(n) Allocates an array of strings, like DIM, but can allocate only array with name A$ BYTE serves as PEEK when used as a function (e.g. PRINT BYTE(11123)) and POKE when used as a command (e.g. BYTE 11123,123). CALL n Calls BASIC subroutine as GOSUB in most other BASICs (e.g. CALL 100+4X) CHR$(n) converts an ASCII numeric code into a corresponding character (string) DOT x, y draws (command) or inspects (function) a pixel at given coordinates (0<=x<=63, 0<=y<=47). DOT displays the clock or time controlled by content of Y$ variable. Not in standard ROM EDIT n causes specified program line to be edited ELSE standard part of IF-ELSE construct (Galaksija did not use THEN) EQ compare alphanumeric values X$ and Y$ FOR standard FOR loop GOTO standard GOTO command HOME equivalent of standard BASIC CLS command - clears the screen HOME n protects n characters from the top of the screen from being scrolled away IF standard part of IF-ELSE construct (Galaksija did not use THEN) INPUT user entry of variable INT(n) a function that returns the greatest integer value equal to or lesser than n KEY(n) test whether a particular keyboard key is pressed LIST lists the program. Optional numeric argument specifies the first line number to begin listing with. MEM returns memory consumption data (need details here) NEW clears the current BASIC program NEW n clears BASIC program and moves beginning of BASIC area NEXT standard terminator of FOR loop OLD loads a program from tape OLD n loads program to different address PTR Returns address of the variable PRINT Printing numeric or string expression. RETURN Return from BASIC subroutine RND function (takes no arguments) that returns a random number between 0 and 1. RUN runs (executes) BASIC program. Optional numeric argument specifies the line number to begin execution with. SAVE saves a program to tape. Optional two arguments specify memory range to be saved (need details here). STEP standard part of FOR loop STOP stops execution of BASIC program TAKE replacement for READ and RESTORE. If the parameter is variable name, acts as READ, if it is number, acts as RESTORE UNDOT x, y "undraws" (resets) at specified coordinates (see DOT) UNDOT Stops the clock, not part of ROM USR Calls machine code subroutine WORD Double byte PEEK and POKE == ROM "B"/"2" additional symbols and keywords == The extended BASIC features, in ROM "B" or "2", contained one extra reserved symbol and 22 extra keywords: % /LABEL ABS(x) ARCTG(x) COS(x) COSD(x) DEL DUMP EXP(x) INP(x) LDUMP LLIST LN(x) LPRINT OUT PI POW(x,y) REN SIN(x), SIND(x) SQR(x) TG(x) TGD(x)
Read more →
Change data capture

In databases, change data capture (CDC) is a set of software design patterns used to determine and track the data that has changed (the "deltas") so that action can be taken using the changed data. The result is a delta-driven dataset. CDC is an approach to data integration that is based on the identification, capture and delivery of the changes made to enterprise data sources. For instance it can be used for incremental update of data loading. CDC occurs often in data warehouse environments since capturing and preserving the state of data across time is one of the core functions of a data warehouse, but CDC can be utilized in any database or data repository system. == Methodology == System developers can set up CDC mechanisms in a number of ways and in any one or a combination of system layers from application logic down to physical storage. In a simplified CDC context, one computer system has data believed to have changed from a previous point in time, and a second computer system needs to take action based on that changed data. The former is the source, the latter is the target. It is possible that the source and target are the same system physically, but that would not change the design pattern logically. Multiple CDC solutions can exist in a single system. === Timestamps on rows === Tables whose changes must be captured may have a column that represents the time of last change. Names such as LAST_UPDATE, LAST_MODIFIED, etc. are common. Any row in any table that has a timestamp in that column that is more recent than the last time data was captured is considered to have changed. Timestamps on rows are also frequently used for optimistic locking so this column is often available. === Version numbers on rows === Database designers give tables whose changes must be captured a column that contains a version number. Names such as VERSION_NUMBER, etc. are common. One technique is to mark each changed row with a version number. A current version is maintained for the table, or possibly a group of tables. This is stored in a supporting construct such as a reference table. When a change capture occurs, all data with the latest version number is considered to have changed. Once the change capture is complete, the reference table is updated with a new version number. (Do not confuse this technique with row-level versioning used for optimistic locking. For optimistic locking each row has an independent version number, typically a sequential counter. This allows a process to atomically update a row and increment its counter only if another process has not incremented the counter. But CDC cannot use row-level versions to find all changes unless it knows the original "starting" version of every row. This is impractical to maintain.) === Status indicators on rows === This technique can either supplement or complement timestamps and versioning. It can configure an alternative if, for example, a status column is set up on a table row indicating that the row has changed (e.g., a boolean column that, when set to true, indicates that the row has changed). Otherwise, it can act as a complement to the previous methods, indicating that a row, despite having a new version number or a later date, still shouldn't be updated on the target (for example, the data may require human validation). === Time/version/status on rows === This approach combines the three previously discussed methods. As noted, it is not uncommon to see multiple CDC solutions at work in a single system, however, the combination of time, version, and status provides a particularly powerful mechanism and programmers should utilize them as a trio where possible. The three elements are not redundant or superfluous. Using them together allows for such logic as, "Capture all data for version 2.1 that changed between 2005-06-01 00:00 and 2005-07-01 00:00 where the status code indicates it is ready for production." === Triggers on tables === May include a publish/subscribe pattern to communicate the changed data to multiple targets. In this approach, triggers log events that happen to the transactional table into another queue table that can later be "played back". For example, imagine an Accounts table, when transactions are taken against this table, triggers would fire that would then store a history of the event or even the deltas into a separate queue table. The queue table might have schema with the following fields: Id, TableName, RowId, Timestamp, Operation. The data inserted for our Account sample might be: 1, Accounts, 76, 2008-11-02 00:15, Update. More complicated designs might log the actual data that changed. This queue table could then be "played back" to replicate the data from the source system to a target. Data capture offers a challenge in that the structure, contents and use of a transaction log is specific to a database management system. Unlike data access, no standard exists for transaction logs. Most database management systems do not document the internal format of their transaction logs, although some provide programmatic interfaces to their transaction logs (for example: Oracle, DB2, SQL/MP, SQL/MX and SQL Server 2008). Other challenges in using transaction logs for change data capture include: Coordinating the reading of the transaction logs and the archiving of log files (database management software typically archives log files off-line on a regular basis). Translation between physical storage formats that are recorded in the transaction logs and the logical formats typically expected by database users (e.g., some transaction logs save only minimal buffer differences that are not directly useful for change consumers). Dealing with changes to the format of the transaction logs between versions of the database management system. Eliminating uncommitted changes that the database wrote to the transaction log and later rolled back. Dealing with changes to the metadata of tables in the database. CDC solutions based on transaction log files have distinct advantages that include: minimal impact on the database (even more so if one uses log shipping to process the logs on a dedicated host). no need for programmatic changes to the applications that use the database. low latency in acquiring changes. transactional integrity: log scanning can produce a change stream that replays the original transactions in the order they were committed. Such a change stream include changes made to all tables participating in the captured transaction. no need to change the database schema == Confounding factors == As often occurs in complex domains, the final solution to a CDC problem may have to balance many competing concerns. === Unsuitable source systems === Change data capture both increases in complexity and reduces in value if the source system saves metadata changes when the data itself is not modified. For example, some Data models track the user who last looked at but did not change the data in the same structure as the data. This results in noise in the Change Data Capture. === Tracking the capture === Actually tracking the changes depends on the data source. If the data is being persisted in a modern database then Change Data Capture is a simple matter of permissions. Two techniques are in common use: Tracking changes using database triggers Reading the transaction log as, or shortly after, it is written. If the data is not in a modern database, CDC becomes a programming challenge. === Push versus pull === Push: the source process creates a snapshot of changes within its own process and delivers rows downstream. The downstream process uses the snapshot, creates its own subset and delivers them to the next process. Pull: the target that is immediately downstream from the source, prepares a request for data from the source. The downstream target delivers the snapshot to the next target, as in the push model. === Alternatives === Sometimes the slowly changing dimension is used as an alternative method. CDC and SCD are similar in that both methods can detect changes in a data set. The most common forms of SCD are type 1 (overwrite), type 2 (maintain history) or 3 (only previous and current value). SCD 2 can be useful if history is needed in the target system. CDC overwrites in the target system (akin to SCD1), and is ideal when only the changed data needs to arrive at the target, i.e. a delta-driven dataset.
Read more →
Data governance

Data governance is a term used on both a macro and a micro level. The former is a political concept and forms part of international relations and Internet governance; the latter is a data management concept and forms part of corporate/organizational data governance. Data governance involves delegating authority over data and exercising that authority through decision-making processes. It plays a role in enhancing the value of data assets. == Macro level == Data governance at the macro level involves regulating cross-border data flows among countries, which is more precisely termed international data governance. This field was first formed in the early 2000s, and consists of "norms, principles and rules governing various types of data." There have been several international groups established by research organizations that aim to grant access to their data. These groups that enable an exchange of data are, as a result, exposed to domestic and international legal interpretations that ultimately decide how data is used. However, as of 2023, there are no international laws or agreements specifically focused on data protection. == Data governance (Data Management) == Data governance is the set of principles, policies, and processes that guide the effective and responsible use of data within an organization. It creates a framework for decision making, accountability, and oversight across the data lifecycle, from creation and storage to sharing and disposal. Data governance is closely linked with data management, which provides the practical methods to carry out governance objectives. These methods include data quality assurance, metadata management, master data management, security controls, and compliance monitoring. Together, governance and management aim to maximize the value of data as a strategic asset, reduce risks from misuse or inaccuracy, and ensure compliance with regulatory, ethical, and business requirements. The importance of this discipline has grown with the rise of big data, cloud computing, and artificial intelligence, where consistent standards and stewardship are essential for privacy protection, interoperability, and informed decision making. == Data governance drivers == While data governance initiatives can be driven by a desire to improve data quality, they are often driven by C-level leaders responding to external regulations. In a recent report conducted by the CIO WaterCooler community, 54% stated the key driver was efficiencies in processes; 39% - regulatory requirements; and only 7% customer service. Examples of these regulations include Sarbanes–Oxley Act, Basel I, Basel II, HIPAA, GDPR, cGMP, and a number of data privacy regulations. To achieve compliance with these regulations, business processes and controls require formal management processes to govern the data subject to these regulations. Successful programs identify drivers that are meaningful to both supervisory and executive leadership. Common themes among the external regulations center on the need to manage risk. The risks can be financial misstatement, inadvertent release of sensitive data, or poor data quality for key decisions. Methods to manage these risks vary from industry to industry. Examples of commonly referenced best practices and guidelines include COBIT, ISO/IEC 38500, and others. The proliferation of regulations and standards creates challenges for data governance professionals, particularly when multiple regulations overlap the data being managed. Organizations often launch data governance initiatives to address these challenges. == Data governance initiatives (Dimensions) == Data governance initiatives improve the quality of data by assigning a team responsible for data's accuracy, completeness, consistency, timeliness, validity, and uniqueness. This team usually consists of executive leadership, project management, line-of-business managers, and data stewards. The team usually employs a methodology for tracking and improving enterprise data, such as Six Sigma, and tools for data mapping, profiling, cleansing, and monitoring data. Data governance initiatives may be aimed at achieving a number of objectives including offering better visibility to internal and external customers (such as supply chain management), compliance with regulatory law, improving operations after rapid company growth or corporate mergers, or to aid the efficiency of enterprise knowledge workers by reducing confusion and error and increasing their scope of knowledge. Many data governance initiatives are also inspired by past attempts to fix information quality at the departmental level, which can lead to incongruent and redundant data quality processes. Most large companies have many applications and databases that can not easily share information. Therefore, knowledge workers within large organizations may not have access to the data they need to best do their jobs. When they do have access to the data, the data quality may be poor. By setting up a data governance practice or corporate data authority (individual or area responsible for determining how to proceed, in the best interest of the business, when a data issue arises), these problems can be mitigated. == Implementation == Implementation of a data governance initiative may vary in scope as well as origin. Sometimes, an executive mandate will arise to initiate an enterprise-wide effort. Sometimes the mandate will be to create a pilot project or projects, limited in scope and objectives, aimed at either resolving existing issues or demonstrating value. Sometimes, an initiative originates from lower down in the organization's hierarchy and will be deployed in a limited scope to demonstrate value to potential sponsors higher up in the organization. The initial scope of an implementation can vary greatly as well, from review of a one-off IT system to a cross-organization initiative. == Data governance tools == Leaders of successful data governance programs declared at the Data Governance Conference in Orlando, FL, in December 2006, that data governance is about 80 to 95 percent communication. That stated, it is a given that many of the objectives of a data governance program must be accomplished with appropriate tools. Many vendors are now positioning their products as data governance tools. Due to the different focus areas of various data governance initiatives, a given tool may or may not be appropriate. Additionally, many tools that are not marketed as governance tools address governance needs and demands.
Read more →