AI Analytics And Strategic Decision Making

AI Analytics And Strategic Decision Making — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

  • Per-pixel lighting

    Per-pixel lighting

    In computer graphics, per-pixel lighting refers to any technique for lighting an image or scene that calculates illumination for each pixel on a rendered image. This is in contrast to other popular methods of lighting such as vertex lighting, which calculates illumination at each vertex of a 3D model and then interpolates the resulting values over the model's faces to calculate the final per-pixel color values. Per-pixel lighting is commonly used with techniques, such as blending, alpha blending, alpha to coverage, anti-aliasing, texture filtering, clipping, hidden-surface determination, Z-buffering, stencil buffering, shading, mipmapping, normal mapping, bump mapping, displacement mapping, parallax mapping, shadow mapping, specular mapping, shadow volumes, high-dynamic-range rendering, ambient occlusion (screen space ambient occlusion, screen space directional occlusion, ray-traced ambient occlusion), ray tracing, global illumination, and tessellation. Each of these techniques provides some additional data about the surface being lit or the scene and light sources that contributes to the final look and feel of the surface. Most modern video game engines implement lighting using per-pixel techniques instead of vertex lighting to achieve increased detail and realism. The id Tech 4 engine, used to develop such games as Brink and Doom 3, was one of the first game engines to implement a completely per-pixel shading engine. All versions of the CryENGINE, Frostbite Engine, and Unreal Engine, among others, also implement per-pixel shading techniques. Deferred shading is a recent development in per-pixel lighting notable for its use in the Frostbite Engine and Battlefield 3. Deferred shading techniques are capable of rendering potentially large numbers of small lights inexpensively (other per-pixel lighting approaches require full-screen calculations for each light in a scene, regardless of size). == History == While only recently have personal computers and video hardware become powerful enough to perform full per-pixel shading in real-time applications such as games, many of the core concepts used in per-pixel lighting models have existed for decades. Frank Crow published a paper describing the theory of shadow volumes in 1977. This technique uses the stencil buffer to specify areas of the screen that correspond to surfaces that lie in a "shadow volume", or a shape representing a volume of space eclipsed from a light source by some object. These shadowed areas are typically shaded after the scene is rendered to buffers by storing shadowed areas with the stencil buffer. Jim Blinn first introduced the idea of normal mapping in a 1978 SIGGRAPH paper. Blinn pointed out that the earlier idea of unlit texture mapping proposed by Edwin Catmull was unrealistic for simulating rough surfaces. Instead of mapping a texture onto an object to simulate roughness, Blinn proposed a method of calculating the degree of lighting a point on a surface should receive based on an established "perturbation" of the normals across the surface. == Hardware rendering == Real-time applications, such as video games, usually implement per-pixel lighting through the use of pixel shaders, allowing the GPU hardware to process the effect. The scene to be rendered is first rasterized onto a number of buffers storing different types of data to be used in rendering the scene, such as depth, normal direction, and diffuse color. Then, the data is passed into a shader and used to compute the final appearance of the scene, pixel-by-pixel. Deferred shading is a per-pixel shading technique that has recently become feasible for games. With deferred shading, a "g-buffer" is used to store all terms needed to shade a final scene on the pixel level. The format of this data varies from application to application depending on the desired effect, and can include normal data, positional data, specular data, diffuse data, emissive maps and albedo, among others. Using multiple render targets, all of this data can be rendered to the g-buffer with a single pass, and a shader can calculate the final color of each pixel based on the data from the g-buffer in a final "deferred pass". Because deferred shading assumes only one visible fragment per pixel sample, transparent objects are generally handled in a separate forward pass. == Software rendering == Per-pixel lighting is also performed in software on many high-end commercial rendering applications which typically do not render at interactive framerates. This is called offline rendering or software rendering. NVidia's mental ray rendering software, which is integrated with such suites as Autodesk's Softimage is a well-known example.

    Read more →
  • Scripped

    Scripped

    Scripped was an online screenplay services company offering three services: script writing, script registration, and script coverage. Scripped did not facilitate collaboration among screenwriters. It combined with Zhura in 2010. According to Techcrunch, Scripped had more than 60,000 writers as of March 2010. Scripped was administered by Sunil Rajaraman, Ryan Buckley and Zak Freer. Actor, writer, and director Edward Burns and screenwriter Steven E. de Souza joined Scripped's Board of Advisers in May 2008. In 2008, the company formed a partnership with Write Brothers, makers of Movie Magic Screenwriter software. On March 29, 2010, Scripped announced that it closed $250,000 in private investment and merged with competitor Zhura. Scripped's CEO, Sunil Rajaraman, remains the merged company's Chief Executive Officer. On April 1, 2015, citing a serious technical failure, Scripped shuttered its service. As part of the announcement, it was disclosed that their backup servers had failed as well, losing all of its users' stored scripts. The website URL currently redirects to WriterDuet's website, another online scriptwriting service; Scripped had advertised WriterDuet in Scripped's shutdown open letter. == Features == The Scripped Writer provided a built-in screenplay template which formatted the document to a standard for scripts as recommended by the AMPAS. The screenplay document was composed of seven elements: scene, action, character, dialog, parenthetical, transition and general. Each element had a specific style to which the Scripped Writer conformed as text was entered. Like other client-side screenplay software, Scripped offered Tab-Enter toggling between screenplay elements, making the writing process much faster. Text files could be imported into the Scripped Writer and automatically conformed to the screenplay template. Completed scripts could be exported as PDF files. In May 2011 the administrators of Scripped launched Scripted.com - a sister site focused on freelance writing jobs. Subsequent to the service's launch, the company was renamed to Scripted, Inc.

    Read more →
  • Color reproduction

    Color reproduction

    Color reproduction is an aspect of color science concerned with producing light spectra that evoke a desired color, either through additive (light emitting) or subtractive (surface color) models. It converts physical correlates of color perception (CIE 1931 XYZ color space tristimulus values and related quantities) into light spectra that can be experienced by observers. In this way, it is the opposite of colorimetry. It is concerned with the faithful reproduction of a color in one medium, with a color in another, so it is a central concept in color management and relies heavily on color calibration. For example, food packaging must be able to faithfully reproduce the colors of the foods therein in order to appeal to a customer. This involves proper color calibration of at least four devices: Lighting, which must have a high color rendering index and not give a color cast to the object. Camera, which measures the reflected spectrum of the object and converts to a trichromatic color space (e.g. RGB). Screen, which reproduces color so a designer can proof the captured image and make color corrections as necessary. Printer, which reproduces the final color on paper.

    Read more →
  • Cloud printing

    Cloud printing

    There are, in essence, three kinds of Cloud printing. == Benefits == 76% of IT teams have moved, or plan to move, their print workflows to the cloud due to its simplicity. Consumers can print easily to any printer from their PC, tablet or smartphone, while the Cloud print service monitors the supplies level. Many printer vendors such as Lexmark propose an automatic supplies shipment based on the real-time analysis of the printer supplies and user behavior to ensure printing will always be possible. For IT department, Cloud Printing eliminates the need for print servers and represents the only way to print from Cloud virtual desktops and servers. For consumers, cloud ready printers eliminate the need for PC connections and print drivers, enabling them to print from mobile devices. As for publishers and content owners, cloud printing allows them to "avoid the cost and complexity of buying and managing the underlying hardware, software and processes" required for the production of professional print products. Leveraging cloud print for print on demand also allows businesses to cut down on the costs associated with mass production. Moreover, cloud printing can be considered more eco-friendly, as it significantly reduces the amount of paper used (13% reduction in print jobs yearly) and lowers carbon emissions from transportation. As many companies move their IT to the Cloud, some adopting the Windows 365 and Azure Virtual Desktop services from Microsoft, the connection from the Cloud environment to the on-premise printers become an issue as opening ports for incoming print flow traffic is not an option. In 2020, at the exact same time Google discontinued its Google Print offer, Microsoft has announced its Universal Print service offer, aimed at making printing compatible with Cloud Desktop environments, making printing driver-free and simple with no client to install on PC. With Universal Print Microsoft has built a disrupting architecture with a value proposition commodifying printers, removing print servers and drivers, allowing to move printers to VLAN for security purpose and printing from anywhere. Clients are free to use any printer from any model as they all work the same, clients are not tied anymore to any printer brand and that gave a significant boost to the Cloud print market. That Microsoft Universal Print architecture provides APIs to third-party developers who can develop add-ons such as Celiveo 365 to extend Microsoft Cloud Print with added features such as access control on printers and copiers, follow-me pull print, data encryption, advanced usage reporting or charge back. == Providers of Consumer Cloud Printing Solutions == Before 2020 only a handful of providers used to work towards a professional cloud print solution, operating in their own niche or focus on mobile devices. In 2020 Microsoft has boosted that market by announcing its Universal Print Cloud printing service and since then many publishers have started to propose solutions for that growing market. The Covid pandemic also created the need for employees to be able to print at home when using the corporate IT software. Closed VPN often prevent accessing home network printers from corporate laptops and Full Public Cloud solutions are meant to be a solution to that problem. After the decision by Google to terminate Google Cloud Print service on 31 December 2020, most printer vendors released their own mobile cloud solution to fill the gap, while Hewlett-Packard implemented its own cloud print with their ePrint solution. Those solutions are often proprietary, only working on printers proposed by the vendor. Google has decided to let third-party developers develop Cloud Print solutions and to limit its scope to certifying the best Print Management offers compatible with its Chrome Enterprise Cloud ecosystem. == Providers of Corporate Cloud Printing solutions == While many print solutions claim to be "Cloud Printing", there are actually three categories: full Private Cloud, full Public Cloud, and Hybrid Cloud. Their differences are real and have an impact on the overall TCO as the more software there is on-site, the more hidden cost there are. In the Full Public Cloud category, independent SaaS vendors like Celiveo, ezeep , Printix , and Y Soft support a wide range of printer brands and models, allowing clients to buy the best printer without being locked on any brand. They are leveraging cloud computing technology to offer cloud-based print infrastructure and cloud-based printing software as a Service (SaaS). These solutions have integrations to cloud enabled printers or provide embedded printer agents. They feature allow users to print to any printer in any network, isolated network or not, even if that printer is otherwise not reachable from the user's computer. This also allows IT departments to move printers to VLAN for maximum security, like what they are doing with IP phones. Google Chrome Enterprise Cloud ecosystem has its own technical particularities and Google certifies Print Management solutions, ensuring they comply with Google technical requirement, yet letting each solution differentiate from others with specific features or security. Many of solutions for Chrome Enterprise are Hybrid, a few are Full Public Cloud. Industry experts believe that as these services become more popular, users will no longer consider printers as necessary assets but rather as devices that they can access on demand when the need to generate a printed page presents itself. == Caveats of Cloud Printing == == Security == Print jobs flow through Public Internet. It is therefore important to verify no Man-in-the-Middle attack can be performed. The only technical solution is to ensure each printer and PC uses a non-self-generated cryptographic token or certificate allowing TLS mutual authentication and specific data encryption. Self-generated printer certificates are unknown from the Cloud and prevent trusted authentication. Microsoft has implemented its Zero Trust Access security in its Universal Print service, it generates a unique certificate on printers compatible with its service. Other Cloud Printing SaaS providers have followed Microsoft on that High Security path. Print jobs data stored on the Cloud is sensitive as it contains user information as well as all information appearing on pages. Good practices require such data is encrypted at rest and in motion, using asymmetric PKI keys instead of fixed encryption keys. Some solutions require to open incoming traffic ports on the firewall to let Cloud services communicate with printers attached behind that firewall (most of the time for IPP/IPPS flows), some other solutions use a pull model where the communication is always initiated by the printer and no firewall port needs to be open. In terms of security the later is to be preferred.

    Read more →
  • Serverless computing

    Serverless computing

    Serverless computing is "a cloud service category where the customer can use different cloud capability types without the customer having to provision, deploy and manage either hardware or software resources, other than providing customer application code or providing customer data. Serverless computing represents a form of virtualized computing", according to ISO/IEC 22123-2. Serverless computing is a broad ecosystem that includes the cloud provider, function as a service (FaaS), managed services, tools, frameworks, engineers, stakeholders, and other interconnected elements. == Overview == Serverless is a misnomer in the sense that servers are still used by cloud service providers to execute code for developers. The definition of serverless computing has evolved over time, leading to varied interpretations. According to Ben Kehoe, serverless represents a spectrum rather than a rigid definition. Emphasis should shift from strict definitions and specific technologies to adopting a serverless mindset, focusing on leveraging serverless solutions to address business challenges. Serverless computing does not eliminate complexity but shifts much of it from the operations team to the development team. However, this shift is not absolute, as operations teams continue to manage aspects such as identity and access management (IAM), networking, security policies, and cost optimization. Additionally, while breaking down applications into finer-grained components can increase management complexity, the relationship between granularity and management difficulty is not strictly linear. There is often an optimal level of modularization where the benefits outweigh the added management overhead. According to Yan Cui, serverless techniques should be adopted only when they help to deliver customer value faster. And while adopting, organizations should take small steps and de-risk along the way. == Challenges == Serverless applications are prone to fallacies of distributed computing. In addition, they are prone to the following fallacies: Versioning is simple Compensating transactions always work Observability is optional === Monitoring and debugging === Monitoring and debugging serverless applications can present unique challenges due to their distributed, event-driven nature and proprietary environments. Traditional tools may fall short, making it difficult to track execution flows across services. However, modern solutions such as distributed tracing tools (e.g., AWS X-Ray, Datadog), centralized logging, and cloud-agnostic observability platforms are mitigating these challenges. Emerging technologies like OpenTelemetry, AI-powered anomaly detection, and serverless-specific frameworks are further improving visibility and root cause analysis. While challenges persist, advancements in monitoring and debugging tools are steadily addressing these limitations. === Security === According to OWASP, serverless applications are vulnerable to variations of traditional attacks, insecure code, and some serverless-specific attacks (like denial of wallet). So, the risks have changed and attack prevention requires a shift in mindset. === Vendor lock-in === Serverless computing is provided as a third-party service. Applications and software that run in the serverless environment are by default locked to a specific cloud vendor. This issue is exacerbated in serverless computing, as with its increased level of abstraction, public vendors only allow customers to upload code to a FaaS platform without the authority to configure underlying environments. More importantly, when considering a more complex workflow that includes backend-as-a-service (BaaS), a BaaS offering can typically only natively trigger a FaaS offering from the same provider. This makes the workload migration in serverless computing virtually impossible. Therefore, considering how to design and deploy serverless workflows from a multi-cloud perspective could mitigate this. == High-performance computing == Serverless computing may not be ideal for certain high-performance computing (HPC) workloads due to resource limits often imposed by cloud providers, including maximum memory, CPU, and runtime restrictions. For workloads requiring sustained or predictable resource usage, bulk-provisioned servers can sometimes be more cost-effective than the pay-per-use model typical of serverless platforms. However, serverless computing is increasingly capable of supporting specific HPC workloads, particularly those that are highly parallelizable and event-driven, by leveraging its scalability and elasticity. The suitability of serverless computing for HPC continues to evolve with advancements in cloud technologies. == Anti-patterns == The grain of sand anti-pattern refers to the creation of excessively small components (e.g., functions) within a system, often resulting in increased complexity, operational overhead, and performance inefficiencies. Lambda pinball is a related anti-pattern that can occur in serverless architectures when functions (e.g., AWS Lambda, Azure functions) excessively invoke each other in fragmented chains, leading to latency, debugging and testing challenges, and reduced observability. These anti-patterns are associated with the formation of a distributed monolith. These anti-patterns are often addressed through the application of clear domain boundaries, which distinguish between public and published interfaces. Public interfaces are technically accessible interfaces, such as methods, classes, API endpoints, or triggers, but they do not come with formal stability guarantees. In contrast, published interfaces involve an explicit stability contract, including formal versioning, thorough documentation, a defined deprecation policy, and often support for backward compatibility. Published interfaces may also require maintaining multiple versions simultaneously and adhering to formal deprecation processes when breaking changes are introduced. Fragmented chains of function calls are often observed in systems where serverless components (functions) interact with other resources in complex patterns, sometimes described as spaghetti architecture or a distributed monolith. In contrast, systems exhibiting clearer boundaries typically organize serverless components into cohesive groups, where internal public interfaces manage inter-component communication, and published interfaces define communication across group boundaries. This distinction highlights differences in stability guarantees and maintenance commitments, contributing to reduced dependency complexity. Additionally, patterns associated with excessive serverless function chaining are sometimes addressed through architectural strategies that emphasize native service integrations instead of individual functions, a concept referred to as the functionless mindset. However, this approach is noted to involve a steeper learning curve, and integration limitations may vary even within the same cloud vendor ecosystem. Reporting on serverless databases presents challenges, as retrieving data for a reporting service can either break the bounded contexts, reduce the timeliness of the data, or do both. This applies regardless of whether data is pulled directly from databases, retrieved via HTTP, or collected in batches. Mark Richards refers to this as the reach-in reporting anti-pattern. A possible alternative to this approach is for databases to asynchronously push the necessary data to the reporting service instead of the reporting service pulling it. While this method requires a separate contract between services and the reporting service and can be complex to implement, it helps preserve bounded contexts while maintaining a high level of data timeliness. == Principles == Adopting DevSecOps practices can help improve the use and security of serverless technologies. In serverless applications, the distinction between infrastructure and business logic is often blurred, with applications typically distributed across multiple services. To maximize the effectiveness of testing, integration testing is emphasized for serverless applications. Additionally, to facilitate debugging and implementation, orchestration is used within the bounded context, while choreography is employed between different bounded contexts. Ephemeral resources are typically kept together to maintain high cohesion. However, shared resources with long spin-up times, such as AWS RDS clusters and landing zones, are often managed in separate repositories, deployment pipeline, and stacks.

    Read more →
  • MeeMix

    MeeMix

    MeeMix Ltd is a company specializing in personalizing media-related content recommendations, discovery and advertising for the telecommunication industry, founded in 2006. On January 1, 2008, MeeMix launched meemix.com, a public personalized internet radio serving as an online testbed for the development of music taste-prediction technologies. Subsequently, MeeMix released in 2009 a line of Business-to-business commercial services intended to personalize media recommendations, discovery and advertising. MeeMix hybrid taste-prediction technology relies on integrating machine learning algorithms, digital signal processing, behavior analysis, metadata analysis and collaborative filtering, and is provided via API web service. In August 2009, MeeMix was announced as Innovator Nominee in the GSM Association’s Mobile Innovation Grand Prix worldwide contest. As of 2013, MeeMix no longer features internet radios on meemix.com. On Sep 28, 2014, meemix.com went offline.

    Read more →
  • Spike (application)

    Spike (application)

    Spike is a cross-platform email client and AI-powered communication app, available on Windows, MacOS, iOS, Android and the web. It has a chat-like, conversational view for emails with AI-powered inbox management and integrated collaboration features. Depending on the selected plan, it can be used solely as an email application or as a full suite of business communication tools. == History == Founded in 2013 by Erez Pilosof and Dvir Ben-Aroya, Spike is a software application that puts existing e-mails into a multimedia messaging, chat-like interface enhanced with video and voice calls. The application was initially named Hop. In 2019, the developers completed a $5 million funding round including investment from Wix.com and NFX Capital. In 2020, Spike raised $8m in a Series A funding round led by Insight Partners with the participation from previous rounds' investors. In 2021 Spike announced a collaboration with Meta to launch on the Oculus Store and would become one of the first productivity apps to launch in Meta's new virtual world, known as the Metaverse. In June 2023, the company introduced its corporate offering — Teamspace, a corporate communication platform for teams with features such as company-wide channels for broad conversations, private groups for specific topics or projects, direct one-on-one conversations, video meetings, file collaboration, AI-powered email messaging, and custom email domain. It supports file management, search capabilities, and project management. Built on open-protocol technology, Spike Teamspace enables users to send and receive messages from all email providers. Regardless of whether the other party is using Spike. == Company operations == Spike is developed and operated by SpikeNow LTD. Dvir Ben Aroya serves as Spike’s CEO and Erez Pilosof is the CTO. The company is headquartered in Tel Aviv, Israel. == Mode of use == The app enables users to organize email into three types of "conversations,"a traditional inbox/sent format, by subject, or by people. Spike users can also make audio and video calls to each other, and other features include a calendar, contact list, and Groups. Spike is available for Microsoft Windows, MacOS, iOS and Android, and as a web version, and works with Gmail, Outlook, Exchange, iCloud, Yahoo! Mail and IMAP email providers. == Features == Since 2023, the platform features an AI-driven assistant, Magic AI, for customized email creation, document summarization, research, content generation, advanced note-taking, project management, and real-time translation. Since 2023, Spike offers custom email domain management. It supports team collaboration through Channels, uniting members globally with access to historical messages, and combines email with real-time messaging via Conversational Email. The Shared Inbox allows team collaboration on emails, while Groups support private conversations and invitations. It also features integrated video meetings, real-time collaboration on documents and notes, and email hosting with custom domains. Super Search enables retrieval of various content, and the Priority Inbox organizes emails by priority. Collaborative Tasks offer real-time updates and tracking. The platform allows voice message sending from mobile devices and integrates multiple calendar platforms into a unified schedule. File Management optimizes attachment handling, and the Unified Inbox consolidates emails from multiple accounts. Spike ensures data security with AES-256 encryption and private keys. The platform features AI-powered inbox management and communication tools. In May 2025, Spike launched its AI Feed feature, which automatically summarizes unread messages in a unified stream and enables bulk email actions. Additional AI capabilities include email composition assistance, document summarization, content generation, note-taking enhancement, and real-time translation.

    Read more →
  • RCUDA

    RCUDA

    rCUDA, which stands for Remote CUDA, is a type of middleware software framework for remote GPU virtualization. Fully compatible with the CUDA application programming interface (API), it allows the allocation of one or more CUDA-enabled GPUs to a single application. Each GPU can be part of a cluster or running inside of a virtual machine. The approach is aimed at improving performance in GPU clusters that are lacking full utilization. GPU virtualization reduces the number of GPUs needed in a cluster, and in turn, leads to a lower cost configuration – less energy, acquisition, and maintenance. The recommended distributed acceleration architecture is a high performance computing cluster with GPUs attached to only a few of the cluster nodes. When a node without a local GPU executes an application needing GPU resources, remote execution of the kernel is supported by data and code transfers between local system memory and remote GPU memory. rCUDA is designed to accommodate this client-server architecture. On one end, clients employ a library of wrappers to the high-level CUDA Runtime API, and on the other end, there is a network listening service that receives requests on a TCP port. Several nodes running different GPU-accelerated applications can concurrently make use of the whole set of accelerators installed in the cluster. The client forwards the request to one of the servers, which accesses the GPU installed in that computer and executes the request in it. Time-multiplexing the GPU, or in other words sharing it, is accomplished by spawning different server processes for each remote GPU execution request. == rCUDA v20.07 == The rCUDA middleware enables the concurrent usage of CUDA-compatible devices remotely. rCUDA employs either the InfiniBand network or the socket API for the communication between clients and servers. rCUDA can be useful in three different environments: Clusters. To reduce the number of GPUs installed in High Performance Clusters. This leads to energy savings, as well as other related savings like acquisition costs, maintenance, space, cooling, etc. Academia. In commodity networks, to offer access to a few high performance GPUs concurrently to many students. Virtual Machines. To enable the access to the CUDA facilities on the physical machine. The current version of rCUDA (v20.07) supports CUDA version 9.0, excluding graphics interoperability. rCUDA v20.07 targets the Linux OS (for 64-bit architectures) on both client and server sides. CUDA applications do not need any change in their source code in order to be executed with rCUDA.

    Read more →
  • Region Based Convolutional Neural Networks

    Region Based Convolutional Neural Networks

    Region-based Convolutional Neural Networks (R-CNN) are a family of machine learning models for computer vision, and specifically object detection and localization. The original goal of R-CNN was to take an input image and produce a set of bounding boxes as output, where each bounding box contains an object and also the category (e.g. car or pedestrian) of the object. In general, R-CNN architectures perform selective search over feature maps outputted by a CNN. R-CNN has been extended to perform other computer vision tasks, such as: tracking objects from a drone-mounted camera, locating text in an image, and enabling object detection in Google Lens. Mask R-CNN is also one of seven tasks in the MLPerf Training Benchmark, which is a competition to speed up the training of neural networks. == History == The following covers some of the versions of R-CNN that have been developed. November 2013: R-CNN. April 2015: Fast R-CNN. June 2015: Faster R-CNN. March 2017: Mask R-CNN. December 2017: Cascade R-CNN is trained with increasing Intersection over Union (IoU, also known as the Jaccard index) thresholds, making each stage more selective against nearby false positives. June 2019: Mesh R-CNN adds the ability to generate a 3D mesh from a 2D image. == Architecture == For review articles see. === Selective search === Given an image (or an image-like feature map), selective search (also called Hierarchical Grouping) first segments the image by the algorithm in (Felzenszwalb and Huttenlocher, 2004), then performs the following: Input: (colour) image Output: Set of object location hypotheses L Segment image into initial regions R = {r1, ..., rn} using Felzenszwalb and Huttenlocher (2004) Initialise similarity set S = ∅ foreach Neighbouring region pair (ri, rj) do Calculate similarity s(ri, rj) S = S ∪ s(ri, rj) while S ≠ ∅ do Get highest similarity s(ri, rj) = max(S) Merge corresponding regions rt = ri ∪ rj Remove similarities regarding ri: S = S \ s(ri, r∗) Remove similarities regarding rj: S = S \ s(r∗, rj) Calculate similarity set St between rt and its neighbours S = S ∪ St R = R ∪ rt Extract object location boxes L from all regions in R === R-CNN === With R-CNN, prediction follows a two-step process. A preprocessing selective search step generates a large set of candidate objects (typically as many as 2000), known as regions of interest (ROI). These are forwarded to a CNN, which predicts an object class score and bounding box estimate, independently for each ROI. Importantly, the ROIs are heavily filtered to remove excess candidates. This is achieved using two mechanism. Filtering begins by removing ROIs assigned to the background category. This is a specialized category, which is scored by the CNN alongside other categories. An unfortunate reality is that remaining ROIs typically suffer from heavy duplication. Namely, multiple ROIs that cover same objects in the image are all assigned non-background categories. This is resolved by a heuristic non-maximum suppression (NMS) step. === Fast R-CNN === While the original R-CNN independently computed the neural network features on each of as many as two thousand regions of interest, Fast R-CNN runs the neural network once on the whole image. At the end of the network is a ROIPooling module, which slices out each ROI from the network's output tensor, reshapes it, and classifies it. As in the original R-CNN, the Fast R-CNN uses selective search to generate its region proposals. === Faster R-CNN === While Fast R-CNN used selective search to generate ROIs, Faster R-CNN integrates the ROI generation into the neural network itself. === Mask R-CNN === While previous versions of R-CNN focused on object detections, Mask R-CNN adds instance segmentation. Mask R-CNN also replaced ROIPooling with a new method called ROIAlign, which can represent fractions of a pixel.

    Read more →
  • Mobile cloud computing

    Mobile cloud computing

    Mobile Cloud Computing (MCC) is the combination of cloud computing and mobile computing to bring rich computational resources to mobile users, network operators, as well as cloud computing providers. The ultimate goal of MCC is to enable execution of rich mobile applications on a plethora of mobile devices, with a rich user experience. MCC provides business opportunities for mobile network operators as well as cloud providers. More comprehensively, MCC can be defined as "a rich mobile computing technology that leverages unified elastic resources of varied clouds and network technologies toward unrestricted functionality, storage, and mobility to serve a multitude of mobile devices anywhere, anytime through the channel of Ethernet or Internet regardless of heterogeneous environments and platforms based on the pay-as-you-use principle." == Architecture == MCC uses computational augmentation approaches (computations are executed remotely instead of on the device) by which resource-constraint mobile devices can utilize computational resources of varied cloud-based resources. In MCC, there are four types of cloud-based resources, namely distant immobile clouds, proximate immobile computing entities, proximate mobile computing entities, and hybrid (combination of the other three model). Giant clouds such as Amazon EC2 are in the distant immobile groups whereas cloudlet or surrogates are member of proximate immobile computing entities. Smartphones, tablets, handheld devices, and wearable computing devices are part of the third group of cloud-based resources which is proximate mobile computing entities. Vodafone, Orange and Verizon have started to offer cloud computing services for companies. == Challenges == In the MCC landscape, an amalgam of mobile computing, cloud computing, and communication networks (to augment smartphones) creates several complex challenges such as Mobile Computation Offloading, Seamless Connectivity, Long WAN Latency, Mobility Management, Context-Processing, Energy Constraint, Vendor/data Lock-in, Security and Privacy, Elasticity that hinder MCC success and adoption. === Open research issues === Although significant research and development in MCC is available in the literature, efforts in the following domains is still lacking: Architectural issues: A reference architecture for heterogeneous MCC environment is a crucial requirement for unleashing the power of mobile computing towards unrestricted ubiquitous computing. Energy-efficient transmission: MCC requires frequent transmissions between cloud platform and mobile devices, due to the stochastic nature of wireless networks, the transmission protocol should be carefully designed. Context-awareness issues: Context-aware and socially-aware computing are inseparable traits of contemporary handheld computers. To achieve the vision of mobile computing among heterogeneous converged networks and computing devices, designing resource-efficient environment-aware applications is an essential need. Live VM migration issues: Executing resource-intensive mobile application via Virtual Machine (VM) migration-based application offloading involves encapsulation of application in VM instance and migrating it to the cloud, which is a challenging task due to additional overhead of deploying and managing VM on mobile devices. Mobile communication congestion issues: Mobile data traffic is tremendously hiking by ever increasing mobile user demands for exploiting cloud resources which impact on mobile network operators and demand future efforts to enable smooth communication between mobile and cloud endpoints. Trust, security, and privacy issues: Trust is an essential factor for the success of the burgeoning MCC paradigm. It is because the data along with code/component/application/complete VM is offloaded to the cloud for execution. Moreover, just like software and mobile application piracy, the MCC application development models are also affected by the piracy issue. Pirax is known to be the first specialized framework for controlling application piracy in MCC requirements == MCC research groups and activities == Several academic and industrial research groups in MCC have been emerging since last few years. Some of the MCC research groups in academia with large number of researchers and publications include: MDC, Mobile and Distributed Computing research group is at Faculty of Computer and Information Science, King Saud University. MDC research group focuses on architectures, platforms, and protocols for mobile and distributed computing. The group has developed algorithms, tools, and technologies which offer energy efficient, fault tolerant, scalable, secure, and high performance computing on mobile devices. MobCC lab, Faculty of Computer Science and Information Technology, University Malaya. The lab was established in 2010 under the High Impact Research Grant, Ministry of Higher Education, Malaysia. It has 17 researchers and has track of 22 published articles in international conference and peer-reviewed CS journals. ICCLAB, Zürich University of Applied Sciences has a segment working on MCC. The InIT Cloud Computing Lab is a research lab within the Institute of Applied Information Technology (InIT) of Zürich University of Applied Sciences (ZHAW). It covers topic areas across the entire cloud computing technology stack. Mobile & Cloud Lab, Institute of Computer Science, University of Tartu. Mobile & Cloud Lab conducts research and teaching in the mobile computing and cloud computing domains. The research topics of the group include cloud computing, mobile application development, mobile cloud, mobile web services and migrating scientific computing and enterprise applications to the cloud. SmartLab, Data Management Systems Laboratory, Department of Computer Science, University of Cyprus. SmartLab is a first-of-a-kind open cloud of smartphones that enables a new line of systems-oriented mobile computing research. Mobile Cloud Networking: Mobile Cloud Networking (MCN) was an EU FP7 Large-scale Integrating Project (IP, 15m Euro) funded by the European Commission. The MCN project was launched in November 2012 for the period of 36 month. The project was coordinated by SAP Research and the ICCLab at the Zurich University of Applied Science. In total 19 partners from industry and academia established the first vision of Mobile Cloud Computing. The project was primarily motivated by an ongoing transformation that drives the convergence between the Mobile Communications and Cloud Computing industry enabled by the Internet and is considered the first pioneer in the area of Network Function Virtualization.

    Read more →
  • MeeMix

    MeeMix

    MeeMix Ltd is a company specializing in personalizing media-related content recommendations, discovery and advertising for the telecommunication industry, founded in 2006. On January 1, 2008, MeeMix launched meemix.com, a public personalized internet radio serving as an online testbed for the development of music taste-prediction technologies. Subsequently, MeeMix released in 2009 a line of Business-to-business commercial services intended to personalize media recommendations, discovery and advertising. MeeMix hybrid taste-prediction technology relies on integrating machine learning algorithms, digital signal processing, behavior analysis, metadata analysis and collaborative filtering, and is provided via API web service. In August 2009, MeeMix was announced as Innovator Nominee in the GSM Association’s Mobile Innovation Grand Prix worldwide contest. As of 2013, MeeMix no longer features internet radios on meemix.com. On Sep 28, 2014, meemix.com went offline.

    Read more →
  • Mentimeter

    Mentimeter

    Mentimeter (or Menti for short) is a Swedish company based in Stockholm that develops and maintains an eponymous app used to create presentations with real-time feedback. == Foundation and background == Based in Stockholm, Sweden, the Mentimeter app was started by Swedish entrepreneur Johnny Warström and Niklas Ingvar as a response to unproductive meetings. The initial start-up budget was $500,000 raised by a group of prominent investors, including Per Appelgren in 2014, following the market's tendency to invest in Scandinavia. The app also focuses on online collaboration for the education sector, allowing students or public members to answer questions anonymously. The app enables users to share knowledge and real-time feedback on mobile devices with presentations, polls or brainstorming sessions in classes, meetings, gatherings, conferences and other group activities. == Achievements == By 2021, Mentimeter had over 270 million users and was one of Sweden's fastest-growing startups. The company also ranked #10 on 20 Fastest Growing 500 Startups Batch 16 Companies. It was ranked Stockholm's fastest growing company of the 2018 edition of the DI Gasell Award. Mentimeter has a freemium business model.

    Read more →
  • Machine-learned interatomic potential

    Machine-learned interatomic potential

    Machine-learned interatomic potentials (MLIPs), or simply machine learning potentials (MLPs), are interatomic potentials constructed using machine learning. Beginning in the 1990s, researchers have employed such programs to construct interatomic potentials by mapping atomic structures to their potential energies. These potentials are referred to as MLIPs or MLPs. Such machine learning potentials promised to fill the gap between density functional theory, a highly accurate but computationally intensive modelling method, and empirically derived or intuitively-approximated potentials, which were far lighter computationally but substantially less accurate. Improvements in artificial intelligence technology heightened the accuracy of MLPs while lowering their computational cost, increasing the role of machine learning in fitting potentials. Machine learning potentials began by using neural networks to tackle low-dimensional systems. While promising, these models could not systematically account for interatomic energy interactions; they could be applied to small molecules in a vacuum, or molecules interacting with frozen surfaces, but not much else – and even in these applications, the models often relied on force fields or potentials derived empirically or with simulations. These models thus remained confined to academia. Modern neural networks construct highly accurate and computationally light potentials, as theoretical understanding of materials science was increasingly built into their architectures and preprocessing. Almost all are local, accounting for all interactions between an atom and its neighbor up to some cutoff radius. There exist some nonlocal models, but these have been experimental for almost a decade. For most systems, reasonable cutoff radii enable highly accurate results. Almost all neural networks intake atomic coordinates and output potential energies. For some, these atomic coordinates are converted into atom-centered symmetry functions. From this data, a separate atomic neural network is trained for each element; each atomic network is evaluated whenever that element occurs in the given structure, and then the results are pooled together at the end. This process – in particular, the atom-centered symmetry functions which convey translational, rotational, and permutational invariances – has greatly improved machine learning potentials by significantly constraining the neural network search space. Other models use a similar process but emphasize bonds over atoms, using pair symmetry functions and training one network per atom pair. Other models to learn their own descriptors rather than using predetermined symmetry-dictating functions. These models, called message-passing neural networks (MPNNs), are graph neural networks. Treating molecules as three-dimensional graphs (where atoms are nodes and bonds are edges), the model takes feature vectors describing the atoms as input, and iteratively updates these vectors as information about neighboring atoms is processed through message functions and convolutions. These feature vectors are then used to predict the final potentials. The flexibility of this method often results in stronger, more generalizable models. In 2017, the first-ever MPNN model (a deep tensor neural network) was used to calculate the properties of small organic molecules. == Gaussian Approximation Potential (GAP) == One popular class of machine-learned interatomic potential is the Gaussian Approximation Potential (GAP), which combines compact descriptors of local atomic environments with Gaussian process regression to machine learn the potential energy surface of a given system. To date, the GAP framework has been used to successfully develop a number of MLIPs for various systems, including for elemental systems such as carbon, silicon, phosphorus, and tungsten, as well as for multicomponent systems such as Ge2Sb2Te5 and austenitic stainless steel, Fe7Cr2Ni. == Equivariant graph neural networks == A significant limitation of early MPNNs was that they were not inherently equivariant to rotations and reflections of atomic structures — meaning predictions could change depending on how a molecule was oriented in space. Beginning around 2021, a new class of models addressed this by incorporating equivariance directly into the message-passing layers using spherical harmonics and irreducible representations. Notable examples include NequIP (2021), MACE (2022), and GemNet-OC (2022). These equivariant architectures proved substantially more data-efficient and accurate than their predecessors, and became the dominant paradigm for high-accuracy MLIPs. == Universal MLIPs and large-scale datasets == Early MLIPs were system-specific, trained on a few thousand structures of a single material. A major shift occurred with the creation of large, chemically diverse datasets enabling models that generalize across many elements, bonding environments, and application domains — so-called universal MLIPs. A key driver was the Open Catalyst Project (OC20, OC22), a collaboration between Meta AI (FAIR) and Carnegie Mellon University launched in 2020. OC20 comprises approximately 1.3 million DFT relaxations across 82 elements, designed to accelerate the discovery of catalysts for renewable energy applications. It was among the first datasets large enough to train GNNs that generalize across diverse chemical systems, and established a widely-used benchmark for the field. A subsequent dataset, Open Direct Air Capture (OpenDAC 2023 and OpenDAC 2025), applied the same approach to carbon capture, providing a large computational database of metal-organic frameworks and sorbent candidates evaluated for CO₂ capture, generated using nearly 400 million CPU hours of quantum chemistry calculations in collaboration with Georgia Tech. These datasets revealed a new challenge: the GNN architectures most effective for atomic simulations were memory-intensive, as they model higher-order interactions between triplets or quadruplets of atoms, making it difficult to scale model size. Graph Parallelism, introduced by Sriram et al. (ICLR 2022), addressed this by distributing a single input graph across multiple GPUs — a distinct strategy from data parallelism (which distributes training examples) or model parallelism (which distributes layers). This enabled training GNNs with hundreds of millions to billions of parameters for the first time. Building on these foundations, Meta FAIR released the Universal Model for Atoms (UMA) in 2025, trained on approximately 500 million unique 3D atomic structures spanning molecules, materials, and catalysts — the largest training run to date for an MLIP. UMA introduced a Mixture of Linear Experts (MoLE) architecture, enabling one model to learn from datasets generated by different DFT codes and settings without significant inference overhead. It matches or surpasses specialized models across catalysis, materials, and molecular benchmarks without task-specific fine-tuning, and has been described as marking a "pre/post-UMA" divide in the field. == Applications == Catalyst discovery: MLIPs have significantly accelerated the computational screening of heterogeneous catalysts by replacing expensive DFT relaxations with fast neural network surrogates. The Open Catalyst Project explicitly targets this application, aiming to identify new catalysts for green hydrogen production and other renewable energy reactions. Carbon capture: The OpenDAC project applies universal MLIPs to screening sorbent materials for direct air capture of CO₂, a key technology for climate change mitigation. AI-accelerated screening allows evaluation of orders of magnitude more candidate materials than traditional DFT workflows. Drug discovery and molecular design: MLIPs are increasingly used in pharmaceutical research to model molecular conformations and binding energies. The Open Molecules 2025 (OMol25) dataset, released by Meta FAIR in 2025, provides high-accuracy calculations for a large set of molecular systems to support this use case. Materials discovery: Universal MLIPs enable high-throughput screening of novel inorganic materials, including battery electrolytes, semiconductors, and superconductors, by rapidly estimating stability and properties across large chemical spaces.

    Read more →
  • EyeOS

    EyeOS

    eyeOS was a web desktop for cloud computing, whose main purpose is to enable collaboration and communication among users. It is mainly written in PHP, XML, and JavaScript. It is a private-cloud application platform with a web-based desktop interface. eyeOS delivers a whole desktop from the cloud with file management, personal management information tools, and collaborative tools, with the integration of the client's applications. == History == The first publicly available eyeOS version was released on August 1, 2005, as eyeOS 0.6.0 in Olesa de Montserrat, Barcelona (Spain). A worldwide community of developers soon took part in the project and helped improve it by translating, testing, and developing it. After two years of development, the eyeOS Team published eyeOS 1.0 on June 4, 2007. Compared with previous versions, eyeOS 1.0 introduced a complete reorganization of the code and some new web technologies, like eyeSoft, a portage-based web software installation system. Moreover, eyeOS also included the eyeOS Toolkit, a set of libraries allowing easy and fast development of new web applications. With the release of eyeOS 1.1 on July 2, 2007, eyeOS changed its license and migrated from GNU GPL Version 2 to Version 3. Version 1.2 was released just a month after the 1.1 version and integrated full compatibility with Microsoft Word files. eyeOS 1.5 Gala was released on January 15, 2008. This version was the first to support both Microsoft Office and OpenOffice.org file formats for documents, presentations, and spreadsheets. With this version, eyeOS also gained the ability to import and export documents in both formats using server-side scripting. eyeOS 1.6 was released on April 25, 2008, and included many improvements such as synchronization with local computers, drag and drop, a mobile version, and more. eyeOS 1.8 Lars was released on January 7, 2009, and featured a completely rewritten file manager and a new sound API to develop media-rich applications. Later, on April 1, 2009, 1.8.5 was released with a new default theme and some rewritten apps, such as the Word Processor and the Address Book. On July 13, 2009, 1.8.6 was released with an interface for the iPhone and a new version of eyeMail with support for POP3 and IMAP. eyeOS 1.9 was released on December 29, 2009. It was followed up with the 1.9.0.1 release with minor fixes on February 18, 2010. These releases were the last of the "classic desktop" interfaces. A major re-work was completed in March 2010, now called eyeOS 2.x. However, a small group of eyeOS developers still maintain the code within the eyeOS forum, where support is provided, but the eyeOS group itself has stopped active 1.x development. It is now available as the On-eye project on GitHub. Active development was halted on 1.x as of February 3, 2010. eyeOS 2.0 release took place on March 3, 2010. This was a total restructure of the operating system. The 2.x stable is the new series of eyeOS, which is in active development and will replace 1.x as stable in a few months. It includes live collaboration and more social capabilities than eyeOS 1.x. eyeOS then released 2.2.0.0 on July 28, 2010. On December 14, 2010, a working group inside the eyeOS open-source development community began the structure development and further upgrade of eyeOS 1.9.x. The group's main goal is to continue the work eyeOS has stopped on 1.9.x. eyeOS released 2.5 on May 17, 2011. This was the last release under an open source license. It is available on SourceForge for download under another project called eyeOS 2.5 Open Source Version. On April 1, 2014, Telefónica announced their acquisition of eyeOS. eyeOS would maintain its headquarters in the Catalonia, Spain, where their staff would continue to work but now as part of Telefónica. After its integration into Telefónica, eyeOS would continue to function as an independent subsidiary under CEO Michel Kisfaludi. == Structure and API == For developers, EyeOS provides the eyeOS Toolkit, a set of libraries and functions to develop applications for eyeOS. Using the integrated Portage-based eyeSoft system, one can create their own repository for eyeOS and distribute applications through it. Each core part of the desktop is its own application, using JavaScript to send server commands as the user interacts. As actions are performed using AJAX (such as launching an application), it sends event information to the server. The server then sends back tasks for the client to do in XML format, such as drawing a widget. On the server, eyeOS uses XML files to store information. This makes it simple for a user to set up on the server, as it requires zero configuration other than the account information for the first user, making it simple to deploy. To avoid bottlenecks that flat files present, each user's information and settings are stored in different files, preventing resource starvation from occurring, though this in turn may create issues in high volume user environments due to host operating system open file descriptor limits. == Professional edition == A Professional Edition of eyeOS was launched on September 15, 2011, as an operating system for businesses. It uses a new version number and was released under version 1.0 instead of continuing with the next version number in the open source project. The Professional Edition retains the web desktop interface used by the open source version while targeting enterprise users. A host of new features designed for enterprises, like file sharing and synchronization (called eyeSync), Active Directory/LDAP connectivity, system-wide administration controls, and a local file execution tool called eyeRun were introduced. A new suite of Web Apps (a mail client, calendar, instant messaging, and collaboration tools) was also introduced, specific to the enterprise edition for the web desktop. With eyeOS Professional Edition 1.1, a to-do task manager tool, Citrix XenApp integration, and a Facebook like 'wall' for collaboration were introduced. == Awards == 2007 – Received the Softpedia's Pick award. 2007 – Finalist at SourceForge's 2007 Community Choice Awards at the "Best Project" category. The winner for that category was 7-Zip. 2007 – Won the Yahoo! Spain Web Revelation award in the Technology category. 2008 – Finalist for the Webware 100 awards by CNET, under the "Browsing" category. 2008 – Finalist at the SourceForge's 2008 Community Choice Awards at the "Most Likely to Change the World" category. The winner for that category was Linux. 2009 – Selected Project of the Month (August 2009) by SourceForge. 2009 – BMW Innovation Award. 2010 – Winner of Accelera (Ernst & Young). 2010 – Asturias & Girona Spanish Prince award “IMPULSA”. 2011 – Winner of MIT's TR35 award as Innovator of the Year in Spain. == Community == eyeOS community is formed with the eyeOS forums, which reached 10,000 members on April 4, 2008; the eyeOS wiki; and the eyeOS Application Communities, available at the eyeOS-Apps website, hosted and provided by openDesktop.org as well as Softpedia.

    Read more →
  • Distributed file system for cloud

    Distributed file system for cloud

    A distributed file system for cloud is a file system that allows many clients to have access to data and supports operations (create, delete, modify, read, write) on that data. Each data file may be partitioned into several parts called chunks. Each chunk may be stored on different remote machines, facilitating the parallel execution of applications. Typically, data is stored in files in a hierarchical tree, where the nodes represent directories. There are several ways to share files in a distributed architecture: each solution must be suitable for a certain type of application, depending on how complex the application is. Meanwhile, the security of the system must be ensured. Confidentiality, availability and integrity are the main keys for a secure system. Users can share computing resources through the Internet thanks to cloud computing which is typically characterized by scalable and elastic resources – such as physical servers, applications and any services that are virtualized and allocated dynamically. Synchronization is required to make sure that all devices are up-to-date. Distributed file systems enable many big, medium, and small enterprises to store and access their remote data as they do local data, facilitating the use of variable resources. == Overview == === History === Today, there are many implementations of distributed file systems. The first file servers were developed by researchers in the 1970s. Sun Microsystem's Network File System became available in the 1980s. Before that, people who wanted to share files used the sneakernet method, physically transporting files on storage media from place to place. Once computer networks started to proliferate, it became obvious that the existing file systems had many limitations and were unsuitable for multi-user environments. Users initially used FTP to share files. FTP first ran on the PDP-10 at the end of 1973. Even with FTP, files needed to be copied from the source computer onto a server and then from the server onto the destination computer. Users were required to know the physical addresses of all computers involved with the file sharing. === Supporting techniques === Modern data centers must support large, heterogenous environments, consisting of large numbers of computers of varying capacities. Cloud computing coordinates the operation of all such systems, with techniques such as data center networking (DCN), the MapReduce framework, which supports data-intensive computing applications in parallel and distributed systems, and virtualization techniques that provide dynamic resource allocation, allowing multiple operating systems to coexist on the same physical server. === Applications === Cloud computing provides large-scale computing thanks to its ability to provide the needed CPU and storage resources to the user with complete transparency. This makes cloud computing particularly suited to support different types of applications that require large-scale distributed processing. This data-intensive computing needs a high performance file system that can share data between virtual machines (VM). Cloud computing dynamically allocates the needed resources, releasing them once a task is finished, requiring users to pay only for needed services, often via a service-level agreement. Cloud computing and cluster computing paradigms are becoming increasingly important to industrial data processing and scientific applications such as astronomy and physics, which frequently require the availability of large numbers of computers to carry out experiments. == Architectures == Most distributed file systems are built on the client-server architecture, but other, decentralized, solutions exist as well. === Client-server architecture === Network File System (NFS) uses a client-server architecture, which allows sharing of files between a number of machines on a network as if they were located locally, providing a standardized view. The NFS protocol allows heterogeneous clients' processes, probably running on different machines and under different operating systems, to access files on a distant server, ignoring the actual location of files. Relying on a single server results in the NFS protocol suffering from potentially low availability and poor scalability. Using multiple servers does not solve the availability problem since each server is working independently. The model of NFS is a remote file service. This model is also called the remote access model, which is in contrast with the upload/download model: Remote access model: Provides transparency, the client has access to a file. He sends requests to the remote file (while the file remains on the server). Upload/download model: The client can access the file only locally. It means that the client has to download the file, make modifications, and upload it again, to be used by others' clients. The file system used by NFS is almost the same as the one used by Unix systems. Files are hierarchically organized into a naming graph in which directories and files are represented by nodes. === Cluster-based architectures === A cluster-based architecture ameliorates some of the issues in client-server architectures, improving the execution of applications in parallel. The technique used here is file-striping: a file is split into multiple chunks, which are "striped" across several storage servers. The goal is to allow access to different parts of a file in parallel. If the application does not benefit from this technique, then it would be more convenient to store different files on different servers. However, when it comes to organizing a distributed file system for large data centers, such as Amazon and Google, that offer services to web clients allowing multiple operations (reading, updating, deleting,...) to a large number of files distributed among a large number of computers, then cluster-based solutions become more beneficial. Note that having a large number of computers may mean more hardware failures. Two of the most widely used distributed file systems (DFS) of this type are the Google File System (GFS) and the Hadoop Distributed File System (HDFS). The file systems of both are implemented by user level processes running on top of a standard operating system (Linux in the case of GFS). ==== Design principles ==== ===== Goals ===== Google File System (GFS) and Hadoop Distributed File System (HDFS) are specifically built for handling batch processing on very large data sets. For that, the following hypotheses must be taken into account: High availability: the cluster can contain thousands of file servers and some of them can be down at any time A server belongs to a rack, a room, a data center, a country, and a continent, in order to precisely identify its geographical location The size of a file can vary from many gigabytes to many terabytes. The file system should be able to support a massive number of files The need to support append operations and allow file contents to be visible even while a file is being written Communication is reliable among working machines: TCP/IP is used with a remote procedure call RPC communication abstraction. TCP allows the client to know almost immediately when there is a problem and a need to make a new connection. ===== Load balancing ===== Load balancing is essential for efficient operation in distributed environments. It means distributing work among different servers, fairly, in order to get more work done in the same amount of time and to serve clients faster. In a system containing N chunkservers in a cloud (N being 1000, 10000, or more), where a certain number of files are stored, each file is split into several parts or chunks of fixed size (for example, 64 megabytes), the load of each chunkserver being proportional to the number of chunks hosted by the server. In a load-balanced cloud, resources can be efficiently used while maximizing the performance of MapReduce-based applications. ===== Load rebalancing ===== In a cloud computing environment, failure is the norm, and chunkservers may be upgraded, replaced, and added to the system. Files can also be dynamically created, deleted, and appended. That leads to load imbalance in a distributed file system, meaning that the file chunks are not distributed equitably between the servers. Distributed file systems in clouds such as GFS and HDFS rely on central or master servers or nodes (Master for GFS and NameNode for HDFS) to manage the metadata and the load balancing. The master rebalances replicas periodically: data must be moved from one DataNode/chunkserver to another if free space on the first server falls below a certain threshold. However, this centralized approach can become a bottleneck for those master servers, if they become unable to manage a large number of file accesses, as it increases their already heavy loads. The load rebalance problem is NP-hard. In order to get a large number of chunkservers to work in collaboration, and to

    Read more →