Pattern language

A pattern language is an organized and coherent set of patterns, each of which describes a problem and the core of a solution that can be used in many ways within a specific field of expertise. The term was coined by architect Christopher Alexander and popularized by his 1977 book A Pattern Language. A pattern language can also be an attempt to express the deeper wisdom of what brings aliveness within a particular field of human endeavor, through a set of interconnected patterns. Aliveness is one placeholder term for "the quality that has no name": a sense of wholeness, spirit, or grace, that while of varying form, is precise and empirically verifiable. Alexander claims that ordinary people can use this design approach to successfully solve very large, complex design problems. == What is a pattern? == When a designer designs something – whether a house, computer program, or lamp – they must make many decisions about how to solve problems. A single problem is documented with its typical place (the syntax), and use (the grammar) with the most common and recognized good solution seen in the wild, like the examples seen in dictionaries. Each such entry is a single design pattern. Each pattern has a name, a descriptive entry, and some cross-references, much like a dictionary entry. A documented pattern should explain why that solution is good in the pattern's contexts. Elemental or universal patterns such as "door" or "partnership" are versatile ideals of design, either as found in experience or for use as components in practice, explicitly described as holistic resolutions of the forces in recurrent contexts and circumstances, whether in architecture, medicine, software development or governance, etc. Patterns might be invented or found and studied, such as the naturally occurring patterns of design that characterize human environments. Like all languages, a pattern language has vocabulary, syntax, and grammar – but a pattern language applies to some complex activity other than communication. In pattern languages for design, the parts break down in this way: The language description – the vocabulary – is a collection of named, described solutions to problems in a field of interest. These are called design patterns. So, for example, the language for architecture describes items like: settlements, buildings, rooms, windows, latches, etc. Each solution includes syntax, a description that shows where the solution fits in a larger, more comprehensive or more abstract design. This automatically links the solution into a web of other needed solutions. For example, rooms have ways to get light, and ways to get people in and out. The solution includes grammar that describes how the solution solves a problem or produces a benefit. So, if the benefit is unneeded, the solution is not used. Perhaps that part of the design can be left empty to save money or other resources; if people do not need to wait to enter a room, a simple doorway can replace a waiting room. In the language description, grammar and syntax cross index (often with a literal alphabetic index of pattern names) to other named solutions, so the designer can quickly think from one solution to related, needed solutions, and document them in a logical way. In Christopher Alexander's book A Pattern Language, the patterns are in decreasing order by size, with a separate alphabetic index. The web of relationships in the index of the language provides many paths through the design process. This simplifies the design work because designers can start the process from any part of the problem they understand and work toward the unknown parts. At the same time, if the pattern language has worked well for many projects, there is reason to believe that even a designer who does not completely understand the design problem at first will complete the design process, and the result will be usable. For example, skiers coming inside must shed snow and store equipment. The messy snow and boot cleaners should stay outside. The equipment needs care, so the racks should be inside. == Many patterns form a language == Just as words must have grammatical and semantic relationships to each other in order to make a spoken language useful, design patterns must be related to each other in position and utility order to form a pattern language. Christopher Alexander's work describes a process of decomposition, in which the designer has a problem (perhaps a commercial assignment), selects a solution, then discovers new, smaller problems resulting from the larger solution. Occasionally, the smaller problems have no solution, and a different larger solution must be selected. Eventually all of the remaining design problems are small enough or routine enough to be solved by improvisation by the builders, and the "design" is done. The actual organizational structure (hierarchical, iterative, etc.) is left to the discretion of the designer, depending on the problem. This explicitly lets a designer explore a design, starting from some small part. When this happens, it's common for a designer to realize that the problem is actually part of a larger solution. At this point, the design almost always becomes a better design. In the language, therefore, each pattern has to indicate its relationships to other patterns and to the language as a whole. This gives the designer using the language a great deal of guidance about the related problems that must be solved. The most difficult part of having an outside expert apply a pattern language is in fact to get a reliable, complete list of the problems to be solved. Of course, the people most familiar with the problems are the people that need a design. So, Alexander famously advocated on-site improvisation by concerned, empowered users, as a powerful way to form very workable large-scale initial solutions, maximizing the utility of a design, and minimizing the design rework. The desire to empower users of architecture was, in fact, what led Alexander to undertake a pattern language project for architecture in the first place. == Design problems in a context == An important aspect of design patterns is to identify and document the key ideas that make a good system different from a poor system (that may be a house, a computer program or an object of daily use), and to assist in the design of future systems. The idea expressed in a pattern should be general enough to be applied in very different systems within its context, but still specific enough to give constructive guidance. The range of situations in which the problems and solutions addressed in a pattern apply is called its context. An important part in each pattern is to describe this context. Examples can further illustrate how the pattern applies to very different situation. For instance, Alexander's pattern "A PLACE TO WAIT" addresses bus stops in the same way as waiting rooms in a surgery, while still proposing helpful and constructive solutions. The "Gang-of-Four" book Design Patterns by Gamma et al. proposes solutions that are independent of the programming language, and the program's application domain. Still, the problems and solutions described in a pattern can vary in their level of abstraction and generality on the one side, and specificity on the other side. In the end this depends on the author's preferences. However, even a very abstract pattern will usually contain examples that are, by nature, absolutely concrete and specific. Patterns can also vary in how far they are proven in the real world. Alexander gives each pattern a rating by zero, one or two stars, indicating how well they are proven in real-world examples. It is generally claimed that all patterns need at least some existing real-world examples. It is, however, conceivable to document yet unimplemented ideas in a pattern-like format. The patterns in Alexander's book also vary in their level of scale – some describing how to build a town or neighbourhood, others dealing with individual buildings and the interior of rooms. Alexander sees the low-scale artifacts as constructive elements of the large-scale world, so they can be connected to a hierarchic network. === Balancing of forces === A pattern must characterize the problems that it is meant to solve, the context or situation where these problems arise, and the conditions under which the proposed solutions can be recommended. Often these problems arise from a conflict of different interests or "forces". A pattern emerges as a dialogue that will then help to balance the forces and finally make a decision. For instance, there could be a pattern suggesting a wireless telephone. The forces would be the need to communicate, and the need to get other things done at the same time (cooking, inspecting the bookshelf). A very specific pattern would be just "WIRELESS TELEPHONE". More general patterns would be "WIRELESS DEVICE" or "SECONDARY ACTIVITY", suggesting that a secondary activity (such as talking on t

The Visualization Handbook

The Visualization Handbook is a textbook by Charles D. Hansen and Christopher R. Johnson that serves as a survey of the field of scientific visualization by presenting the basic concepts and algorithms in addition to a current review of visualization research topics and tools. It is commonly used as a textbook for scientific visualization graduate courses. It is also commonly cited as a reference for scientific visualization and computer graphics in published papers, with almost 500 citations documented on Google Scholar. == Table of Contents == PART I - Introduction Overview of Visualization - William J. Schroeder and Kenneth M. Martin PART II - Scalar Field Visualization: Isosurfaces Accelerated Isosurface Extraction Approaches -Yarden Livnat Time-Dependent Isosurface Extraction - Han-Wei Shen Optimal Isosurface Extraction - Paolo Cignoni, Claudio Montani, Robert Scopigno, and Enrico Puppo Isosurface Extraction Using Extrema Graphs - Takayuki Itoh and Koji Koyamada Isosurfaces and Level-Sets - Ross Whitaker PART III - Scalar Field Visualization: Volume Rendering Overview of Volume Rendering - Arie E. Kaufman and Klaus Mueller Volume Rendering Using Splatting - Roger Crawfis, Daqing Xue, and Caixia Zhang Multidimensional Transfer Functions for Volume Rendering - Joe Kniss, Gordon Kindlmann, and Charles D. Hansen Pre-Integrated Volume Rendering - Martin Kraus and Thomas Ertl Hardware-Accelerated Volume Rendering - Hanspeter Pfister PART IV - Vector Field Visualization Overview of Flow Visualization - Daniel Weiskopf and Gordon Erlebacher Flow Textures: High-Resolution Flow Visualization - Gordon Erlebacher, Bruno Jobard, and Daniel Weiskopf Detection and Visualization of Vortices - Ming Jiang, Raghu Machiraju, and David Thompson PART V - Tensor Field Visualization Oriented Tensor Reconstruction - Leonid Zhukov and Alan H. Barr Diffusion Tensor MRI Visualization - Song Zhang, David Laidlaw, and Gordon Kindlmann Topological Methods for Flow Visualization - Gerik Scheuermann and Xavier Tricoche PART VI - Geometric Modeling for Visualization 3D Mesh Compression - Jarek Rossignac Variational Modeling Methods for Visualization - Hans Hagen and Ingrid Hotz Model Simplification - Jonathan D. Cohen and Dinesh Manocha PART VII - Virtual Environments for Visualization Direct Manipulation in Virtual Reality - Steve Bryson The Visual Haptic Workbench - Milan Ikits and J. Dean Brederson Virtual Geographic Information Systems - William Ribarsky Visualization Using Virtual Reality - R. Bowen Loftin, Jim X. Chen, and Larry Rosenblum PART VIII - Large-Scale Data Visualization Desktop Delivery: Access to Large Datasets - Philip D. Heermann and Constantine Pavlakos Techniques for Visualizing Time-Varying Volume Data - Kwan-Liu Ma and Eric B. Lum Large-Scale Data Visualization and Rendering: A Problem-Driven Approach - Patrick McCormick and James Ahrens Issues and Architectures in Large-Scale Data Visualization - Constantine Pavlakos and Philip D. Heermann Consuming Network Bandwidth with Visapult - Wes Bethel and John Shalf PART IX - Visualization Software and Frameworks The Visualization Toolkit - William J. Schroeder and Kenneth M. Martin Visualization in the SCIRun Problem-Solving Environment - David M. Weinstein, Steven Parker, Jenny Simpson, Kurt Zimmerman, and Greg M. Jones Numerical Algorithms Group IRIS Explorer - Jeremy Walton AVS and AVS/Express - Jean M. Favre and Mario Valle Vis5D, Cave5D, and VisAD - Bill Hibbard Visualization with AVS - W. T. Hewitt, Nigel W. John, Matthew D. Cooper, K. Yien Kwok, George W. Leaver, Joanna M. Leng, Paul G. Lever, Mary J. McDerby, James S. Perrin, Mark Riding, I. Ari Sadarjoen, Tobias M. Schiebeck, and Colin C. Venters ParaView: An End-User Tool for Large-Data Visualization - James Ahrens, Berk Geveci, and Charles Law The Insight Toolkit: An Open-Source Initiative in Data Segmentation and Registration - Terry S. Yoo amira: A Highly Interactive System for Visual Data Analysis - Detlev Stalling, Malte Westerhoff, and Hans-Christian Hege PART X - Perceptual Issues in Visualization Extending Visualization to Perceptualization: The Importance of Perception in Effective Communication of Information - David S. Ebert Art and Science in Visualization - Victoria Interrante Exploiting Human Visual Perception in Visualization - Alan Chalmers and Kirsten Cater PART XI - Selected Topics and Applications Scalable Network Visualization - Stephen G. Eick Visual Data-Mining Techniques - Daniel A. Keim, Mike Sips, and Mihael Ankerst Visualization in Weather and Climate Research - Don Middleton, Tim Scheitlin, and Bob Wilhelmson Painting and Visualization - Robert M. Kirby, Daniel F. Keefe, and David Laidlaw Visualization and Natural Control Systems for Microscopy - Russell M. Taylor II, David Borland, Frederick P. Brooks, Jr., Mike Falvo, Kevin Jeffay, Gail Jones, David Marshburn, Stergios J. Papadakis, Lu-Chang Qin, Adam Seeger, F. Donelson Smith, Dianne Sonnenwald, Richard Superfine, Sean Washburn, Chris Weigle, Mary Whitton, Leandra Vicci, Martin Guthold, Tom Hudson, Philip Williams, and Warren Robinett Visualization for Computational Accelerator Physics - Kwan-Liu Ma, Greg Schussman, and Brett Wilson

Social media stock bubble

The social media bubble is a hypothesis stating that there was a speculative boom and bust phenomenon in the field of social media in the 2010s, particularly in the United States. The Wall Street Journal defined a bubble as stocks "priced above a level that can be justified by economic fundamentals," but this bubble includes social media. Social networking services (SNS) have seen huge growth since 2006, but some investors believed around 2014-2015, that the "bubble" was similar to the dot-com bubble of the late 1990s and early 2000s. In 2015, Mark Cuban, owner of the Dallas Mavericks NBA team and star of the TV show, Shark Tank, sounded an alarm on his personal blog over the social media bubble, calling it worse than the tech bubble in 2000 due to the lack of liquidity in social media stocks. A year prior, however, Cuban told CNBC that he did not believe social media stocks were on the verge of a bubble. In a letter to investors in 2014, David Einhorn, who runs the hedge-fund Greenlight Capital, wrote that "we are witnessing our second tech bubble in 15 years." He went on to write, "What is uncertain is how much further the bubble can expand, and what might pop it." Einhorn cited several factors supporting the existence an over-exuberance including "rejection of conventional valuation methods" and "huge first day IPO pops for companies that have done little more than use the right buzzwords and attract the right venture capital." Since those claims, services like Facebook, Twitter, Instagram, and Snapchat have grown to become multi-billion-dollar corporations generating enormous revenues, though some continue to lose money. == History of social networking services == Social networking services have grown and evolved with time since the launch of SixDegrees.com in 1997. Cutting edge at its time, SixDegrees.com allowed users to create a profile, invite friends, and connect within its platform. At its peak, SixDegrees.com had more than 3.5 million users. Between 1997 and 2001 more social sites aimed at allowing users to connect with others for personal, professional, or dating reasons. Friendster and MySpace were next to enter the social SNS arena, followed by Facebook in 2004. Even though MySpace had a following of more than 300 million users, it could not compete with Facebook, which now has overtaken the social networking world. However, as development of SNS started to emerge, a market saturation began to take effect. Some classrooms have begun to incorporate technology in daily learning as well as social channels specific to student's course work. Traditional social media sites are used, as are educational oriented sites such as ShowMe and Educreations Interactive Whiteboard. == Controversies == While SNS continue to play an influential role in helping people form real-world connections via the Internet, renewed concerns over the social media bubble have surfaced due to recent controversies. These threats include growing concerns about breaches in data, the rise of bot accounts, and the sharing of fake news on SNS platforms. There are also concerns that big data figures associated with these SNS are inflated or fake, as well as worries about the role the platforms played in national elections (see Russian interference in the 2016 United States elections). These issues have resulted in a lack of trust among the sites' users.

Squeaky Dolphin

Squeaky Dolphin is a program developed by the Government Communications Headquarters (GCHQ), a British intelligence and security organization, to collect and analyze data from social media networks. The program was first revealed to the general public on NBC on 27 January 2014 based on documents previously leaked by Edward Snowden. == Scope of surveillance == According to a document of the GCHQ dated August 2012, the program enables broad, real-time surveillance of the following items: YouTube video views The Like button on Facebook. Facebook has since then encrypted the data. Blogspot/Blogger visits Twitter, which has however encrypted its communications since this presentation was made The program can be supplemented with commercially available analytic software to determine which videos are popular among residents of specific cities. The dashboard software chosen was made by Splunk. The presentation, which was originally shown to an NSA audience and was made public by the NBC, contains a note saying the program was "Not interested in individuals just broad trends!". However, "according to other Snowden documents" obtained by NBC, in 2010, "GCHQ exploited unencrypted data from Twitter to identify specific users around the world and target them with propaganda."

Classora

Classora is a knowledge base for the Internet oriented to data analysis. From a practical point of view, Classora is a digital repository that stores structured information and allows it to be displayed in multiple formats: analytically, graphically, geographically (through maps); as well as carry out OLAP analysis. The information contained in Classora comes from public sources and is uploaded into the system through bots and ETL processes. The Knowledge Base has a commercial API for semantic enhancement, and an open web through which any user can access to part of the information collected (it also allows users to complete data and share opinions). Internally, Classora is organized into Knowledge Units and Reports. A «Knowledge Unit» is any element of the World about which information may be stored and presented in the form of a data sheet (a person, a company, a country, etc.) A «Report» is a group of Knowledge Units: a ranking of companies, a sport classification table, a survey about people, etc. In fact, one of the technical capabilities of Classora is that it allows the comparison of reports and knowledge units gathered from different sources, thereby generating an added value for the media in which this information is published: digital media, interactive TV, etc. == Key definitions == === Knowledge unit === The units of knowledge (also known as entries) in Classora are data sheets that have a certain semantic equivalence with the articles on the Wikipedia: they store information about any element of the world, be it a film, a country, a company or an animal. However, they differ from Wikipedia in that Classora stores structured information, enriched with a metadata layer; and therefore it is able to automatically interpret the meaning of each unit of knowledge. === Data report === A report is a group of units of knowledge in which the repetition of elements is not allowed. This definition includes any list, poll, ranking, etc.; and, in general, any consultation that involves more than one unit of knowledge. Classora excels at the reports management due to its visualization capabilities, being able to display data in the form of tables, graphs and maps. Types of reports: Sports scores: Sports competitions results sanctioned by the competent institution. Rankings and lists: All types of interesting and curious lists, whether they have an implicit order or not. Polls: Units of knowledge that are ranked according to users’ votes. Queries to the Knowledge Base: Questions from users using CQL. Networks of connections: automatically calculated from the reports and the taxonomy of each Knowledge Unit. === Organizational taxonomy === An organizational taxonomy (also referred to as entry type) is a data sheet that brings together the common attributes of a set of units of knowledge. For instance, the organizational taxonomy F1 Driver displays attributes such as date of debut, team, etc.; and the organizational taxonomy Football Club presents attributes such as city, stadium, etc. In Classora, taxonomies are hierarchically organized, so that they inherit attributes from their parent taxonomies. For instance, F1 Driver is a subsidiary taxonomy of Sportsperson, which is a subsidiary taxonomy of Person, which in turn is a subsidiary taxonomy of Organism. The simplest type of entry in Classora is Classora Object. All the other taxonomies are its subsidiaries and inherit its attributes. In fact, the only attribute Classora Object possesses is name (all units of knowledge are required to have one name at least). == Architecture of Classora == === Data Extraction Module === The Data Extraction Module consists of a set of robots coordinated by software that also manages the potential incidents. Most of the information available in Classora is automatically uploaded through those robots, which connect to the main online public sources to gather all types of data. There are three categories of robots: Extraction robots: responsible for the massive uploading of reports from official public sources (FIFA, CIA, IMF, Eurostat...). They are used for either absolute or incremental data uploading. Data scanner robots: responsible for looking for and updating the data of a unit of knowledge. They use specific sources to perform this task: Wikipedia, IMDB, World Bank, etc. Content aggregators: they don’t connect to external sources. Instead, they generate new information using Classora’s internal database. === Participatory Module === In Classora’s Open Website, Internet users may participate providing their knowledge as they would on the Wikipedia. There are different ways to participate: adding or correcting data in the Knowledge Base, voting in surveys (participatory rankings) and creating new Knowledge Units and Data Reports. === Connectivity Module === The Knowledge Base is designed to be embedded in multi-platform, multi-channel systems, thus enabling its integration into mobile devices, tablets, interactive TV, etc. This integration may be carried out through specific plugins (for navigators or other devices) or an API REST that provides content in XML or JSON formats. The API is divided into three blocks of operations. The first one is the block of general utility tools (ranging from autosuggest components about geographical hierarchies to operations to obtain the list of today’s celebrity birthdays, using CQL). The second one is the block of operations for widget generation (graphs, maps, rankings) using information from the knowledge base. Finally, there is a block of operations designed for the publication of free-source content. == Project statistics == As of April 2012, 2,000,000 Knowledge Units, 15,000 Reports, around 10,000 Maps and several million potential Comparative Analyses had been added to Classora. According to the site of web metrics Alexa, Classora Open Website is ranked at 100,557 globally and at 2,880 in the Spanish traffic ranking. Users spend an average of 9 ½ minutes in Classora.

Round-trip engineering

Round-trip engineering (RTE) in the context of model-driven architecture is a functionality of software development tools that synchronizes two or more related software artifacts, such as, source code, models, configuration files, documentation, etc. between each other. The need for round-trip engineering arises when the same information is present in multiple artifacts and when an inconsistency may arise in case some artifacts are updated. For example, some piece of information was added to/changed in only one artifact (source code) and, as a result, it became missing in/inconsistent with the other artifacts (in models). == Overview == Round-trip engineering is closely related to traditional software engineering disciplines: forward engineering (creating software from specifications), reverse engineering (creating specifications from existing software), and reengineering (understanding existing software and modifying it). Round-trip engineering is often wrongly defined as simply supporting both forward and reverse engineering. In fact, the key characteristic of round-trip engineering that distinguishes it from forward and reverse engineering is the ability to synchronize existing artifacts that evolved concurrently by incrementally updating each artifact to reflect changes made to the other artifacts. Furthermore, forward engineering can be seen as a special instance of RTE in which only the specification is present and reverse engineering can be seen as a special instance of RTE in which only the software is present. Many reengineering activities can also be understood as RTE when the software is updated to reflect changes made to the previously reverse engineered specification. === Types === Various books describe two types of RTE: partial or uni-directional RTE: changes made to a higher level representation of a code and model are reflected in lower level, but not otherwise; the latter might be allowed, but with limitations that may not affect higher-level abstractions full or bi-directional RTE: regardless of changes, both higher and lower-level code and model representations are synchronized if any of them altered === Auto synchronization === Another characteristic of round-trip engineering is automatic update of the artifacts in response to automatically detected inconsistencies. In that sense, it is different from forward- and reverse engineering which can be both manual (traditionally) and automatic (via automatic generation or analysis of the artifacts). The automatic update can be either instantaneous or on-demand. In instantaneous RTE, all related artifacts are immediately updated after each change made to one of them. In on-demand RTE, authors of the artifacts may concurrently update the artifacts (even in a distributed setting) and at some point choose to execute matching to identify inconsistencies and choose to propagate some of them and reconcile potential conflicts. === Iterative approach === Round trip engineering may involve an iterative development process. After you have synchronized your model with revised code, you are still free to choose the best way to work – make further modifications to the code or make changes to your model. You can synchronize in either direction at any time and you can repeat the cycle as many times as necessary. == Software == Many commercial tools and research prototypes support this form of RTE; a 2007 book lists Rational Rose, Together, ESS-Model, BlueJ, and Fujaba among those capable, with Fujaba said to be capable to also identify design patterns. == Limitations == A 2005 book on Visual Studio notes for instance that a common problem in RTE tools is that the model reversed is not the same as the original one, unless the tools are aided by leaving laborious annotations in the source code. The behavioral parts of UML impose even more challenges for RTE. Usually, UML class diagrams are supported to some degree; however, certain UML concepts, such as associations and containment do not have straightforward representations in many programming languages which limits the usability of the created code and accuracy of code analysis/reverse engineering (e.g., containment is hard to recognize in the code). A more tractable form of round-trip engineering is implemented in the context of framework application programming interfaces (APIs), whereby a model describing the usage of a framework API by an application is synchronized with that application's code. In this setting, the API prescribes all correct ways the framework can be used in applications, which allows precise and complete detection of API usages in the code as well as creation of useful code implementing correct API usages. Two prominent RTE implementations in this category are framework-specific modeling languages and Spring Roo (Java). Round-trip engineering is critical for maintaining consistency among multiple models and between the models and the code in Object Management Group's (OMG) Model-driven architecture. OMG proposed the QVT (query/view/transformation) standard to handle model transformations required for MDA. To date, a few implementations of the standard have been created. (Need to present practical experiences with MDA in relation to RTE). == Controversies == === Code generation controversy === Code generation (forward-engineering) from models means that the user abstractly models solutions, which are connoted by some model data, and then an automated tool derives from the models parts or all of the source code for the software system. In some tools, the user can provide a skeleton of the program source code, in the form of a source code template where predefined tokens are then replaced with program source code parts during the code generation process. UML (if used for MDA) diagrams specification was criticized for lack the detail which is needed to contain the same information as is covered with the program source. Some developers even claim that "the Code is the design". == Disadvantages == There is a serious risk that the generated code will rapidly differ from the model or that the reverse-engineered model will lose its reflection on the code or a mix of these two problems as result of cycled reengineering efforts. Regarding behavioral/dynamic part of UML for features like statechart diagram there is no equivalents in programming languages. Their translation during code-generation will result in common programming statement (.e.g if,switch,enum) being either missing or misinterpreted. If edited and imported back may result in different or incomplete model. The same goes for code snippets used for code generation stage for the pattern-implementation and user-specific logic: intermixed they may not be easily reverse-engineered back. There is also general lack of advanced tooling for modelling that are comparable to that of modern IDEs (for testing, debugging, navigation, etc.) for general-purpose programming languages and domain-specific languages. == Examples in software engineering == Perhaps the most common form of round-trip engineering is synchronization between UML (Unified Modeling Language) models and the corresponding source code and entity–relationship diagrams in data modelling and database modelling. Round-trip engineering based on Unified Modeling Language (UML) needs three basic tools for software development: Source Code Editor; UML Editor for the Attributes and Methods; Visualisation of UML structure

Clustered file system

A clustered file system (CFS) is a file system which is shared by being simultaneously mounted on multiple servers. There are several approaches to clustering, most of which do not employ a clustered file system (only direct attached storage for each node). Clustered file systems can provide features like location-independent addressing and redundancy which improve reliability or reduce the complexity of the other parts of the cluster. Parallel file systems are a type of clustered file system that spread data across multiple storage nodes, usually for redundancy or performance. == Shared-disk file system == A shared-disk file system uses a storage area network (SAN) to allow multiple computers to gain direct disk access at the block level. Access control and translation from file-level operations that applications use to block-level operations used by the SAN must take place on the client node. The most common type of clustered file system, the shared-disk file system – by adding mechanisms for concurrency control – provides a consistent and serializable view of the file system, avoiding corruption and unintended data loss even when multiple clients try to access the same files at the same time. Shared-disk file-systems commonly employ some sort of fencing mechanism to prevent data corruption in case of node failures, because an unfenced device can cause data corruption if it loses communication with its sister nodes and tries to access the same information other nodes are accessing. The underlying storage area network may use any of a number of block-level protocols, including SCSI, iSCSI, HyperSCSI, ATA over Ethernet (AoE), Fibre Channel, network block device, and InfiniBand. There are different architectural approaches to a shared-disk filesystem. Some distribute file information across all the servers in a cluster (fully distributed). === Examples === == Distributed file systems == Distributed file systems do not share block level access to the same storage but use a network protocol. These are commonly known as network file systems, even though they are not the only file systems that use the network to send data. Distributed file systems can restrict access to the file system depending on access lists or capabilities on both the servers and the clients, depending on how the protocol is designed. The difference between a distributed file system and a distributed data store is that a distributed file system allows files to be accessed using the same interfaces and semantics as local files – for example, mounting/unmounting, listing directories, read/write at byte boundaries, system's native permission model. Distributed data stores, by contrast, require using a different API or library and have different semantics (most often those of a database). === Design goals === Distributed file systems may aim for "transparency" in a number of aspects. That is, they aim to be "invisible" to client programs, which "see" a system which is similar to a local file system. Behind the scenes, the distributed file system handles locating files, transporting data, and potentially providing other features listed below. Access transparency: clients are unaware that files are distributed and can access them in the same way as local files are accessed. Location transparency: a consistent namespace exists encompassing local as well as remote files. The name of a file does not give its location. Concurrency transparency: all clients have the same view of the state of the file system. This means that if one process is modifying a file, any other processes on the same system or remote systems that are accessing the files will see the modifications in a coherent manner. Failure transparency: the client and client programs should operate correctly after a server failure. Heterogeneity: file service should be provided across different hardware and operating system platforms. Scalability: the file system should work well in small environments (1 machine, a dozen machines) and also scale gracefully to bigger ones (hundreds through tens of thousands of systems). Replication transparency: Clients should not have to be aware of the file replication performed across multiple servers to support scalability. Migration transparency: files should be able to move between different servers without the client's knowledge. === History === The Incompatible Timesharing System used virtual devices for transparent inter-machine file system access in the 1960s. More file servers were developed in the 1970s. In 1976, Digital Equipment Corporation created the File Access Listener (FAL), an implementation of the Data Access Protocol as part of DECnet Phase II which became the first widely used network file system. In 1984, Sun Microsystems created the file system called "Network File System" (NFS) which became the first widely used Internet Protocol based network file system. Other notable network file systems are Andrew File System (AFS), Apple Filing Protocol (AFP), NetWare Core Protocol (NCP), and Server Message Block (SMB) which is also known as Common Internet File System (CIFS). In 1986, IBM announced client and server support for Distributed Data Management Architecture (DDM) for the System/36, System/38, and IBM mainframe computers running CICS. This was followed by the support for IBM Personal Computer, AS/400, IBM mainframe computers under the MVS and VSE operating systems, and FlexOS. DDM also became the foundation for Distributed Relational Database Architecture, also known as DRDA. There are many peer-to-peer network protocols for open-source distributed file systems for cloud or closed-source clustered file systems, e. g.: 9P, AFS, Coda, CIFS/SMB, DCE/DFS, WekaFS, Lustre, PanFS, Google File System, Mnet, Chord Project. === Examples === == Network-attached storage == Network-attached storage (NAS) provides both storage and a file system, like a shared disk file system on top of a storage area network (SAN). NAS typically uses file-based protocols (as opposed to block-based protocols a SAN would use) such as NFS (popular on UNIX systems), SMB/CIFS (Server Message Block/Common Internet File System) (used with MS Windows systems), AFP (used with Apple Macintosh computers), or NCP (used with OES and Novell NetWare). == Design considerations == === Avoiding single point of failure === The failure of disk hardware or a given storage node in a cluster can create a single point of failure that can result in data loss or unavailability. Fault tolerance and high availability can be provided through data replication of one sort or another, so that data remains intact and available despite the failure of any single piece of equipment. For examples, see the lists of distributed fault-tolerant file systems and distributed parallel fault-tolerant file systems. === Performance === A common performance measurement of a clustered file system is the amount of time needed to satisfy service requests. In conventional systems, this time consists of a disk-access time and a small amount of CPU-processing time. But in a clustered file system, a remote access has additional overhead due to the distributed structure. This includes the time to deliver the request to a server, the time to deliver the response to the client, and for each direction, a CPU overhead of running the communication protocol software. === Concurrency === Concurrency control becomes an issue when more than one person or client is accessing the same file or block and want to update it. Hence updates to the file from one client should not interfere with access and updates from other clients. This problem is more complex with file systems due to concurrent overlapping writes, where different writers write to overlapping regions of the file concurrently. This problem is usually handled by concurrency control or locking which may either be built into the file system or provided by an add-on protocol. == History == IBM mainframes in the 1970s could share physical disks and file systems if each machine had its own channel connection to the drives' control units. In the 1980s, Digital Equipment Corporation's TOPS-20 and OpenVMS clusters (VAX/ALPHA/IA64) included shared disk file systems.