Talim (Kashmiri: تعليم, Kashmiri pronunciation: [t̪əːliːm], Urdu: تَعْلِیم, Arabic: تعليم, pronounced [taʕ.liːm] ) in textiles is a symbolic code and system of notation that facilitates the creation of intricate patterns in fabrics, such as shawls and carpets, and the written coded plans that include colour schemes and weaving instructions. The term is used in traditional hand-weaving in the Indian subcontinent. Talim was initially used to create certain types of patterns in Kashmiri shawls, and later came to be applied in the production of carpets. == Etymology and origin == The term talim, which refers to a symbolic code and system of notation used by shawl and carpet artisans in their weaving processes, came to the Urdu language from the Arabic noun taʻlim (تعليم), which means "authoritative instruction", "teaching", or "edification". It means the same in Urdu and Kashmiri. The Arabic noun originated from the second form of the Arabic root verb ʻalima (علم), which means "to know". According to a local belief in Kashmir, talim was introduced to them by Persian scholar and Sufi Muslim saint Mir Sayyid Ali Hamadani. The belief notwithstanding, talim might have originated from Kashmir; Amritsar was the only place outside of Srinagar where talim was used, by migrated Kashmiri artisans. == Technique == Whereas carpets are generally woven horizontally, providing weavers with a clear view of the progress they are making in creating designs, in Kashmir, carpets are woven vertically, so the weaver is reliant on the talim. The talim technique forms fabrics by passing the weft thread as per a given script that has design codes. Weavers use talim to weave the desired pattern with planned colours. Talim involves teamwork when applying the technique, as the process of creating intricate fabric designs in weaving begins with the Naqash (designer, who designs using pencils on graphs) meticulously crafting the design on a blank sheet of paper called a naska, and the master, Talim guru, making the colour codes and symbols for weft yarns that would interlace the warp to construct the desired design. He writes on a long strip of paper, in specific symbols, the colour codes, and the number of knots to be woven with each colour. Taraha guru collaborates with talim guru and is known as the artisan responsible for determining the colours. Talim uthana is a process or the act of "picking the codes" from the graph. A clerk called the Talim Navis would record the step-by-step instructions for these numbers and colours, and thousands of low-paid and interchangeable weavers would read or recite the record to carry out the design. Afterward, a talim copyist makes copies, which are needed when multiple looms weave the same product. The script, which has been encoded, is deciphered and translated according to the specific guidelines of weavers in order to incorporate the design that is included within it. Talim has been compared to "hieroglyphics" or as a "notational-cum-cryptographic system", as it is challenging to decipher and is unique to the shawls of Kashmir, which requires expertise to comprehend. According to researcher Gagan Deep Kaur, "The talim is widely held to be a trade secret of the community and has always been fiercely guarded by the owners." Those who use talim for shawl-making do not assign important tasks to women, because of the fear that the technique and knowledge may be divulged to other communities when the women are sent there to be married. The coded cards known as talim in the Kashmiri language were used for creating certain types of patterns in shawl weaving. The talim technique is employed in the creation of kani shawls, which originated from the Kanihama region of the Kashmir valley. Carpet weaving adapted the technique from shawl making. When Kashmiri artisans started to create carpets, they chose to continue using the talim rather than switching to a different method. The resurgence of the carpet industry in Amritsar during the last century resulted in the prevalent use of the talim technique among the local weavers, a majority of whom hailed from the region of Kashmir. === Recitation of codes === Talim was also communicated through recitation accompanied by a melodic chant or song. In traditional weaving practices, the use of chanting was common. The movement of the shuttles was synchronised with the song of the weaver, adding a musical rhythm to the instructions represented through hieroglyphics. The weaver's chant, "Two blue, one red, three yellow, two blue," served as a guide as they wove and replicated the designated pattern. == Usage == The first factories established in Amritsar around 1860 utilised Bokhara designs. However, Kashmiri weavers maintained their traditional techniques and employed the talim, instead of a cartoon, for tying knots. As a result, Amritsar became the second location in the Indian subcontinent to use the talim. The traditional weaving practices are still carried out in some parts of the Indian subcontinent. The exact date when talim was last used in the subcontinent varies depending on the region and the specific weaving community. Indian textile historian Jasleen Dhamija wrote in her 1989 book Handwoven Fabrics of India that there were still some weavers in the Kashmiri village of Kanihama who applied talim in weaving shawls. As of 2022, the carpet weavers in Kashmir were the only remaining users of talim in carpets, according to Zubair Ahmed, director of the Indian Institute of Carpet Technology. The institute aims to preserve traditional Kashmiri carpet designs by digitising talim and training weavers in the technique. == Gallery ==
Artisse AI
Artisse AI is a Hong Kong-based technology company founded by William Wu. The company developed a mobile photography application using generative artificial intelligence to transform selfies into high-quality, personalized images. The app allows users to visualize themselves in various scenarios, outfits, and hairstyles, and they can adjust lighting and ambiance to match their preferences. The app launched in 2023 across multiple markets, including the United States, United Kingdom, Japan, South Korea, Canada, and Australia. By January 2024, users had generated over 5 million images. That same month, the company secured $6.7 million in seed funding to support product development and marketing. == History == Artisse was originally founded in South Korea in 2022 by William Wu. The early concept was connected to a virtual idol initiative developed in collaboration with a K-pop agency, intended to support Wu's blockchain gaming business. The project later evolved into a standalone AI photography application. The current version of the Artisse app was developed following the company's relocation to Hong Kong in 2022. In January 2024, Artisse secured $6.7 million in seed funding, led by The London Fund. The investment was aimed at supporting product development, marketing, and user acquisition. Artisse uses an AI algorithm to create hyperrealistic images from uploaded photos. The app generates personalized images by combining generative AI technology, a global pool of licensed talent, and finished art services. The app works with individual users and businesses, offering professional-grade photos and advertisement images. According to the British newspaper Evening Standard the company has developed the world's first and most advanced AI photographer. It captures 15-30 photos of the user and generates 2D images, placing them in various outfits and locations worldwide. === Catheron Gaming === Artisse AI originated from Catheon Gaming, a blockchain gaming and entertainment company founded in 2021 by William Wu. Catheon Gaming published more than 30 Web3 titles in its first year, developed a blockchain game distribution platform, and offered advisory services to external developers. In 2022, HSBC and KPMG listed Catheon Gaming among the "Top 10 Emerging Giants" in the Asia–Pacific region, selected from a pool of more than 6,000 startups. In June 2023, Catheon Gaming was rebranded as Artisse Interactive, creating two divisions: Artisse Gaming, which continued blockchain and Web3 game development, and Artisse AI, which focused on generative photography technology. == Technology == Artisse uses a proprietary generative AI model combined with open-source imaging frameworks and diffusion models. Users are prompted to upload between 15 and 30 personal images, allowing the AI to train a personalized model in 30 to 40 minutes. After training, the app generates new images based on either textual or visual prompts, with options to adjust elements such as clothing, hairstyles, lighting, and backgrounds. To enhance realism, the app integrates augmented reality features and image refinement tools. The company has introduced features to address representation issues related to body shape and skin tone, although concerns persist about the ethical implications of altering personal traits. == Products == === Artisse mobile app === Available on iOS and Android platforms in 35 languages. Users initially receive 25 free images, after which the app adopts a subscription pricing model ranging from approximately $6 to $30 per month. By early 2024, the app reported around 4,000 paying subscribers out of more than 200,000 downloads. === Business and enterprise services === Artisse provides B2B solutions for creating marketing imagery and partners with agencies like Iconic Management to enable cost-effective virtual photoshoots. Additional features in development include virtual try-on capabilities and augmented reality integration for fashion retail. == Reception == Media coverage has noted the app's photorealistic image outputs with some sources highlighting its ease of use. However, concerns have been raised regarding image authenticity, algorithmic biases, and the potential impact on professional photography and modeling. Artisse has been widely covered by media outlets including TechCrunch, PetaPixel, Forbes Australia, and The Evening Standard. These publications discussed the app's integration of generative AI technology within the consumer photography space, its growing market influence, and its rapid adoption by users worldwide.
Twproject
Twproject (say: T W Project) is a web-based project and groupware management tool created by Open Lab, an Italian software house founded in 2001. It won the 17th Jolt Productivity Award in 2007 in the project management category. In March 2019 it becomes property of Twproject company. It has widespread use in universities as a teaching tool in project management courses. It is used by Oracle Corporation, Prada, Calzedonia, General Electric and many other companies from corporations to small start-ups. == History == April 2001 - The idea of Teamwork came to Open-Lab founders from a need to overcome the PM tools used at that time. It was built in Microsoft ASP and Adobe Flash November 2002 - Open-Lab decide to move from Flash to HTML and from ASP to Java-JSP. Teamwork 2 development is started. June 2004 - Teamwork 2 released, using top open-source technologies like Hibernate, jBlooming, dynamic CSS, Ajax 7 January 2005 - Teamwork goes open source, under LGPL license; remains such until June 2006 (18 months): it is a hit application on SourceForge, with 38.000 downloads, covered by greeting but starving April 2005 - Open-Lab takes the decision to change commercial strategy to finance development of Teamwork version 3 6 June 2006 - Teamwork 3 is finally out (15 months development). New interface, many new features, agile support and much more 27 March 2007 - Teamwork wins the 2007 JOLT Productivity Awards for project management category July 2007 - Teamwork 4 development started: new interface, extended use of new HTML capabilities, JS-oriented interface, start using jQuery February 2009 - Teamwork 4.0 is out February 2010 - Teamwork 4.4: public project pages, Chinese interface. jQuery is getting more space in Teamwork December 2010 - Teamwork 4.6: released Mobile module available for iPhone, Android, BlackBerry. Intensive usage of jQuery June 2011 - Teamwork 4.7: released Issue Kanban / Organizer January 2012 - Teamwork 5.0 development started. Lighter interface, extensive usage of dynamic pages, easier installer and first time approach. Learning curve highly reduced. A jQuery Gantt editor included and released free for the community July 2012 - Teamwork 5 released and also the free online Gantt editor November 2012 - Teamwork 5.1 with new trees and improved model for staffing March 2013 - Teamwork 5.2 with stronger support for customizations and Japanese interface. April 2014 - Teamwork has changed its name in Twproject because the domain teamwork.com has been purchased by Teamwork. April 2013 - Twproject 5.4 with a redesigned more powerful Gantt chart. August 2015 - Twproject 5 finale release. September 2015 - Twproject 6 with a completely redesigned user interface. March 2019 - A new company Twproject srl has been spun off. September 2021 - Twproject 7 has been released introducing WBS based management and workload management. == Features == Project & task management (with Microsoft Project import/export), and JSON format Gantt editor. Uses jQuery Gantt components Time tracking. Several entry points: dashboard, weekly view, issues, start/stop buttons Resource planning with weekly/monthly view, work load overview, unavailability from agenda Issue tracking & planning(with Kanban), e-mail integration, task dedicated inboxes Dashboard configuration, with customizable portlets and layout Message boards Scrum module Meeting and minute management, attached documents Agenda (Integrates with iCal, Microsoft Outlook, Microsoft Entourage, and Google Calendar) Document management, remote file systems link with NTFS, FTP, SVN, S3 (Dropbox, Google drive) Mobile application for iPhone, iPad, Android, Blackberry, Windows phone == Integration == A complete JSON API is available for integrations. The applications runs in Java JDK 8+ on the Hibernate object/relational mapping. The standard distribution uses Apache Tomcat 9, but can run on any J2EE application server. Twproject is tested on these DB servers: MySQL, Oracle, SQL Server, PostgreSql, HSQLDB, but as uses Hibernate can run on many others. There is simple graphical step-by-step installer for Windows, Mac, Linux, .zip/.tar.gz/.rpm packages.
Deconvolution
In mathematics, deconvolution is the inverse of convolution. Both operations are used in signal processing and image processing. For example, it may be possible to recover the original signal after a filter (convolution) by using a deconvolution method with a certain degree of accuracy. Due to the measurement error of the recorded signal or image, it can be demonstrated that the worse the signal-to-noise ratio (SNR), the worse the reversing of a filter will be; hence, inverting a filter is not always a good solution as the error amplifies. Deconvolution offers a solution to this problem. The foundations for deconvolution and time-series analysis were largely laid by Norbert Wiener of the Massachusetts Institute of Technology in his book Extrapolation, Interpolation, and Smoothing of Stationary Time Series (1949). The book was based on work Wiener had done during World War II but that had been classified at the time. Some of the early attempts to apply these theories were in the fields of weather forecasting and economics. == Description == In general, the objective of deconvolution is to find the solution f of a convolution equation of the form: f ∗ g = h {\displaystyle fg=h\,} Usually, h is some recorded signal, and f is some signal that we wish to recover, but has been convolved with a filter or distortion function g, before we recorded it. Usually, h is a distorted version of f and the shape of f can't be easily recognized by the eye or simpler time-domain operations. The function g represents the impulse response of an instrument or a driving force that was applied to a physical system. If we know g, or at least know the form of g, then we can perform deterministic deconvolution. However, if we do not know g in advance, then we need to estimate it. This can be done using methods of statistical estimation or building the physical principles of the underlying system, such as the electrical circuit equations or diffusion equations. There are several deconvolution techniques, depending on the choice of the measurement error and deconvolution parameters: === Raw deconvolution === When the measurement error is very low (ideal case), deconvolution collapses into a filter reversing. This kind of deconvolution can be performed in the Laplace domain. By computing the Fourier transform of the recorded signal h and the system response function g, you get H and G, with G as the transfer function. Using the convolution theorem, F = H / G {\displaystyle F=H/G\,} where F is the estimated Fourier transform of f. Finally, the inverse Fourier transform of the function F is taken to find the estimated deconvolved signal f. Note that G is at the denominator and could amplify elements of the error model if present. === Deconvolution with noise === In physical measurements, the situation is usually closer to ( f ∗ g ) + ε = h {\displaystyle (fg)+\varepsilon =h\,} In this case ε is noise that has entered our recorded signal. If a noisy signal or image is assumed to be noiseless, the statistical estimate of g will be incorrect. In turn, the estimate of ƒ will also be incorrect. The lower the signal-to-noise ratio, the worse the estimate of the deconvolved signal will be. That is the reason why inverse filtering the signal (as in the "raw deconvolution" above) is usually not a good solution. However, if at least some knowledge exists of the type of noise in the data (for example, white noise), the estimate of ƒ can be improved through techniques such as Wiener deconvolution. == Applications == === Seismology === The concept of deconvolution had an early application in reflection seismology. In 1950, Enders Robinson was a graduate student at MIT. He worked with others at MIT, such as Norbert Wiener, Norman Levinson, and economist Paul Samuelson, to develop the "convolutional model" of a reflection seismogram. This model assumes that the recorded seismogram s(t) is the convolution of an Earth-reflectivity function e(t) and a seismic wavelet w(t) from a point source, where t represents recording time. Thus, our convolution equation is s ( t ) = ( e ∗ w ) ( t ) . {\displaystyle s(t)=(ew)(t).\,} The seismologist is interested in e, which contains information about the Earth's structure. By the convolution theorem, this equation may be Fourier transformed to S ( ω ) = E ( ω ) W ( ω ) {\displaystyle S(\omega )=E(\omega )W(\omega )\,} in the frequency domain, where ω {\displaystyle \omega } is the frequency variable. By assuming that the reflectivity is white, we can assume that the power spectrum of the reflectivity is constant, and that the power spectrum of the seismogram is the spectrum of the wavelet multiplied by that constant. Thus, | S ( ω ) | ≈ k | W ( ω ) | . {\displaystyle |S(\omega )|\approx k|W(\omega )|.\,} If we assume that the wavelet is minimum phase, we can recover it by calculating the minimum phase equivalent of the power spectrum we just found. The reflectivity may be recovered by designing and applying a Wiener filter that shapes the estimated wavelet to a Dirac delta function (i.e., a spike). The result may be seen as a series of scaled, shifted delta functions (although this is not mathematically rigorous): e ( t ) = ∑ i = 1 N r i δ ( t − τ i ) , {\displaystyle e(t)=\sum _{i=1}^{N}r_{i}\delta (t-\tau _{i}),} where N is the number of reflection events, r i {\displaystyle r_{i}} are the reflection coefficients, t − τ i {\displaystyle t-\tau _{i}} are the reflection times of each event, and δ {\displaystyle \delta } is the Dirac delta function. In practice, since we are dealing with noisy, finite bandwidth, finite length, discretely sampled datasets, the above procedure only yields an approximation of the filter required to deconvolve the data. However, by formulating the problem as the solution of a Toeplitz matrix and using Levinson recursion, we can relatively quickly estimate a filter with the smallest mean squared error possible. We can also do deconvolution directly in the frequency domain and get similar results. The technique is closely related to linear prediction. === Optics and other imaging === In optics and imaging, the term "deconvolution" is specifically used to refer to the process of reversing the optical distortion that takes place in an optical microscope, electron microscope, telescope, or other imaging instrument, thus creating clearer images. It is usually done in the digital domain by a software algorithm, as part of a suite of microscope image processing techniques. Deconvolution is also practical to sharpen images that suffer from fast motion or jiggles during capturing. Early Hubble Space Telescope images were distorted by a flawed mirror and were sharpened by deconvolution. The usual method is to assume that the optical path through the instrument is optically perfect, convolved with a point spread function (PSF), that is, a mathematical function that describes the distortion in terms of the pathway a theoretical point source of light (or other waves) takes through the instrument. Usually, such a point source contributes a small area of fuzziness to the final image. If this function can be determined, it is then a matter of computing its inverse or complementary function, and convolving the acquired image with that. The result is the original, undistorted image. In practice, finding the true PSF is impossible, and usually an approximation of it is used, theoretically calculated or based on some experimental estimation by using known probes. Real optics may also have different PSFs at different focal and spatial locations, and the PSF may be non-linear. The accuracy of the approximation of the PSF will dictate the final result. Different algorithms can be employed to give better results, at the price of being more computationally intensive. Since the original convolution discards data, some algorithms use additional data acquired at nearby focal points to make up some of the lost information. Regularization in iterative algorithms (as in expectation-maximization algorithms) can be applied to avoid unrealistic solutions. When the PSF is unknown, it may be possible to deduce it by systematically trying different possible PSFs and assessing whether the image has improved. This procedure is called blind deconvolution. Blind deconvolution is a well-established image restoration technique in astronomy, where the point nature of the objects photographed exposes the PSF thus making it more feasible. It is also used in fluorescence microscopy for image restoration, and in fluorescence spectral imaging for spectral separation of multiple unknown fluorophores. The most common iterative algorithm for the purpose is the Richardson–Lucy deconvolution algorithm; the Wiener deconvolution (and approximations) are the most common non-iterative algorithms. For some specific imaging systems such as laser pulsed terahertz systems, PSF can be modeled mathematically. As a result, as shown in the figure, deconvolution of the modeled PS
Research software engineering
Research software engineering is the application of software engineering practices, methods and techniques for research software, i.e. software that was made for and is mainly used within research projects. As usual for software engineering, this also includes knowledge of other (and in this case varying) research fields as well as open science that need to be incorporated into a software development process. The term was proposed in a research paper in 2010 in response to an empirical survey on tools used for software development in research projects. It started to be used in United Kingdom in 2012, when it was needed to define the type of software development needed in research. This focuses on reproducibility, reusability, and accuracy of data analysis and applications created for research. == Support == Various type of associations and organisations have been created around this role to support the creation of posts in universities and research institutes. In 2014 a Research Software Engineer Association was created in UK, which attracted 160 members in the first three months and which lead to the creation of the Society of Research Software Engineering in 2019. Other countries like the Netherlands, Germany, and the USA followed creating similar communities and there are similar efforts being pursued in Asia, Australia, Canada, New Zealand, the Nordic countries, and Belgium. In January 2021 the International Council of RSE Associations was introduced. UK counts over 40 universities and institutes with groups that provide access to software expertise to different areas of research. Additionally, the Engineering and Physical Sciences Research Council created a Research Software Engineer fellowship to promote this role and help the creation of RSE groups across UK, with calls in 2015, 2017, and 2020. The world first RSE conference took place in UK in September 2016 and it has been repeated annually (except for a gap in 2020) since. In 2019 the first national RSE conferences in Germany and the Netherlands were held, next editions were planned for 2020 and then cancelled. US-RSE held its first national conference in 2023. The Research Software Alliance was formed in 2019 to advance the global research software ecosystem by collaborating with decision makers and key influencers. The SORSE (A Series of Online Research Software Events) community was established in late‑2020 in response to the COVID-19 pandemic and ran its first online event in September 2020.
Mobile DevOps
Mobile DevOps is a set of practices that applies the principles of DevOps specifically to the development of mobile applications. Traditional DevOps focuses on streamlining the software development process in general, but mobile development has its own unique challenges that require a tailored approach. Mobile DevOps is not simply as a branch of DevOps specific to mobile app development, instead an extension and reinterpretation of the DevOps philosophy due to very specific requirements of the mobile world. == Rationale == Traditional DevOps approach has been formed around 2007-2008, close to the dates when iOS and Android mobile operating systems were released to the public. The traditional DevOps approach primarily evolved to meet the changing needs of the software development world with the paradigm shift towards continuous and rapid development and deployment (such as in web development, where interpreted languages are more prevalent than compiled languages). While traditional DevOps embraced agility and flexibility, mobile operating system providers steered towards a walled-garden approach with compiled apps with tight controls over how they can be distributed and installed on a mobile device. This difference in the mobile development mindset compared to what the traditional DevOps approach is advocating, is augmented further with the mobile applications to be deployed on a high number of varying devices and operating systems. Eventually, the concept of Mobile DevOps took off as a trend around 2014-2015, in line with the fast growth of the number of applications in mobile app stores. As individuals and corporations alike are developing and publishing more and more mobile applications, the need for efficiency and shorter release cycles increased, which is addressed by the continuous feedback and continuous development approach within the concept of DevOps, while requiring a significant level of adaptation and extension of the traditional DevOps practices. == Mindset shift from traditional DevOps to mobile DevOps == Mobile DevOps has a unique set of challenges and constraints, which solidifies the fact that it needs to be approached as a separate discipline. These challenges can be outlined as follows: Platform-specific requirements and tight controls of mobile operating system providers, where for instance a macOS device is mandatory for iOS application development and release. The walled-garden approach of distributing mobile apps, specifically applying to iOS applications, which comes with app review and app release delays that would not be needed in web development, for instance. Code signing requirements that come with the walled-garden approach, which introduce additional processes in the mobile application build pipeline along with new security concerns. An entire deployment cycle is re-run even in the slightest code change due to how applications are compiled and delivered to the users. The final product is to be deployed to a wide variety of mobile devices worldwide, which requires extensive testing and user feedback. Monitoring mobile applications require additional tools and approaches to be able to get data from an application running on a mobile device while respecting user privacy. Frequent operating system updates by mobile platforms can require rapid adaptation of apps, introducing further complexity to the development and maintenance cycles. == Benefits of mobile DevOps == Mobile DevOps is not an abstract concept and offers a range of benefits that can help improve the efficiency and effectiveness of the mobile app development process. These benefits can even be quantified by collecting the data within the mobile application development lifecycle. The benefits can be categorized into the following areas: Faster Release Cycles: By automating tasks and streamlining the development process, mobile DevOps enables teams to deliver new features and updates more frequently. Improved Quality: Automated testing and continuous monitoring help to identify and fix bugs earlier in the development cycle, leading to higher quality apps. Optimized Resource Utilization: Mobile DevOps promotes optimized resource utilization by automating tasks and streamlining workflows. Furthermore, mobile DevOps practices like containerization can help to create more efficient and scalable development environments. Increased Agility: Mobile DevOps allows teams to be more responsive to changes in the market and user feedback. == List of Dedicated Mobile DevOps Platforms == Even though it is possible to run a mobile DevOps cycle with most of the CI/CD platforms, they may require significant effort compared to non-mobile CI/CD (e.g. you need to bring your own infrastructure or it may require "reinventing the wheel" for commonly used platforms like Jenkins). To overcome the mobile-specific challenges specified, there are certain platforms that are dedicated to the lifecycle of mobile applications. These platforms exclusively focus on DevOps processes for mobile app development and are also referred as mobile CI/CD platforms. Appcircle (Multiplatform | Cloud-based & On-premise) Visual Studio App Center (Multiplatform | Cloud-based) Xcode Cloud (Apple platforms only | Cloud-based)
Apache Parquet
Apache Parquet is a free and open-source column-oriented data storage format in the Apache Hadoop ecosystem inspired by Google Dremel interactive ad-hoc query system for analysis of read-only nested data. It is similar to RCFile and ORC, the other columnar-storage file formats in Hadoop, and is compatible with most of the data processing frameworks around Hadoop. It provides data compression and encoding schemes with enhanced performance to handle complex data in bulk. == History == The open-source project to build Apache Parquet began as a joint effort between Twitter and Cloudera using the record shredding and assembly algorithm as described in Google's Dremel. Parquet was designed as an improvement on the Trevni columnar storage format created by Doug Cutting, the creator of Hadoop. The name 'parquet' (lit. 'small compartment') refers to a style of decorative flooring and was chosen to "evoke the bottom layer of a database with an interesting layout". The first version, Apache Parquet 1.0, was released in July 2013. Since April 27, 2015, Apache Parquet has been a top-level Apache Software Foundation (ASF)-sponsored project. == Features == Apache Parquet is implemented using the record-shredding and assembly algorithm, which accommodates the complex data structures that can be used to store data. The values in each column are stored in contiguous memory locations, providing the following benefits: Column-wise compression is efficient in storage space Encoding and compression techniques specific to the type of data in each column can be used Queries that fetch specific column values need not read the entire row, thus improving performance Apache Parquet is implemented using the Apache Thrift framework, which increases its flexibility; it can work with a number of programming languages like C++, Java, Python, PHP, etc. As of August 2015, Parquet supports the big-data-processing frameworks including Apache Hive, Apache Drill, Apache Impala, Apache Crunch, Apache Pig, Cascading, Presto and Apache Spark. It is one of the external data formats used by the pandas Python data manipulation and analysis library. == Compression and encoding == In Parquet, compression is performed column by column, which enables different encoding schemes to be used for text and integer data. This strategy also keeps the door open for newer and better encoding schemes to be implemented as they are invented. Parquet supports various compression formats: snappy, gzip, LZO, brotli, zstd, and LZ4. === Dictionary encoding === Parquet has an automatic dictionary encoding enabled dynamically for data with a small number of unique values (i.e. below 105) that enables significant compression and boosts processing speed. === Bit packing === Storage of integers is usually done with dedicated 32 or 64 bits per integer. For small integers, packing multiple integers into the same space makes storage more efficient. === Run-length encoding (RLE) === To optimize storage of multiple occurrences of the same value, run-length encoding is used, which is where a single value is stored once along with the number of occurrences. Parquet implements a hybrid of bit packing and RLE, in which the encoding switches based on which produces the best compression results. This strategy works well for certain types of integer data and combines well with dictionary encoding. == Cloud Storage and Data Lakes == Parquet is widely used as the underlying file format in modern cloud-based data lake architectures. Cloud storage systems such as Amazon S3, Azure Data Lake Storage, and Google Cloud Storage commonly store data in Parquet format due to its efficient columnar representation and retrieval capabilities. Data lakehouse frameworks—including Apache Iceberg, Delta Lake, and Apache Hudi —build an additional metadata layer on top of Parquet files to support features such as schema evolution, time-travel queries, and ACID-compliant transactions. In these architectures, Parquet files serve as the immutable storage layer while the table formats manage data versioning and transactional integrity. == Comparison == Apache Parquet is comparable to RCFile and Optimized Row Columnar (ORC) file formats — all three fall under the category of columnar data storage within the Hadoop ecosystem. They all have better compression and encoding with improved read performance at the cost of slower writes. In addition to these features, Apache Parquet supports limited schema evolution, i.e., the schema can be modified according to the changes in the data. It also provides the ability to add new columns and merge schemas that do not conflict. Apache Arrow is designed as an in-memory complement to on-disk columnar formats like Parquet and ORC. The Arrow and Parquet projects include libraries that allow for reading and writing between the two formats. == Implementations == Known implementations of Parquet include: