Category: TalkingData

When discussing ordered grouping, we only studied the possible impact of the order of the set members to be grouped on the grouping operation. However, since we need to consider the order of the set, and the result set is also a set, does its member order also have business significance? It does have, but it is not as important as the order of the original set. There are two aspects to the order of the grouping result set, one is the order of these grouped subsets, and the other is the order of the members of the grouped subsets.

We know that SQL uses the concept of unordered sets in mathematics, so SQL grouping does not pay attention to the order of members in the set to be grouped. The equivalence grouping and none equivalence grouping we discussed earlier have not paid attention to this issue, and the grouping rules are based on the member values themselves. But if an ordered set is used as the object of operation, the influence of member order on grouping must be considered, and there are many applications of ordered grouping in real business. Firstly, there is the sequence number grouping. A simple

Most of your veteran developers must have experienced the rather low database I/O performance. But many of you may not have the perception about how slow is the I/O operation, and particularly, about the gap between database I/O speed and the speeds of other data read/write methods. Java is a commonly used technique used for application development. Now we perform a field test to find out performances of retrieving data from Oracle and MySQL – the two typical types of databases, in Java, and compare them with the performance of retrieving data from the text file. Use the international T-PCH

We have studied the essence of grouping operations, i.e., partitioning a set into several subsets according to certain rules. However, the focus of the discussion is to restore the steps of grouping operations, and there is no discussion of partition rules. In the examples, certain grouping key values are used to define partition rules, which is the method used in SQL. We refer to this partition method as equivalence grouping. Equivalence grouping is equivalent to defining an equivalence relationship on a set: members (records) with equal key values are considered equivalent. The equivalence relationship is a mathematical concept that will not

Grouping is a common operation in databases, but not everyone can deeply understand it. The essence of grouping operation is to partition a set into several subsets according to certain rules, which means that the return value should be a set composed of sets. However, people generally do not care much about the member sets that make up this set (i.e. grouped subsets), but are more interested in the aggregated values of these subsets. Therefore, grouping operation is often accompanied by further aggregate calculation of subsets. SQL is designed in this way. When there is a GROUP BY clause, the

In one of the previous articles, we talked about the hard disk performance characteristics, mainly on the hardware level and on the OS level. Here we look at it on the application software level. Software, in theory, can penetrate the operating system to access disk directly. However, the passage is hardly feasible because it is too inconvenient and loses compatibility. So, let’s just ignore it and move on to the storage forms of the OS, where files hold the major position . Text is the most common file format. It is widely used because it is universally applicable and highly

We will continue to discuss the Lamba syntax in set operations and introduce order related computational capabilities. SQL uses the mathematical concept of unordered sets and does not pay attention to order when traversing sets. But computers can only execute step by step, and there will always be an order when looping. By fully utilizing this order, richer computational requirements can be easily expressed. For example, we want to extract half of the members from a set to form a new set. This may seem like a filtering operation, but the filtering conditions are not related to the set members

A programming language with set-oriented features allows us to write complex operations on sets with very few statements, and the core issue is whether the Lambda syntax is designed conveniently, which directly determines the description efficiency of the programming language. Let’s examine the possible scenarios of set operations from simple to complex, and see which requirements a good Lambda syntax needs to adapt to. Firstly, it is the operation that directly uses set members, such as calculating the sum of set members. This is the simplest case, and using a regular function syntax is ok, to pass in the set

The columnar storage is a commonly used storage technique. Often it implies high performance, and has basically become a standard configuration for today’s analytical databases. The basic principle of columnar storage is reducing the amount of data retrieved from the hard disk.  A data table can have a lot of columns, but the computation may use only a very small number of them. With the columnar storage, useless columns do not need to be retrieved; while with the row-wise storage, all columns need to be scanned. When the retrieved columns only take up a very small part of the total,

Programming was once considered a basic skill in modern society, just like driving, that none professionals should also master, and it is also taught in primary and secondary schools. But in reality, its popularity is far worse than driving, and there are very few people who master this skill. The knowledge learned in school is forgotten because it cannot be used in work. Why is this happening? To answer this question, we first need to answer: Why do we need to learn programming? What do you want to do after learning? Except for a few individuals who are highly interested