Category: TalkingData

Title party, there are thousands of Python programmers, of course, many people can master it. The term “you” here refers to none professionals in the workplace. People in the workplace usually use Excel to process data, but there are also many helpless situations, such as complex calculations, duplicate calculations, automatic processing, etc. When encountering a crash without saving, it can often cause people to collapse. If you can master a programming language, these problems won’t be a problem anymore. So, what should we learn? Countless training institutions and online materials will tell us: Python! The Python code looks very simple,

We have discussed the principles of coding difficulty and complexity, and now focusing on performance issues, running speed is of course a very important thing. We know that software cannot change the performance of hardware, and the CPU and hard disk have their fixed speed. However, we can design low complexity algorithms, which require less computation. As a result, the computer executes fewer actions and naturally becomes faster. It was originally planned to perform 100 million calculations, but if there is a good algorithm that can reduce the computational load to 1 million, it is not surprising that it is

Indexing is commonly used among programmers. Without fully grasping the idea behind the technique, a programmer is always eager to take advantage of it whenever they encounter a query performance problem, only to get disappointed by the result on many occasions when resources are consumed. By analyzing the principles of indexing, the article shows you when is the appropriate time to use an index and how to use it. The purpose of creating an index is to quickly find the record where a certain field value is equal to a given value (find the right person according to an ID

We discussed the advantages of SQL over Java, which is its set-oriented feature. Now let’s take a look at what advantages Java has over SQL. The code in Java may seem long and messy, but upon closer examination, one will find that the computational logic it describes is not difficult, and it basically follows a step-by-step approach to implement business goals. That is to say, Java is tedious to write, not difficult to think. But SQL is different. It is not difficult to understand the technical significance of each subquery, but it is difficult for you to understand what it

Compared with expensive, fast memory, hard disks are much cheaper and about one or two orders of magnitude of slower. But problems of hard disks are more than the slow speed. The basic characteristic of hard disks is that they are not good at handling high-frequency accesses of small amounts. High-frequency accesses of small amounts means that the frequency of data accesses is high and the amount of data retrieved each time is very small. For the memory, it takes almost the same time to retrieve one hundred million bytes of data in one million accesses, 100 bytes at a

For the same data calculation task, written in SQL or Java, the latter is often several times longer than the former. The long code is not only cumbersome to write, but also not conducive to understanding the overall business logic structure, and the algorithm and process are buried in the details. Why is Java so much longer than SQL? Let’s answer this question and introduce the concept of set orientation in programming languages. Firstly, it is about the computing ability of sets, which is easy to understand. Business logic often faces structured data, which typically appears in batches (in the

The sample SQL statements in coursebooks are usually simple and easy to understand. They even read like English sentences, giving the impression that SQL is rather simple and easy to learn. Actually, such a SQL statement consisting of only a few lines of code can only be found in coursebooks and training courses. In real-world businesses, the amount of SQL code is measured by KB instead of the number of lines. One SQL statement having several hundred lines of code and N layers of nested subqueries often reaches 3KB to 5KB in size. Such SQL statements are not easy to

We often hear about the advertisements for the performance of a big data product, saying that it is capable of running “queries over trillion-row tables in seconds”, which means they can get and return data meeting the specified condition from one trillion rows in seconds. Is this true? Probably you do not think it is true if you have read the article How Much Is One Terabyte of Data? ”. To process one trillion rows of data, which is dozens of, even one hundred, terabytes in size, we need tens of thousands of, even hundreds of thousands of, hard disks. This is

Many large databases accumulate a large number of data tables after running for many years, with severe cases reaching tens of thousands, making the database very bloated. These data tables are often many years old, for some of which the construction reasons may have been forgotten or they may no longer be useful, but they are difficult to confirm and you dare not delete them. This brings a huge burden to the operation and maintenance work. Along with these tables, there are still a large number of stored procedures constantly updating data to these tables, occupying computing resources and often

We know that SQL is very similar to English, and simple SQL statements can be read directly as English. Except for SQL, other major programming languages do not have this feature, and even if there are English words in the syntax, they are only used as mnemonics for certain concepts or operations, and what is written is a formal program statement rather than an English sentence. But SQL is different. It will write the entire sentence in a form that conforms to English habits, and also add many unnecessary prepositions, such as FROM being the operation subject of the statement