Category: 6 SPL Case&Testing

It is inconvenient to compute XML data, so a class library is necessary and useful to handle it. In this essay, we will compare four types of XML class libraries – dom4j, MySQL, Scala and esProc SPL, in the aspects of syntactic expressiveness, deployment configurations, and data source support.

It is more difficult to compute JSON data than to compute two-dimensional data. A class library, in this case, is necessary and useful to handle JSON formatted data. In this essay, we will compare four types of the class library – JsonPath, SQLite, Scala and esProc SPL, for computing JSON data in terms of syntactic expressiveness, deployment configurations, and data source support.

As a data analyst, you have a lot of opportunities to process Excel files. At times you need to do the processing using a program. There are several scripting languages to choose. Here we compare them in a number of aspects, such as if they are easy to use when parsing an Excel file, how adaptable they are with different data format, to know the pros and cons of each. The best ones are Python pandas and esProc SPL. Yet the latter is even better.

SQL is database-based. That means files need to be first loaded into the database before we can process them with the popular query language. The data loading is not always convenient. So tools that can directly execute SQL on files were designed to avoid troubles. These tools differ widely in working environment, usability, SQL execution capability and format requirement. I tried a number of them and found that esProc is the easiest and most efficient to use for data analysts. Let me explain my findings. Go to Looking for the Best SQL-On-File Tool for Analyst for more infomation.

ClickHouse, as an open-source analysis database, is known for being fast. Is that true? Let’s test and verify it through comparative test. ClickHouse vs Oracle First of all, we conduct a comparative test on ClickHouse (CH for short) and Oracle database (ORA for short) under the same hardware and software environment, and use the internationally recognized TPC-H as the test benchmark. The test is to execute the calculation requirements (Q1 to Q22) defined with 22 SQL statements for 8 tables. The test is performed on one machine within 12 threads, and the total data amount is around 100G. Since the

In terms of structured data calculation outside the database, Stream has taken a step from scratch; Kotlin has slightly enhanced this capability, but the nature of the compiled language makes it impossible to go further; To really solve the problem of structured data computing outside the database, SPL, a professional structured data computing language, is needed. In java development, we often encounter the situation that it is inconvenient to use database but structured data calculation is needed. For a long time, Java has not provided a class library to deal with this situation. Even basic calculations such as sorting and

T+0 query refers to real-time data query, and the latest data can be queried during data statistics. There are also T+1, T+7, etc., which means that you can only query data 1 day or 7 days ago. Obviously, T+N query is not real-time. This article will discuss some main T+0 query technologies. Database The simplest way to realize T+0 query through database is to directly use a single business database for a query. All data are stored in one database, and it can naturally realize real-time data query. However, this method is only applicable to scenarios where the amount of data

A complete microservice includes service management (registration/discovery/logoff), service gateway, service fault tolerance, service communication, service monitoring, service security, deployment, etc. It is often unrealistic to use the native development language to develop from scratch, so the existing microservice development framework is usually used to build when implementing microservices, such as Spring cloud, Dubbo, Istio, etc. In microservices, data processing is the core, and it is also where the workload is large and the framework itself cannot help. For analytical businesses, data computing is the core of the core. This article will discuss and compare the methods of implementing data computing

The reporting application is liable to change. Frequent changes to the data source (data preparation process) are one aspect. It is crucial for reporting applications to hot-swap reports without terminating the application. In this essay, we’ll look at and compare several hot-swapping techniques in terms of report data source changes.  SQL It is fairly easy to achieve HotSwap if we implement the report data source preparation using SQL in the database. Most reporting tools have interpreted execution templates. Developers edit the data preparation SQL statement (which is usually packaged into the report template) and update the template to bring about

In some report development scenarios, the original data will undergo complicated computations through a number of steps before it is presented as expected in the desired layout. Finding a simple and efficient way or tool to get the multistep, complicated preparations done is crucial to increasing report development performance. This essay will examine several such methods to try to give you the best choice. Complicated SQL/Stored procedure RDB data sources have strong computing power. Using SQL enables us to handle most of the data processing scenarios. Yet some SQL queries are too hard to write for ordinary report development staff,