Category: 5 SPL Learning Material

In big data computing scenarios, many complicated algorithms produce concrete intermediate results that result in performance-reducing hard disk reads/writes. Here’s an example. The account trading table trades has three fields – id (account id), dt (trading date) and amount (trading amount), and we are trying to find records having transactions for n days in a row and sum amounts by the weekday when the transaction happens. Let’s call the first half of the computation compute1 and the second half compute2. For compute1, we sort records by account and trading date and store the sorted data before we start to get

Principle of multipurpose traversal When traversing the data table on external storage, most of the time is spent on reading data from the hard disk. Therefore, we would hope to do as many things as possible over a single reading, that is, the data read out during the traversal process can be used to the maximum extent. For example, we hope to perform two calculations when grouping and aggregating the orders table. The first one (compute1) is to count the sales by product group to get the result (result1), and the second one (compute2) is to count the maximum order

Performing statistical analysis based on detail user/account data is a common computing scenario, covering computing goals from user behavior analysis, bank account analytics, calculating funnel conversion rate to insurance policy analysis, and so on. We call all of them account analysis. The scenario has certain characteristics – a huge volume of total data versus a small amount of data in each account; long total time span versus short time for each computation; and no exchanges between accounts versus probably complicated computations within one account. Generally, this type of computing scenario involves filtering on time dimension. If data can be ordered

A table storing data physically (which is simply called physical table) usually uses some storage mechanisms to boost performance and reduce space usage. But they usually lead to lower data readability and higher coding complexity. For example, using integers to represent enumerated values or using bits of integer field to store boolean values. Integers and binary digits are not as easy to read as strings and booleans. And we need to write code for type conversion and binary computation over these physical values. A SPL pseudo table is a logical table defined on a physical table. We can define easier

SPL offers two ways to connect to databases: direct connection via JDBC/ODBC and JNDI connection pooling. Suppose there is employees table (below) in MyCompany database in Mysql. Let’s use the two ways to connect to it. Direct connection JDBC Download the corresponding JDBC driver, such as Mysql-connector-java-***.jar, from MySQL official website, put it under {SPL installation directory}/common/jdbc/, and start the SPL Desktop. Quick connection Directly connect to MyCompany database by using Mysql driver class to connect to the URL in a SPL script: A 1 =connect(“com.Mysql.jdbc.Driver”,“jdbc:Mysql://127.0.0.1:3306/MyCompany?user=root&password=123456”) 2 =A1.query(“select * from employees”) 3 >A1.close() After execution, A2 retrieves data from employees

JOIN is used to combine records from two or more tables. This article will explore the solutions and basic principles of SPL for join problems. The data interdependence between tables is called the inter table association relationship. The association relationships between tables can be divided into the following types: one-to-one, many-to-one, one-to-many, and many-to-many. We can join two or more tables through inter table association relationships to achieve the goal of multi table association queries. Many to many association: A record of Table A can correspond to any number of records of Table B; At the same time, a record

JOIN is used to combine records from two or more tables. This article will explore the solutions and basic principles of SPL for join problems. The data interdependence between tables is called the inter table association relationship. The association relationships between tables can be divided into the following types: one-to-one, many-to-one, one-to-many, and many-to-many. We can join two or more tables through inter table association relationships to achieve the goal of multi table association queries. Many to one association refers to any number of records of one table to correspond to one record of another table. A many-to-one association relationship,

— esProc includes supports for Spring data source Introduction esProc has a rich set of data source configuration classes that can meet most configuration requirements. Yet, when trying to dock a native data source into esProc, we find that a special interface is needed.The Spring data source factory:com.scudata.common.SpringDBSessionFactory.java，And a general-purpose data source factory are thus created.com.scudata.common.DataSourceSessionFactory.java。Each of them can implement the ISessionFactory interface. Just Invoke their unique static method create to generate ISessionFactory and pass it to the esProc runtime environment. This essay explains the role of com.scudata.common.SpringDBSessionFactory.java for building microservices based on Spring Cloud.The general-purpose data source factory com.scudata.common.DataSourceSessionFactory.java

As a structured computation engine, SPL brings a lot of brand-new concepts, which go far beyond the traditional relational-algebra-based coding systems. Maybe they are strange to programmers or users familiar with SQL or Java. This essay is intended to help understanding those SPL concepts correctly and master the key ones quickly. Discreteness and general sets In SPL, record is a basic data type, and a data table is a set of records. A record which is also a member of a data table is allowed to exist and operate separately and repeatedly outside of the table. Such a record is