Category: 4 SPL Big Data Computing

I. Move data out of database When a TP database is overloaded, we can utilize SPL to undertake AP task, and the first thing to do is to move data out of the TP database. Connecting to relational database to retrieve data is usually implemented with JDBC. Take Oracle database as an example, and refer to TPC-H for data structure. The script to move the ORDERS out of the database and store as SPL high-performance composite table is as follows: A 1 =connect@l(“oracle12c”) 2 =A1.cursor@x(“O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT FROM ORDERS ORDER BY 1”) 3 =file(“orders.ctx”).create(#o_orderkey,o_custkey,o_orderstatus,o_totalprice,o_orderdate,o_orderpriority,o_clerk,o_shippriority,o_comment) 4 >A3.append@i(A2) Retrieving data through JDBC is slow,

Problem description A flexible data structure means that each record of a data table has different data structure. Usually, the fields of data table are divided into two parts, one part is the fields that are common to all records (common field for short), and the other part is the fields that vary from record to record (flexible field for short). There may be as many as hundreds of flexible fields, yet each record may have only a few of them. In complex situations, these flexible fields are further classified. Here is a student score table, and this table does

The objective of this practice is to search for the member with the highest similarity to the target vector from a large number of high-dimensional (such as 512-dimensional) binary vectors. A binary vector refers to the vector whose dimension values are either 0 or 1; the similarity refers to being close in distance in high-dimensional space. There are many types of distances, such as Euclidean distance, Mahalanobis distance and cos similarity, among which, the cos similarity is the most commonly used method in the industry to measure the similarity between vectors. The larger the cos similarity, the closer the distance

This series of articles explain how to perform multidimensional analysis (OLAP) in steps with detailed samples. Click to learn more in Multidimensional Analysis Backend Practice 8: Primary-sub Table and Parallel Calculation. Multidimensional Analysis Backend Practice 8: Primary-sub Table and Parallel Calculation

This series of articles explain how to perform multidimensional analysis (OLAP) in steps with detailed samples. Click to learn more in Multidimensional Analysis Backend Practice 7: Boolean Dimension and Binary Dimension. Multidimensional Analysis Backend Practice 7: Boolean Dimension and Binary Dimension

This series of articles explain how to perform multidimensional analysis (OLAP) in steps with detailed samples. Click to learn more in Multidimensional Analysis Backend Practice 6: big fact table associate with small dimension table. Multidimensional Analysis Backend Practice 6: big fact table associate with small dimension table

This series of articles explain how to perform multidimensional analysis (OLAP) in steps with detailed samples. Click to learn more in Multidimensional Analysis Backend Practice 5: small fact table associate with small dimension table. Multidimensional Analysis Backend Practice 5: small fact table associate with small dimension table

This series of articles explain how to perform multidimensional analysis (OLAP) in steps with detailed samples. Click to learn more in Multidimensional Analysis Backend Practice 4: Pre-aggregate and Redundant Sorting. Multidimensional Analysis Backend Practice 4: Pre-aggregate and Redundant Sorting

This series of articles explain how to perform multidimensional analysis (OLAP) in steps with detailed samples. Click to learn more in Multidimensional Analysis Backend Practice 3: Dimensions Sorting and Compression. Multidimensional Analysis Backend Practice 3: Dimensions Sorting and Compression

This series of articles explain how to perform multidimensional analysis (OLAP) in steps with detailed samples. Click to learn more in Multidimensional Analysis Backend Practice 2: Data Type Optimization. Multidimensional Analysis Backend Practice 2: Data Type Optimization