Category: 5 SPL Learning Material

QVS SRV is simply called QSRV, which operates as an independent WebServer process. It can be deployed independently to manage resource files on QVS, such as reading, uploading and downloading data files stored on the remote server. Installing esProc Enterprise Edition To deploy QSRV, first you need to install esProc Enterprise Edition, which includes the complete QVS project. Then deploy QSRV based on QVS according to the following directions.Download esProc Enterprise Edition：https://www.esproc.com/download-esproc/The application is located in [installation directory]\esProc\q-server\qvs. The jars QSRV server needs are located in [installation directory]\esProc\lib and [installation directory]\esProc\q-server\webapps\qvs\WEB-INF\lib.According to the cloud storage service QSRV will access, copy

SPL has implemented a set of functions for various cloud object storage (Amazon S3, Alibaba Cloud OSS, Google GCS, Microsoft Azure), which can read and write data on the cloud as easily as reading and writing local data files. This makes it easy to apply SPL’s flexible computing power to cloud data. Let’s take Amazon S3 as an example to understand its detailed usage. Amazon S3 Create/Close Connection The API for reading and writing S3 in Java requires four parameters, access key, secret key, region, endpoint; Passing them in SPL creates an S3 connection: s3_open(accessKey : secretKey : region :

Through esProc Desktop’s Office add-in service, you can use SPL.X() function to execute the SPL script, get the result set, and insert its data to Excel. For the Windows system, download the latest version of esProc Desktop and Office Add-in patch package. First install esProc Desktop and then the Office Add-in Service. For the Mac system, Office Add-in Service is built into the Desktop installation package. For Windows Both Windows system and Excel should be 64-bit. Dependency files nodejs: The directory whereNode.js is located. The full path is【esProc Desktop installation directory】/esProcDesk/nodejs. exceladdin: The directory whereSPLforExcel add-ins are located. The full

In some real-world scenarios, script file accesses need to be strictly controlled through different privileges in order to ensure data security. Only privileged users are allowed to check the result data of executing certain script files, or scripts can only be executed on certain types of hardware. To meet these needs and the like, both esProc Enterprise edition and Standard edition offer script file (.splx) encryption functionality. The functionality allows users to customize the encryption/decryption function by defining the privileges as needed, and executes encryption on the .splx file while recording the decryption function information in the encrypted file. Here

esProc Enterprise Edition provides the Remote Server (abbreviated as RSRV). The feature enables users to upload/download files to/from a remote server or the remote storage, or modify and debug a SPL script on the remote server. You need to deploy the Remote Server (RSRV) application before using the remote server. esProc Enterprise Edition has the QVS deployed in its built-in Tomcat, so you can directly use it after installing esProc in your remote machines. You can also deploy the RSRV in an independent application. Quick start esProc Enterprise edition offers a complete set of RSRV project in its installer. The

Ⅰ Metadata permission (table and field access control) A WEB system usually involves multiple users. There are two methods to control the scopes of metadata information different users can access. One is to create separate metadata files for different types of users, deploy multiple DQL services and connect to the appropriate DQL service for the current user via JDBC; the other is to let users share the metadata, and on WEB side select the appropriate information and remove tables/fields the current user does not have privilege to access. Multiple DQL services Look at the first method. Create manager.glmd, leader.glmd and

Ⅰ Deploy DQL service The startup program of DQL server is {installation directory}/esProc/bin/startDQLSERVER.bat (……startDQLSERVER.sh for Linux). Before starting the DQL server, first deploy the created metadata file TPCH.glmd on DQL server. The server configuration file is located in {installation directory}/esProc/esproc-services/, in which each folder corresponds to a DQL service. Copy the default datalogic service as a new service and name it TPCH: Then edit server.xml to add the TPCH service. Server address and server port number are also configured: Get into TPCH directory, copy the metadata file TPCH.glmd to TPCH/conf/, and edit TPCH/service.xml to configure the metadata file: After the

Key characteristics that enable SPL to outperform SQL In performance optimization cases, often SPL runs faster than SQL, having performance orders of magnitude higher. Our explanation of SPL’s winning is this: the language can implement low-complexity algorithms involving less amount of computation that SQL cannot implement; and this results in higher performance; SQL can only turn to the database optimizer, which, however, doesn’t work in the complex computing scenarios.There is nothing wrong with this logic when discussed in the abstract. But the specific technical factors are still wanted. The fundamental factor lies in the discreteness that we’ve talked about many

After we retrieve data from the data source as a table sequence in SPL, we can perform all SQL-style operations, such as filtering, aggregation, inter-column calculations, sorting, grouping & aggregation, top-N, post-grouping top-N, distinct, post-grouping distinct, and join to name a few. In this essay, we take the file data source to explain how to achieve those operations on a SPL table sequence. 1.  Filtering A filtering operation selects records that meet the specified condition (s) from a data table. Example: Get scores of students in class 10 from the following student scores table Students_scores.txt. The first row of the file

Both Parquet and ORC are columnar storage file formats of Hadoop, which offers Java API to read files of both formats. esProc encapsulates the Java code for implementing the file read to create easy-to-use SPL functions. These functions enable users to load Parquet/ORC files directly to esProc for computations. In this essay, we’ll take a close look at how you can use these functions to read and handle ORC/Paquet data. Create/Close Hive connection Similar to database connection, SPL also uses the “Create/Close” pair to connect to Hive. hive_open(hdfsUrl,thriftUrl,dbName,hdfsUserName)  Parameter hdfsUrl is address of HDFS server, parameter thriftUrl is URI of