Category: 2-SPL-Application-Development

Databases have closed data processing capacity. Being closed means that data to be computed or processed must be loaded into databases in advance. There is a clear line between in-database data and outside database data. Databases are generally used for OLTP and OLAP. OLTP requires data consistency that makes sense only when discussed within a defined scope. This automatically results in a closed database system that will and can only ensure consistency of data within it. OLAP, however, does not demand data consistency. But customarily, data warehouses for performing them use databases. As a result, these data warehouses also become

It is common to use the database to store the data for the reason that the database not only provides a variety of storage strategies but also meets the requirements of data consistency. Moreover, it is convenient to do data calculation based on the database. However, this storage way has some disadvantages, and it is not the best option in some scenarios. Another way to store the data is the file. The file system is more open and flexible, but the file itself has no computing power and cannot ensure the data consistency; nevertheless, using files to store data is

It is common that a contemporary information system has multiple or diverse data sources, especially an analytic application, which often needs to perform data analysis on several sources, sometimes of different types, through mixed computation. There are a variety of data sources, including RDB, NoSQL, files like CSV and Excel, JSON, XML, HDFS, Elasticsearch, Kafka, etc. Implementing a mixed computation on heterogeneous sources is not easy because they vary greatly in their computing ability. RDBs boast powerful ability on which computations can rely on, while NoSQL databases have rather weak computing ability and files do not have any. It is

What kind of code is considered low？ The term “Low-Code” has become popular in the past years, springing into existence a number of corresponding entrepreneurial teams. We only knew there were long codes and short codes before, only now do we know there are still high and low codes. Intuitively, the so-called low code is to make the code easier to write, in other words, the amount of code (which can be understood as workload) is less when doing the same task; In addition, another important indicator of low code is that the requirements for developers should be low enough.

As microservices get popular, there are and will be more and more applications developed based on microservices framework. To accomplish data processing tasks within microservices conveniently and efficiently thus becomes an inevitable issue. For a conventional monolithic application, we can implement computations in the database using the specialized, set-oriented SQL. The database’s powerful computing abilities enable quite easy data processing. In the microservices frameworks, a database is mainly used to realize data persistence and data is processed at the application (server) side. Hardcoding in a high-level language (mainly in Java) is the main approach used to process data at the server

Whether reporting, querying or analyzing, it always involves the data preparation. Data preparation is to process the raw data into a form that can be presented or analyzed (such as the common report dataset). However, data preparation is often not easy and may face various complex situations. Challenges in data preparation Diverse data sources The data of business comes from diverse sources at present. Theoretically, any data source can be used as the source of report data at the same time, such as RDB, NoSQL, CSV, Excel, Json, Hadoop, etc., which leads to the problem of diverse data sources in report development.

Reporting tools are now mostly used for building reports within applications. A mature reporting tool provides a lot of visualization configurations, a wealth of chart types, and various other functionalities, including export and print, making report development convenient and simple. In real-world situations, however, there is a certain thorny problem at a deep level during the report development process that bothers even reporting tool veterans much. Why do reporting tools have these problems? It appears that report development is to present data as a table or chart (graph) in a specified layout, which is reporting tools good at. Yet, the original

MongoDB is a typical NoSQL database. Its document-oriented structure makes both storage and access convenient and efficient. But the database has rather weak computing ability. Computations on MongoDB data, particularly complex ones, are hard to handle. A data computing engine having powerful computing capability is needed to work with MongoDB to achieve relevant computing tasks. The open-source esProc SPL is a specialized structured data computation engine. It supplies rich class libraries and all-around, database-independent computational capabilities. SPL has an independent procedural syntax that is particularly good at handling complex computations. It can help MongoDB increase its ability to compute, accomplish

Both XML and JSON are structured text formats good at recording hierarchical data that carries general and rich information. They are often used to interchange data and transmit information in systems like WebService/RESTful and Microservices. The hierarchical data has much more complicated structure than the traditional two-dimensional tables, and are difficult to process after being retrieved. In the days when there were no specialized JSON/XML processing technologies, JAVA developers usually turned to hardcoding or loaded JSON/XML data to the database for further processing. Hardcoding has weak computing ability, produces too much code and has very slow development efficiency. Loading data

Parsing, generating, querying and computing Excel files are common tasks Java often handles. As Excel has complicated file format, it is hard to read /write Excel files by writing your own code. But this is made convenient by various class libraries like POI, EasyExcel and JExcel, where POI is the best. POI has all-around and comprehensive abilities to read/write Excel files POI can read/write Excel files of various formats, from the older binary format (xls) to the current OOXML (xlsx) format. It supports both large memory jobs by reading/writing the whole Excel file at once and small memory jobs by