Majops Skeleton Scripts
for Single Pipelines
for Single Pipelines
If you would like to build a script in your single pipeline according to Majops, you always have to make use of one of Majops Skeleton Scripts.
There are 3 types of Majops Skeleton Scripts for a single pipeline:
- Process scripts
- Highway bash script
- Start bash script
Each type of Majops Skeleton Script will be explained below.
Process scripts:
Process scripts are used for processing the data in a pipeline. These process scripts will be called from the highway bash script. The process scripts are in several programming languages. The choice of which programming language is chosen depends on the task to be carried out in that process.
Structure of a Majops Skeleton Process Script:
Each Majops Skeleton Process Script has the same structure for each programming language. Its structure could be divided in 6 parts:
- Imports of libraries
- Declarations of global variables
- The function/method showUsageInformation()
- All the other functions/methods for the script itself
- The function main()
- Additional information of the script itself for:
- Remarks
- Descriptions how the script works
- Possible future improvements
The Majops Skeleton Process Scripts can be downloaded for the following programming languages:
Bash, C, C++, D, Go, Java, Lua, Pascal, Perl, Python2, Python3, R, Ruby, Scala.
Click here to go to the download page for the Majops Skeleton Process Scripts for a single pipeline.
Bash, C, C++, D, Go, Java, Lua, Pascal, Perl, Python2, Python3, R, Ruby, Scala.
Click here to go to the download page for the Majops Skeleton Process Scripts for a single pipeline.
Instruction movie about the process scripts:
In the next instruction movie, there will be explained about the process scripts. Directly after that, a process script is programmed as example.
Instruction movie 4: Explanation about the process scripts.
In this instruction movie, there will be explained about the process scripts. Directly after that, a process script is programmed as example.
In this instruction movie, there will be explained about the process scripts. Directly after that, a process script is programmed as example.
Highway bash script:
In the highway bash script, process scripts and/or normal programs are called after one another. The highway bash script can be called by a start bash script or by another highway bash script.
It is also possible that in a highway bash script another pipeline(s) is/are called.
Click here to go to the download page for the Majops Skeleton Highway Bash Script for a single pipeline.
It is also possible that in a highway bash script another pipeline(s) is/are called.
Click here to go to the download page for the Majops Skeleton Highway Bash Script for a single pipeline.
Instruction movie about the highway bash script:
In the next instruction movie, there will be explained about the highway bash script. Directly after that, an example of a pipeline is built from that same highway bash script.
Instruction movie 5: Explanation about the highway bash script.
In this instruction movie, there is explained about the highway bash script. Direct after that, an example of a pipeline is built from that same highway bash script.
In this instruction movie, there is explained about the highway bash script. Direct after that, an example of a pipeline is built from that same highway bash script.
Start bash script:
Each single pipeline has 1 start bash script. This start bash script is only used to start the pipeline. The start bash script generates the right command line arguments for the highway bash script it starts, depending on the answers of the user.
It is always in the programming language Bash and it should always be built at the last moment of building a single pipeline.
Click here to go to the download page for the Majops Skeleton Start Bash Script for a single pipeline.
It is always in the programming language Bash and it should always be built at the last moment of building a single pipeline.
Click here to go to the download page for the Majops Skeleton Start Bash Script for a single pipeline.
Structure of the Majops Skeleton Start Bash Script of a single pipeline:
The Majops Skeleton Start Bash Script has the following structure:
- Additional information of the single pipeline as comments
- An if statement for the Show Usage Information, in which information about the single pipeline is given.
- Code that will show the possible input files and/or input directories to the user, when he starts the single pipeline with start bash script.
- Code that will show questions to the user, in which he has to answer those for generating the right input for the highway script of the pipeline.
- An alphanumeric overview of all the scripts and/or files that should be present in the single pipeline (zip file).
These files are checked, if each of them is present. If one is absent, the pipeline will be aborted and a list of missing scripts and/or files will be shown on the screen to the user. - The highway bash script of the pipeline is called with the right generated command line arguments. After that, the pipeline starts running.
Reasons for the existence of the start bash script:
There are several reasons why the start bash script has been introduced in Majops. The first reason is that a user can recognize very easily and find out how a single pipeline should be started.
The second reason is that a user can see immediately, which highway bash script starts the pipeline. The highway of a pipeline is mostly recognized by a bash script that ends with "...Pipeline.sh" in its file name. If you have one highway bash script in your pipeline, nothing is happening. But problems will arise, if you have more highway bash scripts in your pipeline. A user cannot see anymore which highway bash script is the one that will start the pipeline. By using a start bash script, that problem is solved.
The third reason is that a user does not have to find out what command line arguments he has to type in for the highway bash script, that is first called. Because the start bash script asks questions to the user, which he has to answer, the start bash script generates the right command line arguments for the user for the highway bash script.
The fourth reason is that the start bash script can check the presence of every file and directory. If one is absent, the pipeline will not continue.
The second reason is that a user can see immediately, which highway bash script starts the pipeline. The highway of a pipeline is mostly recognized by a bash script that ends with "...Pipeline.sh" in its file name. If you have one highway bash script in your pipeline, nothing is happening. But problems will arise, if you have more highway bash scripts in your pipeline. A user cannot see anymore which highway bash script is the one that will start the pipeline. By using a start bash script, that problem is solved.
The third reason is that a user does not have to find out what command line arguments he has to type in for the highway bash script, that is first called. Because the start bash script asks questions to the user, which he has to answer, the start bash script generates the right command line arguments for the user for the highway bash script.
The fourth reason is that the start bash script can check the presence of every file and directory. If one is absent, the pipeline will not continue.
Working of the start bash script:
Figure 6 shows how the start bash script really works. You can see that when the start bash script is started, the user has to answer questions, after which right command line arguments are generated for the highway bash script for starting the pipeline.
Figure 6: Working of the start bash script.
Here is shown how the start bash script works. When the start bash script is started, the user has to answer questions, after which right command line arguments are generated for the highway bash script for starting the pipeline.
Here is shown how the start bash script works. When the start bash script is started, the user has to answer questions, after which right command line arguments are generated for the highway bash script for starting the pipeline.
Instruction movie about the start bash script:
In the next instruction movie, there will be explained about the start bash script.
Instruction movie 6: Explanation about the start bash script.
In this instruction movie, there is explained about the start bash script.
In this instruction movie, there is explained about the start bash script.
Instruction movie about finalizing a single pipeline with the start bash script:
In the next instruction movie, there will be explained how a single pipeline is finalized with the start bash script. First, the start bash script itself is programmed. After that, the Show Usage Information's of the start bash script and the highway bash script are written. Finally, everything in the pipeline is started. At the end, a flowchart is also drawn in Dia for the single pipeline.
Instruction movie 7: Finalizing a single pipeline with the start bash script.
A single pipeline is finalized with the start bash script. First, the start bash script itself is programmed. After that, the Show Usage Information's of the start bash script and the highway bash script are written. Finally, everything in the single pipeline is started. At the end, a flowchart is also drawn for the single pipeline.
A single pipeline is finalized with the start bash script. First, the start bash script itself is programmed. After that, the Show Usage Information's of the start bash script and the highway bash script are written. Finally, everything in the single pipeline is started. At the end, a flowchart is also drawn for the single pipeline.
The single pipeline 'Several Numbers Calculation Maker Pipeline' in instruction movie 7 can be downloaded below.
Several Numbers Calculation Maker Pipeline
|
|