Global section

jflow WEB server and parameters can be defined in the global section.

Property Default Description
makeflow /usr/bin/makeflow The path to the makeflow executable.
batch_system_type local The name of the batch system type to use. Supported batch systems are:
  • local
  • condor
  • sge
  • moab
  • cluster
  • wq
  • hadoop
  • mpi-queue
batch_options - Those options will be added to all submitted batch files.
limit_submission 100 The limit number of jobs submitted in parallel.
server_socket_host 127.0.0.1 The WEB address to use to run the workflow management system server.
server_socket_port 8080 The WEB server socket port to use to run the workflow management system server.
date_format %d/%m/%Y The default date format to use in the application for date parameters. This must be a date format string.

Email section

jflow offers the possibility to send emails to the users at the end of a workflow execution. This can be done by editing the email section of the application.properties file.

To access this functionality, smtp_server, smtp_port, from_address and from_password are required parameters. The values to_address, subject and message can be globaly set from the jflow configuration file or can be set within a workflow by using respectivly the self.set_to_address(), self.set_subject() and self.set_message() methods. This allows to request, as a workflow input, a user email in order to send him a notification when the workflow is over.

Property Default Description
smtp_server - The smtp server used to send the email.
smtp_port - The smtp server port number.
from_address - Email used as "from" while sending the email.
from_password - The password associated with from_address on the smtp server.
subject - The subject of the mail. This value will be used for all workflow. It can be overloaded within the workflow implementation using self.set_subject("subject").
Placeholders can be used to access workflow attributes or unparametred methods: ###attribute### or ###method()###. For timestamps you can apply date function with the following syntax: ###attribute|date### or ###method()|date###".
message - The mail content. This value will be used for all workflow. It can be overloaded within the workflow implementation using self.set_message("message").
Placeholders can be used to access workflow attributes or unparametred methods: ###attribute### or ###method()###. For timestamps you can apply date function with the following syntax: ###attribute|date### or ###method()|date###".

Storage section

This section configure the paths used by all workflows.

On a multi-user computer or cluster you can use placeholder ###USER###. ###USER### will be replace by the environment variable $USER.

Property Default Description
log_file <path>/jflow.log The path to the log file for workflows execution. Errors and logs will be stored in this file.
work_directory <path>/work The path to the work directory. This is the location where workflows will write results. If using a cluster, this location should be accessible by all cluster nodes.
tmp_directory <path>/tmp The path to the tmp directory. If using a cluster, this location should be accessible by all cluster nodes.

Softwares section

While retrieving a path from a workflow using self.get_exec_path('my_software'), jflow will first look in the sofwares section of the configuration file to get the full path of 'my_software'. If this one is nout found, jflow will then search within the workflows/bin/ directory. Finally, jflow search in the system PATH variable to find the right path.

# where could be find binaries
[softwares]
cutadapt = /path/to/cutadapt

Components section

In a production environment, it can be useful to custom DMS options for each components. To do so, within the components section in the jflow configuration file, add a line following the schema: [component_name].batch_options = [options].

Cpu and memory values are then avalaible in the component via the methods get_cpu() and get_memory() on local and SGE enviroment only/

Local configuration

you can provide option with format mem=XX[K|M|G] cpu=XX"

[components]
BWAIndex.batch_options = "mem=1G"
BWAmem.batch_options = "mem=10G cpu=4"

SGE configuration

you can provide option as you would provide to SGE

[components]
BWAIndex.batch_options = -l mem=1G -l h_vmem=1G -q myflowq
BWAmem.batch_options = -l mem=10G -l h_vmem=10G -q myflow2q

Load modules

If a component required a module load, it can be set here. Module load will be run on the computing node, just before job execution.

[components]
markduplicates.genomestrip.modules = bioinfo/Java8
BWAmem.modules = bioinfo/bwa-0.7.15

Workflows section

Each workflow has a default group. In this section you can set a group name for each one of your workflow, thus while using the availablewf() plugin, you will be able to specify which group of workflow you want to get. You can set the group of a workflow by adding a line of the following schema : [workflow_classname].group = [group_name].

[workflows]
QuickStart.group = tutorial

Ressources section

In a production environment, it can be useful to set an ensemble of variables, such as path to databanks, for multiple workflows. Thus, jflow gives access to ressources settled within the application.properties file. All the values settled within this section are then accessible from any Component or Workflow through the get_resource() class method.

[resources]
ecoli_db = /path/to/ecoli536
phix_db = /path/to/phi.fa
...