reproman-run

Synopsis

reproman-run [--version] [-h]
           [-l {critical,error,warning,info,debug,1,2,3,4,5,6,7,8,9}]
           [-m MESSAGE] [-r RESOURCE] [--resref-type TYPE]
           [--list {submitters,orchestrators,parameters,}]
           [--submitter NAME] [--orchestrator NAME] [--batch-spec PATH]
           [--batch-parameter PATH] [--job-spec PATH]
           [--job-parameter PARAM] [-i PATH] [-o PATH] [--follow [ACTION]]
           ...

Description

Run a command on the specified resource.

Two main options control how the job is executed: the orchestator and the submitter. The orchestrator that is selected controls details like how the data is made available on the resource and how the results are fetched. The submitter controls how the job is submitted on the resource (e.g., as a condor job). Use –list to see information on the available orchestrators and submitters.

Unless –follow is specified, the job is started and detached. Use reproman jobs to list and fetch detached jobs.

Options

COMMAND

command for execution. [Default: None]

–version

show the program’s version and license information and exit

-h, –help, –help-np

show this help message and exit. –help-np forcefully disables the use of a pager for displaying the help message

-l {critical,error,warning,info,debug,1,2,3,4,5,6,7,8,9}, –log-level {critical,error,warning,info,debug,1,2,3,4,5,6,7,8,9}

level of verbosity. Integers provide even more debugging information

-m MESSAGE, –message MESSAGE

Message to use when saving the run. The details depend on the orchestator, but in general this message will be used in the commit message. [Default: None]

-r RESOURCE, –resource RESOURCE

Name or ID of the resource to operate on. To see available resources, run ‘reproman ls’. Constraints: value must be a string [Default: None]

–resref-type TYPE

A resource can be referenced by its name or ID. In the unlikely case that a name collides with an ID, explicitly specify ‘name’ or ‘id’ to disambiguate. Constraints: value must be one of (‘auto’, ‘name’, ‘id’) [Default: ‘auto’]

–list {submitters,orchestrators,parameters,}

Show available submitters, orchestrators, or job parameters. If an empty string is given, show all. [Default: None]

–submitter NAME, –sub NAME

Name of submitter. The submitter controls how the command should be submitted on the resource (e.g., with CONDOR_SUBMIT). Use –list to see available submitters. Constraints: value must be one of (None, ‘pbs’, ‘condor’, ‘slurm’, ‘local’, ‘lsf’) [Default: None]

–orchestrator NAME, –orc NAME

Name of orchestrator. The orchestrator performs pre- and post-command steps like setting up the directory for command execution and storing the results. Use –list to see available orchestrators. Constraints: value must be one of (None, ‘plain’, ‘datalad-pair’, ‘datalad-no-remote’, ‘datalad-pair-run’, ‘datalad- local-run’) [Default: None]

–batch-spec PATH, –bs PATH

YAML file that defines a series of records with parameters for commands. A command will be constructed for each record, with record values available in the command as well as the inputs and outputs as {P[KEY]}. See –batch-parameter for an alternative method for simple combinations. [Default: None]

–batch-parameter PATH, –bp PATH

Define batch parameters with ‘KEY=val1,val2,…’. Different keys can be specified by giving multiple values, in which case the product of the values are taken. For example, ‘subj=mei,satsuki’ and ‘day=1,2’ would expand to four records, pairing each subj with each day. Values can be a glob pattern to match against the current working directory. See –batch-spec for specifying more complex records. This option can be given more than once. [Default: None]

–job-spec PATH, –js PATH

YAML files that define job parameters. Multiple paths can be given. If a parameter is defined in multiple specs, the value from the last path that defines it is used. Use –list to see available parameters for the built-in templates. This option can be given more than once. [Default: None]

–job-parameter PARAM, –jp PARAM

A job parameter in the form KEY=VALUE. If the same parameter is defined via a job spec, the value given here takes precedence. The values are available as fields in the templates used to generate both the run script and submission script. Use –list to see available parameters for the built-in templates. This option can be given more than once. [Default: None]

-i PATH, –input PATH

An input path to the command. How input paths are used depends on the orchestrator, but, at the very least, the orchestrator should try to make these paths available on the resource. This option can be given more than once. [Default: None]

-o PATH, –output PATH

An output path to the command. How output paths are handled depends on the orchestrator. This option can be given more than once. [Default: None]

–follow [ACTION]

Continue to follow the submitted command instead of submitting it and detaching. Constraints: value must be one of (False, True, ‘stop’, ‘stop-if-success’, ‘delete’, ‘delete-if-success’) [Default: False]

Authors

reproman is developed by The ReproMan Team and Contributors <team@reproman.org>.