reproman.cmd

Wrapper for command and function calls, allowing for dry runs and output handling

class reproman.cmd.GitRunner(cwd=None, env=None, protocol=None)[source]

Bases: reproman.cmd.Runner

Runner to be used to run git and git annex commands

Overloads the runner class to check & update GIT_DIR and GIT_WORK_TREE environment variables set to the absolute path if is defined and is relative path

static get_git_environ_adjusted(env=None)[source]

Replaces GIT_DIR and GIT_WORK_TREE with absolute paths if relative path and defined

run(cmd, env=None, *args, **kwargs)[source]

Runs the command cmd using shell.

In case of dry-mode cmd is just added to commands and it is actually executed otherwise. Allows for separately logging stdout and stderr or streaming it to system’s stdout or stderr respectively.

Note: Using a string as cmd and shell=True allows for piping,
multiple commands, etc., but that implies shlex.split() is not used. This is considered to be a security hazard. So be careful with input.
Parameters:
  • cmd (str, list) – String (or list) defining the command call. No shell is used if cmd is specified as a list
  • log_stdout (bool, optional) – If True, stdout is logged. Goes to sys.stdout otherwise.
  • log_stderr (bool, optional) – If True, stderr is logged. Goes to sys.stderr otherwise.
  • log_online (bool, optional) – Either to log as output comes in. Setting to True is preferable for running user-invoked actions to provide timely output
  • expect_stderr (bool, optional) – Normally, having stderr output is a signal of a problem and thus it gets logged at ERROR level. But some utilities, e.g. wget, use stderr for their progress output. Whenever such output is expected, set it to True and output will be logged at DEBUG level unless exit status is non-0 (in non-online mode only, in online – would log at DEBUG)
  • expect_fail (bool, optional) – Normally, if command exits with non-0 status, it is considered an ERROR and logged accordingly. But if the call intended for checking routine, such alarming message should not be logged as ERROR, thus it will be logged at DEBUG level.
  • cwd (string, optional) – Directory under which run the command (passed to Popen)
  • env (string, optional) – Custom environment to pass
  • shell (bool, optional) – Run command in a shell. If not specified, then it runs in a shell only if command is specified as a string (not a list)
Returns:
Return type:

(stdout, stderr)
Raises:

CommandError – if command’s exitcode wasn’t 0 or None. exitcode is passed to CommandError’s code-field. Command’s stdout and stderr are stored in CommandError’s stdout and stderr fields respectively.
class reproman.cmd.Runner(cwd=None, env=None, protocol=None)[source]

Bases: object

Provides a wrapper for calling functions and commands.

An object of this class provides a methods that calls shell commands or python functions, allowing for protocolling the calls and output handling.

Outputs (stdout and stderr) can be either logged or streamed to system’s stdout/stderr during execution. This can be enabled or disabled for both of them independently. Additionally, a protocol object can be a used with the Runner. Such a protocol has to implement reproman.support.protocol.ProtocolInterface, is able to record calls and allows for dry runs.

call(f, *args, **kwargs)[source]

Helper to unify collection of logging all “dry” actions.

Calls f if Runner-object is not in dry-mode. Adds f along with its arguments to commands otherwise.

f : callable

*args, **kwargs:
Callable arguments
commands
cwd
dry
env
log(msg, level=10)[source]

log helper

Logs at DEBUG-level by default and adds “Protocol:”-prefix in order to log the used protocol.

protocol
run(cmd, log_stdout=True, log_stderr=True, log_online=False, expect_stderr=False, expect_fail=False, cwd=None, env=None, shell=None)[source]

Runs the command cmd using shell.

In case of dry-mode cmd is just added to commands and it is actually executed otherwise. Allows for separately logging stdout and stderr or streaming it to system’s stdout or stderr respectively.

Note: Using a string as cmd and shell=True allows for piping,
multiple commands, etc., but that implies shlex.split() is not used. This is considered to be a security hazard. So be careful with input.
Parameters:
  • cmd (str, list) – String (or list) defining the command call. No shell is used if cmd is specified as a list
  • log_stdout (bool, optional) – If True, stdout is logged. Goes to sys.stdout otherwise.
  • log_stderr (bool, optional) – If True, stderr is logged. Goes to sys.stderr otherwise.
  • log_online (bool, optional) – Either to log as output comes in. Setting to True is preferable for running user-invoked actions to provide timely output
  • expect_stderr (bool, optional) – Normally, having stderr output is a signal of a problem and thus it gets logged at ERROR level. But some utilities, e.g. wget, use stderr for their progress output. Whenever such output is expected, set it to True and output will be logged at DEBUG level unless exit status is non-0 (in non-online mode only, in online – would log at DEBUG)
  • expect_fail (bool, optional) – Normally, if command exits with non-0 status, it is considered an ERROR and logged accordingly. But if the call intended for checking routine, such alarming message should not be logged as ERROR, thus it will be logged at DEBUG level.
  • cwd (string, optional) – Directory under which run the command (passed to Popen)
  • env (string, optional) – Custom environment to pass
  • shell (bool, optional) – Run command in a shell. If not specified, then it runs in a shell only if command is specified as a string (not a list)
Returns:
Return type:

(stdout, stderr)
Raises:

CommandError – if command’s exitcode wasn’t 0 or None. exitcode is passed to CommandError’s code-field. Command’s stdout and stderr are stored in CommandError’s stdout and stderr fields respectively.
reproman.cmd.get_runner(*args, **kwargs)[source]

Just a little helper to hardlink files’s load