Mortar has joined Datadog, the leading SaaS-based monitoring service for cloud applications. Read more about what this means here.

ShellScript Tasks

ShellScript tasks provide a way to run bash commands from within luigi.

mortar-luigi

These Tasks are part of the mortar-luigi project, an open-source collection of extensions for using Mortar from within Luigi. It is installed automatically when you use the mortar local:luigi or mortar luigi commands.

ShellScriptTask

Runs a shellscript.

Example Usage

from mortar.luigi import shellscript

class MyShellScriptTask(shellscript.ShellScriptTask):

    token_path = 'path-to-token-output'

    def subprocess_commands(self):
        return 'cd my/dir; ls;

    def requires(self):
        return [OtherTask()]

This task runs a series of shellscript commands defined in subprocess commands. After successful completion, it will write an output token at the given token_path location.

requires needs to be defined for every Luigi task, and it indicates dependencies for the task. This can either be other Luigi tasks, data in a specified location, or nothing.