ll.make – Object oriented make replacement

ll.make provides tools for building projects.

Like make it allows you to specify dependencies between files and actions to be executed when files don’t exist or are out of date with respect to one of their sources. But unlike make you can do this in an object oriented way and targets are not only limited to files.

Relevant classes are:

• Project, which is the container for all actions in a project and

• Action (and subclasses), which are used to transform input data and read and write files (or other entities like database records).

A simple script that copies a file foo.txt to bar.txt reencoding it from "latin-1" to "utf-8" in the process looks like this:

from ll import make

class MyProject(make.Project):
   name = "Acme.MyProject"

   def create(self):
      make.Project.create(self)
      source = self.add(make.FileAction("foo.txt"))
      temp = source.callattr("decode", "iso-8859-1")
      temp = temp.callattr("encode", "utf-8")
      target = self.add(make.FileAction("bar.txt", temp))
      self.writecreatedone()

p = MyProject()
p.create()

if __name__ == "__main__":
   p.build("bar.txt")

ll.make.filechanged(key)

Get the last modified date (or bigbang, if the file doesn’t exist).

class ll.make.Level

Bases: object

Stores information about the recursive execution of Actions.

ll.make.report(func)

Standard decorator for Action.get() methods.

This decorator handles proper reporting of nested action calls. If it isn’t used, only the output of calls to Project.writestep() will be visible to the user.

exception ll.make.RedefinedTargetWarning

Bases: Warning

Warning that will be issued when a target is added to a project and a target with the same key already exists.

exception ll.make.UndefinedTargetError

Bases: KeyError

Exception that will be raised when a target with the specified key doesn’t exist within the project.

ll.make.getoutputs(project, since, input)

Recursively iterate through the object input (if it’s a tuple, list or dict) and return a tuple containing:

• An object (data) of the same structure as input, where every action object encountered is replaced with the output of that action;

• A timestamp (changed) which the newest timestamp among all the change timestamps of the actions encountered.

If none of the actions has any data newer than since (i.e. none of the actions produced any new data) data will be nodata.

class ll.make.Action

Bases: object

An Action is responsible for transforming input data into output data. It may have no, one or many inputs which themselves may be other actions. It fetches, combines and transforms the output data of those actions and returns its own output data.

__init__()

Create a new Action instance.

get(project, since)

This method (i.e. the implementations in subclasses) is the workhorse of ll.make. get() must return the output data of the action if this data has changed since since (which is a datetime.datetime object in UTC). If the data hasn’t changed since since the special object nodata must be returned.

In both cases the action must make sure that the data is internally consistent, i.e. if the input data is the output data of other actions self has to ensure that those other actions update their data too, independent from the fact whether get() will return new data or not.

Two special values can be passed for since:

bigbang

This timestamp is older than any timestamp that can appear in real life. Since all data is newer than this, get() must always return output data.

bigcrunch

This timestamp is newer than any timestamp that can appear in real life. Since there can be no data newer than this, get() can only return output data in this case if ensuring internal consistency resulted in new data.

In all cases get() must set the instance attribute changed to the timestamp of the last change to the data before returning. In most cases this if the newest changed timestamp of the input actions.

execute(project, *args, **kwargs)

Execute the action: transform the input data in args and kwargs and return the resulting output data. This method must be implemented in subclasses.

getkey()

Get the nearest key from self or its inputs. This is used by ModuleAction for the filename.

call(*args, **kwargs)

Return a CallAction for calling selfs output with positional arguments from args and keyword arguments from kwargs.

getattr(attrname)

Return a GetAttrAction for getting selfs attribute named attrname.

callattr(attrname, *args, **kwargs)

Return a CallAttrAction for calling selfs attribute named attrname with positional arguments from args and keyword arguments from kwargs.

__iter__()

Return an iterator over the input actions of self.

iterallinputs()

Return an iterator over all input actions of self (i.e. recursively).

findpaths(input)

Find dependency paths leading from self to the other action input. I.e. if self depends directly or indirectly on input, this generator will produce all paths p where p[0] is self and p[-1] is input and p[i+1] in p[i] for all i in range(len(p)-1).

class ll.make.ObjectAction

Bases: ll.make.Action

An ObjectAction returns an object.

class ll.make.TransformAction

Bases: ll.make.Action

A TransformAction depends on exactly one input action and transforms the input data into output data.

class ll.make.CollectAction

Bases: ll.make.TransformAction

A CollectAction is a TransformAction that simply outputs its input data unmodified, but updates a number of other actions in the process.

addinputs(*otherinputs)

Register all actions in otherinputs as additional actions that have to be updated before self is updated.

class ll.make.PhonyAction

Bases: ll.make.Action

A PhonyAction doesn’t do anything. It may depend on any number of additional input actions which will be updated when this action gets updated. If there’s new data from any of these actions, a PhonyAction will return None (and nodata otherwise as usual).

__init__(*inputs, **kwargs)

Create a PhonyAction object. doc describes the action and is printed by the method Project.writephonytargets().

addinputs(*inputs)

Register all actions in inputs as additional actions that have to be updated once self is updated.

class ll.make.FileAction

Bases: ll.make.TransformAction

A FileAction is used for reading and writing files (and other objects providing the appropriate interface).

__init__(key, input=None, encoding=None, errors=None)

Create a FileAction object with key as the “filename”. key must be an object that provides a method open() for opening readable and writable streams to the file. input is the data written to the file (or the action producing the data). encoding is the encoding to be used for reading/writing. If encoding is None binary i/o will be used. errors is the codec error handling name for encoding/decoding text.

write(project, data)

Write data to the file and return it.

read(project)

Read the content from the file and return it.

get(project, since)

If a FileAction object doesn’t have an input action it reads the input file and returns the content if the file has changed since since (otherwise nodata is returned).

If a FileAction object does have an input action and the output data from this input action is newer than the file self.key the data will be written to the file. Otherwise (i.e. the file is up to date) the data will be read from the file.

chmod(mode=420)

Return a ModeAction that will change the file permissions of self to mode.

chown(user=None, group=None)

Return an OwnerAction that will change the user and/or group ownership of self.

class ll.make.MkDirAction

Bases: ll.make.TransformAction

This action creates the a directory (passing through its input data).

__init__(key, mode=511)

Create a MkDirAction instance. mode (which defaults to 0o777) will be used as the permission bit pattern for the new directory.

execute(project, data)

Create the directory with the permission bits specified in the constructor.

class ll.make.PipeAction

Bases: ll.make.TransformAction

This action pipes the input through an external shell command.

__init__(input, command)

Create a PipeAction instance. command is the shell command to be executed (which must read it’s input from stdin and write its output to stdout).

class ll.make.CacheAction

Bases: ll.make.TransformAction

A CacheAction is a TransformAction that passes through its input data, but caches it, so that it can be reused during the same build round.

class ll.make.GetAttrAction

Bases: ll.make.TransformAction

This action gets an attribute from its input object.

class ll.make.CallAction

Bases: ll.make.Action

This action calls a function or any other callable object with a number of arguments. Both positional and keyword arguments are supported and the function and the arguments can be static objects or actions.

class ll.make.CallAttrAction

Bases: ll.make.Action

This action calls an attribute of an object with a number of arguments. Both positional and keyword arguments are supported and the object, the attribute name and the arguments can be static objects or actions.

class ll.make.CommandAction

Bases: ll.make.TransformAction

This action executes a system command (via os.system()) and passes through the input data.

__init__(command, input=None)

Create a new CommandAction object. command is the command that will executed when execute() is called.

class ll.make.ModeAction

Bases: ll.make.TransformAction

ModeAction changes file permissions and passes through the input data.

__init__(input=None, mode=420)

Create an ModeAction object. mode (which defaults to 0644) will be use as the permission bit pattern.

execute(project, data, mode)

Change the permission bits of the file self.getkey().

class ll.make.OwnerAction

Bases: ll.make.TransformAction

OwnerAction changes the user and/or group ownership of a file and passes through the input data.

__init__(input=None, user=None, group=None)

Create a new OwnerAction object. user can either be a numerical user id or a user name or None. If it is None no user ownership will be changed. The same applies to group.

execute(project, data, user, group)

Change the ownership of the file self.getkey().

class ll.make.ModuleAction

Bases: ll.make.TransformAction

This action will turn the input string into a Python module.

__init__(input=None)

Create an ModuleAction.

This object must have an input action (which might be a FileAction that creates the source file).

addinputs(*inputs)

Register all actions in inputs as modules used by this module. These actions must be ModuleAction objects too.

Normally it isn’t necessary to call the method directly. Instead fetch the required module inside your module like this:

from ll import make

mymodule = make.currentproject.get("mymodule.py")

This will record your module as depending on mymodule, so if mymodule changes, your module will be reloaded too. For this to work you need to have an ModuleAction added to the project with the key "mymodule.py".

class ll.make.FOPAction

Bases: ll.make.TransformAction

This action transforms an XML string (containing XSL-FO) into PDF. For it to work Apache FOP is required. The command line is hardcoded but it’s simple to overwrite the class attribute command in a subclass.

class ll.make.AlwaysAction

Bases: ll.make.Action

This action always returns None as new data.

class ll.make.NeverAction

Bases: ll.make.Action

This action never returns new data.

class ll.make.Project

Bases: dict

A Project collects all Action objects from a project. It is responsible for initiating the build process and for generating a report about the progress of the build process.

strtimedelta(delta)

Return a nicely formatted and colored string for the datetime.timedelta value delta. delta may also be None in with case "0" will be returned.

strdatetime(dt)

Return a nicely formatted and colored string for the datetime.datetime value dt.

strcounter(counter)

Return a nicely formatted and colored string for the counter value counter.

strerror(text)

Return a nicely formatted and colored string for the error text text.

strkey(key)

Return a nicely formatted and colored string for the action key key.

straction(action)

Return a nicely formatted and colored string for the action action.

__setitem__(key, target)

Add the action target to self as a target and register it under the key key.

add(target, key=None)

Add the action target as a target to self. If key is not None, target will be registered under this key, otherwise it will be registered under its own key (i.e. target.key).

__getitem__(key)

Return the target with the key key. If an key can’t be found, it will be wrapped in a ll.url.URL object and retried. If key still can’t be found a UndefinedTargetError will be raised.

has_key(key)

Return whether the target with the key key exists in the project.

__contains__(key)

Return whether the target with the key key exists in the project.

create()

Create all dependencies for the project. Overwrite in subclasses.

This method should only be called once, otherwise you’ll get lots of RedefinedTargetWarnings. But you can call clear() to remove all targets before calling create(). You can also use the method recreate() for that.

recreate()

Calls clear() and create() to recreate all project dependencies.

argparser()

Return an argparse parser for parsing the command line arguments. This can be overwritten in subclasses to add more arguments.

parseargs(args=None)

Use the parser returned by argparser() to parse the argument sequence args, modify self accordingly and return the result of the parsers parse_args() call.

get(target)

Get up-to-date output data from the target target (which must be an action registered with self (or the id of one). During the call the global variable currentproject will be set to self.

build(*targets)

Rebuild all targets in targets. Items in targets must be actions registered with self (or their ids).

buildwithargs(args=None)

For calling make scripts from the command line. args defaults to sys.argv. Any positional arguments in the command line will be treated as target ids. If there are no positional arguments, a list of all registered PhonyAction objects will be output.

write(*texts)

All screen output is done through this method. This makes it possible to redirect the output (e.g. to logfiles) in subclasses.

writeln(*texts)

All screen output is done through this method. This makes it possible to redirect the output (e.g. to logfiles) in subclasses.

writeerror(*texts)

Output an error.

warn(warning, stacklevel)

Issue a warning through the Python warnings framework

writestacklevel(level, *texts)

Output texts indented level levels.

writestack(*texts)

Output texts indented properly for the current nesting of action execution.

writestep(action, *texts)

Output texts as the description of the data transformation done by the action arction.

writenote(action, *texts)

Output texts as the note for the data transformation done by the action action.

writecreatedone()

Can be called at the end of an overwritten create() to report the number of registered targets.

writephonytargets()

Show a list of all PhonyAction objects in the project and their documentation.

findpaths(target, source)

Find dependency paths leading from the action target to the action source.