Resources for “Make”#
Manual#
Discussions#
Recursive Make Considered Harmful. This is a well-known paper on why you shouldn’t use nested makefiles. To summarise: if you do this Make can’t see the entire DAG and that leads to problems.
Non-Recursive Make Considered Harmful: This is a research paper describing the failings of Make for large and complex builds.
Blogs#
Of course we are not the first to suggest the use of Make for reproducibility! The blog posts cited below were found after the above tutorial was written, but can add further information and examples.
Reproducibility is hard. Discusses making a research project reproducible using Make.
GNU Make for Reproducible Data Analysis. Argues for using Make for reproducible analysis in a similar vein as we do above.
Reproducible Bioinformatics Pipelines using Make. A quite extensive tutorial on using Make for data analysis.
Automatic Data-analysis Pipelines. A similar tutorial that uses R for the analysis.
Writing a reproducible Paper. A similar tutorial with Python using variables to populate tables in the manuscript.
Tools#
Plot the DAG of the Makefile with makefile2graph.
Alternatives to Make#
There are many alternatives to Make. Below are some that caught our eye and that might be worth a look.
SnakeMake. A Python3-based alternative to Make. Snakemake supports multiple wildcards in filenames, supports Python code in rules, and can run workflows on workstations, clusters, the grid, and in the cloud without modification.
Tup. A fast build system that processes prerequisites bottom-up instead of Make’s top-down. The speed looks impressive and the paper describing it is interesting, but for small projects Make’s speed will not be a bottleneck. The Tupfile syntax is not compatible with that of Makefiles.
Bazel. An open-source version of Google’s Blaze build system.
Buck. Facebook’s build system.