Cross-platform testsuite for phylogenetic likelihoods

Hi everyone.

1. I am working on assembling a collection of (alignment, tree, model, parameters) combinations with known likelihoods to check that phylogeny programs are computing likelihoods correctly. So far, I have methods to test bali-phy, revbayes, PAUP*, raxml-ng, PhyML and IQ-Tree.

Hopefully this would also help other projects implement new models by checking that a new implementation obtains the correct likelihood. At the moment, we don’t check optimization quality, just accuracy for a user-specified set of parameters.

2. The gitlab URL is: https://gitlab.com/testiphy/testiphy.git

Here is what the output currently looks like for raxml-ng

    $ ./testiphy raxml-ng
    Running tests for 'raxml-ng':

    Running test: tests/likelihood/gtr_inv/1  ... ok
    Running test: tests/likelihood/gtr_expected_gamma_inv/1 ... ok
    Running test: tests/likelihood/gtr_expected_gamma_inv/2 ... ok
    Running test: tests/likelihood/1  ... ok
    SUCCESS! (0 unexpected failures, 0 expected failures, 4 tests total)

There are about 37 tests total, including for a variety of codon models, but for most of these I have only written control files for bali-phy or revbayes.

3. The raxml-ng driver looks like:

class raxml_ng(Program):
    def __init__(self, cmd):
        Program.__init__(self,cmd)
        self.name = "raxml-ng"
        self.likelihood_regex = r".*. initial LogLikelihood: ([^ ]+)$"
        self.extra_args = ['--opt-branches','off','--opt-model','off','--threads','1','--evaluate','--redo']

4. Other projects should be able to use testiphy in continuous integration tests via .travis.yml, then I’d be happy to help. This has helped me catch typos/thinkos that affect the likelihood during development.

• see https://github.com/bredelings/BAli-Phy/blob/master/.travis.yml (uses meson)
• see https://github.com/bredelings/BAli-Phy/blob/3.0.2/.travis.yml (uses autotools)

5. If anyone is interested in contributing by (i) improving the python code (ii) adding test-cases, (iii) requesting new features to enable practical testing, then please let me know.

I hope this project will be an opportunity to get people from different phylogenetics software communities communicating more, so if you want to incorporate these tests into your software, please let me know.

thanks,

-BenRI

Thanks for sharing this Ben, we will definitely start experimenting with this

Alexis