Skip to content

Alignment Filtering

Parts of a SEGUL commandThe command segul align filter --dir alignments/ --min-len 500 has five parts: the program segul, the command align for the task group, the subcommand filter for the task, the input --dir alignments/ (folder or files), and the option --min-len 500 (filtering method).segulProgramalignCommandtask groupfilterSubcommandtask--dir alignments/Inputfolder or files--min-len 500Optionsfiltering method

In a typical genomic workflow, you may want to filter problematic alignments before running a phylogenetic analysis. This feature provides multiple ways to filter alignments.

Available filtering methods (required SEGUL v0.23.0 and later):

  1. Min or max alignment length
  2. Data matrix completeness
  3. Min, max, or percentage of parsimony informative sites
  4. Proportion of missing data
  5. Minimum taxon counts
  6. Taxon ID

The command is structured as below:

Terminal window
segul align filter <input-option> [alignment-path] <filtering-option> <value>

To filter alignments in the alignments/ directory with a minimal length of 500 bp, use the command below:

Terminal window
segul align filter --dir alignments/ --input-format nexus --min-len 500

All command examples:

# Minimum alignment length
segul align filter --dir alignments/ --input-format nexus --min-len 500
# Maximum alignment length
segul align filter --dir alignments/ --input-format nexus --max-len 800
# Data matrix completeness
segul align filter --dir alignments/ --input-format nexus --percent 0.8
# Using multiple data matrix completeness values
segul align filter --dir alignments/ --input-format nexus --npercent 0.9 0.8 0.75
# Minimum parsimony informative sites
segul align filter --dir alignments/ --input-format nexus --min-pinf 50
# Maximum parsimony informative sites
segul align filter --dir alignments/ --input-format nexus --max-pinf 200
# Percentage of minimal parsimony informative sites
segul align filter --dir alignments/ --input-format nexus --percent-inf 0.8
# Proportion of missing data
segul align filter --dir alignments/ --missing-data 0.3
# Minimum taxon counts
segul align filter --dir alignments/ --min-ntax 100
# Taxon ID
segul align filter --dir alignments/ --taxon-id path/to/taxon_ids.txt

To check the available filtering options and other command arguments, use the --help option:

Terminal window
segul align filter --help

You can filter alignments based on their length using the --min-len option. For example, if you want to filter alignments in the alignments/ below:

Terminal window
alignments/
├── locus_1.nexus
├── locus_2.nexus
└── locus_3.nexus

To filter alignments with a minimum length of 200 bp, use the command below:

Terminal window
segul align filter --dir alignments/ --input-format nexus --min-len 200

To filter alignments based on the maximum length, use the --max-len option. For example:

Terminal window
segul align filter --dir alignments/ --input-format nexus --max-len 800

Filtering based on data matrix completeness

Section titled “Filtering based on data matrix completeness”

This filtering method is based on the percentage of minimal taxa present in your alignments, similar to phyluce_align_get_only_loci_with_min_taxa in Phyluce pipeline. For example, given a collection of alignments with 100 taxa with 90 percent completeness, SEGUL will filter alignments containing at least 90 taxa.

We use the --percent option with input in decimals to filter alignments based on data matrix completeness. For example, here we will filter alignments in the alignments directory with 80 percent data matrix completeness:

Terminal window
segul align filter --dir alignments/ --input-format nexus --percent 0.8

You can also write the percentage as .8, and it will parse the same way you write it with 0.

Using multiple data matrix completeness values

Section titled “Using multiple data matrix completeness values”

Sometimes, we want to build phylogenetic trees with different data matrix completeness. Using segul, we can do it in a single command using the npercent option:

Terminal window
segul align filter --dir alignments/ --input-format nexus --npercent 0.9 0.8 0.75

SEGUL will create an output directory for each data matrix completeness value, with the percentage values suffixed on the directory names. For the example above, it will create three directories named alignments_90p, alignments_80p, and alignments_75p.

Filtering based on parsimony informative sites

Section titled “Filtering based on parsimony informative sites”

SEGUL provides two ways of filtering based on parsimony informative sites:

This option filters alignments that contain at least the specified number of informative sites.

For example, in the command below, we will filter alignments in the alignments directory with a minimum containing 50 parsimony informative sites:

Terminal window
segul align filter --dir alignments/ --input-format nexus --min-pinf 50

To filter alignments based on the maximum number of informative sites, use the --max-pinf option. For example:

Terminal window
segul align filter --dir alignments/ --input-format nexus --max-pinf 200

Percentage of minimal parsimony informative sites

Section titled “Percentage of minimal parsimony informative sites”

This value is counted based on the highest number of informative sites on your alignments. It is similar to computing data matrix completeness but based on the number of informative sites. You only need to input the percentage value. SEGUL will determine the highest number of informative sites across all input alignments.

For example, the highest number of informative sites in the input alignments is 100. We will filter with a minimal 80 percent of the highest parsimony informative value. Using the --percent-inf .8 option, SEGUL will filter alignments with at least containing 80 parsimony informative sites:

Terminal window
segul align filter --dir alignments/ --percent-inf 0.8

Filtering based on the proportion of missing data

Section titled “Filtering based on the proportion of missing data”

The proportion of missing data is calculated using the following formula:

Proportion of missing data = (Total missing data in alignment) / (Total number of characters in alignment)

It works at per alignment level. For example, if an alignment has 1000 characters and 300 of them are missing (gaps or ambiguous characters), the proportion of missing data would be 0.3 (or 30%).

SEGUL will will copy the matching alignments to the output directory. To filter alignments based on the proportion of missing data, use the --missing-data option. For example:

Terminal window
segul align filter --dir alignments/ --missing-data 0.3

To filter alignments based on the minimum number of taxa present in each alignment, use the --min-ntax option. For example, to filter alignments with at least 100 taxa, use the command below:

Terminal window
segul align filter --dir alignments/ --min-ntax 100

The taxon ID is the same as your sequence ID. To use this feature, you need to provide a text file containing a list of taxon IDs (one taxon ID per line). SEGUL will filter alignments that contain the taxon IDs specified in the file. For example, if you have a file named taxon_ids.txt with the following content:

taxon_1
taxon_2
taxon_3

You can use the following command to filter alignments in the alignments/ directory that contain these taxon IDs:

Terminal window
segul align filter --dir alignments/ --taxon-id path/to/taxon_ids.txt

By default, SEGUL output directories for filtering are based on the input directory names suffixed with filtering values. The format is as follows:

[input-dir](underscore)[filtering-values]

For example, if your input directory is named alignments/ and the filtering option is a minimal alignment length of 500 bp, the output directory will be alignments_500bp.

The --output or -o option allows you to change the prefix names of the output directory.

By default, SEGUL copies the alignments that match the filtering option. Instead of copying the files, you can concatenate them using the --concat flag. All options for the align concat subcommand are available for this task. You are, however, required to specify the --output or -o name and the partition format --partition-format or -p option.

For example, in the command below, we will filter based on alignments with a length of 500 bp and will concat the result:

Terminal window
segul align filter --dir alignments/ --input-format nexus --min-len 500 --concat --partition-format raxml -o concat_alignment

You can use --input or -i to specify a single alignment file or multiple alignments using wildcards. For example:

Terminal window
segul align filter --input alignments/*.fasta --min-len 300

By default, SEGUL assumes the input alignments are DNA sequences. It will fails to run if the data is amino acid sequences. To filter amino acid alignments, use the --datatype aa option. For example:

Terminal window
segul align filter --dir alignments/ --input-format fasta --datatype aa --min-len 300
OptionDescription
-d, --dirDirectory with alignments
-i, --inputAlignment files. Allows wildcards
-f, --input-formatInput format: auto (default), fasta, nexus, or phylip
--datatypeData type: dna (default), aa, or ignore
--min-lenMinimum alignment length
--max-lenMaximum alignment length
--percentMinimum data matrix completeness, as a proportion of taxa
--npercentSeveral data matrix completeness values. Writes one output per value
--min-pinfMinimum number of parsimony informative sites
--max-pinfMaximum number of parsimony informative sites
--percent-infMinimum parsimony informative sites, as a proportion of the highest count
--missing-dataMaximum proportion of missing data
--min-ntaxMinimum number of taxa
--taxon-idText file of taxon IDs, one per line
--concatConcatenate the filtered alignments instead of copying them
-F, --output-formatOutput format with --concat: nexus (default), phylip, fasta, nexus-int, phylip-int, or fasta-int
-p, --partition-formatPartition format with --concat: nexus (default), charset, or raxml
--codonWrite the partition as a codon model, with --concat
--prefixPrefix of the output files, with --concat
-o, --outputOutput directory. Default: named after the input directory and filter value
--forceOverwrite an existing output directory
--logLog file path. Default: segul.log

Use one filtering option per run.

This material is based upon work supported by the National Science Foundation under grant numbers: DEB-1754393 and DEB-2244754. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.