Bioinformatics Project

I’ve started working in a lab on a genomics project for grad school. The goal is investigating the relationships between transposons and antibiotic resistance in Drosophelia(fruit fly).

I’m not going to go into the details of the biology piece of the project in this post but want to detail the informatics pipeline I’ve been building and some additional adjacent important pieces of info I’ve learned along the way. This post may be a work in progress over the semester.

Filetypes

This is an interesting and important element of bioinformatics. I’m working with genomic data so all my exposure to these filetypes are nucleic acid based at the moment, so I will focus on those.

The two primary datatypes I’ve encountered so far are as follows:

.fasta file types
- These are raw sequence data.
- They can be nucleotides or amino acids.
- Subtypes of the fasta files are .fna and .faa which are fasta files but for nucleic acid and amino acids respectfully.
- As you can see in the following code snippet, a fasta file is just a file with a header and a bunch of single letter representations of nucleic acids(or amino acids).

>NC_004354.4 Drosophila melanogaster chromosome X
TGTAGATTCAATTGGTCAACGCAACAGTTATGTGAAAACATGGGGTTGCGGAGGTTTTTTGAATGAGACGAACATTTACAGCAGAGGAAAAAGCCTCTGTTTTTGAACTATGGAAGAACG......

.gff file types
- This is an corrolary file in tandem with the .fasta.
- It doesn’t have raw genome data, it’s purpose is to map genome coordinates to data about a specific section of the genome.
- So each row in this file has a field for a starting base and a field for an ending base and then annotation about what the specific sequence entails.
- In the following row from a .gff, you can see the starting base is located at 18709865 and ends at 18711638 in chromosome NC_004354.4 and codes for a Cyp6g1 gene. (NC_004354.4 is the official NCBI RefSeq accession number for the X chromosome of Drosophila melanogaster)

NC_004354.4    RefSeq    gene    18709865    18711638    .    +    .    ID=gene-Dmel_CG6585;Name=Cyp308a1

Bioinformatics tools

I’ve been using several bioinformatics tools so far.

Earl Gray
- This is a tool used to identify and classify transposable elements in genomes.
- https://github.com/TobyBaril/EarlGrey
- This is ultimately a pipeline built of several tools.
- The ultimate goal is that it checks for known TE’s, then runs a subtool called RepeatModeler which looks for novel TE’s based on pattern matching with various degrees of freedom.
- This tool was significantly difficult to build for me as there’s so many subtools and one wouldn’t discover an error in a subtool until hours into the genome run.
- Anytime I encountered an error I would devise a fix and add it to my build scripts here:
  - https://github.com/0foo/bioinformatics_pipeline/tree/master/scripts
  - Running these scripts in order should build the pipeline to run EarlGray with it’s default settings.
datasets
- tool by the NCBI to query and download data from their databases
- https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/
- You can use this to pull down genome data
- For example: this pulls down a json file of all the D.M. genomes available in the NCBI database.
- datasets summary genome taxon "Drosophila melanogaster" > "$CATALOG_FILE"
bedtools
- fast cli command suite for essentially working with genomes by co-ordinates.
- Can extract, compare, manipulate, based on co-ordinate.
- Works primarily with .gff files discussed earlier
Meme suite with FIMO
- Apparently this is a suite used to analyze a nucleic or amino acid sequence to identify known motifs, for example: gene sequences representing transcription factor binding site motifs.
- If you have a sequence of nucleotides or amino acids it will try to identify any motifs located inside of this sequence.
- I used it to identify sequences for known transcription factor binding site motifs.
- I have more to explore with this tool.
RepeatModeler
- https://github.com/Dfam-consortium/RepeatModeler
- A subtool run inside of Earl Gray pipeline but deserves its own mention.
- Looks for de novo transposable elements in genomic sequences
- It uses numerous tools and databases to do it’s work which make it somewhat fragile to setup
  - https://github.com/Dfam-consortium/RepeatModeler#source-distribution-installation
RepeatMasker
- Also, part of the Earl Gray pipeline
- Annotates the genome by masking the found repeat sequences.
- More detail here to be added soon.

Databases used

dfam database
- Essentially a database of Transposable elements
- https://www.dfam.org/home
blast database
- A type of database that you can convert a fasta file to with the makeblastdb command
- It’s optimized to do ultra fast sequence alignment and homology searches
NCBI database
- Incredible wealth of data that contains a great deal of genomic and proteomic datasets…plus much much more life sciences tools, literature, and data.

Earl Gray pipeline

This is deserving an entire post of it’s own, which I will go into soon.

Earl Gray output

A full log file run(minus repetetive progress update lines) of the complete Earl Gray pipeline can be seen here:
- logfile
An EarlGray run takes several hours on my average machine.
The following are the files output by EarlGray after finishing its run.
More detail to come on the content and purpose of each of these files.

root@bioinformatics-host /workspace/analysis/GCF_000001215.4/earlgray_out/GCF_000001215.4_EarlGrey (master) : tree -L 2
.
|-- GCF_000001215.4_Curated_Library
|   |-- 7227.RepeatMasker.lib
|   `-- GCF_000001215.4_combined_library.fasta
|-- GCF_000001215.4_Database
|   |-- GCF_000001215.4-families.fa
|   |-- GCF_000001215.4-families.stk
|   |-- GCF_000001215.4-rmod.log
|   |-- GCF_000001215.4.nhr
|   |-- GCF_000001215.4.nin
|   |-- GCF_000001215.4.njs
|   |-- GCF_000001215.4.nnd
|   |-- GCF_000001215.4.nni
|   |-- GCF_000001215.4.nog
|   |-- GCF_000001215.4.nsq
|   `-- GCF_000001215.4.translation
|-- GCF_000001215.4_EarlGrey.log
|-- GCF_000001215.4_RepeatLandscape
|   |-- GCF_000001215.4.filteredRepeats.withDivergence.gff
|   |-- GCF_000001215.4_classification_landscape.pdf
|   |-- GCF_000001215.4_split_class_landscape.pdf
|   |-- GCF_000001215.4_summary_table.tsv
|   `-- GCF_000001215.4_superfamily_div_plot.pdf
|-- GCF_000001215.4_RepeatMasker
|   |-- GCF_000001215.4_Release_6_plus_ISO1_MT_genomic.fna.prep.align
|   |-- GCF_000001215.4_Release_6_plus_ISO1_MT_genomic.fna.prep.cat.gz
|   |-- GCF_000001215.4_Release_6_plus_ISO1_MT_genomic.fna.prep.masked
|   |-- GCF_000001215.4_Release_6_plus_ISO1_MT_genomic.fna.prep.out
|   `-- GCF_000001215.4_Release_6_plus_ISO1_MT_genomic.fna.prep.tbl
|-- GCF_000001215.4_RepeatMasker_Against_Custom_Library
|   |-- GCF_000001215.4_Release_6_plus_ISO1_MT_genomic.fna.prep.align
|   |-- GCF_000001215.4_Release_6_plus_ISO1_MT_genomic.fna.prep.cat.gz
|   |-- GCF_000001215.4_Release_6_plus_ISO1_MT_genomic.fna.prep.masked
|   |-- GCF_000001215.4_Release_6_plus_ISO1_MT_genomic.fna.prep.out
|   `-- GCF_000001215.4_Release_6_plus_ISO1_MT_genomic.fna.prep.tbl
|-- GCF_000001215.4_RepeatModeler
|   `-- RM_232123.TueAug42120492026
|-- GCF_000001215.4_heliano
|-- GCF_000001215.4_mergedRepeats
|   `-- looseMerge
|-- GCF_000001215.4_strainer
|   |-- GCF_000001215.4-families.fa.strained
|   `-- TS_GCF_000001215.4-families.fa_2839
`-- GCF_000001215.4_summaryFiles
    |-- GCF_000001215.4.familyLevelCount.kable
    |-- GCF_000001215.4.familyLevelCount.txt
    |-- GCF_000001215.4.filteredRepeats.bed
    |-- GCF_000001215.4.filteredRepeats.gff
    |-- GCF_000001215.4.highLevelCount.kable
    |-- GCF_000001215.4.highLevelCount.txt
    |-- GCF_000001215.4.summaryPie.pdf
    |-- GCF_000001215.4_classification_landscape.pdf
    |-- GCF_000001215.4_combined_library.fasta
    |-- GCF_000001215.4_divergence_summary_table.tsv
    |-- GCF_000001215.4_split_class_landscape.pdf
    `-- GCF_000001215.4_superfamily_div_plot.pdf

14 directories, 42 files

TO BE CONTINUED!

2026-08-06