by Ned Thaddeus Taylor

ATHENA (Adaptive Training for High Efficiency Neural network Applications) is a Fortran library for developing and handling neural networks (with a focus on convolutional neural networks).

This repository has been migrated from the University of Exeter GitLab to GitHub to facilitate community interaction and support. The latest version, updates, and collaboration now take place on this GitHub repository.

GitLab Repository (Archived): https://git.exeter.ac.uk/hepplestone/athena

It was decided that this project should be migrated to allow for better community support (i.e. allowing community users to raise issues). All information has been ported over where possible. Issues have not been migrated over, these can be found in the old repository. Releases prior to 1.2.0 have not been migrated over, but they can still be found as tags in this repository.

The ATHENA library leverages Fortran's strong support of array arithmatics, and its compatibility with parallel and high-performance computing resources. Additionally, there exist many improvements made available since Fortran 95, specifically in Fortran 2018 (Reid 2018) (and upcoming ones in Fortran 2023), as well as continued development by the Fortran Standards committee. All of this provides a clear incentive to develop further libraries and frameworks focused on providing machine learning capabilities to the Fortran community.

While existing Fortran-based libraries, such as neural-fortran (Curcic 2019), address many aspects of neural networks, ATHENA provides implementation of some well-known features not currently available within other libraries; these features include batchnormalisation, regularisation layers (such as dropout and dropblock), and average pooling layers. Additionally, the library provides support for 1, 2, and 3D input data for most features currently implemented; this includes 1, 2, and 3D data for convolutional layers. Finally, the ATHENA library supports many convolutional techniques, including various data padding types, and stride.

One of the primary intended applications of ATHENA is in materials science, which heavily utilises convolutional and graph neural networks for learning based on charge densities and atomic structures. Given the unique data structure of atomic configurations, specifically their graph-based nature, a specialised API must be developed to accommodate these needs.

• Reid, J. (2018). The new features of fortran 2018. SIGPLAN Fortran Forum, 37(1), 5–43. https://doi.org/10.1145/3206214.3206215
• Curcic, M. (2019). A parallel fortran framework for neural networks and deep learning. SIGPLAN Fortran Forum, 38(1), 4–21. https://doi.org/10.1145/3323057.3323059

ATHENA is distributed with the following directories:

For extended details on the functionality of this library, please check out the wiki

NOTE: There currently exists no manual document. This will be included at a later date

The ATHENA library can be obtained from the git repository. Use the following commands to get started:

  git clone https://github.com/nedtaylor/athena.git
  cd athena

The library has the following dependencies:

• A Fortran compiler (compatible with Fortran 2018 or later)
• fpm or CMake for building the library

The library has been developed and tested using the following compilers:

• gfortran -- gcc 13.2.0
• ifort -- Intel 2021.10.0.20230609
• ifx -- IntelLLVM 2023.2.0

The library has also been tested with and found to support the following libraries:

• gfortran -- gcc 12.3

The library is set up to work with the Fortran Package Manager (fpm).

Run the following command in the repository main directory:

  fpm build --profile release

To check whether ATHENA has installed correctly and that the compilation works as expected, the following command can be run:

  fpm test --profile release

This runs a set of test programs (found within the test/ directory) to ensure the expected output occurs when layers and networks are set up.

Run the following commands in the directory containing CMakeLists.txt:

  mkdir build  
  cd build  
  cmake [-DCMAKE_BUILD_TYPE="optim;mp"] ..  
  make install  

This will build the library in the build/ directory. All library files will then be found in:

  ${HOME}/.local/athena

Inside this directory, the following files will be generated:

  include/athena.mod
  lib/libathena.a

To check whether ATHENA has installed correctly and that the compilation works as expected, the following command can be run:

  ctest

This runs a set of test programs (found within the test/ directory) to ensure the expected output occurs when layers and networks are set up.

After the library has been installed, a set of example programs can be compiled and run to test the capabilities of ATHENA on the MNIST dataset. Some of the examples can be run as-is, and do not require external databases. For those that require the MNIST (a set of 60,000 hand-written numbers for training and 10,000 for testing, 0-9) dataset (i.e. 'example/mnist_' directories ), the dataset must first be downloaded. The example program has been developed to accept a text-based format of the MNIST dataset. The .txt database that these examples have been developed for can be found here: https://github.com/halimb/MNIST-txt/tree/master

The link to the original MNIST database is: http://yann.lecun.com/exdb/mnist/

NOTE: For the mnist examples, the MNIST dataset must be downloaded. By default, the database is expected to be found in the directory path ../../DMNIST. However, this can be chaned by editing the following line in the example/mnist[_VAR]/test_job.in file to point to the desired path:

  dataset_dir = "../../DMNIST"

Using fpm, the examples are built alongside the library. To list all available examples, use:

  fpm run --example --list

To run a particular example, execute the following command:

  fpm run --example [NAME] --profile release

where [NAME] is the name of the example found in the list.

To compile and run the examples, run the following commands in the directory containing CMakeLists.txt:

  cd example/mnist
  make build optim [FC=FORTRAN-COMPILER]
  ./bin/athena_test -f test_job.in

After the example program is compiled, the following directories will also exist:

The example will perform a train over the MNIST dataset. Once complete, it will print its weights and biases to file, and test the trained network on the training set. The output from this can then be compared to the file expected_output_COMPILER.txt.

In the tools/ directory, there exist scripts that take utilise the wandb python package (Weights and Biases, a machine learning data tracker). Wandb is a Python module and, as such, a Python interface has been provided to call and run the Fortran example. The Python interface then reads the Fortran output files and logs the results to the wandb project.

Example wandb project link: https://wandb.ai/ntaylor/cnn_mnist_test/overview?workspace=user-ntaylor

To call/reference the ATHENA library in a program, include the following use statement at the beginning of the necessary Fortran file: use athena

During compilation, include the following flags in the compilation (gfortran) command:

-I${HOME}/.local/athena/include -L${HOME}/.local/athena/lib -lathena

• Ned Thaddeus Taylor

Please note that this project adheres to the Contributing Guide. If you are interested in contributing to this project, please contact Ned Taylor.

This work is licensed under an MIT license.

Automated reporting on unit test code coverage in the README is achieved through utilising the cmake-modules and dynamic-badges-action projects.