research-project/README.md

Functional MRI for Neural Networks

An attempt to create real-time in depth visualisation for neural networks.

Requirements

The following programs/libraries need to be installed on your system:

• CMake (at least 3.8)
• Caffe
• Google log (glog)
• OpenCV (at least 2.6, 3.x works fine.)
• Boost System
• PNG++

Note that most of these are dependencies of Caffe, and thus will probably already be on your system. For OS-specific instructions, see below.

Arch Linux

1. Install either caffe or caffe-cpu from the AUR.
2. Install other dependencies:
   • png++
   • cmake
   • freeglut
3. Do the build as normal

Ubuntu

Caffe is available from the Ubuntu repositories from version 17.10 (Artful Aardvark) onwards, and can be used to simplify the installation process. The commands required are as follows:

# Install dependencies
sudo apt install libcaffe-cpu-dev freeglut3-dev libpng++-dev \
    build-essential cmake libopencv-dev libboost-system-dev \
    libgoogle-glog-dev libblas-dev libprotobuf-dev

# Do the build normally

Older versions of Ubuntu need some help, as the caffe package does not exist, and the supplied cmake version is too old.

1. First, install a recent version of cmake. Any version from 3.8 onwards will do.
2. Then, install caffe from source. Installation instructions can be found on the Caffe website. Be sure to also install the development headers.
3. Run the build as normal.

Building

The build process is based on CMake. This means you can easily create a build as follows. Starting from the project root:

mkdir build && cd build
cmake ..
make

Compilation is a little slow due to the inclusion of Boost.

Usage

This program can operate on most Caffe models, provided they don't split/merge the input, since the visualisers cannot handle more than one input.

There is one valid model provided, Caffenet. It can be downloaded by running ./download-models.sh from the data folder.

After that, you can, for example, run the program as follows (assuming you are in the build directory)

./fmri -n ../data/models/caffenet/model-dedup.prototxt \
    -w ../data/models/caffenet/bvlc_reference_caffenet.caffemodel \
    -l ../data/ilsvrc12/synset_words.txt \
    ../data/samples/*.jpg

This will run the network on the deduplicated caffenet (see: limitations) with the correct weights file and the labels file supplied, on all jpegs located in the samples folder. More advanced usage can be discovered using the -h option of the executable.

Controls

You can move around with the WASD keys, and look around using the mouse. Arrow keys change the currently loaded input.