Installation

Requirements

The pipeline has different requirements depending on your deployment target. Here are the core requirements for each deployment option:

Local Deployment

For local deployment, you need:

  • nextflow (version 22.10.8 recommended)

  • docker (19.03+ if going to use GPUs, e.g. for spikesort_kilosort* workflows)

  • figurl (optional, for cloud visualization)

SLURM Deployment

For SLURM cluster deployment:

  • nextflow (version 22.10.8 recommended)

  • apptainer or singularity

  • Access to a SLURM cluster

  • figurl (optional, for cloud visualization)

Installation Steps

Local Setup

  1. Install Nextflow:

    Follow the Nextflow installation guide

  2. Install Docker:

    Follow the Docker installation instructions

  3. (Optional) Set up Figurl:

    1. Initialize Kachery Client:

      1. Register at kachery.vercel.app using your GitHub account.

      2. Go to settings and provide your name, an email address and a short description of your research purpose.

      3. Set the KACHERY_API_KEY environment variable with your assigned API key.

    2. Set credentials:

      • Click on settings and generate a new API key.

      • Set environment variables:

      export KACHERY_API_KEY="your-client-id"
# Optional: Set custom Kachery zone
export KACHERY_ZONE="your-zone"

    3. (optional) Set up a custom kachery zone:

      If you plan to use the Figurl service extensively, please consider creating your own “zone”. Follow the instructions in the Kachery documentation.

SLURM Setup

  1. Install Nextflow on your cluster environment

  2. Ensure Apptainer/Singularity is available

  3. Set up environment variables:

    # Optional: Set custom Apptainer (or Singularity) cache directory
export NXF_APPTAINER_CACHEDIR="/path/to/cache"
# export NXF_SINGULARITY_CACHEDIR="/path/to/cache"

  4. (Optional) Follow the same Figurl setup steps as in the local deployment

Clone the Repository

Clone the pipeline repository:

git clone https://github.com/AllenNeuralDynamics/aind-ephys-pipeline.git
cd aind-ephys-pipeline
cd pipeline

The pipeline is now ready to be configured and run on your chosen platform.