Skip to content

Overview

A Scientific Pipeline is a containerized sequence of tools that are structured together to perform data analysis.

Pipelines enable scalable and reproducible workflows that ensure data reusability. Workflow management systems like Nextflow and WDL will version pipelines, so that researchers can ensure identical workflows when repeating data analysis for their samples.

Scientific Pipelines can be easily adopted into data analysis workflows since they do not require extensive coding expertise to deploy. However, users may generally require IT expertise, to allocate computational resources required for each stage of the pipeline run.

Quark simplifies pipeline deployment for both bench-scientists and bioinformaticians through a simplified no-code, no-IT and single-click interface.

Quark Pipelines

Quark provides bench-scientists and bioinformaticians an intuitive self-service environment that runs scientific pipelines out-of-the-box from the platform.

Pipelines can be identified, imported, and run independently on Quark without needing coding expertise. To view, import or build a pipeline, select Pipelines from the Navigator pane to the left of the Quark screen, which displays the following tabs:

  • Dashboard : The Dashboard provides a status summary of all pipeline runs on the platform for that calendar month.

  • Runs : Displays a list of all runs completed on the platform, including details such as Name, Status, Deployed Pipeline, Start and Finish times, and Total Runtime. The Search bar allows users to retrieve their runs using the Run Name assigned at the time of initiating a pipeline run, or using the name of the Pipeline. The Filter option allows users to access Runs based on their Status, Start / Finish times, or Created date.

  • Launchpad : provides a comprehensive and up-to-date list of all workflows available on the Quark platform. Users can run the workflow of their choice out-of-the-box from the Launchpad. Users can either: Navigate through the Filter option next to the Search bar, and find a workflow that matches project requirements or, Directly use the search bar on the top left to find the workflow of their choice. The Filter allows users to identify pipelines by Type or Category. The Sort By function sorts pipelines by Name or by version (Last Release).

  • My Pipelines : allows users to import and build pipelines. The window provides a list of all the pipelines published and deployed by the user. The Search bar accesses pre-configured pipelines.

Running Pipelines on Quark

Users can run their pipelines on Quark in three ways:

  • Quark enables users to run nf-core and AWS Healthomics pipelines out-of-the-box from the Launchpad.
  • Users can also import a pipeline from different workflow management systems (Nextflow, WDL, Snakemake) and configure them according to their project needs, from My Pipelines.
  • Quark additionally enables bioinformaticians to build new pipelines on the platform using a drag-and-drop interface called the Pipeline Builder, from My Pipelines.

The Results of all pipeline runs are retrievable from the Runs tab. Quark's Vizapp provides a curated collection of visualization tools that enable users to run their secondary data analyses from the Vizapp.