Imaging Automation & Feedback

Our instrumentation is equipped with several options to develop automated routines, allowing the automation of image acquisition, analysis or the integration of both in feedback microscopy solutions.

Microscopy Automation ↔ Adaptive Feedback Microscopy ↔ smart Microscopy

Imaging Automation & Feedback definitions

0.) Automation: setting a repetitive or remote routine (remote pipetting, correction collars, temperature control).

1.) Automatic imaging: Unsupervised acquisition of multi-dimensional data (multi-color time series, etc.): configuring micro-manager to run a camera, batch data processing, etc.

2.) Guided acquisition: imaging driven by data extracted from previous analysis: segmentation + high resolution imaging.

3.) Smart microscopy: imaging driven by real-time image analysis: object tracking, power adjustment.


Automation is the use of artificial means to execute tasks without the intervention of a human.

It is mostly used for repetitive, labor-intensive or remote tasks. In the IOF facility multiple automation solutions have been incorporated or developed to our machine park. Some examples:

Remote Pipette and Remotely-Controlled Correction Collar (R-CCC)

In both cases the problem is safely access the system while it’s imaging, particularly when this disrupts the experiment or endangers the operator. The Remote Pipette can deliver reagents inside a closed chamber allowing, for instance, to monitor the process by TIRF microscopy, which would be otherwise impossible. Similarly, adjusting the correction collar manually is not feasible for laser imaging modes, but straightforward using a remotely-controlled unit (Leica DIVE, Zeiss LSM-04).

Water Immersion Dispenser (WID)

The WID allows to switch from air to water objectives (e.g. for overview + scanning) without stopping experiments. The replenishment can be scheduled according to imaging conditions.

There are WIDs installed in the Ti2-02 and 03 widefield microscopes and at the CSU-W1-01 / 02 and 03 spinning disk confocals.


Sample Flipper

The sample flipper allows certain samples to rest vertically (in their natural position) while being tilted for imaging cycles only. This solution is portable and it can be installed in many different microscopes. It can be used in combination with (ISTA) tracker.




Data infrastructure

Together, IT and us established a file server infrastructure that safely and continuously pulls data from the microscopes, making it available almost immediately for its analysis or processing. For the near future, we are working towards offering automatic analysis pipelines (stitching, fusion, etc.). These or other procedures can be carried out automatically as soon as the data arrives the data server.


Automatic Imaging

Automation, in the context of imaging, starts as soon as images are not saved by hand (e.g. taking photographs). It is an indispensable requirement for acquiring time series. Every modern microscope can be considered automatic, as long as it incorporates image acquisition.

Multi-block acquisition

Most vendors offer multi-block acquisition options for complex imaging routines. In these setups, multiple imaging sessions (imaging protocol, settings, positions, etc.) can be enclosed in separate blocks and executed in a sequence, or multiple times in loops. These modules provide already great flexibility for automatic imaging.

  • Carl Zeiss: Experiment designer
  • Leica: Live Data Mode
  • Nikon: Sequence Acquisition and JOBS
  • Olympus: Experiment Manager

To allow the Zeiss Experiment Designer to execute a large number of blocks without running into RAM memory issues we have the Multiblock Made Easy macro.

Custom setups

Some of our automation projects consisted in re-purposing retired microscopes to extend our machine park (Zeiss Examiner D1). In general retired optical setups contain many fully functional parts that cannot be used as intended originally. In this cases, building a new automatic setup requires rebuilding hardware (HW) and reimplementing software (SW). We also find the appropriate components and a way to automate them (Bioluminescence Imaging System) or build the complete optical system (Top-View optics for AFM).

Vertical Confocal microscopes

The LSM-05  and LSM-06 are dedicated systems for plant imaging. Other than the adaptations for setting up the stands vertically, these setups required the re-calibration of the confocal components and the design and fabrication of many specific parts: magnetic shields, sample holders, etc.

Zeiss Examiner D1

A microscope stand is re-purposed as a ‘new’ upright widefield system, compatible with multi-dimensional imaging approaches (Zeiss ZEN Connect)






Bioluminescence Imaging System

The bioluminescence setup acquires long exposure time images using an EMCCD camera, driven by micro-manager.





Top-View Optics for Atomic Force Microscopy

The top-view optics module is an accessory Long Working Distance  Reflection Microscope, that allows the location of alignment marks in non-transparent samples for Atomic Force Microscopy. Its camera is controlled within the JPK native AFM software.




Guided Acquisition

Guided Acquisition refers to experiments where data acquired initially defines the subsequent imaging procedure on the fly. It is not interactive during the acquisition but remains useful for high throughput applications under steady conditions.

Overview and high resolution scan

One of the most common use cases of guided acquisition is the automatic calculation of Regions Of Interest (ROIs) based on low resolution images. This avoids the time consuming process of selecting manually tens or hundreds of scanning locations.

This pipeline has been established for Nikon and Carl Zeiss systems, under the JOBS and OAD environments, respectively. See our toolbox section for more details.





Smart Microscopy

Smart Microscopy is the application of robotics in the strict sense: real-time data acquired drives the acquisition process. Image acquisition, analysis and instrument control coordinate together along the experiment.

ISTA tracker

Smart microscopy for analyzing sample movement during the acquisition and keeping it within the field of view. Originally made for the Zeiss Vertical Microscopes, it has been made available at all Zen Blue confocal microscopes.

The systems below are fully automatic, providing interfaces for smart microscopy. A few examples: the General Analysis module of NIS-Elements or a connected Fiji plugin can provide signals for starting an injection phase or for zooming into a dissected Region Of Interest (ROI). These implementations are flexible and can be requested as projects.

Laser Cutter 2

An in-house developed UV micro-ablation system developed for inter-operating with a CSU-W1-SoRa Spinning Disk confocal microscope.





Micro-manipulation setup

Made for the application and measurement of cell-cell interaction forces. Currently available at the Leica Stellaris, a version with smart capabilities is under development for the Nikon W1-02.



Under development

  • ISTA tracker for NIS-Elements: It is still in preliminary planning stage. Once available, it will boost the fast-imaging capabilities of the IOF Widefield and Spinning Disk microscopes.

    Light intensity adaptation: Based on Image intensity measurement the necessary excitation light power will be calculated and adjusted in real time

The automation toolbox
Environments for guided acquisition and smart microscopy

These are well established packages available in our systems. All of them are aimed to be used in combination with other packages (e.g. analysis) within the same environment. Nevertheless, inter-process communication allows data exchange with external software as well.

  • Nikon JOBS: module to assist in setting up complex imaging pipelines, such as an automatic overview, analysis and high-resolution scanning. More about it at the Nikon official site.
  • Nikon MACRO: a C interpreter for high-end applications (e.g. Laser cutter, microfluidics, micro-manipulation).
  • Open Application Development: data driven acquisition (offline) but still usable in some smart applications. Zeiss OAD on Github.
  • Guided Acquisition: module for smart microscopy. The acquisition parameters can be modified during the acquisition, from real-time feedback data. More about it on Github.
Interfaces for sequential experiments (multiple blocks)

When experiments consist not only in a uniform 5D (XYZ + time + color) series but a sequence of them.

  • Experiment Designer (Carl Zeiss): It is a module available for Zen Blue and Zen Black, available for all our Zeiss confocal microscopes. Multiple blocks allow running separate imaging sessions in a sequence or loop. A use case consists in setting up independent Z-stacks in multi-point experiments.
  • Live data mode (Leica): Offers similar capabilities as the Experiment Designer. Nevertheless, in these systems different stacks can be defined within the Leica Navigator. Instead, a use case here could be to set different illumination powers per location in a multi-point experiment.
  • Sequence Acquisition (Nikon): The equivalent functionality in the Nikon NIS-Elements software. It offers a interface similar to the rest of the experiment setup boxes.
  • Experiment Manager (Olympus): CellSens has a graphical built-in interface for setting up experiments, providing nearly arbitrary flexibility for setting up sequential tasks.


Some interactive modules related to setting up the imaging of large areas or volumes or high throughput experiments.

  • ZEN Connect a module from Zeiss that allows pyramidal integration of images acquired at multiple microscopes, not necessarily from Zeiss only (further information here). IOF works towards integrating Zen Connect with our Data Infrastructure. Zen connect is already available at Zeiss Axioscan Z1 / LSM800-01inv / -02up / -03inv / -04inv / Examiner D1.
  • Leica Navigator: It brings a satellite view of the entire stage area, independent tiles and Z-stacks, focus points, etc.
  • Zeiss sample navigator: Provides overview capabilities for exploring large samples offline. Multipoint or tile series can be defined later without re-visiting the sample later.
  • Advanced tile interfaces (Nikon): NIS-Elements offers a JOBS-derived interface for tiles, where arbitrary contours can be defined and a macro for navigating and incorporating saved positions into experiments.

Image Analysis implementations
  • Light sheet data reconstruction & deconvolution pipeline

Project request: Automatic / feedback microscopy