Delivering data reduction pipelines to science users

Wolfram Freudling; Martino Romaniello

doi:10.1117/12.2232734

11 July 2016 Delivering data reduction pipelines to science users

Wolfram Freudling, Martino Romaniello

Proceedings Volume 9910, Observatory Operations: Strategies, Processes, and Systems VI; 99101U (2016) https://doi.org/10.1117/12.2232734
Event: SPIE Astronomical Telescopes + Instrumentation, 2016, Edinburgh, United Kingdom

Abstract

The European Southern Observatory has a long history of providing specialized data processing algorithms, called recipes, for most of its instruments. These recipes are used for both operational purposes at the observatory sites, and for data reduction by the scientists at their home institutions. The two applications require substantially different environments for running and controlling the recipes. In this papers, we describe the ESOReflex environment that is used for running recipes on the users’ desktops. ESOReflex is a workflow driven data reduction environment. It allows intuitive representation, execution and modification of the data reduction workflow, and has facilities for inspection of and interaction with the data. It includes fully automatic data organization and visualization, interaction with recipes, and the exploration of the provenance tree of intermediate and final data products. ESOReflex uses a number of innovative concepts that have been described in Ref. 1. In October 2015, the complete system was released to the public. ESOReflex allows highly efficient data reduction, using its internal bookkeeping database to recognize and skip previously completed steps during repeated processing of the same or similar data sets. It has been widely adopted by the science community for the reduction of VLT data.

Citation Download Citation

Wolfram Freudling and Martino Romaniello "Delivering data reduction pipelines to science users", Proc. SPIE 9910, Observatory Operations: Strategies, Processes, and Systems VI, 99101U (11 July 2016); https://doi.org/10.1117/12.2232734

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available