Acquisition of Spectra

Summary

This package supports direct acquisition of spectra only from Ocean Optics spectrometers. It also allows computations on RAW spectral counts to be done with the spectrometer off-line. What distinguishes it from other software is the implementation of special algorithms for measurement protocols that enhance the signal to noise ratio by one order of magnitude, allowing the measurement of ultraviolet-B radiation in sunlight using array spectrometers.

• Allows spectral data acquisition from within an R session in near real-time.
• Allows conversion of raw-counts data acquired using SpectraSuite or OceanView on PCs and possibly using the open-source SeaBreeze based software running on the Raspberry Pi single-board microcomputer. It also supports computations based on files acquired autonomously with the Jaz spectrometer.
• Irradiance, fluence, reflectance, transmittance, absorptance and absorbance can be derived from raw spectra.
• Measurements of both continuous and pulsed light sources is possible.
• Different measurement protocols, data averaging, integration time bracketing, automatic adjustment of integration time can be used and corrections for stray light and slit function can be applied.
• Acquisition of time series of spectra is also possible in two different ways: at timed intervals, or as fast as possible using buffered transfer from the spectrometer. The minimum delay depends on the spectrometer but can be as short as a few microseconds.
• Data and plots are saved to files asynchronously to avoid blocking the user interface or data acquisition.

Although not directly supported by this package, the code it includes can serve as a basis for equivalent protocols and corrections of data acquired with software and/or spectrometers from other suppliers.

Introduction

This document describes only high level functions for interactive data acquisition. We first show simple examples of their use, assuming that R, the needed R packages, one of Temurin 8 OpenJDK or Oracle’s Java 8 JDK or Amazon’s Corretto Java 8 OpenJDK plus Ocean Optics’ free OmniDriver runtime are all installed and working (see this package’s README and the README of package ‘rOmniDriver’ for details).

In this vignette we provide a brief tutorial on how to carry out measurements.

The User Guide menu of the on-line documentation includes in addition to the vignettes included in the package, two on-line only chapters: a tutorial on how to measure spectral irradiance with ‘oocquire’ and a description of the algorithms used.

The package exports both high level user-friendly functions and the lower level functions used to build them. What we describe here is the use of the higher level functions.

Building this vignette

# change this to TRUE to run acquisition examples
# these examples require user interaction to complete!!
sr.online <- FALSE

NOTE: This vignette can be built even if a spectrometer is not connected as the code chunks with code that “talks” with the instrument are by default not evaluated. Set above sr.online <- TRUE and connect a spectrometer for which a calibration is available if you intend to rebuild this vignette with live output from a measurement session.

Measurement of light spectra

An emission spectrum of light emitted by a source or the spectrum of light received on a surface describes the energy or photon distribution as a function of wavelength. A calibration describes the relationship between the signal response of a sensor and a physical quantity. For spectral measurements this involves a description of the response of the spectrometer to light of each measurable wavelength and a calibration of detector pixels or monochromator positions to true wavelengths

Thus to acquire spectral irradiance, a suitable instrument calibration and correction method description should be also available and in practice implemented in software. Advanced algorithms implemented in ‘ooacquire’ require a special characterization of the spectrometer. If such characterization is not available, a simpler approach, as normally used with most other software, is also available in ‘ooacquire’. If no calibration is available in the format used by this package, an attempt is made to retrieve calibration data from the connected spectrometer. In most cases the wavelength calibration can be retrieved, but the multipliers needed to compute spectral irradiance are stored in memory only if expressly saved for this type of measurements as a result of a calibration. An irradiance calibration is not needed to measure quantities like spectral reflectance and transmittance that are relative to a reference. Protocols, characterization and calibration information can all be stored once in the special format used by ‘ooacquire’ if available. Subsequently, the acquisition of spectra does not differ much and any differences are reflected in the user interface as long as a calibration is available. What differs is the quality of the spectral data that is acquired.

In the code chunks below we use calibrations included as part of the package. As the serial number stored with the calibration is validated against that retrieved from the instrument, the examples below will work only with our own instruments, rather than with the ones you may intend to use. In most cases you will need to either import calibration data as provided by Ocean Optics/Ocean Insight or for more complex correction algorithms, manually create R objects with calibration and characterisation data obtained from other sources. Such setup needs to be done only once, and updated when the spectrometer is recalibrated.

When sunlight or radiation emitted by lamps driven by direct or alternating current are measured the integration time can be used as reference to compute fluxes such as energy irradiance or photon irradiance. In the case of many lamps driven by alternating current accurate measurements require us to consider the line frequency (50 Hz in Europe, 60 Hz in USA) because light output will usually vary cyclically at twice these frequencies (100 Hz or 120 Hz) making it necessary to use integration times that are multiples of the duration of one half cycle of line frequency (10 ms and 8.3333 ms, respectively) or alternatively use long-enough integration times for several cycles to be included in the measurement interval. The same considerations apply to transmittance, reflectance, and absorptance measurements using continuous radiation sources.

When measuring radiation from pulsed light sources such as xenon flashes we express the measurements as spectral power or photons per event (pulse or “flash”) and the integration time must simply be long enough to encompass a known number of whole pulses. We use the number of pulses to compute fluence rate per pulse or event. The same considerations apply to transmittance, reflectance and absorptance measurements using pulsed radiation sources.

During a measurement event either a single spectrum or a time series of spectra are acquired and stored in one or two data objects, together with metadata describing the spectrometer and settings used. By default, both raw-counts data and the spectra expressed in physical units are stored in two R objects. Optionally, it is possible to store only raw-counts data in a single R object and computation of spectra expressed in physical units done “off-line”.

The functions described below by default acquire for each measurement event one or more raw-counts spectra for each spectrum expressed in physical units. The interactive spectral-data-acquisition functions by default compute the corresponding spectrum or spectra expressed in a physical quantity, and save the data, both raw and computed, asynchronously, creating one .Rda R-data file per measurement event.

After each measurement event, the operator is by default, presented with an annotated plot. The annotations that the user can adjust before continuing for example choosing energy vs. photon units. The plot corresponding to the measurement is saved to a .pdf file, also before a new measurement is started. This approach to saving the data ensures that in case of a computer failure or program crash, at most data for a single measurement event will be lost.

Spectra grouped into collections can be also saved and plotted at any time during a measurement session. When saving a collection all spectra expressed in physical units are collected into a .mspct object, and the corresponding raw-counts-spectra are collected into an R list object. Summaries of the measurements can be computed and saved as a data.frame. All these R objects are saved into a single .Rda file. The summaries are in addition saved into a .csv file.

A measurement event can consist of a single “light” spectrum or of a time series of “light” spectra. When reference spectra are acquired, the “light” spectra in a measurement event share the “dark” and, if used, also the “filter” spectra. The “dark” and “filter” spectra are used as references for the conversion of “light” raw-counts-spectra into physical quantities such as spectral irradiance.

Each raw-counts spectrum can consist in one or more scans or readings of the detector array, taken using identical or different settings in the spectrometer, depending on the protocol used. Once the desired protocol is selected, the measurements can be acquired with little intervention from the user, and the settings are retained across successive measurement events, unless expressly modified.

Whether a time series of spectra or a single spectra is acquired depends on the arguments passed to parameters of the function.

It is possible to select a different quantity to be saved to files by passing an argument to qty.out:

• "raw" skips computations of spectra in physical units and saves to file only raw-counts data with metadata. In this case conversion into physical units must be done at a later time. The descriptor of the instrument if missing is retrieved from the instrument.
• "cps" (default if no spectral irradiance calibration is available) converts raw-counts into linearised counts-per-second and saves to file both raw-counts data with metadata and counts-per second spectral data. The descriptor of the instrument if missing is retrieved from the instrument. Plots can be displayed on screen and are by default saved as .pdf files.
• "irrad" (default if a spectral irradiance calibration is available) converts raw-counts into corrected spectral irradiance (a flux rate expressed per unit time) and saves to a file both raw-counts data with metadata and spectral irradiance data. If the descriptor and spectral irradiance calibration are not available in R but available in the memory of the instrument they are retrieved. Plots can be displayed on screen and are by default saved as .pdf files.
• "fluence" same as “irrad”`, but the quantity computed is spectral fluence (a flux expressed per illumination event, usually one flash per measurement).

The saved data objects include metadata as attributes:

The attributes in the last two rows of the table, together with always saving raw-counts spectra to files, allows recalculation of physical quantities and provides a built-in trace of the origin of the data.

Spectral irradiance

Spectral energy irradiance (Wm⁻²nm⁻¹) and spectral photon irradiance (mols⁻¹m^m − 2nm⁻¹$) are fluxes expressed as energy or quanta, respectively per unit area of a receiving surface. They can be inter-converted and there is no difference in how they are measured or a need for different calibration.

After loading the package as shown in the section below and connecting a spectrometer by USB, calling function acq_irrad_interactive() opens a connection to the spectrometer and starts an interactive spectral-data acquisition session. The function takes several arguments, which have as defaults the most frequently used values. However, frequently, users will want to pass arguments in the call to set the interface mode.

Of the formal parameters of function acq_irrad_interactive(), interface.mode selects which settings can be controlled interactively during a session, i.e., which settings can be modified by the user at run time (in other words, it alters which options are visible or not in menus as well as some of the default settings). The simplified interfaces make measurements simpler and faster in specific situations, while the defaults for the session can always be changed by passing arguments when calling function acq_irrad_interactive(). The default interface.mode = "auto" retains the behaviour of earlier versions of package ‘ooacquire’, but interface.mode = "simple" gives access to all the parameters that need to be set at runtime for some common measurements. Modes "manual" and "full" make it possible to manually set the integration time, retaining the ability to automatically adjust it on demand. Mode “fluence” makes the interface suitable for the measurement of pulsed light sources and “series” makes it possible to acquire time series of spectra as a single measurement event. Variations of the modes ending in .attr enable the user interface for entering for each measurement event the texts to be stored in attributes comment and what.measured.

The correction.method and descriptor parameters are used to pass the calibration and protocol information specific to a given spectrometer. For spectrometers “known” to package ‘ooacquire’ these also have suitable defaults which are selected after retrieving from the spectrometer its serial number. These parameters are not totally redundant as for a given spectrometer multiple correction algorithms, and/or calibrations for different entrance optics, may be available and selected by overriding the defaults. In all cases these arguments are checked to match the serial number of the attached spectrometer, and rejected in cases of mismatch.

If no calibration information is available based on the serial number or the user has not supplied valid calibration information as arguments to the call, an attempt is made to retrieve an iradiance calibration from the spectrometer EEPROM. Because of the way in which irradiance calibrations are stored in Ocean Optics spectrometers. Calibration data retrieved from the EEPROM can only be used if the user provides information about the diffuser used as entrance optics and the retrieved calibration is valid, thus among other things, done using the same optical fibre as being used for the measurements. If a calibration cannot be retrieved from the EEPROM or information about the diffuser is not supplied, this last attempt also fails. In this case the function falls back into a mode that returns counts-per-second instead of spectral irradiance, with a warning. A wavelength calibration remains as a requirement but it can be usually retrieved from the spectrometer EEPROM.

Saving of plots as PDF files is controlled through a logical argument passed to parameter save.pdfs. In the case of saving of collections of spectra and summaries, passing FALSE to save.summaries and save.collections disables the user interface for their creation.

Arguments passed to other formal parameters only alter the starting default values presented interactively to the user.

library(ooacquire)

## Loading required package: photobiology

## Documentation at https://docs.r4photobiology.info/

## Loading required package: ggspectra

## Loading required package: ggplot2

## Loading required package: lubridate

## 
## Attaching package: 'lubridate'

## The following objects are masked from 'package:base':
## 
##     date, intersect, setdiff, union

## 'rOmniDriver': Communication with spectrometers is disabled because environment variable OOI_HOME is not set to a path.
## To acquire new data, please, make sure that the OmniDriver driver is installed and that the environment variable OOI_HOME is set to its path.

## Package 'ooacquire' is Off-line: package 'rOmniDriver'needs to be installed and enabled to access spectrometers.

acq_irrad_interactive(interface.mode = "simple")

acq_irrad_interactive(correction.method = ooacquire::MAYP11278_ylianttila.mthd,
                      descriptors = ooacquire::MAYP11278_descriptors)

acq_irrad_interactive()

acq_irrad_interactive(HDR.mult = 1,
                      tot.time.range = c(0, Inf))

acq_irrad_interactive(HDR.mult = c(short = 1, long = 10), # the default
                      tot.time.range = c(5, 15), # the default
                      correction.method = ooacquire::MAYP11278_ylianttila.mthd,
                      descriptors = ooacquire::MAYP11278_descriptors)

acq_irrad_interactive(HDR.mult = c(1, 10),
                      tot.time.range = 10)

acq_irrad_interactive(HDR.mult = c(1, 10), 
                      tot.time.range = c(10, 20))

acq_irrad_interactive(target.margin = 0.25)

acq_irrad_interactive(interface.mode = "series",
                      seq.settings = list(start.boundary = "minute",
                                          initial.delay = 0, # seconds
                                          step.delay = 60, # seconds
                                          num.steps = 60))

acq_irrad_interactive(interface.mode = "series",
                      tot.time.range = c(5, 10),
                      target.margin = 0.1,
                      qty.out = "irrad",
                      HDR.mult = c(0.3, 1, 3, 10),
                      seq.settings = list(start.boundary = "minute",
                                          initial.delay = 0,
                                          step.delay = 60,
                                          num.steps = 60))

acq_irrad_interactive(qty.out = "fluence")

acq_fraction_interactive(ref.value = 0.97)

acq_fraction_interactive(qty.out = "Rfr", type = "specular")

Rolling your own

The functions for interactive acquisition of spectral data described above, although useful in many cases, are meant also as examples. In cases of routine measurements one could simplify menus by removing entries that are not needed, or even converting the functions to non-interactive, and possibly enhancing support for scheduled sequences of spectral data acquisitions. This should be fairly simple, not requiring advanced knowledge of the R language as we export lower level functions that can be used as building blocks for new variations. It should be even possible to use these functions to develop a user interface based of package ‘shinny’.

Using scripts

It is also possibly to combine lower level functions using R scripts instead of through writing higher level R functions that call them. Several example scripts are included with the package, including scripts based on low level functions and scripts that demonstrate different ways of using the interactive functions, including in combination with trigger functions.

I recommend that you open these example scripts, and save them to your own working directory before sourcing them or editing them.

You will need first to locate where in your computer’s file system the ‘ooacquire’ package is installed, and where the file you are interested in is, and to copy it to your own workspace. Here we copy the script called "irrad-acq-interac.R".

filepath <- system.file("example-scripts", "irrad-acq-interac.R", package="ooacquire")
file.copy(from = filepath, to = ".")

Here we copy all the example scripts.

folderpath <- system.file("example-scripts", package="ooacquire")
list.files(path = folderpath, pattern = ".*[.]R")
file.copy(from = list.files(path = folderpath, pattern = ".*[.]R", full.names = TRUE), to = ".")