--- title: "Acquisition of Spectra" subtitle: "`ooacquire` `r packageVersion('ooacquire')`" author: "Pedro J. Aphalo" date: "`r Sys.Date()`" output: rmarkdown::html_vignette: toc: yes vignette: > %\VignetteIndexEntry{Acquisition of Spectra} %\VignetteEncoding{UTF-8} %\VignetteEngine{knitr::rmarkdown} editor_options: markdown: wrap: 72 --- ## Summary This package supports direct acquisition of spectra only from Ocean Optics spectrometers. It also allows computations on RAW spectral counts to be done with the spectrometer off-line. What distinguishes it from other software is the implementation of special algorithms for measurement protocols that enhance the signal to noise ratio by one order of magnitude, allowing the measurement of ultraviolet-B radiation in sunlight using array spectrometers. - Allows spectral data acquisition from within an R session in near real-time. - Allows conversion of raw-counts data acquired using SpectraSuite or OceanView on PCs and possibly using the open-source SeaBreeze based software running on the Raspberry Pi single-board microcomputer. It also supports computations based on files acquired autonomously with the Jaz spectrometer. - Irradiance, fluence, reflectance, transmittance, absorptance and absorbance can be derived from raw spectra. - Measurements of both continuous and pulsed light sources is possible. - Different measurement protocols, data averaging, integration time bracketing, automatic adjustment of integration time can be used and corrections for stray light and slit function can be applied. - Acquisition of time series of spectra is also possible in two different ways: at timed intervals, or as fast as possible using buffered transfer from the spectrometer. The minimum delay depends on the spectrometer but can be as short as a few microseconds. - Data and plots are saved to files asynchronously to avoid blocking the user interface or data acquisition. Although not directly supported by this package, the code it includes can serve as a basis for equivalent protocols and corrections of data acquired with software and/or spectrometers from other suppliers. ## Introduction This document describes only high level functions for interactive data acquisition. We first show simple examples of their use, assuming that R, the needed R packages, one of Temurin 8 **OpenJDK** or Oracle's Java 8 **JDK** or Amazon's Corretto Java 8 **OpenJDK** plus Ocean Optics' free OmniDriver runtime are all installed and working (see this package's README and the README of package 'rOmniDriver' for details). In this vignette we provide a brief tutorial on how to carry out measurements. The User Guide menu of the [on-line documentation](https://docs.r4photobiology.info/ooacquire/) includes in addition to the vignettes included in the package, two on-line only chapters: [a tutorial on how to measure spectral irradiance](https://www.r4photobiology.info/pages/acq-irrad-tutorial.html) with 'oocquire' and a [description of the algorithms used](https://www.r4photobiology.info/pages/ooacquire-algorithms.html). The package exports both high level user-friendly functions and the lower level functions used to build them. What we describe here is the use of the higher level functions. ## Building this vignette ```{r "setup", include=FALSE} require("knitr") opts_knit$set(cache = FALSE, root.dir = system.file("extdata", package = "ooacquire")) ``` ```{r} # change this to TRUE to run acquisition examples # these examples require user interaction to complete!! sr.online <- FALSE ``` **NOTE:** This vignette can be built even if a spectrometer is not connected as the code chunks with code that "talks" with the instrument are by default not evaluated. Set above `sr.online <- TRUE` and connect a spectrometer for which a calibration is available if you intend to rebuild this vignette with live output from a measurement session. ## Measurement of light spectra An emission spectrum of light emitted by a source or the spectrum of light received on a surface describes the energy or photon distribution as a function of wavelength. A calibration describes the relationship between the signal response of a sensor and a physical quantity. For spectral measurements this involves a description of the response of the spectrometer to light of each measurable wavelength and a calibration of detector pixels or monochromator positions to true wavelengths Thus to acquire spectral irradiance, a suitable instrument calibration and correction method description should be also available and in practice implemented in software. Advanced algorithms implemented in 'ooacquire' require a special characterization of the spectrometer. If such characterization is not available, a simpler approach, as normally used with most other software, is also available in 'ooacquire'. If no calibration is available in the format used by this package, an attempt is made to retrieve calibration data from the connected spectrometer. In most cases the wavelength calibration can be retrieved, but the multipliers needed to compute spectral irradiance are stored in memory only if expressly saved for this type of measurements as a result of a calibration. An irradiance calibration is not needed to measure quantities like spectral reflectance and transmittance that are relative to a reference. Protocols, characterization and calibration information can all be stored once in the special format used by 'ooacquire' if available. Subsequently, the acquisition of spectra does not differ much and any differences are reflected in the user interface as long as a calibration is available. What differs is the quality of the spectral data that is acquired. In the code chunks below we use calibrations included as part of the package. As the serial number stored with the calibration is validated against that retrieved from the instrument, the examples below will work only with our own instruments, rather than with the ones you may intend to use. In most cases you will need to either import calibration data as provided by Ocean Optics/Ocean Insight or for more complex correction algorithms, manually create R objects with calibration and characterisation data obtained from other sources. Such setup needs to be done only once, and updated when the spectrometer is recalibrated. When sunlight or radiation emitted by lamps driven by direct or alternating current are measured the integration time can be used as reference to compute fluxes such as energy irradiance or photon irradiance. In the case of many lamps driven by alternating current accurate measurements require us to consider the line frequency (50 Hz in Europe, 60 Hz in USA) because light output will usually vary cyclically at twice these frequencies (100 Hz or 120 Hz) making it necessary to use integration times that are multiples of the duration of one half cycle of line frequency (10 ms and 8.3333 ms, respectively) or alternatively use long-enough integration times for several cycles to be included in the measurement interval. The same considerations apply to transmittance, reflectance, and absorptance measurements using continuous radiation sources. When measuring radiation from pulsed light sources such as xenon flashes we express the measurements as spectral power or photons per event (pulse or "flash") and the integration time must simply be long enough to encompass a known number of *whole* pulses. We use the number of pulses to compute fluence rate per pulse or event. The same considerations apply to transmittance, reflectance and absorptance measurements using pulsed radiation sources. During a measurement event either a single spectrum or a time series of spectra are acquired and stored in one or two data objects, together with metadata describing the spectrometer and settings used. By default, both raw-counts data and the spectra expressed in physical units are stored in two R objects. Optionally, it is possible to store only raw-counts data in a single R object and computation of spectra expressed in physical units done "off-line". The functions described below by default acquire for each measurement event one or more raw-counts spectra for each spectrum expressed in physical units. The interactive spectral-data-acquisition functions by default compute the corresponding spectrum or spectra expressed in a physical quantity, and save the data, both raw and computed, asynchronously, creating one `.Rda` R-data file per measurement event. After each measurement event, the operator is by default, presented with an annotated plot. The annotations that the user can adjust before continuing for example choosing energy vs. photon units. The plot corresponding to the measurement is saved to a `.pdf` file, also before a new measurement is started. This approach to saving the data ensures that in case of a computer failure or program crash, at most data for a single measurement event will be lost. Spectra grouped into collections can be also saved and plotted at any time during a measurement session. When saving a collection all spectra expressed in physical units are collected into a `.mspct` object, and the corresponding raw-counts-spectra are collected into an R list object. Summaries of the measurements can be computed and saved as a `data.frame`. All these R objects are saved into a single `.Rda` file. The summaries are in addition saved into a `.csv` file. A measurement event can consist of a single *"light"* spectrum or of a time series of *"light"* spectra. When reference spectra are acquired, the *"light"* spectra in a measurement event share the *"dark"* and, if used, also the *"filter"* spectra. The *"dark"* and *"filter"* spectra are used as references for the conversion of *"light"* raw-counts-spectra into physical quantities such as spectral irradiance. Each raw-counts spectrum can consist in one or more scans or readings of the detector array, taken using identical or different settings in the spectrometer, depending on the protocol used. Once the desired protocol is selected, the measurements can be acquired with little intervention from the user, and the settings are retained across successive measurement events, unless expressly modified. Whether a time series of spectra or a single spectra is acquired depends on the arguments passed to parameters of the function. | Parameter | Buffered | Fast series | Timed series| Timed single| Single | |:------------|:-----------:|:-----------:|:-----------:|:-----------:|:--------------:| | interface.mode | "series" | "series" | "series" | "series" | "auto"/"simple"| | step.delay | 0 s | 0 s | > 0 s | NA | NA | | start.delay | >= 0 s | >= 0 s | >= 0 s | > 0 s | = 0 s | | HDR | no | yes | yes/no | yes/no | yes/no | : Timing of the acquisition of spectra. Buffered acquisition is as fast as the spectrometer allows, with the frequency controlled by the total integration time plus an overhead < 1 ms. In the case of Buffered acquisition there is minimal dead time between acquisitions. Fast acquisition has more dead time as settings in the spectrometer need to we modified two or more times per measuring event. While Buffered and Fast series are acquired as fast as possible, in the case of a timed series, spectra are retrieved at specific times, with dead/waiting time in-between. It is possible to select a different quantity to be saved to files by passing an argument to `qty.out`: - `"raw"` skips computations of spectra in physical units and saves to file only raw-counts data with metadata. In this case conversion into physical units must be done at a later time. The descriptor of the instrument if missing is retrieved from the instrument. - `"cps"` (default if no spectral irradiance calibration is available) converts raw-counts into linearised counts-per-second and saves to file both raw-counts data with metadata and counts-per second spectral data. The descriptor of the instrument if missing is retrieved from the instrument. Plots can be displayed on screen and are by default saved as `.pdf` files. - `"irrad"` (default if a spectral irradiance calibration is available) converts raw-counts into corrected spectral irradiance (a flux rate expressed per unit time) and saves to a file both raw-counts data with metadata and spectral irradiance data. If the descriptor and spectral irradiance calibration are not available in R but available in the memory of the instrument they are retrieved. Plots can be displayed on screen and are by default saved as `.pdf` files. - `"fluence"` same as "irrad"\`, but the quantity computed is spectral fluence (a flux expressed per illumination event, usually one flash per measurement). The saved data objects include metadata as attributes: | purpose | included | accessor | |:--------------------------------------------------------------------------|:---------:|:------------------:| | user comment | optional | `comment()` | | what was measured | optional | `what_measured()` | | where was measured | optional | `where_measured()` | | when was measured in UTC time | automatic | `when_measured()` | | simply how was measured | automatic | `how_measured()` | | instrument descriptor including calibration, serial number, configuration | automatic | | | instrument settings used | automatic | | : Attributes of spectral objects. The attributes in the last two rows of the table, together with always saving raw-counts spectra to files, allows recalculation of physical quantities and provides a built-in trace of the origin of the data. ## Spectral irradiance Spectral energy irradiance ($W m^{-2} nm^{-1}$) and spectral photon irradiance ($mol s^{-1} m^{m-2} nm^{-1}$\$) are fluxes expressed as energy or quanta, respectively per unit area of a receiving surface. They can be inter-converted and there is no difference in how they are measured or a need for different calibration. After loading the package as shown in the section below and connecting a spectrometer by USB, calling function `acq_irrad_interactive()` opens a connection to the spectrometer and starts an interactive spectral-data acquisition session. The function takes several arguments, which have as defaults the most frequently used values. However, frequently, users will want to pass arguments in the call to set the interface mode. Of the formal parameters of function `acq_irrad_interactive()`, `interface.mode` selects which settings can be controlled interactively during a session, i.e., which settings can be modified by the user at run time (in other words, it alters which options are visible or not in menus as well as some of the default settings). The simplified interfaces make measurements simpler and faster in specific situations, while the defaults for the session can always be changed by passing arguments when calling function `acq_irrad_interactive()`. The default `interface.mode = "auto"` retains the behaviour of earlier versions of package 'ooacquire', but `interface.mode = "simple"` gives access to all the parameters that need to be set at runtime for some common measurements. Modes `"manual"` and `"full"` make it possible to manually set the integration time, retaining the ability to automatically adjust it on demand. Mode "fluence" makes the interface suitable for the measurement of pulsed light sources and "series" makes it possible to acquire time series of spectra as a single measurement event. Variations of the modes ending in `.attr` enable the user interface for entering for each measurement event the texts to be stored in attributes `comment` and `what.measured`. | Parameter | Single | Single | Single | Single | Single | |:------------|:-----------:|:-----------:|:-----------:|:-----------:|:-----------:| | interface.mode | "auto" | "simple" | "full" | "manual" | "fluence" | | step.delay | NA | NA | NA | NA | NA | | start.delay | NA | NA | NA | NA | NA | | HDR | yes | yes | yes | yes | (yes) | | target.margin | yes | fixed | yes | yes | no | | default integ. | _tune_ | _tune_ | _tune_ | _user-set_ | _user-set_ | : Interface modes for acquisition of non-timed single spectra per measurement event. | Parameter | Buffered | Fast series | Timed series| Timed single| |:------------|:-----------:|:-----------:|:-----------:|:-----------:| | interface.mode | "series" | "series" | "series" | "series" | | step.delay | 0 s | 0 s | > 0 s | NA | | start.delay | >= 0 s | >= 0 s | >= 0 s | > 0 s | | HDR | no | yes | yes/no | yes/no | | target.margin | yes | yes | yes | yes | | default integ. | _tune_ | _tune_ | _tune_ | _tune_ | : Interface modes for acquisition of time series of spectra and timed single spectra per measurement event. The `correction.method` and `descriptor` parameters are used to pass the calibration and protocol information specific to a given spectrometer. For spectrometers "known" to package 'ooacquire' these also have suitable defaults which are selected after retrieving from the spectrometer its serial number. These parameters are not totally redundant as for a given spectrometer multiple correction algorithms, and/or calibrations for different entrance optics, may be available and selected by overriding the defaults. In all cases these arguments are checked to match the serial number of the attached spectrometer, and rejected in cases of mismatch. If no calibration information is available based on the serial number or the user has not supplied valid calibration information as arguments to the call, an attempt is made to retrieve an iradiance calibration from the spectrometer EEPROM. Because of the way in which irradiance calibrations are stored in Ocean Optics spectrometers. Calibration data retrieved from the EEPROM can only be used if the user provides information about the diffuser used as entrance optics and the retrieved calibration is valid, thus among other things, done using the same optical fibre as being used for the measurements. If a calibration cannot be retrieved from the EEPROM or information about the diffuser is not supplied, this last attempt also fails. In this case the function falls back into a mode that returns counts-per-second instead of spectral irradiance, with a warning. A wavelength calibration remains as a requirement but it can be usually retrieved from the spectrometer EEPROM. Saving of plots as PDF files is controlled through a `logical` argument passed to parameter `save.pdfs`. In the case of saving of collections of spectra and summaries, passing `FALSE` to `save.summaries` and `save.collections` disables the user interface for their creation. Arguments passed to other formal parameters only alter the starting default values presented interactively to the user. ### Acquiring a single irradiance spectrum #### A simple example Assuming a calibration is available the minimal steps to measure spectral irradiance are as follows. 0. (See the README files of R packages ['ooacquire'](https://docs.r4photobiology.info/ooacquire/) and ['rOmnidriver'](https://docs.r4photobiology.info/rOmniDriver/) for software installation instructions). 1. Start R and load the package. ```{r} library(ooacquire) ``` 2. Connect an Ocean Optics spectrometer to a USB port. To avoid problems use a short and high quality USB cable, if this is not possible use an active USB extension cable. In all cases use a short and known good USB cable when using 'ooacquire' or a given spectrometer for the first time and later keep this short and good USB cable always at hand to debug any possible USB connection problems if they happen. 3. Start an interactive measuring session, in this example accepting all defaults except for the choice of a simplified user interface. ```{r, eval=sr.online} acq_irrad_interactive(interface.mode = "simple") ``` If everything works as expected, the version of OmniDriver that has been found will be reported. If one or more spectrometers are found, they will be listed with their serial numbers for the user to choose one of them. Once a connection and software are initialized data acquisition is possible. 4. Follow the prompts to acquire spectra. Many settings can be adjusted during a data acquisition session. Here, using `interface.mode = "simple"` fewer of these settings can be modofied interactively than when using, for example, `interface.mode = "full"`, which has additional menu options in the user interface. If one knows in advance what settings will need, these can be passed as arguments when starting the session by calling `acq_irrad_interactive()`. The values passed as arguments become the new defaults or fixed depending on the `interface.mode`. In most cases when values supplied by the user fall outside those accepted by the connected instrument, these values are adjusted to the nearest valid value. In the remaining cases a new value is requested with a message. We can pass the objects containing the correction method definition and the instrument descriptor that includes the calibration data as arguments. ```{r, eval=sr.online} acq_irrad_interactive(correction.method = ooacquire::MAYP11278_ylianttila.mthd, descriptors = ooacquire::MAYP11278_descriptors) ``` In this case, this corresponds to a spectrometer whose data are available in the package and recognized automatically. Thus is equivalent to accepting all defaults as below. The defaults are dynamic, based on the serial number of the spectrometer unit that is attached. ```{r, eval=sr.online} acq_irrad_interactive() ``` However, for this spectrometer additional correction methods are available. Two calibrations with different entrance optics are available, and can be selected by means of an argument passed to parameter `entrance.optics`. In the case of calibrations done by Ocean Insight or with their software, and retrieved from the spectrometer EEPROM we need to pass an argument to parameter `area` indicating the effective area of the diffuser, or the name of one of the entrance optics known to 'ooacquire'. Although we discuss settings as arguments to formal parameters of function `acq_irrad_interactive()`, the settings discussed below can be also set interactively by the user, at least in one of the supported interface modes. The functions described here, except when in interface mode `"manual"`, will set a suitable integration time automatically. #### Protocols for enhanced performance Dynamic range is the difference between the smallest and largest values that can be detected in a measurement. Noise is the component of the measured value that is not a reflection of what we want to measure. An array detector not only responds to light, but it is also to a small extent excited also by thermal radiation, electrical fields and even cosmic particles. These events are infrequent and so during a single integration are likely to affect only individual pixels and different pixels in different integrations. A further problem is stray light, photons that imping on the "wrong" pixel because of internal reflections in the optical bench of the spectrometer. Different protocols and correction algorithms can mitigate the effect of these problems on the quality of the spectral data. If we use a single value for integration time (the equivalent of exposure time in a photographic camera) the realized dynamic range is the intrinsic dynamic range of the array detector and electronics of the spectrometer. In this case we pass a single numeric value as argument to `HDR.mult` to indicate acquisition using a single integration time. Except in very unusual cases, we use 1, because `HDR.mult` is a multiplicative factor applied to the optimal integration time. ```{r, eval=sr.online} acq_irrad_interactive(HDR.mult = 1, tot.time.range = c(0, Inf)) ``` Averaging of multiple "scans" mitigates "random" noise because the longer the total measurement time the most likely it is that all pixels are similarly affected by the noise. The noise is still present, but it affects more evenly the different pixels. The approach used in `acq_irrad_interactive()` is for the user to supply an upper and lower limit to the sum of integration times from multiple scans and the software determines the number of scans that are needed at the time the optimal integration time is tuned. Keeping the `tot.time.range` approximately constant can help keep the noise levels consistent across multiple measurement events. The argument passed to `tot.time.range`, a vector of one or two numbers, indicates the allowed range for the sum of the integration times in seconds per spectrum acquired. If two identical numbers or a single number are passed, this exact time is used. The integration time for individual scans can be adjusted automatically to ensure good performance of the instrument and used to compute the number of scans to average. The maximum and minimum integration times supported by the spectrometer hardware set the effective limits for the integration time. The smallest argument accepted for `tot.time.range` is zero and the largest `Inf`. Setting `tot.time.range = c(0, Inf)` as in the example above ensures that only one integration per measurement will be used, and that the integration times used will be from the whole range the spectrometer supports. It is possible to use in a call to `acq_irrad_interactive()` a value larger than one for `HDR.mult` even when a single integration time is used. This is unusual, but if we want to increase the raw-counts in a low-signal region, by forcing clipping in a different region of spectrum this is an alternative approach to manually setting the integration time to a value given in seconds. The dynamic range in the spectral data can be increased by splicing spectra measured using increasing integration times (similar to "exposure bracketing" as used in photography followed by merging to ensure enough detail is captured in both shade and highlights). In most cases for this to significantly improve the quality of the spectral data we also need to average data from several successive integrations to control the noise. The defaults in `acq_irrad_interactive()` are thus to use the optimal integration time an integration time ten times longer and to keep the total measuring time, the sum of multiple integration events, between 5 and 10 s. ```{r, eval=sr.online} acq_irrad_interactive(HDR.mult = c(short = 1, long = 10), # the default tot.time.range = c(5, 15), # the default correction.method = ooacquire::MAYP11278_ylianttila.mthd, descriptors = ooacquire::MAYP11278_descriptors) ``` The resulting spectra are merged choosing the *most suitable* of the used integration times for each wavelength region. Usually using two integration times with 1 and 10 as `HDR.mult` is a good compromise. However, there is no built-in limitation in the package code, making it possible to use a vector of two, three, four or more values, which combined with a very long total acquisition time of the order of minutes, can yield very good control of noise for light sources with stable output. Consequently, although the default setting `HDR.mult = c(1, 10)` tends to work well, settings like `HDR.mult = c(1, 5, 25)` may be useful in special circumstances. We next set total acquisition time to a fixed length of 10 seconds. Once this setting is active, each time the integration time is automatically set, both the integration time and number of scans are adjusted so that total acquisition time is exactly 10 seconds, for both short and long integration times. ```{r, eval=sr.online} acq_irrad_interactive(HDR.mult = c(1, 10), tot.time.range = 10) ``` If long integration times are needed, the setting above may result in the use of suboptimal integration times, as the desired integration time may have be to be shortened until it becomes an exact fraction of the total acquisition time. To avoid this, we can supply instead of a single fixed value for the total acquisition time, a range of values. ```{r, eval=sr.online} acq_irrad_interactive(HDR.mult = c(1, 10), tot.time.range = c(10, 20)) ``` Sometimes we may want to avoid long integration times even at the cost of not using the whole dynamic range of the detector. For example, when following some fast kinetics, we would usually need the integration time not to exceed a certain time, say 50 milliseconds, we would simply use smaller values, such as `tot.time.range = c(0.03, 0.05)`. When the integration time for individual scans is adjusted automatically the target value of maximum instrument counts is computed proportionally to the maximum counts specification of the spectrometer. The head-room as a fraction of one can be set by the user as depending on the stability or not of the irradiance we need more or less head-room to accomodate increases in irradiance. Parameter `target.margin` allows the user to set a *safety margin* different from the default of 0.1 (or 10%). The example below could help prevent clipping in cases when irradiance changes so fast that it could be up to 25% higher during actual measurement than during the most recent tuning of the integration time. ```{r, eval=sr.online} acq_irrad_interactive(target.margin = 0.25) ``` Although above we described the use of `HDR.mult` as a step in splicing spectra, it is also possible at a later time also the discard spectra acquired using specific multipliers and recalculate the irradiance from selected raw-counts spectra. In some cases with extremely variable light levels when acquiring time series of spectra, it can be wise to use a rather wide range of values in the vector passed to `HDR.mult`, which also accepts values smaller than one, achieving a similar effect as `target.margin` (`target.margin = 0.25` is equivalent to `HDR.mult = 0.75`). The automatic tuning step can be also skipped. For example the value used in the previous measurement event can be reused, or the duration set manually by the user. None of the approaches described above can control stray light caused by reflections in the optical bench. Correction for stray light in the case of some spectrometers makes it necessary to measure it using a short pass filter. The function also supports this type of protocols with two references, and uses it for spectrometers for which special characterization is available, to allow the measurement of UV-B radiation in sunlight. ### Acquiring a time series of irradiance spectra Measuring a time series of irradiance spectra using the approaches described above can be difficult. Triggering acquisition of spectra at regular intervals manually is difficult for long time steps and impossible for very short ones. In most cases we also need to measure many light spectra for each "dark" reference measurement. Interface mode `"series"` introduced in version 0.3.0 of 'ooacquire' and enhanced in later versions targets this type of measurements. For measuring a time series, we pass `interface.mode = "series"` in the call to `acq_irrad_interactive()`. We can set the default parameters for the series through arguments. The following example sets the default for a series with measuring events triggered at the start of each minute, with 60 s time step between successive `"light"` spectra and 60 spectra, for a total time expanse of 1 h. We also set the initial delay to zero, so that once we start the light measurement event by pressing enter, the acquisition of the time series starts at the beginning of the next minute. ```{r, eval=sr.online} acq_irrad_interactive(interface.mode = "series", seq.settings = list(start.boundary = "minute", initial.delay = 0, # seconds step.delay = 60, # seconds num.steps = 60)) ``` More realistically, we would set some other defaults. The example below has worked well in sunlight. ```{r, eval=sr.online} acq_irrad_interactive(interface.mode = "series", tot.time.range = c(5, 10), target.margin = 0.1, qty.out = "irrad", HDR.mult = c(0.3, 1, 3, 10), seq.settings = list(start.boundary = "minute", initial.delay = 0, step.delay = 60, num.steps = 60)) ``` That the acquisition of spectra is continuously running in the spectrometer is something to be aware of, because it means that this is what determines the true time step between successively retrieved spectra and that, say using a `step.delay` that is not a multiple of the frequency at which the spectrometer is acquiring the spectra will result only in missing the retrieval of some of the spectra measured by the spectrometer, but not in a different effective time step! In other words with short time steps we need to be aware that new spectra are acquired at time steps that are a multiple of the **integration time** setting of the spectrometer plus some small overhead that depends on the model of the spectrometer. This is tricky as the length of the integration time needs to be adjusted based on irradiance. The problem barely exists if we average multiple spectra during a measurement, or the time step includes some idle time between acquisitions. The OmniDriver API contains special functions for acquisition of spectra at high speed. These functions store the spectra in a computer memory buffer for later retrieval by user code, minimizing the overhead, but they do not allow control over the timing of successively acquired spectra. The `"series"` mode uses them only for `step.delay = 0` if no HDR integration time bracketing is used. In all other cases normal API functions are used so as to allow the use of HDR bracketing or the choice of the time step between successive acquisitions of spectra. Because, of how OO array spectrometers work, some limitations exist. When using `step.delay = 0` the time step is controlled by the length of the measurement, or the integration time if spectra are not averaged for each measurement. In my test with a specific PC under Windows 10 together with a USB2000 spectrometer and without bracketing I was able to use a minimum `step.delay = 0.052` with `tot.time.range = 0.050` with `HDR.mult = 1`. From these tests I infer that the USB2000 spectrometer is constantly measuring spectra with an overhead of $2\,ms$ (this delay is even shorter in modern spectrometer models) and that the overhead incurred by 'ooacquire' in reading a spectrum is less than 50 ms. In the memory buffered mode with a Maya 2000 Pro I was able to measure using an integration time of $7.2\,ms$ a series with a median time step of less than 7.23 ms or an overhead of less than $30\,\mu s$. However, I observed some small variation in the length of successive time step across a series. Anyway time stamps are saved with a resolution of $1\,\mu s$. **The time overhead depends on the spectrometer, the PC and most likely the operating system and possibly the version of OmniDriver, Java, R and 'ooacquire'. Consequently before attempting measurements using `step.delay` values that are not much longer than the largest value in the vector passed to `tot.time.range`, some experimentation will be needed. The package attempts to detect `step.delay` values that are too short to be achieved and overrides them with `step.delay = 0`**. When using HDR-integration-time bracketing, the spectrometer is not free running or continuously acquiring spectra. Settings need to be changed, acquisition of spectra restarted and the spectral data subsequently acquired only after a freshly measured spectrum is available. This adds a lot of overhead compared to retrieving spectra that are being measured by the spectrometer using multitasking while the PC is busy with other tasks, such as retrieving the previous spectrum. Furthermore, this also means that when using HDR bracketing, the overhead depends on the `tot.time.range` and `HDR.mult` settings and on the integration time. In contrast, when the settings are not modified between successively acquired spectra the overhead is much less and less dependent on the integration time and arguments passed to other parameters. So with care about choosing matching integration times and `step.delay` values without HDR bracketing one can achieve as little as 50 ms between acquired spectra at timed steps. However, when using HDR bracketing, a rough guide is that the minimum time step that can be achieved is about twice the sum of the summed integration time values used. When `tot.time.range` is set as a range, then the longest time should be considered as limiting. The high speed acquisition mode retrieves as quickly as possible the spectra as they are acquired, and the time step is simply the integration time plus the spectrometer overhead. To enable this mode we use `step.delay = 0`. The time stamp used for spectra in 'ooacquire' until version 0.3.0 has been the time when spectra are retrieved from the spectrometer. This is close enough for most individual measurements and series with time steps of several seconds, but when dealing with fast time series this can be inadequate. If `step.delay = 0` is used, the time for the first spectrum is set when the retrieval into the memory buffer is started, but the times for the later spectra are set based on time differences computed from time stamps retrieved from OmniDriver. The theoretical time resolution is better than a microsecond for objects of R's class `POSIXct`, and this class is used in package 'photobiology' to record acquisition times. ------------------------------------------------------------------------ **Measurement of series of spectra with single dark and filter measurements makes it imperative that the functioning of the spectrometer is stable. Consequently, when measuring series, specially if they are longer than a few seconds, do make sure that the temperature of the spectrometer has stabilized and that it remains stable throughout the acquisition of the series. For example, protect the spectrometer from direct sunlight or other sources of energy that can increase its temperature, while allowing ventilation to allow the dissipation of the heat generated by the spectrometer itself. Of course, active temperature control is the best approach. Passive temperature equalization can take 30 to 45 min if the difference between the storage temperature and the temperature under which the spectrometer is used is large (e.g., more than 10 C).** **As the spectrometer electronics dissipate energy, even when used at the same temperature at which it was stored, the spectrometer should be allowed to warm up for 10 to 20 minutes before attempting to measure a time series. Although allowing the spectrometer to warm up for several minutes is always beneficial, when a single reference dark measurement is used for a series of light measurements, it is crucial that the dark reference is not invalidated by changes in the temperature of the spectrometer.** ------------------------------------------------------------------------ ### Spectral fluence Function `acq_irrad_interactive()` has replaced function `acq_fluence_interactive`. Fluence spectra are returned by `acq_irrad_interactive()` when `qty.out = "fluence"`. Some of the defaults for other parameters also change. Otherwise all the functionality described above for irradiance measurements is available except the `"series"` interface mode. If the duration of the exposure to a continuous light source is known, then spectral irradiance multiplied by this time duration gives spectral fluence, and can be obtained by measuring spectral irradiance as described in the previous section. In this section we discuss the case when spectral fluence is measured directly, using an integration time that encompasses the whole duration of the irradiation event, especially when the duration of this event is unknown or spectral irradiance varies through the irradiation event and we are interested in the accumulated exposure or "dose". Fluence, also called radiant exposure, is defined per exposure event rather than per unit time. When measuring pulsed light sources, the length of exposure is determined by the duration of the pulse, which is usually not known with precision. In the discussion below we assume that the quantity of interest is fluence per pulse. For this type of measurements the integration time setting in the spectrometer only affects the window within which pulses will be measured, consequently, it is always set manually. The way of controlling the number of photons reaching the detector during an integration is through the number of pulses per integration and/or the fluence provided by each pulse (e.g., by altering the distance between source and instrument entrance optics or the power settings of the source). We start the session as described above for irradiance, but using function `acq_fluence_interactive()`. Using defaults except for `qty.out`. ```{r, eval = sr.online} acq_irrad_interactive(qty.out = "fluence") ``` The integration time can be set when calling the function to start a session or later interactively. Another important parameter is the number of exposure events. This determines how the resulting data are expressed. If we consider that each individual pulse is an exposure event, and for example, we trigger five flashes per spectrometer integration (or "scan"), we would set `num.exposures = 5`. By default the user is prompted to manually trigger a pulse. However, when triggering of pulses can be automated, a function accepting arguments for number of pulses to parameter `n` (the number of pulses), a delay in seconds to `delay` and a pulse length in seconds to `duration` can be passed as argument to parameter `f.trigger.on`. Of these currently only `n` is implemented, and for future compatibility arguments for `delay` and `duration` must be accepted, and possibly ignored. **It is important to be aware that multitasking if used, must be implemented in the function passed as argument. For automation, the pulse trigger function should return to the caller before the pulses are triggered and the train of pulses start with enough delay for the integration to be started before them. This can be implemented in R code using package 'mirai' or as a delay implemented in the hardware or software used to trigger the flash of light in the source.** ### Spectral transmittance, absorptance and reflectance **All features available in `acq_irrad_interactive()` have now been implemented in function `acq_fraction_interactive()`. This means that series of spectra can be acquired to measure the dynamics of chemical reactions and repeats can be used to measure several samples reusing the same dark and reference spectra. It is also possible to use the HDR feature to improve the noise to signal ratio. Trigger functions work as in `acq_irrad_interactive()`.** A single function can be used for the acquisition of data where the quantities of interest are expressed relative to a reference. This is the case for transmittance, reflectance and absorptance. The same parameters as discussed for `acq_irrad_interactive()` are available in function `acq_fraction_interactive()`. We will here describe three additional parameters only present in this function. The first one is `ref.value` which needs to be supplied if the reference target is imperfect, such as a white reference patch which reflects less than 100%. The default is `ref.value = 1`, but this parameter not only accepts numeric values as argument but also transmittance or reflectance spectra for the target used as reference. Consequently if the reflectance of an "imperfect" reference is know, such reference can be used. (e.g. In the case of measuring the reflectance of very dark objects, it might be convenient to use a grey reference instead of a white reference). For a reference with 97% reflectance across the whole range of wavelengths of interest one would use. ```{r, eval=sr.online} acq_fraction_interactive(ref.value = 0.97) ``` Parameters `qty.out` and `type` determine the class of object and the quantity stored in the returned objects. If the optical set-up used is for measuring specular spectral reflectance, we use the following code. This information will also be used when plotting the data. ```{r, eval=sr.online} acq_fraction_interactive(qty.out = "Rfr", type = "specular") ``` Wen using *light sources with pulsed emission* we start a session by calling function `acq_fraction_pulsed_interactive()` instead of `acq_fraction_interactive` and we set manually the integration time as when measuring spectral fluence. In other respects the procedure is similar to that for continuous light sources except for the need to either manually or automatically trigger the light pulses. ## Rolling your own The functions for interactive acquisition of spectral data described above, although useful in many cases, are meant also as examples. In cases of routine measurements one could simplify menus by removing entries that are not needed, or even converting the functions to non-interactive, and possibly enhancing support for scheduled sequences of spectral data acquisitions. This should be fairly simple, not requiring advanced knowledge of the R language as we export lower level functions that can be used as building blocks for new variations. It should be even possible to use these functions to develop a user interface based of package 'shinny'. ## Using scripts It is also possibly to combine lower level functions using R scripts instead of through writing higher level R functions that call them. Several example scripts are included with the package, including scripts based on low level functions and scripts that demonstrate different ways of using the interactive functions, including in combination with trigger functions. I recommend that you open these example scripts, and save them to your own working directory before sourcing them or editing them. You will need first to locate where in your computer's file system the 'ooacquire' package is installed, and where the file you are interested in is, and to copy it to your own workspace. Here we copy the script called `"irrad-acq-interac.R"`. ```{r, eval=FALSE} filepath <- system.file("example-scripts", "irrad-acq-interac.R", package="ooacquire") file.copy(from = filepath, to = ".") ``` Here we copy all the example scripts. ```{r, eval=FALSE} folderpath <- system.file("example-scripts", package="ooacquire") list.files(path = folderpath, pattern = ".*[.]R") file.copy(from = list.files(path = folderpath, pattern = ".*[.]R", full.names = TRUE), to = ".") ```