1   Introduction¶

This technote is to detail out the observing scripts (jobs) and notebook development cycle from a simple test or idea, developed inside a notebook, to evolving into a method that could be called by other notebooks, and/or into a script to be called via the scriptQueue, then finally to the level of being a sanctioned and regularly maintained Job as part of operations.

Details for each step include which repos should be used for the different aspects of development and the level of testing/review required at each stage. The note is built upon a sample use-case which was encountered early on in the commissioning phase of the AuxTel, where a user wants to derive a high-level piece of functionality which requires new additions to code at multiple levels to implement as a final production script. In short, the use-case is:

• Slew to a target
• Take an image and perform basic ISR such that it can be analyzed.
• Find the brightest point source in the image, determine its location, calculate the telescope offset required to put the star on a specific pixel (within a tolerance).
• Perform a series of observations using multiple instrument setups and exposure times. This will also require changing telescope focus (and pointing) for each setup.

A general overview of the development flow, as viewed from the user, is as follows:

• Draft, test and flush-out their desired functionality in a notebook.

  This may include creating drafts of functions (e.g. calculating offsets), performing calls to high-level classes (e.g. slew telescope)

• Create observing utilities to perform specific tasks, which may be sufficiently generic such that they may be used by other use-cases (e.g. find a star and put it on pixel [x,y]).

• Create a job, which is runnable by the Queue, to perform these tasks.

  Note that at this point the utilities may still be rough, and certain functionality might be better accomplished in lower classes (e.g. attcs) but that functionality does not yet exist.

• Request new functionality in lower-level control classes (e.g. attcs)

• Migrate/evolve the utilities and Job to a production level for regular use.

2   Jupyter notebooks¶

Jupyter notebooks (henceforth referred to as notebooks) will be the primary tool used in system verification and commissioning. The environment permits the simultaneous control of observatory functionality, data reduction/analysis tasks and documentation. This is the natural starting point for development of ideas and demonstrating proof of concept(s). In the use-case referenced in this technote, notebooks are the starting point, where the user is free to do as they wish with its structure/content etc.

User’s notebooks are currently stored in the ts_notebooks repository. There is also a section where individuals create directories with their identifying name (e.g. pingraham). Notebooks should be cleared of all data prior to committing/pushing, to prevent the repo size from ballooning. The repo also holds a series of examples which ranges in from telescope operation to EFD mining/analysis.

Users should still follow the T&S development guidelines when using this repo. That means, create a ticket branch to work on, commit code and, once ready, open a PR to have their work integrated to the develop branch. Content added to the users directory are still subjected to the PR process but only to guarantee that the content was cleared out and that no changes where made to other users content (without permission). Contents in the examples directory will be subject to a more rigorous review process.

It is understood that the practice of storing notebooks, particularly the personal notebooks, will not scale into commissioning. It is anticipated that this repo will split into multiple components such as example notebooks, operations-focused notebooks (where they will be run by operators to diagnose or characterize certain behaviour), and personal notebooks. The details of this organization are beyond the scope of this technote. Until the re-organisation is completed, tags will be made of the repo at least every ~6 months, after which all files larger than 20 MB (TBR) or older than 1 year will be deleted from the develop branch.

3   Observing Utilities¶

Observing utilities are user-defined methods that perform tasks that are not already part of the control classes code base that operates the observatory (the Control Classes discusses this in further detail). An example of functionality contained in a utility would be the reduction/analysis of an image. In the use-case discussed in this document, the user defines methods that perform basic ISR on an image, finds the center of the star, and calculates the required offset.

The repo sanctioned for the development and use of such functions is the ts_observing_utilities repo, which follows a standard package format such as used with ts_standardscripts. Users develop their functions on a branch and the functions must go through a review (PR) process prior to being merged to the develop branch. This area is designed to act as a staging area prior to having their functionality either moved into control classes, or promoted to sanctioned utilities which would be contained in the ts_observatory_controls repo (discussed in Control Classes section).

The development practices of this area are purposefully loose to promote rapid coding and integration. Although functions should follow a generic standard, the only strict requirement is that each function possess a deprecation date. This is required to guard against bit-rot. As will be discussed below, all scripts under development must have unit tests to verify the desired utilities are not expiring. This is discussed further in Control Utilities. Utilities with deprecation dates that are 60 (TBD) days past will be removed.

4   Control Classes¶

Control Classes perform coordination of CSC functionality at a high-level. An example of such an operation is slewing the telescope and dome, discussed in more detail below. Because these classes are used throughout many areas of operations, high levels of unit and integration testing are required; especially if utilities are contained outside the class. High-level control classes live in their own repository (ts_observatory_control). These classes are written and tightly controlled by the T&S team.

In the example use-case for this technote, the user wishes to take images with multiple instrument setups. Because the focus changes with different glass thicknesses and wavelength, this is the type of functionality that really should belong in the standard control-classes. However, while this use-case was being developed, that functionality didn’t exist and was therefore contained in a utility (in ts_observing_utilities).

To remedy this, the proper path forward is to request that the additional functionality be added. To do this, the user should file a JIRA ticket with the requested functionality for review. This will trigger discussion on whether the functionality should indeed be implemented. Upon conclusion of that discussion, a user can either wait for it to be implemented or make the changes themselves and submit a pull-request.

In the meantime, the utility in ts_observing_utilities must remain until the functionality gets included in the control-classes. Once included, the utility could be deprecated and the appropriate code updated accordingly.

from lsst.ts.standardscripts.auxtel.attcs import ATTCS
attcs = ATTCS()
await attcs.start_task
await attcs.slew_icrs(ra="20:25:38.85705", dec="-56:44:06.3230", sky_pos=0., target_name="Alf Pav")

from lsst.ts.standardscripts.auxtel.attcs import ATTCS
attcs = ATTCS()
await attcs.start_task
await attcs.slew_object(name="Alf Pav", sky_pos=0.)

from lsst.ts.standardscripts.auxtel.latiss import LATISS
latiss = LATISS()
await latiss.start_task
exp_id = await latiss.take_engtest(exptime=10, filter='RG06', grating='empty_1')

5   Jobs for the Queue¶

The Queue (currently ScriptQueue) is the mechanism to run scripts in an automated fashion during commissioning and operations. The level of robustness required for these scripts is divided among those still in development, and those which are in full production.