Copilot

copilot.py is a script that watches for new images, measures them, and produces a plot, recent.png to help observers understand what the conditions are doing.

Start it from the mzls@mayall-idl.kpno.noao.edu account,

cd ~/obsbot
python copilot.py &

It will monitor the $MOS3_DATA directory, and whenever a new image comes it, it will measure it, update a little database, and create a "strip plot" showing the conditions. The plot should be shown in a python plot window. The plot is also saved to the file "recent.png"; you could for example view it using

eog recent.png &

When a focus sequence is taken, it will run an analysis on the image and show another plot (also saved as focus.png).

The database is in ~/obsbot/obsdb/mosaic3.sqlite3 for MzLS.

Copilot Options

• -h, --help show this help message and exit
• --ext=EXT Extension to read for computing observing conditions: default "N4" for DECam, "im4" for Mosaic3
• --extnum=EXTNUM Integer extension to read
• --rawdata=RAWDATA Directory to monitor for new images: default $MOS3_DATA if set, else "rawdata"
• --n-fwhm=N_FWHM Number of stars on which to measure FWHM
• --no-db Do not append results to database
• --no-focus Do not analyze focus frames
• --fits=FITS Write database to given FITS table
• --plot Plot recent data and quit
• --plot-filename=PLOT_FILENAME Save plot to given file, default recent.png
• --nightplot, --night Plot tonight's data and quit
• --qa-plots Create QA plots
• --keep-plots Do not remove PNG-format plots (normally merged into PDF)
• --mjdstart=MJDSTART MJD (UTC) at which to start plot
• --mjdend=MJDEND MJD (UTC) at which to end plot (default: now, which is 57440.940)
• --skip Skip images that already exist in the database
• --threads=THREADS Run multi-threaded when processing list of files on command-line
• --fix-db
• --tiles=TILES Tiles table, default obstatus/mosaic-tiles_obstatus.fits
• --no-show Do not show plot window, just save it.

Things to Watch For on the Copilot Plot

• Vertical red line accompanied by lack of purple points

This is BAD and signifies that no images have been written out since the last purple point. When this occurs check if you have been taking data. It means that while the telescope may be offsetting and camera opening and closing the shutter properly, no files are being written out! Need to stop (using "touch quit") and restart MOSAIC3 software. Sigh.

• Vertical red line in top panel accompanied by "pixcnt" error message

This means that the frame taken has shifted pixels. You do not need to interrupt observing. Please log the exposure number of the bad frame both in the observing logs and in the bad_expid.txt file.

• Variations in transparency / seeing

May indicate that one has to change to using only pass2 and pass3 or just pass 3.

An example plot:

The top panel shows the seeing. Any problems with the data we know how to detect are marked with red vertical bands. These include:

• md5sum -- this image contains identical pixels as a previous image. This requires action! Please see the problem description https://desi.lbl.gov/trac/wiki/PublicPages/MayallZbandLegacy/NotesforObservers/Problems#a5.Repeatimages HERE.
• pixcnt -- the image readout is shifted. Please have a look at this exposure and note in the logs which CCD has the shift. The shifts are pretty easily recognized in the *mscexam* IRAF window -- blocks of one of the images will be shifted to the left or right. Do not stop observing. This problem is described https://desi.lbl.gov/trac/wiki/PublicPages/MayallZbandLegacy/NotesforObservers/Problems#a4.Shiftedimages here
• (unlabelled bar) -- it has been a while since the last image has been seen. Have images stopped being written? See problem description https://desi.lbl.gov/trac/wiki/PublicPages/MayallZbandLegacy/NotesforObservers/Problems#a0.Noimagesbeingwritten here

The second panel shows the Sky brightness. The "nominal" brightness is shown by the magenta horizontal line. In this panel, the pass number of previous exposures is shown. (This corresponds to the pass we thought the image was going to be, not what the actual conditions delivered.)

The "transparency" panel shows what fraction of the light from stars is getting through the atmosphere to us. Around 90% usually indicates clear photometric conditions.

The "exposure time" panel is complicated. The hollow black dots show the actual exposure time of the images we took. The magenta dots show the exposure time we should have taken, given the measured conditions in that image; it's post-facto what we should have done. We have lower and upper bounds on the exposure times, 80 and 250 seconds. If the "post-facto" target exposure time is below or above the bounds, it is shown with a triangle, and a circle will also be plotted at the lower or upper bound line.

The "depth factor" plot just shows our actual exposure time divided by the post-facto target exposure time. 1 indicates that we're doing exactly what we should be doing. Greater than 1 indicates we're over-exposing, and less than 1 indicates that we're under-exposing. The vertical labels show the exposure numbers.

Note that zeros will not show up on the plot. The times of focus frames will be marked with a black dot labelled "F".

Notes

At the moment, copilot.py just waits for new images to appear, but you can also tell it to process a set of images (eg, if it crashed and you want it to run the backlog) by specifying the files on the command line. You can also tell it to skip files that already exist in the database via --skip, eg,

python copilot.py --skip $MOS3_DATA/mos3.*.fits

The database just records our CCD measurements, so it's not valuable. If things somehow go wrong and the database gets corrupted or you get git wedged or something, you can always delete it and start over via:

python obsdb/manage.py flush
python obsdb/manage.py migrate