Copilot

copilot.py is a script that watches for new images, measures them, and produces a plot, recent.png to help observers understand what the conditions are doing.

Start it from the mzls@mayall-idl.kpno.noao.edu account,

cd ~/obsbot
python copilot.py &
eog recent.png &

It will monitor the $MOS3_DATA directory, and whenever a new image comes it, it will measure it, update a little database, and create a "strip plot" showing the conditions.

The database is in ~/obsbot/obsdb/obsdb.sqlite3.

Copilot Options

• -h, --help show this help message and exit
• --ext=EXT Extension to read for computing observing conditions: default "N4" for DECam, "im4" for Mosaic3
• --extnum=EXTNUM Integer extension to read
• --rawdata=RAWDATA Directory to monitor for new images: default $MOS3_DATA if set, else "rawdata"
• --portion=PORTION Portion of the night: default 1.0
• --n-fwhm=N_FWHM Number of stars on which to measure FWHM
• --no-db Do not append results to database
• --no-focus Do not analyze focus frames
• --fits=FITS Write database to given FITS table
• --plot Plot recent data and quit
• --plot-filename=PLOT_FILENAME Save plot to given file, default recent.png
• --nightplot, --night Plot tonight's data and quit
• --qa-plots Create QA plots
• --keep-plots Do not remove PNG-format plots (normally merged into PDF)
• --mjdstart=MJDSTART MJD (UTC) at which to start plot
• --mjdend=MJDEND MJD (UTC) at which to end plot (default: now, which is 57440.940)
• --skip Skip images that already exist in the database
• --threads=THREADS Run multi-threaded when processing list of files on command-line
• --fix-db
• --tiles=TILES Tiles table, default obstatus/mosaic-tiles_obstatus.fits
• --no-show Do not show plot window, just save it.

Things to Watch For

• Vertical red lines accompanied by lack of purple points

This is BAD and signifies that no images have been written out since the last purple point. When this occurs check if you have been taking data. It means that while the telescope may be offsetting and camera opening and closing the shutter properly, no files are being written out! Need to stop (using "touch quit") and restart MOSAIC3 software. Sigh.

• Vertical red lines accompanied with "pixcnt" error message

An example plot:

The top panel shows the seeing. Any problems with the data we know how to detect are marked with red vertical bands. These include:

• readtime -- MAY indicate a duplicated image ("md5sum" is a better indicator). ALSO, typing "hselect mos3*fits[0] $I,readtime yes" in the IRAF window can show you the readtime header jkeyword values for the images. If the last several are identical, then this requires action.
• md5sum -- this image contains identical pixels as a previous image. This requires action! Please see the problem description https://desi.lbl.gov/trac/wiki/PublicPages/MayallZbandLegacy/NotesforObservers/Problems#a5.Repeatimages HERE.
• pixcnt -- the image readout is shifted. Please have a look at this exposure and note in the logs which CCD has the shift. The shifts are pretty easily recognized in the *mscexam* IRAF window -- blocks of one of the images will be shifted to the left or right. Do not stop observing. This problem is described https://desi.lbl.gov/trac/wiki/PublicPages/MayallZbandLegacy/NotesforObservers/Problems#a4.Shiftedimages here
• (unlabelled bar) -- it has been a while since the last image has been seen. Have images stopped being written? See problem description https://desi.lbl.gov/trac/wiki/PublicPages/MayallZbandLegacy/NotesforObservers/Problems#a0.Noimagesbeingwritten here

The second panel shows the Sky brightness. The "nominal" brightness is shown by the magenta horizontal line. In this panel, the pass number of previous exposures is shown. (This corresponds to the pass we thought the image was going to be, not what the actual conditions delivered.)

The "transparency" panel shows what fraction of the light from stars is getting through the atmosphere to us. Around 90% usually indicates clear photometric conditions.

The "exposure time" panel is complicated. The hollow black dots show the actual exposure time of the images we took. The magenta dots show the exposure time we should have taken, given the measured conditions in that image; it's post-facto what we should have done. We have lower and upper bounds on the exposure times, 80 and 250 seconds. If the "post-facto" target exposure time is below or above the bounds, it is shown with a triangle, and a circle will also be plotted at the lower or upper bound line.

The "depth factor" plot just shows our actual exposure time divided by the post-facto target exposure time. 1 indicates that we're doing exactly what we should be doing. Greater than 1 indicates we're over-exposing, and less than 1 indicates that we're under-exposing. The vertical labels show the exposure numbers.

Note that zeros and focus frames will not show up on the plot.

Notes

At the moment, copilot.py just waits for new images to appear, but you can also tell it to process a set of images (eg, if it crashed and you want it to run the backlog) by specifying the files on the command line. You can also tell it to skip files that already exist in the database via --skip, eg,

python copilot.py --skip $MOS3_DATA/mos3.*.fits

The database just records our CCD measurements, so it's not valuable. If things somehow go wrong and the database gets corrupted or you get git wedged or something, you can always delete it and start over via:

python obsdb/manage.py flush