Copilot

copilot.py is a script that watches for new images, measures them, and produces a plot to help observers understand what the conditions are doing.

Start it from the mzls@mayall-idl.kpno.noao.edu account,

cd ~/obsbot
python copilot.py &

It will monitor the $MOS3_DATA directory, and whenever a new image comes it, it will measure it, update a little database, and create a "strip plot" showing the conditions. The plot should be shown in a python plot window. The plot is also saved to the file recent.png; you could for example view it using

eog recent.png &

When a focus sequence is taken, it will run an analysis on the image and show another plot (also saved as focus.png).

The database is in ~/obsbot/obsdb/mosaic3.sqlite3 for MzLS.

If you want to make a plot showing your whole night, you can do

python copilot.py --night

and it will save the plot night.png.

Copilot Options (not usually needed)

• -h, --help show this help message and exit
• --ext=EXT Extension to read for computing observing conditions: default "N4" for DECam, "im4" for Mosaic3
• --extnum=EXTNUM Integer extension to read
• --rawdata=RAWDATA Directory to monitor for new images: default $MOS3_DATA if set, else "rawdata"
• --n-fwhm=N_FWHM Number of stars on which to measure FWHM
• --no-db Do not append results to database
• --no-focus Do not analyze focus frames
• --fits=FITS Write database to given FITS table
• --plot Plot recent data and quit
• --plot-filename=PLOT_FILENAME Save plot to given file, default recent.png
• --nightplot, --night Plot tonight's data and quit
• --qa-plots Create QA plots
• --keep-plots Do not remove PNG-format plots (normally merged into PDF)
• --mjdstart=MJDSTART MJD (UTC) at which to start plot
• --mjdend=MJDEND MJD (UTC) at which to end plot (default: now, which is 57440.940)
• --skip Skip images that already exist in the database
• --threads=THREADS Run multi-threaded when processing list of files on command-line
• --fix-db
• --tiles=TILES Tiles table, default obstatus/mosaic-tiles_obstatus.fits
• --no-show Do not show plot window, just save it.

Things to Watch For on the Copilot Plot

• Vertical red line labelled *MISSING IMAGE!*

This can be BAD -- it signifies that no images have been written out since the last purple point. When this occurs, check if you have been taking data. It means that while the telescope may be offsetting and camera opening and closing the shutter properly, no files are being written out! Need to stop (using "touch ~/exec/mosbot/quit") and restart MOSAIC3 software. Sigh.

• Vertical red line in top panel accompanied by "pixcnt" error message

This means that the frame taken has shifted pixels. You do not need to interrupt observing. Please log the exposure number of the bad frame both in the observing logs and in the bad_expid.txt file.

• Variations in transparency / seeing

May indicate that one has to change to using only pass2 and pass3 or just pass 3.

An example plot:

The top panel shows the seeing. Any problems with the data we know how to detect are marked with red vertical bands. These include:

• md5sum -- this image contains identical pixels as a previous image. This requires action! Please see the problem description https://desi.lbl.gov/trac/wiki/PublicPages/MayallZbandLegacy/NotesforObservers/Problems#a5.Repeatimages HERE.
• pixcnt -- the image readout is shifted. Please have a look at this exposure and note in the logs which CCD has the shift. The shifts are pretty easily recognized in the *mscexam* IRAF window -- blocks of one of the images will be shifted to the left or right. Do not stop observing. This problem is described https://desi.lbl.gov/trac/wiki/PublicPages/MayallZbandLegacy/NotesforObservers/Problems#a4.Shiftedimages here
• MISSING IMAGE -- it has been a while since the last image has been seen. Have images stopped being written? See problem description https://desi.lbl.gov/trac/wiki/PublicPages/MayallZbandLegacy/NotesforObservers/Problems#a0.Noimagesbeingwritten here

The second panel shows the Sky brightness. The "nominal" brightness is shown by the magenta horizontal line. In this panel, the pass number of previous exposures is shown. (This corresponds to the pass we thought the image was going to be, not what the actual conditions delivered.)

The "transparency" panel shows what fraction of the light from stars is getting through the atmosphere to us. Around 90% usually indicates clear photometric conditions.

The "exposure time" panel is complicated. The filled magenta dots show the actual exposure time of the images we took. The black circles show the exposure time we should have taken, given the measured conditions in that image; it's post-facto what we should have done. We have lower and upper bounds on the exposure times, 80 and 250 seconds. If the "post-facto" target exposure time is below or above the bounds, it is shown with a triangle, and a circle will also be plotted at the lower or upper bound line. Triangles above the upper bound indicate images that might not be as deep as desired. Sometimes a line is visible in this plot (e.g. see 10hr - 11hr above 250 sec). This is the predicted max exposure time to saturate (really to get to a "do not cross" number of counts) the background. This relates to safety of the detectors. If you notice real exposure times are getting close to this line, its time to stop tonight.sh and take stock. This most easily would happen with moon and clouds which make the background high.

[depth factor has been removed from the plot. This description is useful when looking at plots created before about February 2016] The "depth factor" plot just shows our actual exposure time divided by the post-facto target exposure time. 1 indicates that we're doing exactly what we should be doing. Greater than 1 indicates we're over-exposing, and less than 1 indicates that we're under-exposing. The vertical labels show the exposure numbers.

The final panel is a strip chart showing the RA and DEC offsets from tile center determined post facto. When offsets are greater than a total of 10", the telescope zero point should be updated. Pointing and tracking are currently excellent at the Mayall. The actual telescope pointing updates should be made using output from mosstat. Thus the copilot plot may show non zero, but constant offsets. The goal is to get the mosstat and copilot zero points to agree.

Focus frames will be marked with a dot labelled "F", and Zero frames will be marked "Z".

Notes

At the moment, copilot.py just waits for new images to appear, but you can also tell it to process a set of images (eg, if it crashed and you want it to run the backlog) by specifying the files on the command line. You can also tell it to skip files that already exist in the database via --skip, eg,

python copilot.py --skip $MOS3_DATA/mos3.*.fits

The database just records our CCD measurements, so it's not valuable. If things somehow go wrong and the database gets corrupted or you get git wedged or something, you can always delete it and start over via:

python obsdb/manage.py flush
python obsdb/manage.py migrate