OBSERVING INSTRUCTIONS FOR MzLS

Still need to incorporate the following instructions:

• Afternoon biases and flats
• Pointing the telescope, and re-zeroing the pointing using the numbers reported by MOSSTAT
• Focusing the camera
• Running the focus loop (or choosing not to)
• Instructions for converting the JSON files into the ones that the telescope can read (what are these called?)
• ALMANAC code, which we've not yet ported from the DECam version
• Coverage plots, which we've not yet ported from the DECam version

These are instructions for using the MzLS planning scripts for the nightly program. To see the Mosaic observing manual, go here: ​http://www.noao.edu/kpno/mosaic/manual.

Log on to the mzls account on the mayall-idl.kpno.noao.edu machine (which can only be done from within the KPNO network or using a VPN):

Username : mzls
Password : Ask Arjun, David, or previous observers

Two environment variables define the locations of data and code:

$MOS3_DATA= -- This is where our raw data files are written
$MOS3_OBS=$HOME/mosaic3 -- This contains the observing product with code and versioned log files.
$PS1CAT_DIR=$HOME/ps1/chunks-qz-star-v2 -- Pan-STARRS1 catalogs used by the IDL DECSTAT routine for computing astrometric offsets and photometry

These path names are set in the .bashrc file, or can be set for example with "export MOS3_DATA=/mosaic3/data2/observer".

Other directories in the home directory are:

data/ -- Link to the top-level data directory, where each night is stored in subdirectories like "20151213"
exec/ -- Cross-mount to mosaic.kpno.noao.edu:/home/observer/exec
products/ -- Code checked out from the SDSS and the DESI svn repositories

Documentation for the IDL scripts can be printed from the IDL prompt with the DOC_LIBRARY command, for example:

      IDL> doc_library,'mosstat'

Date convention

All dates in log files are set to the local date of the beginning of the night. For example, any data taken during the night of December 13/14, 2015 will be written as 2015-12-13. This is consistent with how the NOAO Science Archive timestamps and saves the raw data files.

Official Nightly Strategy

Update 13 Dec 2015 Do the following to update the current NMSL scripts at any time.

• login to mayall-idl as mzls
• cd products/mosaic3
• python2.7 py/mosaic3strategy.py -t obstatus/mosaic-tiles_obstatus.fits -d 2015-12-13 -p 1 -pass 1 --moon-sep-limit 50 --start-date 2015-12-14 --start-time 02:00:00 -seez 1.15 -sbz 18.45 --dir <dir_name>

Rename the script, so we don't overwrite it later:

mv mosaic_2015-12-13_plan.json mosaic_2015-12-13_plan1.json mv strategy_2015-12-13.txt strategy_2015-12-13_plan1.txt

The mosaicstrategy script outputs a JSON file (e.g., mosaic_2015_12-13_plan.json in the above example). In order to produce the execution script for NOCS, run Behzad Abareshi's script:

python ./py/jnox.py --in mosaic_2015-12-13_plan1.json --out mzls_2015-12-13_plan1.sh

Then copy this output (e.g., mzls_2015-12-13_plan1.sh) to the exec directory of the observer account on the mosaic3 computer:

scp mzls_2015-12-13_plan1.sh observer@mosaic3:/home/observer/exec/

Start uptiles running in an IDL window on mayall-idl; this will always use the most up-to-date version of the mosaic-tiles_obstatus.fits file. Once this finishes running (usually a couple of minutes), in a nocs terminal window on mosaic3, cd to exec and execute the script:

~/exec> ./mzls_2015-12-13_plan1.sh

Instructions and details for mosaicstrategy.py

written by A. Patej

mosaicstrategy.py is a Python script that can generate a nightly observing plan. It requires several Python modules:

• numpy
• pylab (Matplotlib)
• ephem (Pyephem)
• pyfits

The code is adopted from the DecamLegacy project. See those wiki pages for more information.

Selection of Pass 1, 2, or 3

It is very important to adhere to the strategy of when to observe pass 1, 2, or 3 tiles, in particular that pass 1 tiles are always in photometric good-seeing conditions. The overall strategy is outlined at MayallZbandLegacy/ObservingStrategy].

The strategy is as follows:

• Pass 1: Photometric and seeing < 1.3 arc sec
• Pass 2: Photometric or seeing < 1.3 arc sec, or all Pass 1 tiles completed
• Pass 3: Unphotometric and seeing > 1.3 arc sec, or all Pass 1 and Pass 2 tiles completed

These need not be observed in any particular order, e.g. pass 3 observations can certainly occur before pass 1.

If conditions change, then you should definitely change to a different pass. For example, if clouds begin to roll in, you should stop any pass 1 observations right away!

Running the script

At a minimum, the mosaicstrategy.py script can be run with these inputs:

-t *name of file with tile centers*
-d *UT date at start of night in format yyyy-mm-dd*
-p *portion of night (1.0, 0.5 (first half), -0.5 (second half))*
-pass *desired pass - input 1, 2, or 3*
--dir *name of the dir where the output scripts are located*

For instance, to generate a pass 3 plan for the entire night of April 26, 2015, run:

python ./nightlystrategy.py -t moasic-tiles_obstatus.fits -d 2015-04-26 -p 1.0 -pass 3 --dir obsScripts [[BR]]
python ./py/jnox.py --in mosaic_2015-04-26_plan.json --out mzls_2015-04-26_plan.sh

Additional optional arguments:

1. Observing conditions:
   -sb(f) *sky brightness in filter f; replace (f) = g, r, or z* in mag/sqarcsec
   -see(f) *seeing in filter f; replace (f) = g, r, or z* in arcsec
   -transparency *value of the transparency* (0-1.0)
   

2. Observing constraints:
   --airmass-limit *maximum allowed airmass, default is 2.3*
   --moon-sep-limit *minimum allowed moon separation, default is 50.0 deg*
   --moon-alt-limit *moon altitude in deg required for switching between z and g,r*
   --min-dec *Declination minimum limit for choosing tiles
   --max-dec *Declination maximum limit for choosing tiles
   

3. Custom Scheduling:
   It's possible to create a plan for a specific block of time. To do so, use the following options:
   --start-date *Date of start of observations (UTC) in format yyyy-mm-dd; overrides the default start of night, which is calculated using --date and --portion. Requires --start-time to also be set*
   --start-time *Time of start of observations (UTC) in format hh:mm:ss; overrides the default start of night, which is calculated using --date and --portion. Requires --start-date to also be set*
   --end-date *Date of end of observations (UTC) in format yyyy-mm-dd; overrides the default end of night, which is calculated using --date and --portion. Requires --end-time to also be set*
   --end-time *Time of end of observations (UTC) in format hh:mm:ss; overrides the default end of night, which is calculated using --date and --portion. Requires --end-date to also be set*
   

More complicated example using these options:
Assuming 18.2 as z-band sky brightness, 1.2" seeing in z and 1.0" seeing in r, photometric conditions (pass 1) and allowing a moon separation of 50 deg and including DESI tiles:

python ./mosaicstrategy.py -t mosaic-tiles_obstatus.fits -d 2015-04-26 -p 1.0 -pass 1 -sbz 18.2 -seer 1.0 -seez 1.2 --moon-sep-limit 50.0 --expand [[BR]]
python ./py/jnox.py --in mosaic_2015-04-26_plan.json --out mzls_2015-04-26_plan.sh

Example using custom scheduling:
To generate a plan for just two hours on the night that starts on 2015-04-26 between 05:00 and 07:00 UTC, and restricting the DEC range:

python ./mosaicstrategy.py -t mosaic-tiles_obstatus.fits -d 2015-04-26 -p 1.0 -pass 1 --start-date 2015-04-27 --start-time 05:00:00 --end-date 2015-04-27 --end-time 07:00:00 --min-dec=34 --max-dec=50 --moon-sep-limit 50.0 [[BR]]
python ./py/jnox.py --in mosaic_2015-04-26_plan.json --out mzls_2015-04-26_plan.sh

Outputs

This script will output many data files to your current directory:

1. a single text file called "strategy_mm-dd.txt", with information about all the pointings for the night (exp time, filter, approximately when they should be run, airmass, etc). There is a corresponding plot of what has been observed previously (by pass) and what you will be observing given the plan for the night: saved as a pdf, 'plot_plan_yyy-mm-dd.pdf' (gray = z, red = r, green = g). Planets and moon are plotted as well.

2. many executable script files of the form "Mosaic_Survey<tile id>.sh" for the night corresponding to the plan. There is one script file for each pointing in the plan. A single meta-script is produced that will call those ".sh" files: moasic_<ut date>_plan.sh.

3. A FITS file called "mosaic-tiles_obstatus_update_yyyy-mm-dd.fits" that is a temporary updated mosaic obstatus file that assumes all planned observations were completed. This should not be used during the run; instead use the updated tiles file obtained from running the idl program, uptiles (see section on "Creating the Updated Observing Status File" below).

Updating the "observed tiles" file

The FITS file listing which tiles have been completed should be updated throughout the night. This file is $MOS3_OBS/obstatus/mosaic-tiles_obstatus.fits. The following IDL command will monitor exposures as they are taken throughout the night, automatically updating this file:

   IDL> muptiles

At the end of the night, you should check the updated tile file into the svn repository:

   cd $MOS3_OBS/obstatus
   svn commit obstatus/mosaic-tiles_obstatus.fits --username <YOUR-NAME>

Checking the Sky Brightness, Seeing and Transparency

The observing strategy files take as input these environmental conditions for setting the exposure times to achieve the required MzLS survey depth.

From an IDL prompt, use the MOSSTAT routine to analyze the latest image on disk:

   IDL> mosstat

There are keyword options that allow you to choose different exposure numbers or CCDs within that exposure. For example, to analyze chip 'im16' of the exposure number 12345, type:

   IDL> mosstat, 12345, ext='im16'

The full documentation can be seen with:

   IDL> doc_library,'mosstat'

If the telescope pointing has gone off by more than about 30 arcsec (which happens!), then this routine will likely fail to match stars. If this happens, the operator will need to recenter the telescope.

THE FOLLOWING NOT YET UPDATED FOR MOSAIC-3 One can also check the observing progress using the almanac command:

IDL> almanac, 10001, /noprint

will print out a summary of all the frames from 10001 to the present in a nice tabular form (widen the window). This is useful for checking how the seeing and sky brightness have varied since frame 10001, and whether one should create and upload a new JSON observing script. The /noprint just prevents the decstat output for every frame; if you want to see that, then don't use the /noprint keyword.

Creating an almanac file with the observations for the night

THE FOLLOWING NOT YET UPDATED FOR MOSAIC-3

At the end of the night, an almanac file should be generated for the night and checked into svn.

cd products/mosaic3/logs
idl
   almanac, <START-EXPOSURE>, <END-EXPOSURE>
   exit
svn add Almanac*
svn commit Almanac*

For example, for the night of March 26/27, 2015, this is done with:

cd products/mosaic3/logs
idl
   almanac, 424978, 425143
   exit
svn add Almanac_2015-03-26.*
svn commit Almanac_2015-03-26.*

There are actually two versions of this file, one that is an ASCII file (with .txt extension) and one that is a FITS file (with .fits extension).

Writing the human logs

THE FOLLOWING NOT YET UPDATED FOR MOSAIC-3

Please write about what happened during your night: weather conditions, which pass you observed, and telescope problems. Follow the example on the pages at DecamLegacy/ObservingLogs.

Catastrophically bad frames (such as saturated frames, or where the telescope moved) are recorded and svn-checked-in to products/mosaic3/logs/bad_expid.txt.

In case of technical difficulties

We have in the past encountered technical difficulties that have restricted where we can observe to a specific range in RA/Dec.

If this happens during your run, you have several options:

1. First, try running mosaicstrategy.py with an appropriate --airmass-limit and/or --moon-sep-limit to try to keep to the required range of tiles.
   
2. Shorten your observing period (using --start-date,--start-time and --end-date,--end-time) such that it's scheduling fewer tiles.
3. Increase the assumed seeing such that it schedules longer exposures.

For reference, the tiles we’d be observing with MOSAIC3 in z-band are:

IN_DESI = 1 DEC > 30 88 < RA < 301 PASS <= 3

That’s 48760 tiles.

Example run-through for a given night

=

Log into the mzls account on the mayall-idl machine.

user@anyshell>ssh ??@??

Update everything

Update the code, log files and most importantly the tile file:

cd ~/products/mosaic3
svn up
cd ~/products/observing
svn up

Create nightly plan

Create the nightly plan from the available tiles. This will read the tile file to see which ones are already completed.

cd ~/products/mosaic3/json
python py/mosaicstrategy.py -t obstatus/mosaic-tiles_obstatus.fits -d *UT date at ''start'' of night* -p *portion of night* -pass *pass number*
python ./py/jnox.py --in *JSON input file --out *Output shell script
scp *output shell script to the /home/observer/exec area on mosaic3

Go to /home/observer/exec, and execute the mzls*.sh script. This must be done from the observer@mosaic account?

Start the automated script for updating the tile file

From a terminal on the mayall-idl compute, in the mzls account, start this running:

idl
   muptiles

From another terminal in the mzls account, start this running:

idl
   mosstat_continuous

This will run mosstat on each frame as it shows up and display the results on the screen.

Create the Almanac and Almanac plot files

THE FOLLOWING NOT YET UPDATED FOR MOSAIC-3 At the end of the night, create the Almanac files and check them into svn.

- cd products/mosaic3/logs
- idl
- almanac,<first exp num>
- plotalmanac,'Almanac_date.fits',ps='plot_Almanac_date'
- exit
- svn add *Almanac_date*
- svn commit

Create a log file

THE FOLLOWING NOT YET UPDATED FOR MOSAIC-3

Generating a Coverage Map

The code pro/coverage_plot.pro will generate coverage maps for a given date range using the info in mosaic-tiles_obstatus.fits. To run it, login as ??@?? and start IDL:

IDL>.com coverage_plot

To produce the plot for a single night use, for example:

IDL> coverage_plot, datemin='2015-04-07', datemax='2015-04-07'

To produce a plot for a range of nights use:

IDL> coverage_plot, datemin='date-start', datemax='date-end'

To produce a plot for all nights to date use:

IDL> coverage_plot

All plots will be put in $??_OBS/obstatus/Plots. The default is to output a PDF but there are flags to output other formats.

The code pro/coverage_plot_alldates.pro will produce a set of cumulative coverage plots (one per day) over a date range. Within IDL when logged in as ??@?? you enter:

IDL> .com coverage_plot_alldates
IDL> coverage_plot_alldates, datemin='date-start', datemax='date-end'

All plots, consecutively numbered, will be put in $??_OBS/obstatus/Plots/PDFs and can be combined into a movie, e.g with the free graphicconverter ​http://www.lemkesoft.de/en/image-editing-slideshow-browser-batch-conversion-metadata-and-more-on-your-mac/ program on Macs.

Ben Weaver would prefer that we do not upload any plots to SVN but rather just the code. So feel free to mail plots around but there is no need to check them in.

---

Pages linking to PublicPages/MayallZbandLegacy/NotesforObservers:

• MayallZbandLegacy
• MayallZbandLegacy/NotesforObservers
• MayallZbandLegacy/ObservingSchedule
• MayallZbandLegacy/Run08
• MayallZbandLegacy/Run13
• PublicPages/MayallZbandLegacy
• PublicPages/MayallZbandLegacy/NotesforObservers/Copilot