= OBSERVING INSTRUCTIONS FOR MzLS =

'''Still need to incorporate the following instructions:'''
  - '''Running the guiding loop -- although we typically do not guide on the Mayall for these 1-2 min exposures'''
  - '''Coverage plots, which we've not yet ported from the DECam version'''

These are instructions for observing for MzLS. 

[[PageOutline]]


----

= VERY VERY IMPORTANT = 

== 1. DO NOT OVEREXPOSE MOSAIC CCDs to LIGHT! ==

Prolonged exposure can damage and even destroy the devices. The following rules are in effect: 

1. Operation on sky is restricted to between 12 deg twilights. 
2. Operation is restricted to when OAs or other NOAO technical/scientific staff are present
3. The cameras and MOSAIC software should be shut down when observing / NOAO personnel are not present
4. Sky values should be kept < 40,000 adu (<30,000 adu is preferable). 

== 2. PLEASE TALK TO THE OAs ABOUT SAFETY AT THE TELESCOPE ==

1. Do not leave the control room at night without telling the OA
2. Do not go into the dome without the OA's permission
3. Do not go exploring by yourself
4. Carry a flashlight and radio at night

----

= Read the Manual for MOSAIC operations =

To see the Mosaic1.1 manual, go here: http://www.noao.edu/kpno/mosaic/manual.
Mosaic3 also runs NOCS. We'll hopefully have an updated manual soon.

= Some Preliminaries = 

Who is your OA for tonight? See [http://www.noao.edu/perl/Calcium?Op=ShowIt&CookieParams=1&CalendarName=oa&Amount=Month&NavType=Absolute&Type=Block]

Observing can only be done from the Mayall control room or from within the KPNO/NOAO network using a VPN. 

The telescope and low-level instrument control runs on observer@mayall-2.kpno.noao.edu .
Log on to the mayall-2 computer: [[BR]]
    Username : 4meter [[BR]]
    Password : <Ask someone on our team or the staff> [[BR]]
To run MOSAIC3, use the MOSAIC3 Menu GUI. Or you can also log into the mosaic3 computer from mayall-2:[[BR]]
    ssh -XY observer mosaic3[[BR]]
and use command-line nocs commands. [[BR]]
'''PLEASE BE CAREFUL ON THIS COMPUTER - IT RUNS THE CONTROLLERS AND THE INSTRUMENT!!!'''

Our observing scripts and data reduction runs on mzls@mayall-idl.kpno.noao.edu :
    Username : mzls  [[BR]]
    Password : <Ask someone on our team or previous observers> [[BR]]

These environment variables define the locations of data and code:
   '''$HOME=/home/mzls'''[[BR]]
   '''$MOS3_DATA=''' -- This is where our raw data files are written. Should be something like /mosaic3/data2/observer/20160202. This is a network mounted disk on the machine mosaic3. [[BR]]
   '''$MOS3_OBS=$HOME/products/mosaic3''' -- This contains the observing product with code and versioned log files. [[BR]]
   '''$PS1CAT_DIR=$HOME/ps1/chunks-qz-star-v2''' -- Pan-STARRS1 catalogs used by the IDL DECSTAT routine for computing astrometric offsets and photometry [[BR]]
These path names are set in the .bashrc file, or can be set for example with "export MOS3_DATA=/mosaic3/data2/observer".
The $MOS3_DATA directory must be updated to point to the current night's data.

Other directories in the home directory are:
   '''astrometry/, legacypipe, obsbot, tractor, wcslib''' -- Python code for running copilot [[BR]]
   '''data/''' -- Link to the top-level data directory, where each night is stored in subdirectories like "20151213" [[BR]]
   '''exec/''' -- Cross-mount to mosaic.kpno.noao.edu:/home/observer/exec [[BR]]
   '''products/''' -- Code checked out from the SDSS and the DESI svn repositories [[BR]]
 
Documentation for the IDL scripts can be printed from the IDL prompt with the DOC_LIBRARY command, for example:
{{{
      idl
      IDL> doc_library,'mosstat'
}}}

= Date convention =

All dates in log files are set to the local date of the '''beginning of the night'''.  For example,
any data taken during the night of December 13/14, 2015 will be written as 2015-12-13.
This is consistent with how the NOAO Science Archive timestamps and saves the raw data files.

= Are we there yet? = 

The tiles we’d be observing with MOSAIC3 in z-band are tracked in the file ~/products/mosaic3/obsstatus/mosaic-tiles_obstatus.fits with the following cuts:
{{{
  IN_DESI = 1
  DEC >= 30
  88 < RA < 301
  PASS <= 3
}}}
That’s 44,422 tiles.
This will likely be further limited further to the 41,188 tiles at DEC >= 34.
About 5% of the tiles have been removed from the list where there's a star
brighter than V=6 within 0.35 deg of the tile centers.

----
= AFTERNOON DOMEFLATS =
----
[wiki:MayallZbandLegacy/NotesforObservers/Domeflats]

----
= EXAMPLE RUN THROUGH FOR A GIVEN NIGHT =
----


= [1] Update everything =

Log into the mzls account on the mayall-idl machine.

Update the code, log files and most importantly the tile file: (if people have been working on the code and tile file during the day, this step may not be necessary)
{{{
cd ~/products/mosaic3
svn up
cd ~/products/observing
svn up
}}}

= [2] Set the paths =

The data path must be changed to point to the current night's data.
Edit the entry for MOS3_DATA ~/.bashrc file, for example if the start of the night is 13 Dec 2015 change this to
{{{
export MOS3_DATA=/mosaic3/data2/observer/20151213
}}}

= [3] Create nightly plan =

Arjun or David should have provided a set of 3 nightly plan files in JSON format that specify
pass 1, pass 2 or pass 3 tiles for timestamps through the night, which will appear after you update the code in 
[https://desi.lbl.gov/trac/browser/decam/code/mosaic3/trunk/json ~products/mosaic3/json] .
The selection of pass number while observing will depend upon the weather conditions as
described at [wiki:MayallZbandLegacy/ObservingStrategy], and will either be selected
automatically or can be forced by the user.

Copy the 3 plan files for tonight's local date to the default location used by "obsbot".
For example, for the night 11/12 Feb 2016:
{{{
   cp ~/products/mosaic3/json/2016-02-11-p1.json obsbot/pass1.json
   cp ~/products/mosaic3/json/2016-02-11-p2.json obsbot/pass2.json
   cp ~/products/mosaic3/json/2016-02-11-p3.json obsbot/pass3.json
}}}

The IDL
[https://desi.lbl.gov/trac/browser/decam/code/observing/trunk/pro/mstrategy.pro MSTRATEGY code]
is used to create the above plan files,
as described at [wiki:MayallZbandLegacy/ObservingStrategy#NightlyStrategy]

Deprecated instructions using a modified version of "nightlystrategy" is at [wiki:MayallZbandLegacy/NightlyStrategyOld]

= [4] Start up mosaic control software =

On mayall-2, double click on the MOSAIC3 icon, which brings up the MOSAIC3 Menu on the left edge of the screen. 
1. Start the camera control program by pressing the yellow "Start Cameras" button. Wait for this to finish, then dismiss the screen by typing any key as instructed
2. Start the MOSAIC3 NOCS software by pressing the blue "Start MOSAIC" button. This launches a blue xterm. Move it out of the way and watch all the windows come up
3. Rearrange the desktop as needed. 

If, for some reason, the buttons do not work, you can start up the software on a command line as follows: 
On mayall-2 open a terminal window and: 
{{{
ssh -XY observer@mosaic3
nocs start ccp (this is equivalent to the "Start Cameras" button on the MOSAIC3 Menu)
nocs start all (this is equivalent to the "Start MOSAIC" button on the MOSAIC3 Menu)
}}}
Once nocs is up and running, rearrange windows as desired, and check the status of the system by typing
the following in a nocs terminal window:
{{{
nocs status all
nocs fullstatus ccp
}}}

4. Launch CCD temperature monitor from "MOSAIC Temps" icon (CCD and Dewar temps should be around 173C and 90C respectively)
5. Launch TCS acorn monitor from "VDU" icon
6. 4MAPS monitor from "4MAPS" icon
7. Launch Truss temperature monitor from "Truss C" icon

All these icons are on the right side of the left hand screen of the mayall-2 computer display. Rearrange the busy desktop as needed ...

= [5] Set the PROP-ID and Project Info =

On the NGUI window, press the "Set Project" button and fill out the relevant information. For example, 
{{{
Principal Investigator: Arjun Dey
PIs Email Address: dey@noao.edu
Actual Observers: Tristram Shandy, Bertram Wooster
Observing Assistant: Karen Butler
OA's Email Address: 4meter@noao.edu
Proposal Identifier: 2016A-0453
Telescope System: KPNO Mayall 4m
Science Instrument: Mosaic 3
}}}
''Please ensure the Proposal Identifier number is correct!''

Then, in one of the NOCS xterm windows, type: "nocs set project"

= [6] Take a test image to ensure system is working =

Check that all is well by taking a test zero exposure. 
On the IRAF window, check the image statistics by cd-ing to the correct directory and typing "mscstat <filename>"
All rms values should be about 4-6 adu/pix; the exception is amplifier [6] which has an rms~8-10adu/pix. 
If any one amplifier shows very high noise, then execute the following commands in a nocs window:
{{{
nocs reset ccp
nocs init ccp
}}}
Then take two more zeros; the first one will be junk, but the second one should be OK.

= [7] Take dome flats and zeros =


During the afternoon (after 4pm), take dome flats with the telescope pointed at the white spot. [[BR]]

Instructions for Domeflats are here: [wiki:MayallZbandLegacy/NotesforObservers/Domeflats]

Go eat dinner [[BR]]

= [8] Just before observing = 

1. Start the may all-idl:1 VNC session.  This session should automatically start MUPTILES and MOSSTAT.  

If it doesn't, open a terminal in the VNC session and start the automated script for updating the tile file:

1a) The FITS file listing which tiles have been completed should be updated throughout the night.
This file is '''$MOS3_OBS/obstatus/mosaic-tiles_obstatus.fits'''.  The following IDL command
will monitor exposures as they are taken throughout the night, automatically updating this file.
From a terminal on the mayall-idl compute, in the mzls account, start this running:
{{{
idl
   muptiles
}}}
At the end of the night, you should check the updated tile file into the svn repository.

1b). Start the automated script for monitoring the data quality

From another terminal in the mzls account, start this running: 
{{{
idl
   mosstat_continuous
}}}
This will run mosstat on each frame as it shows up and display the results on the screen. 

= [9] Observe - start of night =

''No on-sky observations are permitted before 12 deg twilight.'' 

1. Take a zero image to ensure everything is working

2. Set the focus to some approximate value based on the Truss temperature and the formula: 
- Focus(zd) = -8400+(1.4-Ttruss)x110
where Ttruss is in deg C

3. At start of night, check telescope pointing and zero the telescope coordinates using a bright star placed on the telescope boresight (defined as the center of the mosaic3 focal plane).  When taking the image of the bright star, make sure to create an OBJECT script with NGUI (not a TEST, because MOSSTAT will ignore it), zd filter, 1 - 5 sec exposure.  Watch for MOSSTAT to process the image and report the RA, Dec offsets.  If mosstat fails, it could be because (a) the telescope is mis-pointed, (b) the telescope is out of focus (set the focus approx using the formula above and try again). Once you are able to determine offsets,  give these offsets to the OA with **opposite** sign.  That is, if mosstat reads "RA,Dec offsets = -15.34, 18.32", then you need to provide the OA with the offsets of +15,-18 to zero the telescope coordinates. You can take another OBJECT frame if you want to check to make sure the offset went in the right direction.

4. Move to the beginning of the first MzLS tile position.  What position is that, you ask?  One way to find out is to run mosbot.py (see step 5 below), and look for RA, DEC in expose-1.sh in the exec/mosbot/ directory.  Tell these coordinates to the OA to move the telescope. Another way is to look at teh first entry in the ~/obsbot/pass1.json file on the mayall-idl computer - remember, it is in degrees, so you will have to divide the RA by 15. Getting within a degree is close enough, so no need to be super precise.

5. Focus the telescope
 - create a focus script using the NGUI: exposure time 5 or 10 seconds, zd filter, -100 micron focus steps, click Midpoint to *on*, 9 exposures.
 - run the focus script from the /home/observer/exec directory 
 - analyze the focus image using mscstarfocus 
    - edit the /data3/observer/mscfoc.cl script to correct the name of the image that needs to be analyzed 
    - run 'cl < /data3/observer/mscfoc.cl' in iraf window in data directory.
    - mark about 10 stars around the image; to get a quick idea if you have covered the right focus range mark "g" on the top star in a sequence which will pop up a graph that you will need to type "q" to get out of; mark the top star in each remaining sequence using "m"; "q" to quit; "d" to delete bad points; "q" to quit, then will get best focus value. 
 - log the Truss temperature 
 - set the telescope focus in the Configuration Monitor: Enter the value in Pedestal focus, **hit return**, and then hit Apply.  If you forget the return, it will do nothing. 
 - focus the guiders; this way you can use the guider images to monitor focus drifts

**[wiki:MayallZbandLegacy/NotesforObservers/FocusInfo Example of a focus sequence]

=== Observe - all night long! ===

6. From the mzls@mayall-idl window, generate the top-level observing script (tonight.sh) assuming that we have three plan files named pass1.json, etc.
{{{
   cd ~/obsbot
   python mosbot.py pass1.json pass2.json pass3.json
}}}
This will start the `mosbot` script, which will watch the $MOS3_DATA directory for new images, analyze them, and update FUTURE exposure scripts,
choosing the pass number and setting the exposure time.

See [wiki:MayallZbandLegacy/NotesforObservers/MosBot]

Note that if you need to escape the script at some point during the night (see below), you will need to CTRL-C to stop mosbot.py and **restart** it when you are ready to start up again.  If you forget this, you will be observing tiles that you already observed earlier in the night.

7. From the observer@mayall-3 xterm window, start taking exposures using the top-level observing script (tonight.sh):
{{{
   cd ~/exec/mosbot
   ./tonight.sh
}}}

8. Start running "copilot" in the mayall-idl VNC window. Copilot keeps a beautiful running plot of observing conditions.  From any mzls@mayall-idl window:
{{{
   cd ~/obsbot
   python copilot.py
}}}

Whenever a new image is detected in the data directory (as defined by $MOS3_DATA), a new image is generated as ~/obsbot/recent.png .  That image can also be displayed by anyone that can see this directory.  It could be displayed with continuous updating with
{{{
   eog ~/obsbot/recent.png
}}}
However, running copilot automatically displays this.

See [wiki:MayallZbandLegacy/NotesforObservers/Copilot]

8. Monitor focus by checking the image quality on each frame. Keep track of the truss temperature variation and use the information to modify  the focus as needed. Note that the focus may not respond quickly to changes in temperature, so monitor the images carefully before adjusting focus.  The Mayall has astigmatism, so one can sometimes tell from the shape of the images which way to move the focus.  To stop and do a focus sequence:
- Create a file to tell obsbot to quit
{{{
   touch ~/exec/mosbot/quit
}}}
- Wait for the current exposure to complete (at which point the above file is automatically removed)
- Run a focus sequence (see link below for instructions)
  - [wiki:MayallZbandLegacy/NotesforObservers/FocusInfo Example of a focus sequence]
- Re-start the observing as described above.  **IMPORTANT** make sure to re-run `mosbot.py` so that you don’t repeat exposures from the beginning of the night!

Keep an eye on the CCD and dewar temps (should be around 173C and 90C respectively)

OK - you are off and running! Congratulations!!!

=== If you have problems ... ===

See **[wiki:MayallZbandLegacy/NotesforObservers/Problems]

=== Checking the Sky Brightness, Seeing and Transparency  ===

The "copilot" python script mostly deprecates this, but we have IDL code for analyzing the images.

From an IDL prompt, use the MOSSTAT routine to analyze the latest image on disk:
{{{
   IDL> mosstat
}}}
There are keyword options that allow you to choose different exposure numbers or CCDs
within that exposure.  For example, to analyze chip 'im16' of the exposure number 12345, type:
{{{
   IDL> mosstat, 12345, ext='im16'
}}}
The full documentation can be seen with:
{{{
   IDL> doc_library,'mosstat'
}}}
To just have mosstat run continuously whenever each image appears, use
{{{
   IDL> mosstat_continuous
}}}


=== Please write useful human logs ===

Keep a log about weather conditions, which pass you observed,
and telescope problems.  Follow the example on the pages at [wiki:MayallZbandLegacy/ObservingLogs] .

Catastrophically bad frames (such as saturated frames, or where the telescope moved) are recorded and svn-checked-in to
[https://desi.lbl.gov/trac/browser/decam/code/mosaic3/trunk/obstatus/bad_expid.txt ~products/mosaic3/obstatus/bad_expid.txt] .

= [10] End of night =

No on-sky observations are permitted with MOSAIC3 before 10 deg twilight or after 10 deg dawn. No twilight flats are allowed. 

Once you exit the observing script (using "touch quit" or waiting for it to end), take a zero image. This ensures that the 
dark slide is put in place and the instrument is ready for shut down.

=== Shut down the software ===

1. Ensure that the "Shutter" and "ready" are both in the "Dark" position on the MCCD gui
2. Press the red "Stop MOSAIC" button on the MOSAIC Menu GUI. Wait for this to finish. 
3. Then press the yellow "Stop Cameras" button on the MOSAIC Menu GUI.

If the buttons do now work, then go to one of the NOCS xterm windows and type:
{{{
     nocs stop all (equivalent to pressing the "Stop MOSAIC" button)
}}}
Once nocs is shut down, type
{{{
     nocs stop ccp (equivalent to pressing the "Stop Cameras" button)
}}}
Once this is done, type 
{{{
     nocs status all
     nocs fullstatus ccp
}}}
and make sure everything is shut down. Only when this is done, is it safe for the OAs to put on the lights in the dome. 


=== Check in the updated tile file and updated bad exposure file ===

At the end of the night, you should check the updated tile file into the svn repository. On the 
mayall-idl computer: 
{{{
   cd $MOS3_OBS/obstatus
   svn -m "observing update of tile file and bad exposure list” commit
}}}
If this does not work for some reason, try 
{{{
  cd $MOS3_OBS/obstatus 
  svn -m "observing update of tile file and bad exposure list” commit -username <YOUR USERNAME>
}}}
and enter your wiki log in and password when prompted.

You can also check in the files individually as

{{{
  cd ~/products/mosaic3/obstatus
  svn -m "bad exp update" commit bad_expid.txt
  svn -m "obstatus update" commit mosaic-tiles_obstatus.fits
}}}


=== Create the Almanac and Almanac plot files  ===

At the end of the night, create the Almanac files and check them into svn. Do this using 
a terminal window on the mayall-idl VNC: 
{{{ 
- cd products/mosaic3/logs
- idl
- almanac,<first exp num>
- plotalmanac,'Almanac_date.fits',ps='plot_Almanac_date'
- exit
- svn add *Almanac_date*
- svn commit *Almanac*
}}}

For example, for the night of March 26/27, 2015, this is done with:
{{{
cd products/mosaic3/logs
idl
   almanac, 424978, 425143
   exit
svn add Almanac_2015-03-26.*
svn commit Almanac_2015-03-26.*
}}}

There are actually two versions of this file, one that is an ASCII file (with .txt extension)
and one that is a FITS file (with .fits extension).

=== Create the Coverage and Summary files for the Night  ===

In a mayall-idl window on the VNC: 
{{{
- idl
- IDL> moscoverage_plot
}}}
In a mayall-idl window:
{{{
- python copilot.py --night
- mv night.png <date>.png (e.g., 2016-02-11.png)
}}}
You can then mail these plots out to mayall-obs@desi.lbl.gov to let your collaborators know how the night went.

=== Thank the OA and go to bed ===
[[BackLinks]]