= OBSERVING INSTRUCTIONS FOR MzLS =

These are instructions for using the MzLS planning scripts for the nightly program. To see the Mosaic observing manual, go here: http://www.noao.edu/kpno/mosaic/manual.

[[PageOutline]]

Log on to the ?? account on the observer2 machine:
    Username : ?? [[BR]]
    Password : Ask Arjun, David, or previous observers [[BR]][[BR]]

Two environment variables define the locations of data and code:
   '''$??_DATA=/home4/images/fits/2014B-0404''' -- This is where our raw data files are written [[BR]]
   '''$??_OBS=$HOME/observing''' -- This contains the observing product with code and versioned log files. [[BR]]
To set these, type "setenv ??_DATA /home4/images/fits/2014B-0404"

Other directories in the home directory:
   '''idl/''' -- IDL products that we make use of, including the idlutils product and the Goodard libraries (a more current version than in idlutils, that does funpack) [[BR]]
   '''!ObservingStrategy/''' -- Outputs from Anna Patej's observing strategy code [[BR]]
   '''OBSOLETE/''' -- Obsolete code, data files, log files, etc. [[BR]]
   '''ps1/''' -- Doug Finkbeiner's Pan-STARRS1 catalogs used by the IDL DECSTAT routine for computing astrometric offsets and photometry [[BR]]
   '''rawdata@''' -- Convenient link to the raw data directory $??_DATA [[BR]]

Documentation for the IDL scripts can be printed from the IDL prompt with the DOC_LIBRARY command, for example:
{{{
      IDL> doc_library,'decstat'
}}}


= Date convention =

All dates in log files are set to the local date of the '''beginning of the night'''.  For example,
any data taken during the night of August 17/18, 2014 will be written as 2014-08-17.
This is consistent with how the NOAO Science Archive timestamps and saves the raw data files.
(This convention has changed from what we did at the start of the survey, but Greg has made
everything consistent.)


= Visualizing the available tiles (not required) =

The IDL routine WHICHTILES_TONIGHT can be used to visualize the available tiles for the night:
{{{
IDL> whichtiles_tonight
SYNTAX: whichtiles_tonight,utdate,goodtiles,band,moonlimit=moonlimit,amlimit=amlimit
   Inputs:
       utdate - string - the UT date for the ''midpoint'' of the night (e.g., "2014-08-12")
       band - string - the observation band preferred (e.g., "g", "z", or "r") - default is "z"
   Keywords:
       moonlimit = minimum acceptable angle from the moon (default=20 deg)
       amlimit = maximum acceptable airmass (default=2.2)
       ps - string - name of an output postscript file; if not provided, defaults to a screen plot
   Output: 
        goodtiles = boolean 1 or 0 for each tile in decam-tiles.fits

To execute the program for a given night:
IDL> whichtiles_tonight,'2014-08-15',good,'z',ps='plan_zband_aug15ut'
IDL> whichtiles_tonight,'2014-08-15',good,'r',moonlimit=50,ps='plan_rband_aug15ut'
IDL> whichtiles_tonight,'2014-08-15',good,'g',moonlimit=100,amlimit=1.8,ps='plan_gband_aug15ut'
}}}

These will produce encapsulated postscript files that are useful for visualizing the 
possible fields for tonight.

= Official Nightly Strategy =
'''Update 4/9/2015'''
Python is now available to the decals user. Do the following to update the current NMSL scripts at any time.

        - login to the home directory of ??
        - cd observing
	- python2.7 py/nightlystrategy.py -t obstatus/mosaic-tiles_obstatus.fits -d 2015-04-08 -p 1 -pass 1 --moon-sep-limit 40 --start-date 2015-04-09 --start-time 02:00:00 -seez 1.15 -sbz 18.45 --dir /home/observer/exec/<dir_name>

Assuming that uptiles is running in an IDL window on ??, this will always use the most up-to-date version of the mosaic-tiles_obstatus.fits file. Once this finishes running (usually a couple of minutes), cd to <dir_name>:

	> ./mosaic_<UTdate>_plan.sh

= Instructions and details for nightlystrategy.py = 
'''written by A. Patej'''  

nightlystrategy.py is a Python script that can generate a nightly observing plan. It requires several Python modules:
    - numpy
    - pylab (Matplotlib)
    - ephem (Pyephem)
    - pyfits

=== Selection of Pass 1, 2, or 3 ===

It is '''very important''' to adhere to the strategy of when to observe pass 1, 2, or 3 tiles,
in particular that pass 1 tiles are always in photometric good-seeing conditions.
The overall strategy is outlined at [[wiki:MayallZbandLegacy/ObservingStrategy]]].

The strategy is as follows:
 * **Pass 1:** Photometric and seeing < 1.3 arc sec
 * **Pass 2:** Photometric or seeing < 1.3 arc sec, or all Pass 1 tiles completed
 * **Pass 3:** Unphotometric and seeing > 1.3 arc sec, or all Pass 1 and Pass 2 tiles completed
These need not be observed in any particular order, e.g. pass 3 observations can certainly
occur before pass 1.

If conditions change, then you should definitely change to a different pass.
For example, if clouds begin to roll in, you should stop any pass 1 observations right away!

=== Running the script ===

At a minimum, the nightlystrategy.py script can be run with these inputs:

    -t *name of file with tile centers* [[BR]]
    -d *UT date at ''start'' of night in format yyyy-mm-dd* [[BR]]
    -p *portion of night (1.0, 0.5 (first half), -0.5 (second half))* [[BR]]
    -pass *desired pass - input 1, 2, or 3* [[BR]]
    --dir *name of the dir where the output scripts are located* 
For instance, to generate a pass 3 plan for the entire night of August 14, 2014, I could run:
{{{
python ./nightlystrategy.py -t moasic-tiles_obstatus.fits -d 2014-08-14 -p 1.0 -pass 3 --dir /home/observer/exec/mzls-utdate/
}}}
    
Additional optional arguments:
1. Observing conditions: [[BR]]
   -sb(f) *sky brightness in filter f; replace (f) = g, r, or z* [[BR]]
   -see(f) *seeing in filter f; replace (f) = g, r, or z* [[BR]]
   -transparency *value of the transparency* [[BR]]

2. Observing constraints: [[BR]]
   --airmass-limit *maximum allowed airmass, default is 2.3* [[BR]]
   --moon-sep-limit *minimum allowed moon separation, default is 50.0 deg* [[BR]]
   --moon-alt-limit *moon altitude required for switching between z and g,r* [[BR]]

3. Custom Scheduling: [[BR]]
  It's possible to create a plan for a specific block of time. To do so, use the following options:[[BR]]
  --start-date *Date of start of observations (UTC) in format yyyy-mm-dd; overrides the default start of night, which is calculated using --date and --portion. Requires --start-time to also be set* [[BR]]
  --start-time *Time of start of observations (UTC) in format hh:mm:ss; overrides the default start of night, which is calculated using --date and --portion. Requires --start-date to also be set*[[BR]]
  --end-date *Date of end of observations (UTC) in format yyyy-mm-dd; overrides the default end of night, which is calculated using --date and --portion. Requires --end-time to also be set*[[BR]]
  --end-time *Time of end of observations (UTC) in format hh:mm:ss; overrides the default end of night, which is calculated using --date and --portion. Requires --end-date to also be set*[[BR]]


More complicated example using these options: [[BR]]
Assuming 18.2 as z-band sky brightness, 1.2" seeing in z and 1.0" seeing in r, photometric conditions (pass 1) and allowing a moon separation of 40 deg and including DESI tiles:
{{{
python ./nightlystrategy.py -t mosaic-tiles_obstatus.fits -d 2015-03-24 -p 1.0 -pass 1 -sbz 18.2 -seer 1.0 -seez 1.2 --moon-sep-limit 40.0 --expand
}}}

Example using custom scheduling: [[BR]]
To generate a plan for just two hours on the night that starts on 2015-04-06 between 05:00 and 07:00 UTC:
{{{
python ./nightlystrategy.py -t mosaic-tiles_obstatus.fits -d 2015-04-06 -p 1.0 -pass 1 --start-date 2015-04-07 --start-time 05:00:00 --end-date 2015-04-07 --end-time 07:00:00 --moon-sep-limit 40.0
}}}

=== Outputs ===

This script will yield 3 output data files:

    1. text file called "strategy_mm-dd.txt", with information about all the pointings for the night (exp time, filter, approximately when they should be run, airmass, etc). If there's a late start or gap in observing due weather, etc you can look in this file and find the tile corresponding to your revised start time and remove all the ones prior to it in the Exposure Queue. It is a good idea to check this file before accepting the plan.
    
    2. JSON script called "decals_mm-dd_plan.json" for the night corresponding to the plan in this text file as well as some plots of what has been currently observed on previous nights and what you will be observing given the plan for the night, which are saved as a pdf, 'plot_plan_mm-dd.pdf' (gray = z, red = r, green = g)

   3. A FITS file called "decam-tiles_obstatus_update_yyyy-mm-dd.fits" that is a temporary updated decam obstatus file that assumes all planned observations were completed. This should not be used during the run; instead use the updated tiles file obtained from running the idl program, uptiles (see section on "Creating the Updated Observing Status File" below).


= Updating the "observed tiles" file =

The FITS file listing which tiles have been completed should be updated throughout the night.
This file is '''$DECALS_OBS/obstatus/decam-tiles_obstatus.fits'''.  The following IDL command
will monitor exposures as they are taken throughout the night, automatically updating this file:
{{{
   IDL> uptiles
}}}

At the end of the night, you should check the updated tile file into the svn repository:
{{{
   cd $DECALS_OBS/obstatus
   svn commit obstatus/decam-tiles_obstatus.fits --username <YOUR-NAME>
}}}

= Use of Steve Kent's tools for monitoring observing =

During observing, please check frames as they read out. To use kenttools remotely, you must be 
logged in as ''DECamObserver'' rather than as ''declsp''. To load kenttools log onto observer2 and 
type "observer" at the unix shell prompt. This will load up an environment in which you can run the 
kenttools commands.

prompt>

The following commands from kenttools are 
very useful: 
       center - find the RA and DEC offsets[[BR]]
       seeingall - measures the seeing[[BR]]
       psc - provides a heap of useful data[[BR]]
       bigload - display the last image[[BR]]
       load - display a single CCD from the last image[[BR]]
       inv - show an inventory of all the fits files taken during the night[[BR]]

More details about kenttools can be found here:  https://cdcvs.fnal.gov/redmine/projects/desops/wiki/Introduction_to_kentools 

In addition to kenttools, there is an even more useful version of inv in godb called "qcInv". To use
this instead, log onto ?? as ?? " and type "godb" and then "qcInv" at the 
''pgtcl>'' prompt.


= Checking the Sky Brightness, Seeing and Transparency  =

The observing strategy files take as input these environmental conditions for
setting the exposure times to achieve the required MzLS survey depth.

From an IDL prompt, use the DECSTAT routine to analyze the latest image on disk:
{{{
   IDL> decstat
}}}
There are keyword options that allow you to choose different exposure numbers or CCDs
within that exposure.  For example, to analyze chip 'S4' of the exposure number 123456, type:
{{{
   IDL> decstat, 123456, ext='S4'
}}}
The full documentation can be seen with:
{{{
   IDL> doc_library,'decstat'
}}}

If the telescope pointing has gone off by more than about 30 arcsec (which happens!), then
this routine will likely fail to match stars.  In that case, the kenttools routine '''center''' described above
should be used to find the telescope offset.  Stop observing and ask the telescope operator to apply
a pointing offset.

One can also check the observing progress using the almanac command: 
  IDL> almanac, 10001, /noprint
will print out a summary of all the frames from 10001 to the present in a nice tabular form (widen the window). 
This is useful for checking how the seeing and sky brightness have varied since frame 10001, and whether 
one should create and upload a new JSON observing script. The /noprint just prevents the decstat output 
for every frame; if you want to see that, then don't use the /noprint keyword. 

= Creating an almanac file with the observations for the night =

At the end of the night, an almanac file should be generated for the night and checked into svn.
{{{
cd $??_OBS/observing/logs
idl
   almanac, <START-EXPOSURE>, <END-EXPOSURE>
   exit
svn add Almanac*
svn commit Almanac*
}}}
For example, for the night of March 26/27, 2015, this is done with:
{{{
cd $DECALS_OBS/observing/logs
idl
   almanac, 424978, 425143
   exit
svn add Almanac_2015-03-26.*
svn commit Almanac_2015-03-26.*
}}}

There are actually two versions of this file, one that is an ASCII file (with .txt extension)
and one that is a FITS file (with .fits extension).

= Writing the human logs =

Please write about what happened during your night: weather conditions, which pass you observed,
and telescope problems.  Follow the example on the pages at [wiki:DecamLegacy/ObservingLogs].

Catastrophically bad frames (such as saturated frames, or where the telescope moved) are recorded
and svn-checked-in to  '''observing/logs/bad_expid.txt'''.

Historically, we also made some files in $DECALS_OBS/logs/logfile-$DATE.txt that are
output from the kenttools "inv" command.  These are probably not really needed any longer.

= In case of technical difficulties =

We have in the past encountered technical difficulties that have restricted where we can observe to a specific range in RA/Dec. For instance, in April 2015, we had a night during which the windscreen was stuck, requiring us to stick to dec < -5 deg tiles. 

If this happens during your run, you have several options:[[BR]]
1. First, try running nightlystrategy.py with an appropriate --airmass-limit and/or --moon-sep-limit to try to keep to the required range of tiles. [[BR]]
2. If you run out of tiles with these options, add the --expand option, which allows the selection of DESI tiles in addition to SDSS tiles, in addition to any airmass-limit you need. 
3. '''Shorten''' your observing period (using --start-date,--start-time and --end-date,--end-time)  such that it's scheduling fewer tiles.
4. Increase the assumed seeing such that it schedules longer exposures.

If these options don't work, you will have to modify our observing footprint, which is encoded in decam-tiles_obstatus.fits. This file has, among other things, three columns named 'IN_SDSS', 'IN_DESI', and 'IN_DES'. For each tile the value in these columns is either 1 or 0.

We define our fiducial footprint as 'IN_SDSS' = 1 & 'IN_DES' = 0. If you allow the selection of DESI tiles (using --expand), then the footprint is defined by ( 'IN_SDSS' = 1 & 'IN_DES' = 0 ) or ( 'IN_DESI' = 1 & 'IN_DES' = 0). If you have no tiles to observe that are within the region defined this way, you need to modify the tiles file.

In this case, please do the following:
1. Make a copy of decam-tiles_obstatus.fits (call it something like temp_decam-tiles_yyyy-mm-dd.fits)
2. Open temp_decam-tiles_yyyy-mm-dd.fits using your favorite fits file editor (python/pyfits, idl, etc) and modify the file to set 'IN_SDSS' = 0 & 'IN_DESI' = 0 for all tiles that are outside the range allowed by the telescope that night. Then set 'IN_SDSS' = 1 & 'IN_DESI' = 1 for tiles that satisfy your allowed RA/dec range and also some possible quality constraints. For instance, in April 2015, David suggested adding the following tiles to satisfy dec < -5:
{{{
a) IN_DESI = 0 
b) EBV_MED < 0.1 
c) 120 < RA < 270 
d) DEC > -19 
}}}
3. Use the modified temp_decam-tiles_yyyy-mm-dd.fits as your -t argument in nightlystrategy. For instance,
{{{
python ./nightlystrategy.py -t temp_decam-tiles_yyyy-mm-dd.fits -d yyyy-mm-dd -p 1.0 -pass 3 --expand
}}}

4. Set two versions of uptiles running, one to update the official "decam-tiles_obstatus.fits" file, so that we have an official record of the tiles being observed, the other to update 'temp_decam-tiles_yyyy-mm-dd.fits,' which you can use to recalculate the JSONs if conditions change. You can run uptiles as follows:
{{{
uptiles
uptiles, tfile='temp_decam-tiles_yyyy-mm-dd.fits'
}}}

5. At the end of the night, make sure check in decam-tiles_obstatus.fits to svn; you can leave 'temp_decam-tiles_yyyy-mm-dd.fits' on observer2, or place it in a temp folder on svn. 


= Example run-through for a given night =

Log into the declsp account on the observer2 machine.
{{{
user@anyshell>ssh declsp@observer2
}}}

=== Update everything ===

Update the code, log files and most importantly the tile file:
{{{
cd ~/observing
svn update
}}}

=== Create nightly plan ===

Create the nightly plan from the available tiles.  This will read the tile file to see which ones are already completed.
{{{
cd ~/observing
python2.7 py/nightlystrategy.py -t obstatus/decam-tiles_obstatus.fits -d *UT date at ''start'' of night* -p *portion of night* -pass *pass number*
}}}

The output JSON file is then loaded into SISPI. The DECaLS JSONS that will be loaded into SISPI should be placed in observer2.ctio.noao.edu:/usr/remote/user/DECamObserver/ExposureScripts/User_scripts/BigProjects/DECaLS/

In the observer console, select the tab "Exposure Control" and press "Load Exposure Script." Select your JSON and press "submit." The exposures will then be appended to the bottom of the Exposure Queue.

You can manually modify the exposure queue. Make sure the padlock icon is unlocked if you wish to make changes. You can select a queued exposure by clicking on it; you can press "delete" to delete it, or use your cursor to click and drag it to a new position. To select a range of exposures, click on the first one, then shift+click on the last one.

=== Start the automated script for updating the tile file ===

From a terminal in the declsp account, start this running:
{{{
idl
   uptiles
}}}

From another terminal in the declsp account, start this running: 
{{{
idl
   decstat_continuous
}}}
This will run decstat on each frame as it shows up and display the results on the screen. 

=== Start a window for running kenttools ===

Kenttools can only be run as the "DECamObserver", and not from our "declsp" account.  (It was possible to do directly from
our account before March 2015, so something changed.)

Open a new window logged in as ''DECamObserver'' on observer2:
{{{
observer> observer
prompt> inv
}}}

=== Create the Almanac and Almanac plot files  ===

At the end of the night, create the Almanac files and check them into svn.
{{{ 
- cd observing/logs
- idl
- almanac,<first exp num>
- plotalmanac,'Almanac_date.fits',ps='plot_Almanac_date'
- exit
- svn add *Almanac_date*
- svn commit
}}}

=== Create a log file ===
{{{
- cd logs/
- observer (kent tootls on observer1)
- inv
- cut and paste the output from the first exposure number to the bottom 
  into a file log_date.txt (on observer2)
- use this file to determine the number of g, r, and z images for the 
  summary table on the wiki: grep " g " log_date.txt | wc
}}}

=== Generating a Coverage Map ===

The code pro/coverage_plot.pro will generate coverage maps for a given date range using the info in decam-tiles_obstatus.fits.  To run it, login as declsp@observer 2 and start IDL:
{{{
IDL>.com coverage_plot
}}}
To produce the plot for a single night use, for example:
{{{
IDL> coverage_plot, datemin='2015-04-07', datemax='2015-04-07'
}}}
To produce a plot for a range of nights use:
{{{
IDL> coverage_plot, datemin='date-start', datemax='date-end'
}}}
To produce a plot for all nights to date use:
{{{
IDL> coverage_plot
}}}
All plots will be put in $DECALS_OBS/obstatus/Plots.  The default is to output a PDF but there are flags to output other formats.  

The code pro/coverage_plot_alldates.pro will produce a set of cumulative coverage plots (one per day) over a date range.  Within IDL when logged in as declsp@observer2 you enter:
{{{
IDL> .com coverage_plot_alldates
IDL> coverage_plot_alldates, datemin='date-start', datemax='date-end'
}}}

All plots, consecutively numbered, will be put in $DECALS_OBS/obstatus/Plots/PDFs and can be combined into a movie, e.g with the free graphicconverter [http://www.lemkesoft.de/en/image-editing-slideshow-browser-batch-conversion-metadata-and-more-on-your-mac/] program on Macs.  

Ben Weaver would prefer that we do not upload any plots to SVN but rather just the code.  So feel free to mail plots around but there is no need to check them in.

[[BackLinks]]