= OBSERVING INSTRUCTIONS FOR MzLS (PROPID 2016A-0453) = '''Still need to incorporate the following instructions:''' - '''Running the guiding loop -- although we typically do not guide on the Mayall for these 1-2 min exposures''' These are instructions for observing for MzLS. [[PageOutline]] ---- = VERY VERY IMPORTANT = == 1. DO NOT OVEREXPOSE MOSAIC CCDs to LIGHT! == Prolonged exposure can damage and even destroy the devices. The following rules are in effect: 1. Operation on sky is restricted to between [wiki:MayallZbandLegacy/NotesforObservers/TenDegTwilight 10 deg twilights]. 2. Operation is restricted to when OAs or other NOAO technical/scientific staff are present 3. The cameras and MOSAIC software should be shut down when observing / NOAO personnel are not present 4. Sky values should be kept < 40,000 adu (<30,000 adu is preferable). == 2. PLEASE TALK TO THE OAs ABOUT SAFETY AT THE TELESCOPE == 0. Please read the [wiki:MayallZbandLegacy/NotesforObservers/Safety Safety Rules] carefully and FOLLOW THEM! 1. Do not leave the control room at night without telling the OA 2. Do not go into the dome without the OA's permission 3. Do not go exploring by yourself 4. Carry a flashlight and radio at night ---- = Read the Manual for MOSAIC operations = To see the Mosaic1.1 manual, go here: http://www.noao.edu/kpno/mosaic/manual [[BR]] To see the Mosaic3 manual go here: http://www.noao.edu/kpno/mosaic/manual/Mosaic3Manual.pdf [[BR]] = Some Preliminaries = Who is your OA for tonight? See [http://www.noao.edu/perl/Calcium?Op=ShowIt&CookieParams=1&CalendarName=oa&Amount=Month&NavType=Absolute&Type=Block the OA Schedule] Observing can only be done from the Mayall control room or from within the KPNO/NOAO network using a VPN. The telescope and low-level instrument control runs on observer@mayall-2.kpno.noao.edu . Log on to the mayall-2 computer: [[BR]] Username : 4meter [[BR]] Password : [[BR]] To run MOSAIC3, use the MOSAIC3 Menu GUI. Or you can also log into the mosaic3 computer from mayall-2:[[BR]] ssh -XY observer mosaic3[[BR]] and use command-line nocs commands. [[BR]] '''PLEASE BE CAREFUL ON THIS COMPUTER - IT RUNS THE CONTROLLERS AND THE INSTRUMENT!!!''' Our observing scripts and data reduction runs on mzls@mayall-idl.kpno.noao.edu : Username : mzls [[BR]] Password : [[BR]] These environment variables define the locations of data and code: '''$HOME=/home/mzls'''[[BR]] '''$MOS3_DATA=''' -- This is where our raw data files are written. Should be something like /mosaic3/data2/observer/20160202. This is a network mounted disk on the machine mosaic3. [[BR]] '''$MOS3_OBS=$HOME/products/mosaic3''' -- This contains the observing product with code and versioned log files. [[BR]] '''$PS1CAT_DIR=$HOME/ps1/chunks-qz-star-v2''' -- Pan-STARRS1 catalogs used by the IDL DECSTAT routine for computing astrometric offsets and photometry [[BR]] These path names are set in the .bashrc file, or can be set for example with "export MOS3_DATA=/mosaic3/data2/observer". The $MOS3_DATA directory must be updated to point to the current night's data. Other directories in the home directory are: '''astrometry/, legacypipe, obsbot, tractor, wcslib''' -- Python code for running copilot [[BR]] '''data/''' -- Link to the top-level data directory, where each night is stored in subdirectories like "20151213" [[BR]] '''exec/''' -- Cross-mount to mosaic.kpno.noao.edu:/home/observer/exec [[BR]] '''products/''' -- Code checked out from the SDSS and the DESI svn repositories [[BR]] Documentation for the IDL scripts can be printed from the IDL prompt with the DOC_LIBRARY command, for example: {{{ idl IDL> doc_library,'mosstat' }}} = Date convention = All dates in log files are set to the local date of the '''beginning of the night'''. For example, any data taken during the night of December 13/14, 2015 will be written as 2015-12-13. This is consistent with how the NOAO Science Archive timestamps and saves the raw data files. = Are we there yet? = The tiles we are observing with MOSAIC3 in z-band are tracked in the file ~/products/mosaic3/obsstatus/mosaic-tiles_obstatus.fits with the following cuts: {{{ IN_DESI = 1 DEC >= 30 88 < RA < 301 PASS <= 3 }}} That’s 44,422 tiles. This will likely be further limited further to the 41,188 tiles at DEC >= 34. About 5% of the tiles have been removed from the list where there's a star brighter than V=6 within 0.35 deg of the tile centers. The tiles are broken into three passes. Each pass, pass1, pass2, and pass3, covers the basic footprint and each is offset optimally from the others. The tile are thus now fixed or defined on the sky for the duration of the survey. By definition: **pass1** is the high quality, photometric coverage. To the greatest extent possible, we want to ensure pass one tiles are the best. As such, pass1 should be executed on photometric nights with good seeing (< 1.3"). The observer should not start pass1 if its a not a true photometric night or if 3-4 hour contiguous blocks are not stable and clear. The robot observing program will choose pass1 for seeing < 1.25", transparency > 90% and sky brightness not worse than 0.25 mag brighter than the fiducial. But the observer should really be setting pass1 based on whether its a very stable photometric night from beginning to end. Forcing the pass is described here: [wiki: MosBot]. **pass2** is the next best pass. Seeing should be < 1.3" or the weather is photometric (but seeing worse than 1.3"). Pass2 and 3 may alternate throughout a given night and the mosbot observing robot will do this automatically based on seeing, transparency, and sky brightness. Pass2 could also be done if no pass1 tiles are available. The robot will stick with pass2 when conditions are a little worse than this. Specifically seeing < 2" and trans > 70%. Observers can force pass3 if they think the conditions are poor, even though the robot might say pass2. This is especially encouraged if conditions are variable on short timescales and the robot is moving frequently between pass2 and pass3. **pass3** is the filler or worst pass executed when conditions are bad. Seeing > 1.3'' and its not photometric (trans < 90%). Pass3 may be done in good conditions when no pass1 or pass2 tiles are available. ---- = READ THIS IN PREP FOR OBSERVING = * [wiki:MayallZbandLegacy/NotesforObservers/MosBot mosbot notes] * [wiki:MayallZbandLegacy/NotesforObservers/Copilot copilot notes] ---- = AFTERNOON DOMEFLATS = ---- [wiki:MayallZbandLegacy/NotesforObservers/Domeflats] ---- = EXAMPLE RUN THROUGH FOR A GIVEN NIGHT = ---- = [1] Update everything = Log into the mzls account on the mayall-idl machine. Update the code, log files and most importantly the tile file: (if people have been working on the code and tile file during the day, this step may not be necessary) {{{ cd ~/products/mosaic3 svn up cd ~/products/observing svn up }}} = [2] Set the paths = The data path must be changed to point to the current night's data. Edit the entry for MOS3_DATA ~/.bashrc file, for example if the start of the night is 13 Dec 2015 change this to {{{ export MOS3_DATA=/mosaic3/data2/observer/20151213 }}} Don't forget to '''''source'' the file''' if you plan to continue to use this terminal. = [3] Create nightly plan = There are 3 nightly plan files, one with a list of tiles to observe for each of the 3 passes. These files are in JSON format, and specify the pass 1, pass 2 or pass 3 tiles to be observed at each timestamp through the night. The selection of pass number while observing will depend upon the weather conditions as described at [wiki:MayallZbandLegacy/ObservingStrategy], and will either be selected automatically or can be forced by the user. The IDL [https://desi.lbl.gov/trac/browser/decam/code/observing/trunk/pro/mstrategy.pro MSTRATEGY code] is used to create the above plan files, as described at [wiki:MayallZbandLegacy/ObservingStrategy#NightlyStrategy] Arjun or David may have already provided a set of plan files for the night and posted a note to the mayall-obs e-mail list saying so. These files would be checked into the "mosaic3" product here: {{{ ~/products/mosaic3/json/-p1.json ~/products/mosaic3/json/-p2.json ~/products/mosaic3/json/-p3.json }}} and copied into the observing directory where they are read by [wiki:MayallZbandLegacy/NotesforObservers/MosBot mosbot] here: {{{ ~/obsbot/pass1.json ~/obsbot/pass2.json ~/obsbot/pass3.json }}} If the plan files have not been generated for the night, please do so as follows: {{{ idl mstrategy, copydir='/home/mzls/obsbot' exit }}} Then check in the plan files to the "mosaic3" product as follows: {{{ cd ~/products/mosaic3/json svn add *.json svn commit *.json -m "plan files for tonight" }}} There are plot files in that directory too. We don't check those in, but it's informative to post the planned coverage map ~/products/mosaic3/json/.pdf to mayall-obs@desi.lbl.gov . Deprecated instructions using a modified version of "nightlystrategy" is at [wiki:MayallZbandLegacy/NightlyStrategyOld] = [4] Start up mosaic control software = On mayall-2, double click on the MOSAIC3 icon, which brings up the MOSAIC3 Menu on the left edge of the screen. 1. Start the camera control program by pressing the yellow "Start Cameras" button. Wait for this to finish, then dismiss the screen by typing any key as instructed 2. Start the MOSAIC3 NOCS software by pressing the blue "Start MOSAIC" button. This launches a blue xterm. Move it out of the way and watch all the windows come up 3. Rearrange the desktop as needed. If, for some reason, the buttons do not work, you can start up the software on a command line as follows: On mayall-2 open a terminal window and: {{{ ssh -XY observer@mosaic3 nocs start ccp (this is equivalent to the "Start Cameras" button on the MOSAIC3 Menu) nocs start all (this is equivalent to the "Start MOSAIC" button on the MOSAIC3 Menu) }}} Once nocs is up and running, rearrange windows as desired, and check the status of the system by typing the following in a nocs terminal window: {{{ nocs status all nocs fullstatus ccp }}} If you want to know what these commands actually do, see here: [wiki:MayallZbandLegacy/NotesforObservers/MOSAICGUI_Notes] 4. Launch CCD temperature monitor from "MOSAIC Temps" icon (CCD and Dewar temps should be around 173C and 90C respectively) 5. Launch TCS acorn monitor from "VDU" icon 6. 4MAPS monitor from "4MAPS" icon 7. Launch Truss temperature monitor from "Truss C" icon All these icons are on the right side of the left hand screen of the mayall-2 computer display. Rearrange the busy desktop as needed ... = [5] Set the PROP-ID and Project Info = On the NGUI window, press the "Set Project" button and fill out the relevant information. For example, {{{ Principal Investigator: Arjun Dey PIs Email Address: dey@noao.edu Actual Observers: Tristram Shandy, Bertram Wooster Observing Assistant: Karen Butler OA's Email Address: 4meter@noao.edu Proposal Identifier: 2016A-0453 Telescope System: KPNO Mayall 4m Science Instrument: Mosaic 3 }}} ''Please ensure the Proposal Identifier number is correct!'' Then, in one of the NOCS xterm windows, type: "nocs set project" = [6] Take a test Zero image to ensure system is working = Check that all is well by taking a test zero exposure. On the IRAF window, check the image statistics by cd-ing to the correct directory and typing "mscstat " All rms values should be about 4-6 adu/pix; the exception is amplifier [6] which has an rms~8-10adu/pix. If any one amplifier shows very high noise, then execute the following commands in a nocs window: {{{ nocs reset ccp nocs init ccp }}} Then take two more zeros; the first one will be junk, but the second one should be OK. = [7] Take dome flats and zeros = During the afternoon (after 4pm), take dome flats with the telescope pointed at the white spot. [[BR]] Instructions for Domeflats are here: [wiki:MayallZbandLegacy/NotesforObservers/Domeflats] Go eat dinner [[BR]] = [8] Just before observing = 0. It is best to start things up after sunset and to be ready to start observing before the 10 deg twilight mark. 10 deg twilight times for KPNO are listed here: Before 10 deg twilight, you can do all the steps in [8] (i.e., this one) 1. Start the mayall-idl:1 VNC session. {{{ A. In a terminal window from mayall-2, ssh to mzls@mayall-idl and log in using the mzls password B. type: ~/bin/vnc C. Now double click the VNC gui in the mayall-2 dock (i.e., the thing with all the icons at the bottom of the bottom left terminal). This launches a dialog box asking if you want to connect to mayall-idl.kpno.noao.edu:1 . Say "connect" and type in the mzls password. The VNC starts up. }}} The VNC session should automatically start MUPTILES, MOSSTAT and COPILOT. Rearrange as desired. If it doesn't, open terminals in the VNC session and manually run the scripts for monitoring the observations and updating the tile file: 1a) The FITS file listing which tiles have been completed should be updated throughout the night. This file is '''$MOS3_OBS/obstatus/mosaic-tiles_obstatus.fits'''. The following IDL command will monitor exposures as they are taken throughout the night, automatically updating this file. From a terminal on the mayall-idl VNC, in the mzls account, start this running: {{{ idl muptiles }}} At the end of the night, you should check the updated tile file into the svn repository. 1b). In a second terminal on the VNC tart the automated script for monitoring the data quality {{{ idl mosstat_continuous }}} This will run mosstat on each frame as it shows up and display the results on the screen. 1c). In a third terminal on the VNC start running [wiki:MayallZbandLegacy/NotesforObservers/Copilot copilot]: {{{ cd ~/obsbot python copilot.py }}} 2. Take a zero image to ensure everything is working. Once the image has been written, use the IRAF window and run mscstat to make sure the system came up OK. See step [6] above. 3. Set the focus to some approximate value based on the Truss temperature and the formula: - Focus(zd) = -8400+(1.4-Ttruss)x110 where Ttruss is in deg C 4. Ask the telescope operator to point the telescope to a 5th mag star and to center it on the MOSAIC3 focal plane. 5. Wait patiently for 10 deg twilight. = [9] Get Ready to Observe - start of night = ''No on-sky observations are permitted before 10 deg twilight.'' [wiki:MayallZbandLegacy/NotesforObservers/TenDegTwilight] 1. At start of night, check telescope pointing and zero the telescope coordinates using a bright star placed on the telescope boresight (defined as the center of the mosaic3 focal plane). When taking the image of the bright star, make sure to create an OBJECT script with NGUI (not a TEST, because MOSSTAT will ignore it), zd filter, 1 - 5 sec exposure. Watch for MOSSTAT to process the image and report the RA, Dec offsets. If mosstat fails, it could be because (a) the telescope is mis-pointed, (b) the telescope is out of focus (set the focus approx using the formula above and try again). Once you are able to determine offsets, give these offsets to the OA with **opposite** sign. That is, if mosstat reads "RA,Dec offsets = -15.34, 18.32", then you need to provide the OA with the offsets of +15,-18 to zero the telescope coordinates. You can take another OBJECT frame if you want to check to make sure the offset went in the right direction. 2. Move to the beginning of the first MzLS tile position. What position is that, you ask? One way to find out is to run mosbot.py (see step 5 below), and look for RA, DEC in expose-1.sh in the exec/mosbot/ directory. Tell these coordinates to the OA to move the telescope. Another way is to look at the first entry in the ~/obsbot/pass1.json file on the mayall-idl computer - remember, it is in degrees, so you will have to divide the RA by 15. Getting within a degree is close enough, so no need to be super precise. 3. Focus the telescope - create a focus script using the NGUI: exposure time 5 or 10 seconds, zd filter, -100 micron focus steps, click Midpoint to *on*, 9 exposures. - run the focus script from the /home/observer/exec directory - analyze the focus image using mscstarfocus - edit the /data3/observer/mscfoc.cl script to correct the name of the image that needs to be analyzed - run 'cl < /data3/observer/mscfoc.cl' in iraf window in data directory. - mark about 10 stars around the image; to get a quick idea if you have covered the right focus range mark "g" on the top star in a sequence which will pop up a graph that you will need to type "q" to get out of; mark the top star in each remaining sequence using "m"; "q" to quit; "d" to delete bad points; "q" to quit, then will get best focus value. - log the Truss temperature - set the telescope focus in the Configuration Monitor: Enter the value in Pedestal focus, **hit return**, and then hit Apply. If you forget the return, it will do nothing. - focus the guiders; this way you can use the guider images to monitor focus drifts **[wiki:MayallZbandLegacy/NotesforObservers/FocusInfo Example of a focus sequence] = [10] Observe - all night long! = 1. From the mzls@mayall-idl window, generate the top-level observing script (tonight.sh) assuming that we have three plan files named pass1.json, etc. {{{ cd ~/obsbot python mosbot.py pass1.json pass2.json pass3.json }}} This will start the [wiki:MayallZbandLegacy/NotesforObservers/MosBot mosbot] script, which will watch the $MOS3_DATA directory for new images, analyze them, and update FUTURE exposure scripts, choosing the pass number and setting the exposure time. If the conditions look marginal, and you only want to run with pass3, say, start mosbot using {{{ python mosbot.py pass3.json pass3.json pass3.json }}} If you want to allow only pass 2 and pass 3, then use: {{{ python mosbot.py pass2.json pass3.json pass3.json }}} If the conditions started out well, and you launched mosbot with all three passes, but then deteriorated and now you want to force [wiki:MayallZbandLegacy/NotesforObservers/MosBot mosbot] to only use pass2 and pass3, you can create a file in the ~/exec/mosbot/ directory called nopass1: {{{ cd ~/exec/mosbot touch nopass1 }}} You can add a "nopass2" file to restrict the observing to pass 3. For more options with mosbot, see [wiki:MayallZbandLegacy/NotesforObservers/MosBot] Note that if you need to escape the script at some point during the night (see below), you will need to CTRL-C to stop mosbot.py and **restart** it when you are ready to start up again. If you forget this, you will be observing tiles that you already observed earlier in the night. 2. From the observer@mosaic3 xterm window, start taking exposures using the top-level observing script (tonight.sh): {{{ cd ~/exec/mosbot ./tonight.sh }}} 3. [wiki:MayallZbandLegacy/NotesforObservers/Copilot "copilot"] should already be running. If not, start running "copilot" in the mayall-idl VNC window. Copilot keeps a beautiful running plot of observing conditions. From any mzls@mayall-idl window: {{{ cd ~/obsbot python copilot.py }}} Whenever a new image is detected in the data directory (as defined by $MOS3_DATA), a new image is generated as ~/obsbot/recent.png . That image can also be displayed by anyone that can see this directory. It could be displayed with continuous updating with {{{ eog ~/obsbot/recent.png }}} However, running copilot automatically displays this. See [wiki:MayallZbandLegacy/NotesforObservers/Copilot] 4. Monitor focus by checking the image quality on each frame or keeping an eye on the mosstat PSF display. Keep track of the truss temperature variation and use the information to modify the focus as needed. Note that the focus may not respond quickly to changes in temperature, so monitor the images carefully before adjusting focus. The Mayall has astigmatism, so one can sometimes tell from the shape of the images which way to move the focus. To stop and do a focus sequence: - Create a file to tell obsbot to quit {{{ touch ~/exec/mosbot/quit }}} - Wait for the current exposure to complete (at which point the above file is automatically removed) - Run a focus sequence (see link below for instructions) - [wiki:MayallZbandLegacy/NotesforObservers/FocusInfo Example of a focus sequence] - Re-start the observing as described above. **IMPORTANT** make sure to re-run `mosbot.py` so that you don’t repeat exposures from the beginning of the night! Keep an eye on the CCD and dewar temps (should be around 173K and 90K respectively) OK - you are off and running! Congratulations!!! === If you have problems ... === See **[wiki:MayallZbandLegacy/NotesforObservers/Problems] === Checking the Sky Brightness, Seeing and Transparency === The "copilot" python script and mosstat should already be running in a VNC. If they are not, you can launch xterms, login to the mayall-idl computer as mzls, and run these individually. From an IDL prompt, use the MOSSTAT routine to analyze the latest image on disk: {{{ IDL> mosstat }}} There are keyword options that allow you to choose different exposure numbers or CCDs within that exposure. For example, to analyze chip 'im16' of the exposure number 12345, type: {{{ IDL> mosstat, 12345, ext='im16' }}} The full documentation can be seen with: {{{ IDL> doc_library,'mosstat' }}} To just have mosstat run continuously whenever each image appears, use {{{ IDL> mosstat_continuous }}} === Please write useful human logs === Keep a log about weather conditions, which pass you observed, and telescope problems. Follow the example on the pages at [wiki:MayallZbandLegacy/ObservingLogs] . Catastrophically bad frames (such as saturated frames, or where the telescope moved) are recorded and svn-checked-in to [https://desi.lbl.gov/trac/browser/decam/code/mosaic3/trunk/obstatus/bad_expid.txt ~products/mosaic3/obstatus/bad_expid.txt] . = [11] End of night = No on-sky observations are permitted with MOSAIC3 before [wiki:MayallZbandLegacy/NotesforObservers/TenDegTwilight 10 deg twilight or after 10 deg dawn]. No twilight flats are allowed. Once you exit the observing script (using "touch quit" or waiting for it to end), take a zero image. This ensures that the dark slide is put in place and the instrument is ready for shut down. === 1. Shut down the software === 1. Ensure that the "Shutter" and "ready" are both in the "Dark" position on the MCCD gui 2. Press the red "Stop MOSAIC" button on the MOSAIC Menu GUI. Wait for this to finish. 3. Then press the yellow "Stop Cameras" button on the MOSAIC Menu GUI. If the buttons do not work, then go to one of the NOCS xterm windows and type: {{{ nocs stop all (equivalent to pressing the "Stop MOSAIC" button) }}} Once nocs is shut down, type {{{ nocs stop ccp (equivalent to pressing the "Stop Cameras" button) }}} Once this is done, type {{{ nocs status all nocs fullstatus ccp }}} and make sure everything is shut down. Only when this is done, is it safe for the OAs to put on the lights in the dome. Also, exit the window that was running mosbot, by typing CTRL-C Exit the xterm and close it. === 2. Check in the updated tile file and updated bad exposure file === At the end of the night, you should check the updated tile file into the svn repository. On the mayall-idl computer: {{{ cd $MOS3_OBS/obstatus svn -m "observing update of tile file and bad exposure list” commit }}} If this does not work for some reason, try {{{ cd $MOS3_OBS/obstatus svn -m "observing update of tile file and bad exposure list” commit -username }}} and enter your wiki log in and password when prompted. You can also check in the files individually as {{{ cd ~/products/mosaic3/obstatus svn -m "bad exp update" commit bad_expid.txt svn -m "obstatus update" commit mosaic-tiles_obstatus.fits }}} === 3. Create the Almanac and Almanac plot files === At the end of the night, create the Almanac files and check them into svn. Do this using a terminal window on the mayall-idl VNC: {{{ - cd products/mosaic3/logs - idl - almanac - plotalmanac,'Almanac_date.fits',ps='plot_Almanac_date' - exit - svn add *Almanac_date* - svn commit *Almanac* }}} For example, for the night of March 26/27, 2015, this is done with: {{{ cd products/mosaic3/logs idl almanac exit svn add Almanac_2015-03-26.* svn commit Almanac_2015-03-26.* }}} There are actually two versions of this file, one that is an ASCII file (with .txt extension) and one that is a FITS file (with .fits extension). === 4. Create the Coverage and Summary files for the Night === In a mayall-idl window on the VNC: {{{ - idl - IDL> moscoverage_plot }}} This will produce a pdf plot of the current cumulative coverage. The plot is placed in the directory ~/products/obstatus/plots/ by the script. The file name has the form coverage.till_2016-MM-DD.pdf. log into a mayall-idl window; cd to obsbot; then: {{{ - python copilot.py --night - mv night.png .png (e.g., 2016-02-11.png) }}} You can then mail these plots out to mayall-obs@desi.lbl.gov to let your collaborators know how the night went. === 5. Close down everything === We update the .bashrc for the next night, so it is best if you close the VNC windows and exit any xterms on mosaic3 and mayall-idl. On a mayall-idl xterm, type: {{{ ~/bin/stopvnc }}} Exit from the window running mosbot by typing CTRL-C, and exit from that xterm. === End of night check list === * Dark slides in * MOSAIC3 NOCS shut down * MOSAIC3 cameras shut down * Almanac file created * mosaic-tile_obstatus.fits, almanac, bad_expid.txt files commited to SVN * Log out of all mosaic3 windows * Log out of all mayall-idl windows * Mayall-idl VNC shut down * Observing logs on wiki up-to-date and closed * Summary e-mail describing night sent out to mayall-obs@desi.lbl.gov === Thank the OA and go to bed === [[BackLinks]]