IDL Basics

• Index:
  • x,y,var - A single element variable of arbitrary data type.
  • arr - A multiple element array of arbitrary data type and size.
  • struc - A structure containing multiple subcomponents of arbitrary type and size (may include arrays as subelements).
  • FUNCTION() - Parenthesis denote an IDL function.
  • generic - Generic example to fill out code.
  • result=function(input) - Sample line or stub of code, as entered at the IDL prompt.
  
  
  
• Data Types - IDL makes use of the same data types as most programming languages, the most common being: int, long, float, double, string. Generally you can convert from one type to another on the fly by, for example:x=DOUBLE(x)The exceptions being attempts to convert a string of alphanumeric characters into a numeric data type, which works only for appropriately formatted strings or attempts to convert a data type to integer, which requires a different function such as ROUND(), FLOOR() or CEIL().
  
  
• Referencing Arrays - IDL arrays are very flexible, which can both greatly simplify programming tasks but also lead to some very frustrating and unexpected program behavior. Single array elements are referenced in brackets as x=arr[3], while multiple elements can be referenced in multiple ways: x=arr[[3,4,8,11]] for separated elements and x=arr[2:48] for a continuous subsample. Arrays can contain multiple dimensions, and a reference to an element of, for example, a three dimensional array would take the form x=arr[2,5,3]
  
  
• Structures - Most of the DEEP data are in large catalogs stored in various FITS files. Structure subelements are referenced with '.' as in x=struc.var or x=struc.arr[3]. Structures can both contain arrays and be contained within arrays, so a reference that pulls out a subset of wavelengths contained within an array of spectra could look like: spec=spectra_catalog[20].lambda[0:200].

Useful General Commands

• ?function or command name - Opens up the interactive IDL help dialog, and searches for the appropriate entry.
• help,var
  help,struc,/str - prints out information about the variable in question, the /str switch prints out information on a structure's subcomponents.
• print,var - prints the value(s) contained in the requested variable
• WHICH,'function name' - IDL searches for and prints out the location of the program text file that contains the function in question. This also works for programs not built in to IDL, and can be very useful for finding the text of custom data handling programs for additional help. (The name must be in quotes)

Useful Array Commands/Functions

• x=MIN(arr) - Returns the minimum value contained within arr. (Counterpart is MAX().)
• x=MEDIAN(arr) - Returns the median value contained within arr. (Similarly, MEAN() returns the average value.)
• x=STDDEV(arr) - Returns the standard deviation or rough width about the mean of a data vector.
• x=SORT(arr) - Returns a sorted list running from min to max of the elements in arr. However, it returns not the sorted values themselves, but the indices of the elements in the array that correspond to those values. To use the results, reference it as arr_new=arr[x], or cut out a step and set arr_new=arr[SORT(arr)]
• x=WHERE(arr GT 10) - The where command is a powerful pruning tool for selecting out subsamples of arrays. The conditional statement may be made arbitrarily complex to select a very restricted subset. For example, to make a high redshift, faint galaxy cut in a DEEP catalog structure called 'zcat': inds=WHERE(zcat.z GT 1.0 AND zcat.MAGB GT 25) gives you a vector called inds that contains the zcat index values for the high redshift, faint galaxies you're looking for.
  
  If no matches are found, the output of a WHERE call is -1. For this reason, it is sometimes useful to have a variable that stores the number of matches. The WHERE command takes an optional variable name for this purpose, to be placed after the conditional statement and a comma. For example: x=WHERE(arr GT 100,count) stores the number of matches found (or -1 for no matches) in the count variable, which you can refer to as needed.
• MATCH,arr1,arr2,result_arr1,result_arr2 - The match command takes two input arrays, here arr1 and arr2 and compares their contents to see where they overlap. The indices for any matches the routine finds are stored in two output arrays, here result_arr1 and result_arr2. For DEEP2 data, this command is probably most useful for matching up data between catalogs using the catalog OBJNO field. The following lines, for instance, will match up the entries in two catalogs called cat1 and cat2, then use the results to create two new catalogs that are matched entry for entry: MATCH,cat1.objno,cat2.objno,match1,match2
  cat1_m=cat1[match1]
  cat2_m=cat2[match2] At this point, the indices of cat1_m point to objects identical to those in cat2_m, and you can make much easier use of the information in both catalogs simultaneously.
• cat=MRDFITS('input_filename.fits',1) - Astronomers use .fits files to store data. This format includes useful header information followed by data that is stored in tabular form. Fits files are unreadable as plain text, so specialized routines are required to read/write them. The MRDFITS routine searches for the named file, and if found, reads the data in that file into, in this case, the cat variable. Use help,cat,/str to list the contents of the database. Fits files may contain multiple "pages," or separate arrays where another set of data is stored. Most DEEP-related files do not take advantage of this feature, but if you encounter one that does, change the final number in the MRDFITS call to reference the page you want.
• MWRFITS,cat,'output_filename.fits',/create - Counterpart to MRDFITS, this routine writes out a data structure to a fits file. If a file already exists with the same name as specified for output, MWRFITS will write a new page of data to that file. Setting the /create switch will force MWRFITS to overwrite the file, starting from scratch.

DEEP2-Specific Functions

• cat=STRUCTINDEX(['zcat.latest.fits']) - STRUCTINDEX is a function that returns a catalog of objects drawn from the input files (this will almost always be just zcat.latest.fits as listed). This catalog contains a few physical parameters like observed magnitudes in B, R, and I band, in addition to a redshift (z) and redshift quality for each object.
• cat=READ_KZCAT() - As with STRUCTINDEX, this function returns a catalog of DEEP objects. This routine returns a catalog for which the redshift values may be out of date and should therefore not be used for those data. It does however contain entries for U-B (cat.UB) and B-band absolute magnitude (cat.MB) from the DEEP k-correction code and is very useful for selecting subsamples on the color magnitude diagram.
• coadd=COADD_SPECTRA(cat[arr]) - This routine returns a coadded spectrum compiled from those objects in the input catalog. This catalog must contain at least the subcomponents: objname, z, maskname, date.
• ew=MEASURE_EW(spec,[[cl1,cl2],[ch1,ch2]],[f1,f2]) - This function returns the equivalent width in angstroms of a feature based on the listed inputs: spec - a spectral structure from DEEP2 containing at least the fields LAMBDA and SPEC (output from coadd_spectra should work with this function with no further tweaking);
  [[cl1,cl2],[ch1,ch2]] - the values that define the edges of the pseudocontinuum passbands in angstroms, in the format [[blue passband minimum, blue passband maximum],[red passband minimum, red passband maximum]];
  [f1,f1] - the values that define the edges of the feature to be measured in the format [feature minimum, feature maximum].

Plotting Functions

• PLOT,x_arr,y_arr - The plot command spawns a new window, displaying the x,y data pairs. There are innumerable other keyword switches to add to tweak various aspects of the resultant plot, and you should use the IDL help function to explore these in more depth. The most important ones however are, xrange=[min,max], which forces the x-axis to fit the specified range (yrange scales the y axis the same way); xtitle='X axis label' which creates a text label for the x (or y) axis; psym=3 specifies the type of symbol to use to display points (the most useful psym values run from 1-7, beyond which there are more esoteric meanings). If undefined, IDL defaults to connecting the points from one to the next; color=1 which colors the entire plot with a color from the IDL color table matching the number specified. (The most useful are 1-red, 2-green, 3-blue) Also, see the FSC_COLOR entry.
  
  A complete plot call might therefore look like:
  >plot,x,y,xrange=[0,100],yrange=[-1,11],xtitle='Input current',ytitle='Output voltage',psym=2,color=1
• PLOTSYM,0,2,/fill - This command allows the user to set a separate plot symbol type. The first argument defines the shape (0 corresponds to a circle, 3 - star, 4 - triangle, 8 - square), the second is optional and defines the size for IDL to use. /Fill is also optional, and instructs IDL to draw the points as filled shapes (as opposed to outlines if /fill is not present). You may also use the color= in this function (though I prefer to set the color in the plot or overplot command). Once you have used this command to define a point type, it is referenced as psym=8 in a plot or overplot command. For more info, check out the program text, which is linked here
• plot,blah,blah,color=FSC_COLOR('color name') - If IDL's default color tables are misbehaving, this command can be entered as the argument for any color= call. The argument to FSC_COLOR can be a number, but is likely better remembered as the string containing a color's name. Valid color names can be found on the chart linked here.
• OPLOT,x,y - Same as plot, but this plots the x,y pairs over a previously drawn plot, especially useful for comparing one spectrum to another. Psym still works for oplot, but you should not use title or range keywords in an oplot call.
• SPLOT,x,y - Same as plot, but spawns an interactive plot window that allows you to use your mouse to scroll and zoom on sections of the plot.
• PLOTHIST,x,bin=0.5 - Plots a histogram of the input data vector values, binned up in increments of the specified size. Takes many of the same keywords as PLOT. (The bin keyword is optional, but plothist defaults to 1, which is not generally useful.)
• SET_PLOT,'ps', - This sets the location to which the output of plotting functions should be displayed. 'ps' sets it to a PostScript file, which will be further specified by DEVICE as below. 'x' sets it to display to the screen.
• DEVICE - Controls more specifically where the output to (generally) 'ps' goes. Device also takes a number of optional formatting commands like xsize, ysize, xoffset, yoffset which can be defined in inches, pixels, etc. by setting the appropriate switch. If you plan to display color, you also need to set the /color switch. For example, you might start a new plot with:
  DEVICE,filename=output_plot.ps,/color,xsize=4,ysize=5,xoffset=2,yoffset=2,/inches
  and end it once you've finished inputting plot commands with:
  DEVICE,/CLOSE
  
• XYOUTS,xpos,ypos,Text to print to plot or variable name - Used for writing plot captions/figures, instead of plotting points, xyouts attempts to write text to the plot at the x,y position given in the function call. This input can either be a string or a numeric variable which it will attempt to format appropriately. You can use tags like charsize=0.8,charthick=3 as well as color tags to control the appearance of this text.
• POLYFILL,[x1,x1,x2,x2],[y1,y2,y2,y1] - Polyfill, as the name implies, shades a polygonal region of the plot. You can use the color= switch to control the shade used. The input coordinates should be in the form of an array containing the x positions of each corner of the polygon, followed by an array containing the y positions of the corners. Polyfill can be used to create arbitrary shapes, with the restriction that it requires at least three points. Other keywords include /LINE_FILL, which fills the polygon with lines instead of a solid color, and to be used with LINE_FILL, ORIENTATION=(degrees from horizontal) to rotate the direction of the lines, as well as SPACING=(line spacing in cm) to further change the fill pattern.
• !P.MULTI=[0,1,2,0,0] - This command changes the way in which multiple plots are written to the screen or to a file. Most useful will likely be the second and third entries, which change the number of plots to draw in the x and y directions respectively before IDL attempts to reset to a new page. (1,2, as in the example will force IDL to draw two plots per page, one above another.) IDL will assign each command you enter to the active plot window until it reaches the next plot/plothist command, at which point it will create a new plot which becomes the active plot window. To reset this to the default, use !P.MULTI=[0,0,0,0,0] (Note: If you're sending output to a postscript file, the !P.MULTI command can screw up how postscript attempts to view the file. If this is the case, you'll need to override the postscript window size, for which I have found the following to be suitable: device,filename='whatever.ps',xsize=7.25,ysize=10.0,xoffset=0.5,yoffset=0.5,/inches

Notes on type conversion and strings

• str_var=STRING(var) - As mentioned in the introduction to data types, most IDL variables can be converted from one type to another with a function like, for instance, one that converts an integer to a double-precision floating point variable, as in: like x=DOUBLE(y)
  If you want to use string based functions, such as xyouts, with variables that are numeric by default, you can convert them to strings the same way, as in: x=STRING(y)
• str_var_2=STRCOMPRESS(str_var_1,/REMOVE_ALL) - Some strings contain lots of extraneous white space, and the STRING() operation can sometimes add extra on its own. So if you notice that and IDL routine produces plot keys/labels with lots of blank space, you can trim down the result of converting a number to a string so that the variable contains nothing but useful characters with the string compress function.
• str_var=TEXTOIDL('sample text') - IDL does have the ability to print subscripts/superscripts and scientific characters, but it does so in a very nonintuitive way. It's usually easier to use the TEXTOIDL function, which attempts to parse the string in the argument into something readable in TEX. TEX, or LATEX is a computer language used to produce scientific manuscripts. The most common uses you are likely to encounter in plotting with IDL follow:
  • Subscript: str_var=TEXTOIDL('Text_{subscript text}') - Especially useful for variables with an index
  • Superscript: str_var=TEXTOIDL('Text^{subscript text}') - Especially useful for exponents
  • Greek characters: str_var=TEXTOIDL('\Letter') - Produces the Greek character specified. This is case sensitive, so \alpha will produce a lowercase alpha, \Alpha will produce uppercase. This format can be used for lots of other LaTeX-supported symbols, too. For example, there's a reference with many such characters linked here
• Strings can also be used with the '+' operator, which for variables of string data type acts as a concatenation operator. So if you have a complicated string to use as the argument to xtitle or xyouts, for instance, you can set it up beforehand as, for instance:
  key=TEXTOIDL('Red objects with \Chi^{2}')+'= '+STRCOMPRESS(STRING(cat.chisq),/REMOVE_ALL)
  xyouts,x1,y1,key,charthick=3,color=1

EXAMPLES

	cat=READ_KZCAT()

	z_trim=WHERE(cat.z GT 1.0)
	cat1=cat[z_trim]	

	color_trim=WHERE(cat1.UB LT 1.0 and cat1.UB GT 0.8)
	cat2=cat1[color_trim]

	qual_trim=WHERE(cat2.zquality EQ 4)
	cat_final=cat2[qual_trim]

	bright_sort=SORT(cat_final.MB)
	cat_final=cat_final[bright_sort]

	coadd_1=COADD_SPECTRA(cat_final[0:199])
	coadd_2=COADD_SPECTRA(cat_final[200:399])

	set_plot,'ps'
	device,filename='coadd_compare.ps',/color
	plot,coadd_1.lambda,coadd_1.spec,thick=3,xrange=[3650,4100],xtitle='Restframe wavelength',yrange=[0,500],ytitle='flux'
	oplot,coadd_2.lambda,coadd_2.spec,thick=3,color=1	
	device,/close
	set_plot,'x'		
	

	cat=STRUCTINDEX(['zcat.latest.fits'])

	z_trim=WHERE(cat.z GT 0.5 AND cat.Z LT 1.0)
	cat1=cat[z_trim]	

	qual_trim=WHERE(cat1.zquality EQ 4)
	cat_2=cat1[qual_trim]

	width=STDDEV(cat_2.magb)
	med_point=MEDIAN(cat_2.magb)

	outlier_trim=WHERE(cat_2.magb GT med_point-5.0*width AND cat_2.magb LT med_point+5.0*width)
	cat_final=cat_2[outlier_trim]

	xmin=MIN(cat_final.magb)
	xmax=MAX(cat_final.magb)
	xrange=ABS(xmax-xmin)

	set_plot,'ps'
	device,filename='brightness_distribution.ps'
	plothist,cat_final.MAGB,bin=(xrange/20.0),xrange=[xmin-0.1*width,xmax+0.1*width],thick=3,xtitle=TEXTOIDL('M_{B} apparent')
	device,/close
	set_plot,'x'		
	

	zcat=STRUCTINDEX(['zcat.latest.fits'])
	kzcat=READ_KZCAT()
        
        match,zcat.objno,kzcat.objno,zm,km
        zcat=zcat[zm]
        kzcat=kzcat[km]

        qual_z_trim=WHERE(zcat.zquality eq 4 and zcat.z LT 0.85 and zcat.z GT 0.75)
        zcat=zcat[qual_z_trim]
        kzcat=kzcat[qual_z_trim]
        
        color_trim=WHERE(kzcat.UB LT 0.8 and kzcat.MB LT -21)
        zcat=zcat[color_trim]
        kzcat=kzcat[color_trim]

        blue_coadd=COADD_SPECTRA(zcat[ 0:MIN([200,N_ELEMENTS(zcat.objno)-1]) ])

        oii_ew=MEASURE_EW(blue_coadd,[[3696.3,3716.3],[3738.3,3758.3]],[3716.3,3738.3])

        print,oii_ew

	

        kcat=READ_KZCAT()

        samp1=WHERE(kcat.zquality eq 4 and kcat.z lt 1.0 and kcat.z GT 0.9 and kcat.ub GT 1.05)
        samp2=WHERE(kcat.zquality eq 4 and kcat.z lt 1.0 and kcat.z GT 0.9 and kcat.ub LT 0.95)

        red_coadd=coadd_spectra(kcat[samp1[0:MIN([75,N_ELEMENTS(samp1)-1])]])
        blue_coadd=coadd_spectra(kcat[samp2[0:MIN([75,N_ELEMENTS(samp2)-1])]])

        red_ew=MEASURE_EW(red_coadd,[[3696.3,3716.3],[3738.3,3758.3]],[3716.3,3738.3])
        blue_ew=MEASURE_EW(blue_coadd,[[3696.3,3716.3],[3738.3,3758.3]],[3716.3,3738.3])
        !P.MULTI=[0,1,3,0,0]
        set_plot,'ps'
        device,filename="sample4_output.ps",/color,$
          xsize=7.25,ysize=10.0,xoffset=0.5,yoffset=0.5,/inches

        plot,kcat.mb,kcat.ub,psym=3,xtitle='U-B',ytitle=TEXTOIDL('M_{B,abs}'),$
          thick=3,charthick=3,xrange=[-25,-14],yrange=[-0.2,1.7]
        plotsym,0,0.5,/fill
        oplot,kcat[samp1].mb,kcat[samp1].ub,psym=8,color=FSC_COLOR('red')
        oplot,kcat[samp2].mb,kcat[samp2].ub,psym=8,color=FSC_COLOR('blue')

        plot,[0],[0],xrange=[3600,3900],yrange=[0,MAX([MAX(red_coadd.spec),$
                                                       MAX(blue_coadd.spec)])],$
          xtitle='Restframe wavelength',ytitle='coadd flux',thick=3,charthick=3
        oplot,red_coadd.lambda,red_coadd.spec,thick=3,color=FSC_COLOR('red')
        oplot,blue_coadd.lambda,blue_coadd.spec,thick=3,color=FSC_COLOR('blue')
        
        plot,[0],[0],xrange=[0,100],yrange=[0,100],color=FSC_COLOR('white')
        str_red_ew=STRCOMPRESS(STRING(red_ew),/REMOVE_ALL)
        str_blue_ew=STRCOMPRESS(STRING(blue_ew),/REMOVE_ALL)

        xyouts,10,90,'OII EW in red coadd = '+str_red_ew,charthick=3,color=FSC_COLOR('red')
        xyouts,10,80,'OII EW in blue coadd = '+str_blue_ew,charthick=3,color=FSC_COLOR('blue')
        device,/close
        set_plot,'x'
        !P.MULTI=[0,0,0,0,0]

	

spec=MRDFITS('brblue_coadd.fits',1)
bpass=[WHERE(spec.lambda GT 3850 and spec.lambda LT 3950)]
rpass=[WHERE(spec.lambda GT 4000 and spec.lambda LT 4100)]

D4000=TOTAL(spec.spec[rpass])/TOTAL(spec.spec[bpass])

D4_str=STRCOMPRESS(STRING(D4000),/REMOVE_ALL)
xtext=TEXTOIDL('\lambda_{restframe} [\AA]')
ytext='Coadd Flux'
ttext='Coadd of 250 bright blue objects'

blue_xcoord=[3850,3850,3950,3950]
red_xcoord=[4000,4000,4100,4100]
ycoord=[-0.03,0.37,0.37,-0.03]

plot,[0],[0],xrange=[3500,4500],yrange=[-0.05,0.4],xtitle=xtext,ytitle=ytext,title=ttext,$
  charthick=2,thick=3
polyfill,blue_xcoord,ycoord,color=FSC_COLOR('sky blue')
polyfill,red_xcoord,ycoord,color=FSC_COLOR('pink')

oplot,spec.lambda,spec.spec,thick=3,color=FSC_COLOR('navy')

xyouts,4150,0.22,'D4000 = '+D4_str,charthick=2,charsize=1.2