WDmodel.io module

I/O methods. All the submodules of the WDmodel package use this module for almost all I/O operations.

WDmodel.io._read_ascii(filename, **kwargs)[source]

Read ASCII files

Read space separated ASCII file, with column names provided on first line (leading # optional). kwargs are passed along to numpy.genfromtxt(). Forces any string column data to be encoded in ASCII, rather than Unicode.

Parameters

  • filename (str) – Filename of the ASCII file. Column names must be provided on the first line.

  • kwargs (dict) – Extra options, passed directly to numpy.genfromtxt()

Returns

out – Record array with the data. Field names correspond to column names in the file.

Return type

numpy.recarray

See also

numpy.genfromtxt()

WDmodel.io.copy_params(params)[source]

Returns a deep copy of a dictionary. Necessary to ensure that dictionaries that nest dictionaries are properly updated.

Parameters

params (dict or Object) – Any python object for which a deepcopy needs to be created. Typically a parameter dictionary such as that from WDmodel.io.read_params()

Returns

params – A deepcopy of the object

Return type

Object

Notes

Simple wrapper around copy.deepcopy()

WDmodel.io.get_filepath(infile)[source]

Returns the full path to a file. If the path is relative, it is converted to absolute. If this file does not exist, it is treated as a file within the WDmodel package. If that file does not exist, an error is raised.

Parameters

infile (str) – The name of the file to set the full path for

Returns

pkgfile – The path to the file

Return type

str

Raises

IOError – If the infile could not be found at location or inside the WDmodel package.

WDmodel.io.get_options(args, comm)[source]

Get command line options for the WDmodel fitter package

Parameters

  • args (array-like) – list of the input command line arguments, typically from sys.argv

  • comm (None or mpi4py.mpi.MPI instance) – Used to communicate options to all child processes if running with mpi

Returns

  • args (Namespace) – Parsed command line options

  • pool (None or :py:class`emcee.utils.MPIPool`) – If running with MPI, the pool object is used to distribute the computations among the child process

Raises

ValueError – If any input value is invalid

WDmodel.io.get_outfile(outdir, specfile, ext, check=False, redo=False, resume=False)[source]

Formats the output directory, spectrum filename, and an extension into an output filename.

Parameters

  • outdir (str) – The output directory name for the output file

  • specfile (str) – The spectrum filename

  • ext (str) – The output file’s extension

  • check (bool, optional) – If True, check if the output file exists

  • redo (bool, optional) – If False and the output file already exists, an error is raised

  • resume (bool, optional) – If False and the output file already exists, an error is raised

Returns

outfile – The output filename

Return type

str

Raises

IOError – If check is True, redo and resume are False, and outfile exists.

Notes

We set the output file based on the spectrum name, since we can have multiple spectra per object.

If outdir is configured by set_objname_outdir_for_specfile() for specfile, it’ll include the object name.

See also

set_objname_outdir_for_specfile()

WDmodel.io.get_params_from_argparse(args)[source]

Converts an argparse.Namespace into an ordered parameter dictionary.

Parameters

args (argparse.Namespace) – The parsed command-line options from WDmodel.io.get_options()

Returns

params – The parameter dictionary

Return type

collections.OrderedDict

Raises

RuntimeError – If format of argparse.Namespace is invalid. or If parameter is fixed but value is None. or If parameter value is out of bounds.

Notes

Assumes that the argument parser options were names

  • <param>_value : Value of the parameter (float or None)

  • <param>_fix : Bool specifying if the parameter

  • <param>_bounds : tuple with lower limit and upper limit

where <param> is one of WDmodel.io._PARAMETER_NAMES

See also

WDmodel.io.read_params(), WDmodel.io.get_options()

WDmodel.io.get_phot_for_obj(objname, filename)[source]

Gets the measured photometry for an object from a photometry lookup table.

Parameters

  • objname (str) – Object name to look for photometry for

  • filename (str) – The spectrum FWHM lookup table filename

Returns

phot – The photometry of objname with dtype=[('pb', 'str'), ('mag', '<f8'), ('mag_err', '<f8')]

Return type

numpy.recarray

Raises

  • RuntimeError – If there are no matches in the photometry lookup file or if there are multiple matches for an object in the photometry lookup file

  • ValueError – If the photometry or the photometry uncertainty values are not finite or if the photometry uncertainties are less <= 0

Notes

The lookup file must be readable by read_phot()

The column name with the object name objname expected to be obj

If column names for magnitudes are named <passband>, the column names for errors in magnitudes in passband must be ‘d’+<passband_name>.

WDmodel.io.get_pkgfile(infile)[source]

Returns the full path to a file inside the WDmodel package

Parameters

infile (str) – The name of the file to set the full package filename for

Returns

pkgfile – The path to the file within the package.

Return type

str

Raises

IOError – If the pkgfile could not be found inside the WDmodel package.

Notes

This allows the package to be installed anywhere, and the code to still determine the location to a file included with the package, such as the model grid file.

WDmodel.io.get_spectrum_resolution(specfile, spectable, fwhm=None, lamshift=None)[source]

Gets the measured FWHM from a spectrum lookup table.

Parameters

  • specfile (str) – The spectrum filename

  • spectable (str) – The spectrum FWHM lookup table filename

  • fwhm (None or float, optional) – If specified, this overrides the resolution provided in the lookup table. If None lookups the resultion from spectable.

  • lamshift (None or float, optional) – If specified, this overrides the wavelength shift provided in the lookup table. If None lookups the wavelength shift from spectable.

Returns

  • fwhm (float) – The FWHM of the spectrum file. This is typically used as an initial guess to the WDmodel.fit fitter routines.

  • lamshift (float) – The wavelength shift to apply additively to the spectrum. This is not a fit parameter, and is treated as an input

Raises

RuntimeWarning – If the spectable cannot be read, or the specfile name indicates that this is a test, or if there are no or multiple matches for specfile in the spectable

Notes

If the specfile is not found, it returns a default resolution of 5 Angstroms, appropriate for the instruments used in our program.

Note that there there’s some hackish internal name fixing since T. Matheson’s table spectrum names didn’t match the spectrum filenames.

WDmodel.io.make_outdirs(dirname, redo=False, resume=False)[source]

Makes output directories

Parameters

  • dirname (str) – The output directory name to create

  • redo (bool, optional) – If False the directory will not be created if it already exists, and an error is raised

  • resume (bool, optional) – If False the directory will not be created if it already exists, and an error is raised

Returns

None – If the output directory dirname is successfully created

Return type

None

Raises

  • IOError – If the output directory exists

  • OSError – If the output directory could not be created

Notes

If the options are parsed by get_options() then only one of redo or resume can be set, as the options are mutually exclusive. If redo is set, the fit is redone from scratch, while resume restarts the MCMC sampling from the last saved chain position.

WDmodel.io.read_fit_inputs(input_file)[source]

Read the fit input HDF5 file produced by write_fit_inputs() and return numpy.recarray instances with the data.

Parameters

input_file (str) – The HDF5 fit inputs filename

Returns

  • spec (numpy.recarray) – The spectrum with dtype=[('wave', '<f8'), ('flux', '<f8'), ('flux_err', '<f8')]

  • cont_model (numpy.recarray) – The continuuum model. Has the same structure as spec.

  • linedata (numpy.recarray) – The observations of the spectrum corresponding to the hydrogen Balmer lines. Has dtype=[('wave', '<f8'), ('flux', '<f8'), ('flux_err', '<f8'), ('line_mask', 'i4')]

  • continuumdata (numpy.recarray) – Data used to generate the continuum model. Has the same structure as spec.

  • phot (None or numpy.recarray) – None or the photometry with dtype=[('pb', 'str'), ('mag', '<f8'), ('mag_err', '<f8')]

  • fit_config (dict) –

    Dictionary with various keys needed to configure the fitter

    • rvmodel : {'ccm89','od94','f99', 'custom'} - Parametrization of the reddening law.

    • covtype : {'Matern32', 'SHO', 'Exp', 'White'}- kernel type used to parametrize the covariance

    • coveps : float - Matern32 kernel precision

    • phot_dispersion : float - Excess dispersion to add in quadrature with photometric uncertainties

    • scale_factor : float - Flux scale factor

Raises

  • IOError – If all the fit inputs could not be restored from the HDF5 input_file

  • RuntimeWarning – If the input_file includes a phot group, but the data cannot be loaded.

See also

write_fit_inputs()

WDmodel.io.read_full_model(input_file)[source]

Read the full SED model from an output file.

Parameters

input_file (str) – Input HDF5 SED model filename

Returns

spec – Record array with the model SED. Has dtype=[('wave', '<f8'), ('flux', '<f8'), ('flux_err', '<f8')]

Return type

numpy.recarray

Raises

  • KeyError – If any of wave, flux or flux_err is not found in the file

  • ValueError – If any value is not finite or if flux or flux_err have any values <= 0

WDmodel.io.read_mcmc(input_file)[source]

Read the saved HDF5 Markov chain file and return samples, sample log probabilities and chain parameters

Parameters

input_file (str) – The HDF5 Markov chain filename

Returns

  • samples (array-like) – The model parameter sample chain

  • samples_lnprob (array-like) – The log posterior corresponding to each of the samples

  • chain_params (dict) –

    The chain parameter dictionary

    • param_names : list - list of model parameter names

    • samptype : {'ensemble','pt','gibbs'} - the sampler to use

    • ntemps : int - the number of chain temperatures

    • nwalkers : int - the number of Goodman & Ware walkers

    • nprod : int - the number of production steps of the chain

    • ndim : int - the number of model parameters in the chain

    • thin : int - the chain thinning if any

    • everyn : int - the sparse of spectrum sampling step size

    • ascale : float - the proposal scale for the sampler

Raises

IOError – If a key in the fit_config output is missing

WDmodel.io.read_model_grid(grid_file=None, grid_name=None)[source]

Read the Tlusty/Hubeny grid file

Parameters

  • grid_file (None or str) – Filename of the Tlusty model grid HDF5 file. If None reads the TlustyGrids.hdf5 file included with the WDmodel package.

  • grid_name (None or str) – Name of the group name in the HDF5 file to read the grid from. If None uses default

Returns

  • grid_file (str) – Filename of the HDF5 grid file

  • grid_name (str) – Name of the group within the HDF5 grid file with the grid arrays

  • wave (array-like) – The wavelength array of the grid with shape (nwave,)

  • ggrid (array-like) – The surface gravity array of the grid with shape (ngrav,)

  • tgrid (array-like) – The temperature array of the grid with shape (ntemp,)

  • flux (array-like) – The DA white dwarf model atmosphere flux array of the grid. Has shape (nwave, ngrav, ntemp)

Notes

There are no easy command line options to change this deliberately because changing the grid file essentially changes the entire model, and should not be done lightly, without careful comparison of the grids to quantify differences.

See also

WDmodel.WDmodel

WDmodel.io.read_params(param_file=None)[source]

Read a JSON file that configures the default guesses and bounds for the parameters, as well as if they should be fixed.

Parameters

param_file (str, optional) – The name of the input parameter file. If not the default file provided with the package, WDmodel_param_defaults.json, is read.

Returns

params – The dictionary with the parameter values, bounds, scale and if fixed. See notes for more detailed information on dictionary format and WDmodel_param_defaults.json for an example file for param_file.

Return type

dict

Notes

params is a dict the parameter names, as defined with WDmodel.io._PARAMETER_NAMES as keys

Each key must have a dictionary with keys:

  • value : value

  • fixed : a bool specifying if the parameter is fixed (True) or allowed to vary (False)

  • scale : a scale parameter used to set the step size in this dimension

  • bounds : An upper and lower limit on parameter values

The default bounds are set by the grids available for the DA White Dwarf atmospheres, and by reasonable plausible ranges for the other parameters. Don’t muck with them unless you really have good reason to.

This routine does not do any checking of types, values or bounds. This is done by WDmodel.io.get_params_from_argparse() before the fit. If you setup the fit using an external code, you should check these values.

WDmodel.io.read_pbmap(filename, **kwargs)

Read passband obsmode mapping table - wraps _read_ascii()

WDmodel.io.read_phot(filename, **kwargs)

Read photometry - wraps _read_ascii()

WDmodel.io.read_reddening(filename, **kwargs)

Read J. Holberg’s custom reddening function - wraps _read_ascii()

WDmodel.io.read_spec(filename, **kwargs)[source]

Read a spectrum

Wraps _read_ascii(), adding testing of the input arrays to check if the elements are finite, and if the errors and flux are strictly positive.

Parameters

  • filename (str) – Filename of the ASCII file. Must have columns wave, flux, flux_err

  • kwargs (dict) – Extra options, passed directly to numpy.genfromtxt()

Returns

spec – Record array with the spectrum data. Has dtype=[('wave', '<f8'), ('flux', '<f8'), ('flux_err', '<f8')]

Return type

numpy.recarray

Raises

ValueError – If any value is not finite or if flux or flux_err have any values <= 0

See also

numpy.genfromtxt(), _read_ascii()

WDmodel.io.read_spectable(filename, **kwargs)

Read spectrum FWHM table - wraps _read_ascii()

WDmodel.io.set_objname_outdir_for_specfile(specfile, outdir=None, outroot=None, redo=False, resume=False, nocreate=False)[source]

Sets the short human readable object name and output directory

Parameters

  • specfile (str) – The spectrum filename

  • outdir (None or str, optional) – The output directory name to create. If None this is set based on specfile

  • outroot (None or str, optional) – The output root directory under which to store the fits. If None the default is 'out'

  • redo (bool, optional) – If False the directory will not be created if it already exists, and an error is raised

  • resume (bool, optional) – If False the directory will not be created if it already exists, and an error is raised

  • nocreate (bool, optional) – If True then creation of output directories is not even attempted

Returns

  • objname (str) – The human readable object name based on the spectrum

  • dirname (str) – The output directory name created if successful

See also

make_outdirs()

WDmodel.io.write_fit_inputs(spec, phot, cont_model, linedata, continuumdata, rvmodel, covtype, coveps, phot_dispersion, scale_factor, outfile)[source]

Save all the inputs to the fitter to a file

This file is enough to resume the fit with the same input, redoing the output, or restoring from a failure.

Parameters

  • spec (numpy.recarray) – The spectrum with dtype=[('wave', '<f8'), ('flux', '<f8'), ('flux_err', '<f8')]

  • phot (None or numpy.recarray) – None or the photometry with dtype=[('pb', 'str'), ('mag', '<f8'), ('mag_err', '<f8')]

  • cont_model (numpy.recarray) – The continuuum model. Must have the same structure as spec. Produced by WDmodel.fit.pre_process_spectrum(). Used by WDmodel.viz

  • linedata (numpy.recarray) – The observations of the spectrum corresponding to the hydrogen Balmer lines. Must have dtype=[('wave', '<f8'), ('flux', '<f8'), ('flux_err', '<f8'), ('line_mask', 'i4')] Produced by WDmodel.fit.pre_process_spectrum() Used by WDmodel.viz

  • continuumdata (numpy.recarray) – Data used to generate the continuum model. Must have the same structure as spec. Produced by WDmodel.fit.pre_process_spectrum()

  • rvmodel ({'ccm89','od94','f99', 'custom'}) – Parametrization of the reddening law. Used to initialize WDmodel.WDmodel.WDmodel() instance.

  • covtype ({'Matern32', 'SHO', 'Exp', 'White'}) – stationary kernel type used to parametrize the covariance in WDmodel.covariance.WDmodel_CovModel

  • coveps (float) – If covtype is 'Matern32' a celerite.terms.Matern32Term is used to approximate a Matern32 kernel with precision coveps.

  • phot_dispersion (float, optional) – Excess photometric dispersion to add in quadrature with the photometric uncertainties phot.mag_err in WDmodel.likelihood.WDmodel_Likelihood.

  • scale_factor (float) – Factor by which the flux must be scaled. Critical to getting the right uncertainties.

  • outfile (str) – Output HDF5 filename

Notes

The outputs are stored in a HDF5 file with groups

  • spec - storing the spectrum and scale_factor

  • cont_model - stores the continuum model

  • linedata - stores the hydrogen Balmer line data

  • continuumdata - stores the data used to generate cont_model

  • fit_config - stores covtype, coveps and rvmodel as attributes

  • phot - only created if phot is not None, stores phot, phot_dispersion

WDmodel.io.write_full_model(full_model, outfile)[source]

Write the full SED model to an output file.

Parameters

  • full_model (numpy.recarray) – The SED model with dtype=[('wave', '<f8'), ('flux', '<f8'), ('flux_err', '<f8')]

  • outfile (str) – Output HDF5 SED model filename

Notes

The output is written into a group model with datasets

  • wave : array-like - the SED model wavelength

  • flux : array-like - the SED model flux

  • flux_err : array-like - the SED model flux uncertainty

WDmodel.io.write_params(params, outfile)[source]

Dumps the parameter dictionary params to a JSON file

Parameters

  • params (dict) – A parameter dict such as that produced by WDmodel.io.read_params()

  • outfile (str) – Output filename to save the parameter dict as a JSON file.

Notes

params is a dict the parameter names, as defined with WDmodel.io._PARAMETER_NAMES as keys

Each key must have a dictionary with keys:

  • value : value

  • fixed : a bool specifying if the parameter is fixed (True) or allowed to vary (False)

  • scale : a scale parameter used to set the step size in this dimension

  • bounds : An upper and lower limit on parameter values

Any extra keys are simply written as-is JSON doesn’t preserve ordering necessarily. This is imposed by WDmodel.io.read_params()

See also

WDmodel.io.read_params()

WDmodel.io.write_phot_model(phot, model_mags, outfile)[source]

Write the photometry, model photometry and residuals to an output file.

Parameters

  • phot (None or numpy.recarray) – None or the photometry with dtype=[('pb', 'str'), ('mag', '<f8'), ('mag_err', '<f8')]

  • model_mags (None or numpy.recarray) – The model magnitudes. Has dtype=[('pb', 'str'), ('mag', '<f8')]

  • outfile (str) – Output space-separated text filename

Notes

The data is saved to a space-separated ASCII text file with 6 decimal places of precision.

The order of the columns is

  • pb : array-like - the observation’s passband

  • mag : array-like - the observed magnitude

  • mag_err : array-like - the observed magnitude uncertainty

  • model_mag : array-like - the model magnitude

  • res_mag : array-like - the magnitude residual

WDmodel.io.write_spectrum_model(spec, model_spec, outfile)[source]

Write the spectrum and the model spectrum and residuals to an output file.

Parameters

  • spec (numpy.recarray) – The spectrum with dtype=[('wave', '<f8'), ('flux', '<f8'), ('flux_err', '<f8')]

  • model_spec (numpy.recarray) – The model spectrum. Has dtype=[('wave', '<f8'), ('flux', '<f8'), ('norm_flux', '<f8'), ('flux_err', '<f8')]

  • outfile (str) – Output space-separated text filename

Notes

The data is saved to a space-separated ASCII text file with 8 decimal places of precision.

The order of the columns is

  • wave : array-like - the spectrum wavelength

  • flux : array-like - the observed flux

  • flux_err : array-like - the observed flux uncertainty

  • norm_flux : array-like - the model flux without the Gaussian process covariance model

  • model_flux : array-like - the model flux

  • model_flux_err : array-like - the model flux uncertainty

  • res_flux : array-like - the flux residual