Developer Docs
Documentation for all the under-the-hood classes and functions that most users won’t need to interact with.
Base Classes
These classes are the under-the-hood classes that provide much of the infrastructure for pyuvdata’s user classes.
UVParameter
is the base class for the attributes on pyuvdata’s user classes,
allowing the attributes to carry information like expected type and shape,
acceptable values or ranges and tolerances. We also have some type specific
parameter objects (e.g. for angles and earth locations) with extra functionality.
UVBase
is the base class for pyuvdata’s user classes which has the common
infrastructure to turn the UVParameter attributes into properties and check the
attribute shapes and values.
- class pyuvdata.parameter.UVParameter(name, *, required=True, value=None, spoof_val=None, form=(), description='', expected_type=<class 'int'>, acceptable_vals=None, acceptable_range=None, tols=(1e-05, 1e-08), strict_type_check=False, ignore_eq_none: bool = False, setter: callable | None = None)[source]
Data and metadata objects for interferometric data sets.
- Parameters:
name (str) – The name of the attribute. Used as the associated property name in classes based on UVBase.
required (bool) – Flag indicating whether this is required metadata for the class with this UVParameter as an attribute. Default is True.
value – The value of the data or metadata.
spoof_val – A fake value that can be assigned to a non-required UVParameter if the metadata is required for a particular file-type. This is not an attribute of required UVParameters.
form (‘str’, int or tuple) – Either ‘str’ or an int (if a single value) or tuple giving information about the expected shape of the value. Elements of the tuple may be the name of other UVParameters that indicate data shapes.
- Form examples:
‘str’: a string value
- (‘Nblts’, 3): the value should be an array of shape:
Nblts (another UVParameter name), 3
(): a single numeric value
3: the value should be an array of shape (3, )
description (str) – Description of the data or metadata in the object.
expected_type – The type that the data or metadata should be. Default is int or str if form is ‘str’.
acceptable_vals (list, optional) – List giving allowed values for elements of value.
acceptable_range (2-tuple, optional) – Tuple giving a range of allowed magnitudes for elements of value.
tols (float or 2-tuple of float) – Tolerances for testing the equality of UVParameters. Either a single absolute value or a tuple of relative and absolute values to be used by np.isclose()
strict_type_check (bool) – When True, the input expected_type is used exactly, otherwise a more generic type is found to allow changes in precisions or to/from numpy dtypes to not break checks.
setter (function, optional) – An optional function to be called after the parameter is set. The function should take a single argument, the instance of the class to which the parameter is attached.
- name
The name of the attribute. Used as the associated property name in classes based on UVBase.
- Type:
str
- required
Flag indicating whether this is required metadata for the class with this UVParameter as an attribute. Default is True.
- Type:
bool
- value
The value of the data or metadata.
- spoof_val
A fake value that can be assigned to a non-required UVParameter if the metadata is required for a particular file-type. This is not an attribute of required UVParameters.
- form
Either ‘str’ or an int (if a single value) or tuple giving information about the expected shape of the value. Elements of the tuple may be the name of other UVParameters that indicate data shapes.
- Form examples:
‘str’: a string value
- (‘Nblts’, 3): the value should be an array of shape:
Nblts (another UVParameter name), 3
(): a single numeric value
3: the value should be an array of shape (3, )
- Type:
‘str’, int or tuple
- description
Description of the data or metadata in the object.
- Type:
str
- expected_type
The type that the data or metadata should be. Default is int or str if form is ‘str’.
- acceptable_vals
List giving allowed values for elements of value.
- Type:
list, optional
- acceptable_range
Tuple giving a range of allowed magnitudes for elements of value.
- Type:
2-tuple, optional
- tols
Relative and absolute tolerances for testing the equality of UVParameters, to be used by np.isclose()
- Type:
2-tuple of float
- strict_type_check
When True, the input expected_type is used exactly, otherwise a more generic type is found to allow changes in precisions or to/from numpy dtypes to not break checks.
- Type:
bool
- expected_shape(uvbase)[source]
Get the expected shape of the value based on the form.
- Parameters:
uvbase (object) – Object with this UVParameter as an attribute. Needed because the form can refer to other UVParameters on this object.
- Returns:
tuple – The expected shape of the value.
- class pyuvdata.parameter.AngleParameter(name, *, required=True, value=None, spoof_val=None, form=(), description='', expected_type=<class 'int'>, acceptable_vals=None, acceptable_range=None, tols=(1e-05, 1e-08), strict_type_check=False, ignore_eq_none: bool = False, setter: callable | None = None)[source]
Subclass of UVParameter for Angle type parameters.
Adds extra methods for conversion to & from degrees (used by UVBase objects for _degrees properties associated with these parameters).
- Parameters:
name (str) – The name of the attribute. Used as the associated property name in classes based on UVBase.
required (bool) – Flag indicating whether this is required metadata for the class with this UVParameter as an attribute. Default is True.
value – The value of the data or metadata.
spoof_val – A fake value that can be assigned to a non-required UVParameter if the metadata is required for a particular file-type. This is not an attribute of required UVParameters.
form (‘str’, int or tuple) – Either ‘str’ or an int (if a single value) or tuple giving information about the expected shape of the value. Elements of the tuple may be the name of other UVParameters that indicate data shapes.
- Form examples:
‘str’: a string value
- (‘Nblts’, 3): the value should be an array of shape:
Nblts (another UVParameter name), 3
(): a single numeric value
3: the value should be an array of shape (3, )
description (str) – Description of the data or metadata in the object.
expected_type – The type that the data or metadata should be. Default is int or str if form is ‘str’.
acceptable_vals (list, optional) – List giving allowed values for elements of value.
acceptable_range (2-tuple, optional) – Tuple giving a range of allowed magnitudes for elements of value.
tols (float or 2-tuple of float) – Tolerances for testing the equality of UVParameters. Either a single absolute value or a tuple of relative and absolute values to be used by np.isclose()
strict_type_check (bool) – When True, the input expected_type is used exactly, otherwise a more generic type is found to allow changes in precicions or to/from numpy dtypes to not break checks.
- name
The name of the attribute. Used as the associated property name in classes based on UVBase.
- Type:
str
- required
Flag indicating whether this is required metadata for the class with this UVParameter as an attribute. Default is True.
- Type:
bool
- value
The value of the data or metadata.
- spoof_val
A fake value that can be assigned to a non-required UVParameter if the metadata is required for a particular file-type. This is not an attribute of required UVParameters.
- form
Either ‘str’ or an int (if a single value) or tuple giving information about the expected shape of the value. Elements of the tuple may be the name of other UVParameters that indicate data shapes.
- Form examples:
‘str’: a string value
- (‘Nblts’, 3): the value should be an array of shape:
Nblts (another UVParameter name), 3
(): a single numeric value
3: the value should be an array of shape (3, )
- Type:
‘str’, int or tuple
- description
Description of the data or metadata in the object.
- Type:
str
- expected_type
The type that the data or metadata should be. Default is int or str if form is ‘str’.
- acceptable_vals
List giving allowed values for elements of value.
- Type:
list, optional
- acceptable_range
Tuple giving a range of allowed magnitudes for elements of value.
- Type:
2-tuple, optional
- tols
Relative and absolute tolerances for testing the equality of UVParameters, to be used by np.isclose()
- Type:
2-tuple of float
- strict_type_check
When True, the input expected_type is used exactly, otherwise a more generic type is found to allow changes in precicions or to/from numpy dtypes to not break checks.
- Type:
bool
- class pyuvdata.parameter.LocationParameter(name, *, required=True, value=None, spoof_val=None, description='', tols=0.001)[source]
Subclass of UVParameter for location type parameters.
Supports either
astropy.coordinates.EarthLocation
objects orlunarsky.MoonLocation
objects. Adds extra methods for conversion to & from lat/lon/alt in radians or degrees (used by UVBase objects for _lat_lon_alt and _lat_lon_alt_degrees properties associated with these parameters).- Parameters:
name (str) – The name of the attribute. Used as the associated property name in classes based on UVBase.
required (bool) – Flag indicating whether this is required metadata for the class with this UVParameter as an attribute. Default is True.
value – The value of the data or metadata.
description (str) – Description of the data or metadata in the object.
tols (float or 2-tuple of float) – Tolerances for testing the equality of UVParameters. Either a single absolute value or a tuple of relative and absolute values to be used by np.isclose()
- name
The name of the attribute. Used as the associated property name in classes based on UVBase.
- Type:
str
- required
Flag indicating whether this is required metadata for the class with this UVParameter as an attribute. Default is True.
- Type:
bool
- value
The value of the data or metadata.
- spoof_val
Always set to None.
- form
Always set to None.
- Type:
int
- description
Description of the data or metadata in the object.
- Type:
str
- expected_type
Set to EarthLocation or MoonLocation
- acceptable_vals
Always set to None.
- Type:
list, optional
- acceptable_range
Always set to None.
- Type:
2-tuple, optional
- tols
Relative and absolute tolerances for testing the equality of UVParameters, to be used by np.isclose()
- Type:
2-tuple of float
- strict_type_check
When True, the input expected_type is used exactly, otherwise a more generic type is found to allow changes in precicions or to/from numpy dtypes to not break checks.
- Type:
bool
- property frame
Get the frame.
- property ellipsoid
Get the ellipsoid.
- set_lat_lon_alt(lat_lon_alt, ellipsoid=None)[source]
Set value from (latitude, longitude, altitude) tuple in radians.
- Parameters:
lat_lon_alt (3-tuple of float) – Tuple with the latitude (radians), longitude (radians) and altitude (meters) to use to set the value attribute.
ellipsoid (str or None) – Ellipsoid to use to convert between lat/lon/alt and xyz. Only used for MoonLocation objects.
- set_lat_lon_alt_degrees(lat_lon_alt_degree, ellipsoid=None)[source]
Set value from (latitude, longitude, altitude) tuple in degrees.
- Parameters:
lat_lon_alt (3-tuple of float) – Tuple with the latitude (degrees), longitude (degrees) and altitude (meters) to use to set the value attribute.
ellipsoid (str or None) – Ellipsoid to use to convert between lat/lon/alt and xyz. Only used for MoonLocation objects.
- class pyuvdata.parameter.SkyCoordParameter(name, *, required=True, value=None, spoof_val=None, form=(), description='', acceptable_range=None, radian_tol=4.84813681109536e-09)[source]
Subclass of UVParameter for SkyCoord parameters.
Needed for handling tolerances properly. The tols attribute is interpreted as the tolerance of the sky separation in radians.
- Parameters:
name (str) – The name of the attribute. Used as the associated property name in classes based on UVBase.
required (bool) – Flag indicating whether this is required metadata for the class with this UVParameter as an attribute. Default is True.
value – The value of the data or metadata.
spoof_val – A fake value that can be assigned to a non-required UVParameter if the metadata is required for a particular file-type. This is not an attribute of required UVParameters.
form (‘str’, int or tuple) – Either ‘str’ or an int (if a single value) or tuple giving information about the expected shape of the value. Elements of the tuple may be the name of other UVParameters that indicate data shapes.
- Form examples:
‘str’: a string value
- (‘Nblts’, 3): the value should be an array of shape:
Nblts (another UVParameter name), 3
(): a single numeric value
3: the value should be an array of shape (3, )
description (str) – Description of the data or metadata in the object.
acceptable_range (2-tuple, optional) – Tuple giving a range of allowed magnitudes for elements of value.
radian_tol (float) – Tolerance of the sky separation in radians.
- name
The name of the attribute. Used as the associated property name in classes based on UVBase.
- Type:
str
- required
Flag indicating whether this is required metadata for the class with this UVParameter as an attribute. Default is True.
- Type:
bool
- value
The value of the data or metadata.
- spoof_val
A fake value that can be assigned to a non-required UVParameter if the metadata is required for a particular file-type. This is not an attribute of required UVParameters.
- form
Either ‘str’ or an int (if a single value) or tuple giving information about the expected shape of the value. Elements of the tuple may be the name of other UVParameters that indicate data shapes.
- Form examples:
‘str’: a string value
- (‘Nblts’, 3): the value should be an array of shape:
Nblts (another UVParameter name), 3
(): a single numeric value
3: the value should be an array of shape (3, )
- Type:
‘str’, int or tuple
- description
Description of the data or metadata in the object.
- Type:
str
- expected_type
Always set to SkyCoord.
- acceptable_range
Tuple giving a range of allowed magnitudes for elements of value.
- Type:
2-tuple, optional
- tols
Set to (0, radian_tol).
- Type:
2-tuple of float
- strict_type_check
When True, the input expected_type is used exactly, otherwise a more generic type is found to allow changes in precicions or to/from numpy dtypes to not break checks.
- Type:
bool
- class pyuvdata.uvbase.UVBase[source]
Base class for objects with UVParameter attributes.
This class is intended to be subclassed and its init method should be called in the subclass init after all associated UVParameter attributes are defined. The init method of this base class creates properties (named using UVParameter.name) from all the UVParameter attributes on the subclass. AngleParameter and LocationParameter attributes also have extra convenience properties defined:
AngleParameter:
UVParameter.name+’_degrees’
LocationParameter:
UVParameter.name+’_lat_lon_alt’ UVParameter.name+’_lat_lon_alt_degrees’
- prop_fget(param_name)[source]
Getter method for UVParameter properties.
- Parameters:
param_name (str) – Property name to get, corresponds the the UVParameter.name.
- Returns:
fget – getter method to use for the property definition.
- prop_fset(param_name)[source]
Setter method for UVParameter properties.
- Parameters:
param_name (str) – Property name to set, corresponds the the UVParameter.name.
- Returns:
fset – setter method to use for the property definition.
- degree_prop_fget(param_name)[source]
Degree getter method for AngleParameter properties.
- Parameters:
param_name (str) – Property name to get, corresponds the the UVParameter.name with “_degrees” appended.
- Returns:
fget – getter method to use for the property definition.
- degree_prop_fset(param_name)[source]
Degree setter method for AngleParameter properties.
- Parameters:
param_name (str) – Property name to set, corresponds the the UVParameter.name with “_degrees” appended.
- Returns:
fset – setter method to use for the property definition.
- lat_lon_alt_prop_fget(param_name)[source]
Lat/lon/alt getter method for LocationParameter properties.
- Parameters:
param_name (str) – Property name to get, corresponds the the UVParameter.name with “_lat_lon_alt” appended.
- Returns:
fget – getter method to use for the property definition.
- lat_lon_alt_prop_fset(param_name)[source]
Lat/lon/alt setter method for LocationParameter properties.
- Parameters:
param_name (str) – Property name to set, corresponds the the UVParameter.name with “_lat_lon_alt” appended.
- Returns:
fset – setter method to use for the property definition.
- lat_lon_alt_degrees_prop_fget(param_name)[source]
Lat/lon/alt degree getter method for LocationParameter properties.
- Parameters:
param_name (str) – Property name to get, corresponds the the UVParameter.name with “_lat_lon_alt_degrees” appended.
- Returns:
fget – getter method to use for the property definition.
- lat_lon_alt_degrees_prop_fset(param_name)[source]
Lat/lon/alt degree setter method for LocationParameter properties.
- Parameters:
param_name (str) – Property name to set, corresponds the the UVParameter.name with “_lat_lon_alt_degrees” appended.
- Returns:
fset – setter method to use for the property definition.
- required()[source]
Iterate over all required UVParameter attributes.
- Yields:
UVParameter – required UVParameters on this object.
- extra()[source]
Iterate over all non-required UVParameter attributes.
- Yields:
UVParameter – optional (non-required) UVParameters on this object.
- check(*, check_extra=True, run_check_acceptability=True, ignore_requirements=False)[source]
Check that required parameters exist and have the correct shapes.
Optionally, check that the values are acceptable.
- Parameters:
check_extra (bool) – If true, check shapes and values on all parameters, otherwise only check required parameters.
run_check_acceptability (bool) – Option to check if values in parameters are acceptable.
ignore_requirements (bool) – Do not error if a required parameter isn’t set. This allows the user to run the shape/acceptability checks on parameters in a partially-defined UVData object.
- Returns:
bool – True if the checks pass.
- Raises:
ValueError – If required UVParameter values have not been set or if set UVParameters values do not have the expected names, shapes, types or values.
File Type Specific Classes
These classes inherit from pyuvdata’s user classes and hold the file type specific code. The read and write methods on the user classes convert between the user classes and the file-specific classes automatically as needed, so users generally do not need to interact with these classes, but developers may need to.
UVData Classes
- class pyuvdata.uvdata.fhd.FHD[source]
Defines a FHD-specific subclass of UVData for reading FHD save files.
This class should not be interacted with directly, instead use the read_fhd method on the UVData class.
- read_fhd(vis_files: list[str] | ndarray | str, *, params_file: str, obs_file: str | None = None, flags_file: str | None = None, layout_file: str | None = None, settings_file: str | None = None, background_lsts=True, read_data=True, run_check=True, check_extra=True, run_check_acceptability=True, strict_uvw_antpos_check=False, check_autos=True, fix_autos=True, use_future_array_shapes=None, astrometry_library=None)[source]
Read in data from a list of FHD files.
- Parameters:
vis_files (array_like of str) – The FHD data (or model) visibility save files. Can be None if read_data is False.
params_file (str) – The FHD params save file. Required.
obs_file (str) – The FHD obs save file. Required if read_data is False.
flags_file (str) – The FHD data (or model) flag save file. Required if read_data is True.
layout_file (str) – The FHD layout save file. Required for correct antenna metadata.
settings_file (str) – The FHD settings text file. Required for full history information.
background_lsts (bool) – When set to True, the lst_array is calculated in a background thread.
read_data (bool) – Read in the visibility, nsample and flag data. If set to False, only the metadata will be read in. Setting read_data to False results in a metadata only object. If read_data is False, an obs file must be included in the filelist. Note that if read_data is False, Npols is derived from the obs file and reflects the number of polarizations used in the FHD run. If read_data is True, Npols is given by the number of visibility data files provided in filelist.
run_check (bool) – Option to check for the existence and proper shapes of parameters after after reading in the file (the default is True, meaning the check will be run).
check_extra (bool) – Option to check optional parameters as well as required ones (the default is True, meaning the optional parameters will be checked).
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after reading in the file (the default is True, meaning the acceptable range check will be done).
strict_uvw_antpos_check (bool) – Option to raise an error rather than a warning if the check that uvws match antenna positions does not pass.
check_autos (bool) – Check whether any auto-correlations have non-zero imaginary values in data_array (which should not mathematically exist). Default is True.
fix_autos (bool) – If auto-correlations with imaginary values are found, fix those values so that they are real-only in data_array. Default is True.
use_future_array_shapes (bool) – Defunct option, will result in an error in version 3.2.
astrometry_library (str) – Library used for calculating LSTs. Allowed options are ‘erfa’ (which uses the pyERFA), ‘novas’ (which uses the python-novas library), and ‘astropy’ (which uses the astropy utilities). Default is erfa unless the telescope_location frame is MCMF (on the moon), in which case the default is astropy.
- Raises:
IOError – If root file directory doesn’t exist.
ValueError – If required files are missing or multiple files for any polarization are included in filelist. If there is no recognized key for visibility weights in the flags_file.
- class pyuvdata.uvdata.mir.Mir[source]
A class for Mir file objects.
This class defines an Mir-specific subclass of UVData for reading and writing Mir files. This class should not be interacted with directly, instead use the read_mir and write_mir methods on the UVData class.
- read_mir(filepath, *, antenna_nums=None, antenna_names=None, bls=None, time_range=None, lst_range=None, polarizations=None, catalog_names=None, corrchunk=None, receivers=None, sidebands=None, select_where=None, apply_tsys=True, apply_flags=True, apply_dedoppler=False, pseudo_cont=False, rechunk=None, compass_soln=None, swarm_only=True, codes_check=True, run_check=True, check_extra=True, run_check_acceptability=True, strict_uvw_antpos_check=False, allow_flex_pol=True, check_autos=True, fix_autos=True)[source]
Read in data from an SMA MIR file, and map to a UVData object.
Note that with the exception of filepath, most of the remaining parameters are used to sub-select a range of data.
- Parameters:
filepath (str) – The file path to the MIR folder to read from.
antenna_nums (array_like of int, optional) – The antennas numbers to include when reading data into the object (antenna positions and names for the removed antennas will be retained unless keep_all_metadata is False). This cannot be provided if antenna_names is also provided.
antenna_names (array_like of str, optional) – The antennas names to include when reading data into the object (antenna positions and names for the removed antennas will be retained unless keep_all_metadata is False). This cannot be provided if antenna_nums is also provided.
bls (list of tuple, optional) – A list of antenna number tuples (e.g. [(0, 1), (3, 2)]) specifying baselines to include when reading data in to the object.
time_range (array_like of float, optional) – The time range in Julian Date to include when reading data into the object, must be length 2. Some of the times in the file should fall between the first and last elements.
lst_range (array_like of float, optional) – The local sidereal time (LST) range in radians to keep in the object, must be of length 2. Some of the LSTs in the object should fall between the first and last elements. If the second value is smaller than the first, the LSTs are treated as having phase-wrapped around LST = 2*pi = 0, and the LSTs kept on the object will run from the larger value, through 0, and end at the smaller value.
polarizations (array_like of int, optional) – The polarizations numbers to include when reading data into the object, each value passed here should exist in the polarization_array.
catalog_names (str or array-like of str) – The names of the phase centers (sources) to include when reading data into the object, which should match exactly in spelling and capitalization.
corrchunk (int or array-like of int) – Correlator “chunk” (spectral window) to include when reading data into the object, where 0 corresponds to the pseudo-continuum channel.
receivers (str or array-like of str) – The names of the receivers (“230”, “240”, “345”, “400”) to include when reading data into the object.
sidebands (str or array-like of str) – The names of the sidebands (“l” for lower, “u” for upper) to include when reading data into the object.
select_where (tuple or list of tuples, optional) – Argument to pass to the MirParser.select method, which will downselect which data is read into the object.
apply_flags (bool) – If set to True, apply “wideband” flags to the visibilities, which are recorded by the realtime system to denote when data are expected to be bad (e.g., antennas not on source, dewar warm). Default it true.
apply_tsys (bool) – If set to False, data are returned as correlation coefficients (normalized by the auto-correlations). Default is True, which instead scales the raw visibilities and forward-gain of the antenna to produce values in Jy (uncalibrated).
apply_dedoppler (bool) – If set to True, data will be corrected for any doppler-tracking performed during observations, and brought into the topocentric rest frame (default for UVData objects). Default is False.
pseudo_cont (boolean) – Read in only pseudo-continuum values. Default is false.
rechunk (int) – Number of channels to average over when reading in the dataset. Optional argument, typically required to be a power of 2.
compass_soln (str) – Optional argument, specifying the path of COMPASS-derived flagging and bandpass gains solutions, which are applied prior to any potential spectral averaging (as triggered by using the rechunk keyword).
swarm_only (bool) – By default, only SMA SWARM data is loaded. If set to false, this will also enable loading of older ASIC data.
codes_check (bool) – Option to check the cross-check the internal metadata codes, and deselect data without valid matches, useful for automatically handling various data recording issues. Default is True.
run_check (bool) – Option to check for the existence and proper shapes of parameters before writing the file.
check_extra (bool) – Option to check optional parameters as well as required ones.
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters before writing the file.
strict_uvw_antpos_check (bool) – Option to raise an error rather than a warning if the check that uvw coordinates match antenna positions does not pass.
allow_flex_pol (bool) – If only one polarization per spectral window is read (and the polarization differs from window to window), allow for the UVData object to use “flexible polarization”, which compresses the polarization-axis of various attributes to be of length 1, sets the flex_spw_polarization_array attribute to define the polarization per spectral window. Default is True.
check_autos (bool) – Check whether any auto-correlations have non-zero imaginary values in data_array (which should not mathematically exist). Default is True.
fix_autos (bool) – If auto-correlations with imaginary values are found, fix those values so that they are real-only in data_array. Default is True.
- class pyuvdata.uvdata.miriad.Miriad[source]
Defines a Miriad-specific subclass of UVData for reading and writing Miriad files.
This class should not be interacted with directly, instead use the read and write_miriad methods on the UVData class.
- read_miriad(filepath, *, antenna_nums=None, ant_str=None, bls=None, polarizations=None, time_range=None, read_data=True, projected=None, correct_lat_lon=True, background_lsts=True, run_check=True, check_extra=True, run_check_acceptability=True, strict_uvw_antpos_check=False, calc_lst=True, fix_old_proj=False, fix_use_ant_pos=True, check_autos=True, fix_autos=True, use_future_array_shapes=None, astrometry_library=None)[source]
Read in data from a miriad file.
- Parameters:
filepath (str) – The miriad root directory to read from.
antenna_nums (array_like of int, optional) – The antennas numbers to read into the object.
bls (list of tuple, optional) – A list of antenna number tuples (e.g. [(0, 1), (3, 2)]) or a list of baseline 3-tuples (e.g. [(0, 1, ‘xx’), (2, 3, ‘yy’)]) specifying baselines to include when reading data into the object. For length-2 tuples, the ordering of the numbers within the tuple does not matter. For length-3 tuples, the polarization string is in the order of the two antennas. If length-3 tuples are provided, polarizations must be None.
ant_str (str, optional) – A string containing information about what antenna numbers and polarizations to include when reading data into the object. Can be ‘auto’, ‘cross’, ‘all’, or combinations of antenna numbers and polarizations (e.g. ‘1’, ‘1_2’, ‘1x_2y’). See tutorial for more examples of valid strings and the behavior of different forms for ant_str. If ‘1x_2y,2y_3y’ is passed, both polarizations ‘xy’ and ‘yy’ will be kept for both baselines (1, 2) and (2, 3) to return a valid pyuvdata object. An ant_str cannot be passed in addition to any of antenna_nums, bls or polarizations parameters, if it is a ValueError will be raised.
polarizations (array_like of int or str, optional) – List of polarization integers or strings to read-in. e.g. [‘xx’, ‘yy’, …]
time_range (list of float, optional) – len-2 list containing min and max range of times in Julian Date to include when reading data into the object. e.g. [2458115.20, 2458115.40]
read_data (bool) – Read in the uvws, times, visibility and flag data. If set to False, only the metadata that can be read quickly (without reading the data) will be read in. For Miriad, some of the normally required metadata are not fast to read in (e.g. uvws, times) so will not be read in if this keyword is False. Therefore, setting read_data to False results in an incompletely defined object (check will not pass).
projected (bool or None) – Option to force the dataset to be labelled as projected or unprojected regardless of the evidence in the file. The default is None which means that the projection will be set based on the file contents. Be careful setting this keyword unless you are confident about the contents of the file.
correct_lat_lon (bool) – Option to update the latitude and longitude from the known_telescopes list if the altitude is missing.
background_lsts (bool) – When set to True, the lst_array is calculated in a background thread.
run_check (bool) – Option to check for the existence and proper shapes of parameters after after reading in the file (the default is True, meaning the check will be run). Ignored if read_data is False.
check_extra (bool) – Option to check optional parameters as well as required ones (the default is True, meaning the optional parameters will be checked). Ignored if read_data is False.
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after reading in the file (the default is True, meaning the acceptable range check will be done). Ignored if read_data is False.
strict_uvw_antpos_check (bool) – Option to raise an error rather than a warning if the check that uvws match antenna positions does not pass.
calc_lst (bool) – Recalculate the LST values that are present within the file, useful in cases where the “online” calculate values have precision or value errors. Default is True.
fix_old_proj (bool) – Applies a fix to uvw-coordinates and phasing, assuming that the old phase method was used prior to writing the data, which had errors of the order of one part in 1e4 - 1e5. See the phasing memo for more details.
fix_use_ant_pos (bool) – If setting fix_old_proj to True, use the antenna positions to derive the correct uvw-coordinates rather than using the baseline vectors. Default is True.
check_autos (bool) – Check whether any auto-correlations have non-zero imaginary values in data_array (which should not mathematically exist). Default is True.
fix_autos (bool) – If auto-correlations with imaginary values are found, fix those values so that they are real-only in data_array. Default is True.
use_future_array_shapes (bool) – Defunct option, will result in an error in version 3.2.
astrometry_library (str) – Library used for calculating LSTs. Allowed options are ‘erfa’ (which uses the pyERFA), ‘novas’ (which uses the python-novas library), and ‘astropy’ (which uses the astropy utilities). Default is erfa unless the telescope_location frame is MCMF (on the moon), in which case the default is astropy.
- Raises:
IOError – If root file directory doesn’t exist.
ValueError – If incompatible select keywords are set (e.g. ant_str with other antenna selectors, times and time_range) or select keywords exclude all data or if keywords are set to the wrong type. If the metadata are not internally consistent.
- write_miriad(filepath, *, clobber=False, run_check=True, check_extra=True, run_check_acceptability=True, strict_uvw_antpos_check=False, no_antnums=False, calc_lst=False, check_autos=True, fix_autos=False)[source]
Write the data to a miriad file.
- Parameters:
filename (str) – The miriad root directory to write to.
clobber (bool) – Option to overwrite the filename if the file already exists.
run_check (bool) – Option to check for the existence and proper shapes of parameters after before writing the file (the default is True, meaning the check will be run).
check_extra (bool) – Option to check optional parameters as well as required ones (the default is True, meaning the optional parameters will be checked).
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters before writing the file (the default is True, meaning the acceptable range check will be done).
strict_uvw_antpos_check (bool) – Option to raise an error rather than a warning if the check that uvws match antenna positions does not pass.
no_antnums (bool) – Option to not write the antnums variable to the file. Should only be used for testing purposes.
calc_lst (bool) – Recalculate the LST values upon writing the file. This is done to perform higher-precision accounting for the difference in MIRAD timestamps vs pyuvdata (the former marks the beginning of an integration, the latter marks the midpoint). Default is False, which instead uses a simple formula for correcting the LSTs, expected to be accurate to approximately 0.1 µsec precision.
check_autos (bool) – Check whether any auto-correlations have non-zero imaginary values in data_array (which should not mathematically exist). Default is True.
fix_autos (bool) – If auto-correlations with imaginary values are found, fix those values so that they are real-only in data_array. Default is False.
- Raises:
ValueError – If the frequencies are not evenly spaced or are separated by more than their channel width.
TypeError – If any entry in extra_keywords is not a single string or number.
- class pyuvdata.uvdata.ms.MS[source]
Defines a class for reading and writing casa measurement sets.
This class should not be interacted with directly, instead use the read_ms method on the UVData class.
- write_ms(filepath, *, force_phase=False, model_data=None, corrected_data=None, flip_conj=False, clobber=False, run_check=True, check_extra=True, run_check_acceptability=True, strict_uvw_antpos_check=False, check_autos=True, fix_autos=False)[source]
Write a CASA measurement set (MS).
- Parameters:
filepath (str) – The MS file path to write to.
force_phase (bool) – Option to automatically phase drift scan data to zenith of the first timestamp.
model_data (ndarray) – Optional argument, which contains data to be written into the MODEL_DATA column of the measurement set (along with the data, which is written into the DATA column). Must contain the same dimensions as data_array.
corrected_data (ndarray) – Optional argument, which contains data to be written into the CORRECTED_DATA column of the measurement set (along with the data, which is written into the DATA column). Must contain the same dimensions as data_array.
clobber (bool) – Option to overwrite the file if it already exists.
flip_conj (bool) – If set to True, and the UVW coordinates are flipped (i.e., multiplied by -1) and the visibilities are complex conjugated prior to write, such that the data are written with the “opposite” conjugation scheme to what UVData normally uses. Note that this is only needed for specific subset of applications that read MS-formatted data, and should only be used by expert users. Default is False.
run_check (bool) – Option to check for the existence and proper shapes of parameters before writing the file.
check_extra (bool) – Option to check optional parameters as well as required ones.
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters before writing the file.
strict_uvw_antpos_check (bool) – Option to raise an error rather than a warning if the check that uvws match antenna positions does not pass.
check_autos (bool) – Check whether any auto-correlations have non-zero imaginary values in data_array (which should not mathematically exist). Default is True.
fix_autos (bool) – If auto-correlations with imaginary values are found, fix those values so that they are real-only in data_array. Default is False.
- read_ms(filepath, *, data_column='DATA', pol_order='AIPS', background_lsts=True, run_check=True, check_extra=True, run_check_acceptability=True, strict_uvw_antpos_check=False, ignore_single_chan=True, raise_error=True, read_weights=True, allow_flex_pol=False, check_autos=True, fix_autos=True, use_future_array_shapes=None, astrometry_library=None)[source]
Read in a casa measurement set.
- Parameters:
filepath (str) – The measurement set root directory to read from.
data_column (str) – name of CASA data column to read into data_array. Options are: ‘DATA’, ‘MODEL’, or ‘CORRECTED_DATA’
pol_order (str or None) – Option to specify polarizations order convention, options are ‘CASA’, ‘AIPS’, or None (no reordering). Default is ‘AIPS’.
background_lsts (bool) – When set to True, the lst_array is calculated in a background thread.
run_check (bool) – Option to check for the existence and proper shapes of parameters after after reading in the file (the default is True, meaning the check will be run).
check_extra (bool) – Option to check optional parameters as well as required ones (the default is True, meaning the optional parameters will be checked).
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after reading in the file (the default is True, meaning the acceptable range check will be done).
strict_uvw_antpos_check (bool) – Option to raise an error rather than a warning if the check that uvws match antenna positions does not pass.
ignore_single_chan (bool) – Some measurement sets (e.g., those from ALMA) use single channel spectral windows for recording pseudo-continuum channels or storing other metadata in the track when the telescopes are not on source. Because of the way the object is strutured (where all spectral windows are assumed to be simultaneously recorded), this can significantly inflate the size and memory footprint of UVData objects. By default, single channel windows are ignored to avoid this issue, although they can be included if setting this parameter equal to True.
raise_error (bool) – The measurement set format allows for different spectral windows and polarizations to have different metdata for the same time-baseline combination, but UVData objects do not. It also allows for timescales that are not supported by astropy. If any of these problems are detected, by default the reader will throw an error. However, if set to False, the reader will simply give a warning and try to do the best it can. If the problem is with differing metadata, it will use the first value read in the file as the “correct” metadata in the UVData object. If the problem is with the timescale, it will just assume UTC.
read_weights (bool) – Read in the weights from the MS file, default is True. If false, the method will set the nsamples_array to the same uniform value (namely 1.0).
allow_flex_pol (bool) – If only one polarization per spectral window is read (and the polarization differs from window to window), allow for the UVData object to use “flexible polarization”, which compresses the polarization-axis of various attributes to be of length 1, sets the flex_spw_polarization_array attribute to define the polarization per spectral window. Default is True.
check_autos (bool) – Check whether any auto-correlations have non-zero imaginary values in data_array (which should not mathematically exist). Default is True.
fix_autos (bool) – If auto-correlations with imaginary values are found, fix those values so that they are real-only in data_array. Default is True.
use_future_array_shapes (bool) – Defunct option, will result in an error in version 3.2.
astrometry_library (str) – Library used for calculating LSTs. Allowed options are ‘erfa’ (which uses the pyERFA), ‘novas’ (which uses the python-novas library), and ‘astropy’ (which uses the astropy utilities). Default is erfa unless the telescope_location frame is MCMF (on the moon), in which case the default is astropy.
- Raises:
IOError – If root file directory doesn’t exist.
ValueError – If the data_column is not set to an allowed value. If the data are have multiple subarrays or are multi source or have multiple spectral windows. If the data have multiple data description ID values.
- class pyuvdata.uvdata.mwa_corr_fits.MWACorrFITS[source]
UVData subclass for reading MWA correlator fits files.
This class should not be interacted with directly; instead use the read_mwa_corr_fits method on the UVData class.
- correct_cable_length(cable_lens, ant_1_inds, ant_2_inds)[source]
Apply a cable length correction to the data array.
- Parameters:
cable_lens (list of strings) – A list of strings containing the cable lengths for each antenna.
ant_1_inds (array) – An array of indices for antenna 1
ant_2_inds (array) – An array of indices for antenna 2
- flag_init(num_fine_chan, edge_width=80000.0, start_flag=2.0, end_flag=0.0, flag_dc_offset=True)[source]
Apply routine flagging to the MWA Correlator FITS file data.
Includes options to flag the coarse channel edges, beginning and end of obs, as well as the center fine channel of each coarse channel.
- Parameters:
edge_width (float) – The width to flag on the edge of each coarse channel, in hz. Set to 0 for no edge flagging.
start_flag (float) – The number of seconds to flag at the beginning of the observation. Set to 0 for no flagging.
end_flag (floats) – The number of seconds to flag at the end of the observation. Set to 0 for no flagging.
flag_dc_offset (bool) – Set to True to flag the center fine channel of each coarse channel.
- Raises:
ValueError – If edge_width is not an integer multiple of the channel_width of the data (0 also acceptable). If start_flag is not an integer multiple of the integration time (0 also acceptable). If end_flag is not an integer multiple of the integration time (0 also acceptable).
- van_vleck_correction(ant_1_inds, ant_2_inds, flagged_ant_inds, cheby_approx, data_array_dtype)[source]
Apply a van vleck correction to the data array.
For an explanation of the Van Vleck corrections used and their implementation in this code, see the memos at https://github.com/EoRImaging/Memos/blob/master/PDFs/007_Van_Vleck_A.pdf and https://github.com/EoRImaging/Memos/blob/master/PDFs/008_Van_Vleck_B.pdf
- Parameters:
ant_1_inds (array) – An array of indices for antenna 1.
ant_2_inds (array) – An array of indices for antenna 2.
flagged_ant_inds (numpy array of type int) – List of indices of flagged antennas.
cheby_approx (bool) – Option to implement the van vleck correction with a chebyshev polynomial. approximation.
data_array_dtype (numpy dtype) – Datatype to store the output data_array as.
- read_mwa_corr_fits(filelist, *, use_aoflagger_flags=None, remove_dig_gains=True, remove_coarse_band=True, correct_cable_len=True, correct_van_vleck=False, cheby_approx=True, flag_small_auto_ants=True, phase_to_pointing_center=False, propagate_coarse_flags=True, flag_init=True, edge_width=80000.0, start_flag='goodtime', end_flag=0.0, flag_dc_offset=True, remove_flagged_ants=True, background_lsts=True, read_data=True, data_array_dtype=<class 'numpy.complex64'>, nsample_array_dtype=<class 'numpy.float32'>, run_check=True, check_extra=True, run_check_acceptability=True, strict_uvw_antpos_check=False, check_autos=True, fix_autos=True, use_future_array_shapes=None, astrometry_library=None)[source]
Read in MWA correlator gpu box files.
The default settings remove some of the instrumental effects in the bandpass by dividing out the coarse band shape (for legacy data only) and the digital gains, and applying a cable length correction. If the desired output is raw correlator data, set remove_dig_gains=False, remove_coarse_band=False, correct_cable_len=False, and phase_to_pointing_center=False.
- Parameters:
filelist (list of str) – The list of MWA correlator files to read from. Must include at least one fits file and only one metafits file per data set.
use_aoflagger_flags (bool) – Option to use aoflagger mwaf flag files. Defaults to true if aoflagger flag files are submitted.
remove_dig_gains (bool) – Option to divide out digital gains.
remove_coarse_band (bool) – Option to divide out coarse band shape.
correct_cable_len (bool) – Option to apply a cable delay correction.
correct_van_vleck (bool) – Option to apply a van vleck correction.
cheby_approx (bool) – Only used if correct_van_vleck is True. Option to implement the van vleck correction with a chebyshev polynomial approximation.
flag_small_auto_ants (bool) – Only used if correct_van_vleck is True. Option to completely flag any antenna for which the autocorrelation falls below a threshold found by the Van Vleck correction to indicate bad data. Specifically, the threshold used is 0.5 * integration_time * channel_width. If set to False, only the times and frequencies at which the auto is below the threshold will be flagged for the antenna.
phase_to_pointing_center (bool) – Option to phase to the observation pointing center.
propagate_coarse_flags (bool) – Option to propagate flags for missing coarse channel integrations across frequency.
flag_init (bool) – Set to True in order to do routine flagging of coarse channel edges, start or end integrations, or the center fine channel of each coarse channel. See associated keywords.
edge_width (float) – Only used if flag_init is True. The width to flag on the edge of each coarse channel, in hz. Errors if not equal to integer multiple of channel_width. Set to 0 for no edge flagging.
start_flag (float or str) – Only used if flag_init is True. The number of seconds to flag at the beginning of the observation. Set to 0 for no flagging. Default is ‘goodtime’, which uses information in the metafits file to determine the length of time that should be flagged. Errors if input is not a float or ‘goodtime’. Errors if float input is not equal to an integer multiple of the integration time.
end_flag (floats) – Only used if flag_init is True. The number of seconds to flag at the end of the observation. Set to 0 for no flagging. Errors if not equal to an integer multiple of the integration time.
flag_dc_offset (bool) – Only used if flag_init is True. Set to True to flag the center fine channel of each coarse channel.
remove_flagged_ants (bool) – Option to perform a select to remove antennas flagged in the metafits file. If correct_van_vleck and flag_small_auto_ants are both True then antennas flagged by the Van Vleck correction are also removed.
background_lsts (bool) – When set to True, the lst_array is calculated in a background thread.
read_data (bool) – Read in the visibility, nsample and flag data. If set to False, only the metadata will be read in. Setting read_data to False results in a metadata only object.
data_array_dtype (numpy dtype) – Datatype to store the output data_array as. Must be either np.complex64 (single-precision real and imaginary) or np.complex128 (double-precision real and imaginary).
nsample_array_dtype (numpy dtype) – Datatype to store the output nsample_array as. Must be either np.float64 (double-precision), np.float32 (single-precision), or np.float16 (half-precision). Half-precision is only recommended for cases where no sampling or averaging of baselines will occur, because round-off errors can be quite large (~1e-3).
run_check (bool) – Option to check for the existence and proper shapes of parameters after after reading in the file (the default is True, meaning the check will be run).
check_extra (bool) – Option to check optional parameters as well as required ones (the default is True, meaning the optional parameters will be checked).
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after reading in the file (the default is True, meaning the acceptable range check will be done).
strict_uvw_antpos_check (bool) – Option to raise an error rather than a warning if the check that uvws match antenna positions does not pass.
check_autos (bool) – Check whether any auto-correlations have non-zero imaginary values in data_array (which should not mathematically exist). Default is True.
fix_autos (bool) – If auto-correlations with imaginary values are found, fix those values so that they are real-only in data_array. Default is True.
use_future_array_shapes (bool) – Defunct option, will result in an error in version 3.2.
astrometry_library (str) – Library used for calculating LSTs. Allowed options are ‘erfa’ (which uses the pyERFA), ‘novas’ (which uses the python-novas library), and ‘astropy’ (which uses the astropy utilities). Default is erfa unless the telescope_location frame is MCMF (on the moon), in which case the default is astropy.
- Raises:
ValueError – If required files are missing or multiple files metafits files are included in filelist. If files from different observations are included in filelist. If files in fileslist have different fine channel widths If file types other than fits, metafits, and mwaf files are included in filelist.
- class pyuvdata.uvdata.uvfits.UVFITS[source]
Defines a uvfits-specific subclass of UVData for reading and writing uvfits.
This class should not be interacted with directly, instead use the read_uvfits and write_uvfits methods on the UVData class.
- read_uvfits(filename, *, antenna_nums=None, antenna_names=None, ant_str=None, bls=None, frequencies=None, freq_chans=None, times=None, time_range=None, lsts=None, lst_range=None, polarizations=None, blt_inds=None, phase_center_ids=None, catalog_names=None, keep_all_metadata=True, read_data=True, background_lsts=True, run_check=True, check_extra=True, run_check_acceptability=True, strict_uvw_antpos_check=False, fix_old_proj=False, fix_use_ant_pos=True, check_autos=True, fix_autos=True, use_future_array_shapes=None, astrometry_library=None)[source]
Read in header, metadata and data from a uvfits file.
- Parameters:
filename (str) – The uvfits file to read from.
antenna_nums (array_like of int, optional) – The antennas numbers to include when reading data into the object (antenna positions and names for the removed antennas will be retained unless keep_all_metadata is False). This cannot be provided if antenna_names is also provided. Ignored if read_data is False.
antenna_names (array_like of str, optional) – The antennas names to include when reading data into the object (antenna positions and names for the removed antennas will be retained unless keep_all_metadata is False). This cannot be provided if antenna_nums is also provided. Ignored if read_data is False.
bls (list of tuple, optional) – A list of antenna number tuples (e.g. [(0, 1), (3, 2)]) or a list of baseline 3-tuples (e.g. [(0, 1, ‘xx’), (2, 3, ‘yy’)]) specifying baselines to include when reading data into the object. For length-2 tuples, the ordering of the numbers within the tuple does not matter. For length-3 tuples, the polarization string is in the order of the two antennas. If length-3 tuples are provided, polarizations must be None. Ignored if read_data is False.
ant_str (str, optional) – A string containing information about what antenna numbers and polarizations to include when reading data into the object. Can be ‘auto’, ‘cross’, ‘all’, or combinations of antenna numbers and polarizations (e.g. ‘1’, ‘1_2’, ‘1x_2y’). See tutorial for more examples of valid strings and the behavior of different forms for ant_str. If ‘1x_2y,2y_3y’ is passed, both polarizations ‘xy’ and ‘yy’ will be kept for both baselines (1, 2) and (2, 3) to return a valid pyuvdata object. An ant_str cannot be passed in addition to any of antenna_nums, antenna_names, bls args or the polarizations parameters, if it is a ValueError will be raised. Ignored if read_data is False.
frequencies (array_like of float, optional) – The frequencies to include when reading data into the object, each value passed here should exist in the freq_array.
freq_chans (array_like of int, optional) – The frequency channel numbers to include when reading data into the object. Ignored if read_data is False.
times (array_like of float, optional) – The times to include when reading data into the object, each value passed here should exist in the time_array. Cannot be used with time_range, lsts, or lst_array.
time_range (array_like of float, optional) – The time range in Julian Date to keep in the object, must be length 2. Some of the times in the object should fall between the first and last elements. Cannot be used with times.
lsts (array_like of float, optional) – The local sidereal times (LSTs) to keep in the object, each value passed here should exist in the lst_array. Cannot be used with times, time_range, or lst_range.
lst_range (array_like of float, optional) – The local sidereal time (LST) range in radians to keep in the object, must be of length 2. Some of the LSTs in the object should fall between the first and last elements. If the second value is smaller than the first, the LSTs are treated as having phase-wrapped around LST = 2*pi = 0, and the LSTs kept on the object will run from the larger value, through 0, and end at the smaller value.
polarizations (array_like of int, optional) – The polarizations numbers to include when reading data into the object, each value passed here should exist in the polarization_array. Ignored if read_data is False.
blt_inds (array_like of int, optional) – The baseline-time indices to include when reading data into the object. This is not commonly used. Ignored if read_data is False.
phase_center_ids (array_like of int, optional) – Phase center IDs to include when reading data into the object (effectively a selection on baseline-times). Cannot be used with catalog_names.
catalog_names (str or array-like of str, optional) – The names of the phase centers (sources) to include when reading data into the object, which should match exactly in spelling and capitalization. Cannot be used with phase_center_ids.
keep_all_metadata (bool) – Option to keep all the metadata associated with antennas, even those that do not have data associated with them after the select option.
read_data (bool) – Read in the visibility, nsample and flag data. If set to False, only the metadata will be read in. Setting read_data to False results in a metadata only object.
background_lsts (bool) – When set to True, the lst_array is calculated in a background thread.
run_check (bool) – Option to check for the existence and proper shapes of parameters after after reading in the file (the default is True, meaning the check will be run). Ignored if read_data is False.
check_extra (bool) – Option to check optional parameters as well as required ones (the default is True, meaning the optional parameters will be checked). Ignored if read_data is False.
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after reading in the file (the default is True, meaning the acceptable range check will be done). Ignored if read_data is False.
strict_uvw_antpos_check (bool) – Option to raise an error rather than a warning if the check that uvws match antenna positions does not pass.
fix_old_proj (bool) – Applies a fix to uvw-coordinates and phasing, assuming that the old phase method was used prior to writing the data, which had errors of the order of one part in 1e4 - 1e5. See the phasing memo for more details. Default is False.
fix_use_ant_pos (bool) – If setting fix_old_proj to True, use the antenna positions to derive the correct uvw-coordinates rather than using the baseline vectors. Default is True.
check_autos (bool) – Check whether any auto-correlations have non-zero imaginary values in data_array (which should not mathematically exist). Default is True.
fix_autos (bool) – If auto-correlations with imaginary values are found, fix those values so that they are real-only in data_array. Default is True.
use_future_array_shapes (bool) – Defunct option, will result in an error in version 3.2.
astrometry_library (str) – Library used for calculating LSTs. Allowed options are ‘erfa’ (which uses the pyERFA), ‘novas’ (which uses the python-novas library), and ‘astropy’ (which uses the astropy utilities). Default is erfa unless the telescope_location frame is MCMF (on the moon), in which case the default is astropy.
- Raises:
IOError – If filename doesn’t exist.
ValueError – If incompatible select keywords are set (e.g. ant_str with other antenna selectors, times and time_range) or select keywords exclude all data or if keywords are set to the wrong type. If the metadata are not internally consistent or missing.
- write_uvfits(filename, *, write_lst=True, force_phase=False, run_check=True, check_extra=True, run_check_acceptability=True, strict_uvw_antpos_check=False, check_autos=True, fix_autos=False, use_miriad_convention=False)[source]
Write the data to a uvfits file.
If using this method to write out a data set for import into CASA, users should be aware that the importuvifts task does not currently support reading in data sets where the number of antennas is > 255. If writing out such a data set for use in CASA, we suggest using the measurement set writer (UVData.write_ms) instead.
- Parameters:
filename (str) – The uvfits file to write to.
write_lst (bool) – Option to write the LSTs to the metadata (random group parameters).
force_phase (bool) – Option to automatically phase drift scan data to zenith of the first timestamp.
run_check (bool) – Option to check for the existence and proper shapes of parameters before writing the file.
check_extra (bool) – Option to check optional parameters as well as required ones.
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters before writing the file.
strict_uvw_antpos_check (bool) – Option to raise an error rather than a warning if the check that uvws match antenna positions does not pass.
check_autos (bool) – Check whether any auto-correlations have non-zero imaginary values in data_array (which should not mathematically exist). Default is True.
fix_autos (bool) – If auto-correlations with imaginary values are found, fix those values so that they are real-only in data_array. Default is False.
use_miriad_convention (bool) – Option to use the MIRIAD baseline convention, and write to BASELINE column. This mode is required for UVFITS files with >256 antennas to be readable by MIRIAD, and supports up to 2048 antennas. The MIRIAD baseline ID is given by bl = 256 * ant1 + ant2 if ant2 < 256, otherwise bl = 2048 * ant1 + ant2 + 2**16. Note MIRIAD uses 1-indexed antenna IDs, but this code accepts 0-based.
- Raises:
ValueError – The object contains unprojected data and force_phase keyword is not set. If the frequencies are not evenly spaced or are separated by more than their channel width. The polarization values are not evenly spaced. If the timesys parameter is set to anything other than “UTC” or None.
TypeError – If any entry in extra_keywords is not a single string or number.
- class pyuvdata.uvdata.uvh5.UVH5[source]
A class for UVH5 file objects.
This class defines an HDF5-specific subclass of UVData for reading and writing UVH5 files. This class should not be interacted with directly, instead use the read_uvh5 and write_uvh5 methods on the UVData class.
- read_uvh5(filename, *, antenna_nums=None, antenna_names=None, ant_str=None, bls=None, frequencies=None, freq_chans=None, times=None, time_range=None, lsts=None, lst_range=None, polarizations=None, blt_inds=None, phase_center_ids=None, catalog_names=None, keep_all_metadata=True, read_data=True, data_array_dtype=<class 'numpy.complex128'>, multidim_index=False, remove_flex_pol=True, background_lsts=True, run_check=True, check_extra=True, run_check_acceptability=True, strict_uvw_antpos_check=False, fix_old_proj=None, fix_use_ant_pos=True, check_autos=True, fix_autos=True, use_future_array_shapes=None, blt_order: tuple[str] | ~typing.Literal['determine'] | None = None, blts_are_rectangular: bool | None = None, time_axis_faster_than_bls: bool | None = None, recompute_nbls: bool | None = None, astrometry_library: str | None = None)[source]
Read in data from a UVH5 file.
- Parameters:
filename (str) – The UVH5 file to read from.
antenna_nums (array_like of int, optional) – The antennas numbers to include when reading data into the object (antenna positions and names for the removed antennas will be retained unless keep_all_metadata is False). This cannot be provided if antenna_names is also provided. Ignored if read_data is False.
antenna_names (array_like of str, optional) – The antennas names to include when reading data into the object (antenna positions and names for the removed antennas will be retained unless keep_all_metadata is False). This cannot be provided if antenna_nums is also provided. Ignored if read_data is False.
bls (list of tuple, optional) – A list of antenna number tuples (e.g. [(0, 1), (3, 2)]) or a list of baseline 3-tuples (e.g. [(0, 1, ‘xx’), (2, 3, ‘yy’)]) specifying baselines to include when reading data into the object. For length-2 tuples, the ordering of the numbers within the tuple does not matter. For length-3 tuples, the polarization string is in the order of the two antennas. If length-3 tuples are provided, polarizations must be None. Ignored if read_data is False.
ant_str (str, optional) – A string containing information about what antenna numbers and polarizations to include when reading data into the object. Can be ‘auto’, ‘cross’, ‘all’, or combinations of antenna numbers and polarizations (e.g. ‘1’, ‘1_2’, ‘1x_2y’). See tutorial for more examples of valid strings and the behavior of different forms for ant_str. If ‘1x_2y,2y_3y’ is passed, both polarizations ‘xy’ and ‘yy’ will be kept for both baselines (1, 2) and (2, 3) to return a valid pyuvdata object. An ant_str cannot be passed in addition to any of antenna_nums, antenna_names, bls args or the polarizations parameters, if it is a ValueError will be raised. Ignored if read_data is False.
frequencies (array_like of float, optional) – The frequencies to include when reading data into the object, each value passed here should exist in the freq_array. Ignored if read_data is False.
freq_chans (array_like of int, optional) – The frequency channel numbers to include when reading data into the object. Ignored if read_data is False.
times (array_like of float, optional) – The times to include when reading data into the object, each value passed here should exist in the time_array. Cannot be used with time_range, lsts, or lst_array.
time_range (array_like of float, optional) – The time range in Julian Date to include when reading data into the object, must be length 2. Some of the times in the file should fall between the first and last elements. Cannot be used with times.
lsts (array_like of float, optional) – The local sidereal times (LSTs) to keep in the object, each value passed here should exist in the lst_array. Cannot be used with times, time_range, or lst_range.
lst_range (array_like of float, optional) – The local sidereal time (LST) range in radians to keep in the object, must be of length 2. Some of the LSTs in the object should fall between the first and last elements. If the second value is smaller than the first, the LSTs are treated as having phase-wrapped around LST = 2*pi = 0, and the LSTs kept on the object will run from the larger value, through 0, and end at the smaller value.
polarizations (array_like of int, optional) – The polarizations numbers to include when reading data into the object, each value passed here should exist in the polarization_array. Ignored if read_data is False.
blt_inds (array_like of int, optional) – The baseline-time indices to include when reading data into the object. This is not commonly used. Ignored if read_data is False.
phase_center_ids (array_like of int, optional) – Phase center IDs to include when reading data into the object (effectively a selection on baseline-times). Cannot be used with catalog_names.
catalog_names (str or array-like of str, optional) – The names of the phase centers (sources) to include when reading data into the object, which should match exactly in spelling and capitalization. Cannot be used with phase_center_ids.
keep_all_metadata (bool) – Option to keep all the metadata associated with antennas, even those that do not have data associated with them after the select option.
read_data (bool) – Read in the visibility, nsample and flag data. If set to False, only the metadata will be read in. Setting read_data to False results in a metadata only object.
data_array_dtype (numpy dtype) – Datatype to store the output data_array as. Must be either np.complex64 (single-precision real and imaginary) or np.complex128 (double- precision real and imaginary). Only used if the datatype of the visibility data on-disk is not ‘c8’ or ‘c16’.
multidim_index (bool) – If True, attempt to index the HDF5 dataset simultaneously along all data axes. Otherwise index one axis at-a-time. This only works if data selection is sliceable along all but one axis. If indices are not well-matched to data chunks, this can be slow.
remove_flex_pol (bool) – If True and if the file is a flex_pol file, convert back to a standard UVData object.
background_lsts (bool) – When set to True, the lst_array is calculated in a background thread.
run_check (bool) – Option to check for the existence and proper shapes of parameters after after reading in the file (the default is True, meaning the check will be run). Ignored if read_data is False.
check_extra (bool) – Option to check optional parameters as well as required ones (the default is True, meaning the optional parameters will be checked). Ignored if read_data is False.
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after reading in the file (the default is True, meaning the acceptable range check will be done). Ignored if read_data is False.
strict_uvw_antpos_check (bool) – Option to raise an error rather than a warning if the check that uvws match antenna positions does not pass.
fix_old_proj (bool) – Applies a fix to uvw-coordinates and phasing, assuming that the old phase method was used prior to writing the data, which had errors of the order of one part in 1e4 - 1e5. See the phasing memo for more details. Default is to apply the correction if the attributes phase_center_app_ra and phase_center_app_dec are missing (as they were introduced alongside the new phasing method).
fix_use_ant_pos (bool) – If setting fix_old_proj to True, use the antenna positions to derive the correct uvw-coordinates rather than using the baseline vectors. Default is True.
check_autos (bool) – Check whether any auto-correlations have non-zero imaginary values in data_array (which should not mathematically exist). Default is True.
fix_autos (bool) – If auto-correlations with imaginary values are found, fix those values so that they are real-only in data_array. Default is True.
use_future_array_shapes (bool) – Defunct option, will result in an error in version 3.2.
blt_order (tuple of str or “determine”, optional) – The order of the baseline-time axis in the file. This can be determined, or read directly from file, however since it has been optional in the past, many existing files do not contain it in the metadata. Some reading operations are significantly faster if this is known, so providing it here can provide a speedup. Default is to try and read it from file, and if not there, just leave it as None. Set to “determine” to auto-detect the blt_order from the metadata (takes extra time to do so).
blts_are_rectangular (bool, optional) – Whether the baseline-time axis is rectangular. This can be read from metadata in new files, but many old files do not contain it. If not provided, the rectangularity will be determined from the data. This is a non-negligible operation, so if you know it, it can be provided here to speed up reading.
time_axis_faster_than_bls (bool, optional) – If blts are rectangular, this variable specifies whether the time axis is the fastest-moving virtual axis. Various reading functions benefit from knowing this, so if it is known, it can be provided here to speed up reading. It will be determined from the data if not provided.
recompute_nbls (bool, optional) – Whether to recompute the number of unique baselines from the data. Before v1.2 of the UVH5 spec, it was possible to have an incorrect number of baselines in the header without error, so this provides an opportunity to rectify it. Old HERA files (< March 2023) may have this issue, but in this case the correct number of baselines can be computed more quickly than by fully re=computing, and so we do this.
astrometry_library (str) – Library used for calculating LSTs. Allowed options are ‘erfa’ (which uses the pyERFA), ‘novas’ (which uses the python-novas library), and ‘astropy’ (which uses the astropy utilities). Default is erfa unless the telescope_location frame is MCMF (on the moon), in which case the default is astropy.
- Returns:
None
- Raises:
IOError – If filename doesn’t exist.
ValueError – If the data_array_dtype is not a complex dtype. If incompatible select keywords are set (e.g. ant_str with other antenna selectors, times and time_range) or select keywords exclude all data or if keywords are set to the wrong type.
- write_uvh5(filename, *, clobber=False, chunks=True, data_compression=None, flags_compression='lzf', nsample_compression='lzf', data_write_dtype=None, add_to_history=None, run_check=True, check_extra=True, run_check_acceptability=True, strict_uvw_antpos_check=False, check_autos=True, fix_autos=False)[source]
Write an in-memory UVData object to a UVH5 file.
- Parameters:
filename (str) – The UVH5 file to write to.
clobber (bool) – Option to overwrite the file if it already exists.
chunks (tuple or bool) – h5py.create_dataset chunks keyword. Tuple for chunk shape, True for auto-chunking, None for no chunking. Default is True.
data_compression (str) – HDF5 filter to apply when writing the data_array. Default is None (no filter/compression). In addition to the normal HDF5 filter values, the user may specify “bitshuffle” which will set the compression to 32008 for bitshuffle and will set the compression_opts to (0, 2) to allow bitshuffle to automatically determine the block size and to use the LZF filter after bitshuffle. Using bitshuffle requires having the hdf5plugin package installed. Dataset must be chunked to use compression.
flags_compression (str) – HDF5 filter to apply when writing the flags_array. Default is the LZF filter. Dataset must be chunked.
nsample_compression (str) – HDF5 filter to apply when writing the nsample_array. Default is the LZF filter. Dataset must be chunked.
data_write_dtype (numpy dtype) – The datatype of output visibility data. If ‘None’, then the same datatype as data_array will be used. The user may specify ‘c8’ for single-precision floats or ‘c16’ for double-presicion. Otherwise, a numpy dtype object must be specified with an ‘r’ field and an ‘i’ field for real and imaginary parts, respectively. See uvh5.py for an example of defining such a datatype.
run_check (bool) – Option to check for the existence and proper shapes of parameters before writing the file.
check_extra (bool) – Option to check optional parameters as well as required ones.
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters before writing the file.
strict_uvw_antpos_check (bool) – Option to raise an error rather than a warning if the check that uvws match antenna positions does not pass.
check_autos (bool) – Check whether any auto-correlations have non-zero imaginary values in data_array (which should not mathematically exist). Default is True.
fix_autos (bool) – If auto-correlations with imaginary values are found, fix those values so that they are real-only in data_array. Default is False.
- Returns:
None
- Raises:
IOError – If the file located at filename already exists and clobber=False, an IOError is raised.
Notes
The HDF5 library allows for the application of “filters” when writing data, which can provide moderate to significant levels of compression for the datasets in question. Testing has shown that for some typical cases of UVData objects (empty/sparse flag_array objects, and/or uniform nsample_arrays), the built-in LZF filter provides significant compression for minimal computational overhead.
Note that for typical HERA data files written after mid-2020, the bitshuffle filter was applied to the data_array. Because of the lack of portability, it is not included as an option here; in the future, it may be added. Note that as long as bitshuffle is installed on the system in a way that h5py can find it, no action needs to be taken to _read_ a data_array encoded with bitshuffle (or an error will be raised).
- initialize_uvh5_file(filename, *, clobber=False, chunks=True, data_compression=None, flags_compression='lzf', nsample_compression='lzf', data_write_dtype=None)[source]
Initialize a UVH5 file on disk to be written to in parts.
- Parameters:
filename (str) – The UVH5 file to write to.
clobber (bool) – Option to overwrite the file if it already exists.
chunks (tuple or bool) – h5py.create_dataset chunks keyword. Tuple for chunk shape, True for auto-chunking, None for no chunking. Default is True.
data_compression (str) – HDF5 filter to apply when writing the data_array. Default is None (no filter/compression). In addition to the normal HDF5 filter values, the user may specify “bitshuffle” which will set the compression to 32008 for bitshuffle and will set the compression_opts to (0, 2) to allow bitshuffle to automatically determine the block size and to use the LZF filter after bitshuffle. Using bitshuffle requires having the hdf5plugin package installed. Dataset must be chunked to use compression.
flags_compression (str) – HDF5 filter to apply when writing the flags_array. Default is the LZF filter. Dataset must be chunked.
nsample_compression (str) – HDF5 filter to apply when writing the nsample_array. Default is the LZF filter. Dataset must be chunked.
data_write_dtype (str or numpy dtype) – The datatype of output visibility data. If ‘None’, then double- precision floats will be used. The user may specify ‘c8’ for single-precision floats or ‘c16’ for double-presicion. Otherwise, a numpy dtype object must be specified with an ‘r’ field and an ‘i’ field for real and imaginary parts, respectively. See uvh5.py for an example of defining such a datatype.
- Returns:
None
- Raises:
IOError – If the file located at filename already exists and clobber=False, an IOError is raised.
Notes
When partially writing out data, this function should be called first to initialize the file on disk. The data is then actually written by calling the write_uvh5_part method, with the same filename as the one specified in this function. See the tutorial for a worked example.
The HDF5 library allows for the application of “filters” when writing data, which can provide moderate to significant levels of compression for the datasets in question. Testing has shown that for some typical cases of UVData objects (empty/sparse flag_array objects, and/or uniform nsample_arrays), the built-in LZF filter provides significant compression for minimal computational overhead.
Note that for typical HERA data files written after mid-2018, the bitshuffle filter was applied to the data_array. Because of the lack of portability, it is not included as an option here; in the future, it may be added. Note that as long as bitshuffle is installed on the system in a way that h5py can find it, no action needs to be taken to _read_ a data_array encoded with bitshuffle (or an error will be raised).
- write_uvh5_part(filename, *, data_array, flag_array, nsample_array, check_header=True, antenna_nums=None, antenna_names=None, ant_str=None, bls=None, frequencies=None, freq_chans=None, times=None, time_range=None, lsts=None, lst_range=None, polarizations=None, blt_inds=None, phase_center_ids=None, catalog_names=None, run_check_acceptability=True, add_to_history=None)[source]
Write out a part of a UVH5 file that has been previously initialized.
- Parameters:
filename (str) – The file on disk to write data to. It must already exist, and is assumed to have been initialized with initialize_uvh5_file.
data_array (array of float) – The data to write to disk. A check is done to ensure that the dimensions of the data passed in conform to the ones specified by the “selection” arguments.
flag_array (array of bool) – The flags array to write to disk. A check is done to ensure that the dimensions of the data passed in conform to the ones specified by the “selection” arguments.
nsample_array (array of float) – The nsample array to write to disk. A check is done to ensure that the dimensions fo the data passed in conform to the ones specified by the “selection” arguments.
check_header (bool) – Option to check that the metadata present in the header on disk matches that in the object.
run_check_acceptability (bool) – If check_header, additional option to check acceptable range of the values of parameters after reading in the file.
antenna_nums (array_like of int, optional) – The antennas numbers to include when writing data into the object (antenna positions and names for the excluded antennas will be retained). This cannot be provided if antenna_names is also provided.
antenna_names (array_like of str, optional) – The antennas names to include when writing data into the object (antenna positions and names for the excluded antennas will be retained). This cannot be provided if antenna_nums is also provided.
bls (list of tuples, optional) – A list of antenna number tuples (e.g. [(0, 1), (3, 2)]) or a list of baseline 3-tuples (e.g. [(0, 1, ‘xx’), (2, 3, ‘yy’)]) specifying baselines to write to the file. For length-2 tuples, the ordering of the numbers within the tuple does not matter. For length-3 tuples, the polarization string is in the order of the two antennas. If length-3 tuples are provided, the polarizations argument below must be None.
ant_str (str, optional) – A string containing information about what antenna numbers and polarizations to include when writing data into the object. Can be ‘auto’, ‘cross’, ‘all’, or combinations of antenna numbers and polarizations (e.g. ‘1’, ‘1_2’, ‘1x_2y’). See tutorial for more examples of valid strings and the behavior of different forms for ant_str. If ‘1x_2y,2y_3y’ is passed, both polarizations ‘xy’ and ‘yy’ will be written for both baselines (1, 2) and (2, 3) to reflect a valid pyuvdata object. An ant_str cannot be passed in addition to any of the above antenna args or the polarizations arg.
frequencies (array_like of float, optional) – The frequencies to include when writing data to the file.
freq_chans (array_like of int, optional) – The frequency channel numbers to include when writing data to the file.
times (array_like of float, optional) – The times in Julian Day to include when writing data to the file.
time_range (array_like of float, optional) – The time range in Julian Date to include when writing data to the file, must be length 2. Some of the times in the object should fall between the first and last elements. Cannot be used with times.
lsts (array_like of float, optional) – The local sidereal times (LSTs) to keep in the object, each value passed here should exist in the lst_array. Cannot be used with times, time_range, or lst_range.
lst_range (array_like of float, optional) – The local sidereal time (LST) range in radians to keep in the object, must be of length 2. Some of the LSTs in the object should fall between the first and last elements. If the second value is smaller than the first, the LSTs are treated as having phase-wrapped around LST = 2*pi = 0, and the LSTs kept on the object will run from the larger value, through 0, and end at the smaller value.
polarizations (array_like of int, optional) – The polarizations to include when writing data to the file.
blt_inds (array_like of int, optional) – The baseline-time indices to include when writing data to the file. This is not commonly used.
phase_center_ids (array_like of int, optional) – Phase center IDs to include when reading data into the object (effectively a selection on baseline-times). Cannot be used with catalog_names.
catalog_names (str or array-like of str) – The names of the phase centers (sources) to include when reading data into the object, which should match exactly in spelling and capitalization. Cannot be used with phase_center_ids.
add_to_history (str) – String to append to history before write out. Default is no appending.
- Returns:
None
- Raises:
AssertionError – An AsserionError is raised if: (1) the location specified by filename does not exist; (2) the data_array, flag_array, and nsample_array do not all have the same shape; (3) the shape of the data arrays do not correspond to the sizes specified by the properties to write out.
Notes
When partially writing out data, this function should be called after calling initialize_uvh5_file. The same filename is passed in, with an optional check to ensure that the object’s metadata in-memory matches the header on-disk. See the tutorial for a worked example.
UVCal Classes
- class pyuvdata.uvcal.calfits.CALFITS[source]
Defines a calfits-specific class for reading and writing calfits files.
This class should not be interacted with directly, instead use the read_calfits and write_calfits methods on the UVCal class.
- write_calfits(filename, *, run_check=True, check_extra=True, run_check_acceptability=True, clobber=False)[source]
Write the data to a calfits file.
- Parameters:
filename (str) – The calfits file to write to.
run_check (bool) – Option to check for the existence and proper shapes of parameters before writing the file.
check_extra (bool) – Option to check optional parameters as well as required ones.
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters before writing the file.
clobber (bool) – Option to overwrite the filename if the file already exists.
- read_calfits(filename, *, read_data=True, background_lsts=True, run_check=True, check_extra=True, run_check_acceptability=True, use_future_array_shapes=None, astrometry_library=None)[source]
Read data from a calfits file.
- Parameters:
filename (str) – The calfits file to read from.
read_data (bool) – Read in the gains or delays, quality arrays and flag arrays. If set to False, only the metadata will be read in. Setting read_data to False results in a metadata only object.
background_lsts (bool) – When set to True, the lst_array is calculated in a background thread.
run_check (bool) – Option to check for the existence and proper shapes of parameters after reading in the file.
check_extra (bool) – Option to check optional parameters as well as required ones.
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after reading in the file.
use_future_array_shapes (bool) – Defunct option, will result in an error in version 3.2.
astrometry_library (str) – Library used for calculating LSTs. Allowed options are ‘erfa’ (which uses the pyERFA), ‘novas’ (which uses the python-novas library), and ‘astropy’ (which uses the astropy utilities). Default is erfa unless the telescope_location frame is MCMF (on the moon), in which case the default is astropy.
- class pyuvdata.uvcal.calh5.CalH5[source]
A class for CalH5 file objects.
This class defines an HDF5-specific subclass of UVCal for reading and writing CalH5 files. This class should not be interacted with directly, instead use the read_calh5 and write_calh5 methods on the UVCal class.
- read_calh5(filename: str | ~pathlib.Path | ~pyuvdata.uvcal.calh5.FastCalH5Meta, *, antenna_nums=None, antenna_names=None, frequencies=None, freq_chans=None, spws=None, times=None, time_range=None, lsts=None, lst_range=None, jones=None, phase_center_ids=None, catalog_names=None, read_data=True, gain_array_dtype=<class 'numpy.complex128'>, background_lsts=True, run_check=True, check_extra=True, run_check_acceptability=True, use_future_array_shapes=None, astrometry_library=None)[source]
Read in data from a CalH5 file.
- Parameters:
filename (string, path, FastCalH5Meta, h5py.File) – The CalH5 file to read from. A file name or path or a FastCalH5Meta or h5py File object. Must contains a “/Header” group for CalH5 files conforming to spec.
antenna_nums (array_like of int, optional) – The antennas numbers to include when reading data into the object (antenna positions and names for the removed antennas will be retained). This cannot be provided if antenna_names is also provided. Ignored if read_data is False.
antenna_names (array_like of str, optional) – The antennas names to include when reading data into the object (antenna positions and names for the removed antennas will be retained). This cannot be provided if antenna_nums is also provided. Ignored if read_data is False.
frequencies (array_like of float, optional) – The frequencies to include when reading data into the object, each value passed here should exist in the freq_array. Ignored if read_data is False.
freq_chans (array_like of int, optional) – The frequency channel numbers to include when reading data into the object. Ignored if read_data is False.
spws (array_like of in, optional) – The spectral window numbers to keep in the object. If this is not a wide-band object and frequencies or freq_chans is not None, frequencies that match any of the specifications will be kept (i.e. the selections will be OR’ed together).
times (array_like of float, optional) – The times to include when reading data into the object, each value passed here should exist in the time_array. Cannot be used with time_range.
time_range (array_like of float, optional) – The time range in Julian Date to keep in the object, must be length 2. Some of the times in the object should fall between the first and last elements. Cannot be used with times.
lsts (array_like of float, optional) – The local sidereal times (LSTs) to keep in the object, each value passed here should exist in the lst_array. Cannot be used with times, time_range, or lst_range.
lst_range (array_like of float, optional) – The local sidereal time (LST) range in radians to keep in the object, must be of length 2. Some of the LSTs in the object should fall between the first and last elements. If the second value is smaller than the first, the LSTs are treated as having phase-wrapped around LST = 2*pi = 0, and the LSTs kept on the object will run from the larger value, through 0, and end at the smaller value.
jones (array_like of int, optional) – The jones polarizations numbers to include when reading data into the object, each value passed here should exist in the polarization_array. Ignored if read_data is False.
phase_center_ids (array_like of int, optional) – Phase center IDs to keep on the object (effectively a selection on baseline-times). Cannot be used with catalog_names.
catalog_names (str or array-like of str, optional) – The names of the phase centers (sources) to keep in the object, which should match exactly in spelling and capitalization. Cannot be used with phase_center_ids.
read_data (bool) – Read in the data-like arrays (gains/delays, flags, qualities). If set to False, only the metadata will be read in. Setting read_data to False results in a metadata only object.
background_lsts (bool) – When set to True, the lst_array is calculated in a background thread.
run_check (bool) – Option to check for the existence and proper shapes of parameters after after reading in the file (the default is True, meaning the check will be run). Ignored if read_data is False.
check_extra (bool) – Option to check optional parameters as well as required ones (the default is True, meaning the optional parameters will be checked). Ignored if read_data is False.
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after reading in the file (the default is True, meaning the acceptable range check will be done). Ignored if read_data is False.
use_future_array_shapes (bool) – Option to convert to the future planned array shapes before the changes go into effect by removing the spectral window axis.
astrometry_library (str) – Library used for calculating LSTs. Allowed options are ‘erfa’ (which uses the pyERFA), ‘novas’ (which uses the python-novas library), and ‘astropy’ (which uses the astropy utilities). Default is erfa unless the telescope_location frame is MCMF (on the moon), in which case the default is astropy.
- Returns:
None
- Raises:
IOError – If filename doesn’t exist.
ValueError – If incompatible select keywords are set (e.g. times and time_range) or select keywords exclude all data or if keywords are set to the wrong type.
- write_calh5(filename, *, clobber=False, chunks=True, data_compression=None, flags_compression='lzf', quality_compression='lzf', add_to_history=None, run_check=True, check_extra=True, run_check_acceptability=True)[source]
Write an in-memory UVCal object to a CalH5 file.
Write an in-memory UVCal object to a CalH5 file.
- Parameters:
filename (str) – The CalH5 file to write to.
clobber (bool) – Option to overwrite the file if it already exists.
chunks (tuple or bool) – h5py.create_dataset chunks keyword. Tuple for chunk shape, True for auto-chunking, None for no chunking. Default is True.
data_compression (str) – HDF5 filter to apply when writing the gain_array or delay. Default is None (no filter/compression). In addition to the normal HDF5 filter values, the user may specify “bitshuffle” which will set the compression to 32008 for bitshuffle and will set the compression_opts to (0, 2) to allow bitshuffle to automatically determine the block size and to use the LZF filter after bitshuffle. Using bitshuffle requires having the hdf5plugin package installed. Dataset must be chunked to use compression.
flags_compression (str) – HDF5 filter to apply when writing the flags_array. Default is the LZF filter. Dataset must be chunked.
quality_compression (str) – HDF5 filter to apply when writing the quality_array and/or total_quality_array if they are defined. Default is the LZF filter. Dataset must be chunked.
run_check (bool) – Option to check for the existence and proper shapes of parameters before writing the file.
check_extra (bool) – Option to check optional parameters as well as required ones.
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters before writing the file.
- Returns:
None
- Raises:
IOError – If the file located at filename already exists and clobber=False, an IOError is raised.
Notes
The HDF5 library allows for the application of “filters” when writing data, which can provide moderate to significant levels of compression for the datasets in question. Testing has shown that for some typical cases of UVData objects (empty/sparse flag_array objects, and/or uniform nsample_arrays), the built-in LZF filter provides significant compression for minimal computational overhead.
- class pyuvdata.uvcal.fhd_cal.FHDCal[source]
Defines a FHD-specific subclass of UVCal for reading FHD calibration save files.
This class should not be interacted with directly, instead use the read_fhd_cal method on the UVCal class.
- read_fhd_cal(*, cal_file, obs_file, layout_file=None, settings_file=None, raw=True, read_data=True, background_lsts=True, extra_history=None, run_check=True, check_extra=True, run_check_acceptability=True, use_future_array_shapes=None, astrometry_library=None)[source]
Read data from an FHD cal.sav file.
- Parameters:
cal_file (str or list of str) – The cal.sav file or list of files to read from.
obs_file (str or list of str) – The obs.sav file or list of files to read from.
layout_file (str) – The FHD layout file. Required for antenna_positions to be set.
settings_file (str or list of str, optional) – The settings_file or list of files to read from. Optional, but very useful for provenance.
raw (bool) – Option to use the raw (per antenna, per frequency) solution or to use the fitted (polynomial over phase/amplitude) solution. Default is True (meaning use the raw solutions).
read_data (bool) – Read in the gains, quality array and flag data. If set to False, only the metadata will be read in. Setting read_data to False results in a metadata only object. If read_data is False, a settings file must be provided.
background_lsts (bool) – When set to True, the lst_array is calculated in a background thread.
extra_history (str or list of str, optional) – String(s) to add to the object’s history parameter.
run_check (bool) – Option to check for the existence and proper shapes of parameters after reading in the file.
check_extra (bool) – Option to check optional parameters as well as required ones.
run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after reading in the file.
use_future_array_shapes (bool) – Defunct option, will result in an error in version 3.2.
astrometry_library (str) – Library used for calculating LSTs. Allowed options are ‘erfa’ (which uses the pyERFA), ‘novas’ (which uses the python-novas library), and ‘astropy’ (which uses the astropy utilities). Default is erfa unless the telescope_location frame is MCMF (on the moon), in which case the default is astropy.
UVBeam Classes
- class pyuvdata.uvbeam.beamfits.BeamFITS[source]
Defines a fits-specific subclass of UVBeam for reading and writing beamfits files.
This class should not be interacted with directly, instead use the read_beamfits and write_beamfits methods on the UVBeam class.
The beamfits format supports regularly gridded or healpix beam files. The format defined here for healpix beams is not compatible with true healpix formats because it needs to support multiple dimensions (e.g. polarization, frequency, efield vectors).
- read_beamfits(filename, *, use_future_array_shapes=None, run_check=True, check_extra=True, run_check_acceptability=True, check_auto_power=True, fix_auto_power=True, freq_range=None, az_range=None, za_range=None)[source]
Read the data from a beamfits file.
- Parameters:
filename (str or list of str) – The beamfits file or list of files to read from.
use_future_array_shapes (bool) – Defunct option, will result in an error in version 3.2.
run_check (bool) – Option to check for the existence and proper shapes of required parameters after reading in the file.
check_extra (bool) – Option to check optional parameters as well as required ones.
run_check_acceptabilit (bool) – Option to check acceptable range of the values of required parameters after reading in the file.
check_auto_power (bool) – For power beams, check whether the auto polarization beams have non-zero imaginary values in the data_array (which should not mathematically exist).
fix_auto_power (bool) – For power beams, if auto polarization beams with imaginary values are found, fix those values so that they are real-only in data_array.
freq_range (tuple of float in Hz) – If given, the lower and upper limit of the frequencies to read in. Default is to read in all frequencies. Restricting the frequencies reduces peak memory usage.
az_range (tuple of float in deg) – The azimuth range to read in, if the beam is specified in az/za coordinates. Default is to read in all azimuths. Restricting the azimuth reduces peak memory usage.
za_range (tuple of float in deg) – The zenith angle range to read in, if the beam is specified in za/za coordinates. Default is to read in all za. Restricting the za reduces peak memory.
- write_beamfits(filename, *, run_check=True, check_extra=True, run_check_acceptability=True, check_auto_power=True, fix_auto_power=False, clobber=False)[source]
Write the data to a beamfits file.
- Parameters:
filename (str) – The beamfits file to write to.
run_check (bool) – Option to check for the existence and proper shapes of required parameters before writing the file.
check_extra (bool) – Option to check optional parameters as well as required ones.
run_check_acceptability (bool) – Option to check acceptable range of the values of required parameters before writing the file.
check_auto_power (bool) – For power beams, check whether the auto polarization beams have non-zero imaginary values in the data_array (which should not mathematically exist).
fix_auto_power (bool) – For power beams, if auto polarization beams with imaginary values are found, fix those values so that they are real-only in data_array.
clobber (bool) – Option to overwrite the filename if the file already exists.
- class pyuvdata.uvbeam.cst_beam.CSTBeam[source]
Defines a CST-specific subclass of UVBeam for reading CST text files.
This class should not be interacted with directly, instead use the read_cst_beam method on the UVBeam class.
- name2freq(fname)[source]
Extract frequency from the filename.
Assumes the file name contains a substring with the frequency channel in MHz that the data represents. e.g. “HERA_Sim_120.87MHz.txt” should yield 120.87e6
- Parameters:
fname (str) – Filename to parse.
- Returns:
float – Frequency extracted from filename in Hz.
- read_cst_beam(filename, *, beam_type='power', use_future_array_shapes=None, feed_pol='x', rotate_pol=True, frequency=None, telescope_name=None, feed_name=None, feed_version=None, model_name=None, model_version=None, history='', x_orientation=None, reference_impedance=None, extra_keywords=None, run_check=True, check_extra=True, run_check_acceptability=True, check_auto_power=True, fix_auto_power=True)[source]
Read in data from a cst file.
- Parameters:
filename (str) – The cst file to read from.
beam_type (str) – What beam_type to read in (‘power’ or ‘efield’).
use_future_array_shapes (bool) – Option to convert to the future planned array shapes before the changes go into effect by removing the spectral window axis.
feed_pol (str) – The feed or polarization or list of feeds or polarizations the files correspond to. Defaults to ‘x’ (meaning x for efield or xx for power beams).
rotate_pol (bool) – If True, assume the structure in the simulation is symmetric under 90 degree rotations about the z-axis (so that the y polarization can be constructed by rotating the x polarization or vice versa). Default: True if feed_pol is a single value or a list with all the same values in it, False if it is a list with varying values.
frequency (float or list of float) – The frequency or list of frequencies corresponding to the filename(s). This is assumed to be in the same order as the files. If not passed, the code attempts to parse it from the filenames.
telescope_name (str) – The name of the telescope corresponding to the filename(s).
feed_name (str) – The name of the feed corresponding to the filename(s).
feed_version (str) – The version of the feed corresponding to the filename(s).
model_name (str) – The name of the model corresponding to the filename(s).
model_version (str) – The version of the model corresponding to the filename(s).
history (str) – A string detailing the history of the filename(s).
x_orientation (str, optional) – Orientation of the physical dipole corresponding to what is labelled as the x polarization. Options are “east” (indicating east/west orientation) and “north” (indicating north/south orientation)
reference_impedance (float, optional) – The reference impedance of the model(s).
extra_keywords (dict, optional) – A dictionary containing any extra_keywords.
run_check (bool) – Option to check for the existence and proper shapes of required parameters after reading in the file.
check_extra (bool) – Option to check optional parameters as well as required ones.
run_check_acceptability (bool) – Option to check acceptable range of the values of required parameters after reading in the file.
check_auto_power (bool) – For power beams, check whether the auto polarization beams have non-zero imaginary values in the data_array (which should not mathematically exist).
fix_auto_power (bool) – For power beams, if auto polarization beams with imaginary values are found, fix those values so that they are real-only in data_array.
- class pyuvdata.uvbeam.mwa_beam.MWABeam[source]
Defines an MWA-specific subclass of UVBeam for representing MWA beams.
This class should not be interacted with directly, instead use the read_mwa_beam method on the UVBeam class.
This is based on https://github.com/MWATelescope/mwa_pb/ but we don’t import that module because it’s not python 3 compatible.
Note that the azimuth convention in for the UVBeam object is different than the azimuth convention in the mwa_pb repo. In that repo, the azimuth convention is changed from the native FEKO convention (the FEKO convention is the same as the UVBeam convention). The convention in the mwa_pb repo has a different zero point and a different direction (so it is in a left handed coordinate system).
- read_mwa_beam(h5filepath, *, use_future_array_shapes=None, delays=None, amplitudes=None, pixels_per_deg=5, freq_range=None, run_check=True, check_extra=True, run_check_acceptability=True, check_auto_power=True, fix_auto_power=True)[source]
Read in the full embedded element MWA beam.
Note that the azimuth convention in for the UVBeam object is different than the azimuth convention in the mwa_pb repo. In that repo, the azimuth convention is changed from the native FEKO convention (the FEKO convention is the same as the UVBeam convention). The convention in the mwa_pb repo has a different zero point and a different direction (so it is in a left handed coordinate system).
- Parameters:
h5filepath (str) – path to input h5 file containing the MWA full embedded element spherical harmonic modes. Download via wget http://cerberus.mwa128t.org/mwa_full_embedded_element_pattern.h5
use_future_array_shapes (bool) – Defunct option, will result in an error in version 3.2.
delays (array of ints) – Array of MWA beamformer delay steps. Should be shape (n_pols, n_dipoles).
amplitudes (array of floats) – Array of dipole amplitudes, these are absolute values (i.e. relatable to physical units). Should be shape (n_pols, n_dipoles).
pixels_per_deg (float) – Number of theta/phi pixels per degree. Sets the resolution of the beam.
freq_range (array_like of float) – Range of frequencies to include in Hz, defaults to all available frequencies. Must be length 2.
run_check (bool) – Option to check for the existence and proper shapes of required parameters after reading in the file.
check_extra (bool) – Option to check optional parameters as well as required ones.
run_check_acceptability (bool) – Option to check acceptable range of the values of required parameters after reading in the file.
check_auto_power (bool) – For power beams, check whether the auto polarization beams have non-zero imaginary values in the data_array (which should not mathematically exist).
fix_auto_power (bool) – For power beams, if auto polarization beams with imaginary values are found, fix those values so that they are real-only in data_array.
Other Modules and Functions
aipy extracts
Module for low-level interface to MIRIAD files.
This module extracts some Python code from AIPY used in our MIRIAD I/O
routines. It was copied from AIPY commit
6cb5a70876f33dccdd68d4063b076f8d42d9edae, then reformatted. The only items
used by pyuvdata are uv_selector
and UV
.
- pyuvdata.uvdata.aipy_extracts.uv_selector(uv, ants=-1, pol_str=-1)[source]
Call select on a Miriad object with string arguments for antennas and polarizations.
- Parameters:
uv (UV object) – Miriad data set object
ants (str) – string to select antennas or baselines, e.g. ‘all’, ‘auto’, ‘cross’, ‘0,1,2’, or ‘0_1,0_2’
pol_str (str) – polarizations to select, e.g. ‘xx’, ‘yy’, ‘xy’, ‘yx’
- Returns:
None
- class pyuvdata.uvdata.aipy_extracts.UV(filename, status='old', corrmode='r')[source]
Top-level interface to a Miriad UV data set.
- variables()[source]
Get the list of available variables.
- Returns:
list of str – list of available variables
- items()[source]
Get the list of available header items.
- Returns:
list of str – list of available header items
- select(name, n1, n2, include=True)[source]
Choose which data are returned by read().
- Parameters:
name (str) – This can be: ‘decimate’, ‘time’, ‘antennae’, ‘visibility’, ‘uvrange’, ‘pointing’, ‘amplitude’, ‘window’, ‘or’, ‘dra’, ‘ddec’, ‘uvnrange’, ‘increment’, ‘ra’, ‘dec’, ‘and’, ‘clear’, ‘on’, ‘polarization’, ‘shadow’, ‘auto’, ‘dazim’, ‘delev’
n1, n2 (int) – Generally this is the range of values to select. For ‘antennae’, this is the two antennae pair to select (indexed from 0); a -1 indicates ‘all antennae’. For ‘decimate’, n1 is every Nth integration to use, and n2 is which integration within a block of N to use. For ‘shadow’, a zero indicates use ‘antdiam’ variable. For ‘on’, ‘window’, ‘polarization’, ‘increment’, ‘shadow’ only p1 is used. For ‘and’, ‘or’, ‘clear’, ‘auto’ p1 and p2 are ignored.
include (bool) – If true, the data is selected. If false, the data is discarded. Ignored for ‘and’, ‘or’, ‘clear’.
- read(raw=False)[source]
Return the next data record.
Calling this function causes variables to change to reflect the record which this function returns.
- Parameters:
raw (bool) – if True data and flags are returned seperately
- Returns:
preamble (tuple) – (uvw, t, (i,j)), where uvw is an array of u,v,w, t is the Julian date, and (i,j) is an antenna pair
data (ndarray or masked array) – ndarray if raw is True, otherwise a masked array
flags (ndarray) – only returned if raw is True
- all_data(raw=False)[source]
Provide an iterator over preamble, data.
Allows constructs like:
for preamble, data in uv.all_data(): ...
.- Parameters:
raw (bool) – If True data and flags are returned seperately.
- Returns:
preamble (tuple) – (uvw, t, (i,j)), where uvw is an array of u,v,w, t is the Julian date, and (i,j) is an antenna pair
data (ndarray or masked array of complex) – ndarray if raw is True, otherwise a masked array
flags (ndarray) – only returned if raw is True
- write(preamble, data, flags=None)[source]
Write the next data record.
- Parameters:
preamble (tuple) – (uvw, t, (i,j)), where uvw is an array of u,v,w, t is the Julian date, and (i,j) is an antenna pair
data (masked array of complex) – spectra for this record
- init_from_uv(uv, override=None, exclude=None)[source]
Initialize header items and variables from another UV.
Those in override will be overwritten by override[k], and tracking will be turned off (meaning they will not be updated in pipe()). Those in exclude are omitted completely.
- Parameters:
uv (UV object) – Miriad data set object to initialize from
override (dict) – variables with values to overwrite
exclude (list) – list of variable to exclude
- pipe(uv, mfunc=<function _uv_pipe_default_action>, append2hist='', raw=False)[source]
Pipe in data from another UV.
Uses the function
mfunc(uv, preamble, data)
, which should return(preamble, data)
. If mfunc is not provided, the dataset will just be cloned, and if the returned data is None, it will be omitted.- Parameters:
uv (UV object) – Miriad data set object to pipe from
mfunc (function) – function that defines how the data are piped.
mfunc(uv, preamble, data)
should return(preamble, data)
. Default is_uv_pipe_default_action
which just clones the dataset.append2hist (str) – string to append to history
raw (bool) – if True data and flags are piped seperately
MIR parser
Module for low-level interface to Mir datasets.
This module provides a python interface with Mir datasets, including both metadata and the visibility data itself.
- class pyuvdata.uvdata.mir_parser.MirParser(filepath=None, *, compass_soln=None, make_v3_compliant=None, old_vis_format=None, nchunks=None, has_auto=False, has_cross=True, load_auto=False, load_cross=False, load_raw=False)[source]
General class for reading Mir datasets.
Does lots of cool things! There are static functions that allow you low level access to mir files without needing to create an object. You can also instantiate a MirParser object with the constructor of this class which will only read the metadata into memory by default. Read in the raw data through the use of the load_cross and load_auto flags, or by using the load_data() function once the object is created. This allows for the flexible case of quickly loading metadata first to check whether or not to load additional data into memory.
- copy(metadata_only=False)[source]
Make and return a copy of the MirParser object.
- Parameters:
metadata_only (bool) – If set to True, will forgo copying the attributes vis_data, raw_data`, and auto_data.
- Returns:
MirParser – Copy of self.
- apply_tsys(*, invert=False, force=False, use_cont_det=None)[source]
Apply Tsys calibration to the visibilities.
SMA MIR data are recorded as correlation coefficients. This allows one to apply system temperature information to the data to get values in units of Jy.
- Parameters:
invert (bool) – If set to True, this will effectively undo the Tsys correction that has been applied. Default is False (convert uncalibrated visibilities to units of Jy).
force (bool) – Normally the method will check if tsys has already been applied (or not applied yet, if invert=True), and will throw an error if that is the case. If set to True, this check will be bypassed. Default is False.
use_cont_det (bool) – If set to True, use the continuum tsys data for calculating system temperatures. If set to False, the spectral tsys data will be applied if available. Default is True.
- apply_flags()[source]
Apply online flags to the visibilities.
Applies flagging as recorded by the online system, which are applied on a per-spectral record basis. Users should be aware that this method will only modify data in vis_data, and not the “raw” values stored in raw_data.
- Raises:
ValueError – If vis_data are not loaded.
- load_data(*, load_auto=None, load_cross=None, load_raw=False, apply_tsys=True, allow_downselect=None, allow_conversion=None, use_mmap=True, read_only=False)[source]
Load visibility data into MirParser class.
Method for loading the visibility data into the object. Can either load the compressed “raw data” (which is about half the size in memory, but carries overheads in various function calls _except_ for writing to disk) or the uncompressed “vis data” (floating point values).
Note that this function will only load either vis data or raw data, not both. If told to load both, then the method will default to loading only the vis data.
- Parameters:
load_auto (bool) – Load the autos (floats) into object. Default is True if the object has auto-correlation data, otherwise False.
load_cross (bool) – Load the cross-correlation visibility data (floats) into object. Default is True if if the object has cross-correlation data, otherwise False.
load_raw (bool) – Nominally when loading cross-correlation data, the data are uncompressed when being loaded into the object, and put into the attribute vis_data. However, if set to True, the “raw” (compressed) data are put into the attribute raw_data. Default is False.
apply_tsys (bool) – Apply tsys corrections to the data. Only applicable if loading cross-correlation vis data (where load_raw=False). Default is True.
allow_downselect (bool) – If data has been previously loaded, and all spectral records are currently contained in vis_data, raw_data, and/or auto_data, then down-select from the currently loaded data rather than reading the data from disk. Default is True if all spectral records have been previously loaded, otherwise False.
allow_conversion (bool) – Allow the method to convert previously loaded uncompressed (“raw”) data into “normal” data. Only applicable if loading cross-correlation data (and load_raw=False). Default is True if all of the required spectral records have been loaded into the raw_data attribute.
use_mmap (bool) – If False, then each integration record needs to be read in before it can be parsed on a per-spectral record basis (which can be slow if only reading a small subset of the data). Default is True, which will leverage mmap to access data on disk (that does not require reading in the whole record). There is usually no performance penalty to doing this, although reading in data is slow, you may try seeing this to False and seeing if performance improves.
read_only (bool) – Only applicable if load_cross=True, use_mmap=True, and load_raw=True. If set to True, will return back data arrays which are read-only. Default is False.
- Raises:
UserWarning – If attempting to set both load_vis and load_raw to True. Also if the method is about to attempt to convert previously loaded data.
- unload_data(*, unload_vis=True, unload_raw=True, unload_auto=True)[source]
Unload data from the MirParser object.
Unloads the data-related attributes from memory, if they are loaded. Because these attributes can be formidable in size, this operation will substantially reduce the memory footprint of the MirParser object.
- Parameters:
unload_vis (bool) – Unload the visibilities stored in the vis_data attribute, if loaded. Default is True.
unload_raw (bool) – Unload the raw visibilities stored in the raw_data attribute, if loaded. Default is True.
unload_auto (bool) – Unload the auto-correlations stored in the auto_data attribute, if loaded. Default is True.
- reset()[source]
Reset a MirParser object to its original state.
This method will in effect revert the object to a “pristine” state. Visibility data will be unloaded, changed metadata will be restored, and any rechunking settings will be removed (so that data will be loaded at full spectral resolution).
- save_mask(mask_name: str = None, overwrite=False)[source]
Save masks for later use.
This method will store selection masks, that can later be reloaded. This can be useful for temporarily down-selecting data for various operations (allowing one to restore the old selection mask after that is complete).
- Parameters:
mask_name (str) – Name under which to store the mask. No default.
overwrite (bool) – If set to False and attempting to save a mask whose name is already stored, an error is thrown. If set to True, and existing masks stored under the name given will be overwritten. Default is False.
- restore_mask(mask_name: str, run_update=True)[source]
Restore stored masks.
This method will restore selection masks that have been previously saved.
- Parameters:
mask_name (str) – Name of the previously stored mask to restore.
run_update (bool) – By default, the method will cross-check internal masks and, if needed, will unload or downselect the visibility/autocorrelation data to match the records selected. Setting this to False will bypass this check. This should only be used by advanced users, as this can produce unexpected behaviors.
- read(filepath=None, *, compass_soln=None, make_v3_compliant=None, old_vis_format=None, nchunks=None, has_auto=False, has_cross=True, load_auto=False, load_cross=False, load_raw=False)[source]
Read in all files from a mir data set into predefined numpy datatypes.
The full dataset can be quite large, as such the default behavior of this function is only to load the metadata. Use the keyword params to load other data into memory.
- Parameters:
filepath (str) – Filepath is the path to the folder containing the Mir data set.
compass_soln (str) – Optional argument, specifying the path of COMPASS-derived flagging and bandpass gains solutions, which are loaded into the object.
make_v3_compliant (bool) – Convert the metadata required for filling a UVData object into a v3-compliant format. Only applicable if MIR file version is v1 or v2. Default behavior is to set this to True if the file version is v1 or v2.
old_vis_format (bool) – Prior to the v1 data format, older visibility data was recorded with different header fields and endianness (big-endian versus the current little-endian format). If set to True, data are assumed to be in this “old” format, if set to False, data are assumed to be in the current format. Default behavior is to attempt to automatically determine old versus new based on metadata, and set this value accordingly.
nchunks (int) – Only used if has_auto=True is set, this value is used to set the number of spectral windows in a single auto-correlation set. The default behavior is to set this value automatically depending on file version. This setting should only be used by advanced users in specialized circumstances.
has_auto (bool) – Flag to read auto-correlation data. Default is False.
has_cross (bool) – Flag to read cross-correlation data. Default is True.
load_auto (bool) – Flag to load auto-correlations into memory. Default is False.
load_cross (bool) – Flag to load cross-correlations into memory. Default is False.
load_raw (bool) – Flag to load raw data into memory. Default is False.
- write(filepath=None, *, overwrite=True, load_data=False, append_data=False, check_index=True)[source]
Write a MirParser object to disk in Mir format.
Writes out a MirParser object to disk, in the binary Mir format. This method can worth with either a full dataset, or partial datasets appended together multiple times.
- Parameters:
filepath (str) – Path of the directory to write out the data set. found in filepath. If no previously created data set exists, then a new data set is created on disk, and this parameter is ignored. Default is False.
overwrite (bool) – If set to True, any previously written data in filepath will be overwritten. Default is False. This argument is ignored if append_data is set to True.
load_data (bool) – If set to True, load the raw visibility data. Default is False, which will forgo loading data. Note that if no data are loaded, then the method will then write out a metadata-only object.
append_data (bool) – If set to True, this will allow the method to append data to an existing file on disk. If no such file exists in filepath, then a new file is created (i.e., no appends are performed).
check_index (bool) – Only applicable if append_data=True. If set to True and data are being appended to an existing file, the method will check to make sure that there are no header key conflicts with the data being being written to disk, since this can cause corrupted the metadata. Default is True, users should use this argument with caution, since it can cause the data set on disk to become unusable.
- Raises:
UserWarning – If only metadata is loaded in the MirParser object.
FileExistsError – If a file already exists and cannot append or overwrite.
ValueError – If attempting to append data, but conflicting header keys are detected between the data on disk and the data in the object.
- rechunk(chan_avg=None)[source]
Rechunk a MirParser object.
Spectrally average a Mir dataset. This command attempts to emulate the old “SMARechunker” program within the MirParser object. Users should be aware that running this operation modifies the metadata in such a way that all data loaded will be rechunked in the same manner, until you run the reset method.
Note that this command will only process data from the “normal” spectral windows, and not the pseudo-continuum data (which will remain untouched).
- Parameters:
chan_avg (int) – Number of contiguous spectral channels to average over.
- select(where=None, *, and_where_args=True, and_mask=True, update_data=None, reset=False)[source]
Select a subset of data inside a Mir-formatted file.
This routine allows for one to select a subset of data within a Mir dataset, based on various metadata. The select command is designed to be flexible, allowing for both multiple simultaneous selections and serial calls. Users should be aware that the command is case sensitive, and uses “strict” agreement (i.e., it does not mimic the behavior is np.isclose) with metadata values. By default, multiple selection criteria are combined together via an “and” operation; for example, if you wanted to select only data from Antenna 1 while on 3c84, you would set where=((“source”, “eq”, “3c84”), (“ant”, “eq”, 1)).
The select command will automatically translate information in codes_data, the and the various indexing keys, e.g., it will convert an allowed value for “source” (found in`codes_read`) into allowed values for “isource” (found in in_read).
- Parameters:
where (tuple or list of tuples) – Optional argument, where tuple is used to identify a matching subset of data. Each tuple must be 3 elements in length, consisting of the “selection field”, the “comparison operator”, and the “comparison value”. The selection field match one of the field names inside one of the metadata attributes (e.g., “ant1”, “mjd”, “source”, “fsky”). The comparison operator specifies how the metadata are compared against the selection field. Allowed comparisons include: “eq” or “==” (equal to); “ne” or “!=” (not equal to); “lt” or “<” (less than); “le” or “<=” (less than or equal to); “gt” or “>” (greater than); “ge” or “>=” (greater than or equal to); “between” (between a range of values); “outside” (outside of a range of values). The selection value are the value or sequence of values to compare against that present in the selection field. Note that in most cases, this should be a single number, unless the comparison operator is “between” our “outside” (which requires a this element to be a sequence of length 2), of “eq” or “ne”, where either a single value (string or number), or multiple values can be supplied in a single where statement. Multiple selections can be made by supplying a sequence of 3-element tuples, where the results of each selection are combined based on the value of and_where_args.
and_where_args (bool) – If set to True, then the individual calls to the where method will be combined via an element-wise “and” operator, such that the returned array will report the positions where all criteria are met. If False, results are instead combined via an element-wise “or” operator. Default is True. If supplied, the argument for mask will be combined with the output from the calls to where with the same logic.
and_mask (bool) – If set to True, then the mask generated by the selection criteria above will be combined with the existing mask values using an element-wise “and” operation. If set to False, the two will instead be combined with an element-wise “or” operation. Default is True.
update_data (bool) – Whether or not to update the visibility values (as recorded in the attributes vis_data and raw_data). If set to True, it will force data to be loaded from disk, based on what had been previously loaded. If False, it will unload those attributes. The default is to do nothing if data are not loaded, otherwise to downselect from the existing data in the object if all records are present (and otherwise unload the data).
reset (bool) – If set to True, undoes any previous filters, so that all records are once again visible.Default is False.
- read_compass_solns(filename=None, load_flags=True, load_bandpass=True)[source]
Read in COMPASS-formatted bandpass and flagging solutions.
Reads in an HDF5 file containing the COMPASS-derived flags and gains tables. These solutions are applied as the data are read in (when calling load_data).
- Parameters:
filename (str) – Name of the file containing the COMPASS flags and gains solutions.
load_flags (bool) – If set to True, the COMPASS flags will be read into the MirParser object. Default is True.
load_gains (bool) – If set to True, the COMPASS gains will be read into the MirParser object. Default is True.
- Raises:
UserWarning – If the COMPASS solutions do not appear to overlap in time with that in the MirParser object.
- redoppler_data(freq_shift=None, *, kernel_type='cubic', tol=0.001, flag_adj=True, fix_freq=None)[source]
Re-doppler the data.
Note that this function may be moved out into utils module once UVData objects are capable of carrying Doppler tracking-related information.
- Parameters:
freq_shift (ndarray) – Amount to shift each spectral window by in frequency space. Shape is the same as the attribute sp_data, of dtype float32, in units of GHz. If no argument is provided (or if set to None), then the method will assume you want to redoppler to the topocentric rest frame, using the information stored in the MirParser object. Note that if supplying a value delta_nu, the center of the spectra will be shifted to old_center + delta_nu.
kernel_type (str) – The redoppler_data method allows for several interpolation schemes for adjusting the frequencies of the individual channels. The three supported schemes are “nearest” (nearest-neighbor; choose the closest channel to the one desired), “linear” (linear interpolation; interpolate between the two closest channel), and “cubic” (cubic convolution; see “Cubic Convolution Interpolation for Digital Image Processing” by Robert Keys for more details ). Nearest neighbor is the fastest, although cubic convolution generally provides the best spectral PSF.
tol (float) – If the desired frequency shift is close enough to an integer number of channels, then the method will forgo any attempt and interpolation and will simply return the nearest channel desired. The tolerance for choosing this behavior is given by this parameter, in units of number of channels. The default is 1e-3, which means that if the desired shift is within one one-thousandth of an existing channel, the method will (for the frequency window in question) the nearest-neighbor interpolation scheme.
flag_adj (bool) – Option to flag channels adjacent to those that are flagged, the number of which depends on the interpolation scheme (1 additional channel with linear interpolation, and 3 additional channels with cubic convolution). Set to True by default, which prevents the window from having an inconsistent spectral PSF across the band.
fix_freq (bool) – Only used if freq_shift is left unset (or otherwise set to None). Some versions of MIR data have frequency information incorrectly stored. If set to True, this metadata will be fixed before doppler-shifting the data. By default, the method will apply this correction if the version of the MIR data format is known to have the defective metadata.
- Raises:
ValueError – If tol is outside of the range [0, 0.5], or if kernel_type does not match the list of supported values. Also if providing no argument to freq_shift, but doppler-tracking information cannot be determined (either because it’s the wrong file version or because the receiver code isn’t recognized).
- exception pyuvdata.uvdata.mir_parser.MirPackdataError(message='There was an error parsing a packed data file.')[source]
Class for errors when ingesting packed MIR data.
This class is used to raise error when attempting to read “packed” data sets, which are long byte arrays, within which data has been packed, intermixed with some basic header information. Because the data can be irregularly shaped, file integrity issues (e.g., flipped bits) can make the header information unparsable. This class is used in practice to catch such errors.
MIR metadata
Module for low-level interface to Mir metadata files.
This module provides a python interface for individual Mir metadata files, e.g. “in_read”, “bl_read”, “sp_read”, “we_read”, “eng_read”, “antennas”, and “codes_read”.
- class pyuvdata.uvdata.mir_meta_data.MirMetaData(obj=None, *, filetype=None, dtype=None, header_key_name=None, binary_dtype=None, pseudo_header_key_names=None)[source]
Class for metadata within Mir datasets.
This class is used as the parent class for the different types of metadata tables that exist within a Mir dataset. The object is built around a complex ndarray, which typically contains dozens of fields with different metadata stored. The object also contains a mask, which is useful for marking specific header entries as being in use (particularly when one has multiple MirMetaData objects together, like in the MirParser object).
- copy(skip_data=False)[source]
Make and return a copy of the MirMetaData object.
- Parameters:
skip_data (bool) – If set to True, forgo copying the data-related attributes. Default is False.
- Returns:
new_obj (MirMetaData object) – Copy of the original object.
- where(select_field=None, select_comp=None, select_val=None, *, mask=None, return_header_keys=False)[source]
Find where metadata match a given set of selection criteria.
This method will produce a masking screen based on the arguments provided to determine which entries match a given set of conditions.
- Parameters:
select_field (str) – Field in the metadata to evaluate.
select_comp (str) – Specifies the type of comparison to do between the value supplied in select_val and the metadata. No default, allowed values include: “eq” or “==” (equal to); “ne” or “!=” (not equal to); “lt” or “<” (less than); “le” or “<=” (less than or equal to); “gt” or “>” (greater than); “ge” or “>=” (greater than or equal to); “between” (between a range of values); “outside” (outside of a range of values).
select_val (number of str, or sequence of number or str) – Value(s) to compare data in select_field against. If select_comp is “lt”, “le”, “gt”, “ge”, then this must be a single number. If select_comp is “between” or “outside”, then this must be a list 2 numbers. If select_comp is “eq”/”==” or “ne”/”!=”, then this can be either a single value (number or string) or a sequence of numbers.
mask (ndarray of bool) – Optional argument, of the same length as the MirMetaData object, which is applied to the output of the selection parsing through an element-wise “and” operation. Useful for combining multiple calls to where together.
return_header_keys (bool) – If set to True, return a list of the header key values where matching entries are found. Default is False, which will return an ndarray of type bool, and length equal to that of the MirMetaData object.
- Returns:
return_arr (ndarray of bool or list) – If return_header_keys=False, boolean array marking whether select_field meets the condition set by select_comp and select_val. If return_header_keys=True, then instead of a boolean array, a list of ints (or tuples of ints if the MetaDataObject has only a pseudo header key) corresponding to the header key values.
- Raises:
ValueError – If select_comp is not one of the permitted strings, or if select_field is not one of the fields within data_arr.
- get_value(field_name, *, use_mask=None, where=None, and_where_args=True, header_key=None, index=None, return_tuples=None)[source]
Get values from a particular field or set of fields of the metadata.
This function allows one to get the values for a particular field or set of fields within the metadata. Selection criteria can optionally be specified for gathering only a subset of the metadata for the field(s).
- Parameters:
field_name (str or list of strs) – Fields from which to extract data. Can either be given as either an str or list of strs. Each str must match a field name, as list in the dtype attribute of the object.
use_mask (bool) – If True, consider only data where the internal mask is marked True. Default is True, unless an argument is supplied to index or header_key, in which case the default is False.
where (tuple of sequence of tuples) – Optional argument, each tuple is used to call the where method to identify which index positions match the given criteria. Can be supplied as a sequence of tuples rather than a single tuple, but each much be of length 3, where the first argument is the select_field argument, the second is the select_comp argument, and the last is the select_val argument. See the documentation of where for more details. Cannot be specified with index or header_key.
and_where_args (bool) – If set to True, then the individual calls to the where method will be combined via an element-wise “and” operator, such that the returned array will report the positions where all criteria are met. If False, results are instead combined via an element-wise “or” operator. Default is True. If supplied, the argument for mask will be combined with the output from the calls to where with the same logic.
index (sequence of int) – Optional argument, specifies the index positions at which to extract data from the meta data. Cannot be specified with header_key or where.
header_key (sequence of ints or tuples) – Optional argument, values to match against the header key field, in order to determine which entries of the array to extract. For example, if the header key field “a” has the values [2, 4, 6, 8], setting this argument to [2, 8] will grab the values of field_name in the metadata array at the index positions [0, 3]. Cannot be specified with index or where.
return_tuples (bool) – If set to True, return a list of tuples containing the value of each field (in the order provided in field_name). If False, return an ndarray or list of ndarrays, where each array contains the set of values matching the specified selection criteria. Default is to return tuples if multiple fields are being extracted, otherwise to return an ndarray of values.
- Returns:
value_arr (ndarray or list of ndarrays or tuples) – Values for the specified field name where the selection criteria match. If return_tuples=False, then this will be an ndarray (of varying dtype) if a single field name was supplied, otherwise a list of ndarrays will be returned. If return_tuples=True, then a tuple containing the set of all fields at each index position will be provided.
- Raises:
ValueError – If field_name is not a list, set, tuple, or str.
- set_value(field_name=None, value=None, *, use_mask=None, where=None, and_where_args=True, header_key=None, index=None)[source]
Set values from a particular field of the metadata.
Allows one to set the values of specific field within the metadata, optionally based on a set of selection criteria.
- Parameters:
field_name (str) – Fields from which to extract data. Must match a field name, as list in the dtype attribute of the object.
value (ndarray) – Values to set the field in question to, where the provided selection criteria match. Shape of the array must be broadcastable to either the shape of the internal mask or to the shape of the index or header_key arguments.
use_mask (bool) – If True, consider only data where the internal mask is marked True. Default is True, unless an argument is supplied to index or header_key, in which case the default is False.
where (tuple of sequence of tuples) – Optional argument, each tuple is used to call the where method to identify which index positions match the given criteria. Can be supplied as a sequence of tuples rather than a single tuple, but each much be of length 3, where the first argument is the select_field argument, the second is the select_comp argument, and the last is the select_val argument. See the documentation of where for more details. Cannot be specified with index or header_key.
and_where_args (bool) – If set to True, then the individual calls to the where method will be combined via an element-wise “and” operator, such that the returned array will report the positions where all criteria are met. If False, results are instead combined via an element-wise “or” operator. Default is True. If supplied, the argument for mask will be combined with the output from the calls to where with the same logic.
header_key (sequence of ints or tuples) – Optional argument, values to match against the header key field, in order to determine which entries of the array to extract. For example, if the header key field “a” has the values [2, 4, 6, 8], setting this argument to [2, 8] will set the values of field_name in the metadata array at the index positions [0, 3]. Cannot be specified with index or where.
index (sequence of int) – Optional argument, specifies the index positions at which to set the values of the metadata. Cannot be specified with header_key or where.
- Raises:
UserWarning – If attempting to set the field “dataoff”, which is typically only used for internal indexing purposes, and is generally not modified. Also raised if modifying one of the known (pseudo) header keys.
- get_mask(*, where=None, and_where_args=True, header_key=None, index=None)[source]
Get value of the mask at a set of locations..
This function allows one to get the value(s) of the internal mask. Selection criteria can optionally be specified for accessing the mask at a specific set of positions.
- Parameters:
where (tuple of sequence of tuples) – Optional argument, each tuple is used to call the where method to identify which index positions match the given criteria. Can be supplied as a sequence of tuples rather than a single tuple, but each much be of length 3, where the first argument is the select_field argument, the second is the select_comp argument, and the last is the select_val argument. See the documentation of where for more details. Cannot be specified with index or header_key.
and_where_args (bool) – If set to True, then the individual calls to the where method will be combined via an element-wise “and” operator, such that the returned array will report the positions where all criteria are met. If False, results are instead combined via an element-wise “or” operator. Default is True. If supplied, the argument for mask will be combined with the output from the calls to where with the same logic.
header_key (sequence of ints or tuples) – Optional argument, values to match against the header key field, in order to determine which entries of the array to extract. For example, if the header key field “a” has the values [2, 4, 6, 8], setting this argument to [2, 8] will grab the values internal mask at the index positions [0, 3]. Cannot be specified with index or where.
index (sequence of int) – Optional argument, specifies the index positions at which to extract data from the meta data. Cannot be specified with header_key or where.
- Returns:
mask_arr (ndarray of bool) – Values for mask where the selection criteria match.
- set_mask(*, mask=None, where=None, and_where_args=True, header_key=None, index=None, reset=False, and_mask=True, use_mask=True)[source]
Set the internal object mask.
This function updates the internal mask based on the supplied selection criteria. This internal mask is primarily used to identify which rows of data are “active”, and will affect what some methods return to the user.
- Parameters:
mask (ndarray of bool) – Optional argument, typically of the same length as the MirMetaData object, where True marks which index positions to set. Setting this will cause any arguments passed to where, header_key, and index to be ignored.
where (tuple of sequence of tuples) – Optional argument, each tuple is used to call the where method to identify which index positions match the given criteria. Can be supplied as a sequence of tuples rather than a single tuple, but each much be of length 3, where the first argument is the select_field argument, the second is the select_comp argument, and the last is the select_val argument. See the documentation of where for more details. Cannot be specified with index or header_key.
and_where_args (bool) – If set to True, then the individual calls to the where method will be combined via an element-wise “and” operator, such that the returned array will report the positions where all criteria are met. If False, results are instead combined via an element-wise “or” operator. Default is True. If supplied, the argument for mask will be combined with the output from the calls to where with the same logic.
header_key (sequence of ints or tuples) – Optional argument, values to match against the header key field, in order to determine which entries of the array to extract. For example, if the header key field “hid” has the values [2, 4, 6, 8], setting this argument to [2, 8] will set the mask at index positions [0, 3] to True. Cannot be specified with index or where.
index (sequence of int) – Optional argument, specifies the index positions at which to set the mask to True. Cannot be specified with header_key or where.
reset (bool) – If set to True, reset all values of the mask to True before updating the mask. Default is False.
and_mask (bool) – If set to True, then the mask generated by the selection criteria above will be combined with the existing internal mask using an element-wise “and” operation. If set to False, the two will instead be combined with an element-wise “or” operation. Default is True (i.e., and the masks together).
use_mask (bool) – Only used if an argument for mask is supplied. If set to False, the supplied mask is applied to the mask attribute directly, rather than only to elements where the underlying mask is already set to True. This means that the length of mask will need to be equal to the _mask attribute, rather than the MirMetaData object. Default is True, which covers most typical use cases.
- get_header_keys(*, use_mask=None, where=None, and_where_args=True, index=None, force_list=False)[source]
Get the header keys based on selection criteria.
This function allows one to lookup (pseudo) header keys that match a given set of criteria. Header keys are most commonly used to cross-link various metadata objects.
- Parameters:
use_mask (bool) – If True, consider only data where the internal mask is marked True. Default is True, unless an argument is supplied to index or header_key, in which case the default is False.
where (tuple of sequence of tuples) – Optional argument, each tuple is used to call the where method to identify which index positions match the given criteria. Can be supplied as a sequence of tuples rather than a single tuple, but each much be of length 3, where the first argument is the select_field argument, the second is the select_comp argument, and the last is the select_val argument. See the documentation of where for more details. Cannot be specified with index or header_key.
and_where_args (bool) – If set to True, then the individual calls to the where method will be combined via an element-wise “and” operator, such that the returned array will report the positions where all criteria are met. If False, results are instead combined via an element-wise “or” operator. Default is True. If supplied, the argument for mask will be combined with the output from the calls to where with the same logic.
index (sequence of int) – Optional argument, specifies the index positions at which to extract data from the meta data. Cannot be specified with header_key or where.
force_list (False) – Normally the header keys are returned as an iterable (ndarray in there is a header key, otherwise a list of tuples), but if set to True, the output will instead be a list of ndarray for each one of the fields within the set of (pseudo) header keys.
- Returns:
header_key (ndarray or list) – If force_list=False, then if the object has a normal header key, an ndarray is returned with all keys that match the selection criteria, otherwise a list of tuples is returned. If force_list=True or list of ndarrays is returned – one for each field in the (pseudo) header keys.
- group_by(group_fields=None, *, use_mask=True, where=None, header_key=None, return_index=None, assume_unique=False)[source]
Create groups of index positions based on particular field(s) in the metadata.
This method is a convenience function for creating groups of data based on a particular set of metadata.
- Parameters:
group_fields (str or list of str) – Field or list of fields to group the data by. Must be one of the fields within the dtype of this object.
use_mask (bool) – If True, consider only data where the internal mask is marked True. Default is True.
where (tuple of sequence of tuples) – Optional argument, each tuple is used to call the where method to identify which index positions match the given criteria. Can be supplied as a sequence of tuples rather than a single tuple, but each much be of length 3, where the first argument is the select_field argument, the second is the select_comp argument, and the last is the select_val argument. See the documentation of where for more details. Cannot be specified with index or header_key.
header_key (sequence of ints or tuples) – Optional argument, values to match against the header key field, in order to determine which entries of the array to extract. For example, if the header key field “hid” has the values [2, 4, 6, 8], setting this argument to [2, 8] will set the mask at index positions [0, 3] to True. Cannot be specified with index or where.
return_index (bool) – If False, return the header key values (or pseudo-key tuples) for each element of the group. If True, return instead the index position of the grouped data (if applicable). Note that the index positions are reported after the mask is applied, such that the highest index position will be equal to the sum of the mask values minus 1. Default is False.
assume_unique (bool) – If set to True, assume that the value(s) of group_field are unique per index position, and return the results without making any attempts at grouping, which can produce a moderate increase in speed. Default is False.
- Returns:
group_dict (dict) – A dictionary containing the unique groupings of data depending on the input to group_fields. If a single str is provided, then the keys of the dict will be the unique values of the field, otherwise the keys will be tuples of the unique sets of metadata values for the grouping fields. The values are is either an ndarray of index positions (if return_index=True), an ndarray of header key values (if return_index=True and the object has a valid header key), or a list of tuples (if return_index=True and the object only has a pseudo-index), which correspond to the metadata entries that match the unique key.
- reset_values(field_name=None)[source]
Reset metadata fields to their original values.
Restores the original values for metadata that has been changed, when it has been modified by set_value or __set_item__.
- Parameters:
field_name (str or list of str) – Optional argument, specifies which fields should be restored. Can be either a single field (str) or multiple fields (list of str). Default is to restore all values which have been changed.
- Raises:
ValueError – If the specified field name(s) do not have a backup copy found in the internal backup dictionary.
- reset()[source]
Reset a MirMeteData object.
Restores the object to a “pristine” state, similar to when it was first loaded. Any changed fields are restored, and the mask is reset (selection criteria are unapplied).
- read(filepath=None)[source]
Read in data for a MirMetaData object from disk.
- Parameters:
filepath (str or Path) – Path of the folder containing the metadata in question.
- write(filepath=None, *, overwrite=False, append_data=False, check_index=False)[source]
Write a metadata object to disk.
- Parameters:
filepath (str) – Path of the folder to write the metadata into.
overwrite (bool) – If set to True, allow the file writer to overwrite a previously written data set. Default is False. This argument is ignored if append_data is set to True.
append_data (bool) – If set to True, will append data to an existing file. Default is False.
check_index (bool) – Only applicable if append_data=True. If set to True and data are being appended to an existing file, the method will check to make sure that there are no header key conflicts with the data being written to disk, since this can cause the file to become unusable. Default is False.
- Raises:
FileExistsError – If a file already exists and cannot append or overwrite.
ValueError – If attempting to append data, but conflicting header keys are detected between the data on disk and the data in the object.
- class pyuvdata.uvdata.mir_meta_data.MirInData(obj=None)[source]
Class for per-integration metadata in Mir datasets.
This class is a container for per-integration metadata, using the header key “inhid”. When reading from/writing to disk, the object looks for a file named “in_read”, which is where the online system records this information.
- class pyuvdata.uvdata.mir_meta_data.MirBlData(obj=None)[source]
Class for per-baseline metadata in Mir datasets.
This class is a container for per-baseline metadata, using the header key “blhid”. When reading from/writing to disk, the object looks for a file named “bl_read”, which is where the online system records this information. Note that “per-baseline” here means per-integration, per-sideband, per-receiver/polarization.
- class pyuvdata.uvdata.mir_meta_data.MirSpData(obj=None)[source]
Class for per-spectral window metadata in Mir datasets.
This class is a container for per-spectral window metadata, using the header key “sphid”. When reading from/writing to disk, the object looks for a file named “sp_read”, which is where the online system records this information. Note that “per-spectral window” here means per-integration, per-baseline, per-spectral band number.
- class pyuvdata.uvdata.mir_meta_data.MirEngData(obj=None)[source]
Class for per-antenna metadata in Mir datasets.
This class is a container for per-antenna, per-integration metadata. When reading from/writing to disk, the object looks for a file named “eng_read”, which is where the online system records this information. This object does not have a unique header key, but instead has a pseudo key made up of the integration header ID number (“inhid”) and the antenna number (“antenna”), which should be unique for each entry.
- class pyuvdata.uvdata.mir_meta_data.MirWeData(obj=None)[source]
Class for per-integration weather metadata in Mir datasets.
This class is a container for per-integration weather metadata, using the header key “ints”. When reading from/writing to disk, the object looks for a file named “we_read”, which is where the online system records this information.
- class pyuvdata.uvdata.mir_meta_data.MirCodesData(obj=None)[source]
Class for per-track metadata in Mir datasets.
This class is a container for various metadata, which typically vary per-integration or not at all. When reading from/writing to disk, the object looks for a file named “codes_read”, which is where the online system records this information. This object does not have a unique header key, but instead has a pseudo key made up of the variable name (“v_name”) and the indexing code (“icode”).
The main feature of block of metadata is two-fold. First, it enables one to match strings (for example, like that used for source names) to indexing values that are used by other metadata types (e.g., isource in “in_read”). Second, it enables one to record arbitrary strings that can be used to describe various properties of the whole dataset (e.g., “filever”, which denotes the version).
This object has several methods that are partially inherited from the MirMetaData class, but are modified accordingly to enable better flexibility when attempting to process these string “codes”.
- get_code_names()[source]
Produce a list of code types (v_names) found in the metadata.
- Returns:
code_list (list of str) – A list of all the unique code types, as recorded in the “v_name” field of the metadata.
- where(select_field=None, select_comp=None, select_val=None, *, mask=None, return_header_keys=None)[source]
Find where metadata match a given set of selection criteria.
This method will produce a masking screen based on the arguments provided to determine which entries match a given set of conditions.
- Parameters:
select_field (str) – Field or code type (“v_name”) in the metadata to evaluate.
select_comp (str) – Specifies the type of comparison to do between the value supplied in select_val and the metadata. No default, allowed values include: “eq” or “==” (equal to); “ne” or “!=” (not equal to); “lt” or “<” (less than); “le” or “<=” (less than or equal to); “gt” or “>” (greater than); “ge” or “>=” (greater than or equal to); “between” (between a range of values); “outside” (outside of a range of values).
select_val (number of str, or sequence of number or str) – Value(s) to compare data in select_field against. If select_comp is “lt”, “le”, “gt”, “ge”, then this must be a single number. If select_comp is “between” or “outside”, then this must be a list 2 numbers. If select_comp is “eq”/”==” or “ne”/”!=”, then this can be either a single value (number or string) or a sequence of numbers.
mask (ndarray of bool) – Optional argument, of the same length as the MirMetaData object, which is applied to the output of the selection parsing through an element-wise “and” operation. Useful for combining multiple calls to where together.
return_header_keys (bool) – If set to True, return a list of the header key values where matching entries are found. Default is False if supplying a field name for select_field, and True if supplying a code type for select_field.
- Returns:
return_arr (ndarray of bool or list) – If return_header_keys=False, boolean array marking whether select_field meets the condition set by select_comp and select_val. If return_header_keys=True, then instead of a boolean array, a list of ints (or tuples of ints if the MetaDataObject has only a pseudo header key) corresponding to the header key values. Note that if a code type was supplied for select_field and return_header_keys was not set to False, then the function will return a list of the matching index codes (“icode”) for the given code type.
- Raises:
ValueError – If select_comp is not one of the permitted strings, or if select_field is not one of the fields within the metadata, or a valid code type. Also raised if setting select_comp to anything but “eq” or “ne” when selecting on a code type (other operations not allowed since they are nonsensical for strings).
MirMetaError – If select_field does not match the metadata field types or any of the indexing codes.
- get_codes(code_name=None, return_dict=None)[source]
Get code strings for a given variable name in the metadata.
Look up the code strings for a given variable name (v_name), which typically contain information about the data set as a whole, or information for mapping indexing data from other MirMetaData objects to more easily understood strings of text.
- Parameters:
code_name (str) – Name of the codes, a full listing of which can be provided by the method get_code_name.
return_dict (bool) – If set to True, return a dict with keys and values that map code strings to indexing values, and visa-versa. Useful for mapping values between other MirCodesData and other MirMetaData object types. Default is None, which will return a dict only if code_name has more than one entry or has a known counterpart field in one of the other MirMetaData object types (e.g., “source” maps to “isource” in MirInData).
- Returns:
codes (list or dict) – If return_dict=False, then a list for all code strings is returned. Otherwise, a dict is returned which maps both indexing codes to code strings and visa-versa.
- class pyuvdata.uvdata.mir_meta_data.MirAntposData(obj=None)[source]
Class for antenna position information in Mir datasets.
This class is a container for antenna positions, which are recorded as a text file within a Mir dataset named “antennas”. It has a header key of “antenna”, which is paired to the antenna number in other metadata objects (e.g., “antenna”, “iant1”, “iant2”).
- class pyuvdata.uvdata.mir_meta_data.MirAcData(obj=None, nchunks=None)[source]
Class for per-track metadata in Mir datasets.
This class is a container for per-auto correlation metadata using the header key “achid”. At present, this class is a “synthetic” metadata object, in that it does not match to a natively written file on disk, as recorded by the online system (although it will read and write to the filename “ac_read”). As such, this class should be considered a “work in progress”, whose functionality may evolve considerably in future releases.
UVFlag Functions
Some useful flag handling functions.
- pyuvdata.uvflag.uvflag.and_rows_cols(waterfall)[source]
Perform logical and over rows and cols of a waterfall.
For a 2D flag waterfall, flag pixels only if fully flagged along time and/or frequency
- Parameters:
waterfall (2D boolean array of shape (Ntimes, Nfreqs))
- Returns:
wf (2D array) – A 2D array (size same as input) where only times/integrations that were fully flagged are flagged.
- pyuvdata.uvflag.uvflag.flags2waterfall(uv, *, flag_array=None, keep_pol=False)[source]
Convert a flag array to a 2D waterfall of dimensions (Ntimes, Nfreqs).
Averages over baselines and polarizations (in the case of visibility data), or antennas and jones parameters (in case of calibrationd data).
- Parameters:
uv (A UVData or UVCal object) – Object defines the times and frequencies, and supplies the flag_array to convert (if flag_array not specified)
flag_array (Optional,) – flag array to convert instead of uv.flag_array. Must have same dimensions as uv.flag_array.
keep_pol (bool) – Option to keep the polarization axis intact.
- Returns:
waterfall (2D array or 3D array) – Waterfall of averaged flags, for example fraction of baselines which are flagged for every time and frequency (in case of UVData input) Size is (Ntimes, Nfreqs) or (Ntimes, Nfreqs, Npols).