UVFlag

UVFlag is a main user class that is still in the development and beta-testing phase, which holds metadata and data related to flagging and metric information for interferometric data sets. It provides import and export functionality to and from all file formats supported by pyuvdata.UVData and pyuvdata.UVCal objects as well as an HDF5 file format specified by this object. It supports three different “shapes” of data (all with time and frequency axes): visibility based, antenna based, and waterfall (a single value for the entire array at each time and frequency). It has methods for transforming the data between different shapes and converting metrics to flags, and can be interacted with directly.

Attributes

The attributes on UVFlag hold all of the metadata and data required to specify flagging and metric information for interferometric data sets. Under the hood, the attributes are implemented as properties based on pyuvdata.parameter.UVParameter objects but this is fairly transparent to users.

UVFlag objects can be initialized from a file or a pyuvdata.UVData or pyuvdata.UVCal object (as flag = UVFlag(<filename or object>)). Some of these attributes are required to be set to have a fully defined data set while others are optional. The pyuvdata.UVFlag.check() method can be called on the object to verify that all of the required attributes have been set in a consistent way.

Required

These parameters are required to have a sensible UVFlag object and are required for most kinds of uv data files.

Nants_data

Number of antennas with data present. Only available for “baseline” or “antenna” type objects.May be smaller than the number of antennas in the array

Nants_telescope

Number of antennas in the array. Only available for “baseline” type objects. May be larger than the number of antennas with data.

Nbls

Number of baselines. Only Required for “baseline” type objects.

Nblts

Number of baseline-times (i.e. number of spectra). Not necessarily equal to Nbls * Ntimes

Nfreqs

Number of frequency channels

Npols

Number of polarizations

Nspws

Number of spectral windows (ie non-contiguous spectral chunks). More than one spectral window is not currently supported.

Ntimes

Number of times

ant_1_array

Array of first antenna indices, shape (Nblts). Only available for “baseline” type objects. type = int, 0 indexed

ant_2_array

Array of second antenna indices, shape (Nblts). Only available for “baseline” type objects. type = int, 0 indexed

baseline_array

Array of baseline indices, shape (Nblts). Only available for “baseline” type objects. type = int; baseline = 2048 * (ant1+1) + (ant2+1) + 2^16

freq_array

Array of frequencies, center of the channel, shape (Nspws, Nfreqs), units Hz

history

String of history, units English

label

String used for labeling the object (e.g. ‘FM’). Default is empty string.

lst_array

Array of lsts, center of integration, shape (Nblts), units radians

metric_array

Floating point metric information, only availble in metric mode. shape (Nblts, Nspws, Nfreq, Npols).

mode

The mode determines whether the object has a floating point metric_array or a boolean flag_array. Options: {“metric”, “flag”}. Default is “metric”.

polarization_array

Array of polarization integers, shape (Npols). AIPS Memo 117 says: pseudo-stokes 1:4 (pI, pQ, pU, pV); circular -1:-4 (RR, LL, RL, LR); linear -5:-8 (XX, YY, XY, YX). NOTE: AIPS Memo 117 actually calls the pseudo-Stokes polarizations “Stokes”, but this is inaccurate as visibilities cannot be in true Stokes polarizations for physical antennas. We adopt the term pseudo-Stokes to refer to linear combinations of instrumental visibility polarizations (e.g. pI = xx + yy).

time_array

Array of times, center of integration, shape (Nblts), units Julian Date

type

The type of object defines the form of some arrays and also how metrics/flags are combined. Accepted types:”waterfall”, “baseline”, “antenna”

weights_array

Floating point weight information, shape (Nblts, Nspws, Nfreq, Npols).

Optional

These parameters are defined by one or more type but are not always required. Some of them are required depending on the type (as noted below).

ant_array

Array of antenna numbers, shape (Nants_data), Only available for “antenna” type objects. type = int, 0 indexed

extra_keywords

Any user supplied extra keywords, type=dict.Use the special key ‘comment’ for long multi-line string comments.Default is an empty dictionary.

filename

List of strings containing the unique basenames (not the full path) of input files.

flag_array

Boolean flag, True is flagged, only availble in flag mode. shape (Nblts, Nspws, Nfreq, Npols).

weights_square_array

Floating point weight information about sum of squares of weights when weighted data converted from baseline to waterfall mode.

x_orientation

Orientation of the physical dipole corresponding to what is labelled as the x polarization. Options are “east” (indicating east/west orientation) and “north” (indicating north/south orientation)

Methods

class pyuvdata.UVFlag(indata=None, mode='metric', copy_flags=False, waterfall=False, history='', label='', run_check=True, check_extra=True, run_check_acceptability=True)[source]

Object to handle flag arrays and waterfalls for interferometric datasets.

Supports reading/writing, and stores all relevant information to combine flags and apply to data. Initialization of the UVFlag object requires some parameters. Metadata is copied from indata object. If indata is subclass of UVData or UVCal, the weights_array will be set to all ones. Lists or tuples are iterated through, treating each entry with an individual UVFlag init.

Parameters
  • indata (UVData, UVCal, str, pathlib.Path, list of compatible combination) – Input to initialize UVFlag object. If str, assumed to be path to previously saved UVFlag object. UVData and UVCal objects cannot be directly combined, unless waterfall is True.

  • mode ({“metric”, “flag”}, optional) – The mode determines whether the object has a floating point metric_array or a boolean flag_array.

  • copy_flags (bool, optional) – Whether to copy flags from indata to new UVFlag object

  • waterfall (bool, optional) – Whether to immediately initialize as a waterfall object, with flag/metric axes: time, frequency, polarization.

  • history (str, optional) – History string to attach to object.

  • extra_keywords (dict, optional) – A dictionary of metadata values not explicitly specified by another parameter.

  • label (str, optional) – String used for labeling the object (e.g. ‘FM’).

  • run_check (bool) – Option to check for the existence and proper shapes of parameters after creating UVFlag object.

  • check_extra (bool) – Option to check optional parameters as well as required ones (the default is True, meaning the optional parameters will be checked).

  • run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after creating UVFlag object.

UVParameter objects

For full list see the UVFlag Parameters Documentation. (https://pyuvdata.readthedocs.io/en/latest/uvflag_parameters.html) Some are always required, some are required for certain phase_types and others are always optional.

property data_like_parameters

Return iterator of defined parameters which are data-like.

property pol_collapsed

Determine if this object has had pols collapsed.

clear_unused_attributes()[source]

Remove unused attributes.

Useful when changing type or mode or to save memory. Will set all non-required attributes to None, except x_orientation, extra_keywords, weights_square_array and filename.

antpair2ind(ant1, ant2)[source]

Get blt indices for given (ordered) antenna pair.

Parameters
  • ant1 (int or array_like of int) – Number of the first antenna

  • ant2 (int or array_like of int) – Number of the second antenna

Returns

int or array_like of int – baseline number(s) corresponding to the input antenna number

baseline_to_antnums(baseline)[source]

Get the antenna numbers corresponding to a given baseline number.

Parameters

baseline (int) – baseline number

Returns

tuple – Antenna numbers corresponding to baseline.

get_baseline_nums()[source]

Return numpy array of unique baseline numbers in data.

get_antpairs()[source]

Return list of unique antpair tuples (ant1, ant2) in data.

get_ants()[source]

Get the unique antennas that have data associated with them.

Returns

ndarray of int – Array of unique antennas with data associated with them.

get_pols()[source]

Get the polarizations in the data.

Returns

list of str – list of polarizations (as strings) in the data.

parse_ants(ant_str, print_toggle=False)[source]

Get antpair and polarization from parsing an aipy-style ant string.

Used to support the select function. This function is only useable when the UVFlag type is ‘baseline’. Generates two lists of antenna pair tuples and polarization indices based on parsing of the string ant_str. If no valid polarizations (pseudo-Stokes params, or combinations of [lr] or [xy]) or antenna numbers are found in ant_str, ant_pairs_nums and polarizations are returned as None.

Parameters
  • ant_str (str) – String containing antenna information to parse. Can be ‘all’, ‘auto’, ‘cross’, or combinations of antenna numbers and polarization indicators ‘l’ and ‘r’ or ‘x’ and ‘y’. Minus signs can also be used in front of an antenna number or baseline to exclude it from being output in ant_pairs_nums. If ant_str has a minus sign as the first character, ‘all,’ will be appended to the beginning of the string. See the tutorial for examples of valid strings and their behavior.

  • print_toggle (bool) – Boolean for printing parsed baselines for a visual user check.

Returns

  • ant_pairs_nums (list of tuples of int or None) – List of tuples containing the parsed pairs of antenna numbers, or None if ant_str is ‘all’ or a pseudo-Stokes polarizations.

  • polarizations (list of int or None) – List of desired polarizations or None if ant_str does not contain a polarization specification.

collapse_pol(method='quadmean', run_check=True, check_extra=True, run_check_acceptability=True)[source]

Collapse the polarization axis using a given method.

If the original UVFlag object has more than one polarization, the resulting polarization_array will be a single element array with a comma separated string encoding the original polarizations.

Parameters
  • method (str, {“quadmean”, “absmean”, “mean”, “or”, “and”}) – How to collapse the dimension(s).

  • run_check (bool) – Option to check for the existence and proper shapes of parameters after collapsing polarizations.

  • check_extra (bool) – Option to check optional parameters as well as required ones.

  • run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after collapsing polarizations.

to_waterfall(method='quadmean', keep_pol=True, run_check=True, check_extra=True, run_check_acceptability=True, return_weights_square=False)[source]

Convert an ‘antenna’ or ‘baseline’ type object to waterfall.

Parameters
  • method (str, {“quadmean”, “absmean”, “mean”, “or”, “and”}) – How to collapse the dimension(s).

  • keep_pol (bool) – Whether to also collapse the polarization dimension If keep_pol is False, and the original UVFlag object has more than one polarization, the resulting polarization_array will be a single element array with a comma separated string encoding the original polarizations.

  • run_check (bool) – Option to check for the existence and proper shapes of parameters after converting to waterfall type.

  • check_extra (bool) – Option to check optional parameters as well as required ones.

  • run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after converting to waterfall type.

  • return_weights_square (bool) – Option to compute the sum of the squares of the weights when collapsing baseline object to waterfall. Not used if type is not baseline to begin with. Fills an optional parameter if so.

to_baseline(uv, force_pol=False, run_check=True, check_extra=True, run_check_acceptability=True)[source]

Convert a UVFlag object of type “waterfall” to type “baseline”.

Broadcasts the flag array to all baselines. This function does NOT apply flags to uv.

Parameters
  • uv (UVData or UVFlag object) – Objcet with type baseline to match.

  • force_pol (bool) – If True, will use 1 pol to broadcast to any other pol. Otherwise, will require polarizations match. For example, this keyword is useful if one flags on all pols combined, and wants to broadcast back to individual pols.

  • run_check (bool) – Option to check for the existence and proper shapes of parameters after converting to baseline type.

  • check_extra (bool) – Option to check optional parameters as well as required ones.

  • run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after converting to baseline type.

to_antenna(uv, force_pol=False, run_check=True, check_extra=True, run_check_acceptability=True)[source]

Convert a UVFlag object of type “waterfall” to type “antenna”.

Broadcasts the flag array to all antennas. This function does NOT apply flags to uv.

Parameters
  • uv (UVCal or UVFlag object) – object of type antenna to match.

  • force_pol (bool) – If True, will use 1 pol to broadcast to any other pol. Otherwise, will require polarizations match. For example, this keyword is useful if one flags on all pols combined, and wants to broadcast back to individual pols.

  • run_check (bool) – Option to check for the existence and proper shapes of parameters after converting to antenna type.

  • check_extra (bool) – Option to check optional parameters as well as required ones.

  • run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after converting to antenna type.

to_flag(threshold=inf, run_check=True, check_extra=True, run_check_acceptability=True)[source]

Convert to flag mode.

This function is NOT SMART. Removes metric_array and creates a flag_array from a simple threshold on the metric values.

Parameters
  • threshold (float) – Metric value over which the corresponding flag is set to True. Default is np.inf, which results in flags of all False.

  • run_check (bool) – Option to check for the existence and proper shapes of parameters after converting to flag mode.

  • check_extra (bool) – Option to check optional parameters as well as required ones.

  • run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after converting to flag mode.

to_metric(convert_wgts=False, run_check=True, check_extra=True, run_check_acceptability=True)[source]

Convert to metric mode.

This function is NOT SMART. Simply recasts flag_array as float and uses this as the metric array.

Parameters
  • convert_wgts (bool) – if True convert self.weights_array to ones unless a column or row is completely flagged, in which case convert those pixels to zero. This is used when reinterpretting flags as metrics to calculate flag fraction. Zero weighting completely flagged rows/columns prevents those from counting against a threshold along the other dimension.

  • run_check (bool) – Option to check for the existence and proper shapes of parameters after converting to metric mode.

  • check_extra (bool) – Option to check optional parameters as well as required ones.

  • run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after converting to metric mode.

combine_metrics(others, method='quadmean', inplace=True, run_check=True, check_extra=True, run_check_acceptability=True)[source]

Combine metric arrays between different UVFlag objects together.

Parameters
  • others (UVFlag or list of UVFlags) – Other UVFlag objects to combine metrics with this one.

  • method (str, {“quadmean”, “absmean”, “mean”, “or”, “and”}) – Method to combine metrics.

  • inplace (bool, optional) – Perform combination in place.

Returns

uvf (UVFlag) – If inplace==False, return new UVFlag object with combined metrics.

select(antenna_nums=None, ant_inds=None, bls=None, ant_str=None, frequencies=None, freq_chans=None, times=None, polarizations=None, blt_inds=None, run_check=True, check_extra=True, run_check_acceptability=True, inplace=True)[source]

Downselect data to keep on the object along various axes.

Axes that can be selected along depend on the current type of the object. However some axis may always be selected upon, these include frequencies, times and polarizations. In “baseline” and “antenna” modes, antenna numbers may be selected. In “baseline” mode, antenna pairs may be selected. Specific baseline-time indices can also be selected in “baseline” mode, but this is not commonly used. The history attribute on the object will be updated to identify the operations performed.

Parameters
  • antenna_nums (array_like of int, optional) – The antennas numbers to keep in the object (antenna positions and names for the removed antennas will be retained unless keep_all_metadata is False). This cannot be provided if antenna_names is also provided.

  • bls (list of tuple, optional) – A list of antenna number tuples (e.g. [(0,1), (3,2)]) or a list of baseline 3-tuples (e.g. [(0,1,’xx’), (2,3,’yy’)]) specifying baselines to keep in the object. For length-2 tuples, the ordering of the numbers within the tuple does not matter. For length-3 tuples, the polarization string is in the order of the two antennas. If length-3 tuples are provided, polarizations must be None.

  • ant_str (str, optional) – A string containing information about what antenna numbers and polarizations to keep in the object. Can be ‘auto’, ‘cross’, ‘all’, or combinations of antenna numbers and polarizations (e.g. ‘1’, ‘1_2’, ‘1x_2y’). See tutorial for more examples of valid strings and the behavior of different forms for ant_str. If ‘1x_2y,2y_3y’ is passed, both polarizations ‘xy’ and ‘yy’ will be kept for both baselines (1, 2) and (2, 3) to return a valid pyuvdata object. An ant_str cannot be passed in addition to any of antenna_nums, antenna_names, bls args or the polarizations parameters, if it is a ValueError will be raised.

  • frequencies (array_like of float, optional) – The frequencies to keep in the object, each value passed here should exist in the freq_array.

  • freq_chans (array_like of int, optional) – The frequency channel numbers to keep in the object.

  • times (array_like of float, optional) – The times to keep in the object, each value passed here should exist in the time_array.

  • polarizations (array_like of int or str, optional) – The polarizations numbers to keep in the object, each value passed here should exist in the polarization_array. If passing strings, the canonical polarization strings (e.g. “xx”, “rr”) are supported and if the x_orientation attribute is set, the physical dipole strings (e.g. “nn”, “ee”) are also supported.

  • blt_inds (array_like of int, optional) – The baseline-time indices to keep in the object. This is not commonly used.

  • ant_inds (array_like of int, optional) – The antenna indices to keep in the object. This is not commonly used.

  • run_check (bool) – Option to check for the existence and proper shapes of parameters after downselecting data on this object.

  • check_extra (bool) – Option to check optional parameters as well as required ones (the default is True, meaning the optional parameters will be checked).

  • run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after downselecting data on this object.

  • inplace (bool) – Option to perform the select directly on self or return a new UVData object with just the selected data.

Returns

UVData object or None – None is returned if inplace is True, otherwise a new UVData object with just the selected data is returned

Raises

ValueError – If any of the parameters are set to inappropriate values.

read(filename, history='', run_check=True, check_extra=True, run_check_acceptability=True)[source]

Read in flag/metric data from a HDF5 file.

Parameters
  • filename (str or pathlib.Path) – The file name to read.

  • history (str) – History string to append to UVFlag history attribute.

  • run_check (bool) – Option to check for the existence and proper shapes of parameters after reading data.

  • check_extra (bool) – Option to check optional parameters as well as required ones.

  • run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after reading data.

write(filename, clobber=False, data_compression='lzf')[source]

Write a UVFlag object to a hdf5 file.

Parameters
  • filename (str) – The file to write to.

  • clobber (bool) – Option to overwrite the file if it already exists.

  • data_compression (str) – HDF5 filter to apply when writing the data_array. If no compression is wanted, set to None.

from_uvdata(indata, mode='metric', copy_flags=False, waterfall=False, history='', label='', run_check=True, check_extra=True, run_check_acceptability=True)[source]

Construct a UVFlag object from a UVData object.

Parameters
  • indata (UVData) – Input to initialize UVFlag object.

  • mode ({“metric”, “flag”}, optional) – The mode determines whether the object has a floating point metric_array or a boolean flag_array.

  • copy_flags (bool, optional) – Whether to copy flags from indata to new UVFlag object

  • waterfall (bool, optional) – Whether to immediately initialize as a waterfall object, with flag/metric axes: time, frequency, polarization.

  • history (str, optional) – History string to attach to object.

  • label (str, optional) – String used for labeling the object (e.g. ‘FM’).

  • run_check (bool) – Option to check for the existence and proper shapes of parameters after creating UVFlag object.

  • check_extra (bool) – Option to check optional parameters as well as required ones (the default is True, meaning the optional parameters will be checked).

  • run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after creating UVFlag object.

from_uvcal(indata, mode='metric', copy_flags=False, waterfall=False, history='', label='', run_check=True, check_extra=True, run_check_acceptability=True)[source]

Construct a UVFlag object from a UVCal object.

Parameters
  • indata (UVData) – Input to initialize UVFlag object.

  • mode ({“metric”, “flag”}, optional) – The mode determines whether the object has a floating point metric_array or a boolean flag_array.

  • copy_flags (bool, optional) – Whether to copy flags from indata to new UVFlag object

  • waterfall (bool, optional) – Whether to immediately initialize as a waterfall object, with flag/metric axes: time, frequency, polarization.

  • history (str, optional) – History string to attach to object.

  • label (str, optional) – String used for labeling the object (e.g. ‘FM’).

  • run_check (bool) – Option to check for the existence and proper shapes of parameters after creating UVFlag object.

  • check_extra (bool) – Option to check optional parameters as well as required ones (the default is True, meaning the optional parameters will be checked).

  • run_check_acceptability (bool) – Option to check acceptable range of the values of parameters after creating UVFlag object.

last updated: 2022-02-10