map module

3D maps (electron density, coulombic potential…).

Module author: Michael S. Chapman <chapmanms@missouri.edu>

Authors:

Michael S. Chapman <chapmanms@missouri.edu>

Oregon Health & Science University & University of Missouri

Version:

1, Nov 26, 2024

Changed in version 0.1.0: 8/13/10 Started

Changed in version 0.5.0: 2/13/15 Xplor/CNS map support for Scale/Offset (aka RAve, RSigma).

Changed in version 0.5.0: 2/17/15 Maps written in Xplor/CNS format.

Changed in version 0.5.1: 8/11/15 Forgives missing Scale/Offset w/ warning.

Changed in version 1.0.0: 11/18/20 Python 2.7 –> 3.6

Changed in version 1.0.2: 01/17/21 mrcfile replacing bsoft for .mcr, .ccp4 input.

class map.Arguments(imports=[], main=None, *args, **kwargs)

Bases: Arguments

Manager for command-line arguments from main and imported modules.

This is a template to be copied into modules, for the handling of command- line options.

methods export() and domestic() should be overridden by the module subclass with declarations of the command-line arguments needed by the module.

Parameters:

  • imports ((list of) ArgumentParser(s)) – (list of) module(s) from which to obtain “parent” ArgumentParser objects for inclusion.

  • main (bool|NoneType) – Add information and arguments appropriate for a main program (or not); if None, will determine by whether this class is defined in __main__.

domestic()

Defines options used only when main program and not when imported.

export()

Defines options used in both stand-alone and imported modes.

class map.Map

Bases: MapOrthogonal

Container for general 3D density map and its properties.

Class allows great freedom in the choice of grid in terms of origin, grid steps and orientation. This means that any arbitrary volume (including negative coordinates, and those well outside the unit cell) can be represented with positive indices in the map array. Alignment with a crystal unit cell is possible, but not required. There is no implicit support for symmetry or periodicity.

Variables:

density – in a row-major (C-like) array with slow, medium & fast axes density[s,m,f].

Initialize operators to interconvert map indices & orthogonal coords.

The forward operator consists of a scaling/rotation matrix and a vector which defines the position of the first map element ([0,0,0]). It will be constructed one of two ways, depending on which named arguments are provided:

  1. Calculated from crystallographic parmeters if named argument orthogonalize_unit_cell is provided - see method from_unit_cell().

  2. Direct input of matrix & ortho_origin (default) see method new().

Note: Most crystallographic maps are calculated on grids parallel to the unit cell axes, for which option 1 will be appropriate. However, this is not required here. If the density has been (re-)mapped to a different grid, or used an arbitrary grid placed over a non-periodic structure (electron microscopy), then option 2 would be appropriate. Indeed, some calculations may be more efficient following interpolation of crystallographic maps onto orthogonal grids (prior to this call).

Variables:

  • ~.orthog (numpy.matrix(shape=(3,3))) – matrix to scale/rotate grid to orthonormal Angstrom coord.

  • origin (numpy.matrix(shape=(3,1))) – translation of map origin to orthonormal Angstrom coordinates. Not sure that shape=(3,1); could be shape=(1,3).

  • orthoxyz (bool) – true if map grid orthonormal in (‘X’,’Y’,’Z’) orientation.

  • ~.volume (float) – pixel volume (Angstrom^2).

  • spacing (ndarray(type=float, shape=3)) – perpendicular distances between faces [A, B, C] of the pixel.

  • orthonormal (bool) – True if pixel has orthonormal shape

local_view(about=(3.0, 4.0, 5.0), extent=(1, 5, 5), verbose=True)

Reports density of map near “about”.

Parameters:

  • about (tuple or array (3)) – Orthogonal Angstrom coordinates.

  • extent (tuple (3)) – Size of the viewing box.

  • verbose (bool) – print the density view.

Returns:

(density-view, index of 1st corner, index of local maximum)

Return type:

ndarray(type=float, shape=extent), tuple(3), tuple(3)

zeros_like()

Creates a new empty map with the same gridding as self.

Returns:

a deepcopy of map self on the same grid as self.

Return type:

class Map

class map.ReadMap(filename, permutation='ZYX', magnification=1.0, new_cell=None, normalize=True, verbose=False)

Bases: Map

Container for a map read from an external file.

Variables:

  • density – in a row-major (C-like) array with slow, medium & fast axes density[s,m,f].

  • header – documentation

  • abc_grid – ((a_cell, a_min, a_max), (b_cell, b_min, b_max), (c_cell, c_min, c_max)), containing, for each crystal axis, the number of grid spaces per unit cell, the first and last in the map. The number of grid spaces is the number of grid points less one, if the last is the repeat of the zeroth (a closed cell), or the number of grid points for an open cell.

  • space_group (int or NoneType) – number, if provided by map format

Reads an external map into memory.

The header is parsed and stored in file-type specific attributes, such as self.abc_grids. Information is also converted into Map instance attributes that are universal across all map formats.

Parameters:

  • permutation (str) – which axes align with the (slow, medium, fast) indices of the map array? Ignored for Xplor (as read from the map). Some other maps support different permutations (eg. ccp4), but bsoft does not appear to extract the information from the header.

  • magnification (float) – > 1.0 increase the object’s (EM) magnification by decreasing the map (pixel) dimensions, equivalent to reducing unit cell size.

  • new_cell (tuple(6 x float) or NoneType) – Unit cell parameters to use instead of map header. Use very carefully, as could affect map-model compatibility.

  • normalize (bool) – scale to mean of 0.0 and standard deviation of 1.0 x pixel volume

  • verbose (bool) – echo file name etc.

Variables:

  • permutation (list(str)) – eg [‘X’, ‘Y’, ‘Z’] required for xplor format.

  • unit_cell (list(float)) – (a, b, c, alpha, beta, gamma; Angstrom & degrees)

  • _magnification (float) – cumulative magnification that has been applied.

  • ~.scale (float) – internal values of density are (file values) * scale + offset

  • offset (float) – internal values of density are (file values) * scale + offset

class Scale

Bases: object

Maintains relation: density values = (file values) * scale + offset

input(density, scale, offset)

Input new coefficients and overwrite density w/ scaled values

Parameters:

  • density (ndarray) – unscaled values directly from input file

  • scale (float) – density values to be first multiplied by this

  • offset (float) – this value then to be added

Return internal:

Rtype ndarray:

output(density, original_scale=True, magnification=None)

Output density, scaled by the input coefficients.

Parameters:

  • density (ndarray) – internally-saved density values

  • original_scale (bool) – put map back on scale approximately matching input, undoing normalization & scaling to model (if relevant).

  • magnification (float or NoneType) – rescale output to maintain the same total number of electrons through specified change in magnification.

Return out, scale, offset:

density ready for output & coefficients

Rtype ndarray, float, float:

allow_no_file = False
Variables:

allow_no_file (bool) – allow limited functionality without a map file

compare(other, atol=1e-05, rtol=0.05)

Are two ReadMap objects the same, all elements close?

Parameters:

  • other (ReadMap)

  • atol (float or NoneType) – Absolute tolerance in map element values.

  • rtol (float or NoneType) – Relative tolerance in map element values.

Returns:

True if same

Return type:

bool

Tolerance is assessed as in numpy.testing.assert_allclose.

from_bsoft(filename, verbose=True)

Read map from variety of file formats using bsoft.

Parameters:

  • filename (string) – file name

  • verbose (bool) – diagnostics printed

Deprecated since version 1.0.2: (1/16/21) Default reader for .mrc, .ccp4, .map, and .ccp files will become from_ccpem, using the mrcfile package. This will remove the dependence on an extension to a legacy version of bsoft (v1.9.2). Bsoft does support a wider array of formats, so there will be ongoing evaluation of the need to maintain with the dependency no longer consider required. Thus, consider deprecated in part.

Warning

acell, bcell & ccell from our bsoft interface are wrong, and look like the numbers of grid points in each direction. Have kluged, recalculating from the mx, my, mz and the spacings, issuing a warning. I am not aware that this has been a problem for RSRef, so this may have been a “sleeper” bug. Note that the kluged values can be over-ridden with the new_cell argument when intializing Readmap.

Warning

BSoft assumes a (1, 2, 3) map orientation (x fastest, z sections) and ignores any header information that indicates a different orientation. This has been a convention in EM, as referenced in bsoft.bsoft.i.

from_ccpem(filename, verbose=True)

Read MRC/CCP4 map using CCP-EM python library.

Parameters:

  • filename (string) – file name

  • verbose (bool) – diagnostics printed

Format definition is in: https://www.ccpem.ac.uk/mrc_format/mrc2014.php, with an accompanying paper. Appears that the format is intended to maintain compatibility between EM and macromolecular crystallography. Installation etc.: https://mrcfile.readthedocs.io/en/latest/ . Source documentation: https://mrcfile.readthedocs.io/en/latest/source/mrcfile.html .

Burnley T, Palmer C & Winn M (2017) Recent developments in the CCP-EM software suite. Acta Cryst. D73:469–477. doi: 10.1107/S2059798317007859

Warning

01/15/21 Seeking clarity on map permutation

The following is important only for maps that are not in the usual orientation of (mapc, mapr, maps) = (1, 2, 3) for factional coordinate x along the crystallographic a-axis changing (fastest) with column, y along b with row, and z along c slowest with section.

The format documentation makes it clear that (NX, NY, NZ) refer to (columns/fast, rows/medium, sections/slow) respectively, so these variables might, more aptly be named (NC, NR, NS) as in the legacy format definition and there appears no intent to associate with specific crystal axes.

(NXSTART, NYSTART, NZSTART) cannot be “locations … in unit cell” (as documented). From dtypes.py, and also the legacy format definition, it appears that these integers are the grid coordinates of the first pixel of the map array. Both new & old format definitions indicate column, row & section, so we should probably ignore any inference of crystallographic axes (x, y, z) in the variable names. Indeed, in the legacy format definition, these words in the header are more appropriately named: NCSTART, NRSTART, NSSTART.

(MX, MY, MZ) appear to be the number of voxels that would span each direction for a full unit cell. Both current & legacy format descriptions as well as module mrcfile.dtypes state that these pertain to the (x, y, z) directions. This is inconsistent with multiple examples in EM programs, comparing NZ and MZ without permuting the axes. So it appears that either:

  • EM software has diverged from crystallographic such that the format is common only for (MAPC, MAPR, MAPS) = (1, 2, 3).

  • The documentation for (MX, MY, MZ) is incorrect, and these really refer to column, row and section directions.

abc_grid and subsequent calculation of matrices that map cartesian Angstrom coordinates to map grid space require hashing of the map header parameters according to the map permuation parameters (mapc, mapr, maps). If the documentation for (mx, my, mz) is correct, then (mx, my, mz) would be the exception to hashing, because they already corresond to the (x, y, z) or (a, b, c) directions respectively.

When such hashing is assumed for all header parameters except (mx, my, mz), map-model correlation remains high for a test map that is re-oriented using the crystallographic program mapman.

If we assume that the test validates our interpretation, this would mean that MZ and NZ would not be comparable for permuations other than (mapc, mapr, maps) = (1, 2, 3). It therefore seems that (some or all) EM programs assume a (1, 2, 3) orientation, apparently acknowledged in sources cited in bsoft.bsoft.i, perhaps explaining why variables once called (NC, NR, NS) are now (nx, ny, nz) in the mrcfile definition. With a unified format, the new notation will confusing for crystallographic applications that should support multiple map orientations.

It is unclear whether any EM software supports map writing in different orientations, and whether the header is then written correctly. Clarification has been sought from ccpem (1/15/21), and until then a warning is written for all non-(1, 2, 3) orientations that poor model-map agreement could be the result of an incorrect header in an EM map.

from_file(filename, verbose=True)

Read map from a variety of file types.

If no file name provided, prepare for a minimalist run.

Parameters:

  • filename (str)

  • verbose (bool)

Returns:

map read successfully.

Return type:

bool

Changed in version 1.0.1: Default reader for .ccp4, .mrc, .ccp & .map files is switched from bsoft to ccpem (mrcfile). To override and use bsoft, within program, execute the following within the python interpreter::

py

from pasto import map

map.ReadMap.use_mrcfile=False

exit()

from_xplor(filename, verbose=True)

Read from X-plor formatted file.

Parameters:

  • filename (string) – file name

  • verbose (bool) – diagnostics printed

magnify(magnification=1.0)

Change the (EM) magnification.

Parameters:

magnification (float) – values > 1.0 increase the object’s (EM) magnification by decreasing the map (pixel) dimensions, equivalent to reducing unit cell size.

Returns:

magnification

Return type:

float

to_file(filename, verbose=True)

Write map to a variety of file types.

Parameters:

  • filename (str)

  • verbose (bool)

Returns:

map written successfully.

Return type:

bool

to_xplor(filename, autodoc=False, verbose=True)

Write to X-plor formatted file.

Parameters:

  • filename (str) – file name

  • autodoc (bool) – appends to header, file name, program, date, user

  • verbose (bool) – diagnostics printed

Error

The final line of xplor/cns map files is often mis-understood with some programs setting offset & scale (RAVE & RSIGMA in CNS) arbitrarily to 0.0 & 1.0 (incl. phenix), others ignoring the line. CNS xmread.f:312 shows stored density = input * RSIGMA + RAVE and CNS xmwrite.f:600 output density = (stored - RAVE) / RSIGMA. RAVE and RSIGMA should usually be the average, standard deviation for a crystal asymmetric unit, which is why they do not match the average & standard deviation for some arbitrary map volume. CNS print statements suggest that densities are on an absolute (e/Å3) scale, so RAVE, RSIGMA are converting between absolute and standard deviation scales. As our program supports maps of arbitrary volume, and does not consider asymmetric units, the appropriate values cannot be calculated. Values obtained from an input xplor/cns map will be used, optionally modified to maintain an absolute scale with changing magnification.

use_mrcfile = True
Variables:

use_mrcfile (bool) – ccp4/ccpem for .mrc, .ccp4, .ccp & .map maps, not bsoft