density module¶
Electron density calculated from individual atoms.
Module author: Michael S. Chapman <chapmami@ohsu.edu>
Authors: | Michael S. Chapman <chapmami@ohsu.edu>, Oregon Health & Science University |
---|---|
Version: | 0.5, March 23, 2016 |
Changed in version 02/14/2010.
Changed in version 11/4/12: - split from atomic_density.py
Changed in version 0.4.2: (10/24/13)
Changed in version 0.5.0: (5/1/15) ReStructured Text documentation
Todo
partials calculation - exclude parameter types that are not needed. Make xyz the default (as per CNS), else read from “refine”.
-
class
density.
Arguments
(imports=[], main=False, *args, **kwargs)¶ Bases:
argparser.ArgumentParser
Command-line arguments for atomic density calculation / map comparison.
Parameters: - imports ((list of) ArgumentParser method(s)) – method(s) adding arguments to ArgumentParser instance. (Alternative to ArgumentParser parents w/ better maintained groups.)
- main (bool) – Parser for main program (i.e. not listed within an ArgumentParser parents argument). Adds default program information.
-
static
export
()¶
-
exception
density.
ArrayBoundsError
(*args)¶ Bases:
exceptions.Exception
Wrap pending exception (from numpy), if any, in addition to other info.
-
class
density.
AtomicDensity
(method='dstar-shells', form_factors=None, d_min=None, d_max=None, shells=15, filter_resolution=None, use_cache=True, cut_off=7.5, verbose=False)¶ Bases:
object
Electron density surrounding isolated atom by various methods.
Density calculation can be accelerated through the use of previously cached densities from atoms of the same type and same (similar!) B-factor. See “cache_*” methods below. If enabled, method density will take advantage of the cache. The cache stores tables of density (and partial derivatives) with respect to distance (from center of atom) and rasterized B-factor. The default caching has some limitations:
Caching requires that the atomic density cut-off distance (beyond which density assumed zero) remain constant, as it should for a run of RSRef (some checks provided).
Chaching ignores differences less than b_precision (see self.cache_init()). So, there are limitations on its use during B-factor/envelope refinement:
For refinement of overall B-factors or EM envelope or the resolution of a low-pass Butterworth filter, previous densities need to be purged with cache_reset(). Caching will still help in calculating densities for atoms with identical individual B-factors.
For refinement of individual B-factors, caching should be disabled with cache_off() as caching will ignore the effects of small changes between cycles and risks one atom coming within b_precision of another. With freely changing B-factors, there is little to be gained by caching anyway.
With group B-factor refinement, it is safest to disable (cache_off()). Caching may speed computation if there are many atoms of the same B-factor in each group. If you want to risk it, use cache_reset() between cycles and use a small cache_init(b_precision=0.000001) to minimize the chance of atoms being accidentally considered the same.
Caching should also be disabled (cache_off()) with: (1) finite difference estimates of partial derivatives (unit-testing); and (2) direct (vs. interpolative) density calculation (method = ‘_calculate_all_pts’, used mostly for unit-testing).
Variables: - dst_limit – limits of the resolution bins, going from low to high resolution.
- dst – (geometric) mean resolution of each bin.
- dst_limit_cubed – limits (above) cubed.
- butterworth_order (float) – for low-pass filter on atomic density. If reset, re-bind self.filter_resolution to force re-calculation of the filter values.
Parameters: - method (str) – to be used in atomic density calculation.
Currently only
dstar-shells
(as implemented in the original 1995RSRef
, which requires parametersform_factors
,shells
,d_min
,d_max
. In the future, might also support spherical Gaussian and other possibilities. - form_factors (
form_factor_table.FormFactorTable
) – tabulated standard scattering factors for all atom types. - d_min (float) – high resolution limit (minimal d-spacing; Å).
- d_max (float) – low resolution limit (maximal d-spacing; Å).
- shells (int) – number of shells to use between d_min & d_max. Accuracy increases linearly & speed decreases with larger number. For a single C-atom (B=0), 5 shells gives 0.9% precision, 20 shells (overkill) 0.6%, compared to direct transform.
- filter_resolution (float) – Soft high resolution limit (Å) for a Butterworth 5th order low pass filter to be applied to the model density, or None
- use_cache (bool) – speed calculation by saving density (/partials) vs. distance, initial setting. See notes on AtomicDensity class documentation, and methods cache_on and cache_off etc..
- cut_off (float) – atomic density considered zero beyond this distance in Å
-
cache_init
(distance=None, b_precision=0.003, verbose=False)¶ Prepares cache for tables of atomic density vs. distance.
Parameters: - distance (float or ndarray-like(type=float)) – distances from atom for density calculation.
- b_precision (float) – B-factors considered same if w/in b_precision. Default is the formatting precision of a PDB file.
-
cache_off
(reset=False, verbose=False)¶ Switches off cache storage and use of atomic densities / derivatives.
Parameters: - reset (bool) – force clearing of the cache - use if don’t trust current values for future use.
- verbose (bool) – provide documentation.
-
cache_on
(distance=None, b_precision=None, reset=False, verbose=False)¶ Switches on cache storage and use of atomic densities / derivatives.
Parameters: - distance (float or ndarray-like(type=float)) – distances from atom for density calculation, checked if provided and cache reinitialized if necessary.
- b_precision (float) – B-factors considered same if w/in b_precision. Default is the formatting precision of a PDB file, checked if provided and cache reinitialized if necessary.
- reset (bool) – force clearing of the cache - use if don’t trust prior calculations.
- verbose (bool) – provide documentation.
-
cache_reset
(verbose=False)¶ Reset the density cache, called for example when added_b changed.
-
density
(distance=None, atom_type='C', type_num=None, b=0.0, b_refining=True, check_caching=False)¶ Electron density (& derivatives) @ distance(s) from atom, using caching.
Pass through routine for _density, adding support for caching.
Caching saves up to ~1.8 ms/atom, which can be 50% for small atom boxes (1/2-width of ~d_min). This depends on the variety of atom types and B. No measurable overhead for 10,000 atoms w/ different B-factors, but may be switched off. (Of greatest use when densities are cached with unit occupancies.)
Parameters: - distance (float or ndarray-like(type=float) or NoneType) – distance(s) from atom center (self.cache_distance if None).
- atom_type (str) – as in PDB file.
- type_num (int) – (optional) numerical atom type designator that has
precedence over
atom_type
. (Minor speed increase.) Must correspond toatom_type
inform_factors
. - b (float) – atomic temperature factor (Å2).
- b_refining (bool) – cache will store whether this atom has a B-factor that is being refined for a more atomic control of the use of cached data that is not yet implemented.
- check_caching (bool) – will perform a (computer intensive) check asserting that cached densities are the same as calculated de novo (for unit testing).
Returns: electron density at
distance
(s) from atom center.Return type: float or ndarray(type=float)
-
density_array
= <numpy.lib.function_base.vectorize object>¶
-
density_wo_cache
(distance, atom_type='C', type_num=None, b=0.0)¶ Electron density and partial derivatives at a distance from atom center.
Parameters: - distance (float or ndarray-like(type=float)) – distance(s) from atom center.
- atom_type (string) – as in PDB file.
- type_num (int) – (optional) numerical atom type designator that has
precedence over
atom_type
. (Minor speed increase.) Must correspond toatom_type
inform_factors
. - b (float, Å2) – atomic temperature factor (already incorporating overall B and /or envelope function).
Returns: electron density at
distance
(s) from atom center and its its derivatives with respect to distance, B-factor and resolution. (The resolution partial derivative is with respect to a low-pass filter to be applied to the atomic density.)Return type: 4 * (float or ndarray(type=float))
Note
Vectorized: if
distance
is array-like, the density is calculated at all distances in little more than the time for a single distance.Todo
Consider wrapping in another layer of vectorization for calculation over multiple atoms.
Todo
Consider class method that pre-calculates attributes that depend on distance and not on atom type or B-factor.
-
distances
(spacing=0.05)¶ Distances for density calculation for interpolation and caching.
Parameters: spacing (float) – spacing between points (to be multipled by high resolution limit). With linear interpolation, preliminary tests indicate no loss of precision with spacing = 1/20. With cubic interpolation, 1/2 may suffice. Computation may depend inversely on spacing. Returns: distances (Å) from atom center where density will be calculated. Return type: ndarray(dtype=float)
-
filter_resolution
¶
-
profile
(b=10.0, atom_type='C', max_dist=4.0, points=17)¶ Print calculated density profile of an isolated atom.
Incorporates resolution limits and filter resolution that have been specified, but not (automatically) any overall B or envelope function.
Parameters: - b (float) – thermal factor (add in any overall B or envelope function)
- atom_type (str) –
- max_dist (flost) – tabulate up to this distance from atom center
- points (int) – number of columns in table
-
three_d_transform_weights
(distance)¶ Pre-calculations for 3D Fourier transform of centro-symmetric function.
For example, weights to calculate atomic density from scattering factors do not depend on the atomic parameters, and may be pre-calculated.
The returned weights are the 3D Fourier transforms of spherical shells, calculated as the difference between the FTs of solid spheres. The function to be multiplied by the returned array must be sampled at the same
self.dst
points. These are calculated fromself.d_min
,self.d_max
andself.shells
on class instantiation. Take care as there is no checking.For further efficiency, the results could be tabulated for a range of distances for subsequent interpolation.
Todo
Add calculation of partial derivatives of the weights with respect to distance (see rres_local.c).
Parameters: distance (float or ndarray) – distance(s) (from the center) at which Fourier transform is to be evaluated. Returns: weights, derivatives-by-distance which when multiplied by some centro-symmetric function, sampled at self.dst
, and summed, yield the 3D Fourier transform and its derivative atdistance
from the center (from a 1D summation). weights is 1-D if distance is scalar, or a 2-D array with a row for each distance.Return type: 2 ndarray(type=float, shape=(distance.size, self.shells)) Attention
distance
> 0.0001 is imposed to avoid instability in the numerical calculation of the G-function weight 2πrd*cos{2πrd*}/(2πr)3.
-
class
density.
ModelMap
(comparison_map=None, calculator=None, near=3.6, cut_off=7.5, require_pts=2.0, symmetry=None)¶ Calculation of density near model atoms and comparison to observed density.
Calculations are on the grid of the (input) experimental map. Several methods are provided for calculating the model density in decreasing order of likely interest:
- calculate_box_mask: Atom-masked distance interpolation <~ 3ms/atom with bounds-checking.
- calculate_spherical_mask: Atom-masked distance interpolation ~3ms/atom with bounds-checking. (This might be the most erudite, but it is slower to determine which voxels are w/in the sphere than vectorized calculation for all voxels within the enclosing rhomboid.)
- calculate_interpolate: No masking (all map) calculation ~12ms/atom.
- calculate_all_pts: Direct independent calculation each pt, ~300ms/atom
Calculate_box_mask is the default, but the simpler, more accurate methods are maintained for regression testing.
Variables: scale_model_to_observed (bool) – default = True. False can be used to obtain statistics for models so far out of density that scale nears zero. However, scaling to the model (False) can lead to a trivial solution in refinement where Bs are refined higher, occupancies lower to reduce the scale & residual. False is a fragile option, because I have not figured the effects on partial calculation, which I have programmed to fail in this case.
Parameters: - comparison_map (
Map
) – experminental map. - calculator (
AtomicDensity
) – definition of how the density of atoms is calculated. - near (float, Å) – use grid points within this distance of any atom for residuals etc.
- cut_off (float, Å) – atomic density considered zero beyond this distance.
- require_pts (float) – check that all map grid points within this distance of any atom evaluated are available (if > 0.0)
- symmetry (
Symmetry
) – to apply to the atomic model.
Todo
Consider whether better to initialize instance of AtomicDensity within ModelMap.
Todo
Methods to pre-processes the comparison map: resample, expand symmetry etc..
-
class
AtomMask
(modelmap, cut_off=5.0, checkbounds=False)¶ Bases:
object
Supports operations on the map that are masked to the locale of atoms.
Variables: - mask – True for map pixels w/in cut_off of any atom being evaluated.
- stray_atoms – number of atoms with out-of-bounds grid points; could be used for warnings / errors if not self.checkbounds.
Parameters: - modelmap (
ModelMap
) – the current ModelMap instance, usually “self” - cut_off (float) – distance from atom (A) beyond which density is set zero or otherwise ignored.
- checkbounds (bool) – throw ArrayBoundError exception if grid points within cut_off are outside the map. Set to False if using a larger cut_off (to include contributions of neighboring atoms) than will be used for model/map comparisons. Instead, use ModelMap.inbounds to check.
Warning
checkbounds is not implemented for all density calculation methods.
Note
map density assumed to start at array index [0,0,0].
-
distances
(xyz, evaluate=True)¶ Distances of neighboring map grid points to atom. Molecular mask.
Parameters: - xyz (nd.matrix, shape=(3,1)) – atomic coordinate, orthogonal A
- evaluate (bool) – True if the that is being evaluated. Exceptions will only be thrown for out-of-bounds atoms that are being evaluated.
Returns: (map_slice, distances), box surrounding the atom & distances to grid points within the box from the atom.
Return type: tuple of 3 slice objects, float ndarray.shape(map_slice)
Todo
faster bounds-checking on limits rather than array assingment?
-
ModelMap.
atom_box
(distance)¶ Range of map that could be within distance of a neighboring atom.
Sphere is expanded such that if centered on the nearest grid point to an atom, the box encloses all grid points within distance of the atom.
Parameters: distance (float) – Å. Returns: Half-width of enclosing box in map grid steps. Return type: int ndarray, shape=3
-
ModelMap.
atomic_partials_current
¶ Whether atomic partial derivatives are current with latest model.
-
ModelMap.
calculate
(atoms, added_b=0.0, lazy=True, tolerance=None, method=None, scale_constants=None, local_scale=True, verbose=False)¶ Calculate density from model atoms by one of several methods.
Parameters: - atoms (
Atoms
) – list of coordinates & atomic properties - added_b (float, Å2) – overall isotropic B-factor and/or EM envelope exponential to be added to atomic B-factors
- lazy (bool) – Perform the calculation only if the structure or map has changed since the last calculation (change > tolerance, see below).
- tolerance (float) – for paramter p & p’, deemed changed if abs(p-p’) > tolerance*max(abs(p),abs(p’)) and max(abs(p),abs(p’)) > tolerance. Should not be less than machine precision.
- method (str) – one of _select_method.methods.keys() that controls whether density values are interpolated and grid points masked. Set by ModelMap.__init__, and should only be reset here for unit testing.
- scale_constants (tuple(float, float)) – if prior scaling coefficients are to be used, this tuple should contain the multiplier and constant to be added subsequently. scale_constants=None will force recalculation.
- local_scale (bool) – for scaling use only grid points close to atoms Local scaling must be used with masked density calculation, but may need to be turned off for good overall R-factor statistics.
Returns: density, nearest_atom, multipler, constant (that were used to scale calculated map)
Return type: float ndarray, float ndarray, float, float
- atoms (
-
ModelMap.
changed
(atoms=None, added_b=0.0, tolerance=None, verbose=False)¶ Test if changes in atoms / parameters require density recalculation.
Parameters: - atoms (
Atoms
or NoneType) – atomic coordinates, B-factors, occupancies If None, will not check for changes in atoms. - added_b (float) – combined overall B-factor / EM envelope correction.
- tolerance (float) – for paramter p & p’, deemed changed if abs(p-p’) > tolerance*max(abs(p),abs(p’)) and max(abs(p),abs(p’)) > tolerance. Should not be less than machine precision.
Returns: True if any parameter changed
Return type: bool
- atoms (
-
ModelMap.
correlation
(verbose=True)¶ Pearson correlation coefficient model vs experimental map, local & overall.
Todo
support masking of map.
Returns: correlation coefficients - local & overall Return type: float, float
-
ModelMap.
distances
(atom, map_index)¶
-
ModelMap.
distances_assume_orthonormal
(atom, map_index)¶
-
ModelMap.
dmagnification
(function='sq_diff', atoms=None, dxyz=None)¶ Partial of the objective function with respect to magnification.
This method calculates the partial derivative needed to optimize the magnification of the input map for best agreement with the atomic model.
Magnification is used in the microscopic sense. An increase means that the field of view (the map) contains less object, i.e. the Å per pixel are reduced and the unit cell dimensions are decreased. This is the inverse of the scale parameter in EMAN. Magnification is defined as a scaling from the Cartesian-space origin.
Conceptually, changed magnification can be thought of as either:
- Division of the map dimensions by M.
- Magnification of the coordinates by M.
In calculating the partial derivative, the 2nd approach can take advantage of atomic partial derivatives (if already calculated), while the 1st approach might be quicker if atomic partials are not available. Note that we do not actually magnify the coordinates as this would distort stereochemistry.
Approach 2: We can take advantage of the partial derivatives of the objective function, Γ with respect to the atomic coordinates xi, ∂Γ/∂xi
∂Γ/∂M = Σi ∂Γ/∂xi . ∂xi/∂M = Σi ∂Γ/∂xi . xi
Approach 1: This requires no prior calculation of coordinate partials. It should be quicker when independent of coordinate refinement, but the current implementation is significantly less accurate for reasons that are not clear.
Consider a least-squares residual:
Σ = Σgrid (ρo - ρc)2
Adjusting (only) the observed map:
∂Γ/∂M = 2(ρo - ρc) . ∂ρo/∂M
By increasing the magnification, we change the map index coordinates m at any point in Carthesian space, m, where D,t is the deorthogonalization operator such that m = Dx + t.
∂ρo/∂M = ∂ρo /∂m . ∂m/∂M
The first term is just the gradient of the observed map, calculated numerically by central differences. Now the 2nd term. For a fixed set of atomic coordinates, the map coordinates will change w/ magnification:
m = Dx + t; ∂m/∂M = Dx = (m - t)/M
Here, we will restrict ourselves to evaluating the derivative at M=1, because we need matching ρo, ρc grid points to avoid interpolation. i.e. this approach will work when the maps are updated between every cycle, so that (locally) M=1. In summary, in approach 2, we evaluate:
∂Γ/∂M = 2(ρo - ρc) . ∂ρo/∂m . (m - t)
Accuracy has been assessed by comparison to finite difference calculations. Accuracy depends on the magnification error, resolution, distance of atoms from density, etc.. Test cases so far, give a convergence radius of approach 1 at about 2.5% and approach 2 at about 4%. Within this, errors range from 10-30% for approach 1 and 1-15% for approach 2. It is not clear whether the larger errors of approach 1 result from a undetected bug, or intrinsic limitations of the algorithm. Neither are very impressive, but both are robust in the sign of the derivative changing as you go through the optimum. Thus, both should allow refinement, even if conergence is slow. (It is suspected that convergence is poor, because small changes in magnification result in large displacements which can place atoms close to mismatched peaks in the density.)
Parameters: - function (str) – objective function, one of the supported strings below.
- atoms (
Atoms
) – coordinates etc., required if to use partials. - dxyz (ndarray.shape(3,N)) – partial derivatives of the objective function re: atoms.xyz. If not None, approach 2 (see above) will be used.
Warning
map_calc.calculate() must have been run w/ latest atoms (approach 1) - no checks!
Warning
dxyz must be current with atoms (approach 2) - no checks!
Returns: ∂Γ/∂M Return type: float Warning
Possible bug in approach 1 with derivatives up to 30% in error. It might also be a limitation of the algorithm, see comments in “Accuracy” above.
-
ModelMap.
getVolume
(threshold=None, ignoreMask=False)¶
-
ModelMap.
image_refine
(atoms, magnification=1.0, added_b=0.0, resolution=0.0, optimize_magnification=False, optimize_overall_b=False, optimize_filter=False, finite_difference=False, constrained=True, bounds={'magnification': (0.9, 1.1), 'resolution': (0.6, 1.5)}, max_cycles=10, min_improvement=0.0, min_grad=0.0, verbosity=0)¶ Returns image (map) parameters fit to model (w/o applying them).
(Optionally constrained to improve stability.)
Differs from RSRef.refine in that all refining atoms will be used without reference to atom or group selections.
Warning
Convergence can be poor, due to:
- Covariance of parameters: envelope and resolution have similar effects upon the density. This becomes a greater problem at lower resolution and with smaller structures (fewer data points). The solution may be to refine only one of the paramters, in turn, starting with the worst.
- Inappropriate (internal) scaling of different parameter types, so that shifts are concentrated in a sub-set of components. As appropriate values depend on the system and stage of refinement, default values may need to be adjusted. See command-line options.
- Low pass resolution << (hard) resolution limit, so that the filter resolution has little effect.
- Analytical partial derivatives are approximated by sum of partials from individual atoms. If map_use is small relative to the actual width of an atom’s density, then the impact of changing its density parameters on neighboring atoms may be underestimated. This can be mitigated by using a larger map_use (just takes longer), or setting atom_extent = map_use, or using finite difference partials - but only if it looks like there is a problem.
- Poor initial guesses for parameters: A large error in magnification (off by 10%) can yeild erroneous partials for the B-factors (atoms are too far from correct locations).
Warning
Constrained refinement (default, see below) may be needed to avoid an excessive trial shift on the first cycle resulting in out-of-bounds atoms. The problem has been lessened, but not completely eliminated by the rescaling of internal refinement units (Sept 2012).
Parameters: - atoms (
Atoms
) – Atomic coordinates etc. - magnification (float) – starting magnification.
- added_b (float) – Overall B-factor and/or EM envelope exponential (Å2)
- resolution (float) – for low-pass filtering of atomic density.
- optimize_magnification (bool) – Refine magnification (else fix).
- optimize_overall_b (bool) – Refine overall b-factor or EM envelope function (else fix).
- optimize_filter (bool) – Refine low-pass filter parameters (else fix). (May not be possible to refine filter and B-factor/envelope simultaneously.)
- finite_difference (bool) – Calculate derivatives by finite differences instead of analytically. Takes ~2 x longer, but may be more stable, particularly at low resolution, because it is not subject to the same approximations as the analytical derivatives.
- constrained (bool) – Use the l_bfgs_b algorithm, constrained within limits defined below. If False, will use bfgs unconstrained optimization which, to date, has never converged as quickly. Note that magnification refinement is likely to fail without constraints as the first trial step often takes the atoms outside the map.
- bounds (dictionary of tuples, keywords: ‘magnification’, ‘added_b’) – fractional limits on refined parameter values. Thus (0.9, 1.1) would correspond to +/-10%. Defaults to None.
- max_cycles (int) – maximum number of max_cycles (hard limit for bfgs; approx. for constrained l-bfgs, as converted to a corresponding limit on function / gradient calculations).
- min_improvement (float) – refinement (constrained l_bfgs_b only) is stopped when relative change in function < min_improvement, i.e. 1.e-2 would stop when change < 1% of current residual value.
- min_grad (float) – refinement stopped when the largest derivative magnitude falls below this value (internal parameter units).
- verbosity (int) – -1 for silent, 0 for terse, 1 for verbose
Returns: refined incremental magnification factor, added overall B, resolution for low-pass filtering of atomic density.
Return type: float x 3
Todo
Consider combined atomic / image refinement, but suspect that would not converge well.
Todo
Consider refinement of map origin, but this would be equivalent to overall rigid model (translational) refinement.
See also
refine_magnification_no_atomic_partials
for an approximation that is quicker if atomic partials are not available.Publications describing work using l_bfgs_b, or commercial products using it, should cite one of the Nocedal references [Byrd-1995] [Zhu-1997].
-
ModelMap.
inbounds
(atoms, radius=2.0, terminate=True, out=<open file '<stderr>', mode 'w'>, verbose=True)¶ Check that spheres on each atom fall within bounds of the map.
Parameters: - atoms (
Atoms
) – list of coordinates & atomic properties - radius (float) – sphere placed at each atom
- terminate (bool) – stop (else only warn) for out-of bounds atom
- out (file) – file-stream for warning/error messages
- verbose (bool) – print diagnostics even if passes check
Returns: covered
Return type: bool
Todo
Additional testing needed. To date: comparison w/ orthogonal routine; with all atoms refining and moving atom Å at a time towards edge, i.e. informally.
- atoms (
-
ModelMap.
indices_near_index
(distance)¶ Logical mask of grid-points close to the central grid index.
Also provides masks appropriate for non-integral points rounded to the central grid index. Thus, this is appropriate for approximate screening of which grid points might be close to atoms.
The 3 masks returned are:
neighbor_guess: points within distance of the central grid point.
- neighbor_possibly: if the central grid point is merely a rounded
coordinate, then this mask covers all grid points that might be within distance of the original coordinate.
- neighbor_definitely: if the central grid point is merely a rounded
coordinate, then this mask covers all grid points that must be within distance of the original coordinate.
Parameters: distance (float) – Orthogonal Å coordinates. Returns: 3 spherical masks. Return type: ndarray(boolean, shape=(2n+1,2n+1,2n+1))
-
ModelMap.
magnify
(magnification=1.0, recalculate=True, atoms=None, added_b=0.0)¶ Change the (EM) magnification for the comparison map (observed).
Calls
map.ReadMap.magnify
thennew_comparison
& optionallycalculate
.Parameters: - magnification (float) – values > 1.0 increase the object’s (EM) magnification by decreasing the map (pixel) dimensions, equivalent to reducing unit cell size.
- recalculate (bool) – model density at new grid points. False only if sure that will be doing it independently later.
- atoms (
Atoms
) – Atomic coordinates etc. - added_b (float) – Overall B-factor and/or EM envelope exponential.
Returns: new value for change = 1.0
Return type: float
-
ModelMap.
new_comparison
(updated_map=None, zero_maps=True)¶ Re-initialize after updating the comparison_map (magnification etc.).
Reitinitialization is needed if a change is made to the comparison map, including magnification.
Parameters: - updated_map (
Map
) – experminental map. - zero_maps (bool) – reset the model / difference maps to zero. Setting to zero (True) & forcing recalculation is conservative approach, but False might save time if restoring previously calculated map (if nothing else has changed). False gives improved unittests of B-factor; worse for magnification. Needs work to figure out what is going on.
- updated_map (
-
ModelMap.
partial_stats
= True¶
-
ModelMap.
partials
(atoms, parameterization, added_b=0.0, lazy=True, tolerance=None, method=None)¶ Partials of (ρo - ρc):sup:2 with respect to the model parameters.
Parameters: - atoms (
Atoms
) – coordinate set - parameterization (AtomicParameterization or sub-class) – definition of what parameters are refining and how they depend on atomic parameters. This is used here to limit calculation of partials to those that are needed.
- added_b (float, Å2) – overall isotropic B-factor and/or EM envelope exponential to be added to atomic B-factors
- lazy (bool) – Recalculate density only if the structure or map has changed since the last calculation (change > tolerance, see below). Will recalculate partials no matter what.
- tolerance (float) – for paramter p & p’, deemed changed if abs(p-p’) > tolerance*max(abs(p),abs(p’)) and max(abs(p),abs(p’)) > tolerance. Should not be less than machine precision.
- method (str) – one of _select_method.methods.keys() that controls whether density values are interpolated and grid points masked. Set by ModelMap.__init__, and should only be reset here for unit testing. (All methods were timed by fair comparison, but only ‘box-masked interpolation’ (self._calculate_box_mask, self._partials_box_mask), has subsequently been optimized and extended to cover all needed partials.)
Returns: parital derivatives of (ρo - ρc):sup:2 with respect to the x,y,z coordinates of each atom (in one array), and with respect to atomic B-factor, occupancy and low-pass filter resolution.
Return type: ndarray.shape(3,natoms), 3 x ndarray.shape(natoms)
Todo
add support for other types of objective functions, e.g. CC.
Warning
requires: self.difference.density from prior self.calculate
Todo
conditional call of calculate if required.
- atoms (
-
ModelMap.
refine_magnification
(atoms, magnification=1.0, added_b=0.0, constrained=True, precision=0.01, verbosity=0)¶ Lst sq refinement of observed map magnification using atomic partials.
(Optionally constrained to improve stability.) If refinement is successful, updates the comparison (observed) map.
Atoms are moved temporarily to mimic a change in map magnification. When the map is actually changed at the end, expect a modest change in the residual as the numbers of grid points close to atoms (and local scaling) will change a little.
Todo
Had hoped to deprecate refine_magnification when image_refine tested. However, suspect that poor combined refinement (magnification w/ other parameters) may be due to the sampling of new grid points when the map magnification is changed on each shift / iteration. Thus, need to try an image_refine_2 in which the magnification refinement of refine_magnification is combined with the other parameters of image_refine.
Parameters: - atoms (
Atoms
) – Atomic coordinates etc. - magnification (float) – starting magnification.
- added_b (float) – Overall B-factor and/or EM envelope exponential (Å2).
- constrained (bool) – Use the l_bfgs_b algorithm, constrained within +/- 10% changes to the magnification. Else use bfgs optimization.
- precision (float) – constrained refinement is stopped when change in function (relative to starting value) < precision, i.e. precision of 1.e-2 stops when change < 1% of initial value.
- verbosity (int) – -1 for silent, 0 for terse, 1 for verbose
Returns: refined incremental magnification factor
Return type: float
Todo
Add partial to atomic partials for joint refinement.
See also
refine_magnification_no_atomic_partials
for an approximation that is quicker if atomic partials are not available.Publications describing work using l_bfgs_b, or commercial products using it, should cite one of the Nocedal references [Byrd-1995] [Zhu-1997].
- atoms (
-
ModelMap.
refine_magnification_no_atomic_partials
(atoms, magnification=1.0, added_b=0.0, constrained=True, precision=0.01, verbosity=0)¶ Standalone lst sq refinement of magnification using map gradient.
(Optionally constrained to improve stability.)
Deprecated since version Takes: 50% longer and has 1% cf. 0.001% accuracy, relative to
refine_magnification
- which has been deprecated in favor ofimage_refine
. Use only when atomic partials not available.Parameters: - atoms (
Atoms
) – Atomic coordinates etc. - magnification (float) – starting magnification.
- added_b (float) – Overall B-factor and/or EM envelope exponential (Å2).
- constrained (bool) – Use the l_bfgs_b algorithm, constrained within +/- 5% changes to the magnification. Else use bfgs optimization.
- precision (float) – stopped when change in function (relative to starting value) < precision, i.e. precision of 1.e-2 stops when change < 1% of initial value.
- verbosity (int) – -1 for silent, 0 for terse, 1 for verbose
Returns: refined incremental magnification factor
Return type: float
Todo
Add partial to atomic partials for joint refinement.
See also
refine_magnification_no_atomic_partials
for an approximation that is quicker if atomic partials are not available.Todo
Lazy hashing to avoid recalculation.
Todo
Unit test with more than one atom.
Todo
lbgfs in refinement?
Publications describing work using l_bfgs_b, or commercial products using it, should cite one of the Nocedal references [Byrd-1995] [Zhu-1997].
- atoms (
-
ModelMap.
rfactor
(conventional=True, verbose=True)¶ Real-space R between model & experimental maps w/o recalculation or scaling.
Calculated locally and over entire map (see scale for scaling). Requires prior calculation of density from current model & scaling. The R-factor is analogous to conventional reciprocal space R, w/o Alwyn Jones’ factor of two in the denominator.
Parameters: - conventional (bool) – Denominator is Σρobs, else 0.5Σ(ρobs + ρcalc)
- verbose (bool) – print statistics.
Returns: R = |ρobs + ρcalc|/Denominator
Return type: float
-
ModelMap.
rms
(verbose=True)¶ RMS difference between model & experimental density w/o recalculation or scaling.
Requires prior calculation of density from current model & scaling.
Returns: rms difference, local & overall Return type: float
-
ModelMap.
scale_model_to_observed
= True¶
-
ModelMap.
sorted_correlation
(indexList, useVolume=True, useModel=True, verbose=True)¶ Pearson correlation coefficients between model & experimental maps.
Calculated using the first n grid points in order of either distance from nearest atom (useVolume=True) or model map value (useVolume=False and useModel=True) or experimental map value (useVolume=False and useModel=False)
Returns: list of correlation coefficients Return type: [(int,float,float)]
-
ModelMap.
sq_diff
(verbose=True)¶ Squared difference between model & experimental density w/o recalculation or scaling.
Requires: prior calculation of density from current model & scaling.
Returns: squared difference, local & overall Return type: 2 floats
-
ModelMap.
threshold_correlation
(useModel=True, threshold=2.0, verbose=True, ignoreMask=False)¶ Pearson correlation coefficient between model & experimental maps.
Calculated over all grid points for which model map exceeds thresholdValue standard deviations above mean
Returns: correlation coefficient Return type: float
-
ModelMap.
togrid
(atom, map_index)¶ Distance(s) & partial derivatives of distance from atom to map grid point(s).
Partial derivatives are with respect to the x,y,z atomic coordinates.
Parameters: - atom (numpy matrix (float, shape=(3,1))) – atom coordinates (orthogonal Å)
- map_index (numpy matrix (integer, shape=(3,N))) – N > 0 grid point(s)
Returns: distance(s), xyz-partial derivative(s) (Å)
Return type: float numpy.ndarray shape(N) & shape(3,N)
-
ModelMap.
tolerance
= 4.4408920985006262e-16¶
-
ModelMap.
warnings_issued
= []¶ Kluge to circumvent repeated warnings.
-
ModelMap.
write_calculated
(filename, header='')¶ Write density calculated from atomic model to file.
Parameters: - filename (str) – extension gives file format
- header (str) –
-
ModelMap.
write_comparison
(filename, header='')¶ Write observed density that has been scaled to the atomic model.
Parameters: - filename (str) – extension gives file format
- header (str) –
-
ModelMap.
write_difference
(filename, header='')¶ Write difference density (observed - model) to file.
Parameters: - filename (str) – extension gives file format
- header (str) –
-
density.
correl_coeff
(array1, array2)¶ Pearson correlation coefficient using NumPy not SciPy
Parameters: - array1 (nd.array) –
- array2 (nd.array) –
Returns: Pearson correlation coefficient
Return type: float
-
density.
scale
(to_scale, fixed, mask=True, verbose=False)¶ Least-squares linear scaling of optionally masked identically shaped arrays.
Parameters: - to_scale (ndarray) –
- fixed (ndarray, shape=to_scale.shape) –
- mask (bool ndarray or logical) – optionally uses only points where mask = True
Returns: scaled contents of to_scale, multiplier that was used, constant that was subsequently added.
Return type: ndarray, shape=to_scale.shape, float, float