optimize module¶
Interface to scipy.optimize for refinement of atomic + additional parameters.
Defines default options, objective functions, and manages parameters in two layers: AtomicParameterization and its subclasses support designation of which and how atomic parameters are to be refined (overall, group, torsion, individual); Parameters scales the different units and packs atomic or derived parameters into the flat arrays expected by scipy.optimize.
Module author: Michael S. Chapman <chapmami@ohsu.edu>
Authors: | Michael S. Chapman <chapmami@ohsu.edu>, Oregon Health & Science University |
---|---|
Version: | 0.5, March 23, 2016 |
Changed in version 03/30/2011.
Changed in version 0.4.2: 10/24/2013
Changed in version 0.5.0: (5/1/15) ReStructured Text documentation
Changed in version 0.5.3: 9/27/15 Compatibility with torsion.py tolerance of missing OXT.
Changed in version 0.5.5: 10/13/15 Anisotropic Us can ride with atom rotations.
Changed in version 0.5.6: 10/22/15 van der Waals restraint.
Todo
Support use of different scipy optimizers. fmin_bgfs converges better than fmin_l_bgfs, but (on laptop) runs out of memory refining ~1500 xyz’s being refined. Other optimizers could be tried.
Todo
strip setting/oldsetting from do_individual, do_group, do_torsion
Todo
filtering on torsion change should be an option, not hard-wired.
-
class
optimize.
AnalyzeSubCommands
(parent, prompt='sub-command')¶ Bases:
cmd2nest.SubCmd
,cmd2nest.ProgCmd
Sub-commands for refinement analysis.
Parameters: - parent (
TopCmd
instance) – Cmd object whence this call was made, usually self. - prompt (str) – concatonated with the parent’s prompt.
-
do_convergence
(instance, arg)¶ Analyze the gradients and shifts of prior refinement. Usage: convergence [options] arg
- Options:
-h, --help show this help message and exit -s, --summary Summary only without itemizing groups or dihedrals.
-
do_dihedrals
(arg=None, opts=None)¶ Changes in torsion angles and their impact.
- parent (
-
class
optimize.
Arguments
(imports=[], main=False, *args, **kwargs)¶ Bases:
argparser.ArgumentParser
Argument parser for optimization target function.
The refinement of atomic and image parameters each brings together units on very different scales. The optimizer knows nothing of units. This can lead to two problems:
- Some types of parameters can be frozen if the optimizer calculates derivatives that are are an insigificant component of the gradient vector.
- On the first cycle, the step search down the gradient vector starts with a completely arbitrary step which could be to an unreasonable value from which refinement cannot recover.
Generally refinement works best when internal units are chosen such that the partial derivatives of different types of parameters are of commensurate magnitude (or within ~10-fold), ideally throughout refinement. The values set here work for selected test cases.
Appropriate values might depend on the stage of refinement and its parameterization. For example, larger rigid groups generally have numerically larger partial derivatives with respect to rotation. To see the magnitudes of different types of partial derivative, use the analyze option of refine, and verbosity > 2 for image_refine. See comments in
density.ModelMap.image_refine
for more detailed discussion of these parameters. Each of the parameters is a multiplicative scale constant applied to the input parameter values to convert them to internal units.Parameters: - imports ((list of) ArgumentParser method(s)) – method(s) adding arguments to ArgumentParser instance. (Alternative to ArgumentParser parents w/ better maintained groups.)
- main (bool) – Parser for main program (i.e. not listed within an ArgumentParser parents argument). Adds default program information.
-
static
export
()¶ Adding options used in both stand-alone and imported modes.
-
class
optimize.
AtomicParameterization
(atoms=None)¶ Bases:
dict
(Derived) parameters and their dependence on atomic parameters.
Implemented as nested dict-like objects w/ restricted keys and value types. Access either in dict or attribute format, eg. instance[‘position’][‘individual’] or instance.position.individual. This top level (AtomicParameteization) defines parameter types that are varied: position, bfactor, occupancy... Each of these is a dict-like Groupings object that defines how these (atomic) parameters are to be combined and/or depend on other parameters that will be optimized: overall, group, torsion or individual (atom). Each of these is a Grouping object that defines (a sequence of) atomic selection(s) (derived from input logical expressions), or a boolean defining a default, or None. Only some combinations are valid.
Changed in version 09/25/13: - default is nothing refined.
Warning
copy.deepcopy() fails to copy attributes beyond the standard dict and AtomicParameterization().update(old) only does a shallow copy of the underlying Groupings items. Thus, if needed, a specific deepcopy will have to be implemented.
Parameters: atoms (Atoms) – coordinates to be parameterized. -
bfactor
¶ Var: Convenience synonym for dictionary item.
-
changing
¶ Atoms for which any parameter is changing.
-
nothing
()¶ Unsets everything.
-
occupancy
¶ Var: Convenience synonym for dictionary item.
-
position
¶ Var: Convenience synonym for dictionary item.
-
sorted_keys
()¶ Orders xyz, B, occ, then other parameters alphabetically.
-
width
= 132¶ Var: line width for representation.
-
-
class
optimize.
AtomicRefinement
(atoms, refine, verbosity=0)¶ Bases:
optimize.Optimization
Atomic model optimization parameterized as rigid groups, torsions, atoms.
See also
Until joint refinement implemented here, see alternative image optimization in
density.ModelMap.image_refine
etc.Initialize a refinement session, defining the parameterization.
Joint refinement of occupancy and B-factors for individual atoms requires very high resolution data. Convergence can be improved by setting stringent convergence criteria, eg. min_improvement = 1e.-9.
Parameters: - atoms (
Atoms
) – starting atomic model to be refined. - refine (
AtomicParameterization
(subclass)) – Defines what is to be refined & how. - verbosity (int) – -1 for silent, 0 for terse, 1 for verbose, > 1 for (debugging) information on each function or gradient evaluation.
-
static
selectionPair
(selections, atoms)¶ Expand defaults for a 2-tuple of selections.
Polymorphic torsion input allowing True/False, single selection or tuple with selection as A.Group, A.Selection, string argument to A.Selection or None
Parameters: - selections – see above
- atoms (
Atoms
) – coordinate set from which to make selections
- atoms (
-
class
optimize.
BGrouping
(**kwarg)¶ Bases:
optimize.Groupings
Ways in which thermal factors are to be refined.
Parameters: atoms (Atoms) – Coordinates to be parameterized.
-
class
optimize.
CallBackEmulator
(method, objective, verbose=True)¶ Bases:
object
Calling callback from f() when optimizers don’t support directly.
Guesses when a new cycle is beginning, calling callback(). In SciPy versions 0.11 & lower, l_bfgs does not support callback.
Deprecated since version SciPy: 0.11 supports callback so completely obsolete.
Warning
Difficult to replicate exactly the sequence of calls within scipy.fmin. Calling at the end of fprime for l_bfgs_b works as long as there are no additional step evaluations, which would occur if under conditions of poor convergence. No easy resolution, because would need to know which function evaluation is going to be the last. The simple approach is to ignore the effects on the parameters and residual of the subsequent step optimization, because repeated callback for updated step-sizes will spoil integrated statistics.
Parameters: - method (str) – name of the optimizer in use.
- objective (
Objective
) – location of the callback method.
-
argument
()¶ Optimizer keyword argument consistent w/ native or emulated callback.
Returns: key-word argument for scipy.optimize routine. Return type: dict Parameters: callback – callable if supported
-
queryCallBack
(param, calledFrom='f', stage='start', anyCycle=False)¶ Call callback() on appropriate cycles if not called directly.
Parameters: - param (ndarray) – parameter array from optimizer, passed to callback.
- anyCycle (bool) – don’t restrict to the start of an iteration.
-
whenCall
= {'l_bfgs_b': [{'gradientsCompleted': 2, 'evaluationsCompleted': 2, 'function': 'fprime', 'stage': 'end'}, {'gradientsCompleted': 1, 'evaluationsCompleted': 1, 'function': 'fprime', 'stage': 'end'}]}¶
-
class
optimize.
Commands
¶ Bases:
object
Optimization cmd2 commands for importing/subclassing to other modules.
-
do_analyze
(arg=None, opts=None)¶ Analyze prior refinement: shifts, gradients & impact of parameters.
-
-
class
optimize.
DihedralsSubSubCommands
(parent, prompt='sub-command')¶ Bases:
cmd2nest.SubCmd
,cmd2nest.ProgCmd
Sub-commands for analysis of refined torsion angles.
Parameters: - parent (
TopCmd
instance) – Cmd object whence this call was made, usually self. - prompt (str) – concatonated with the parent’s prompt.
-
do_hinges
(instance, arg)¶ Find hinges in dihedral changes. Usage: hinges [options] arg
- Options:
-h, --help show this help message and exit -g INT, --gap=INT # residues w/o dihedral changes that can be bridged in a hinge -t FLOAT, --threshold=FLOAT above which dihedral rotations considered hinges. Degrees if > 1.0, else fraction of total change. -p, --pseudo combine phi_i with psi_i-1, else individual dihedrals. -s START, --start=START start of an explicitly defined hinge: chain-residue, requires –end, repeatable. -e END, --end=END end of an explicitly defined hinge: chain-residue, requires –end, repeatable.
-
do_impact
(instance, arg)¶ Estimate impact of refined torsion angle changes on superimposition. Usage: impact [options] arg
- Options:
-h, --help show this help message and exit -i, --iterative Iteratively determines/applies highest impact dihedral changes. This rarely-needed option is compute- intensive, because it iteratively finds the single, highest impact dihedral, applies the rotation, and repeats. Otherwise, by default, impact is assessed more rapidly from the dot product of the gradient and shift vectors, integrated over refinement iterations. -I IDENTIFY, --identify=IDENTIFY List this number of the top-impact dihedrals. -p, --pseudo Use approximation to pseudo-torsion angles (phi_i + psi_{i-1}) instead of individual phi, psi. (Limits output options, but speeds option iterative.) -R, --recover Use previous calculation of impact, other options invalid.
-
do_paint
(instance, arg)¶ Output coordinates, B-factors set to dihedral change for visualization. Usage: paint [options] output PDB file (w/ corrupted B-factors!)
- Options:
-h, --help show this help message and exit -p, --pseudo Use pseudo-torsion angle (phi_i + psi_i-1),else individual phi, psi. -n, --normalized Scaled between 0 & 100.
- parent (
-
class
optimize.
Documentation
¶ User-level documentation for imports.
-
class
optimize.
ExtendedParameterization
(*args, **kwargs)¶ Bases:
optimize.AtomicParameterization
Parameterization going beyond atomic paramters.
-
class
optimize.
Fit
(atoms, model_vs_map, refine, link=None, unlink=None, reference=None, symmetry=None, verbosity=0)¶ Bases:
optimize.AtomicRefinement
Fitting atomic structure to field as rigid groups, torsions, atoms.
Joint refinement of occupancy and B-factors for individual atoms requires very high resolution data. Convergence can be improved by setting stringent convergence criteria, eg. min_improvement = 1e.-9.
Initialize a refinement session, defining the parameterization.
:param
Atoms
atoms: starting atomic model to be refined. :paramModelMap
model_vs_map: data/methods for model/map comparison :param refine: Defines what is to be refined & how. :type refine:AtomicParameterization
(subclass) :param reference: if torsion angle parameter changes are to be restrained(option.torsion_weight), an optional reference state of the moving structure (same atoms, order etc.) can be provided (eg. input structure); else any restraint will apply from the start of each refinement batch.Parameters: - symmetry – molecular and lattice (crystallographic) symmetry as appropriate, or None.
- verbosity (int) – -1 for silent, 0 for terse, 1 for verbose, > 1 for (debugging) information on each function or gradient evaluation.
Todo
link, unlink: (a) need user-input mechanism; (b) may not be fully supported yet, 4/19/13.
-
class
optimize.
GroupGp
(*arg, **kwarg)¶ Bases:
optimize.Grouping
Group(s) of atoms with collective parameters.
-
expand
()¶ Iterator over (multiple) changing selection(s).
Returns: atom selection. Return type: Atoms.Selection
See also
selections for named iterator
-
parse
(text, atoms=None)¶ From a single text criterion or sequence thereof (list, dict).
-
selections
¶ Iterator over (multiple) changing selection(s).
Returns (name (str), atom selection (Atoms.Selection)).
-
-
class
optimize.
Grouping
(*arg, **kwarg)¶ Bases:
object
How atoms depend on one another.
Polymorphic storage, either as pre-defined atom selection(s), or True, False or None from which selection generated in a subclass-specific way. Should not access directly: use one of the sub-classes.
Parameters: - arg (polymorphic) – (optional) value with which to initialize
- atoms (
Atoms
) – Default coordinates, used to determine selections if defaults or text criteria used later w/o atoms-specification. - notify (callable) – Callable with single argument to which any change in the instance is posted. Can be used to trigger recalculation of dependent properties.
-
as_count
= False¶ Var: abbreviate with a count, else as ellipsis (bool).
-
assign
(value)¶ Polymorphic setter
1st assumes subclass legal value, evaluating as string selection directive(s) if not.
-
static
boolabbrev
(boolar)¶ Boolean array abbreviated w/ T/F (ellipsis if long) or count.
-
changing
¶ Atoms that are moving (Atoms.Selection boolean ndarray).
-
check_type
()¶
-
default
()¶ A sensible object value, not automatically set by __init__.
-
default_selection
()¶ Default atom selection if instance set to True.
-
max_len
= 33¶ Var: beyond which Selection printed abbreviated. (int >= 5, odd.)
-
nothing
()¶ Nothing to be optimized.
-
parse
(text)¶ Evaluated from text input.
This default allows assignment to a single group. Some sub-classes will over-ride.
-
selections
¶ Iterator over (multiple) changing selection(s).
For polymorphic compatibility with GroupGp subclass, returns: (NoneType, Atoms.Selection)
-
value
¶
-
class
optimize.
Groupings
(atoms=None)¶ Bases:
dict
The (multiple) Grouping(s) by which atomic parameter type is refined.
Access through subclasses: XYZGroupings, BGroupings, OGroupings.
Implemented as a dictionary with restricted keys and value types, and with equivalent attribute-like properties for convenience.
Parameters: atoms (Atoms) – Coordinates to be parameterized. -
changed
(*value)¶ Trigger recalculation of properties.
-
changing
¶ Bool array of atomic parameters changing under any parameterization.
These would be the atomic parameters for which partial derivatives would be needed.
-
default
()¶ Sets and returns the default.
-
group
¶ Var: Convenience synonym for dictionary item.
-
individual
¶ Var: Convenience synonym for dictionary item.
-
nothing
()¶ Unsets everything.
-
overall
¶ Var: Convenience synonym for dictionary item.
-
own
(*keys)¶ Take ownership of item(s) possibly copied from other Groupings.
-
torsion
¶ Var: Convenience synonym for dictionary item.
-
update
(other)¶ dict.update, ignoring other’s keys that are invalid in self.
Also takes ownership of component Grouping.notify.
-
-
class
optimize.
History
¶ Bases:
object
Starting, penultimate & final parameters, residual, gradients.
During fmin optimizations, parameters are updated at the start of a cycle & residuals calculated. Gradients calculated in one cycle are used to calculate shifts for the next cycle. Thus, during refinement shifts should be compared with the gradients of the previous cycle. After refinement, if wrapUp() is called, the final parameters and objective function value will be pushed onto the stack if changed, but there is no new calculation of the gradient. Thus, at the end, the last cycle shifts and gradients correspond to one another, as do those of the penultimate cycle.
-
deriv
= None¶ Gradient vector at start, penultimate and after last cycle.
-
impact
¶
-
info
= False¶ Extra information tracked for program development.
-
normalizeImpact
= True¶ Var: normalize grad-estimated impact to match residual (rms) change, so that when multiplied by parameter shifts, approximates actual improvement (bool). In practice, difference is subtle & not clear which is better. (To change interactively, try: py import optimize; optimize.History.normalizeImpact=False
).
-
param
= None¶ Parameters at start, penultimate and after last cycle.
-
residual
= None¶ Residual at start, penultimate and after last cycle.
-
saveGradients
(deriv, push=True)¶ Save gradients (deriv).
Parameters: push (bool) – Push current value to penultimate, just overwrite.
-
saveParameters
(param, push=True)¶ Save parameter array.
Parameters: push (bool) – Push current value to penultimate, just overwrite.
-
saveResidual
(residual, push=True)¶ Save parameter array.
Parameters: push (bool) – Push current value to penultimate, just overwrite.
-
trackImpact
(gradientUpdated=True)¶ Track estimated impact on refinement of each parameter.
Tracks the 4 combinations of metric = residual or root(residual) and assessment = change or grad(), so that the choice can be made later when analyzing impact.
The metric of improvement can be the change in (least squares) residual or (perhaps preferably) its square root (corresponding to an rms difference). RMS reduces the influence of early cycles with large |grad|, and appears just slightly better with arginine kinase (subjective assessment).
grad(metric) is a first order approximation to the derivatives used in lbgfs & other optimizers. (It ignores curvature and interdependence of parameters, but it seems impractical to try to use a higher order approximation.) The simple change in metric (per cycle) is a zeroth order approximation, assuming contributions of parameters in simple proportion to their change.
All combinations of above are accumulated in History() during refinement so that here the appropriate choice can be analyzed per residue.
grad(sqrt(residual)) (proportional to grad(rms)) is given by: Δp∂R0.5/∂p = Δp(1/2)(1/R0.5)(∂R/∂p)
Parameters: gradientUpdated (bool) – if the gradient has been pushed to the history stack since the last update of parameters, then recent parameter changes need to be paired with penultimate gradients. Returns: self Return type: History
-
update
(param, residual, gradient)¶ Push latest values onto stack, appropriate at end of cycle.
-
wrapUp
(param, residual)¶ Push just parameters & residual onto stack (only if changed).
Appropriate after end of refinement. :return: changed :rtype: bool
-
-
class
optimize.
ImpactSubSubSubCommands
(parent, prompt='sub-command')¶ Bases:
cmd2nest.SubCmd
,cmd2nest.ProgCmd
Sub-commands for analysis of impact of refined torsion angles.
Parameters: - parent (
TopCmd
instance) – Cmd object whence this call was made, usually self. - prompt (str) – concatonated with the parent’s prompt.
-
do_color
(instance, arg)¶ Output coordinates, B-factors set to percent impact on fit of dihedrals. Usage: color [options] output PDB file
- Options:
-h, --help show this help message and exit -p, --pseudo Use pseudo-torsion angle (phi_i + psi_i-1), else individual phi, psi; ignored if impact –pseudo.
-
do_pickle
(instance, arg)¶ Save impact-needed data, so impact.py can replot without recaclulation (when tweaking plot). WARNING: objects picked by superpose must be compatible with impact - has not been checked recently. Usage: pickle [options] FILE.jar - pickled impact object for later analysis.
- Options:
-h, --help show this help message and exit
-
do_plot
(instance, arg)¶ Graph impact of dihedrals on fit. Usage: plot [options] arg
- Options:
-h, --help show this help message and exit - -p impact|change, –prior=impact|change
- Replace impact or dihedral change with values from previous refinement (to superimpose restrained impact upon unrestrained dihedral changes).
-c, --changeOnly plot only dihedral changes, not estimate of impact. -t, --tty Display graph on terminal (not recommended for background jobs). -f FILE, --file=FILE save plot in graphics file of type given by extension (.emf, eps, jpeg, jpg, pdf, png, ps, raw, rgba, svg, svgz, tif, tiff) -A FILE, --annotation=FILE File containing additional plot commands, overriding command-line input. -O, --overall Dihedral changes (not impact) relative to reference (input) structure not refinement batch.
-
do_print
(instance, arg)¶ Tabulate changes in dihedrals & impact on fit. Usage: print [options] arg
- Options:
-h, --help show this help message and exit -O, --overall Dihedral changes (not impact) relative to reference (input) structure not refinement batch.
- parent (
-
class
optimize.
IndividualGp
(*arg, **kwarg)¶ Bases:
optimize.Grouping
Selection of atoms to be refined individually.
-
default
()¶ A sensible object value, not automatically set by __init__.
-
-
class
optimize.
OGrouping
(**kwarg)¶ Bases:
optimize.Groupings
Ways in which occupancies are to be refined.
Parameters: atoms ( Atoms
) – Coordinates to be parameterized.
-
class
optimize.
Objective
(model_vs_map, parameters, local_scale=True, lazy=True, reference=None, verbosity=0, target=None, symmetry=None)¶ Bases:
object
Callable methods for function, partials to be optimized.
Variables: continuation (bool) – True (only) if additional cycles w/o changes to model or map calculation parameters. (Continues w/ prior numbering and skips first function evaluation.
Parameters: - model_vs_map (
ModelMap
) – data/methods for model/map comparison - parameters (
Parameters
) – refinement parameters - local_scale (bool) – scale maps using only grid points close to atoms
- lazy (bool) – only recalculate density when parameters have changed
- reference (
Atoms
or NoneType) – if torsion angle changes are to be restrained (option.torsion_weight), an optional reference state of the moving structure (same atoms, order etc.) can be provided (eg. input structure); else any restraint will apply from the start of each refinement batch. - verbosity (int) – increase for greater diagnostic output
:param
Atoms
target: known coordinates to which restrained :paramSymmetry
symmetry: molecular and lattice symmetry operators.Prone to large initial shift on 1st cycle when model is ~perfect with bfgs. Should be used with l_bgfs_b.
Todo
consider merits of passing VDWrestraint instance (containing Symmetry instance) instead of Symmetry. This would be more consistent with model_vs_map, but would be inefficient if other features need symmetry in the fiture.
-
add_torsion_restraint
()¶ Begin applying restraint on changes if torsion angle refinement.
-
callback
(param, parameters_only=False)¶ Function to be called by optimizer at the end of each iteration.
Optionally calculates the correlation between the latest shifts and the gradients from the last evaluation of the previous cycle. (Empirically, get higher correlations with the final gradient estimate rather then the first.)
Parameters: - param – refining parameters, x_k; Note not yet updated to x_k+1 as one might expect at the end of a cycle.
- parameters_only (ndarray) – Only the parameters have been updated from xk to xk+1, as occurs between the last callback and the end of refinement. Use True after end of refinement.
-
depends_on_anisou
= False¶ If False, Objective independent of anisotropic U which are “riding” and can be ignored until final update.
-
f
(param)¶ Objective function callable for optimizer, with diagnostic output.
Parameters: param (1D ndarray) – refining parameters
-
fprime
(param)¶ Partials callable for optimizer, with diagnostic output.
Partial derivatives are mostly calculated by methods called and documented therein. Exceptions are some of the group and overall partial derivatives calculated from atomic partials.
For (multiplicative) group occupancies: Let:
- Γ be objective function;
- O be group (overall) occupancy;
- oi be the atomic occupancy of atom i;
- ρo be observed density;
- ρic be (scaled) calculated density for atom i;
Γ = Σi(ρo - Ooiρic)2
∂Γ/∂O = Σi(ρo - ρic)oiρic)
∂Γ/∂oi = 2(ρo - ρic)Oρic)
Substituting: 2(ρo - ρic)ρic = ∂Γ/oi/O
∂Γ/∂O = Σi(∂Γ/∂oi)oi/O
For group B-factors, as they are additive (B_overall + B_atom), group partial is just the sum of the individual atomic partials.
Parameters: param (ndarray) – refining parameters
-
function_detail
= 3¶
-
progress
= 1¶
-
step_detail
= 2¶
- model_vs_map (
-
class
optimize.
Optimization
(verbosity=0)¶ Bases:
object
Refinements defined by their parameterization.
-
analyze
(**kwargs)¶
-
refine
(max_cycles=10, min_improvement=0.0, min_grad=0.0, method='l_bfgs_b', verbosity=None, reference=None)¶ Refinement of previously instantiated parameters.
Parameters: - max_cycles (int) – maximum number of max_cycles (hard limit for bfgs; approx. for l-bfgs, as have to guess a corresponding limit on function calculations).
- min_improvement (float) – refinement (l_bfgs_b) is stopped when relative change in function < min_improvement, i.e. precision of 1.e-2 stops when change < 1% of current residual value.
- min_grad (float) – refinement stopped when the largest derivative magnitude falls below this value (internal parameter units as reported by analyze).
- method (str) –
bfgs
to approximate full Hessian (2nd derivatives; small number of parameters); orl_bfgs_b
for limited memory approximation (most atomic models). l_bfgs_b also supports constrained refinement. - verbosity (int or NoneType) – -1 for silent, 0 for terse, 1 for verbose, > 1 for (debugging) information on each function or gradient evaluation. If None, use value given on instantiation.
- reference (
Atoms
or NoneType) – optional self.atoms with different xyz positions, from where torsion angle changes (not impact) will be measured. Must be same atoms / order as self.atoms. This can be different from and does not reset the reference for torsion parameter restraints that is set when the objective function is instantiated. (Sorry for confusing use of same variable name.)
Todo
add default ‘auto’ method that would set according to number of parameters.
Publications describing work using l_bfgs_b, or commercial products using it, should cite one of the Nocedal references [Byrd-1995] [Zhu-1997].
Todo
Once SciPy ver >= 0.12 becomes prevalent, remove callback_supported from Objective, and callBackEmulator from optimize.
-
-
class
optimize.
OverallGp
(*arg, **kwarg)¶ Bases:
optimize.Grouping
Dummy - Just True (all atoms) or False (None)!
-
default
()¶
-
nothing
()¶
-
parse
(text)¶ Evaluated from text input. Only True|False.
-
-
class
optimize.
ParameterizeSubCommands
(parent, prompt='sub-command')¶ Bases:
cmd2nest.SubCmd
,cmd2nest.ProgCmd
Sub-commands for parameterization.
Parameters: - parent (
TopCmd
instance) – Cmd object whence this call was made, usually self. - prompt (str) – concatonated with the parent’s prompt.
-
do_SELECTION_EXPR
()¶
-
do_clear
(rest)¶ Switch off all refinement of requested parameter type. (Individual parameterizations can be switched off with”group None”, “individual None”, “torsion None”, “overall False”)
-
do_group
(instance, arg)¶ Select atoms to be refined as one or more groups. Usage: group [options] GROUP | COLLECTION | SELECTION | SELECTION_EXPR | GROUP_EXPR:
- SELECTION: a previously saved single Selection, given as: COLLECTION[‘ITEM’].
- (COLLECTION or [‘ITEM’] can be omitted if defaults were used in the corresponding select command.)
SELECTION_EXPR: boolean Selection expression, see help select.
GROUP_EXPR: list, dictionary of tuple of multiple quoted SELECTION_EXPR.
GROUP | COLLECTION: name of previously saved (cmd select) Selections.
- Options:
-h, --help show this help message and exit
-
do_individual
(instance, arg)¶ Select atoms to be refined individually. (More appropos with exptl. data than superposition.)
Usage: individual [options] SELECTION_EXPR | SELECTION:
SELECTION_EXPR: a boolean expression, see help select; SELECTION: name of a previously saved Selection, given as: COLLECTION[ITEM]. (COLLECTION or [ITEM] can be omitted if defaults were used in the corresponding select command.)- Options:
-h, --help show this help message and exit
-
do_overall
(instance, arg)¶ Refine requested parameter type as a single group.
If just refining overall position, overlay is better (wider convergence/faster). Use overall when also simultaneously refining sub-groups, torsion angles etc.. Usage: overall [options] [True | False]
- Options:
-h, --help show this help message and exit
-
do_print
(instance, arg)¶ Print parameterization. (Options invoked until reset.) Usage: print [options] arg
- Options:
-h, --help show this help message and exit -w INT, --width=INT Line width (def. 132). -a SIZE, --abbreviate=SIZE Selections longer than SIZE abbreviated w/ ellipsis (def. 33). -c, --count Report number of moving atoms in selections. -b, --boolean Report selections as T/F boolean arrays (def).
-
do_torsion
(instance, arg)¶ Select variable dihedrals and atoms whose positions depend on them.
Usage: torsion [options] [SELECTION_EXPR] - used by -f/-V and overules -d defailt.
- Options:
-h, --help show this help message and exit -d SELECTION_EXPR, --dihedrals=SELECTION_EXPR optimize all variable dihedrals (phi, psi) within this selection. See help select. Default: macromolecule -a SELECTION_EXPR, --atoms=SELECTION_EXPR atoms to be moved by dihedral rotations if linked to variable bonds (which must be fully enclosed by this Selection). See help select. Default: all atoms - macromolecule - -f phi|psi|pseudo, –fix=phi|psi|pseudo
- Fix all dihedrals of named type within atoms in SELECTION_EXPR (must be only option/argument.
- -t NUMBER(int)|FRACTION(float), –top=NUMBER(int)|FRACTION(float)
- Fix all but these dihedral angles (must be only option/argument).
- -V phi|psi|pseudo, –vary=phi|psi|pseudo
- Fix all dihedrals of named type within atoms in SELECTION_EXPR (must be only option/argument.
-v, --verbose List selected dihedrals (–top).
-
help_SELECTION_EXPR
()¶
-
param_type
= {'position': 'xyz', 'bfactor': 'B', 'occupancy': 'occ'}¶
-
postloop
()¶
-
preloop
()¶
-
which
()¶ Iterates over the parameter types to be modified.
- parent (
-
class
optimize.
Parameters
(atoms, refine, link=None, unlink=None)¶ Bases:
object
Maintains correspondence between refining objects & flat array for optimize.
Variables: array (ndarray) – a packed flat array containing parameters to be refined. Starts with atomic coordinates for atoms being refined individually, all x values, then y, z, B and occupancy. Torsion angles (if refining) follow, then group and overall rotational and translational parameters. The parameters to be refined are designated by dict-like Parameterization object refine. This is like supported_modes, below, except that for xyz, B & occ, an atom Selection will often replace True or False.
Maps refining parameters into a linear array.
Saves slices for extraction of updated parameters. Although atoms and groups may contain non-contiguous sequences of atoms, the refinable parameters will be loaded contiguously, so that they can be recovered using a single slice. Saves the atomic coordinates used on instantiation for extraction so that any changes outside of Parameters.array will be ignored.
Parameters: - atoms (
Atoms
) – atomic coordinates etc. - refine (
AtomicParameterization
(or subclass)) – Dict-like object defining parameters to be refined and their dependence on atomic coordinates. - link ((Group of) 2-atom atoms.Selection(s)) – Additional pair(s) of atoms to be linked for joint rotations about (other) torsion bonds.
- unlink ((Group of) 2-atom atoms.Selection(s)) – Disconnect pair(s) of atoms from those automatically linked so that not rotated together about torsion bonds.
-
analyze
(itemize=True, all_phipsi=True)¶ Analyze history of parameters and/or gradients.
Parameters: - itemize (bool) – list each group or torsion angle separately.
- all_phipsi (bool) – list fixed phi, psi as well as variable.
-
check_support
(refine)¶ Sets defaults (False) & checks that requested refinement modes supported
Parameters: refine (dict(dict(bool)|bool)) – Dictionary defining what is to be refined. Raises AssertionError: if an unsupported refinement mode is requested or for unrecognized keywords. Returns: updated refine Return type: dict
-
static
check_supported
(refine)¶ Checks compatibility of requested refinement modes.
Checks that modes and groupings within Parameterization refine are consistent with class Parameters. Any supported modes that are missing from refine are filled as False.
Changed in version 10/2/13: - type and much of the value checking is now part of class Parameterization. No longer changes refine.
Parameters: refine (Parameterization (dict-like)) – Defines what to be refined & how to group atomic parameters. Raises AssertionError: if an unsupported refinement mode is requested or for unrecognized keywords. Returns: updated refine Return type: Parameterization
-
debug_instances
= 0¶
-
extract
(array=None, update_anisotropic=True)¶ Regenerate atomic coordinates from flat parameter array, self.array.
Parameters: - array (1D ndarray or NoneType) – optional array containing (all) refinement parameters with which to update self.array before extraction (shapes must be identical).
- update_anisotropic (bool) – by setting to False, if anisotropic Us are merely riding with atomic positions and do not impact the optimization, their calculation can be deferred until a final update on convergence.
Returns: atomic coordinates
Return type: class Atoms
Warning
updates from atoms on instantiation, so ignores changes outside of refinement.
Todo
Save last parameters after extracting / scaling.
Changed in version 2/28/13: - No longer update self.array with array.
-
grouping
= 'group'¶
-
pack_like
(dxyz=None, db=None, docc=None, dmagnification=None, drigid=None, dbgroup=None, doccgroup=None, dxyzoverall=None, dboverall=None, dtorsion=None)¶ Scale and pack derivatives into 1D parameter-like array.
Must be packed in the same order as Parameters.array so that arrangement of partials will correspond to parameters. Scaling is from physical to internal units to balance the refinement of different parameters according to default or user-provided unit.per* attributes.
Warning
No attempt is made to chain the derivatives from the different types of parameters. Thus, the derivatives are at best an approximation if the same atoms are simultaneously refined by group, torsion and/or overall parameterization.
Parameters: - dxyz (nd.array.shape(3,-1)) – partials re: atomic positions
- db (ndarray) – partials re: atomic B-factors
- docc (ndarray) – partials re: atomic occupancies
- dmagnification (flaot) – partial re: overall magnification
- drigid (nd.array.shape(6,-1)) – partials for rigid group rotations/translations, noting that 1st 3 elements (rotation) will be scaled by self.scale_rotation.
- dbgroup (ndarray) – partials re: group B-factors.
- doccgroup (ndarray) – partials re: group occupancies.
Returns: packed array arranged like self.array
Return type: ndarray
Todo
support for rigid groups - must be consistent w/ __init__.
Todo
support skipping non-refining atoms.
Todo
From Brynmor: Would it be unwise to allow functions to be passed as parameters instead of arrays, so unnecessary values aren’t computed?
-
param
= 'occupancy'¶
-
partials
= "{\n 'position':{'individual':dxyz, 'group':drigid, \n 'overall':dxyzoverall, 'torsion': dtorsion},\n 'bfactor': {'individual':db, 'group':dbgroup, \n 'overall':dboverall},\n 'occupancy':{'individual':docc, 'group':doccgroup, \n 'overall':False},\n 'magnification': dmagnification\n }"¶
-
scale_rotation
¶ Rigid rotation scale factor (float).
Operators use radian units which can (without scaling) lead to numerically large partial derivatives relative to positional derivatives. To mitigate this, optimization will use, as parameters, rotation * scale_rotation (which is equivalent to using units of rad / scale_rotation). Thus, use 1000.0 to use mrad units, or choose a custom number to make commensurate the rotational and positional derivatives. The need for scaling diminished with the move to rotations about the group center (instead of 0,0,0), with 30 degree rotations commensurate with 1A shifts for an amino acid (as in unit-test). It remains to be seen whether scaling will be useful for larger groups at lower resolution. Currently applied to rigid, but not torsional rotations.
-
supported_modes
= "{\n 'position':{'individual':True, 'group':True, 'overall':True,\n 'torsion': True},\n 'bfactor': {'individual':True, 'group':True, 'overall':True},\n 'occupancy':{'individual':True, 'group':True, 'overall':False},\n 'magnification': False,\n 'resolution': False\n }"¶
-
torsion_change
()¶ Torsion angle differences (current - start), degrees.
Returns: changes, degrees Return type: dnarray(dtype=float) Changed in version 12/9/13: - wrap now correctly accounts for internal units.
-
value
= {'overall': False, 'individual': True, 'group': True}¶
- atoms (
-
class
optimize.
RSParameterization
(*args, **kwargs)¶ Bases:
optimize.ExtendedParameterization
Parameterization specific for RSRef.
-
magnification
¶ Var: Convenience synonym for dictionary item.
-
resolution
¶ Var: Convenience synonym for dictionary item.
-
-
class
optimize.
ResidueInfo
(atoms=None, prior=None, impactByDelta=None)¶ Tables for analysis, printing etc..
Empty table.
Parameters: - atoms (
Atoms
) – coordinate set. - prior (
ResidueInfo
) – previous instance to archive. - impactByDelta (bool) – impact is set to the accumulated change in dihedral, appropriate only if being refined with an L1Norm restraint on total dihedral change, so that change will be a function of the contribution to residual improvement. Else: impact is measured from cycle-by-cycle partial derivatives or changes in residual, as controled elsewhere.
Warning
If impactByDelta, impact in ResidueTable will differ from that in History.
-
angleEntry
= {'impact': None, 'used': None, 'rank': None, 'to_start': None, 'paramIndex': None, 'change': None, 'value': None}¶ Information for a single dihedral angle. Dictionary keys:
paramIndex location in array of refining torsion angles. change change in the value (degrees) through refinement. to_start dihedral rotation fr optional reference to refinement start. value final value (degrees). impact estimated impact of this angle on the residual or RMSD. rank 0 (highest impact) through 2N (phi or psi). used whether this dihedral is to be used further (bool).
-
changeArray
(individual=False)¶ List of torsion angle changes, suitable for histograms.
Parameters: individual (bool) – individual φ & ψ, else combine φi, φi-1 to approximate pseudo-torsion angles. Returns: torsion angle changes Return type: ndarray
-
color
(pdb='history_in_B.pdb', metric='impact', pseudo=True, normalized=True)¶ Write PDB file with B-factors reflecting torsion impact/change.
Parameters: - pdb (str) – name of coordinate file for fake B-factors.
- metric (str) – ‘impact’ or (angle) ‘change’.
- pseudo (bool) – sum φi, φi-1 when coloring atoms.
- normalized (bool) – color according to the relative magnitude instead of actual +/- degrees.
Returns: atoms for 3D display (if color; differences stored as B-factors).
Return type: Atoms or NoneType
-
combine
(residueID)¶ Combine φi with ψi-1.
Parameters: residueID ((str, int, str, int)) – key for residue / pseudo-residue Returns: (combined change in dihedrals, combined impact of dihedrals) Return type: (float, float)
-
combined
= None¶ True if impact measured after combining φi w/ φi-1. Else add separately calcualted impacts.
-
combined_bond_ids
(residueID)¶ Bond numbers for φi and ψi-1.
Parameters: residueID ((str, int, str, int)) – (chain/segID, residue number, insert, extension) Returns: index offsets for ψi-1 & φi (starting at 0 for the 1st dihedral in the parameter arrays). Return type: int, int
-
copy
()¶ Deepcopy, except that _saved set to None to avoid recursion.
Returns: copy. Return type: ResidueInfo
-
empty
(residueNumber, chain='', segID='', insert='', residueType=None, extension=0)¶ Empty entry for a residue.
Parameters: - residueNumber (int) –
- chain (char) – takes precedence over segID if non-blank.
- segID (str (up to 4 char)) – as in XPlor/CNS PDB file.
- insert (char) – residue insertion label, as in PDB.
- residueType (str) – eg. Ala, Asn
- extension (int) – values >= 1 designate pseudo-residue entries used, for example, for orphan dihedral angles after breaks / termini.
Warning
completely untested!
-
emptyID
(residueID, residueType=None)¶ Add empty entry for residue defined by tuple.
Parameters: - residueID ((str, int, str, int)) – (chain/segID, residue number, insert, extension)
- residueType (str) – e.g. Cys, Pro, etc.
Note
with extension >= 1 will insert a pseudo-residue after the same residue extension-1, if possible, else append (out of order) at end.
Changed in version 5/22/14: - enforce python base types on residueID. If elements copied from arrays, may be numpy.string, which if None, will be zero-length, which is not supported by pickle.load().
-
emptyLikeAtom
(atomNumber, extension=0)¶ Add empty entry (if none exists) for residue containing atomNumber.
Parameters: - atomNumber (int) – number in self.atoms.
- extension (int) – see add(), used for pseudo-residue if >= 1.
Returns: residueID = (chain/segID, residue number, insert, extension)
Return type: (str, int, str, int)
-
flatten
(attr, individual=None, asarrays=False)¶ Flat lists for chain, residue, insert, dihedral and selected attr.
Entries where attr is None are skipped.
Parameters: - attr (str) – attribute (dictionary element) or to be returned.
- individual (bool or NoneType) – attr defined for each dihedral (True) or just for each residue (False), or, if unambiguous, will be determined from attr (None).
- asarrays (bool) – ndarrays will be returned instead of lists.
- index (int or NoneType) – element if attr for each angle or residue is a list.
Returns: chain, residue, residue_insert, dihedral, attribute
Return type: ([str], [int], [str], [str] or NoneType, [float]) as arrays or lists
-
fromParameters
(parameters, reference=None, only_refining=False)¶ New residue entries w/ angles from Parameters.
Note that this is specific for selection for a particular refinement run.
Parameters: - parameters (
Parameters
) – refinement parameters. - reference (
Atoms
or NoneType) – optional self.atoms with different xyz positions, from where torsion angle changes (not impact) will be measured. Must be same atoms / order as self.atoms. - only_refining (bool) – include only torsion angles that are refining.
Returns: self
- parameters (
-
getCombined
(residueID, metric='impact')¶ Combined value of metric, calculating if not already available.
Parameters: - residueID (tuple) – identifier, see emptyID
- metric (str) – ‘impact’ or (angle) ‘change’.
Returns: value of metric
Return type: float
-
impactArray
(individual=False)¶ List of impacts, suitable for histograms.
Parameters: individual (bool) – individual φ & ψ, else combine φi and ψi-1 to approximate pseudo-torsion angles. Returns: impact values Return type: ndarray
-
impactDelta
(identify=False)¶ Impact from net torsion angle changes.
Requires a prior cycle of refinement with an L1Norm restraint sufficient that only the most impactful changes have been made. Note that impactDelata is not really evaluating impact, just copying the dihedral changes so that the class can be polymorphic.
Parameters: identify (int or bool) – list top n dihedrals by impact. Returns: self, updated [‘impact’] and [angle][‘impact’] values. Return type: ResidueInfo See also
impactHistory and impactSequential for real calculations of impact.
Changed in version 03/3/14: added filter, xChange.
-
impactHistory
(grad=False, rms=True, filter=False, xChange=False, identify=False)¶ Parameter changes x improvement, integrated over refinement cycles.
If self.impactByDelta, impact will be estimated from the accumulated change in dihedral, possible if refinement is with an L1Norm restraint that is high enough to ensure that changes only occur where they have impact upon the residueal. Otherwise a more sophisticated attempt is made to estimate impact from the changes in residual and/or partial derivatives, cycle-by-cycle.
Detailed discussion follows, but with arginine kinase, the best and most robust options are grad=False, rms=True, filter=xChange=False. Second is grad=True, rms=False, filter=True, xChange=False, and this actually works better w/ an idealized insulin fragment test. Other combinations are substantially worse.
The metric of improvement can be the change in (least squares) residual or (perhaps preferably) its square root (corresponding to an rms difference). RMS reduces the influence of early cycles with large changes, and appears just slightly better with arginine kinase (subjective assessment).
grad(metric) is a first order approximation to the derivates used in lbgfs & other optimizers. (It ignores curvature and interdependence of parameters.) If grad=False, the change in residual (or rms) each cycle is used instead of the calculated partial derivative, apportioning it among the parameters according to the magnitude of each parameter change. This is therefore a zeroth order approximation, that can work if the total torsion angle change is (highly) restrained during refinement.
Note that the combination grad=True, rms=True uses the square root of the residual gradient, not the gradient of the RMSD.
With filter=True, the impact is additionally normalized by the sum over cycles of (parameter change / absolute parameter change), i.e. removing the effect of transient oscillations in parameters. For grad=True, this is formally equivalent to calculating the cycle-average residual gradient and then multiplying by the net parameter change. Empirically, this helps (only) with grad=True.
All combinations of above are accumulated in History() during refinement so that here the appropriate choice can be analyzed per residue.
Parameters: - rms (bool) – metric is square root of residual, else residual.
- grad (bool) – use 1st partial derivatives w/ respect to each parameter, else use just the change in metric on each cycle.
- filter (bool) – normalize for each parameter by (net parameter change / sum over cycles of absolute changes).
- xChange (bool) – multiply impact by the absolute value of the total change in torsion angle. Destabilizing?
- identify (int or bool) – list top n dihedrals by impact.
Returns: self, updated [impact] and [angle][impact] values.
Return type: Changed in version 03/3/14: added filter, xChange.
-
impactSequential
(objective_f, combined=False, rmsNumData=None, identify=False)¶ Iterative impact of φ, ψ angles on objective function.
Parameters: - objective_f (callable) – object that returns the objective function value with a single argument that is the parameter array.
- combined (bool) – assess impact jointly, φi with ψi-1
- rmsNumData (int or NoneType) – Number of data (sampling) points if impact on residual is to be converted to an RMSD. 0/None for least-squares residual.
- identify (int or bool) – list top n dihedrals by impact.
Returns: self
Return type:
-
load
(filename='residueInfo.jar')¶ Serialize self (for replotting by impact.py).
Can’t pickle self.parameters as nested class Unit. Thus, can’t pickle self, and somewhat incomplete.
-
name
= None¶ Tuple key identifying each residue/pseudo residue in input order.
-
neighbors
()¶ Set values priorResidue and nextResidue to the previous and next residueIDs in a chain of sequentially numbered residues.
After breaks / termini, inserts / appends an empty extension residue so that combined φi and ψi-1 can be analyzed.
-
overall
¶ Dihedral changes from reference structure instead of start of refinement batch.
-
pickle
(filename='residueInfo.jar')¶ Serialize self (for replotting by impact.py).
Can’t pickle self.parameters as nested class Unit. Thus, can’t pickle self, and somewhat incomplete.
-
pickles
= ('residue', 'name', 'combined', 'impactType')¶ Attributes to be pickled/loaded.
-
plot
(prior=None, tty=False, plotFile='impact.png', limit=1.0, annotation=None, overall=False, changeOnly=False, verbose=False)¶ Iterative impact of each residue (φi & ψi-1) on objective function.
Parameters: - prior (str) – replace ‘impact’ or dihedral ‘change’ with values from prior refinement, supporting superimposition of restrained impact and unrestrained dihedral changes.
- tty (bool) – screen-plot impact of each torsion angle on objective function.
- plotFile (str) – save, file-type by extension, eg. .png, .ps, .pdf.
- limit (float) – the axis range for dihedral changes will be limited to limit * max(change). Use limit < 1.0 to highlight smaller changes.
- annotation (str) – file containing additional pylab plotting commands.
- overall (bool) – dihedral changes (not impact) calculated from reference (input) structure rather than start of refinement batch.
- changeOnly (bool) – plot only dihedral changes, skipping impact estimate.
Todo
Consider taking out starting impact as less useful.
-
print_all
= False¶ Print dihedral information even for unchanging bonds.
-
rankImpact
(combine=True, verbose=False)¶ In-place ranking: 0 for highest impact through N.
Sets ‘rank’ elements of self.residue dictionaries and saves ordered ranking lists.
Parameters: - combine – returned bonds are pairs for ψi-1 & φi, else single φ or ψ bonds.
- verbose (int) – how many of the top impact dihedrals to print
Variables: - ranking – highest to lowest impact list of dihedrals that are variable.
- combined_ranking – highest to lowest impact list of pseudo-torsion angles that are variable, given as tuple pairs of the constituent individual bond indices.
Returns: bond indices, starting with highest impact, either ranking or combined_ranking depending on the value of combine.
Return type: list(int) or list((int, int)) if combine
Todo
what happens to fixed dihedrals?
Todo
rasterize so that ~same impact –> same rank?
-
read
(infile)¶ Read from a formatted file to regenerate most of a residue table.
Parameters: infile (file) – text file containing a residue table. Warning
The following residue entries cannot be regenerated: priorResidue, nextResidue, and the following entries for each angle: to_start, paramIndex. Furthermore, when a reference structure was used to build the table read, the to_start value will have been incorporated within the change value. Furthermore, the input table may not have contained residues that were entirely fixed.
Todo
test with some fixed dihedrals.
-
residue
= None¶ Contains dictionary for each residue/pseudo residue.
-
residueEntry
= {'impact': None, 'nextResidue': None, 'phi': {'impact': None, 'used': None, 'to_start': None, 'rank': None, 'value': None, 'paramIndex': None, 'change': None}, 'used': None, 'rank': None, 'priorResidue': None, 'psi': {'impact': None, 'used': None, 'to_start': None, 'rank': None, 'value': None, 'paramIndex': None, 'change': None}, 'type': None, 'change': None}¶ Information for a residue / pseudo-residue. Dictionary keys:
- type
- eg. Ala, Asn...
- phi
- angleEntry dictionary.
- psi
- angleEntry dictionary.
- priorResidue
- pointer to the prior residue (residueID tuple).
- nextResidue
- pointer to the next residue (residueID tuple).
- impact
- per-residue estimate of influence on residual or RMS, calculated from effect of both φi and ψi-1.
- rank
- 0 (highest impact) through N.
- used
- residue to be used further (bool).
-
save
()¶ Saved state is over-written by current state.
-
saved
¶
- atoms (
-
class
optimize.
Superimposition
(pair, refine, reference=None, verbosity=0)¶ Bases:
optimize.AtomicRefinement
Morphing one structure to another as rigid groups, torsions, atoms.
Extends AtomicRefinement to work with a pairing of 2 coordinate sets, over-riding the default objective function and disabling refinement of Bs & occupancy.
Initialize a refinement session, defining the parameterization.
Parameters: - pair (
superpose.Pair
) – two structures with desired atoms paired - refine (
AtomicParameterization
(subclass)) – Defines what is to be refined & how. - reference (
Atoms
or NoneType) – if torsion angle changes are to be restrained (option.torsion_weight), an optional reference state of the moving structure (same atoms, order etc.) can be provided (eg. input structure); else any restraint will apply from the start of each refinement batch.
- pair (
-
class
optimize.
Tasks
¶ Bases:
object
Holder of optimization task methods to be subclassed by other programs.
Superclass for optimization-related common tasks.
Warning
If this class defines pre-requisites on which other tasks depend, then this superclass must be instantiated before the other tasks are defined. If this class depends on pre-requisites defined elsewhere, then it must be instantiated afterwards. It may not be possible to satisfy both requirements!
-
analyze
(itemize=True)¶ Pass-back after checking prerequisites for analyze sub-comds.
-
analyze_prereq
(**kwargs)¶ Pass-through, trying to meet the prerequisites.
-
convergence
(itemize=True)¶ Analyze the gradients and shifts of prior refinement.
-
dihedral_hinges
(pseudo=False, **kwargs)¶ Find & document hinges from dihedral changes.
-
dihedral_paint
(**kwargs)¶ “Color” PDB B-factors by change in dihedrals.
-
fix_dihedrals
(top=0, pseudo=True, verbose=False)¶ Fix all but the top impact previously-declared dihedral angles.
Parameters: - top (int or float) – number of highest impact bonds to retain, or fraction if less than 1.
- pseudo (bool) – use pseudo-dihedrals of ψi-1 & φi, else individual dihedrals. Unless very high resolution, use pseudo=True as these torsions are nearly covariant, so want to combine.
- verbose (bool) – document which dihedrals are selected.
-
impact
(iterative=False, rmsd=False, pseudo=False, identify=False, pickle='impact.jar', **arguments)¶ Estimate impact of each torsion angle change on fit.
Requires self.refinement to be bound to an existing refinement object.
Parameters: - iterative (bool) – from change in residual (or rmsd) on applying torsion rotations in order from highest estimated impact to lowest (from starting structure). This is computationally intensive. Default (False) is to estimate from product of parameter changes and partial derivatives integrated over refinement cycles.
- rmsd (bool) – use rmsd instead of residual.
- pseudo (bool) – when calculating iterative impact, changes in (ψi-1, φi) are made jointly. Else calculate for each individual dihedral. In all other situations the combined impact is simply the sum of the individually calculated impacts.
- identify (int or bool) – list top n dihedrals by impact.
- pickle (str) – save a copy of the residueInfo object after calculating impact into this file for re-plotting with impact.py.
- arguments (dict) – additional arguments to be passed through to residueInfo.plot().
- tty (bool) – plot on terminal.
- plotFile – save plot in file-type given extension (.png, .ps, .pdf etc.).
Todo
add a prerequisite that parameters included torsion?
-
impact_paint
(**kwargs)¶ “Color” PDB B-factors by impact of dihedrals on fit.
-
impact_paint_prereq
(*args, **kwargs)¶ Pass-through, trying to meet the prerequisites.
-
impact_pickle
(pickle='impact.jar')¶ Pickle (save) impact object for later analysis.
-
impact_pickle_prereq
(*args, **kwargs)¶ Pass-through, trying to meet the prerequisites.
-
impact_plot
(**arguments)¶ Graph impact of dihedrals on fit vs. residue.
-
impact_plot_prereq
(*args, **kwargs)¶ Pass-through, trying to meet the prerequisites.
-
impact_prereq
(*args, **kwargs)¶ Pass-through, trying to meet the prerequisites.
-
impact_print
(**arguments)¶ Print table of change in dihedrals & impact on fit.
-
impact_print_prereq
(*args, **kwargs)¶ Pass-through, trying to meet the prerequisites.
-
select_dihedrals
(fix=True, angle='pseudo', selection='all', verbose=False)¶ Fix or vary a selection of previously-declared dihedral angles.
By modifying a prior selection of dihedrals (initiated with user-command parameter> torsion), this routine can make quick changes with simple user-input and without recalculating the covalent connectivity. However, if not previously initialized, the default of all-possible dihedrals may be inefficient, if not inappropriate. An alternative implementation would be to specify prior to calculation of the connectivity, but currently this would involve user-specification of atom selections rather than the more intuitive angle specification.
Parameters: - fix (bool) – fix, else vary dihedrals.
- angle (str) – type of angle, ‘phi’, ‘psi’, or ‘pseudo’ (ψi-1 | φi).
- selection (str) – expression for set of atoms within which all angle will be selected. See documentation for Selection expressions.
- verbose (bool) – document which dihedrals are selected.
-
-
class
optimize.
TorsionGp
(*arg, **kwarg)¶ Bases:
optimize.Grouping
Defines dihedral angles and the atoms whose positions depend on them.
Parameters: arg (iterable containing TorsionParameterization or 1 or more Selection/selection expressions.) – optional designation of atom selections containing variable dihedrals (and dependent atoms). -
changing
¶ Atoms that are moving, i.e. movable AND connected to variable dihedral.
(Atoms.Selection boolean ndarray)
-
default
()¶ A sensible object value, not automatically set by __init__.
-
default_selection
()¶ Default if value set to True.
-
dihedrals_within
¶ Atoms within which dihedrals are variable.
Needs to handle non-TorsionParameterization defaults, else refer to TorsionParameterization.
-
movable
¶ Atoms that can be moved if connected to variable dihedrals.
Needs to handle non-TorsionParameterization defaults, else refer to TorsionParameterization.
-
nothing
()¶ Nothing to be optimized.
-
parse
(*arg, **kwarg)¶ Parse text input.
Parameters: arg – 0, 1 or 2 Selections or string selection expressions.
-
selections
¶ Iterator over (multiple) changing selection(s).
Returns (name (str), atom selection (Atoms.Selection)).
-
-
class
optimize.
TorsionParameterization
(*iterable, **kwargs)¶ Bases:
list
Atoms within which dihedrals rotated & postitions that depend on them.
A first selection defines atoms between which dihedral angles will be deemed variable. A second selection can be used to limit the atoms that will be moved as a result of varying above dihedrals. The second selection must include all of the first selection. The default is all atoms.
Todo
Check for connectivity here so that default second selection can be all connected atoms, improving efficiency of residual & derivative calculations.
The class manages translation from input Selection expressions to boolean Selection arrays, interpolating defaults as needed. The following alternates to an input Selection are also acceptable:
- An Atoms.Group of Selections which will be immediately flattened to a single Selection with Group.inAny().
- True: All atoms.
- False: No atoms.
- None: No atoms if 1st selection; all atoms if 2nd selection.
Properties dihedrals_within and movable will either be:
- Atoms.Selection booleans, True for each atom selected.
- None if no atoms are selected.
The class is implemented as a list with validation. Properties are recalculated if items or attributes have been changed since last access.
Initiates with arguments: 0, 1 or 2 selections as in class doc.
Maintains the following properties which are bool-like atoms.Selection arrays:
- dihedrals_within: the set of atoms between which variable dihedrals are to be found.
- movable: atoms that can be moved if connected to variable dihedrals.
- connected: all atoms that are connected to variable dihedrals.
- changing: the logical combination (movable & connected).
Also maintains property connect, which is a
TAProtein
object that handles the dependence of atoms on dihedrals. Its compute-intense calculation is just-in-time if any dependencies have been changed. Finally, maintains property variable, a boolean list of which connect.bond_atoms elements are currently variable.Parameters: - iterable (list, dict or tuple of Selection, Group or str.) – optional iterable yielding 1 or 2 atom Selection boolean arrays (or expressions to generate them). The first denotes the atoms, between any pairs of which eligible dihedrals are variable. The Selection will be available as property dihedrals_within. The second denotes the atoms allowed to move as a result of changing dihedral angles. It must be the same as or a super-set of the first (default is all atoms). The Selection will be available as property movable.
- atoms (Atoms.atoms) – Coordinate set. If not specified, will attempt to extract from Selection(s) used in instantiating. Will be needed to expand defaults into Selections, so if defaults not used on instantiation, can be set later with defaults().
- selection_expr (str) – default criterion for Atoms.Selection(). Can be set later with defaults().
-
append
(value)¶
-
changed
()¶ Determine whether new
TAProtein
instance will be needed.Cognisant that there are various combinations of self that can mean that there are no atoms depending on torsion angles.
Returns: whether self indicates new dependence of atoms on dihedrals. Return type: bool
-
changing
¶ Atoms both user-selected as movable & connected to variable dihedrals.
-
check
()¶
-
connect
¶ TAprotein instance managing connections between dihedrals and atoms.
-
connected
¶ Atoms linked to dihedrals thro’ (multiple) bonds or specified links.
-
connectivity
= True¶ atoms deemed dependent on variable dihedrals will be filtered to include only those that can be connected to the dihedrals through covalent bonds or links entered by the user. True will lead to more initial computing, perhaps paid off by decreased calculation of partial derivatives. It may need to be set to False for some of the unittesting to work.
-
default_selection_expr
= "S('all')"¶
-
defaults
(**kwargs)¶ Reset attributes needed to calculate default selections.
Parameters: - atoms (Atoms.atoms) – Coordinate set.
- selection_expression (str) – criterion for Atoms.Selection().
-
dihedrals_within
¶ Selection of atoms within which dihedrals (phi, psi) to be refined.
Returns self[1] (Atoms.Selection of NoneType) with defaults expanded or None if zero.
-
efficiency_check
= True¶
-
efficiency_level
= 0.9¶
-
extend
(seq)¶
-
insert
(i, value)¶
-
movable
¶ Selection of atoms whose positions depend on variable dihedrals.
Returns self[1] (Atoms.Selection of NoneType) with defaults expanded or None if zero.
-
parse_selection
(value)¶ String or Group reduced to Selection, else pass-through.
-
variable
¶ Which connect.bond_atoms entries define variable dihedrals (bool list).
-
class
optimize.
Unit
¶ Bases:
object
Number of internal optimizer units per physical unit.
Absolute and relative units used in optimizer may need adjustment to avoid:
- Large initial trial shift exiting bounds / convergence radius. (The first trial shift in l-bfgs appears to be arbitrarily one unit shift.)
- Poor convergence / freezing of parameters with relatively small paritals.
The conversion constants should be set so that the partial derivatives are approximately commensurate, and changes in different types of parameters are proportionate to improvements expected. Initial tests suggest that commensurate means within 10-100 fold. Defaults have been set, but, as the appropriate values depend on the system and stage of refinement, user-adjustment of over-riding defaults attributes are commonly needed. In RSRef, this can be done with a command such as:
py Q.option.units_per_envelope=0.01
.Variables: - per_rad – for rigid rotations.
- per_A2 – for B-factors, thermal displacements, envelope functions
- per_A – for atom / group positions.
- per_frac_occ – for atom / group occupancies.
- per_torsion – for torsion angle refinement (radians).
-
class
optimize.
XYZGrouping
(**kwarg)¶ Bases:
optimize.Groupings
Ways in which positions are to be refined.
Parameters: atoms ( Atoms
) – Coordinates to be parameterized.
-
optimize.
zero
(selection)¶ Query whether empty selection.
Returns: True if selection has zero atoms. Return type: bool