syncopy.specest.compRoutines.FooofSpy#
- class syncopy.specest.compRoutines.FooofSpy(*argv, **kwargs)[source]#
Compute class that checks parameters and adds metadata to output spectral data.
Sub-class of
ComputationalRoutine
, see Design Guide: Syncopy Compute Classes for technical details on Syncopy’s compute classes and metafunctions.See also
syncopy.freqanalysis
parent metafunction
- __init__(*argv, **kwargs)#
Instantiate a
ComputationalRoutine
subclass- Parameters:
*argv (tuple) – Tuple of positional arguments passed on to
computeFunction()
**kwargs (dict) – Keyword arguments passed on to
computeFunction()
- Returns:
obj – Usable class instance for processing Syncopy data objects.
- Return type:
instance of
ComputationalRoutine
-subclass
Methods
__init__
(*argv, **kwargs)Instantiate a
ComputationalRoutine
subclasscompute
(data, out[, parallel, ...])Central management and processing method
computeFunction
(trl_dat[, foi, timeAxis, ...])Run FOOOF
compute_parallel
(data, out)Concurrent computing kernel
compute_sequential
(data, out)Sequential computing kernel
This reverts the special packaging applied to the fooof backend function return values to fit them into the hdf5 container.
Reformat the gaussian and peak params for inclusion in the 2nd return value and hdf5 file.
initialize
(data, out_stackingdim[, ...])Perform dry-run of calculation to determine output shape
preallocate_output
(out[, parallel_store])Storage allocation and provisioning
process_metadata
(data, out)Meta-information manager
write_log
(data, out[, log_dict])Processing of output log
Attributes
The keys available in the metadata returned by this function.
- static computeFunction(trl_dat, foi=None, timeAxis=0, output='fooof', fooof_settings=None, noCompute=False, chunkShape=None, method_kwargs=None)#
Run FOOOF
- Parameters:
trl_dat (2D
numpy.ndarray
) – Uniformly sampled multi-channel time-seriesfoi (1D
numpy.ndarray
) – Frequencies of interest (Hz) for output. If desired frequencies cannot be matched exactly the closest possible frequencies (respecting data length and padding) are used.timeAxis (int) – Index of running time axis in trl_dat (0 or 1)
output (str) – Output of FOOOF; one of
availableFOOOFOutputs
fooof_settings (dict or None) – Can contain keys ‘in_freqs’ (the frequency axis for the data) and ‘freq_range’ (post-processing range for fooofed spectrum).
noCompute (bool) – Preprocessing flag. If True, do not perform actual calculation but instead return expected shape and
numpy.dtype
of output array.chunkShape (None or tuple) – If not None, represents shape of output spec (respecting provided values of nTaper, keeptapers etc.)
method_kwargs (dict) – Keyword arguments passed to
fooofspy()
controlling the spectral estimation method
- Returns:
spec – Complex or real spectrum of (padded) input data.
- Return type:
Notes
This method is intended to be used as
computeFunction()
inside aComputationalRoutine
. Thus, input parameters are presumed to be forwarded from a parent metafunction. Consequently, this function does not perform any error checking and operates under the assumption that all inputs have been externally validated and cross-checked.See also
syncopy.freqanalysis
parent metafunction
- valid_kws = ['in_freqs', 'freq_range', 'fooof_opt', 'out_type', 'foi', 'timeAxis', 'output', 'fooof_settings', 'noCompute', 'chunkShape', 'method_kwargs', 'fooof_settings']#
- metadata_keys = ('aperiodic_params', 'error', 'gaussian_params', 'n_peaks', 'peak_params', 'r_squared')#
The keys available in the metadata returned by this function. These come from fooof and correspond to the attributes of the fooof.FOOOF instance (with an ‘_’ suffix in fooof, e.g., aperiodic_params corresponds to fooof.FOOOF.aperiodic_params_). Please refer to the FOOOF docs for the meanings.
- process_metadata(data, out)[source]#
Meta-information manager
- Parameters:
data (syncopy data object) – Syncopy data object that has been processed
out (syncopy data object) – Syncopy data object holding calculation results
- Returns:
Nothing
- Return type:
None
Notes
This routine is an abstract method and is thus intended to be overloaded. Consult the developer documentation (Design Guide: Syncopy Compute Classes) for further details.
See also
write_log
Logging of calculation parameters
- static encode_singletrial_metadata_fooof_for_hdf5(metadata_fooof_backend)[source]#
Reformat the gaussian and peak params for inclusion in the 2nd return value and hdf5 file.
For several channels, the number of peaks may differ, and thus we cannot simply call something like np.array(gaussian_params) in that case, as that will create an array of dtype ‘object’, which is not supported by hdf5. We could use one return value (entry in the ‘metadata_fooof_backend’ dict below) per channel to solve that, but in this case, we decided to vstack the arrays instead. When extracting the data again (in process_metadata()), we need to revert this. That is possible because we can see from the n_peaks return value how many (and thus which) rows belong to which channel.
- static decode_metadata_fooof_alltrials_from_hdf5(metadata_fooof_hdf5)[source]#
This reverts the special packaging applied to the fooof backend function return values to fit them into the hdf5 container.
In the case of FOOOF, we had to np.vstack the gaussian_params and peak_params, and we now revert this.
Of course, you do not have to undo things if you are fine with passing them to the frontend the way they are stored in the hdf5.
Keep in mind that this is not directly the inverse of the function called in the cF, because: - that function prepares data from a single backend function call, while this function has to unpack the data from all cF function calls. - the input metadata to this function is a standard dict that has already been pre-processed by the general-purpose metadata extraction function metadata_from_hdf5_file().
- __init__(*argv, **kwargs)#
Instantiate a
ComputationalRoutine
subclass- Parameters:
*argv (tuple) – Tuple of positional arguments passed on to
computeFunction()
**kwargs (dict) – Keyword arguments passed on to
computeFunction()
- Returns:
obj – Usable class instance for processing Syncopy data objects.
- Return type:
instance of
ComputationalRoutine
-subclass
- compute(data, out, parallel=False, parallel_store=None, method=None, mem_thresh=0.5, log_dict=None, parallel_debug=False)#
Central management and processing method
- Parameters:
data (syncopy data object) – Syncopy data object to be processed (has to be the same object that was used by
initialize()
in the pre-calculation dry-run).out (syncopy data object) – Empty object for holding results
parallel (bool) – If True, processing is performed in parallel (i.e.,
computeFunction()
is executed concurrently across trials). If parallel is False,computeFunction()
is executed consecutively trial after trial (i.e., the calculation realized incomputeFunction()
is performed sequentially).parallel_store (None or bool) – Flag controlling saving mechanism. If None,
parallel_store = parallel
, i.e., the compute-paradigm dictates the employed writing method. Thus, in case of parallel processing, results are written in a fully concurrent manner (each worker saves its own local result segment on disk as soon as it is done with its part of the computation). If parallel_store is False and parallel is True the processing result is saved sequentially using a mutex. If both parallel and parallel_store are False standard single-process HDF5 writing is employed for saving the result of the (sequential) computation.method (None or str) – If None the predefined methods
compute_parallel()
orcompute_sequential()
are used to control the actual computation (specifically, callingcomputeFunction()
) depending on whether parallel is True or False, respectively. If method is a string, it has to specify the name of an alternative (provided) class method (starting with the word “compute_”).mem_thresh (float) – Fraction of available memory required to perform computation. By default, the largest single trial result must not occupy more than 50% (
mem_thresh = 0.5
) of available single-machine or worker memory (if parallel is False or True, respectively).log_dict (None or dict) – If None, the log properties of out is populated with the employed keyword arguments used in
computeFunction()
. Otherwise, out’s log properties are filled with items taken from log_dict.parallel_debug (bool) – If True, concurrent processing is performed using a single-threaded scheduler, i.e., all parallel computing task are run in the current Python thread permitting usage of tools like pdb/ipdb, cProfile and the like in
computeFunction()
. Note that enabling parallel debugging effectively runs the given computation on the calling machine locally thereby requiring sufficient memory and CPU capacity.
- Returns:
Nothing – The result of the computation is available in out once
compute()
terminated successfully.- Return type:
None
Notes
This routine calls several other class methods to perform all necessary pre- and post-processing steps in a fully automatic manner without requiring any user-input. Specifically, the following class methods are invoked consecutively (in the given order):
preallocate_output()
allocates a (virtual) HDF5 dataset of appropriate dimension for storing the resultcompute_parallel()
(orcompute_sequential()
) performs the actual computation via concurrently (or sequentially) callingcomputeFunction()
process_metadata()
attaches all relevant meta-information to the result out after successful termination of the calculationwrite_log()
stores employed input arguments in out.cfg and out.log to reproduce all relevant computational steps that generated out.
Parallel processing setup and input sanitization is performed by ACME. Specifically, all necessary information for parallel execution and storage of results is propagated to the
ParallelMap
context manager.See also
initialize
pre-calculation preparations
preallocate_output
storage provisioning
compute_parallel
concurrent computation using
computeFunction()
compute_sequential
sequential computation using
computeFunction()
process_metadata
management of meta-information
write_log
log-entry organization
acme.ParallelMap
concurrent execution of Python callables
- compute_parallel(data, out)#
Concurrent computing kernel
- Parameters:
data (syncopy data object) – Syncopy data object, ignored for parallel case. The input data information is already contained in the workerdicts, which are stored in self.pmap (expanded from the inargs in compute()) and can be accessed from it.
out (syncopy data object) – Empty object for holding results
- Returns:
Nothing
- Return type:
None
Notes
The actual reading of source data and writing of results is managed by the decorator
syncopy.shared.kwarg_decorators.process_io()
.See also
compute
management routine invoking parallel/sequential compute kernels
compute_sequential
serial processing counterpart of this method
- compute_sequential(data, out)#
Sequential computing kernel
- Parameters:
data (syncopy data object) – Syncopy data object to be processed
out (syncopy data object) – Empty object for holding results
- Returns:
Nothing
- Return type:
None
Notes
This method most closely reflects classic iterative process execution: trials in data are passed sequentially to
computeFunction()
, results are stored consecutively in a regular HDF5 dataset (that was pre-allocated bypreallocate_output()
). Since the calculation result is immediately stored on disk, propagation of arrays across routines is avoided and memory usage is kept to a minimum.See also
compute
management routine invoking parallel/sequential compute kernels
compute_parallel
concurrent processing counterpart of this method
- initialize(data, out_stackingdim, chan_per_worker=None, keeptrials=True)#
Perform dry-run of calculation to determine output shape
- Parameters:
data (syncopy data object) – Syncopy data object to be processed (has to be the same object that is passed to
compute()
for the actual calculation).out_stackingdim (int) – Index of data dimension for stacking trials in output object
chan_per_worker (None or int) – Number of channels to be processed by each worker (only relevant in case of concurrent processing). If chan_per_worker is None (default) by-trial parallelism is used, i.e., each worker processes data corresponding to a full trial. If chan_per_worker > 0, trials are split into channel-groups of size chan_per_worker (+ rest if the number of channels is not divisible by chan_per_worker without remainder) and workers are assigned by-trial channel-groups for processing.
keeptrials (bool) – Flag indicating whether to return individual trials or average
- Returns:
Nothing
- Return type:
None
Notes
This class method has to be called prior to performing the actual computation realized in
computeFunction()
.See also
compute
core routine performing the actual computation
- preallocate_output(out, parallel_store=False)#
Storage allocation and provisioning
- Parameters:
out (syncopy data object) – Empty object for holding results
parallel_store (bool) – If True, a directory for virtual source files is created in Syncopy’s temporary on-disk storage (defined by syncopy.__storage__). Otherwise, a dataset of appropriate type and shape is allocated in a new regular HDF5 file created inside Syncopy’s temporary storage folder.
- Returns:
Nothing
- Return type:
None
See also
compute
management routine controlling memory pre-allocation
- write_log(data, out, log_dict=None)#
Processing of output log
- Parameters:
data (syncopy data object) – Syncopy data object that has been processed
out (syncopy data object) – Syncopy data object holding calculation results
log_dict (None or dict) – If None, the log properties of out is populated with the employed keyword arguments used in
computeFunction()
. Otherwise, out’s log properties are filled with items taken from log_dict.
- Returns:
Nothing
- Return type:
None
See also
process_metadata
Management of meta-information