public class DatasetUtils extends Object
Modifier and Type | Field and Description |
---|---|
protected static org.slf4j.Logger |
utilsLogger
Setup the logging facilities
|
Constructor and Description |
---|
DatasetUtils() |
Modifier and Type | Method and Description |
---|---|
static Dataset |
append(IDataset a,
IDataset b,
int axis)
Append copy of dataset with another dataset along n-th axis
|
static IntegerDataset |
calcIndexesFromPositions(List<? extends Dataset> positions,
int[] shape,
int... mode)
Calculate indexes in given shape from datasets of position
|
static List<IntegerDataset> |
calcPositionsFromIndexes(Dataset indices,
int[] shape)
Calculate positions in given shape from a dataset of 1-D indexes
|
static CompoundDataset |
cast(Class<? extends Dataset> clazz,
Dataset... a)
Cast array of datasets to a compound dataset
|
static <T extends Dataset> |
cast(Class<T> clazz,
IDataset d)
Cast a dataset
|
static CompoundDataset |
cast(Dataset[] a,
int dtype)
Deprecated.
Please use the class-based methods in DatasetUtils,
such as
cast(Class, Dataset...) |
static Dataset |
cast(IDataset d,
boolean repeat,
int dtype,
int isize)
Deprecated.
Please use the class-based methods in DatasetUtils,
such as
cast(Class, IDataset) |
static Dataset |
cast(IDataset d,
int dtype)
Deprecated.
Please use the class-based methods in DatasetUtils,
such as
cast(Class, IDataset) |
static <T extends Dataset> |
cast(int isize,
Class<T> clazz,
IDataset d,
boolean repeat)
Cast a dataset
|
static double[] |
centroid(Dataset a,
Dataset... bases)
Get the centroid value of a dataset, this function works out the centroid in every direction
|
static Dataset |
choose(IntegerDataset index,
Object[] choices,
boolean throwAIOOBE,
boolean clip)
Choose content from choices where condition is true, otherwise use default.
|
static Dataset |
coerce(Dataset a,
Object obj)
Create a copy that has been coerced to an appropriate dataset type
depending on the input object's class
|
static <T extends CompoundDataset> |
compoundCast(Class<T> clazz,
Dataset... a)
Cast array of datasets to a compound dataset
|
static Dataset |
concatenate(IDataset[] as,
int axis)
Concatenate the set of datasets along given axis
|
static Dataset |
convertToDataset(IDataset data)
Convert (if necessary) a dataset obeying the interface to our implementation
|
static <T extends Dataset> |
copy(Class<T> clazz,
IDataset d)
Cast a dataset
|
static Dataset |
copy(IDataset d,
int dtype)
Deprecated.
Please use the class-based methods in DatasetUtils,
such as
copy(Class, IDataset) |
static <T extends CompoundDataset> |
createCompoundDataset(Class<T> clazz,
Dataset... datasets)
Create a compound dataset from given datasets
|
static CompoundDataset |
createCompoundDataset(Dataset... datasets)
Create a compound dataset from given datasets
|
static CompoundDataset |
createCompoundDataset(Dataset dataset,
int itemSize)
Create a compound dataset from given dataset
|
static CompoundDataset |
createCompoundDataset(int dtype,
Dataset... datasets)
Deprecated.
Please use the class-based methods in DatasetUtils,
such as
createCompoundDataset(Class, Dataset...) |
static CompoundDataset |
createCompoundDatasetFromLastAxis(Dataset a,
boolean shareData)
Create a compound dataset by using last axis as elements of an item
|
static Dataset |
createDatasetFromCompoundDataset(CompoundDataset a,
boolean shareData)
Create a dataset from a compound dataset by using elements of an item as last axis
|
static Object |
createJavaArray(Dataset a)
Create Java array (of arrays) from dataset
|
static List<Double> |
crossings(Dataset xAxis,
Dataset yAxis,
double yValue)
Find x values of all the crossing points of the dataset with the given y value
|
static List<Double> |
crossings(Dataset xAxis,
Dataset yAxis,
double yValue,
double xRangeProportion)
Function that uses the crossings function but prunes the result, so that multiple crossings within a
certain proportion of the overall range of the x values
|
static List<Double> |
crossings(Dataset d,
double value)
Find linearly-interpolated crossing points where the given dataset crosses the given value
|
static <T extends Dataset> |
diag(T a,
int offset)
Create a (off-)diagonal matrix from items in dataset
|
static Dataset |
extract(IDataset data,
IDataset condition)
Extract values where condition is non-zero.
|
static <T extends Dataset> |
eye(Class<T> clazz,
int rows,
int cols,
int offset) |
static Dataset |
eye(int rows,
int cols,
int offset,
int dtype)
Deprecated.
Please use the class-based methods in DatasetUtils,
such as
eye(Class, int, int, int) |
static IntegerDataset |
findFirstOccurrences(Dataset a,
Dataset values)
Find first occurrences in one dataset of values given in another sorted dataset
|
static int |
findIndexEqualTo(Dataset a,
double n)
Find absolute index of first value in dataset that is equal to given number
|
static IntegerDataset |
findIndexesForValues(Dataset a,
Dataset values)
Find indexes in sorted dataset of values for each value in other dataset
|
static int |
findIndexGreaterThan(Dataset a,
double n)
Find absolute index of first value in dataset that is greater than given number
|
static int |
findIndexGreaterThanOrEqualTo(Dataset a,
double n)
Find absolute index of first value in dataset that is greater than or equal to given number
|
static int |
findIndexLessThan(Dataset a,
double n)
Find absolute index of first value in dataset that is less than given number
|
static int |
findIndexLessThanOrEqualTo(Dataset a,
double n)
Find absolute index of first value in dataset that is less than or equal to given number
|
static <T extends Dataset> |
flipLeftRight(T a)
Flip items in left/right direction, column-wise, or along second axis
|
static <T extends Dataset> |
flipUpDown(T a)
Flip items in up/down direction, row-wise, or along first axis
|
static IntegerDataset |
indexSort(Dataset a,
Integer axis)
Indirectly sort along given axis
|
static IntegerDataset |
indices(int... shape)
Generate an index dataset for given dataset where sub-datasets contain index values
|
static Dataset |
lnnorm(Dataset a)
Function that returns a normalised dataset which is bounded between 0 and 1
and has been distributed on a natural log scale
|
static Dataset |
lognorm(Dataset a)
Function that returns a normalised dataset which is bounded between 0 and 1
and has been distributed on a log10 scale
|
static void |
makeFinite(Dataset a)
Make floating point datasets contain only finite values.
|
static Dataset |
makeUnsigned(IDataset a)
Make a dataset unsigned by promoting it to a wider dataset type and unwrapping the signs
of its contents
|
static Dataset |
makeUnsigned(IDataset a,
boolean check)
Make a dataset unsigned by promoting it to a wider dataset type and unwrapping the signs
of its contents
|
static List<Dataset> |
meshGrid(Dataset... axes)
Construct a list of datasets where each represents a coordinate varying over the hypergrid
formed by the input list of axes
|
static CompoundDataset |
norm(CompoundDataset a,
boolean overAllElements)
Function that returns a normalised compound dataset which is bounded between 0 and 1.
|
static Dataset |
norm(Dataset a)
Function that returns a normalised dataset which is bounded between 0 and 1
|
static <T extends Dataset> |
put(T a,
Dataset indices,
Object values)
Changes specific items of dataset by replacing them with other array
|
static <T extends Dataset> |
put(T a,
int[] indices,
Object values)
Changes specific items of dataset by replacing them with other array
|
static void |
removeNansAndInfinities(Dataset a,
Number value)
Removes NaNs and infinities from floating point datasets.
|
static <T extends Dataset> |
repeat(T a,
int[] repeats,
int axis)
Constructs a dataset which has its elements along an axis replicated from
the original dataset by the number of times given in the repeats array.
|
static <T extends Dataset> |
resize(T a,
int... shape)
Resize a dataset
|
static <T extends Dataset> |
roll(T a,
int shift,
Integer axis)
Roll items over given axis by given amount
|
static <T extends Dataset> |
rollAxis(T a,
int axis,
int start)
Roll the specified axis backwards until it lies in given position
|
static <T extends Dataset> |
rotate90(T a)
Rotate items in first two dimension by 90 degrees anti-clockwise
|
static <T extends Dataset> |
rotate90(T a,
int k)
Rotate items in first two dimension by 90 degrees anti-clockwise
|
static Dataset |
select(BooleanDataset[] conditions,
Object[] choices,
Object def)
Select content from choices where condition is true, otherwise use default.
|
static Dataset |
select(BooleanDataset condition,
Object x,
Object y)
Select content according where condition is true.
|
static Serializable |
serializeDataset(IDataset data)
Serialize dataset by flattening it.
|
static void |
setShapeToOriginalRank(ILazyDataset a,
int[] originalShape,
int... axes)
Set shape to keep original rank
|
static Dataset |
sliceAndConvertLazyDataset(ILazyDataset lazy)
Slice (or fully load), if necessary, a lazy dataset, otherwise take a slice view and
convert to our dataset implementation.
|
static void |
sort(Dataset a,
Dataset... b)
Sort in place given dataset and reorder ancillary datasets too
|
static <T extends Dataset> |
sort(T a) |
static <T extends Dataset> |
sort(T a,
Integer axis) |
static List<Dataset> |
split(Dataset a,
int[] indices,
int axis)
Split a dataset into parts along given axis
|
static List<Dataset> |
split(Dataset a,
int sections,
int axis,
boolean checkEqual)
Split a dataset into equal sections along given axis
|
static Dataset |
swapAxes(IDataset a,
int axis1,
int axis2)
Swap two axes in dataset
|
static <T extends Dataset> |
take(T a,
Dataset indices,
Integer axis)
Take items from dataset along an axis
|
static <T extends Dataset> |
take(T a,
int[] indices,
Integer axis)
Take items from dataset along an axis
|
static Dataset |
tile(IDataset a,
int... reps)
Construct a dataset that contains the original dataset repeated the number
of times in each axis given by corresponding entries in the reps array
|
static Dataset |
transpose(IDataset a,
int... axes)
Permute copy of dataset's axes so that given order is old order:
|
static void |
unwrapUnsigned(Dataset a,
int bitWidth)
Unwrap dataset elements so that all elements are unsigned
|
protected static final transient org.slf4j.Logger utilsLogger
public DatasetUtils()
public static Dataset append(IDataset a, IDataset b, int axis)
a
- b
- axis
- number of axis (negative number counts from last)public static <T extends Dataset> T put(T a, Dataset indices, Object values)
a
- indices
- dataset interpreted as integersvalues
- public static <T extends Dataset> T put(T a, int[] indices, Object values)
a
- indices
- values
- public static <T extends Dataset> T take(T a, Dataset indices, Integer axis)
indices
- dataset interpreted as integersaxis
- if null, then use flattened viewpublic static <T extends Dataset> T take(T a, int[] indices, Integer axis)
indices
- axis
- if null, then use flattened viewpublic static Dataset tile(IDataset a, int... reps)
a
- reps
- public static Dataset transpose(IDataset a, int... axes)
axisPerm = (p(0), p(1),...) => newdata(n(0), n(1),...) = olddata(o(0), o(1), ...) such that n(i) = o(p(i)) for all iI.e. for a 3D dataset (1,0,2) implies the new dataset has its 1st dimension running along the old dataset's 2nd dimension and the new 2nd is the old 1st. The 3rd dimension is left unchanged.
a
- axes
- if null or zero length then axes order reversedpublic static Dataset swapAxes(IDataset a, int axis1, int axis2)
a
- axis1
- axis2
- public static <T extends Dataset> T sort(T a)
a
- public static <T extends Dataset> T sort(T a, Integer axis)
a
- axis
- to sort along, if null then dataset is first flattenedpublic static void sort(Dataset a, Dataset... b)
a
- dataset to be sortedb
- ancillary datasetspublic static IntegerDataset indexSort(Dataset a, Integer axis)
a
- dataset whose indexes will be sortedaxis
- to sort along, if null then dataset is first flattenedpublic static Dataset concatenate(IDataset[] as, int axis)
as
- axis
- public static List<Dataset> split(Dataset a, int sections, int axis, boolean checkEqual)
a
- sections
- axis
- checkEqual
- makes sure the division is into equal partspublic static List<Dataset> split(Dataset a, int[] indices, int axis)
a
- indices
- axis
- public static <T extends Dataset> T repeat(T a, int[] repeats, int axis)
a
- repeats
- axis
- public static <T extends Dataset> T resize(T a, int... shape)
a
- shape
- @Deprecated public static Dataset copy(IDataset d, int dtype)
copy(Class, IDataset)
d
- The dataset to be copieddtype
- dataset typepublic static <T extends Dataset> T copy(Class<T> clazz, IDataset d)
clazz
- dataset classd
- The dataset to be cast.@Deprecated public static Dataset cast(IDataset d, int dtype)
cast(Class, IDataset)
d
- The dataset to be cast.dtype
- dataset typepublic static <T extends Dataset> T cast(Class<T> clazz, IDataset d)
clazz
- dataset classd
- The dataset to be cast.@Deprecated public static Dataset cast(IDataset d, boolean repeat, int dtype, int isize)
cast(Class, IDataset)
d
- The dataset to be cast.repeat
- repeat elements over itemdtype
- dataset typeisize
- item sizepublic static <T extends Dataset> T cast(int isize, Class<T> clazz, IDataset d, boolean repeat)
clazz
- dataset classd
- The dataset to be cast.repeat
- repeat elements over itemdtype
- dataset typeisize
- item sizepublic static <T extends CompoundDataset> T compoundCast(Class<T> clazz, Dataset... a)
clazz
- dataset classa
- The datasets to be cast.public static CompoundDataset cast(Class<? extends Dataset> clazz, Dataset... a)
clazz
- dataset classa
- The datasets to be cast.@Deprecated public static CompoundDataset cast(Dataset[] a, int dtype)
cast(Class, Dataset...)
a
- The datasets to be cast.public static Dataset makeUnsigned(IDataset a)
a
- public static Dataset makeUnsigned(IDataset a, boolean check)
a
- check
- if true, then check for negative valuespublic static void unwrapUnsigned(Dataset a, int bitWidth)
a
- datasetbitWidth
- width of original primitive in bitspublic static <T extends Dataset> T eye(Class<T> clazz, int rows, int cols, int offset)
clazz
- dataset classrows
- cols
- offset
- dtype
- @Deprecated public static Dataset eye(int rows, int cols, int offset, int dtype)
eye(Class, int, int, int)
rows
- cols
- offset
- dtype
- public static <T extends Dataset> T diag(T a, int offset)
a
- offset
- public static Dataset sliceAndConvertLazyDataset(ILazyDataset lazy) throws DatasetException
lazy
- can be nullDatasetException
public static Dataset convertToDataset(IDataset data)
data
- can be nullpublic static CompoundDataset createCompoundDataset(Dataset... datasets)
datasets
- public static <T extends CompoundDataset> T createCompoundDataset(Class<T> clazz, Dataset... datasets)
clazz
- dataset classdatasets
- @Deprecated public static CompoundDataset createCompoundDataset(int dtype, Dataset... datasets)
createCompoundDataset(Class, Dataset...)
dtype
- datasets
- public static CompoundDataset createCompoundDataset(Dataset dataset, int itemSize)
dataset
- itemSize
- public static CompoundDataset createCompoundDatasetFromLastAxis(Dataset a, boolean shareData)
a
- shareData
- if true, then share datapublic static Dataset createDatasetFromCompoundDataset(CompoundDataset a, boolean shareData)
In the case where the number of elements is one, the last axis is squeezed out.
a
- shareData
- if true, then share datapublic static Dataset coerce(Dataset a, Object obj)
a
- obj
- public static Dataset norm(Dataset a)
a
- datasetpublic static CompoundDataset norm(CompoundDataset a, boolean overAllElements)
a
- datasetoverAllElements
- if true, then normalise over all elements in each itempublic static Dataset lognorm(Dataset a)
a
- datasetpublic static Dataset lnnorm(Dataset a)
a
- datasetpublic static List<Dataset> meshGrid(Dataset... axes)
axes
- an array of 1D datasets representing axespublic static IntegerDataset indices(int... shape)
public static double[] centroid(Dataset a, Dataset... bases)
a
- the dataset to be analysedbases
- the optional array of base coordinates to use as weights.
This defaults to the mid-point of indicespublic static List<Double> crossings(Dataset d, double value)
d
- value
- public static List<Double> crossings(Dataset xAxis, Dataset yAxis, double yValue)
xAxis
- Dataset of the X axis that needs to be looked atyAxis
- Dataset of the Y axis that needs to be looked atyValue
- The y value the X values are required forpublic static List<Double> crossings(Dataset xAxis, Dataset yAxis, double yValue, double xRangeProportion)
xAxis
- Dataset of the X axisyAxis
- Dataset of the Y axisyValue
- The y value the x values are required forxRangeProportion
- The proportion of the overall x spread used to prune resultpublic static Object createJavaArray(Dataset a)
a
- datasetpublic static void removeNansAndInfinities(Dataset a, Number value)
a
- datasetvalue
- replacement valuepublic static void makeFinite(Dataset a)
a
- datasetpublic static int findIndexEqualTo(Dataset a, double n)
a
- n
- public static int findIndexGreaterThan(Dataset a, double n)
a
- n
- public static int findIndexGreaterThanOrEqualTo(Dataset a, double n)
a
- n
- public static int findIndexLessThan(Dataset a, double n)
a
- n
- public static int findIndexLessThanOrEqualTo(Dataset a, double n)
a
- n
- public static IntegerDataset findFirstOccurrences(Dataset a, Dataset values)
a
- values
- sorted 1D dataset of values to findpublic static IntegerDataset findIndexesForValues(Dataset a, Dataset values)
a
- values
- sorted 1D dataset of values to findpublic static <T extends Dataset> T roll(T a, int shift, Integer axis)
a
- shift
- axis
- if null, then roll flattened datasetpublic static <T extends Dataset> T rollAxis(T a, int axis, int start)
a
- axis
- The rolled axis (index in shape array). Other axes are left unchanged in relative positionsstart
- The position with it right of the destination of the rolled axispublic static <T extends Dataset> T flipLeftRight(T a)
a
- dataset must be at least 2Dpublic static <T extends Dataset> T flipUpDown(T a)
a
- datasetpublic static <T extends Dataset> T rotate90(T a)
a
- dataset must be at least 2Dpublic static <T extends Dataset> T rotate90(T a, int k)
a
- dataset must be at least 2Dk
- number of 90-degree rotationspublic static Dataset select(BooleanDataset condition, Object x, Object y)
condition
- boolean datasetx
- y
- public static Dataset select(BooleanDataset[] conditions, Object[] choices, Object def)
conditions
- array of boolean datasetschoices
- array of datasets or objectsdef
- default value (can be a dataset)public static Dataset choose(IntegerDataset index, Object[] choices, boolean throwAIOOBE, boolean clip)
index
- integer dataset (ideally, items should be in [0, n) range, if there are n choices)choices
- array of datasets or objectsthrowAIOOBE
- if true, throw array index out of bound exceptionclip
- true to clip else wrap indices out of bounds; only used when throwAOOBE is falsepublic static List<IntegerDataset> calcPositionsFromIndexes(Dataset indices, int[] shape)
indices
- shape
- public static IntegerDataset calcIndexesFromPositions(List<? extends Dataset> positions, int[] shape, int... mode)
positions
- as a list of datasets where each holds the position in a dimensionshape
- mode
- either null, zero-length, unit length or length of rank of shape where
0 = raise exception, 1 = wrap, 2 = clippublic static Serializable serializeDataset(IDataset data)
data
- public static Dataset extract(IDataset data, IDataset condition)
data
- condition
- should be broadcastable to datapublic static void setShapeToOriginalRank(ILazyDataset a, int[] originalShape, int... axes)
a
- originalShape
- axes
- Copyright © 2014–2019 Eclipse Foundation. All rights reserved.