Documentation - C API
covdet.h File Reference

Covariant feature detectors (covdet) More...

#include <stdio.h>
#include "generic.h"
#include "stringop.h"
#include "imopv.h"
#include "scalespace.h"

Data Structures

struct  VlFrameDisc
 Disc feature frame. More...
struct  VlFrameOrientedDisc
 Oriented disc feature frame An upright frame has angle equal to zero. More...
struct  VlFrameEllipse
 Ellipse feature frame. More...
struct  VlFrameOrientedEllipse
 Oriented ellipse feature frame The affine transformation transforms the ellipse shape into a circular region. More...
struct  VlAffinePatchNormalizer
 Affine shape normalisation. More...
struct  VlCovDet
 Covariant frames detector. More...

Defines

#define VL_COVDET_MAX_ANGLES   4
#define VL_SIFT_NBO   8
#define VL_SIFT_NBP   4

Enumerations

enum  VlCovDetMethod { VL_COVDET_METHOD_DOG = 1, VL_COVDET_METHOD_HESSIAN }
 Image response functions. More...

Functions

int vl_affinepatchnormalizer_normalise (VlAffinePatchNormalizer *self, float const *image, VlAffineShapeEstimatorFrame const *frame, double magnif)
 Normalize the affine shape into a square patch.
vl_size vl_affinepatchnormalizer_get_patch_size (VlAffinePatchNormalizer const *self)
 Get size of the result patch.
float * vl_affinepatchnormalizer_get_patch (VlAffinePatchNormalizer const *self)
 Get patch with normalized kpt neighbourhood.
Create and destroy
VlAffinePatchNormalizervl_affinepatchnormalizer_new (vl_size imageWidth, vl_size imageHeight, vl_size patchSize)
 Create a new Affine shape normalisation object.
void vl_affinepatchnormalizer_delete (VlAffinePatchNormalizer *self)
 Delete Affine shape normalisation object.
VlCovDetvl_covdet_new_disc_detector (vl_size imageWidth, vl_size imageHeight, VlCovDetMethod respFunction, vl_size numOctaves, vl_index firstOctave, vl_size numLevels, vl_bool calcOrient, vl_bool calcDescr)
 Create an isotropic frames (discs) detector.
VlCovDetvl_covdet_new_ellipse_detector (vl_size imageWidth, vl_size imageHeight, VlCovDetMethod imageMeasure, vl_size numOctaves, vl_index firstOctave, vl_size numLevels, vl_size affineWinSize, vl_size descrWinSize, vl_bool calcOrient, vl_bool calcDesc)
 Create a new ellipse frames detector.
void vl_covdet_delete (VlCovDet *self)
 Delete a ::VlFrameDet object.
Process data
void vl_covdet_detect (VlCovDet *self, float const *image)
 Detect frames in an image using defined Covariant frames detector.
void vl_covdet_convert_frames (VlCovDet *self, float const *image, const void *srcFrames, vl_size srcFramesNum, VlFrameType srcFramesType)
 Convert covariant image frames.
void vl_covdet_calc_ssframe_descriptor (VlCovDet *self, float *descriptor, VlScaleSpaceFrame const *frame, double angle0)
 Compute descriptor of a scale space frame.
int vl_covdet_calc_ssframe_orientations (VlScaleSpace *grad, double angles[VL_COVDET_MAX_ANGLES], VlScaleSpaceFrame const *k)
 Calculate the frame orientation(s)
Retrieve data and parameters
vl_size vl_covdet_get_frames_num (VlCovDet const *self)
 Get number of frames.
vl_size vl_covdet_get_descriptors_num (VlCovDet const *self)
 Get number of frame descriptors.
void const * vl_covdet_get_frames_storage (VlCovDet const *self)
 Get pointer to the frames storage.
VlFrameType vl_covdet_get_frames_type (VlCovDet const *self)
 Get output frames type.
VlFrameDisc const * vl_covdet_get_discs (VlCovDet const *self)
 Get disc frames.
VlFrameOrientedDisc const * vl_covdet_get_oriented_discs (VlCovDet const *self)
 Get oriented disc frames.
VlFrameEllipse const * vl_covdet_get_ellipses (VlCovDet const *self)
 Get ellipse frames.
VlFrameOrientedEllipse const * vl_covdet_get_oriented_ellipses (VlCovDet const *self)
 Get oriented ellipses frames.
vl_size vl_covdet_get_descriptor_size (VlCovDet const *self)
 Get size of the SIFT descriptor.
float const * vl_covdet_get_descriptors (VlCovDet const *self)
 Get descriptors of the frames.
VlScaleSpacevl_covdet_get_gss (VlCovDet const *self)
 Get gaussian scale space.
VlScaleSpace const * vl_covdet_get_responses (VlCovDet const *self)
 Get responses scale space.
VlScaleSpacevl_covdet_get_gradient (VlCovDet const *self)
 Get polar gradient scale space.
VlAffineShapeEstimatorvl_covdet_get_affdet (VlCovDet *self)
 Get the Affine shape detector.
VlAffinePatchNormalizervl_covdet_get_affnorm (VlCovDet *self)
 Get affine shape normalisatior.
double vl_covdet_get_peak_thresh (VlCovDet const *self)
 Get peaks treashold.
double vl_covdet_get_edge_thresh (VlCovDet const *self)
 Get edges threshold.
double vl_covdet_get_norm_thresh (const VlCovDet *self)
 Get norm threshold.
double vl_covdet_get_magnif (const VlCovDet *self)
 Get the magnification factor.
double vl_covdet_get_window_size (const VlCovDet *self)
 Get the Gaussian window size.
vl_bool vl_covdet_get_calc_inverse_sm (const VlCovDet *self)
 Find out if the shape matrix is exported as its inverse.
Set parameters
void vl_covdet_set_peak_thresh (VlCovDet *self, double t)
 Set peaks threshold.
void vl_covdet_set_edge_thresh (VlCovDet *self, double t)
 Set edges threshold.
void vl_covdet_set_norm_thresh (VlCovDet *self, double t)
 Set norm threshold.
void vl_covdet_set_magnif (VlCovDet *self, double m)
 Set the magnification factor.
void vl_covdet_set_window_size (VlCovDet *self, double x)
 Set the Gaussian window size.
void vl_covdet_set_calc_inverse_sm (VlCovDet *self, vl_bool v)
 Set whether the shape matrix should be exported as its inverse.

Variables

VlEnumerator vlCovdetMethods [VL_COVDET_METHOD_NUM]
 Mapping between strings and VlCovDetMethod values.

Feature frames

enum  VlFrameType { VL_FRAMETYPE_DISC = 1, VL_FRAMETYPE_ORIENTED_DISC, VL_FRAMETYPE_ELLIPSE, VL_FRAMETYPE_ORIENTED_ELLIPSE }
 Types of feature frames. More...
const char * vlFrameNames [VL_FRAMETYPE_NUM]
 Names of the frame types.
VlEnumerator vlFrameTypes [VL_FRAMETYPE_NUM]
 Mapping between string values and VlFrameType values.
vl_size vl_frame_size (VlFrameType frameType)
 Get the size of a frame structure.
VlFrameType vl_frame_get_type (vl_bool affineAdaptation, vl_bool orientation)
int vl_calc_frame_orientations (float *gradient, int imageWidth, int imageHeight, double angles[VL_COVDET_MAX_ANGLES], float frameX, float frameY, float frameSigma, int frameOctave)
 Calculate the isotropic frame orientation(s)
void vl_sift_calc_descriptor (float const *gradient, int imageWidth, int imageHeight, float *descriptor, double x, double y, double sigma, double angle0, double magnif, double windowSize, double normThreshold, int borderSize)
 Calculate SIFT descriptor from raw data.

Detailed Description

Author:
Karel Lenc
Andrea Vedaldi

Define Documentation

#define VL_COVDET_MAX_ANGLES   4

Maximum number of angles per unoriented frame

#define VL_SIFT_NBO   8

Bins per orientation

#define VL_SIFT_NBP   4

Bins per spatial position


Enumeration Type Documentation

Response functions appliable to the input image used for frames detection.

Enumerator:
VL_COVDET_METHOD_DOG 

Difference of Gaussians

VL_COVDET_METHOD_HESSIAN 

Determinant of hessian matrix

Enumerator:
VL_FRAMETYPE_DISC 

Translation and scale covariant frame

VL_FRAMETYPE_ORIENTED_DISC 

Similarity covariant frame

VL_FRAMETYPE_ELLIPSE 

Affine covariant frame up to rot.

VL_FRAMETYPE_ORIENTED_ELLIPSE 

Affine covariant frame


Function Documentation

void vl_affinepatchnormalizer_delete ( VlAffinePatchNormalizer self)
Parameters:
selfAffine shape normalisation object to be deleted

The function frees the resources allocated by vl_affinepatchnormalizer_new().

float * vl_affinepatchnormalizer_get_patch ( VlAffinePatchNormalizer const *  self) [inline]
Parameters:
selfAffine shape normalisation object.
Returns:
Pointer to the patch of size vl_affinepatchnormalizer_get_patch_size
vl_size vl_affinepatchnormalizer_get_patch_size ( VlAffinePatchNormalizer const *  self) [inline]
Parameters:
selfAffine shape normalisation object.
Returns:
Size of the square patch with norm. kpt. neighbourhood.
VlAffinePatchNormalizer* vl_affinepatchnormalizer_new ( vl_size  imageWidth,
vl_size  imageHeight,
vl_size  patchSize 
)
Parameters:
imageWidthwidth of the original image.
imageHeightheight of the original image.
patchSizesize of the square patch.
Returns:
the new Affine shape normalisation object.

This class handles normalisation of frame affine anisotropic neighbourhood into an isotropic one based on the detected affine shape matrix. The neighbourhood is normalised into a square patch of size patchSize based on the data from original image of size imageWidth imageHeight

See also:
vl_affinepatchnormalizer_delete().
int vl_affinepatchnormalizer_normalise ( VlAffinePatchNormalizer self,
float const *  image,
VlAffineShapeEstimatorFrame const *  frame,
double  magnif 
)
Parameters:
selfAffine shape normalisation object.
imageImage data. Image is of size defined in constructor.
frameFrame which neighbourhood is normalized.
magnifThe magnification of keypoint scale.
Returns:
error code. The function returns 0 if it was not able to normalize due to fact, that the neighbourhood went over the image boundary.

The function normalises anisotropic neighbourhood of a keypoint into an isotropic patch based on its affine frame (

See also:
affine). This step is needed for calculation of affine invariant SIFT descriptor and orientations of the keypoint neughbourhood.

The shape of the neighbourhood is determined by the affine transformation of the frame (parameters frame->axx) and its size is defined by the frame scale (frm->sigma) and the magnification factor (

See also:
vl_affinepatchnormalizer_get_magnif() and
vl_affinepatchnormalizer_set_magnif()).

The result is stored into patch vl_affinepatchnormalizer_get_patch and vl_affinepatchnormalizer_get_patch_size.

See also:
covdet
int vl_calc_frame_orientations ( float *  gradient,
int  imageWidth,
int  imageHeight,
double  angles[VL_COVDET_MAX_ANGLES],
float  frameX,
float  frameY,
float  frameSigma,
int  frameOctave 
)
Parameters:
gradientPointer to polar gradient image.
imageWidthOriginal image width (grad. width is 2*imageWidth).
imageHeightOriginal image height.
anglesOrientations (output) with size VL_KPTS_MAX_ANGLES.
frameXCoordinate x of the frame in gradient.
frameYCoordinate y of the frame in gradient.
frameSigmaScale of the frame in gradient.
frameOctaveOctave of the frame.

The function computes the orientation(s) of the frame k. The function returns the number of orientations found (up to four). The orientations themselves are written to the vector angles.

Returns:
number of orientations found.
void vl_covdet_calc_ssframe_descriptor ( VlCovDet self,
float *  descriptor,
VlScaleSpaceFrame const *  frame,
double  angle0 
)
Parameters:
selfCovariant frames detector object
descriptorSIFT descriptor (output)
frameScaleSpace frame.
angle0Frame orientation.

The function computes the SIFT descriptor of the frame frame of orientation angle0. The function fills the buffer descriptor which must be large enough to hold the descriptor.

Note:
This method is here just from compatibility reasons, can be removed when framedet driver would replace sift driver.
int vl_covdet_calc_ssframe_orientations ( VlScaleSpace grad,
double  angles[VL_COVDET_MAX_ANGLES],
VlScaleSpaceFrame const *  k 
)
Parameters:
gradGradient scale space of double width.
anglesorientations (output) with min size VL_KPTS_MAX_ANGLES.
kframe.

The function computes the orientation(s) of the frame k. The function returns the number of orientations found (up to four). The orientations themselves are written to the vector angles.

Remarks:
The function requires the frame scale level k->s to be in the range s_min+1 and s_max-1 (where usually s_min=-1 and s_max=S+1) and frame octave k->o to be in the range o_min and o_max. If this is not the case, the function returns zero orientations.

This method is here just from compatibility reasons, can be removed when framedet driver would replace sift driver.

Returns:
number of orientations found.
void vl_covdet_convert_frames ( VlCovDet self,
float const *  image,
void const *  srcFrames,
vl_size  srcFramesNum,
VlFrameType  srcFramesType 
)
Parameters:
selfCovariant frames detector object
imageInput image
srcFramesPointer to the input frames storage
srcFramesNumNumber of input frames
srcFramesTypeType of the input frames

Convert array of frames into frames of type defined in the ::VlFrameDet object constructor.

Converted frames are stored in the same storage as detected frames and are accessible using the same methods as for accessing detected frames

See also:
vl_covdet_new_disc_detector(), vl_covdet_new_ellipse_detector()
vl_covdet_get_frames_num(), vl_covdet_get_frames_type()
void vl_covdet_delete ( VlCovDet self)
Parameters:
selfobject to delete.
See also:
vl_covdet_new_disc_detector()
vl_covdet_new_ellipse_detector()
void vl_covdet_detect ( VlCovDet self,
float const *  image 
)
Parameters:
selfCovariant frames detector
imagePointer to image of size defined in self constructor

This functions detects frames in input image and if defined in the constructor also their SIFT descriptors. Detects frames in the image using the configured VlFrameDet object. If defined in the constructor of the object, also computes descriptors of the detected frames.

The results are accessible with functions depending on the detector definition in the constructor:

For accessing the SIFT descriptors of the frames use vl_covdet_get_descriptors(), vl_covdet_get_descriptors_num() and vl_covdet_get_descriptor_size().

VlAffineShapeEstimator * vl_covdet_get_affdet ( VlCovDet self) [inline]

Returns the pointer to affine shape detector used for detecting anisotropic frames (ellipses).

Parameters:
selfCovariant frames detector.
Returns:
Affine shape detector or NULL if self configured for discs.
VlAffinePatchNormalizer * vl_covdet_get_affnorm ( VlCovDet self) [inline]

Returns pointer to the affine shape normalisation object used for normalisation of frame anisotropic neighbourhood into an isotropic neighbourhood.

Parameters:
selfCovariant frames detector.
Returns:
Affine shape normalisator or NULL if self configured for discs.
vl_bool vl_covdet_get_calc_inverse_sm ( const VlCovDet self) [inline]
Parameters:
selfCovariant frames detector.
Returns:
VL_TRUE if shape matrix is exported as its inverse.
vl_size vl_covdet_get_descriptor_size ( VlCovDet const *  self) [inline]
Parameters:
selfCovariant frames detector.
Returns:
size of the SIFT descriptor.
float const * vl_covdet_get_descriptors ( VlCovDet const *  self) [inline]

Returns pointer to array of frame descriptors where each descriptor is of size vl_covdet_get_descriptor_size.

Parameters:
selfCovariant frames detector.
Returns:
pointer to array of descriptors or NULL if descriptors not calc.
vl_size vl_covdet_get_descriptors_num ( VlCovDet const *  self) [inline]
Parameters:
selfCovariant frames detector.
Returns:
number of descriptors.
VlFrameDisc const * vl_covdet_get_discs ( VlCovDet const *  self) [inline]

Returns NULL pointer if this type of frames was not detected.

Parameters:
selfCovariant frames detector.
Returns:
pointer to the list of disc frames.
See also:
vl_covdet_get_frames_num()
double vl_covdet_get_edge_thresh ( VlCovDet const *  self) [inline]
Parameters:
selfCovariant frames detector.
Returns:
Edge threshold.
VlFrameEllipse const * vl_covdet_get_ellipses ( VlCovDet const *  self) [inline]

Return NULL pointer if this type of frames was not detected.

Parameters:
selfCovariant frames detector.
Returns:
pointer to the list of ellipse frames.
See also:
vl_covdet_get_frames_num()
vl_size vl_covdet_get_frames_num ( VlCovDet const *  self) [inline]
Parameters:
selfCovariant frames detector.
Returns:
number of frames.
void const * vl_covdet_get_frames_storage ( VlCovDet const *  self) [inline]
Parameters:
selfCovariant frames detector.
Returns:
pointer to the array of frames.
VlFrameType vl_covdet_get_frames_type ( VlCovDet const *  self) [inline]
Parameters:
selfCovariant frames detector.
Returns:
type of the detected frames.

Return type of frames for which the detector was configured. The type of the stored frames can be accessed by vl_covdet_get_frames_type() and the number of detected frames with vl_covdet_get_frames_num().

See also:
vl_covdet_get_frames_type, vl_covdet_get_frames_num
VlScaleSpace * vl_covdet_get_gradient ( VlCovDet const *  self) [inline]

Returs pointer to the scale space object of the GSS gradients where each level is array of size 2*gssLevelWidth x gssLevelHeight. The first gssLevelWidth x gssLevelHeight layer of the gradient level contains the gradient magnitude and the second the gradient angle (in radians, between 0 and $ 2\pi $) of the scale space level. The layers are aligned as [gradientMagnitude gradientAngle].

Returns NULL if the detector is configured for anisotropic frames detection.

Parameters:
selfCovariant frames detector.
Returns:
Scale space formed from the polar gradient of GSS or NULL.
VlScaleSpace * vl_covdet_get_gss ( VlCovDet const *  self) [inline]
Parameters:
selfCovariant frames detector.
Returns:
Gaussian scale space initialised with the last image.
double vl_covdet_get_magnif ( const VlCovDet self) [inline]
Parameters:
selfCovariant frames detector.
Returns:
magnification factor.
double vl_covdet_get_norm_thresh ( const VlCovDet self) [inline]
Parameters:
selfCovariant frames detector.
Returns:
Norm threshold.
VlFrameOrientedDisc const * vl_covdet_get_oriented_discs ( VlCovDet const *  self) [inline]

Returns NULL pointer if this type of frames was not detected.

Parameters:
selfCovariant frames detector.
Returns:
pointer to the list of oriented disc frames.
See also:
vl_covdet_get_frames_num()
VlFrameOrientedEllipse const * vl_covdet_get_oriented_ellipses ( VlCovDet const *  self) [inline]

Return NULL pointer if this type of frames was not detected.

Parameters:
selfCovariant frames detector.
Returns:
pointer to the list of oriented ellipse frames.
See also:
vl_covdet_get_frames_num()
double vl_covdet_get_peak_thresh ( VlCovDet const *  self) [inline]
Parameters:
selfCovariant frames detector.
Returns:
Peak threshold.
VlScaleSpace const * vl_covdet_get_responses ( VlCovDet const *  self) [inline]

The particular responses values are computed using function VlImRespFunction defined in the covdet object constructor.

In the case of DoG the returned scale space has one level less in each octave than the GSS. In case of other response functions the size is the same.

Parameters:
selfCovariant frames detector.
Returns:
Scale space of responses from the last image.
double vl_covdet_get_window_size ( const VlCovDet self) [inline]
Parameters:
selfCovariant frames detector.
Returns:
standard deviation of the Gaussian window (in spatial bin units).

This value affects only descriptors calculation of isotropic frames (discs)

VlCovDet* vl_covdet_new_disc_detector ( vl_size  imageWidth,
vl_size  imageHeight,
VlCovDetMethod  method,
vl_size  numOctaves,
vl_index  firstOctave,
vl_size  numLevels,
vl_bool  calcOrient,
vl_bool  calcDescr 
)
Parameters:
imageWidthimage width.
imageHeightimage height.
respFunctionimage response function used for frame detection
numOctavesnumber of octaves.
numLevelsnumber of levels per octave.
firstOctavefirst octave index.
calcOrientdo calc frame orientation
calcDescrdo calc frame SIFT descriptor

The function allocates and returns a new Covariant frames detector for the specified image and scale space geometry.

When calcOrient is true, detector detects VlFrameOrientedDisc otherwise it detects VlFrameDisc. Each type has got its own get methods.

Setting numOctaves to a negative value sets the number of octaves to the maximum possible value depending on the size of the image.

Returns:
the new disc frame Covariant frames detector.
See also:
vl_covdet_delete().
VlCovDet* vl_covdet_new_ellipse_detector ( vl_size  imageWidth,
vl_size  imageHeight,
VlCovDetMethod  respFunction,
vl_size  numOctaves,
vl_index  firstOctave,
vl_size  numLevels,
vl_size  affineWindowSize,
vl_size  descrWindowSize,
vl_bool  calcOrient,
vl_bool  calcDesc 
)
Parameters:
imageWidthimage width.
imageHeightimage height.
respFunctionimage response function used for frame detection
numOctavesnumber of octaves.
firstOctavefirst octave index.
numLevelsnumber of levels per octave.
affineWinSizesize of the window for affine shape detection
descrWinSizesize of the patch for sift calculation
calcOrientdo calc frames orientations.
calcDescdo calc frames descriptors

The function allocates and returns a new Covariant frames detector for the specified image and scale space geometry.

When calcOrient is true, detector detects VlFrameOrientedEllipse otherwise it detects VlFrameEllipse. Each type has got its own get methods.

Setting numOctaves to a negative value sets the number of octaves to the maximum possible value depending on the size of the image.

Returns:
the new ellipse frame Covariant frames detector.
See also:
vl_covdet_delete().
void vl_covdet_set_calc_inverse_sm ( VlCovDet self,
vl_bool  v 
) [inline]
Parameters:
selfCovariant frames detector.
vVL_TRUE if shape matrix is exported as it inverse

This parameter affects the detector only when ellipse frames are detected.

void vl_covdet_set_edge_thresh ( VlCovDet self,
double  t 
) [inline]
Parameters:
selfCovariant frames detector.
tEdges threshold.
void vl_covdet_set_magnif ( VlCovDet self,
double  m 
) [inline]
Parameters:
selfCovariant frames detector..
mMagnification factor.
void vl_covdet_set_norm_thresh ( VlCovDet self,
double  t 
) [inline]
Parameters:
selfCovariant frames detector.
tNorm threshold.
void vl_covdet_set_peak_thresh ( VlCovDet self,
double  t 
) [inline]
Parameters:
selfCovariant frames detector.
tPeaks threshold.
void vl_covdet_set_window_size ( VlCovDet self,
double  x 
) [inline]
Parameters:
selfCovariant frames detector.
xGaussian window size (in units of spatial bin).

This is the parameter $ \hat \sigma_{\text{win}} $ of the standard SIFT descriptor sift-tech-descriptor-std.

Affects only calculation of descriptors of isotropic frames (discs)

vl_size vl_frame_size ( VlFrameType  frameType) [inline]
Parameters:
frameTypeidentifier of the type of frame.
Returns:
size of the corresponding frame structure in bytes.
void vl_sift_calc_descriptor ( float const *  gradient,
int  imageWidth,
int  imageHeight,
float *  descriptor,
double  x,
double  y,
double  sigma,
double  angle0,
double  magnification,
double  windowSize,
double  normThreshold,
int  borderSize 
)
Parameters:
gradientimage gradients.
descriptorSIFT descriptor (output).
imageWidthimage width.
imageHeightimage height.
xframe x coordinate.
yframe y coordinate.
sigmaframe scale.
angle0frame orientation.
magnificationmagnification factor
windowSizesize of Gaussian window (in spatial bins)
normThresholdnorm threshold
borderSizeSize of the border of the grad. planes to ignore

The function runs the SIFT descriptor on raw data. Here gradient is a 2 x imageWidth x imageHeight array (by convention, the memory layout is as such the first index is the fastest varying one). The first imageWidth x imageHeight layer of the array contains the gradient magnitude and the second the gradient angle (in radians, between 0 and $ 2\pi $). x, y and sigma give the frame center and scale respectively.

In order to be equivalent to a standard SIFT descriptor the image gradient must be computed at a smoothing level equal to the scale of the frame. In practice, the actual SIFT algorithm makes the following additional approximation, which influence the result:

  • Scale is discretized in S levels.
  • The image is downsampled once for each octave (if you do this, the parameters x, y and sigma must be scaled too).