dnn_Net
import { dnn_Net } from '@banou/opencv-wasm'Use after await initOpenCV(). See the initialization and named imports guide.
Native object: release it with using or delete(). Factories can return null; check before calling methods.
This class allows to create and manipulate comprehensive artificial neural networks.
Neural network is presented as directed acyclic graph (DAG), where vertices are Layer instances, and edges specify relationships between layers inputs and outputs.
Each network layer has unique integer id and unique string name inside its network. LayerId can store either layer name or layer id.
This class supports reference counting of its instances, i. e. copies point to the same instance.
Constructors and members
static new
Create an owned dnn_Net object. Release native handles with using or delete().
new(): dnn_Net;The dnn_Net result.
static readFromModelOptimizer
Destructor frees the net only if there aren't references to the net anymore. Create a network from Intel's Model Optimizer intermediate representation (IR).
readFromModelOptimizer(xml: EmbindString, bin: EmbindString): dnn_Net;xmlXML configuration file with network's topology.
binBinary file with trained weights. Networks imported from Intel's Model Optimizer are launched in Intel's Inference Engine backend.
The dnn_Net result.
static readFromModelOptimizer1
Create a network from Intel's Model Optimizer in-memory buffers with intermediate representation (IR).
readFromModelOptimizer1(bufferModelConfig: ucharVector, bufferWeights: ucharVector): dnn_Net;bufferModelConfigbuffer with model's configuration.
bufferWeightsbuffer with model's trained weights.
Net object.
clone
Create another handle to the same native object. This retains the object without copying its pixels or algorithm state; dispose both handles separately.
clone(): this;The this result.
empty
Returns true if there are no layers in the network.
empty(): boolean;The boolean result.
dump
Dump net to String
dump(): string;String with structure, hyperparameters, backend, target and fusion Call method after setInput(). To see correct backend, target and fusion run after forward().
dumpToFile
Dump net structure, hyperparameters, backend, target and fusion to dot file
See: dump()
dumpToFile(path: EmbindString): void;pathpath to output file with .dot extension
dumpToPbtxt
Dump net structure, hyperparameters, backend, target and fusion to pbtxt file
dumpToPbtxt(path: EmbindString): void;pathpath to output file with .pbtxt extension
Use Netron (https://netron.app) to open the target file to visualize the model. Call method after setInput(). To see correct backend, target and fusion run after forward().
addLayer
Adds new layer to the net.
addLayer(name: EmbindString, type_: EmbindString, dtype: number, params: dnn_LayerParams): number;nameunique name of the adding layer.
type_typename of the adding layer (type must be registered in LayerRegister).
dtypedatatype of output blobs.
paramsparameters which will be used to initialize the creating layer.
unique identifier of created layer, or -1 if a failure will happen.
addLayerToPrev
Adds new layer and connects its first input to the first output of previously added layer. See: addLayer()
addLayerToPrev(name: EmbindString, type_: EmbindString, dtype: number, params: dnn_LayerParams): number;namename argument (EmbindString).
type_type argument (EmbindString).
dtypedtype argument (number).
paramsparams argument (dnn_LayerParams).
The number result.
getLayerId
Converts string name of the layer to the integer identifier.
getLayerId(layer: EmbindString): number;layerlayer argument (EmbindString).
id of the layer, or -1 if the layer wasn't found.
getLayerNames
Return the layer names configured on this dnn_Net object.
getLayerNames(): StringVector;The StringVector result. Release returned native handles with using or delete(), including handles nested in results.
getLayer
Returns pointer to layer with specified id or name which the network use.
getLayer(layerId: number): dnn_Layer | null;layerIdlayer id argument (number).
The dnn_Layer | null result.
getLayer1
deprecated: Use int getLayerId(const String &layer)
getLayer1(layerName: EmbindString): dnn_Layer | null;layerNamelayer name argument (EmbindString).
The dnn_Layer | null result.
getLayer2
Returns pointer to layer with specified id or name which the network use.
getLayer2(layerId: dnn_DictValue): dnn_Layer | null;layerIdlayer id argument (dnn_DictValue).
The dnn_Layer | null result.
connect
FIXIT: CV_WRAP Connects output of the first layer to input of the second layer.
See: setNetInputs(), Layer::inputNameToIndex(), Layer::outputNameToIndex()
connect(outPin: EmbindString, inpPin: EmbindString): void;outPindescriptor of the first layer output.
inpPindescriptor of the second layer input.
Descriptors have the following template <DFN><layer_name>[.input_number]</DFN>:
- the first part of the template <DFN>layer_name</DFN> is string name of the added layer. If this part is empty then the network input pseudo layer will be used;
- the second optional part of the template <DFN>input_number</DFN> is either number of the layer input, either label one. If this part is omitted then the first layer input will be used.
registerOutput
Registers network output with name
Function may create additional 'Identity' layer.
registerOutput(outputName: EmbindString, layerId: number, outputPort: number): number;outputNameidentifier of the output
layerIdidentifier of the second layer
outputPortnumber of the second layer input
index of bound layer (the same as layerId or newly created)
setInputsNames
Sets outputs names of the network input pseudo layer.
Each net always has special own the network input pseudo layer with id=0. This layer stores the user blobs only and don't make any computations. In fact, this layer provides the only way to pass user data into the network. As any other layer, this layer can label its outputs and this function provides an easy way to do this.
setInputsNames(inputBlobNames: StringVector): void;inputBlobNamesinput blob names argument (StringVector).
setInputShape
Specify shape of network input.
setInputShape(inputName: EmbindString, shape: MatShape): void;inputNameinput name argument (EmbindString).
shapeshape argument (MatShape).
forward
Runs forward pass to compute output of layer with name outputName.
By default runs forward pass for the whole network.
forward(outputName: EmbindString): Mat;outputNamename for layer which output is needed to get
blob for first output of specified layer. Release returned native handles with using or delete(), including handles nested in results.
forward1
Runs forward pass to compute output of layer with name outputName.
If outputName is empty, runs forward pass for the whole network.
forward1(outputBlobs: MatVector, outputName: EmbindString): void;2 available overloads
forward1(outputBlobs: MatVector): void;forward1(outputBlobs: MatVector, outputName: EmbindString): void;outputBlobsOutput destination, filled by the native operation. contains all output blobs for specified layer.
outputNamename for layer which output is needed to get
forward2
Runs forward pass to compute outputs of layers listed in outBlobNames.
forward2(outputBlobs: MatVector, outBlobNames: StringVector): void;outputBlobsOutput destination, filled by the native operation. contains blobs for first outputs of specified layers.
outBlobNamesnames for layers which outputs are needed to get
forwardAsync
Runs forward pass to compute output of layer with name outputName.
By default runs forward pass for the whole network.
This is an asynchronous version of forward(const String&). dnn::DNN_BACKEND_INFERENCE_ENGINE backend is required.
forwardAsync(outputName: EmbindString): AsyncArray;outputNamename for layer which output is needed to get
The AsyncArray result.
forwardAndRetrieve
Runs forward pass to compute outputs of layers listed in outBlobNames.
forwardAndRetrieve(outputBlobs: std__vector_cv__MatVector, outBlobNames: StringVector): void;outputBlobsOutput destination, filled by the native operation. contains blobs for first outputs of specified layers.
outBlobNamesnames for layers which outputs are needed to get
setPreferableBackend
Ask network to use specific computation backend where it supported.
See: Backend
setPreferableBackend(backendId: number): void;backendIdbackend identifier.
setPreferableTarget
Ask network to make computations on specific target device.
See: Target
List of supported combinations backend / target:
| DNN_BACKEND_OPENCV | DNN_BACKEND_INFERENCE_ENGINE | DNN_BACKEND_CUDA | |
|---|---|---|---|
| DNN_TARGET_CPU | + | + | |
| DNN_TARGET_OPENCL | + | + | |
| DNN_TARGET_OPENCL_FP16 | + | + | |
| DNN_TARGET_MYRIAD | + | ||
| DNN_TARGET_FPGA | + | ||
| DNN_TARGET_CUDA | + | ||
| DNN_TARGET_CUDA_FP16 | + | ||
| DNN_TARGET_HDDL | + |
setPreferableTarget(targetId: number): void;targetIdtarget identifier.
finalizeNet
Finalizes the network configuration and prepares it for inference.
This method must be called after setting backend/target via setPreferableBackend() and setPreferableTarget(), and before the first forward() call. It creates the underlying execution session (e.g. ONNX Runtime session) on the configured backend/target. If not called explicitly, the first forward() will call it automatically.
Calling finalizeNet() early lets you pay the one-time setup cost at a predictable point and catch configuration errors before inference.
finalizeNet(): void;setTracingMode
Set the tracing mode
setTracingMode(tracingMode: number): void;tracingModethe tracing mode, see DNN_TRACE_*
getTracingMode
Retrieve the current tracing mode
getTracingMode(): number;The number result.
setProfilingMode
Set the profiling mode
setProfilingMode(profilingMode: number): void;profilingModethe profiling mode, see DNN_PROFILE_*
getProfilingMode
Retrieve the current profiling mode
getProfilingMode(): number;The number result.
getModelFormat
Retrieve the current model format, see DNN_MODEL_*
getModelFormat(): number;The number result.
setInput
Sets the new input value for the network
See: connect(String, String) to know format of the descriptor.
If scale or mean values are specified, a final input blob is computed as:
input(n,c,h,w) = scalefactor \times (blob(n,c,h,w) - mean_c)
setInput(blob: Mat, name: EmbindString, scalefactor: number, mean: Scalar): void;4 available overloads
setInput(blob: Mat): void;setInput(blob: Mat, name: EmbindString): void;setInput(blob: Mat, name: EmbindString, scalefactor: number): void;setInput(blob: Mat, name: EmbindString, scalefactor: number, mean: Scalar): void;blobA new blob. Should have CV_32F or CV_8U depth.
nameA name of input layer.
scalefactorAn optional normalization scale.
meanAn optional mean subtraction values.
setParam
Sets the new value for the learned param of the layer.
See: Layer::blobs
Note: If shape of the new blob differs from the previous shape, then the following forward pass may fail.
setParam(layer: number, numParam: number, blob: Mat): void;layername or id of the layer.
numParamindex of the layer parameter in the Layer::blobs array.
blobthe new value.
setParam1
Sets the parameter blob of a layer identified by its name or output tensor name.
setParam1(layerName: EmbindString, numParam: number, blob: Mat): void;layerNamelayer name (classic engine) or raw ONNX output tensor name (ENGINE_NEW).
numParamindex of the constant weight input to update (0 = kernel, 1 = bias, etc.).
blobthe new parameter value.
getParam
Returns parameter blob of the layer.
See: Layer::blobs
getParam(layer: number, numParam: number): Mat;layername or id of the layer.
numParamindex of the layer parameter in the Layer::blobs array.
The Mat result. Release returned native handles with using or delete(), including handles nested in results.
getParam1
Returns parameter blob of the layer.
See: Layer::blobs
getParam1(layerName: EmbindString, numParam: number): Mat;2 available overloads
getParam1(layerName: EmbindString): Mat;getParam1(layerName: EmbindString, numParam: number): Mat;layerNamelayer name argument (EmbindString).
numParamindex of the layer parameter in the Layer::blobs array.
The Mat result. Release returned native handles with using or delete(), including handles nested in results.
getUnconnectedOutLayers
Returns indexes of layers with unconnected outputs.
FIXIT: Rework API to registerOutput() approach, deprecate this call
getUnconnectedOutLayers(): IntVector;The IntVector result. Release returned native handles with using or delete(), including handles nested in results.
getUnconnectedOutLayersNames
Returns names of layers with unconnected outputs.
FIXIT: Rework API to registerOutput() approach, deprecate this call
getUnconnectedOutLayersNames(): StringVector;The StringVector result. Release returned native handles with using or delete(), including handles nested in results.
getLayerShapes
FIXIT: CV_WRAP
The only overload of getLayerShapes that should be kept in 5.x
getLayerShapes(netInputShapes: MatShapeVector, netInputTypes: IntVector, layerId: number, inLayerShapes: MatShapeVector, outLayerShapes: MatShapeVector): void;netInputShapesnet input shapes argument (MatShapeVector).
netInputTypesnet input types argument (IntVector).
layerIdlayer id argument (number).
inLayerShapesOutput destination, filled by the native operation. in layer shapes argument (MatShapeVector).
outLayerShapesOutput destination, filled by the native operation. out layer shapes argument (MatShapeVector).
getFLOPS
FIXIT: CV_WRAP Computes FLOP for whole loaded model with specified input shapes.
getFLOPS(netInputShapes: MatShapeVector, netInputTypes: IntVector): bigint;netInputShapesvector of shapes for all net inputs.
netInputTypesvector of types for all net inputs.
computed FLOP.
getLayerTypes
Returns list of types for layer used in model.
getLayerTypes(layersTypes: StringVector): void;layersTypesOutput destination, filled by the native operation. output parameter for returning types.
getLayersCount
Returns count of layers of specified type.
getLayersCount(layerType: EmbindString): number;layerTypetype.
count of layers
getMemoryConsumption
Computes bytes number which are required to store all weights and intermediate blobs for model.
getMemoryConsumption(netInputShapes: MatShapeVector, netInputTypes: IntVector): dnn_Net_getMemoryConsumptionResult;netInputShapesvector of shapes for all net inputs.
netInputTypesvector of types for all net inputs.
The dnn_Net_getMemoryConsumptionResult result. Scalar output parameters are returned as named fields in this object. Release returned native handles with using or delete(), including handles nested in results.
enableFusion
Enables or disables layer fusion in the network.
enableFusion(fusion: boolean): void;fusiontrue to enable the fusion, false to disable. The fusion is enabled by default.
enableWinograd
Enables or disables the Winograd compute branch. The Winograd compute branch can speed up 3x3 Convolution at a small loss of accuracy.
enableWinograd(useWinograd: boolean): void;useWinogradtrue to enable the Winograd compute branch. The default is true.
getPerfProfile
Returns overall time for inference and timings (in ticks) for layers.
Indexes in returned vector correspond to layers ids. Some layers can be fused with others, in this case zero ticks count will be return for that skipped layers. Supported by DNN_BACKEND_OPENCV on DNN_TARGET_CPU only.
getPerfProfile(timings: DoubleVector): bigint;timingsOutput destination, filled by the native operation. vector for tick timings for all layers.
overall ticks for model inference.
getPerfProfile1
Returns profiling data captured during the last forward pass.
Entries are sorted by time in descending order. Empty vectors are returned if profiling is disabled (DNN_PROFILE_NONE).
getPerfProfile1(names: StringVector, timems: StringVector, counts: StringVector): void;namesOutput destination, filled by the native operation. names argument (StringVector).
timemsOutput destination, filled by the native operation. timems argument (StringVector).
countsOutput destination, filled by the native operation. counts argument (StringVector).
enableKVCache
Enables KV-Cache for all AttentionOnnxI layers
enableKVCache(): void;disableKVCache
Disables KV-Cache for all AttentionOnnxI layers
disableKVCache(): void;resetKVCache
Resets KV-Cache for all AttentionOnnxI layers
resetKVCache(): void;printPerfProfile
Prints the profile captured during the last forward pass in a formatted table using CV_LOG_INFO.
In DNN_PROFILE_DETAILED mode, prints per-layer label, time, and percentage. In DNN_PROFILE_SUMMARY mode, prints per-type count, time, and percentage. Does nothing if profiling is disabled (DNN_PROFILE_NONE) or all timings are zero.
printPerfProfile(): void;These signatures describe this package. Upstream documentation can mention optional backends that are absent from this build. Check runtime compatibility before choosing a backend or file format.