Skip to content
opencv-wasm

dnn_Tokenizer_load

Deep neural networksfunctionOpenCV 5.0.0
import { dnn_Tokenizer_load } from '@banou/opencv-wasm'

Use after await initOpenCV(). See the initialization and named imports guide.

ARGUMENTSmodel_config
FUNCTIONdnn_Tokenizer_load
RETURN TYPEdnn_Tokenizer
Call structure. A void return can still write to destination arguments. The parameter descriptions define inputs, outputs and ownership.

Load a tokenizer from a model directory.

Expects the directory to contain:

  • config.json with field model_type with value "gpt2" or "gpt4".
  • tokenizer.json produced by the corresponding model family.

The argument is a path prefix; this function concatenates file names directly (e.g. model_dir + "config.json"), so model_dir must end with an appropriate path separator.

dnn_Tokenizer_load(model_config: EmbindString): dnn_Tokenizer;
model_config

Path to config.json for model.

Returns

A Tokenizer ready for use. Throws cv::Exception if files are missing or model_type is unsupported.

Upstream declaration ↗

These signatures describe this package. Upstream documentation can mention optional backends that are absent from this build. Check runtime compatibility before choosing a backend or file format.