Smart Engines 2021: Международная награда в ИТ-инновациях

Smart Code Engine SDK Overview

About Smart Code Engine

Smart Code Engine is a multiplatform, stand-alone SDK for recognizing linear and 2D codes, MRZ, bank card numbers and phone numbers, and other codified objects.

Supported operating systems

The following operating systems are supported:

  • iOS;
  • Android;
  • MS Windows;
  • Linux-based systems;
  • Mac OS.

Workflow

General workflow includes the following stages:

  1. creation of an CodeEngine instance, see Creating an CodeEngine instance;
  2. setting up a recognition session:
  1. optional implementation of callbacks (optionally), see Callbacks;
  2. creation of an image object, see Creating an Image;
  3. processing the image, see Processing the image;
  4. getting the recognition result, see Recognition result.

Creating an CodeEngine instance

Create a CodeEngine instance as follows:

// C++
std::unique_ptr<se::code::CodeEngine> engine(
    se::code::CodeEngine::CreateFromEmbeddedBundle());

// Java
import com.smartengines.code.*;
CodeEngine engine = CodeEngine.CreateFromEmbeddedBundle();

Parameters:

  • the CreateFromEmbeddedBundle()  method implies a flag for enabling lazy configuration (true by default). It is an optional parameter. 
    • If lazy configuration is enabled, some of the internal structures will be allocated and initialized only when first needed. 
    • If you disable the lazy configuration, all the internal structures and components will be initialized  during CodeEngine instance creation.

TIP

 Disable lazy configuration for:
• server applications for which the first recognition response time is more important than the total memory consumption;
• measuring the maximum memory consumed by the application

Configuration process may take a while but it only needs to be performed once during the program lifetime. 

Configured CodeEngine is used to spawn recognition sessions – instances of CodeEngineSession, which has actual recognition methods.

ATTENTION!

CodeEngine::CreateFromEmbeddedBundle() is a factory method and returns an allocated pointer. You are responsible for deleting it

Setting up a recognition session

Creating a session setting object

Create CodeEngineSessionSettings from the configured CodeEngine:

// C++

std::unique_ptr<se::code::CodeEngineSessionSettings> settings(

engine->GetDefaultSessionSettings());

// Java
import com.smartengines.code.*;
CodeEngineSessionSettings settings = engine.GetDefaultSessionSettings();

ATTENTION!

CodeEngine::GetDefaultSessionSettings() is a factory method and returns an allocated pointer. You are responsible for deleting it

Specifying recognition engines to run

Specify recognition engines to run:

// C++
settings->SetOption("barcode.enabled", "true"); // Enabling barcode recognition in this session

// Java
settings.SetOption("barcode.enabled", "true"); // Enabling barcode recognition in this session

Specifying additional session options

This step is not required.

Optionally, you may set additional session options:

// C++
settings->SetOption("barcode.COMMON.enabled", "true"); // Enabling the common barcode symbologies recognition in this session

// Java
settings.SetOption("barcode.COMMON.enabled", "true"); // Enabling the common barcode symbologies recognition in this session

See more about options in Session Options.

Spawning  a CodeEngineSession

A personal signature is provided to the customer when the Smart Code Engine product is delivered. This signature is located in the README.html file in the /doc directory. 

Each time a recognition Code Engine session instance is created, the signature must be passed as one of the arguments to the session creation function. It confirms that the caller is authorized to use the library and unlocks the library. 

Functionality is checked offline, the library does not access any external resources.

Spawn an CodeEngineSession:

// C++
const char* signature = "... YOUR SIGNATURE HERE ..."; //Your personal signature you use to start Smart Code Engine session
std::unique_ptr<se::code::CodeEngineSession> session(
    engine->SpawnSession(*settings, signature, &my_workflow_feedback, &my_visualization_feedback));

// Java
import com.smartengines.code.*;
String signature = "... YOUR SIGNATURE HERE ..."; //Your personal signature you use to start Smart Code Engine session
CodeEngineSession session = engine.SpawnSession(
    settings, signature);

ATTENTION!

CodeEngine::SpawnSession() is a factory method and returns an allocated pointer. You are responsible for deleting it

Callbacks

Optionally, you may implement callbacks by using the  CodeEngineWorkflowFeedback and CodeEngineVisualizationFeedback subclasses:

// C++
class MyWorkflowFeedback : public se::code::CodeEngineWorkflowFeedback { /* callbacks */ };
class MyVisualizationFeedback : public se::code::CodeEngineVisualizationFeedback { /* callbacks */ };
// ...
MyWorkflowFeedback my_workflow_feedback;
MyVisualizationFeedback my_visualization_feedback;

// Java
import com.smartengines.code.*;
class MyWorkflowFeedback extends CodeEngineWorkflowFeedback { /* callbacks */ }
class MyVisualizationFeedback extends CodeEngineVisualizationFeedback { /* callbacks */ }
// ...
MyWorkflowFeedback my_workflow_feedback = new MyWorkflowFeedback();
MyVisualizationFeedback my_visualization_feedback = new MyVisualizationFeedback();
CodeEngineSession session = engine.SpawnSession(
    settings, signature, my_workflow_feedback, my_visualization_feedback);

See more about callbacks in Processing Feedback

Creating an Image

Create an Image object which will be used for processing:

// C++
std::unique_ptr<se::common::Image> image(
    se::common::Image::FromFile(image_path)); // Loading from file

// Java
import com.smartengines.code.*;
Image image = Image.FromFile(image_path);

ATTENTION!

Image::FromFile() is a factory method and returns an allocated pointer. You are responsible for deleting it

Processing the image

Call the Process(…) method for processing the image:

// C++
const se::code::CodeEngineResult& result = session->Process(*image);

// Java
import com.smartengines.code.*;
CodeEngineResult result = session.Process(image);

ATTENTION!

CodeEngineResult::Process() is not a factory method, but the returned result object is not independent. The result object lifetime does not exceed the session lifetime

Recognition result

To perform recognition in a video stream, process the frames sequentially within the same session. The processing time is unlimited, but the best practice is  processing frames coming from the stream until result.GetIsTerminal() is true.

Use CodeEngineResult fields to extract recognized information:

// C++
for (auto it_obj = result.ObjectsBegin(); it_obj != result.ObjectsEnd(); ++it_obj) {
    const se::code::CodeObject& code_object = it_obj.GetValue();
    std::string object_type = code_object.GetTypeStr(); // Type of the codified object in string format
    bool is_accepted = code_object.IsAccepted(); // Accept flag value
    for (auto it_field = code_object.FieldsBegin();
         it_field != code_object.FieldsEnd(); ++it_field) {
        const se::code::CodeField &code_field = it_field.GetValue();
        std::string code_field_name = code_field.Name(); // Code field name
        if (code_field.HasOcrStringRepresentation())
            std::string ocr_string = code_field.GetOcrString()
                .GetFirstString()
                .GetCStr(); // UTF-8 string representation of the recognized field
    }
}

// Java
import com.smartengines.code.*;
for (CodeObjectsMapIterator it_obj = result.ObjectsBegin();
     !it_obj.Equals(result.ObjectsEnd()); it_obj.Advance()) {
    CodeObject code_object = it_obj.GetValue();
    String object_type = code_object.GetTypeStr(); // Type of the codified object in string format
    boolean is_accepted = code_object.IsAccepted(); // Accept flag value
    for (CodeFieldsMapIterator it_field = code_object.FieldsBegin();
         !it_field.Equals(code_object.FieldsEnd()); it_field.Advance()) {
        CodeField code_field = it_field.GetValue();
        String code_field_name = code_field.Name(); // Code field name
        if (code_field.HasOcrStringRepresentation())
            String ocr_string = code_field.GetOcrString().GetFirstString().GetCStr(); // UTF-8 string representation of the recognized field
}

SDK Overview

Delivery package

The basic Smart Code Engine delivery package includes: 

  • C++ prebuilt library;
  • documentation;
  • usage examples;
  • prebuilt wrappers for C#/Java/Python/PHP;
  • obj-C/React/Flutter wrapper source code for mobile platform;
  • the wrapper auto generators to build the wrappers for the client’s product environment (If the auto generator fails to start, use the client’s exact version of Python 3 or PHP).

The files are arranged in directories as shown in the table below:

Directory Contents Description
secommon C++ se::common namespace files Common classes, such as Point, OcrString, Image, etc. See Common classes
Files of integration, for example,
Java com.smartengines.common module (one compiled file)
codeengine C++ se::code namespaces  Main classes of Smart Code Engine. See Main classes
Files of integration, for example Java com.smartengines.code module (one compiled file)
doc Documentation See Code documentation
samples Complete compilable and runnable sample code See Samples

Common classes

Common classes, such as Point, OcrString, Image, etc. are located within se::common namespace and are located within a secommon directory.

The C++ common classes headers:

Header Description
#include <secommon/se_export_defs.h>   Contains export-related definitions of Smart Engines libraries
#include <secommon/se_exceptions_defs.h>   Contains the definition of exceptions used in Smart Engines libraries
#include <secommon/se_geometry.h>   Contains geometric classes and procedures (Point, Rectangle, etc.)
#include <secommon/se_image.h>   Contains the definition of the Image class 
#include <secommon/se_string.h>   Contains the string-related classes (MutableString, OcrString, etc.)
#include <secommon/se_string_iterator.h>   Contains the definition of string-targeted iterators
#include <secommon/se_serialization.h>   Contains auxiliary classes related to object serialization (not used in Smart Code Engine)
#include <secommon/se_common.h>   This is an auxiliary header which simply includes all of the above

The same common classes in Java API are located within com.smartengines.common module:

// Java
import com.smartengines.common.*; // Import all se::common classes

Main classes

The main Smart Code Engine classes are located within the se::code namespaces and are located within an codeengine directory.

The C++ main classes headers:

Header Description
#include <codeengine/code_engine.h>   Contains CodeEngine class definition
#include <codeengine/code_engine_session.h>  Contains CodeEngineSession class definition
#include <codeengine/code_engine_session_settings.h>   Contains CodeEngineSessionSettings  class definition
#include <codeengine/code_engine_result.h>   Contains CodeEngineResult class definition
#include <codeengine/code_engine_feedback.h>   Contains CodeEngineWorkflowFeedback and CodeEngineVisualizationFeedback and associated containers
#include <codeengine/code_object_field.h>   Contains CodeField class definition
#include <codeengine/code_object.h>   Contains CodeObject class definition

The same classes in Java API are located within com.smartengines.code module:

// Java
import com.smartengines.code.*; // Import all se::codeengine classes

Documentation

All the classes, their methods, the methods options and options values are described both in comments that are converted into codeengine.pdf included into the documentation directory. 

The documentation is available at doc directory. The doc directory structure:

  • README.html – brief description of Smart Code Engine SDK;
  • best_practices.html – recommendations on product usage;
  • codeengine.pdf – description of the classes, their methods etc.;
  • NOTICE.txt – contains information about the used external software;
  • WHATSNEW.txt – contains information about updates.

Exceptions

The C++ API may throw se::common::BaseException subclasses when the user passes invalid input, makes bad state calls or if something else goes wrong. 

The following exception (se::common::BaseException) subclasses are implemented:

Exception name Description
FileSystemException Thrown if an attempt is made to read from a non-existent file, or other file-system related IO error
InternalException Thrown if an unknown error occurs or if the error occurs within internal system components
InvalidArgumentException Thrown if a method is called with invalid input parameters
InvalidKeyException Thrown if to an associative container the access is performed with an invalid or a non-existent key, or if the access to a list is performed with an invalid or out-of-range index
InvalidStateException Thrown if an error occurs within the system in relation to an incorrect internal state of the system objects
MemoryException Thrown if an allocation is attempted with insufficient RAM
NotSupportedException Thrown when trying to access a method which given the current state or given the passed arguments is not supported in the current version of the library or is not supported at all by design
Uninitialized Object Exception Thrown if an attempt is made to access a non-existent or non-initialized object

Exceptions contain useful human-readable information. Please read e.what() message if exception is thrown. 

Note

 se::common::BaseException is not a subclass of std::exception.
Smart Code Engine interface does not have any dependency on the STL

The thrown exceptions are wrapped in general java.lang.Exception. In Java, the exception type is included in the corresponding message text.

If you face a problem, or contact us at sales@smartengines.com or support@smartengines.com.

Factory methods and memory ownership

Several Smart Code Engine SDK classes have factory methods which return pointers to heap-allocated objects. Caller is responsible for deleting such objects (a caller is probably the one who is reading this right now)

TIP

In C++:
For simple memory management and avoiding memory leaks, use smart pointers, such as std::unique_ptr<T> or std::shared_ptr<T>.

In Java API:
For the objects which are no longer needed, use the .delete() method to force the deallocation of the native heap memory

Setting barcode recognition

The barcode engine recognizes barcodes on the specified set of frames. By default it is disabled. To enable it, set the corresponding session option:

// C++
settings->SetOption("barcode.enabled", "true");

// Java
settings.SetOption("barcode.enabled", "true");

ATTENTION!

By default, all barcode symbologies are disabled

To enable a specific barcode symbology, set the corresponding session option:

// C++
settings->SetOption("barcode.<symbology>.enabled", "true");

// Java
settings.SetOption("barcode.<symbology>.enabled", "true");
The sets of supported symbologies are shown in the table below:

Name Barcodes Description
COMMON AZTEC | QR_CODE | DATA_MATRIX | PDF_417 | CODABAR | CODE_128 | EAN_8 | EAN_13_UPC_A | ITF | UPC_E | CODE_39 | CODE_93 Symbologies that are used most frequently. Each symbology can be enabled separately 
ALL AZTEC | QR_CODE | MICRO_QR | RMQR | DATA_MATRIX | PDF_417 | MICRO_PDF_417 | CODABAR | CODE_128 | EAN_8 | EAN_13_UPC_A | ITF | UPC_E | CODE_39 | CODE_93 The full list of the supported symbologies. Each symbology can be enabled separately
ALL_1D CODABAR | CODE_128 | EAN_8 | EAN_13_UPC_A | ITF | UPC_E | CODE_39 | CODE_93 One dimensional barcodes. Each symbology can be enabled separately
ALL_2D AZTEC | QR_CODE | MICRO_QR | RMQR | DATA_MATRIX | PDF_417 | MICRO_PDF_417 Two dimensional barcodes. Each symbology can be enabled separately

Enable the COMMON set of barcodes:

// C++
settings->SetOption("barcode.COMMON.enabled", "true");

// Java
settings.SetOption("barcode.COMMON.enabled", "true");.

Enable the ALL set of barcodes:

// C++
settings->SetOption("barcode.ALL.enabled", "true");

// Java
settings->SetOption("barcode.ALL.enabled", "true");

ATTENTION!

We recommend to enable only the required symbologies to increase performance

The recognition result contains two fields: 

  • bytes – contains the content representation in base64 format. 
  • value – contains the human readable barcode content if possible, otherwise it contains the copy of the bytes field data. 

The attribute encoding of this field specifies which type of content it has. Common values are utf8 and base64.

There are three main scenarios for barcode recognition: 

  • focused processes barcodes using handheld camera recognition in video stream when the code occupies the significant area of the source image;
  • anywhere processes barcodes on any image in any location. A resource consuming operation. It can be used to process any image from the gallery. It is recommended to use this scenario for recognizing images from the gallery;
  • dummymainly used when the location of the barcode is predefined. 

ATTENTION!

For payment barcodes, it is recommended to set the following options: enabling the QR, Aztec and DataMatrix barcode symbologies, the “focused” recognition mode, max code =1 – restricting the maximum number of recognized barcodes to one 

To enable the required mode, set the session option:

// C++
settings->SetOption("barcode.roiDetectionMode", "focused");

// Java
settings.SetOption("barcode.roiDetectionMode", "focused");

By default, the barcode feedmode is set to single – barcode recognition stops after one barcode is found. To recognize several barcodes, set feedmode to sequence.

// C++
settings->SetOption("barcode.feedMode", "sequence");

// Java
settings.SetOption("barcode.feedMode", "sequence");

There is a set of structured text messages commonly encoded using barcodes. You can provide a hint which structured message you expect to meet in your session. This hint is denoted as a preset and populates the set of fields in which the content is splitted. By default, no preset is enabled. 

To enable a preset, set the special session option:

// C++
settings->SetOption("barcode.preset", "AAMVA");

// Java
settings.SetOption("barcode.preset", "AAMVA");

You can also enable multiple presets. In this case, the value of the session option takes the following form:

// C++
settings->SetOption("barcode.preset", "URL | PAYMENT");

// Java
settings.SetOption("barcode.preset", "URL | PAYMENT");

where the option value contains a set of presets separated by a | sign. If a preset succeeded, CodeField message_type containing the preset’s name is added to the result as well as the corresponding list of CodeFields.

The full list of supported presets is the following: AAMVA, GS1, URL, EMAIL, VCARD, ICALENDAR, PHONE, SMS, ISBN, WIFI, GEO, PAYMENT. Every preset contains its own set of fields to populate:

Preset Fields
AAMVA In accordance with the AAMVA specification, e.g. “AAMVA_VERSION”, “ANSI”, “DAC” and many others
GS1 “BATCH/LOT”, “EXPIRY”, “GTIN”, “SERIAL”, “SSCC/SSCI” and others
URL “URL”
EMAIL “Body”, “Recipients”, “Subject”, “URL mailto”, “Carbon copy” and others
VCARD “vCard version”, “ADR”, “AGENT”, “BDAY”, “CATEGORIES”, “CLASS”, “LABEL”, “EMAIL” and many others
ICALENDAR “EVENT0”, “EVENT1” etc
PHONE “Phone”
SMS “SMS_number”, “Body”
ISBN “ISBN”, “ISBN type”, “Prefix”, “Registration group”
WIFI “Authentication type”, “Password”, “SSID” and others
GEO “Latitude”, “Longitude”, “Query”
PAYMENT “BIC”, “BankName”, “CorrespAcc”, “Name”, “PersonalAcc” and many others

The barcode related options are presented in the table.

Option name Value type Default Description
barcode.enabled “true” or “false” “false” Enables/disables barcode recognition
barcode.ALL.enabled “true” or “false” “false” Enables all barcode symbologies
barcode.COMMON.enabled “true” or “false” “false” Enables common barcode symbologies
barcode.maxAllowedCodes Integer “1” Specifies the max number of recognized codes
barcode.roiDetectionMode “focused”, “anywhere” or “dummy” “focused” Specifies the ROI detection mode
barcode.feedMode “sequence” or “single” “single” Specifies the feed mode of barcode session
barcode.effortLevel “low”, “normal” or “high” “normal” Specifies the recognition effort level
barcode.<symbology>.enabled “true” or “false” “false” Enables/disables specified symbology
barcode.<symbology>.minMsgLen Integer  “1” The minimum length of barcode message
barcode.<symbology>.maxMsgLen Integer  “999” The maximum length of barcode message
barcode.<symbology>.checkSumPresence Integer  “0” Specifies the presence of checksum (for 1d codes)
barcode.<symbology>.extendedMode “true” or “false” “false” Specifies the extended mode (for 1d codes)
barcode.<symbology>.barcodeColour “black”, “white” or “unknown” “unknown” Specifies the expected module color
barcode.preset preset name none Specifies the preset for the decoded barcode content interpretation
barcode.preset.PAYMENT.strictSpecCompliance “true” or “false” “false” Specifies if the PAYMENT preset strictly follows the corresponding specification
barcode.smartPaymentBarDecoding “true” or “false” “false” Enables smart encoding detection for payment barcodes

You can enable text smart encoding detection for payment barcodes which violates the specification. To achieve that you must set the special session option:

// C++
settings->SetOption("barcode.smartPaymentBarDecoding", "true");

// Java
settings.SetOption("barcode.smartPaymentBarDecoding", "true");

If the algorithm succeeds, CodeField smart_payment_bar_decoding_result containing the encoding name is added to the result and may be used in the PAYMENT preset instead of following the specification. The full list of supported encodings is the following: UTF8, CP1251, KOI8_R, ISO8859_5, CP932.

Setting bank card recognition

The bank_card engine recognizes bank cards on the given set of frames. By default it is disabled. To enable it, set the corresponding session option:

// C++
settings->SetOption("bank_card.enabled", "true");

// Java
settings.SetOption("bank_card.enabled", "true");

There are three types of bank cards: 

  • embossed – bank cards with an embossed data pattern;
  • indent – bank cards with an  intent-printed data pattern; 
  • freeform – bank cards with a flat-designed data pattern, where data of interest may be located anywhere in the card.

By default all these types are enabled. The types of recognized bank cards could be explicitly specified using the corresponding session option:

// C++
settings->SetOption("bank_card.embossed.enabled", "true");

// Java
settings.SetOption("bank_card.embossed.enabled", "true");

For embossed and indent bank cards the recognition result may contain the following list of fields:

Field Description
number the number of the bank card (mandatory)
name the cardholder’s name
expiry_date the expiry date in format MM/YY
optional_data_0 additional data line in any format
optional_data_1 additional data line in any format
optional_data_2 additional data line in any format
For freeform bank cards the recognition result may contain the following list of fields:
Field Description
number the number of the bank card (mandatory)
name the cardholder’s name
expiry_date the expiry date in format MM/YY
iban the IBAN number (according to ISO 13616)
secret code VC/CVV code

The capture mode determines where to find the bank card. For this engine, there are two supported capture modes: 

  • mobile – mostly addresses handheld camera recognition in video stream; 
  • anywhere – suitable for scanned images, webcam images, and arbitrary placed bank cards. It is recommended to use this mode for recognizing cards and images from the device gallery.

By default the mobile mode is enabled.

The set of bank card related options is presented in the table.

Option name Value type Default Description
bank_card.enabled “true” or “false” “false” Enables/disables bank card recognition
bank_card.captureMode “mobile” or “anywhere” “mobile” Specifies bank card detection mode
bank_card.enableStoppers “true” or “false” “true” Enables smart text fields stoppers
bank_card.extractBankCardImages “true” or “false” “false” Extracts rectified bank card image and stores it in the relevant CodeObject

Setting MRZ recognition

The MRZ engine recognizes MRZ on the given set of frames according to the ICAO specification link. By default this feature is disabled. To enable it, set the corresponding session option:

// C++
settings->SetOption("mrz.enabled", "true");

// Java
settings.SetOption("mrz.enabled", "true")

The recognition result contains the full_mrz code field alongside with the set of splitted fields whose names depend on the determined MRZ type.

The MRZ engine supports recognition of the following MRZ types:

MRZ type Document Description
ICAO 9303 MRZ MRP (TD3) subtype Machine Readable Passport 2 lines, 44 characters each
ICAO 9303 MRZ MRVA subtype Machine Readable Visa Type A, 2 lines, 44 characters each
ICAO 9303 MRZ MRVB subtype Machine Readable Visa Type B, 2 lines, 36 characters each
ICAO 9303 MRZ TD1 subtype Machine Readable Travel Document Type 1, 3 lines, 30 characters each
ICAO 9303 MRZ TD2 subtype Machine Readable Travel Document Type 2, 2 lines, 36 characters each
MRZ-like zone   Bulgarian vehicle registration certificates 3 lines, 30 characters each
MRZ-like zone Swiss driving license 3 lines, 9 characters in the first line, 30 characters in the second and third lines
MRZ-like zone Ecuador ID 3 lines, 30 characters each
MRZ-like zone French ID card 2 lines, 36 characters each
MRZ-like zone Kenya ID 3 lines, 30 characters each
MRZ-like zone Russian national passport 2 lines, 44 characters each
MRZ-like zone Russian visa 2 lines, 44 characters each

Depending on the MRZ type, the recognition result contain a subset of code fields of the following names: 

Field name Description
mrz_doc_type_code Document type code
mrz_number Document number
mrz_issuer Document issuer
mrz_proprietor Proprietor
mrz_vehicle_number Vehicle number
mrz_line1 Line 1
mrz_line2 Line 2
mrz_line3 Line 3
full_mrz Full MRZ
mrz_vin VIN (Vehicle Identification Number)
mrz_id_number ID number
mrz_cd_number Number check digit
mrz_cd_bgrvrd_1 Bulgaria VRC (Vehicle registration certificate) control digit 1
mrz_cd_bgrvrd_2 Bulgaria VRC (Vehicle registration certificate) control digit 2
mrz_birth_date Birthdate
mrz_name Name
mrz_nationality Nationality
mrz_opt_data_1 Personal number or other optional data elements from line 1
mrz_opt_data_2 Personal number or other optional data elements from line 2
mrz_last_name Last name
mrz_gender Gender
mrz_expiry_date Expiry date
mrz_issue_date Issue date
photo Photo
mrz_cd_birth_date Birthdate check digit
mrz_cd_composite Composite check digit
mrz_cd_expiry_date Expiry date check digit
mrz_cd_opt_data_2 Optional data from line 2 check digit
mrz_authority_code Authority code
mrz_id_visa Visa ID
mrz_invitation_number Invitation number
mrz_cd_name Name check digit
mrz_cd_invitation_number Invitation number check digit
mrz_cd_id_visa Visa ID check digit
mrz_first_name First name
mrz_cd_issue_date Issue date check digit

Setting codified text line recognition

The code_text_line engine recognizes codified text lines, i.e. the lines with preset content. By default it is disabled. To enable it, set the corresponding session option:

// C++
settings->SetOption("code_text_line.enabled", "true");

// Java
settings.SetOption("code_text_line.enabled", "true");

There are two types of supported codified text lines: phone_number and card_number. By default both these types are disabled.

WARNING!

The phone_number and card_number types are mutually exclusive

The type of a codified text line must be explicitly specified by using the corresponding session option:

// C++
settings->SetOption("code_text_line.card_number.enabled", "true");

// Java
settings.SetOption("code_text_line.card_number.enabled", "true");
This engine can recognize both printed and handwritten text lines. 

ATTENTION!

The text line area should occupy at least one third of the specified region area. 

The following phone number formats are supported:

  1. 11 digits – phone numbers starting with 7 or 8 and with codes 3XX, 4XX, 7XX, 8XX, 9XX;
  2. 10 digits – phone numbers starting with 9 (abbreviation of mobile numbers)

Setting payment details recognition

The payment_details engine recognizes payment details presented in the text form required to make a payment in the financial system of the Russian Federation. By default it is disabled. To enable it, set the corresponding session option:

// C++
settings->SetOption("payment_details.enabled", "true");

// Java
settings.SetOption("payment_details.enabled", "true");

To enable recognition of a specific type of payment details, set the corresponding session option:

// C++
settings->SetOption("payment_details.<type>.enabled", "true");

// Java
settings.SetOption("payment_details.<type>.enabled", "true");

The full list of supported types is the following: inn, kpp, rcbic, rus_bank_account, personal_account.

ATTENTION!

By default all supported types of payment details are enabled

For this engine, there are two supported capture modes: mobile and anywhere. The mobile mode mostly addresses handheld camera recognition in videostream, where the target details occupy the most part of the image, while anywhere mode is suitable for images of full page documents. By default the mobile option is enabled.

The Universal Pay technology used in Code Engine allows you to recognize objects in a payment document without specifying the type.

Supported objects:

  • payment barcode according to the GOST R 56042-2014 standard: QR Code, Aztec сode, Data Matrix;
  • bank card;
  • phone number;
  • bankcard number.

The client provides a sequence of images as input, and the session recognizes and puts into the result only those objects for which payment can be made (i.e., other two-dimensional barcodes not in accordance with GOST will be ignored).

One payment object is recognized within one session. After detecting a payment object in a sequence of frames, the session is terminated. One frame is enough for barcodes; for other types of objects, several frames are required.

Examples of the Universal Pay settings:

// C++
settings->SetOption("global.workflow", "universalPay");
settings->SetOption("barcode.enabled", "true");
settings->SetOption("barcode.QR_CODE.enabled", "true");
settings->SetOption("barcode.AZTEC.enabled", "true");
settings->SetOption("barcode.DATA_MATRIX.enabled", "true");
settings->SetOption("barcode.preset", "PAYMENT");
settings->SetOption("barcode.feedMode", "sequence");
settings->SetOption("bank_card.enabled", "true");
settings->SetOption("code_text_line.enabled", "true");
settings->SetOption("code_text_line.phone_number.enabled", "true");
settings->SetOption("code_text_line.card_number.enabled", "true");

// Java
settings.SetOption("global.workflow", "universalPay");
settings.SetOption("barcode.enabled", "true");
settings.SetOption("barcode.QR_CODE.enabled", "true");
settings.SetOption("barcode.AZTEC.enabled", "true");
settings.SetOption("barcode.DATA_MATRIX.enabled", "true");
settings.SetOption("barcode.preset", "PAYMENT");
settings.SetOption("barcode.feedMode", "sequence");
settings.SetOption("bank_card.enabled", "true");
settings.SetOption("code_text_line.enabled", "true");
settings.SetOption("code_text_line.phone_number.enabled", "true");
settings.SetOption("code_text_line.card_number.enabled", "true");

Session options

You can specify some configuration bundle options in runtime by using CodeEngineSessionSettings methods. 

To obtain all currently set option names and their values. use the following procedure:

// C++
for (auto it = settings->Begin(); it != settings->End(); ++it) {
    // it.GetKey() returns the option name
    // it.GetValue() returns the option value
}

// Java
for (StringsMapIterator it = settings.Begin();
     !it.Equals(settings.OptionsEnd());
     it.Advance()) {
    // it.GetKey() returns the option name
    // it.GetValue() returns the option value
}

To change option values, use the SetOption(…) method:

// C++
settings->SetOption("barcode.enabled", "true");
settings->SetOption("barcode.COMMON.enabled", "true");

// Java
settings.SetOption("barcode.enabled", "true");
settings.SetOption("barcode.COMMON.enabled", "true");

Option values are always represented as strings, so if you want to pass an integer or boolean convert it to string first.

Global options:

Option name Value type Default Description
global.enableMultiThreading “true” or “false” “true” Enables parallel execution of internal algorithms
global.rgbPixelFormat String of characters R, G, B, and A “RGB” for 3-channel images, “BGRA” for 4-channel images Sequence of color channels for session.ProcessSnapshot() method image interpretation
global.sessionTimeout Double value “0.0” for server bundles, “5.0” for mobile bundles Session timeout in seconds

Processing Feedback

Smart Code Engine SDK supports optional callbacks during document analysis and recognition process before the Process(…) method is finished. It allows the user to be more informed about the underlying recognition process and also helps create more interactive GUI.

To support callbacks you need to subclasses CodeEngineWorkflowFeedback and CodeEngineVisualizationFeedback classes and implement desirable callback methods:

// C++
class MyWorkflowFeedback : public se::code::CodeEngineWorkflowFeedback {
    public:
      virtual ~OptionalWorkflowFeedBack() override = default;
   
    public:
      virtual void ResultReceived(const se::code::CodeEngineResult& result) override { }
      virtual void SessionEnded() override { }
    };
   
    class MyVisualizationFeedback : public se::code::CodeEngineVisualizationFeedback {
    public:
      virtual ~OptionalVisualizationFeedBack() override = default;
   
    public:
      virtual void FeedbackReceived(
            const se::code::CodeEngineFeedbackContainer& /*feedback_container*/) override { }
    };

    // Java
 import com.smartengines.code.*;
 class MyWorkflowFeedback extends CodeEngineWorkflowFeedback {
   public void ResultReceived(CodeEngineResult result) { }
   public void SessionEnded() { }
 }

 class MyVisualizationFeedback extends CodeEngineVisualizationFeedback {
   public void FeedbackReceived(CodeEngineFeedbackContainer feedback_container) { }
 }

You also need to create an instance of MyWorkflowFeedback and MyVisualizationFeedback somewhere in the code and pass them when you spawn a session:

// C++
MyWorkflowFeedback my_workflow_feedback;
MyVisualizationFeedback my_visualization_feedback;
std::unique_ptr<se::code::CodeEngineSession> session(
    engine->SpawnSession(*settings, signature, &my_workflow_feedback, &my_visualization_feedback));

// Java
import com.smartengines.code.*;
MyWorkflowFeedback my_workflow_feedback = new MyWorkflowFeedback();
MyVisualizationFeedback my_visualization_feedback = new MyVisualizationFeedback();
CodeEngineSession session = engine.SpawnSession(settings, signature, my_workflow_feedback, 
my_visualization_feedback);

ATTENTION!

Do not delete CodeEngineWorkflowFeedback and CodeEngineVisualizationFeedback subclasses instances while CodeEngineSession is alive. We recommend placing them in the same scope

Java API Specifics

Smart Code Engine SDK implies Java API which is automatically generated from C++ interface using the SWIG tool.

Java interface is the same as C++ except for minor differences, please see the provided Java sample.

WARNING!

There are several drawbacks related to Java memory management that you need to consider

Object deallocation

Even though garbage collection is present and works, it’s strongly advised to call obj.delete() functions for our API objects manually because they are wrappers to the heap-allocated memory and their heap size is unknown to the garbage collector, which may result in delayed deletion of objects and thus in high overall memory consumption.

CodeEngine engine = CodeEngine.CreateFromEmbeddedBundle(true); // or any other object
// ...
engine.delete(); // forces and immediately guarantees wrapped C++ object deallocation

Feedback scope

ATTENTION!

When using optional callbacks by subclassing CodeEngineWorkflowFeedback or CodeEngineVisualizationFeedback please make sure that its instance has the same scope as CodeEngineSession. The reason for this is that our API does not own the pointer to the feedback instance which cause premature garbage collection resulting in crash

// Java
// BAD: may cause premature garbage collection of the feedback instance
class MyRecognizer {
    private CodeEngine engine;
    private CodeEngineSession session;
    private void InitializeSmartCodeEngine() {
        // ...
        session = engine.SpawnSession(settings, signature, new MyWorkflowFeedback(), new MyVisualizationFeedback());
        // feedback objects might be garbage collected there because session doesn't own it
    }
}

// Java
// GOOD: reporter have at least the scope of recognition session
class MyRecognizer {
    private CodeEngine engine;
    private CodeEngineSession session;
    private MyWorkflowFeedback workflow_feedback; //
    private MyVisualizationFeedback visualization_feedback; // feedbacks has session's scope
    private void InitializeSmartCodeEngine() {
        // ...
        workflow_feedback = new MyWorkflowFeedback();
        visualization_feedback = new MyVisualizationFeedback();
        session = engine.SpawnSession(settings, signature, workflow_feedback, visualization_feedback);
    }
}

FAQ

  • How can I test the system?

    You can install demo apps from Apple Store and Google Play from the links below:

    Google Play

    Apple Store

  • Which sample do I need?

    You need samples from the codeengine_sample* directory. 

  • How can I install and test your library?

    We provide the classic SDK as a library and its integration samples, you need to install one of the examples (you can find it in the /samples directory).

  • What file formats are supported?

    Supported formats:

    • jpeg, tiff (except TIFF_ZIP and TIFF_JPEG), png, bmp;
    • base64 (the formats from the paragraph above);
    • a file buffer with a preliminary indication of the color scheme, width\height\stride\number of channels.
  • What should I do if I need to recognize an image in PDF format?

    We do not support PDF and recommend rasterizing it (converting it to a supported format) on our side before recognition.

  • How can I update to the new version?

    You need to replace the libraries from /bin, wrappers (from /bindings respectively). You can find it in the provided SDK.

  • I have the SDK version for Windows, but our production system will use any OS from the Linux family. How do I run in a Linux-based docker container?

    Our SDK are platform-dependent, so please contact us at sales@smartengines.com or support@smartengines.com and we will provide you with the required SDK.

  • I have an SDK for Centos 7 and try to run it on Ubuntu/Debian/Alpine. I have a lot of “undefined symbol” errors.

    Please don’t run SDK for operating systems not intended for it and contact us at sales@smartengines.com or support@smartengines.com to provide you with the required SDK.

Common Errors

  1. Failed to verify static auth.

    Maybe, the signature you have specified is invalid.

  2. Failed to initialize CodeEngine: no single engine was provided.

    No engine is enabled in configuration or the bundle contains no one of the enabled engines (for example, you are trying to enable bank cards in a bundle which implies only barcodes).

  3. Failed to initialize CodeEngine: mismatching engine version in config.

     You cannot use a bundle version different from the library version. The files from the same SDK should be used.

  4. libCodeEngine.so:cannot open shared object file:no such file or directory.

    Many integrations require additional assembly of the wrapper to work with the C++ library. This error occurs when the wrapper cannot find the main library at the specified path. It must either be placed in the scope of the code through environment variables, or the wrapper must be compiled with the correct paths to the library.

  5. for PYTHON and PHP:libpython3.x.x:cannot open shared object file: No such file or directory.

    The module you are using is built for a different version of python, /samples/codeengine_sample_*/ contains a script for building the module on your side. Don’t forget that you must have the dev packages installed for your language.

Smart Code Engine: best practices

Mobile recognition

  1. If you know exactly which document the user should scan, draw a frame over a scanning screen in the UI. It helps the user to align the document with the camera, which could speed up the recognition process.
  2. We recommend adding an additional Scan button after opening the camera preview – this forces the user to align the document better and wait for the camera to focus before starting the recognition. It could speed up the capturing process, since the engine will process fewer empty frames.
  3. Use the fields IsAccepted() flag to draw attention to the fields which were recognized with low confidence.
  4. Initialize the recognition engine in an auxiliary background thread of your application before making the document scanning option available – that way the user will not have to wait for the engine to initialize when it’s time to scan the document.
  5. Do not store your personalized signature in a readable form (i.e. in resources), always store it either encoded in the application binary or load it remotely.
  6. For phone numbers and bank card numbers (do not confuse it with bank card recognition), an image with 2:1 proportions is required. For example, if the width of the image is 500px, then its height should be about 250px. This limitation is due to the fact that in these proportions the object occupies the maximum area in the image. This is exactly the scenario our search network is trained for.

Server-side recognition

  1. The recognition engine initialization could take some time, but remember that the engine is constant with regards to recognition. The engine can only be initialized once, after that multiple recognition sessions can be spawned from the same engine. The sessions could be spawned in parallel and they will not influence each other in any way.
  2. To get the maximum from our SDK, please consult the reference manual of the main C++ library interface – it is directly mapped to multiple library wrappers and contains a log of useful functionality.
  3. Remember that you can disable the internal multithreading of the recognition engine, and you can disable the lazy initialization, forcing the engine to load and initialize all resources at once.
  4. If you load the images from a file, make sure that the loaded files are placed in a high-performance disk device – on some PCs, the time required to load an image from a file could be longer than the recognition itself.
  5. Avoid using any pre-processing of the input images – our products best work with the images obtained directly from the capturing device (camera or scanner).

Send Request

Please fill out the form to get more information about the products,pricing and trial SDK for Android, iOS, Linux, Windows.

    Our customers

    Rosbank

    Rosbank has implemented an artificial intelligence solution for the paperwork

    Kaspi.kz

    Smart Engines rolls out its AI-driven software in super app Kaspi.kz

    Vodafone Qatar

    Vodafone Qatar uses Smart Engines’ technology for scanning ID card of Qatar

    2e Systems

    2e Systems use Smart Engines technologies in solutions for the airline industry.

    Send Request

    Please fill out the form to get more information about the products,pricing and trial SDK for Android, iOS, Linux, Windows.