Polimago Classification & Regression to verify the pictusres

Jesse · January 9, 2020, 5:29am

Dear sir or madam:

I want to verify the pictures which used to train to a classifier can be classified to correct class. But the Sample test result is differenent to my C# program test result.

My Experiment below

I create a new Polimago Classification & Regression to make a pass and fail classifier.
First, I load image and create the pass and fail classes.
Second, I do the Create Classifier to train the classifier.
Third, I save the Polimago classifier file.
Forth, I do the Sample Test to check the training result.

Test_Results

After using TeachBench to make classifier, I use the Visual Studio 2019 C# to Load the classifier I trained and Load the pictures which used to train classifier. But the test result is different to I use Teach Bech Sample Test result.

my Cap test

I also try CVB tutorial Handwriting classifier USDigits6000.pcc. The result is also different, show below.

TeachBench_TestDigits

My C# Code

using Stemmer.Cvb;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;

using Cvb;
using CVBImage = Cvb.Image;
using System.IO;


namespace WindowsFormsApp1
{
    public partial class Form1 : Form
    {
        private Cvb.SharedPolimagoClf theClassifier_;// The Polimago classifier to be used.

        string number = "1";
        string file_path_ = @"D:\test\1";

        string file_type_ = "*.bmp";

        int pass_count_ = 0;
        int fail_count_ = 0;
  
        public Form1()
        {
            InitializeComponent();

            LoadClassifier("USDigits6000.pcc");

            Classify();
        }

        private void LoadClassifier(string fileName)
        {
            try
            {
                Cvb.SharedPolimagoClf tmp = Cvb.Polimago.OpenClf(fileName);

                if (tmp != null)
                {
                    theClassifier_ = tmp;
                    Console.WriteLine("Loaded classifier '" + fileName + "'");
                }

            }
            catch (Exception e)
            {
                Console.WriteLine(e.ToString());
            }
        }

        private void Classify()
        {
            try
            {
                DirectoryInfo di = new DirectoryInfo(file_path_);

                foreach (var fi in di.EnumerateFiles(file_type_))
                {
                    Cvb.Image.IMG image = 0;

                    bool r = Cvb.Image.LoadImageFile(fi.FullName, out image);

                    Cvb.Image.IMG normalize_image = 0;
                    Cvb.Image.CreateNormalizedImage(image, CVBImage.TNormalizeMode.Normalize_MeanVariance, 128, 120, out normalize_image);

                    double quality;
                    string cls;

                    double[] confidences;

                    if (theClassifier_.Classify(normalize_image, 0, 0, out cls, out quality, out confidences))
                    {

                        if (cls == number)
                        {
                            pass_count_++;
                        }
                        else
                        {
                            string ng_message = fi.FullName + " ==> " + confidences[0].ToString() + " " + confidences[1].ToString();
                            Console.WriteLine(ng_message);
                            fail_count_++;
                        }
                    }
                }


                Console.WriteLine("Pass count : " + pass_count_.ToString());
                Console.WriteLine("Fail count : " + fail_count_.ToString());
   
            }
            catch (Exception e)
            {
                Console.WriteLine(e.ToString());
            }
        }


    }///////////////////////////////////////////
}

Frank · January 9, 2020, 11:49am

In regard to your code, it seems like you are performing the classification on normalized images. The samples test in the TeachBench is performed on the training set images (that are probably not normalized).

Jesse · January 10, 2020, 12:03am

Dear Frank:

Thank you for helping me.

But my training set images on TeachBench which already be normalized images.

TeachBench’s teach result by normalized images
Test_Results

Frank · January 10, 2020, 6:03am

This is a sample test on the training set and not a hold-out test, is this correct?

Jesse · January 10, 2020, 7:17am

Hi Frank:

THis result is Sample Test result on the training images set.

Test_Results

Phil · January 10, 2020, 8:09am

Hi Jesse,
I’m running your code and check if I have the same result and check if I find any possible explanation. The results should be the same as they use the same functionality under the hood.

Phil · January 10, 2020, 8:25am

In your code you use bmp files to load images. Where did you get the USDigits6000 images from? The Polimago folder has only a xsil stored together with the classifier file.

Jesse · January 10, 2020, 8:41am

Hi Phil:

I click 0~9 class and mouse right click to save the hand writing images.

I save all hand writing images to bmp images.

Thank you for helping me.

Frank · January 10, 2020, 9:32am

By “exporting” the images this way from the XSIL the images might be changed slightly as a result. This might explain the different results when doing the classification manually compared to the sample test.

That said, for this hand writing set, the images are not normalized within the training set (XSIL). Do you skip the normalization when doing your test in your application for this set? Or have you created a new XSIL based on normalized images?

Phil · January 10, 2020, 10:24am

I have tested your code with the digits and found the error. If you skip the normalization in line 69 and classify directly on the image the results match the ones obtained by the teachbench. I also reloaded all images into the teachbench after saving them as .bmp and the results stay the same.

Here is the code for the class 5 (since there is one easy to spot difference)

using Stemmer.Cvb;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;

using Cvb;
using CVBImage = Cvb.Image;
using System.IO;


namespace WindowsFormsApp1
{
    public partial class Form1 : Form
    {
        private Cvb.SharedPolimagoClf theClassifier_;// The Polimago classifier to be used.

        string number = "5";
        string file_path_ = @"D:\test\5";

        string file_type_ = "*.bmp";

        int pass_count_ = 0;
        int fail_count_ = 0;
  
        public Form1()
        {
            InitializeComponent();

            LoadClassifier("USDigits6000.pcc");

            Classify();
        }

        private void LoadClassifier(string fileName)
        {
            try
            {
                Cvb.SharedPolimagoClf tmp = Cvb.Polimago.OpenClf(fileName);

                if (tmp != null)
                {
                    theClassifier_ = tmp;
                    Console.WriteLine("Loaded classifier '" + fileName + "'");
                }

            }
            catch (Exception e)
            {
                Console.WriteLine(e.ToString());
            }
        }

        private void Classify()
        {
            try
            {
                DirectoryInfo di = new DirectoryInfo(file_path_);

                foreach (var fi in di.EnumerateFiles(file_type_))
                {
                    Cvb.Image.IMG image = 0;

                    bool r = Cvb.Image.LoadImageFile(fi.FullName, out image);

                    Cvb.Image.IMG normalize_image = 0;

                    double quality;
                    string cls;

                    double[] confidences;

                    if (theClassifier_.Classify(image, 0, 0, out cls, out quality, out confidences))
                    {

                        if (cls == number)
                        {
                            pass_count_++;
                        }
                        else
                        {
                            string ng_message = fi.FullName + " ==> " + confidences[0].ToString() + " " + confidences[1].ToString();
                            Console.WriteLine(ng_message);
                            fail_count_++;
                        }
                    }
                }


                Console.WriteLine("Pass count : " + pass_count_.ToString());
                Console.WriteLine("Fail count : " + fail_count_.ToString());
   
            }
            catch (Exception e)
            {
                Console.WriteLine(e.ToString());
            }
        }


    }///////////////////////////////////////////
}

The above code returns 0 errors which is the same result as in the teachbench, while your version was returning one false label.
I assume something similar is causing the problem with your original data.

Topic		Replies	Views
The polimago classify function is error Programming Questions csharp	2	721	December 19, 2018
Polimago search and classification - home-made training application C-style API	5	859	October 31, 2018
Polimago: Creating search and classification training sets from one data set Tools display , csharp , polimago	0	733	May 14, 2019
About Polimago Classification and Regression Project Testing Image Tools polimago	1	582	August 19, 2019
Visual Studio 2019 C# Polimago Project Programming Questions csharp	4	775	December 6, 2019

Polimago Classification & Regression to verify the pictusres

Related topics