hw brickbench: Benchmark test for NXT and EV3

Discussion specific to NXT-G, NXC, NBC, RobotC, Lejos, and more.

Re: hw brickbench: Benchmark test for NXT and EV3

Postby mightor » 22 Apr 2014, 07:04

Display output tests are impossible right now, since the functionality has not been implemented yet.

I implemented a native shellsort opcode for char arrays and it showed a 1000-fold speed up, when compared to running the same algorithm in ROBOTC. I haven't had time to make it work for other types yet.

= Xander
| My Blog: I'd Rather Be Building Robots (http://botbench.com)
| RobotC 3rd Party Driver Suite: (http://rdpartyrobotcdr.sourceforge.net)
| Some people, when confronted with a problem, think, "I know, I'll use threads,"
| and then two they hav erpoblesms. (@nedbat)
User avatar
mightor
Site Admin
 
Posts: 1079
Joined: 25 Sep 2010, 15:02
Location: Rotterdam, Netherlands

Re: hw brickbench: Benchmark test for NXT and EV3

Postby HaWe » 22 Apr 2014, 07:13

Xander,
thank you for this information,
but then please just post your code as-it-is, in the version by which you generated your published results!
The RobotC benchmarks can be updated at any time when your code will be available in an updated version, too.
User avatar
HaWe
 
Posts: 2500
Joined: 04 Nov 2014, 19:00

Re: hw brickbench: Benchmark test for NXT and EV3

Postby mightor » 24 Apr 2014, 06:23

Helmut,

I've added it to the samples folder that comes with ROBOTC 4.x. I am not able to access that stuff right now, but I will when I get home.

= Xander

Edit: see attached.
Attachments
benchmark.c
(9.56 KiB) Downloaded 638 times
| My Blog: I'd Rather Be Building Robots (http://botbench.com)
| RobotC 3rd Party Driver Suite: (http://rdpartyrobotcdr.sourceforge.net)
| Some people, when confronted with a problem, think, "I know, I'll use threads,"
| and then two they hav erpoblesms. (@nedbat)
User avatar
mightor
Site Admin
 
Posts: 1079
Joined: 25 Sep 2010, 15:02
Location: Rotterdam, Netherlands

Re: hw brickbench: Benchmark test for NXT and EV3

Postby HaWe » 26 Apr 2014, 09:08

thx for the code, Xander.
at the top it's still announced to be the 3.6 code - is that true?
or is it the 4.0 code?
User avatar
HaWe
 
Posts: 2500
Joined: 04 Nov 2014, 19:00

Re: hw brickbench: Benchmark test for NXT and EV3

Postby mightor » 26 Apr 2014, 16:04

Oh, I didn't change that part, this is what's shipped with 4.x. I'll fix the header in the benchmark when the next batch of functionality gets added to ROBOTC.

= Xander
| My Blog: I'd Rather Be Building Robots (http://botbench.com)
| RobotC 3rd Party Driver Suite: (http://rdpartyrobotcdr.sourceforge.net)
| Some people, when confronted with a problem, think, "I know, I'll use threads,"
| and then two they hav erpoblesms. (@nedbat)
User avatar
mightor
Site Admin
 
Posts: 1079
Joined: 25 Sep 2010, 15:02
Location: Rotterdam, Netherlands

Re: hw brickbench: Benchmark test for NXT and EV3

Postby gloomyandy » 13 May 2014, 13:14

Doc the source code for the leJOS version for the EV3 is here:
https://drive.google.com/file/d/0BwTg7x ... sp=sharing

and the NXT here:
https://drive.google.com/file/d/0BwTg7x ... sp=sharing

For some reason you do not seem to be showing the results for test 0-3 for the EV3 version of leJOS.
User avatar
gloomyandy
 
Posts: 323
Joined: 29 Sep 2010, 05:03

Re: hw brickbench: Benchmark test for NXT and EV3

Postby HaWe » 13 May 2014, 13:41

thank you very much, Andy!
I'll post it here in pure text for better reading:


hw benchmark for leJOS/Java for NXT:
Code: Select all
import javax.microedition.lcdui.Graphics;

import lejos.nxt.Button;
import lejos.nxt.LCD;


//hw brickbench
//benchmark test for NXT/EV3 and similar Micro Controllers
//PL: leJOS/NXT 0.9.1beta
//Autor: (C) Helmut Wunder 2013
// Ported to leJOS by Andy Shaw.
//freie Verwendung für private Zwecke
//für kommerzielle Zwecke nur nach Genehmigung durch den Autor.
//protected under the friendly Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License
//http://creativecommons.org/licenses/by-nc-sa/3.0/
//version 1.08

// PLEASE NOTE: This is not a good example of a Java program. It is a simple port of a C program. No
// attempt has been made to make use of Java features.

public class MBBenchmark
{
    static long[] runtime = new long[8];
    static int[] a = new int[500], b = new int[500], c = new int[500],
            t = new int[500];

    // --------------------------------------------
    // Mersenne Twister
    // --------------------------------------------

    static int[] y = new int[25];
    static int index = 25 + 1;
    static final int M = 7;
    static final int[] A = { 0, 0x8ebfd028 };

    static int randM()
    {

        if (index >= 25)
        {
            int k;
            if (index > 25)
            {
                int r = 9, s = 3402;
                for (k = 0; k < 25; ++k)
                {
                    r = 509845221 * r + 3;
                    s *= s + 1;
                    y[k] = s + (r >> 10);
                }
            }
            for (k = 0; k < 25 - M; ++k)
                y[k] = y[k + M] ^ (y[k] >> 1) ^ A[y[k] & 1];
            for (; k < 25; ++k)
                y[k] = y[k + (M - 25)] ^ (y[k] >> 1) ^ A[y[k] & 1];
            index = 0;
        }

        int e = y[index++];
        e ^= (e << 7) & 0x2b5b2500;
        e ^= (e << 15) & 0xdb8b0000;
        e ^= (e >> 16);
        return e;
    }

    // --------------------------------------------
    // Matrix Algebra
    // --------------------------------------------

    // matrix * matrix multiplication (matrix product)

    static void MatrixMatrixMult(int N, int M, int K, double A[][],
            double B[][], double C[][])
    {
        int i, j, s; // matrix A: N x M // B: M x K // C: N x K
        for (i = 0; i < N; ++i)
        {
            for (j = 0; j < K; ++j)
            {
                C[i][j] = 0;
                for (s = 0; s < M; ++s)
                {
                    C[i][j] = C[i][j] + A[i][s] * B[s][j];
                }
            }
        }
    }

    // matrix determinant

    static double MatrixDet(int N, double A[][])
    {
        int i, j, i_count, j_count, count = 0;
        double[][] Asub = new double[N - 1][N - 1];
        double det = 0;

        if (N == 1)
            return A[0][0];
        if (N == 2)
            return (A[0][0] * A[1][1] - A[0][1] * A[1][0]);

        for (count = 0; count < N; count++)
        {
            i_count = 0;
            for (i = 1; i < N; i++)
            {
                j_count = 0;
                for (j = 0; j < N; j++)
                {
                    if (j == count)
                        continue;
                    Asub[i_count][j_count] = A[i][j];
                    j_count++;
                }
                i_count++;
            }
            det += Math.pow(-1, count) * A[0][count] * MatrixDet(N - 1, Asub);
        }
        return det;
    }

    // --------------------------------------------
    // shell sort
    // --------------------------------------------

    static void shellsort(int size, int[] A)
    {
        int i, j, increment;
        int temp;
        increment = size / 2;

        while (increment > 0)
        {
            for (i = increment; i < size; i++)
            {
                j = i;
                temp = A[i];
                while ((j >= increment) && (A[j - increment] > temp))
                {
                    A[j] = A[j - increment];
                    j = j - increment;
                }
                A[j] = temp;
            }

            if (increment == 2)
                increment = 1;
            else
                increment = (int) (increment / 2.2);
        }
    }

    // --------------------------------------------
    // benchmark test procedures
    // --------------------------------------------

    static int test_Int_Add()
    {
        int i = 1, j = 11, k = 112, l = 1111, m = 11111, n = -1, o = -11, p = -111, q = -1112, r = -11111;
        int x;
        int s = 0;
        for (x = 0; x < 20000; ++x)
        {
            s += i;
            s += j;
            s += k;
            s += l;
            s += m;
            s += n;
            s += o;
            s += p;
            s += q;
            s += r;
        }
        return s;
    }

    static int test_Int_Mult()
    {
        int x, y;
        int s = 0;

        for (y = 0; y < 2000; ++y)
        {
            s = 1;
            for (x = 1; x <= 13; ++x)
            {
                s *= x;
            }
            for (x = 13; x > 0; --x)
            {
                s /= x;
            }
        }
        return s;
    }

    static float test_float_math()
    {

        float s = (float) Math.PI;
        int y;

        for (y = 0; y < 5000; ++y)
        {
            s *= Math.sqrt(s);
            s = (float) Math.sin(s);
            s *= Math.cos(10.5 * s);
            s = (float) Math.sqrt(s);
            s = (float) Math.exp(s);
        }
        return s;
    }

    static int test_rand_MT()
    {
        int s = 0;
        int y;

        for (y = 0; y < 5000; ++y)
        {
            s = randM() % 10001;
        }
        return s;
    }

    static float test_matrix_math()
    {
        int x;

        double[][] A = new double[2][2], B = new double[2][2], C = new double[2][2];
        double[][] O = new double[3][3], T = new double[3][3];
        int s;

        for (x = 0; x < 250; ++x)
        {

            A[0][0] = 1;
            A[0][1] = 3;
            A[1][0] = 2;
            A[1][1] = 4;

            B[0][0] = 10;
            B[0][1] = 30;
            B[1][0] = 20;
            B[1][1] = 40;

            MatrixMatrixMult(2, 2, 2, A, B, C);

            A[0][0] = 1;
            A[0][1] = 3;
            A[1][0] = 2;
            A[1][1] = 4;

            MatrixDet(2, A);

            O[0][0] = 1;
            O[0][1] = 4;
            O[0][2] = 7;
            O[1][0] = 2;
            O[1][1] = 5;
            O[1][2] = 8;
            O[2][0] = 3;
            O[2][1] = 6;
            O[2][2] = 9;

            MatrixDet(3, O);

        }

        s = (int) (O[0][0] * O[1][1] * O[2][2]);
        return s;
    }

    static int test_Sort()
    {
        int y;
        int[] t = new int[500];

        for (y = 0; y < 50; ++y)
        {
            System.arraycopy(a, 0, t, 0, t.length);
            shellsort(500, t);
            System.arraycopy(b, 0, t, 0, t.length);
            shellsort(500, t);
            System.arraycopy(c, 0, t, 0, t.length);
            shellsort(500, t);
        }

        return y;
    }

    static long test_TextOut()
    {

        int y;

        for (y = 0; y < 20; ++y)
        {
            LCD.clear();

            LCD.drawString("" + 0 + " " + 1000 + "  int_Add", 0, 0);
            LCD.drawString("" + 0 + " " + 1010 + "  int_Mult", 0, 1);
            LCD.drawString("" + 0 + " " + 1020 + "  float_op", 0, 2);
            LCD.drawString("" + 0 + " " + 1030 + "  randomize", 0, 3);
            LCD.drawString("" + 0 + " " + 1040 + "  matrix_algb", 0, 4);
            LCD.drawString("" + 0 + " " + 1050 + "  arr_sort", 0, 5);
            LCD.drawString("" + 0 + " " + 1060 + "  display_text", 0, 6);
            LCD.drawString("" + 0 + " " + 1070 + "  testing...", 0, 7);
        }
        return 99;
    }

    static int test_graphics()
    {
        int x = 1, y;
        Graphics g = new Graphics();;
        for (y = 0; y < 100; ++y)
        {
            g.clear();
            g.drawArc(50, 40, 10, 10, 0, 360);
            g.fillArc(30, 24, 10, 10, 0, 360);
            g.drawLine(10, 10, 60, 60);
            g.drawLine(50, 20, 90, 70);
            g.drawRect(20, 20, 40, 40);
            g.fillRect(65, 25, 20, 30);
            g.drawArc(100, 30, 15, 20, 0, 360);
        }
        return x;
    }


    static void displayValue(int testNo, long testTime, String testName)
    {
        LCD.drawInt(testNo, 2, 0, testNo);
        LCD.drawInt((int) testTime, 6, 3, testNo);
        LCD.drawString(testName, 10, testNo);
    }

    static void displayValues()
    {
        displayValue(0, runtime[0], "int_Add");
        displayValue(1, runtime[1], "int_Mult");
        displayValue(2, runtime[2], "float_op");
        displayValue(3, runtime[3], "randomize");
        displayValue(4, runtime[4], "matrix_algb");
        displayValue(5, runtime[5], "arr_sort");
        displayValue(6, runtime[6], "displ_txt");
        displayValue(7, runtime[7], "graphics");
    }

    static public void main(String[] args)
    {

        long time0;
        int x, y;
        float s;
        int i;

        LCD.clear();
        LCD.drawString("hw brickbench", 0, 1);
        LCD.drawString("(C)H.Wunder 2013", 0, 2);
        LCD.drawString("initializing...", 0, 4);

            for (y = 0; y < 500; ++y)
            {
                a[y] = randM() % 30000;
                b[y] = randM() % 30000;
                c[y] = randM() % 30000;
            }

            LCD.clear();

        time0 = System.currentTimeMillis();
        s = test_Int_Add();
        runtime[0] = System.currentTimeMillis() - time0;
        displayValue(0, runtime[0], "int_Add ");
        System.out.println("int_Add " + runtime[0]);

        time0 = System.currentTimeMillis();
        s = test_Int_Mult();
        runtime[1] = System.currentTimeMillis() - time0;
        displayValue(1, runtime[1], "int_Mult");
        System.out.println("int_Mult " + runtime[1]);

        time0 = System.currentTimeMillis();
        s = test_float_math();
        runtime[2] = System.currentTimeMillis() - time0;
        displayValue(2, runtime[2], "float_op");
        System.out.println("float_op " + runtime[2]);

        time0 = System.currentTimeMillis();
        s = test_rand_MT();
        runtime[3] = System.currentTimeMillis() - time0;
        displayValue(3, runtime[3], "randomize");
        System.out.println("randomize " + runtime[3]);

        time0 = System.currentTimeMillis();
        s = test_matrix_math();
        runtime[4] = System.currentTimeMillis() - time0;
        displayValue(4, runtime[4], "matrix_algb");
        System.out.println("matrx_algb " + runtime[4]);

        time0 = System.currentTimeMillis();
        s = test_Sort();
        runtime[5] = System.currentTimeMillis() - time0;
        displayValue(5, runtime[5], "arr_sort");
        System.out.println("arr_sort " + runtime[5]);

        time0 = System.currentTimeMillis();
        s = test_TextOut();
        runtime[6] = System.currentTimeMillis() - time0;
        System.out.println("text " + runtime[6]);
        LCD.clear();
        displayValues();

        time0 = System.currentTimeMillis();
        s = test_graphics();
        runtime[7] = System.currentTimeMillis() - time0;
        System.out.println("graphics " + runtime[7]);
        LCD.clear();
        displayValues();
        Button.waitForAnyPress();
        LCD.clear();
        y = 0;
        for (x = 0; x < 8; ++x)
        {
            y += runtime[x];
        }
        LCD.drawString("Total ms: " + y, 0, 0);
        LCD.drawString("Benchmark: " + (50000000 / y), 0, 1);
        System.out.println("total ms: " + y);
        System.out.println("benchmark: " + 50000000 / y);
        Button.waitForAnyPress();
        LCD.clear();
    }
}


########################################################################################################

hw benchmark for leJOS/Java for EV3:
Code: Select all
import lejos.hardware.Button;
import lejos.hardware.ev3.LocalEV3;
import lejos.hardware.lcd.GraphicsLCD;
import lejos.hardware.lcd.LCD;

//hw brickbench
//benchmark test for NXT/EV3 and similar Micro Controllers
//PL: leJOS/EV3 0.8.0-alpha
//Autor: (C) Helmut Wunder 2013
// Ported to leJOS by Andy Shaw.
//freie Verwendung für private Zwecke
//für kommerzielle Zwecke nur nach Genehmigung durch den Autor.
//protected under the friendly Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License
//http://creativecommons.org/licenses/by-nc-sa/3.0/
//version 1.08

// PLEASE NOTE: This is not a good example of a Java program. It is a simple port of a C program. No
// attempt has been made to make use of Java features.
public class MBBenchmark
{
    static long[] runtime = new long[8];
    static int[] a = new int[500], b = new int[500], c = new int[500],
            t = new int[500];

    // --------------------------------------------
    // Mersenne Twister
    // --------------------------------------------

    static int[] y = new int[25];
    static int index = 25 + 1;
    static final int M = 7;
    static final int[] A = { 0, 0x8ebfd028 };

    static int randM()
    {

        if (index >= 25)
        {
            int k;
            if (index > 25)
            {
                int r = 9, s = 3402;
                for (k = 0; k < 25; ++k)
                {
                    r = 509845221 * r + 3;
                    s *= s + 1;
                    y[k] = s + (r >> 10);
                }
            }
            for (k = 0; k < 25 - M; ++k)
                y[k] = y[k + M] ^ (y[k] >> 1) ^ A[y[k] & 1];
            for (; k < 25; ++k)
                y[k] = y[k + (M - 25)] ^ (y[k] >> 1) ^ A[y[k] & 1];
            index = 0;
        }

        int e = y[index++];
        e ^= (e << 7) & 0x2b5b2500;
        e ^= (e << 15) & 0xdb8b0000;
        e ^= (e >> 16);
        return e;
    }

    // --------------------------------------------
    // Matrix Algebra
    // --------------------------------------------

    // matrix * matrix multiplication (matrix product)

    static void MatrixMatrixMult(int N, int M, int K, double A[][],
            double B[][], double C[][])
    {
        int i, j, s; // matrix A: N x M // B: M x K // C: N x K
        for (i = 0; i < N; ++i)
        {
            for (j = 0; j < K; ++j)
            {
                C[i][j] = 0;
                for (s = 0; s < M; ++s)
                {
                    C[i][j] = C[i][j] + A[i][s] * B[s][j];
                }
            }
        }
    }

    // matrix determinant

    static double MatrixDet(int N, double A[][])
    {
        int i, j, i_count, j_count, count = 0;
        double[][] Asub = new double[N - 1][N - 1];
        double det = 0;

        if (N == 1)
            return A[0][0];
        if (N == 2)
            return (A[0][0] * A[1][1] - A[0][1] * A[1][0]);

        for (count = 0; count < N; count++)
        {
            i_count = 0;
            for (i = 1; i < N; i++)
            {
                j_count = 0;
                for (j = 0; j < N; j++)
                {
                    if (j == count)
                        continue;
                    Asub[i_count][j_count] = A[i][j];
                    j_count++;
                }
                i_count++;
            }
            det += Math.pow(-1, count) * A[0][count] * MatrixDet(N - 1, Asub);
        }
        return det;
    }

    // --------------------------------------------
    // shell sort
    // --------------------------------------------

    static void shellsort(int size, int[] A)
    {
        int i, j, increment;
        int temp;
        increment = size / 2;

        while (increment > 0)
        {
            for (i = increment; i < size; i++)
            {
                j = i;
                temp = A[i];
                while ((j >= increment) && (A[j - increment] > temp))
                {
                    A[j] = A[j - increment];
                    j = j - increment;
                }
                A[j] = temp;
            }

            if (increment == 2)
                increment = 1;
            else
                increment = (int) (increment / 2.2);
        }
    }

    // --------------------------------------------
    // benchmark test procedures
    // --------------------------------------------

    static int test_Int_Add()
    {
        int i = 1, j = 11, k = 112, l = 1111, m = 11111, n = -1, o = -11, p = -111, q = -1112, r = -11111;
        int x;
        int s = 0;
        for (x = 0; x < 20000; ++x)
        {
            s += i;
            s += j;
            s += k;
            s += l;
            s += m;
            s += n;
            s += o;
            s += p;
            s += q;
            s += r;
        }
        return s;
    }

    static int test_Int_Mult()
    {
        int x, y;
        int s = 0;

        for (y = 0; y < 2000; ++y)
        {
            s = 1;
            for (x = 1; x <= 13; ++x)
            {
                s *= x;
            }
            for (x = 13; x > 0; --x)
            {
                s /= x;
            }
        }
        return s;
    }

    static float test_float_math()
    {

        float s = (float) Math.PI;
        int y;

        for (y = 0; y < 5000; ++y)
        {
            s *= Math.sqrt(s);
            s = (float) Math.sin(s);
            s *= Math.cos(10.5 * s);
            s = (float) Math.sqrt(s);
            s = (float) Math.exp(s);
        }
        return s;
    }

    static int test_rand_MT()
    {
        int s = 0;
        int y;

        for (y = 0; y < 5000; ++y)
        {
            s = randM() % 10001;
        }
        return s;
    }

    static float test_matrix_math()
    {
        int x;

        double[][] A = new double[2][2], B = new double[2][2], C = new double[2][2];
        double[][] O = new double[3][3], T = new double[3][3];
        int s;

        for (x = 0; x < 250; ++x)
        {

            A[0][0] = 1;
            A[0][1] = 3;
            A[1][0] = 2;
            A[1][1] = 4;

            B[0][0] = 10;
            B[0][1] = 30;
            B[1][0] = 20;
            B[1][1] = 40;

            MatrixMatrixMult(2, 2, 2, A, B, C);

            A[0][0] = 1;
            A[0][1] = 3;
            A[1][0] = 2;
            A[1][1] = 4;

            MatrixDet(2, A);

            O[0][0] = 1;
            O[0][1] = 4;
            O[0][2] = 7;
            O[1][0] = 2;
            O[1][1] = 5;
            O[1][2] = 8;
            O[2][0] = 3;
            O[2][1] = 6;
            O[2][2] = 9;

            MatrixDet(3, O);

        }

        s = (int) (O[0][0] * O[1][1] * O[2][2]);
        return s;
    }

    static int test_Sort()
    {
        int y;
        int[] t = new int[500];

        for (y = 0; y < 50; ++y)
        {
            System.arraycopy(a, 0, t, 0, t.length);
            shellsort(500, t);
            System.arraycopy(b, 0, t, 0, t.length);
            shellsort(500, t);
            System.arraycopy(c, 0, t, 0, t.length);
            shellsort(500, t);
        }

        return y;
    }

    static long test_TextOut()
    {

        int y;

        for (y = 0; y < 20; ++y)
        {
            LCD.clear();

            LCD.drawString("" + 0 + " " + 1000 + "  int_Add", 0, 0);
            LCD.drawString("" + 0 + " " + 1010 + "  int_Mult", 0, 1);
            LCD.drawString("" + 0 + " " + 1020 + "  float_op", 0, 2);
            LCD.drawString("" + 0 + " " + 1030 + "  randomize", 0, 3);
            LCD.drawString("" + 0 + " " + 1040 + "  matrix_algb", 0, 4);
            LCD.drawString("" + 0 + " " + 1050 + "  arr_sort", 0, 5);
            LCD.drawString("" + 0 + " " + 1060 + "  display_text", 0, 6);
            LCD.drawString("" + 0 + " " + 1070 + "  testing...", 0, 7);
        }
        return 99;
    }

    static int test_graphics()
    {
        int x = 1, y;
        GraphicsLCD g = LocalEV3.get().getGraphicsLCD();
        for (y = 0; y < 100; ++y)
        {
            g.clear();
            g.drawArc(50, 40, 10, 10, 0, 360);
            g.fillArc(30, 24, 10, 10, 0, 360);
            g.drawLine(10, 10, 60, 60);
            g.drawLine(50, 20, 90, 70);
            g.drawRect(20, 20, 40, 40);
            g.fillRect(65, 25, 20, 30);
            g.drawArc(100, 30, 15, 20, 0, 360);
        }
        return x;
    }


    static void displayValue(int testNo, long testTime, String testName)
    {
        LCD.drawInt(testNo, 2, 0, testNo);
        LCD.drawInt((int) testTime, 6, 3, testNo);
        LCD.drawString(testName, 10, testNo);
    }

    static void displayValues()
    {
        displayValue(0, runtime[0], "int_Add");
        displayValue(1, runtime[1], "int_Mult");
        displayValue(2, runtime[2], "float_op");
        displayValue(3, runtime[3], "randomize");
        displayValue(4, runtime[4], "matrix_algb");
        displayValue(5, runtime[5], "arr_sort");
        displayValue(6, runtime[6], "displ_txt");
        displayValue(7, runtime[7], "graphics");
    }

    static public void main(String[] args)
    {

        long time0;
        int x, y;
        float s;
        int i;

        LCD.clear();
        LCD.drawString("hw brickbench", 0, 1);
        LCD.drawString("(C)H.Wunder 2013", 0, 2);
        LCD.drawString("initializing...", 0, 4);

        // Java on the EV3 uses a JIT compiler, run the test multiple times to
        // allow the results to be stable
        for (int ii = 0; ii < 20; ii++)
        {

            for (y = 0; y < 500; ++y)
            {
                a[y] = randM() % 30000;
                b[y] = randM() % 30000;
                c[y] = randM() % 30000;
            }

            LCD.clear();

            time0 = System.currentTimeMillis();
            s = test_Int_Add();
            runtime[0] = System.currentTimeMillis() - time0;
            displayValue(0, runtime[0], "int_Add ");
            System.out.println("int_Add " + runtime[0]);

            time0 = System.currentTimeMillis();
            s = test_Int_Mult();
            runtime[1] = System.currentTimeMillis() - time0;
            displayValue(1, runtime[1], "int_Mult");
            System.out.println("int_Mult " + runtime[1]);

            time0 = System.currentTimeMillis();
            s = test_float_math();
            runtime[2] = System.currentTimeMillis() - time0;
            displayValue(2, runtime[2], "float_op");
            System.out.println("float_op " + runtime[2]);

            time0 = System.currentTimeMillis();
            s = test_rand_MT();
            runtime[3] = System.currentTimeMillis() - time0;
            displayValue(3, runtime[3], "randomize");
            System.out.println("randomize " + runtime[3]);

            time0 = System.currentTimeMillis();
            s = test_matrix_math();
            runtime[4] = System.currentTimeMillis() - time0;
            displayValue(4, runtime[4], "matrix_algb");
            System.out.println("matrx_algb " + runtime[4]);

            time0 = System.currentTimeMillis();
            s = test_Sort();
            runtime[5] = System.currentTimeMillis() - time0;
            displayValue(5, runtime[5], "arr_sort");
            System.out.println("arr_sort " + runtime[5]);

            time0 = System.currentTimeMillis();
            s = test_TextOut();
            runtime[6] = System.currentTimeMillis() - time0;
            System.out.println("text " + runtime[6]);
            LCD.clear();
            displayValues();

            time0 = System.currentTimeMillis();
            s = test_graphics();
            runtime[7] = System.currentTimeMillis() - time0;
            System.out.println("graphics " + runtime[7]);
            LCD.clear();
            displayValues();
        }
        Button.waitForAnyPress();
        LCD.clear();
        y = 0;
        for (x = 0; x < 8; ++x)
        {
            y += runtime[x];
        }
        LCD.drawString("Total ms: " + y, 0, 0);
        LCD.drawString("Benchmark: " + (50000000 / y), 0, 1);
        System.out.println("total ms: " + y);
        System.out.println("benchmark: " + 50000000 / y);
        Button.waitForAnyPress();
        LCD.clear();
    }

}


########################################################################################################

BTW: Is meanwhile everything finished for the EV3, too, e.g. display benchmarks (or will be still anything changed here), and has the float vs. double thing been fixed intermediately?
What about a java-fw-included sort algorithm?

ps
for the NXT code you wrote " leJOS/EV3 0.8.0-alpha" - but it's probably 0.9.anything, pls CMIIW...?

pps
another question re. the Java JIT compiler:
have the benchs been run once or several times before you took the time?
Last edited by HaWe on 13 May 2014, 14:14, edited 1 time in total.
User avatar
HaWe
 
Posts: 2500
Joined: 04 Nov 2014, 19:00

Re: hw brickbench: Benchmark test for NXT and EV3

Postby gloomyandy » 13 May 2014, 14:08

All of the results are as posted. Things may improve over time, but that is how they are at the moment.

The display results could be improved but it is not a priority for us at the moment. The double/float issue is just a difference between Java and the other systems. Java uses double for most floating point operations (in particular the sin/cos/sqrt/pow etc. functions return doubles), the current benchmark uses floats. This means that the results for the floating point tests are comparing Java using doubles against most of the other systems using floats. On most modern systems this is not a big deal, on the EV3 it may be. On the NXT it certainly is (you can see this in the results). But that is how things are if you use standard Java. I've not made any attempt to use Java features for things like array sorting etc.

Ah yes the version number for the NXT should be 0.9.1beta.

Is there a reason for the question marks against the NXT results?
User avatar
gloomyandy
 
Posts: 323
Joined: 29 Sep 2010, 05:03

Re: hw brickbench: Benchmark test for NXT and EV3

Postby HaWe » 13 May 2014, 14:13

version thing fixed,
"?" removed (I simply missed it).

Thanks again!

Ergebnisse / Results (Version 1.17)
Code: Select all
HaWe brickbench © test (time in ms)          NXT         NXT     NXT        NXT           EV3       EV3      EV3      EV3       Arduino     Arduino
progr.language, API                          NXC       RobotC   leJOS   Toppers C/C++  BCC/gpp C   leJOS   RobotC   C#/Mono     Mega2560      Due   
Firmware / vers.                         JH's EFW       3.6     0.9.1   nxtOSEK BIOS    LegoFW**)  loop(?) 4.0.x  loop#3(-#1)  Sketch 1.5 Sketch 1.5
===================================================================================================================================================
 0 integer add/subtr                        3353       1561      727        29            3      4(-xxx)?    336       2(-15)        129         8
 1 integer multiply/division                6441       1150      463       182           13     11(-xxx)?    589      26(-35)       1219         7
 2 float operations                         1716        413     2543       320           50     45(-xxx)?    226     155(-212)       887        83
 3 Mersenne Tw. PRNG (&,^,>>,<<)            5372       1838       73        55            4      7(-xxx)?   1204      13(-19)        315         8
---------------------------------------------------------------------------------------------------------------------------------------------------
 4 matrix algebra (prod/determ.)            3356        585       68        39           10     60(-xxx)?    325      43(-229)       243        23
 5 int array sort [500]                      332      95879    18756       491          113    139(-xxx)?  20932     172(-299)      1366       171
---------------------------------------------------------------------------------------------------------------------------------------------------
 6 display text     (N/A: penalty=1000) *)   781         32      26        17           28     218(-xxx)?  +1000+?   268(-395)     92016     35347
 7 display graphics (N/A: penalty=1000) *)   384       ? 95?   11480      182           51     371(-xxx)?  +1000+?   185(-475)?   224137    138600
===================================================================================================================================================
execution time                            21,735    101,553   35,117     1,362          272    855(-xxx)? <23600+?      864       316153    174247
===================================================================================================================================================
10 runt. code mem. (score= ln^3)        120k(110)  112k(105)      ?   224k(158)   <64M(1340)       ?          ?     42M(1210)   248k(167) 512k(240)           
11 variable mem.   (score= ln^3)          32k(42)    15k(20)      ?     64k(72)   <64M(1340)       ?     15k(20)    42M(1210)     8k  (8)  96k (95)
12 BT raw master+slave   (score 10x)       4 (40)     2 (20)      ?      2 (20)           0        ?         ?            0            0         0
13   RS485/i2c raw master+slv.(score 10x) 32(320)     2 (20)      ?     32(320)           0        ?         ?            0     110(1100) 220(2200)   
14   BT, +MC I/O rem.protocol (score 50x)      0          0       ?          0            0        ?         ?            0            0         0             
15   RS485, +MC I/O rem.prot. (score 50x)      0          0       ?          0            0        ?         ?            0            0         0
16   USB MC Daisy-Chain       (score 50x)      0          0       ?          0            0        ?         ?            0            0         0
17   WiFi MC I/O rem.protocol (score 50x)      0          0       ?          0            0        ?         ?            0            0         0
18 feat. double=64bit float   (score 100)      0          0      100         0          100       100        0          100            0       100
19 feat.>=4D array/matrix op. (score 100)    100          0       ?          0          100        ?         0          100            0         0
20 Recursions      (score 100)                 0          0       ?        100          100        ?         0          100          100       100
21 Multitthreading (score 100)               100        100      100       100         (100)      100      100          100         (100)      100
22 API complete (sensor+motor+screen)         +          +        +        (+)          ---        +         -          (-)           +         +
23 small IDE made-to-measure  (score 500)    500        500       0          0          500        0       500            0          500       500
===================================================================================================================================================
Brick platform                              NXT         NXT     NXT        NXT         EV3       EV3        EV3         EV3     Ardu.2560   Ardu.Due
progr.language, Firmware                    NXC       RobotC   leJOS     nxtOSEK   BCC/gpp C    leJOS     RobotC     C#/Mono      C/C++       C/C++             
===================================================================================================================================================
speed bench score (50,000,000/exec.time)   2,300      0,485    1,423    36,711      183,823    58,479    >2,100?      57,870       0,158     0,287
environnment bench score                   1,312      0,672   ?0,200     0,790    [---3,080]   ?0,200     0,627?       2,820       2,119     2,487
---------------------------------------------------------------------------------------------------------------------------------------------------
brickbench system benchmark                3,612      1,157?   1,623 ?  37.501  [---186,883]   58,579?   >2,700?      60,690       3,359     2,774
===================================================================================================================================================


link to the most updated charts:
http://www.mindstormsforum.de/viewtopic ... 095#p64772
Last edited by HaWe on 03 Jan 2015, 14:08, edited 2 times in total.
User avatar
HaWe
 
Posts: 2500
Joined: 04 Nov 2014, 19:00

Re: hw brickbench: Benchmark test for NXT and EV3

Postby HaWe » 23 Nov 2014, 13:42

still many Lejos and RobotC issues and data missing, but Monobrick has been finished yet by adding display classes (thanks to Vlad Ruzov!).

latest most updated benchmark overview table can be found here:
http://www.mindstormsforum.de/viewtopic ... 095#p64772
Last edited by HaWe on 04 Jan 2015, 13:27, edited 1 time in total.
User avatar
HaWe
 
Posts: 2500
Joined: 04 Nov 2014, 19:00

PreviousNext

Return to Mindstorms Software

Who is online

Users browsing this forum: No registered users and 4 guests

cron