1. Tested compressors.

ACB (powerful universal archiver without any multimedia stuff):
ACB ver.2.00c (Shareware) Copyright(C) 1994-97 George Buyanovsky 04.25.1997
ARHANGEL:
ARchive HANdler by GEorge Lyapko v1.38 Shareware (c) 1997-1998
BMF:
TGA,BMP,RAS,GIF,RAW to BMF file convertor, v.1.1 (C) 1998-1999 by D.Shkarin
BOA:
--- BOA Constrictor Archiver v0.58b -----------------------------------------
--- Copyright (c) 1997-98 Ian Sutton. All Rights Reserved.
ERI:
Eri32 4.1fre (c) RAO Inc. 1991-99                E-mail: Ratush@srsc-gw.sscc.ru
ESP:
ESP v1.92 Extension-Sort Packer (C) 1996-97. GyikSoft
LOCO-I/JPEG-LS:
LOCO-I/JPEG-LS Compressor - V.0.90 *** FOR TESTING AND EVALUATION ONLY ***
HEWLETT-PACKARD LABORATORIES - Compiled Nov 20 1997 12:40:51
(c) COPYRIGHT HEWLETT-PACKARD COMPANY, 1995-1997.
PPMD:
Fast PPM compressor for textual data, variant C, Aug 22 1999
written an distributed to public domain by Dmitry Shkarin (shkarin@arstel.ru)
RAR:
RAR 2.50 Copyright (c) 1993-99 Eugene Roshal 24 March 1999
RKIM:
Standalone RKIVE v2.00 image compressor (v1.03).
Copyright (c) 1999 Malcolm Taylor.
RKUC:
Standalone RKIVE v2.00 universal compressor (v1.04).
Copyright (c) 1999 Malcolm Taylor.
SEA:
SEA v1.3. copyright (c) 1995-97 Photodex corporation. All rights reserved
UHARC:
UHARC 0.2 ----- high compression multimedia archiver ----- BETA version
Copyright (c) 1997 by Uwe Herklotz All rights reserved 21 Dec 1997
UHIC:
UHIC 1.0 ---- high performance 24bit image compressor ---- BETA version
Copyright (c) 1998 by Uwe Herklotz All rights reserved 20 Aug 1998
  If You know good image compressor not included in this tests, send me mail, please.

2. Comparision terms.

Test image suite is well-known Waterloo Bragzone test image set. Paletted image set was done by Image Alchemy v1.9.1. Old JPEG image set plus well-known LENA test image were used for greyscale compression comparision. CALIC, HBB, TMW and D&Y results were taken from G.Deng and H.Ye "Lossless image compression using adaptive predictor combination, symbol mapping and context filtering". New JPEG image set was used for mixed image type compression comparision. Cheng Jiun Yuan`s CALIC implementation(I have no SparcStation for original implementation ;-) was used for this test. LOCO-A results were taken from M.Weinberger, G.Seroussi, G.Sapiro "The LOCO-I Lossless Image Compression Algorithm: Principles and Standardization into JPEG-LS".
All tested programs were runned on PII-233 (overclocked up to 291 MHz) with 32 MB memory in DOS session under Win98(FAT32).
Time was measured by system timer (with 1/18 sec. accuracy). Overall score is calculated by adding compress time, extract time, and time it would take to transfer the compressed file over a 28,800bps network.
Compressed images sizes were taken from archive listing, when it is possible, so summ of compressed images sizes may be not equal to archive size.

3. Comparision results.

Table 1. Paletted set.
Command line Total size
4,161,360		 Encoding
time, sec		 Decoding
time, sec		 Overall
score, sec		 CLEGG
717,106		 FRYMIRE
1,236,176		 LENA
262,930		 MONARCH
394,002		 PEPPERS
262,930		 SAIL
394,002		 SERRANO
500,212		 TULIPS
394,002
ACB u !a *.tga 1,721,200 231.6 231.1 940.8 371,701 280,492 158,221 192,173 119,687 253,785 137,244 207,897
ARHANGEL a -2 -mo2 !a *.tga 1,634,867 24.4 24.7 503.2 358,152 276,260 146,059 177,413 112,326 238,379 134,882 191,121
BMF *.tga 1,670,256 5.6 1.8 471.4 358,240 289,512 146,716 183,704 116,592 238,180 143,908 193,404
BMF -s *.tga 1,533,496 10.4 8.7 445.1 315,176 270,976 131,232 171,008 109,300 226,548 133,588 175,668
BOA -a -m15 !a *.tga 1,638,168 84.6 84.7 624.3 358,981 273,654 147,332 178,474 114,647 236,004 137,852 190,969
ESP !a *.tga 2,139,664 5.3 0.7 600.4 452,696 364,024 186,497 242,384 155,704 295,821 179,751 261,623
PPMD e -o2 *.tga 1,612,224 7.0 8.5 463.3 351,438 273,242 145,961 175,128 112,064 234,260 132,478 187,653
RAR a -m5 -mde !a *.tga 2,089,550 15.8 1.7 597.9 437,388 354,441 184,983 236,985 152,797 291,549 175,971 255,081
RKUC c -x -o3 %1.tga(1) 1,642,936 29.0 33.1 518.5 361,320 274,284 148,860 178,696 115,368 236,368 136,396 191,644
SEA /PNG:100 *.tga *.png 2,144,973 4.3 1.3 601.4 452,172 367,421 187,340 242,968 155,939 296,765 179,484 262,884
UHARC a -m3 -md1024 !a *.tga 1,631,749 70.8 13.3 537.4 350,259 281,349 143,741 179,861 113,575 236,114 137,573 189,082
Table 2. Grey set 1.
Command line Total size
786,648		 Encoding
time, sec		 Decoding
time, sec		 Overall
score, sec		 BIRD
65,554		 BRIDGE
65,554		 CAMERA
65,554		 CIRCLES
65,554		 CROSSES
65,554		 GOLDHILL
65,554		 HORIZ
65,554		 LENA
65,554		 MONTAGE
65,554		 SLOPE
65,554		 SQUARES
65,554		 TEXT
65,554
ACB u !a *.tga 254,941 20.3 20.1 111.2 32,740 56,830 40,664 925 1,161 51,038 195 50,491 28,631 13,157 173 1,473
ARHANGEL a -2 -mm !a *.tga 262,324 4.7 4.5 82.1 32,527 51,841 39,151 1,071 1,117 47,696 153 45,484 26,345 12,455 502 3,578
BMF -f *.tga 225,164 2.3 0.8 65.6 27,736 47,216 35,436 672 632 43,284 192 36,216 22,180 9,836 136 1,588
BMF -f -s *.tga 215,880 5.4 3.1 68.5 26,664 46,044 33,752 624 544 42,064 120 34,708 21,256 8,368 108 1,628
BOA -a -m15 !a *.tga 272,088 8.3 8.5 92.4 32,250 54,477 40,316 750 1,105 49,256 83 49,128 28,470 13,020 72 2,783
ESP /MM /ML7 !a *.tga 262,960 1.0 0.2 74.2 34,545 50,719 39,682 1,177 1,692 46,662 243 46,168 26,724 11,533 327 1,925
LOCOE -N %1.pgm -o%1.jls(1) 246,358 2.5 2.6 73.5 28,436 47,435 35,338 1,250 3,158 43,264 768 37,530 22,307 12,872 632 13,368
RAR a -m5 -mm -mde !a *.tga 298,724 1.9 0.7 85.6 35,798 60,372 44,268 768 1,176 52,317 148 50,761 33,789 17,354 105 1,546
RKIM cx %1.tga(1) 236,304 11.4 6.5 83.5 27,558 46,942 34,758 714 2,198 43,038 182 36,278 21,302 9,438 86 13,810
SEA /PNG:100 *.tga *.png 272,826 0.8 0.3 76.9 36,683 52,397 42,880 1,656 1,924 49,238 655 45,368 27,014 12,185 589 2,237
UHARC a -m3 -md1024 !a *.tga 227,239 10.1 1.8 75.0 32,277 50,624 37,717 821 1,198 46,642 112 44,221 4,384 7,904 80 1,024
Table 3. Grey set 2.
Command line Total size
3,212,002		 Encoding
time, sec		 Decoding
time, sec		 Overall
score, sec		 BARB
262,162		 BOAT
262,162		 FRANCE
334,098		 FROG
309,276		 GOLD
262,162		 LENA
262,162		 LIBRARY
164,114		 MANDRILL
262,162		 MOINTAIN
307,218		 PEPPERS
262,162		 WASHSAT
262,162		 ZELDA
262,162
ACB u !a *.tga 1,831,309 216.6 211.6 936.9 205,960 176,839 13,992 121,484 190,139 181,987 87,050 228,964 187,283 187,156 72,792 177,663
ARHANGEL a -2 -mm !a *.tga 1,723,472 25.8 25.5 530.0 189,071 162,398 25,890 119,770 172,576 162,479 101,427 211,107 185,108 164,746 72,317 156,177
BMF -f *.tga 1,518,540 5.9 2.5 430.2 146,792 136,912 13,364 127,012 152,336 134,676 88,580 193,844 183,084 143,688 71,724 126,528
BMF -f -s *.tga 1,447,028 17.0 13.4 432.4 133,452 129,480 12,772 120,364 147,628 129,980 81,248 188,468 172,852 138,864 70,932 120,988
BOA -a -m15 !a *.tga 1,761,750 75.0 77.0 641.4 198,841 168,991 16,763 120,437 177,021 169,383 91,786 212,219 183,838 172,355 86,194 163,542
ESP /MM /ML7 !a *.tga 1,889,403 6.0 0.7 531.5 198,762 169,228 18,141 193,512 170,886 165,826 105,370 208,440 247,066 168,127 85,075 156,677
LOCOE -N %1.pgm -o%1.jls(1) 1,842,617 4.5 4.6 520.9 155,100 139,256 58,792 233,831 154,391 139,057 104,140 197,804 246,604 147,086 135,309 131,247
RAR a -m5 -mm -mde !a *.tga 1,873,335 12.6 1.9 534.9 219,598 182,081 14,318 146,030 182,251 167,061 95,800 236,177 201,678 184,144 87,101 156,572
RKIM cx %1.tga(1) 1,760,720 43.1 22.6 554.8 139,466 133,094 40,410 231,050 150,262 133,238 105,394 191,674 242,658 141,982 128,210 123,282
SEA /PNG:100 *.tga *.png 1,929,904 3.2 0.8 540.1 186,167 170,991 16,122 207,555 182,343 170,141 98,475 214,236 252,966 179,607 92,101 159,200
UHARC a -m3 -md1024 !a *.tga 1,660,888 52.7 10.3 524.4 182,278 158,679 12,572 124,463 167,526 157,638 87,566 205,477 179,665 160,169 71,218 153,394
Table 4. Color set.
Command line Total size
12,465,360		 Encoding
time, sec		 Decoding
time, sec		 Overall
score, sec		 CLEGG
2,148,978		 FRYMIRE
3,706,188		 LENA
786,450		 MONARCH
1,179,666		 PEPPERS
786,450		 SAIL
1,179,666		 SERRANO
1,498,296		 TULIPS
1,179,666
ACB u !a *.tga 4,564,162 468.3 470.4 2206.5 743,408 194,224 618,602 692,910 600,267 772,176 80,455 862,120
ARHANGEL a -2 -mm !a *.tga 2,997,537 56.5 42.2 931.3 388,267 189,344 440,055 462,304 337,071 566,318 88,898 525,005
BMF -f *.tga 2,920,672 19.5 6.9 837.7 434,116 199,064 422,860 436,924 327,052 527,620 83,100 489,936
BMF -f -q8 *.tga 2,914,824 60.9 6.7 877.3 434,116 199,064 419,636 434,564 327,052 527,356 83,100 489,936
BMF -f -s *.tga 2,723,804 48.8 33.7 839.1 382,824 194,256 403,076 403,776 301,132 492,604 79,284 466,852
BMF -f -s -q8 *.tga 2,713,720 144.4 36.5 934.7 382,824 194,256 402,300 403,776 301,132 492,604 79,284 457,544
BOA -a -m15 !a *.tga 3,619,836 232.1 230.4 1468.0 399,967 308,387 498,699 550,081 389,561 679,988 130,328 662,554
ERI c *.bmp 3,350,604 89.5 16.5 1036.7 406,291 394,336 483,475 470,094 348,162 561,537 156,876 529,833
ESP /MM2 /ML7 !a *.tga 4,037,920 25.7 1.7 1149.0 485,107 520,575 506,031 592,994 404,787 642,331 201,491 683,975
LOCOE -N %1.pnm -o%1.jls(1) 4,651,884 8.3 10.1 1310.6 653,299 935,285 445,799 555,012 385,047 767,374 293,532 616,536
RAR a -m5 -mm -mde !a *.tga 4,144,586 29.9 7.0 1188.2 989,474 238,432 493,686 600,985 406,621 648,369 103,661 663,003
RKIM cx %1.tga(1) 3,855,368 256.7 86.2 1413.8 939,390 ? 621,514 411,410 410,014 299,038 507,642 204,402 461,958
SEA /PNG:100 *.tga *.png 4,274,996 14.6 2.3 1204.4 517,241 371,296 529,463 669,246 476,991 810,031 154,038 746,690
UHARC a -m3 -md1024 !a *.tga 3,314,092 172.6 29.0 1122.2 371,880 216,396 477,916 533,023 377,738 626,876 91,061 619,002
UHIC e %1.tga(1) 3,061,904 57.1 61.5 969.1 375,618 324,196 423,195 432,272 336,801 528,787 129,303 511,732
UHIC m %1.tga(1) 2,964,146 119.6 124.8 1067.8 364,950 305,292 419,046 422,573 325,814 521,432 116,036 489,003
Table 5. Old JPEG image set.
Image Dimensions LOCO BMF -s- CALIC TMW HBB D&Y BMF -s
BALOON 720x576x8 2.90 2.84 2.83 2.66 2.80 2.74 2.67
BARB 720x576x8 4.65 4.44 4.41 4.08 4.28 4.29 4.03
BARB2 720x576x8 4.66 4.56 4.53 4.38 4.48 4.47 4.32
BOARD 720x576x8 3.64 3.58 3.56 ? 3.54 3.48 3.35
BOATS 720x576x8 3.92 3.85 3.84 3.61 3.80 3.75 3.62
GIRL 720x576x8 3.90 3.77 3.77 ? 3.74 3.67 3.54
GOLD 720x576x8 4.47 4.41 4.39 4.27 4.37 4.33 4.27
HOTEL 720x576x8 4.35 4.30 4.25 ? 4.27 4.19 4.09
LENA 512x512x8 4.24 4.10 4.11 3.91 4.07 4.03 3.96
ZELDA 720x576x8 3.87 3.75 3.75 ? 3.72 3.68 3.57
Average, bpp 4.06 3.96 3.94 ? 3.91 3.86 3.74
Table 6. Near-lossless compression.
Image Dimensions LOCO(1) BMF -s-(2,3) BMF -s(2)
1-off 3-off 7-off 1-off 3-off 7-off 1-off 3-off 7-off
BALOON 720x576x8 1.64 0.99 0.61 1.57 0.77 0.34 1.35 0.52 0.20
BARB 720x576x8 3.15 2.16 1.43 2.93 1.95 1.21 2.55 1.57 0.91
BARB2 720x576x8 3.17 2.13 1.38 3.06 2.04 1.25 2.83 1.79 1.05
BOARD 720x576x8 2.20 1.28 0.68 2.14 1.25 0.56 1.91 0.93 0.37
BOATS 720x576x8 2.48 1.51 0.94 2.40 1.46 0.84 2.17 1.20 0.64
GIRL 720x576x8 2.45 1.58 1.07 2.32 1.45 0.83 2.10 1.18 0.58
GOLD 720x576x8 3.00 1.94 1.25 2.90 1.89 1.15 2.77 1.73 0.96
HOTEL 720x576x8 2.87 1.88 1.11 2.80 1.81 1.00 2.60 1.57 0.80
LENA 512x512x8 2.71 1.76 1.14 2.60 1.64 0.90 2.45 1.43 0.68
ZELDA 720x576x8 2.37 1.54 0.97 2.30 1.40 0.64 2.10 1.15 0.45
Average, bpp 2.60 1.68 1.06 2.50 1.57 0.87 2.28 1.31 0.66
Encoding time, sec 4.5 3.9 3.3 5.0/4.1 4.8/3.9 3.9/3.1 21.9 21.0 20.4
Decoding time, sec 4.7 4.1 3.4 3.8/3.5 3.5/3.2 2.9/2.7 22.4 21.5 20.9
Table 7. New JPEG image set.
Image Dimensions Source Colors LOCO(4) BMF -s- CALIC LOCO-A BMF -s
AIR1 1024x1024x24 Aerial Photo RGB 5.43 4.67 5.31 ? 4.49
AIR2 720x1024x24 Aerial Photo RGB 4.11 3.75 3.81 3.58 3.58
BIKE 2048x2056x32 SCID CYMK 3.63 3.28 3.53 3.54 3.08
BIKE3 781x919x24 Crosfield digital scan RGB 4.38 4.43 4.25 4.33 4.23
CAFE 2048x2056x32 SCID CYMK 4.83 4.54 4.70 4.75 4.38
CATS 3072x2048x24 PhotoCD RGB 2.60 1.84 2.52 2.54 1.73
CHART 1752x2375x24 Computer generated CIELab 1.32 1.26 1.30 1.18 1.16
CHART_S 1688x2347x24 Scan of chart RGB 2.77 2.77 2.69 2.65 2.55
CMPND1 512x768x24 Computer generated RGB 1.23 1.10 1.33 1.21 1.06
CMPND2 1024x1400x24 Computer generated RGB 1.29 0.97 1.32 1.25 0.92
CR 1744x2048x8 Computer radiology mono 3.34 3.26 3.22 ? 3.17
CT 512x512x8 Computer tomography mono 1.32 1.27 1.22 ? 1.04
FAXBALLS 1024x512x24 Computer generated CIELab 0.90 0.68 0.82 0.64 0.57
FINGER 512x512x8 Fingerprint mono 5.66 5.50 5.45 5.50 5.28
GOLD 720x576x24/2 CCIR 601 YUV 3.73 3.69 3.66 3.85 3.51
GRAPHIC 2644x3046x24 Computer generated CIELab 2.36 2.29 2.28 ? 1.89
HOTEL 720x576x24/2 CCIR 601 YUV 3.61 3.58 3.55 3.72 3.34
MRI 256x256x8 Magnetic resonance mono 2.98 2.91 2.89 ? 2.69
TOOLS 1524x1200x32 Crosfield drum scan CYMK 5.08 5.01 4.96 5.01 4.86
US 512x448x8 Ultrasound mono 2.63 2.45 2.63 2.45 2.35
WATER 3072x2048x24 PhotoCD RGB 1.79 1.43 1.75 1.75 1.36
WOMAN 2048x2560x32 SCID CYMK 4.20 3.79 4.06 4.11 3.55
X_RAY 2048x1680x8 Medical x-ray mono 2.20 2.19 2.21 ? 2.09
Average, bpp 3.10 2.90 3.02 ? 2.73

4. Notes.

1) Time measurements are slightly inaccurate(0.2-0.5 sec.) because program is loaded many times during test;
2) /t option was used for this test;
3) First time value corresponds BMF program execution time, second one corresponds BMFG program - pure algorithm implementation(like to NLOCOE/NLOCOD pair);
4) The best of three possible modes was selected for each image;