Results for MPI perforance tests on anlspx (logs)
Contents
memcpy
No logfiles for memcpy
pingpong
No logfiles for pingpong
barrier
No logfiles for barrier
vector
No logfiles for vector
circulate
Pipelining pitfalls
mpicc -o circulate.log -mpilog -O -DDO_LOG circulate.c -lm
For n = 20000, m = 20000, T_comm = 0.033053, T_compute = 0.035704, sum = 0.068756, T_both = 0.044878
Writing logfile.
Finished writing logfile.
For n = 500, m = 500, T_comm = 0.000339, T_compute = 0.000895, sum = 0.001234, T_both = 0.001227
Writing logfile.
Finished writing logfile.
3way
Exploring the cost of synchronization delays
mpicc -o bad.log -mpilog -O \
bad.o -lm
[2] Litsize = 8, Time for first send = 0.000271, for second = 0.000126
Writing logfile.
Finished writing logfile.
[2] Litsize = 9, Time for first send = 0.000271, for second = 0.000127
Writing logfile.
Finished writing logfile.
[2] Litsize = 511, Time for first send = 0.000426, for second = 0.000341
Writing logfile.
Finished writing logfile.
[2] Litsize = 512, Time for first send = 0.000421, for second = 0.000343
Writing logfile.
Finished writing logfile.
[2] Litsize = 513, Time for first send = 0.000426, for second = 0.000447
Writing logfile.
Finished writing logfile.
mpicc -c -O bounce.c
mpicc -o bounce.log -mpilog -O \
bounce.o -lm
Writing logfile.
Finished writing logfile.
jacobi
Jacobi Iteration - Example Parallel Mesh
mpicc -o jacobi.log -mpilog -O \
jacobi.o cmdline.o setupmesh.o exchng.o -lm
send/recv: 6 iterations in 0.000831 secs (0.505643 MFlops); diffnorm 0.008134, m=7 n=4 np=1
Writing logfile.
Finished writing logfile.
send/recv: 7 iterations in 0.022050 secs (18.204527 MFlops); diffnorm 0.006899, m=4098 n=4 np=1
Writing logfile.
Finished writing logfile.
send/recv: 24 iterations in 0.021163 secs (0.317540 MFlops); diffnorm 0.009895, m=7 n=10 np=4
Writing logfile.
Finished writing logfile.
send/recv: 25 iterations in 0.304245 secs (18.847991 MFlops); diffnorm 0.138820, m=4098 n=10 np=4
Writing logfile.
Finished writing logfile.
send/recv: 25 iterations in 0.032661 secs (0.857287 MFlops); diffnorm 0.036615, m=7 n=34 np=16
Writing logfile.
Finished writing logfile.
send/recv: 25 iterations in 0.384632 secs (59.635257 MFlops); diffnorm 0.468864, m=4098 n=34 np=16
Writing logfile.
Finished writing logfile.
send/recv: 25 iterations in 0.039409 secs (1.420984 MFlops); diffnorm 0.055291, m=7 n=66 np=32
Writing logfile.
Finished writing logfile.
send/recv: 25 iterations in 2.553498 secs (17.965630 MFlops); diffnorm 0.470684, m=4098 n=66 np=32
Writing logfile.
Finished writing logfile.
Jacobi Iteration - Shift up and down
mpicc -o jacobi.log -mpilog -O \
jacobi.o cmdline.o setupmesh.o exchng.o -lm
shift/sendrecv: 6 iterations in 0.010430 secs (0.040267 MFlops); diffnorm 0.008134, m=7 n=4 np=1
Writing logfile.
Finished writing logfile.
shift/sendrecv: 7 iterations in 0.031929 secs (12.571717 MFlops); diffnorm 0.006899, m=4098 n=4 np=1
Writing logfile.
Finished writing logfile.
shift/sendrecv: 24 iterations in 0.022271 secs (0.301743 MFlops); diffnorm 0.009895, m=7 n=10 np=4
Writing logfile.
Finished writing logfile.
shift/sendrecv: 25 iterations in 0.287593 secs (19.939315 MFlops); diffnorm 0.138820, m=4098 n=10 np=4
Writing logfile.
Finished writing logfile.
shift/sendrecv: 25 iterations in 0.033798 secs (0.828448 MFlops); diffnorm 0.036615, m=7 n=34 np=16
Writing logfile.
Finished writing logfile.
shift/sendrecv: 25 iterations in 0.354620 secs (64.682136 MFlops); diffnorm 0.468864, m=4098 n=34 np=16
Writing logfile.
Finished writing logfile.
shift/sendrecv: 25 iterations in 0.043489 secs (1.287694 MFlops); diffnorm 0.055291, m=7 n=66 np=32
Writing logfile.
Finished writing logfile.
shift/sendrecv: 25 iterations in 2.610529 secs (17.573146 MFlops); diffnorm 0.470684, m=4098 n=66 np=32
Writing logfile.
Finished writing logfile.
Jacobi Iteration - Exchange head-to-head
mpicc -o jacobi.log -mpilog -O \
jacobi.o cmdline.o setupmesh.o exchng.o -lm
head-to-head sendrecv: 6 iterations in 0.010569 secs (0.039740 MFlops); diffnorm 0.008134, m=7 n=4 np=1
Writing logfile.
Finished writing logfile.
head-to-head sendrecv: 7 iterations in 0.031871 secs (12.594921 MFlops); diffnorm 0.006899, m=4098 n=4 np=1
Writing logfile.
Finished writing logfile.
head-to-head sendrecv: 24 iterations in 0.022451 secs (0.299317 MFlops); diffnorm 0.009895, m=7 n=10 np=4
Writing logfile.
Finished writing logfile.
head-to-head sendrecv: 25 iterations in 0.275395 secs (20.822419 MFlops); diffnorm 0.138820, m=4098 n=10 np=4
Writing logfile.
Finished writing logfile.
head-to-head sendrecv: 25 iterations in 0.035362 secs (0.791800 MFlops); diffnorm 0.036615, m=7 n=34 np=16
Writing logfile.
Finished writing logfile.
head-to-head sendrecv: 25 iterations in 0.338730 secs (67.716400 MFlops); diffnorm 0.468864, m=4098 n=34 np=16
Writing logfile.
Finished writing logfile.
head-to-head sendrecv: 25 iterations in 0.042348 secs (1.322384 MFlops); diffnorm 0.055291, m=7 n=66 np=32
Writing logfile.
Finished writing logfile.
head-to-head sendrecv: 25 iterations in 0.412837 secs (111.121890 MFlops); diffnorm 0.470684, m=4098 n=66 np=32
Writing logfile.
Finished writing logfile.
Jacobi Iteration - Nonblocking send/recv
mpicc -o jacobi.log -mpilog -O \
jacobi.o cmdline.o setupmesh.o exchng.o -lm
irecv/isend: 6 iterations in 0.010506 secs (0.039976 MFlops); diffnorm 0.008134, m=7 n=4 np=1
Writing logfile.
Finished writing logfile.
irecv/isend: 7 iterations in 0.032051 secs (12.524079 MFlops); diffnorm 0.006899, m=4098 n=4 np=1
Writing logfile.
Finished writing logfile.
irecv/isend: 24 iterations in 0.021828 secs (0.307864 MFlops); diffnorm 0.009895, m=7 n=10 np=4
Writing logfile.
Finished writing logfile.
irecv/isend: 25 iterations in 0.282947 secs (20.266704 MFlops); diffnorm 0.138820, m=4098 n=10 np=4
Writing logfile.
Finished writing logfile.
irecv/isend: 25 iterations in 0.034289 secs (0.816594 MFlops); diffnorm 0.036615, m=7 n=34 np=16
Writing logfile.
Finished writing logfile.
irecv/isend: 25 iterations in 0.383743 secs (59.773412 MFlops); diffnorm 0.468864, m=4098 n=34 np=16
Writing logfile.
Finished writing logfile.
irecv/isend: 25 iterations in 0.081017 secs (0.691211 MFlops); diffnorm 0.055291, m=7 n=66 np=32
Writing logfile.
Finished writing logfile.
irecv/isend: 25 iterations in 2.575737 secs (17.810511 MFlops); diffnorm 0.470684, m=4098 n=66 np=32
Writing logfile.
Finished writing logfile.
Jacobi Iteration - Nonblocking send/recv for receiver pull
mpicc -o jacobi.log -mpilog -O \
jacobi.o cmdline.o setupmesh.o exchng.o -lm
isend/irecv: 6 iterations in 0.010500 secs (0.040001 MFlops); diffnorm 0.008134, m=7 n=4 np=1
Writing logfile.
Finished writing logfile.
isend/irecv: 7 iterations in 0.032385 secs (12.395047 MFlops); diffnorm 0.006899, m=4098 n=4 np=1
Writing logfile.
Finished writing logfile.
isend/irecv: 24 iterations in 0.022546 secs (0.298058 MFlops); diffnorm 0.009895, m=7 n=10 np=4
Writing logfile.
Finished writing logfile.
isend/irecv: 25 iterations in 0.309266 secs (18.541954 MFlops); diffnorm 0.138820, m=4098 n=10 np=4
Writing logfile.
Finished writing logfile.
isend/irecv: 25 iterations in 0.035612 secs (0.786248 MFlops); diffnorm 0.036615, m=7 n=34 np=16
Writing logfile.
Finished writing logfile.
isend/irecv: 25 iterations in 0.409805 secs (55.972034 MFlops); diffnorm 0.468864, m=4098 n=34 np=16
Writing logfile.
Finished writing logfile.
isend/irecv: 25 iterations in 0.056090 secs (0.998390 MFlops); diffnorm 0.055291, m=7 n=66 np=32
Writing logfile.
Finished writing logfile.
isend/irecv: 25 iterations in 7.550967 secs (6.075407 MFlops); diffnorm 0.470684, m=4098 n=66 np=32
Writing logfile.
Finished writing logfile.
Jacobi Iteration - Synchronous send
mpicc -o jacobi.log -mpilog -O \
jacobi.o cmdline.o setupmesh.o exchng.o -lm
ssend/irecv: 6 iterations in 0.010452 secs (0.040184 MFlops); diffnorm 0.008134, m=7 n=4 np=1
Writing logfile.
Finished writing logfile.
ssend/irecv: 7 iterations in 0.033705 secs (11.909423 MFlops); diffnorm 0.006899, m=4098 n=4 np=1
Writing logfile.
Finished writing logfile.
ssend/irecv: 24 iterations in 0.070785 secs (0.094935 MFlops); diffnorm 0.009895, m=7 n=10 np=4
Writing logfile.
Finished writing logfile.
ssend/irecv: 25 iterations in 0.296525 secs (19.338686 MFlops); diffnorm 0.138820, m=4098 n=10 np=4
Writing logfile.
Finished writing logfile.
ssend/irecv: 25 iterations in 0.182304 secs (0.153589 MFlops); diffnorm 0.036615, m=7 n=34 np=16
Writing logfile.
Finished writing logfile.
ssend/irecv: 25 iterations in 0.488836 secs (46.922938 MFlops); diffnorm 0.468864, m=4098 n=34 np=16
Writing logfile.
Finished writing logfile.
ssend/irecv: 25 iterations in 2.292568 secs (0.024427 MFlops); diffnorm 0.055291, m=7 n=66 np=32
Writing logfile.
Finished writing logfile.
ssend/irecv: 25 iterations in 2.553770 secs (17.963715 MFlops); diffnorm 0.470684, m=4098 n=66 np=32
Writing logfile.
Finished writing logfile.
Jacobi Iteration - Ready send
mpicc -c -O exchng.c
mpicc -o jacobi.log -mpilog -O \
jacobi.o cmdline.o setupmesh.o exchng.o -lm
rsend: 6 iterations in 0.011040 secs (0.038043 MFlops); diffnorm 0.008134, m=7 n=4 np=1
Writing logfile.
Finished writing logfile.
rsend: 7 iterations in 0.032218 secs (12.459113 MFlops); diffnorm 0.006899, m=4098 n=4 np=1
Writing logfile.
Finished writing logfile.
rsend: 24 iterations in 0.043433 secs (0.154722 MFlops); diffnorm 0.009895, m=7 n=10 np=4
Writing logfile.
Finished writing logfile.
rsend: 25 iterations in 0.297904 secs (19.249169 MFlops); diffnorm 0.138820, m=4098 n=10 np=4
Writing logfile.
Finished writing logfile.
rsend: 25 iterations in 0.106975 secs (0.261744 MFlops); diffnorm 0.036615, m=7 n=34 np=16
Writing logfile.
Finished writing logfile.
rsend: 25 iterations in 0.450297 secs (50.938856 MFlops); diffnorm 0.468864, m=4098 n=34 np=16
Writing logfile.
Finished writing logfile.
rsend: 25 iterations in 0.279063 secs (0.200672 MFlops); diffnorm 0.055291, m=7 n=66 np=32
Writing logfile.
Finished writing logfile.
rsend: 25 iterations in 2.604492 secs (17.613879 MFlops); diffnorm 0.470684, m=4098 n=66 np=32
Writing logfile.
Finished writing logfile.
Jacobi Iteration - Overlapping communication
mpicc -o jacobi.log -mpilog -O \
jacobi.o cmdline.o setupmesh.o exchng.o -lm
isend/overlap: 6 iterations in 0.010505 secs (0.039982 MFlops); diffnorm 0.008134, m=7 n=4 np=1
Writing logfile.
Finished writing logfile.
isend/overlap: 7 iterations in 0.034816 secs (11.529577 MFlops); diffnorm 0.006899, m=4098 n=4 np=1
Writing logfile.
Finished writing logfile.
isend/overlap: 24 iterations in 0.021712 secs (0.309501 MFlops); diffnorm 0.009895, m=7 n=10 np=4
Writing logfile.
Finished writing logfile.
isend/overlap: 25 iterations in 0.286178 secs (20.037887 MFlops); diffnorm 0.138820, m=4098 n=10 np=4
Writing logfile.
Finished writing logfile.
isend/overlap: 25 iterations in 0.034726 secs (0.806305 MFlops); diffnorm 0.036615, m=7 n=34 np=16
Writing logfile.
Finished writing logfile.
isend/overlap: 25 iterations in 0.376102 secs (60.987686 MFlops); diffnorm 0.468864, m=4098 n=34 np=16
Writing logfile.
Finished writing logfile.
isend/overlap: 25 iterations in 0.043098 secs (1.299356 MFlops); diffnorm 0.055291, m=7 n=66 np=32
Writing logfile.
Finished writing logfile.
isend/overlap: 25 iterations in 2.560719 secs (17.914965 MFlops); diffnorm 0.470684, m=4098 n=66 np=32
Writing logfile.
Finished writing logfile.
Jacobi Iteration - Overlapping communication (sends first)
mpicc -o jacobi.log -mpilog -O \
jacobi.o cmdline.o setupmesh.o exchng.o -lm
send first/overlap: 6 iterations in 0.010502 secs (0.039994 MFlops); diffnorm 0.008134, m=7 n=4 np=1
Writing logfile.
Finished writing logfile.
send first/overlap: 7 iterations in 0.031823 secs (12.613691 MFlops); diffnorm 0.006899, m=4098 n=4 np=1
Writing logfile.
Finished writing logfile.
send first/overlap: 24 iterations in 0.022416 secs (0.299786 MFlops); diffnorm 0.009895, m=7 n=10 np=4
Writing logfile.
Finished writing logfile.
send first/overlap: 25 iterations in 0.309429 secs (18.532219 MFlops); diffnorm 0.138820, m=4098 n=10 np=4
Writing logfile.
Finished writing logfile.
send first/overlap: 25 iterations in 0.039359 secs (0.711409 MFlops); diffnorm 0.036615, m=7 n=34 np=16
Writing logfile.
Finished writing logfile.
send first/overlap: 25 iterations in 0.404551 secs (56.698936 MFlops); diffnorm 0.468864, m=4098 n=34 np=16
Writing logfile.
Finished writing logfile.
send first/overlap: 25 iterations in 0.042441 secs (1.319488 MFlops); diffnorm 0.055291, m=7 n=66 np=32
Writing logfile.
Finished writing logfile.
send first/overlap: 25 iterations in 3.024517 secs (15.167779 MFlops); diffnorm 0.470684, m=4098 n=66 np=32
Writing logfile.
Finished writing logfile.
Jacobi Iteration - Persistent send/recv
mpicc -o jacobi.log -mpilog -O \
jacobi.o cmdline.o setupmesh.o exchng.o -lm
persistent send/recv: 6 iterations in 0.020108 secs (0.020888 MFlops); diffnorm 0.008134, m=7 n=4 np=1
Writing logfile.
Finished writing logfile.
persistent send/recv: 7 iterations in 0.055665 secs (7.211125 MFlops); diffnorm 0.006899, m=4098 n=4 np=1
Writing logfile.
Finished writing logfile.
persistent send/recv: 24 iterations in 0.062939 secs (0.106769 MFlops); diffnorm 0.009895, m=7 n=10 np=4
Writing logfile.
Finished writing logfile.
persistent send/recv: 25 iterations in 0.330026 secs (17.375574 MFlops); diffnorm 0.138820, m=4098 n=10 np=4
Writing logfile.
Finished writing logfile.
persistent send/recv: 25 iterations in 0.102669 secs (0.272722 MFlops); diffnorm 0.036615, m=7 n=34 np=16
Writing logfile.
Finished writing logfile.
persistent send/recv: 25 iterations in 0.443456 secs (51.724580 MFlops); diffnorm 0.468864, m=4098 n=34 np=16
Writing logfile.
Finished writing logfile.
persistent send/recv: 25 iterations in 0.176189 secs (0.317840 MFlops); diffnorm 0.055291, m=7 n=66 np=32
Writing logfile.
Finished writing logfile.
persistent send/recv: 25 iterations in 2.554920 secs (17.955628 MFlops); diffnorm 0.470684, m=4098 n=66 np=32
Writing logfile.
Finished writing logfile.