== sanity test 123ad: Verify batching statahead works correctly ========================================================== 23:49:21 (1789271361) batch: statahead_max=32 statahead_batch_max=32 mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.batch_stats=0 llite.lustre-ffff9a30f7b4b800.statahead_max=32 llite.lustre-ffff9a30f7b4b800.statahead_batch_max=32 seq.cli-lustre-OST0000-super.width=0x1ffffff seq.cli-lustre-OST0001-super.width=0x1ffffff kvm mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.batch_stats=0 total: 100 open/close in 3.18 seconds: 31.44 ops/second llite.lustre-ffff9a30f7b4b800.statahead_max=0 101 real 0m2.305s user 0m0.004s sys 0m0.319s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9a30f7b4b800.statahead_max=32 llite.lustre-ffff9a30f7b4b800.statahead_max=32 101 real 0m1.074s user 0m0.000s sys 0m0.161s 'ls -l' 100 files with statahead: 1 sec statahead total: 141 statahead wrong: 2 agl total: 135 list_total: 30 fname_total: 102 hit_total: 3807 miss_total: 1533 swrong=2 ewrong=2 - open/close 716 (time 1789271388.77 total 10.00 last 71.60) total: 900 open/close in 12.57 seconds: 71.60 ops/second llite.lustre-ffff9a30f7b4b800.statahead_max=0 1001 real 0m19.389s user 0m0.029s sys 0m2.638s 'ls -l' 1000 files without statahead: 19 sec llite.lustre-ffff9a30f7b4b800.statahead_max=32 llite.lustre-ffff9a30f7b4b800.statahead_max=32 1001 real 0m9.504s user 0m0.029s sys 0m1.278s 'ls -l' 1000 files with statahead: 10 sec statahead total: 142 statahead wrong: 2 agl total: 136 list_total: 31 fname_total: 102 hit_total: 4806 miss_total: 1534 swrong=2 ewrong=2 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 21 seconds llite.lustre-ffff9a30f7b4b800.statahead_stats= statahead total: 142 statahead wrong: 2 agl total: 136 list_total: 31 fname_total: 102 hit_total: 4806 miss_total: 1534 mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.batch_stats= snapshot_time: 1789271454.471993307 secs.nsecs start_time: 1789271366.084354755 secs.nsecs elapsed_time: 88.387638552 secs.nsecs subreqs per batch batches %% cum %% 1: 1 2 2 2: 0 0 2 4: 0 0 2 8: 3 7 10 16: 2 5 15 32: 34 85 100 - open/close 716 (time 1789271466.92 total 10.01 last 71.56) - open/close 1459 (time 1789271476.94 total 20.03 last 74.14) - open/close 2128 (time 1789271486.96 total 30.04 last 66.79) - open/close 2798 (time 1789271496.97 total 40.05 last 66.95) - open/close 3545 (time 1789271506.97 total 50.06 last 74.66) - open/close 4341 (time 1789271516.98 total 60.06 last 79.53) - open/close 5072 (time 1789271527.00 total 70.08 last 72.98) - open/close 5802 (time 1789271537.00 total 80.09 last 72.97) - open/close 6570 (time 1789271547.01 total 90.09 last 76.73) - open/close 7347 (time 1789271557.02 total 100.10 last 77.62) - open/close 7969 (time 1789271567.03 total 110.11 last 62.15) - open/close 8733 (time 1789271577.04 total 120.12 last 76.34) - open/close 9398 (time 1789271587.06 total 130.14 last 66.34) - open/close 9950 (time 1789271597.06 total 140.15 last 55.19) total: 10000 open/close in 140.70 seconds: 71.07 ops/second llite.lustre-ffff9a30f7b4b800.statahead_batch_max=0 llite.lustre-ffff9a30f7b4b800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m58.623s user 0m0.323s sys 0m14.027s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9a30f7b4b800.statahead_batch_max=32 llite.lustre-ffff9a30f7b4b800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.stats=clear ls -l files in d123ad.sanity: 10001 real 1m42.994s user 0m0.301s sys 0m13.267s batched statahead RPC= hit count=9983 mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.batch_stats= snapshot_time: 1789271778.054408782 secs.nsecs start_time: 1789271665.445383021 secs.nsecs elapsed_time: 112.609025761 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 1 0 0 16: 1 0 0 32: 313 99 100 batch: statahead_max=2048 statahead_batch_max=256 mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.batch_stats=0 llite.lustre-ffff9a30f7b4b800.statahead_max=2048 llite.lustre-ffff9a30f7b4b800.statahead_batch_max=256 kvm mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.batch_stats=0 total: 100 open/close in 2.33 seconds: 42.84 ops/second llite.lustre-ffff9a30f7b4b800.statahead_max=0 101 real 0m1.948s user 0m0.009s sys 0m0.304s 'ls -l' 100 files without statahead: 2 sec llite.lustre-ffff9a30f7b4b800.statahead_max=2048 llite.lustre-ffff9a30f7b4b800.statahead_max=2048 101 real 0m0.904s user 0m0.006s sys 0m0.158s 'ls -l' 100 files with statahead: 1 sec statahead total: 2 statahead wrong: 0 agl total: 2 list_total: 2 fname_total: 0 hit_total: 10082 miss_total: 18 swrong=0 ewrong=0 - open/close 706 (time 1789272095.93 total 10.00 last 70.57) total: 900 open/close in 12.82 seconds: 70.22 ops/second llite.lustre-ffff9a30f7b4b800.statahead_max=0 1001 real 0m20.213s user 0m0.030s sys 0m2.745s 'ls -l' 1000 files without statahead: 20 sec llite.lustre-ffff9a30f7b4b800.statahead_max=2048 llite.lustre-ffff9a30f7b4b800.statahead_max=2048 1001 real 0m6.541s user 0m0.035s sys 0m1.292s 'ls -l' 1000 files with statahead: 7 sec statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 swrong=0 ewrong=0 'ls -l' done unlink + rm -r /mnt/lustre/d123ad.sanity: 22 seconds llite.lustre-ffff9a30f7b4b800.statahead_stats= statahead total: 3 statahead wrong: 0 agl total: 3 list_total: 3 fname_total: 0 hit_total: 11081 miss_total: 19 mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.batch_stats= snapshot_time: 1789272159.558164943 secs.nsecs start_time: 1789271781.352100615 secs.nsecs elapsed_time: 378.206064328 secs.nsecs subreqs per batch batches %% cum %% 1: 0 0 0 2: 0 0 0 4: 0 0 0 8: 2 20 20 16: 2 20 40 32: 1 10 50 64: 0 0 50 128: 1 10 60 256: 4 40 100 - open/close 734 (time 1789272171.77 total 10.01 last 73.32) - open/close 1454 (time 1789272181.78 total 20.02 last 71.96) - open/close 2207 (time 1789272191.79 total 30.03 last 75.17) - open/close 2992 (time 1789272201.80 total 40.04 last 78.48) - open/close 3713 (time 1789272211.82 total 50.06 last 71.93) - open/close 4502 (time 1789272221.83 total 60.07 last 78.84) - open/close 5157 (time 1789272231.83 total 70.07 last 65.47) - open/close 5839 (time 1789272241.84 total 80.08 last 68.14) - open/close 6528 (time 1789272251.84 total 90.09 last 68.88) - open/close 7196 (time 1789272261.85 total 100.09 last 66.78) - open/close 7888 (time 1789272271.85 total 110.09 last 69.17) - open/close 8595 (time 1789272281.88 total 120.12 last 70.49) - open/close 9242 (time 1789272291.88 total 130.12 last 64.69) - open/close 9870 (time 1789272301.92 total 140.16 last 62.58) total: 10000 open/close in 142.01 seconds: 70.42 ops/second llite.lustre-ffff9a30f7b4b800.statahead_batch_max=0 llite.lustre-ffff9a30f7b4b800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m50.887s user 0m0.322s sys 0m15.555s unbatched statahead RPC=10004 hit count=9999 llite.lustre-ffff9a30f7b4b800.statahead_batch_max=256 llite.lustre-ffff9a30f7b4b800.statahead_stats=clear mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.batch_stats=clear mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.stats=clear ls -l files in d123ad.sanity: 10001 real 0m45.838s user 0m0.308s sys 0m14.212s batched statahead RPC= hit count=9999 mdc.lustre-MDT0000-mdc-ffff9a30f7b4b800.batch_stats= snapshot_time: 1789272418.680110309 secs.nsecs start_time: 1789272362.855220095 secs.nsecs elapsed_time: 55.824890214 secs.nsecs subreqs per batch batches %% cum %% 1: 113 49 49 2: 25 11 60 4: 14 6 66 8: 10 4 71 16: 8 3 74 32: 15 6 81 64: 6 2 84 128: 1 0 84 256: 35 15 100 unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second unlink(/mnt/lustre/d123ad.sanity/f123ad.sanity0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second seq.cli-lustre-OST0000-super.width=65536 seq.cli-lustre-OST0001-super.width=65536 llite.lustre-ffff9a30f7b4b800.statahead_batch_max=64 llite.lustre-ffff9a30f7b4b800.statahead_max=128