== sanity-quota test 8: Run dbench with quota enabled ==== 02:57:07 (1727765827) Waiting for MDT destroys to complete oleg426-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported pdsh@oleg426-client: oleg426-server: ssh exited with exit code 1 oleg426-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg426-client: oleg426-server: ssh exited with exit code 1 Creating test directory fail_val=0 fail_loc=0 Waiting 90s for 'ugp' Set enough high limit for user: quota_usr Set enough high limit for group: quota_usr /home/green/git/lustre-release/lustre/utils/lfs project -sp 1000 /mnt/lustre/d8.sanity-quota Set enough high limit for project: 1000 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: [bash] [rundbench] [-D] [/mnt/lustre/d8.sanity-quota] [3] [-t] [120] looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 3 -t 120' on /mnt/lustre/d8.sanity-quota at Tue Oct 1 02:57:18 EDT 2024 waiting for dbench pid 24182 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 120 seconds with load 'client.txt' and minimum warmup 24 secs failed to create barrier semaphore 0 of 3 processes prepared for launch 0 sec 3 of 3 processes prepared for launch 0 sec releasing clients 3 108 15.20 MB/sec warmup 1 sec latency 56.838 ms 3 181 10.98 MB/sec warmup 2 sec latency 110.456 ms 3 252 9.57 MB/sec warmup 3 sec latency 93.829 ms 3 312 8.54 MB/sec warmup 4 sec latency 115.300 ms 3 375 8.03 MB/sec warmup 5 sec latency 109.221 ms 3 466 8.16 MB/sec warmup 6 sec latency 94.534 ms 3 597 8.87 MB/sec warmup 7 sec latency 79.581 ms 3 676 7.94 MB/sec warmup 8 sec latency 71.675 ms 3 738 7.10 MB/sec warmup 9 sec latency 77.517 ms 3 793 6.43 MB/sec warmup 10 sec latency 86.156 ms 3 856 5.88 MB/sec warmup 11 sec latency 58.688 ms 3 921 5.44 MB/sec warmup 12 sec latency 72.064 ms 3 975 5.04 MB/sec warmup 13 sec latency 109.480 ms 3 1045 4.90 MB/sec warmup 14 sec latency 87.728 ms 3 1118 4.59 MB/sec warmup 15 sec latency 76.585 ms 3 1169 4.31 MB/sec warmup 16 sec latency 93.625 ms 3 1218 4.07 MB/sec warmup 17 sec latency 93.025 ms 3 1266 3.85 MB/sec warmup 18 sec latency 105.436 ms 3 1393 4.10 MB/sec warmup 19 sec latency 81.548 ms 3 1445 3.90 MB/sec warmup 20 sec latency 75.338 ms 3 1502 3.72 MB/sec warmup 21 sec latency 68.986 ms 3 1575 3.56 MB/sec warmup 22 sec latency 52.981 ms 3 1626 3.41 MB/sec warmup 23 sec latency 112.317 ms 3 1758 0.17 MB/sec execute 1 sec latency 92.263 ms 3 1814 0.17 MB/sec execute 2 sec latency 66.425 ms 3 1862 0.13 MB/sec execute 3 sec latency 81.438 ms 3 1938 0.17 MB/sec execute 4 sec latency 61.830 ms 3 2005 0.17 MB/sec execute 5 sec latency 57.133 ms 3 2057 0.20 MB/sec execute 6 sec latency 75.730 ms 3 2145 0.20 MB/sec execute 7 sec latency 80.448 ms 3 2185 0.20 MB/sec execute 8 sec latency 102.513 ms 3 2267 0.29 MB/sec execute 9 sec latency 109.659 ms 3 2332 0.48 MB/sec execute 10 sec latency 102.727 ms 3 2371 0.46 MB/sec execute 11 sec latency 103.709 ms 3 2416 0.46 MB/sec execute 12 sec latency 123.266 ms 3 2470 0.53 MB/sec execute 13 sec latency 96.127 ms 3 2559 0.57 MB/sec execute 14 sec latency 98.806 ms 3 2593 0.55 MB/sec execute 15 sec latency 159.072 ms 3 2727 1.07 MB/sec execute 16 sec latency 112.459 ms 3 2881 1.08 MB/sec execute 17 sec latency 57.106 ms 3 2959 1.21 MB/sec execute 18 sec latency 68.282 ms 3 3019 1.15 MB/sec execute 19 sec latency 89.016 ms 3 3222 1.23 MB/sec execute 20 sec latency 79.555 ms 3 3302 1.30 MB/sec execute 21 sec latency 66.580 ms 3 3358 1.25 MB/sec execute 22 sec latency 51.680 ms 3 3463 1.45 MB/sec execute 23 sec latency 57.660 ms 3 3541 1.52 MB/sec execute 24 sec latency 67.631 ms 3 3610 1.48 MB/sec execute 25 sec latency 67.210 ms 3 3676 1.44 MB/sec execute 26 sec latency 68.331 ms 3 3764 1.50 MB/sec execute 27 sec latency 72.262 ms 3 3803 1.45 MB/sec execute 28 sec latency 102.252 ms 3 3842 1.40 MB/sec execute 29 sec latency 69.943 ms 3 3871 1.35 MB/sec execute 30 sec latency 91.160 ms 3 3911 1.34 MB/sec execute 31 sec latency 131.778 ms 3 3942 1.30 MB/sec execute 32 sec latency 98.979 ms 3 3981 1.26 MB/sec execute 33 sec latency 67.174 ms 3 4016 1.22 MB/sec execute 34 sec latency 69.369 ms 3 4051 1.19 MB/sec execute 35 sec latency 93.007 ms 3 4078 1.15 MB/sec execute 36 sec latency 124.569 ms 3 4126 1.12 MB/sec execute 37 sec latency 77.985 ms 3 4175 1.09 MB/sec execute 38 sec latency 73.059 ms 3 4223 1.07 MB/sec execute 39 sec latency 78.457 ms 3 4259 1.05 MB/sec execute 40 sec latency 193.229 ms 3 4306 1.04 MB/sec execute 41 sec latency 107.673 ms 3 4352 1.02 MB/sec execute 42 sec latency 89.302 ms 3 4412 1.01 MB/sec execute 43 sec latency 101.471 ms 3 4464 0.99 MB/sec execute 44 sec latency 79.030 ms 3 4496 0.98 MB/sec execute 45 sec latency 121.009 ms 3 4535 0.96 MB/sec execute 46 sec latency 105.077 ms 3 4582 0.98 MB/sec execute 47 sec latency 160.829 ms 3 4639 0.99 MB/sec execute 48 sec latency 87.309 ms 3 4691 0.97 MB/sec execute 49 sec latency 110.957 ms 3 4735 0.95 MB/sec execute 50 sec latency 101.854 ms 3 4789 0.94 MB/sec execute 51 sec latency 81.295 ms 3 4840 0.92 MB/sec execute 52 sec latency 114.986 ms 3 4962 1.07 MB/sec execute 53 sec latency 77.282 ms 3 5014 1.05 MB/sec execute 54 sec latency 94.558 ms 3 5063 1.03 MB/sec execute 55 sec latency 69.212 ms 3 5119 1.02 MB/sec execute 56 sec latency 73.436 ms 3 5161 1.00 MB/sec execute 57 sec latency 75.022 ms 3 5215 0.99 MB/sec execute 58 sec latency 75.316 ms 3 5270 0.97 MB/sec execute 59 sec latency 92.356 ms 3 5326 0.96 MB/sec execute 60 sec latency 79.054 ms 3 5369 0.94 MB/sec execute 61 sec latency 64.706 ms 3 5423 0.93 MB/sec execute 62 sec latency 61.289 ms 3 5477 0.92 MB/sec execute 63 sec latency 83.459 ms 3 5543 0.91 MB/sec execute 64 sec latency 74.638 ms 3 5585 0.90 MB/sec execute 65 sec latency 70.754 ms 3 5638 0.89 MB/sec execute 66 sec latency 93.074 ms 3 5717 0.88 MB/sec execute 67 sec latency 58.189 ms 3 5761 0.87 MB/sec execute 68 sec latency 84.264 ms 3 5843 0.88 MB/sec execute 69 sec latency 95.917 ms 3 5902 0.89 MB/sec execute 70 sec latency 74.751 ms 3 5949 0.88 MB/sec execute 71 sec latency 110.113 ms 3 5991 0.87 MB/sec execute 72 sec latency 100.902 ms 3 6035 0.88 MB/sec execute 73 sec latency 80.789 ms 3 6117 0.88 MB/sec execute 74 sec latency 112.619 ms 3 6156 0.87 MB/sec execute 75 sec latency 117.614 ms 3 6308 0.98 MB/sec execute 76 sec latency 84.578 ms 3 6431 0.98 MB/sec execute 77 sec latency 79.035 ms 3 6488 1.01 MB/sec execute 78 sec latency 91.962 ms 3 6559 1.00 MB/sec execute 79 sec latency 82.399 ms 3 6741 1.02 MB/sec execute 80 sec latency 88.237 ms 3 6821 1.04 MB/sec execute 81 sec latency 106.064 ms 3 6865 1.03 MB/sec execute 82 sec latency 51.832 ms 3 6908 1.02 MB/sec execute 83 sec latency 52.341 ms 3 6998 1.06 MB/sec execute 84 sec latency 54.646 ms 3 7107 1.10 MB/sec execute 85 sec latency 52.976 ms 3 7157 1.10 MB/sec execute 86 sec latency 91.726 ms 3 7212 1.09 MB/sec execute 87 sec latency 96.958 ms 3 7250 1.08 MB/sec execute 88 sec latency 93.750 ms 3 7318 1.10 MB/sec execute 89 sec latency 131.414 ms 3 7349 1.09 MB/sec execute 90 sec latency 103.787 ms 3 7378 1.08 MB/sec execute 91 sec latency 97.493 ms 3 7410 1.06 MB/sec execute 92 sec latency 82.198 ms 3 7444 1.06 MB/sec execute 93 sec latency 76.898 ms 3 7482 1.05 MB/sec execute 94 sec latency 116.286 ms 3 7512 1.04 MB/sec execute 95 sec latency 90.033 ms 3 7535 1.03 MB/sec execute 96 sec latency 125.907 ms 3 7571 1.02 MB/sec execute 97 sec latency 91.548 ms 3 7601 1.01 MB/sec execute 98 sec latency 77.614 ms 3 7645 1.00 MB/sec execute 99 sec latency 69.446 ms 3 7694 0.99 MB/sec execute 100 sec latency 84.021 ms 3 7751 0.98 MB/sec execute 101 sec latency 155.684 ms 3 7801 0.98 MB/sec execute 102 sec latency 76.659 ms 3 7852 0.97 MB/sec execute 103 sec latency 107.764 ms 3 7897 0.96 MB/sec execute 104 sec latency 99.911 ms 3 7968 0.96 MB/sec execute 105 sec latency 77.191 ms 3 8019 0.95 MB/sec execute 106 sec latency 91.886 ms 3 8077 0.95 MB/sec execute 107 sec latency 60.405 ms 3 8144 0.97 MB/sec execute 108 sec latency 77.471 ms 3 8203 0.96 MB/sec execute 109 sec latency 62.284 ms 3 8266 0.95 MB/sec execute 110 sec latency 72.077 ms 3 8315 0.95 MB/sec execute 111 sec latency 81.652 ms 3 8373 0.94 MB/sec execute 112 sec latency 78.356 ms 3 8508 1.01 MB/sec execute 113 sec latency 73.297 ms 3 8574 1.00 MB/sec execute 114 sec latency 75.035 ms 3 8633 0.99 MB/sec execute 115 sec latency 65.978 ms 3 8684 0.99 MB/sec execute 116 sec latency 92.921 ms 3 8731 0.98 MB/sec execute 117 sec latency 62.172 ms 3 8794 0.97 MB/sec execute 118 sec latency 119.745 ms 3 8835 0.96 MB/sec execute 119 sec latency 113.405 ms 3 cleanup 120 sec 3 cleanup 121 sec 3 cleanup 122 sec 0 cleanup 123 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 3757 45.064 193.215 Close 2761 5.257 52.920 Rename 156 76.877 160.805 Unlink 750 28.275 109.904 Qpathinfo 3451 17.449 159.049 Qfileinfo 589 0.678 9.302 Qfsinfo 612 30.678 102.706 Sfileinfo 300 35.446 121.313 Find 1299 28.343 146.242 WriteX 1820 4.814 43.102 ReadX 5838 0.121 7.429 LockX 12 5.355 11.254 UnlockX 12 4.575 12.435 Flush 258 25.710 100.335 Throughput 0.964043 MB/sec 3 clients 3 procs max_latency=193.229 ms stopping dbench on /mnt/lustre/d8.sanity-quota at Tue Oct 1 02:59:45 EDT 2024 with return code 0 clean dbench files on /mnt/lustre/d8.sanity-quota /mnt/lustre/d8.sanity-quota /mnt/lustre/d8.sanity-quota removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client2/~dmtmp/SEED' removed directory: 'clients/client2/~dmtmp/EXCEL' removed directory: 'clients/client2/~dmtmp/PWRPNT' removed directory: 'clients/client2/~dmtmp/WORDPRO' removed directory: 'clients/client2/~dmtmp/PM' removed directory: 'clients/client2/~dmtmp/WORD' removed directory: 'clients/client2/~dmtmp/COREL' removed directory: 'clients/client2/~dmtmp/PARADOX' removed directory: 'clients/client2/~dmtmp/ACCESS' removed directory: 'clients/client2/~dmtmp' removed directory: 'clients/client2' removed directory: 'clients' removed 'client.txt' /mnt/lustre/d8.sanity-quota dbench successfully finished /home/green/git/lustre-release/lustre/utils/lfs project -C /mnt/lustre/d8.sanity-quota Delete files... Wait for unlink objects finished... Waiting for MDT destroys to complete oleg426-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported oleg426-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported pdsh@oleg426-client: oleg426-server: ssh exited with exit code 1 pdsh@oleg426-client: oleg426-server: ssh exited with exit code 1