== sanity-quota test 8: Run dbench with quota enabled ==== 14:33:47 (1722796427) Waiting for MDT destroys to complete oleg141-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported pdsh@oleg141-client: oleg141-server: ssh exited with exit code 1 oleg141-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg141-client: oleg141-server: ssh exited with exit code 1 Creating test directory fail_val=0 fail_loc=0 Waiting 90s for 'ugp' Set enough high limit for user: quota_usr Set enough high limit for group: quota_usr lfs project -sp 1000 /mnt/lustre/d8.sanity-quota Set enough high limit for project: 1000 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: [bash] [rundbench] [-D] [/mnt/lustre/d8.sanity-quota] [3] [-t] [120] looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 3 -t 120' on /mnt/lustre/d8.sanity-quota at Sun Aug 4 14:33:55 EDT 2024 waiting for dbench pid 25038 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 120 seconds with load 'client.txt' and minimum warmup 24 secs failed to create barrier semaphore 2 of 3 processes prepared for launch 0 sec 3 of 3 processes prepared for launch 0 sec releasing clients 3 197 23.51 MB/sec warmup 1 sec latency 28.553 ms 3 372 19.92 MB/sec warmup 2 sec latency 26.883 ms 3 646 21.06 MB/sec warmup 3 sec latency 40.857 ms 3 898 16.26 MB/sec warmup 4 sec latency 20.301 ms 3 1093 13.76 MB/sec warmup 5 sec latency 25.281 ms 3 1297 11.80 MB/sec warmup 6 sec latency 24.606 ms 3 1580 11.20 MB/sec warmup 7 sec latency 32.204 ms 3 1814 9.87 MB/sec warmup 8 sec latency 30.979 ms 3 2130 8.89 MB/sec warmup 9 sec latency 23.182 ms 3 2404 8.40 MB/sec warmup 10 sec latency 25.299 ms 3 2774 8.72 MB/sec warmup 11 sec latency 24.589 ms 3 3201 8.57 MB/sec warmup 12 sec latency 29.632 ms 3 3530 8.85 MB/sec warmup 13 sec latency 24.605 ms 3 3775 8.52 MB/sec warmup 14 sec latency 19.961 ms 3 3954 8.01 MB/sec warmup 15 sec latency 21.607 ms 3 4120 7.52 MB/sec warmup 16 sec latency 20.455 ms 3 4357 7.15 MB/sec warmup 17 sec latency 62.441 ms 3 4669 7.01 MB/sec warmup 18 sec latency 15.548 ms 3 4919 6.99 MB/sec warmup 19 sec latency 24.809 ms 3 5132 6.78 MB/sec warmup 20 sec latency 31.856 ms 3 5318 6.48 MB/sec warmup 21 sec latency 26.086 ms 3 5542 6.21 MB/sec warmup 22 sec latency 18.679 ms 3 5756 5.98 MB/sec warmup 23 sec latency 18.757 ms 3 6673 17.47 MB/sec execute 1 sec latency 23.956 ms 3 7080 15.41 MB/sec execute 2 sec latency 21.530 ms 3 7289 11.33 MB/sec execute 3 sec latency 24.093 ms 3 7454 8.97 MB/sec execute 4 sec latency 21.436 ms 3 7630 7.19 MB/sec execute 5 sec latency 20.673 ms 3 7877 6.18 MB/sec execute 6 sec latency 39.595 ms 3 8085 5.52 MB/sec execute 7 sec latency 25.800 ms 3 8298 5.27 MB/sec execute 8 sec latency 24.079 ms 3 8567 5.68 MB/sec execute 9 sec latency 22.999 ms 3 8784 5.17 MB/sec execute 10 sec latency 39.814 ms 3 9158 4.80 MB/sec execute 11 sec latency 26.069 ms 3 9514 4.77 MB/sec execute 12 sec latency 25.020 ms 3 9784 5.07 MB/sec execute 13 sec latency 30.312 ms 3 10118 5.26 MB/sec execute 14 sec latency 24.850 ms 3 10540 5.62 MB/sec execute 15 sec latency 25.802 ms 3 10766 5.57 MB/sec execute 16 sec latency 19.504 ms 3 10944 5.43 MB/sec execute 17 sec latency 22.399 ms 3 11101 5.18 MB/sec execute 18 sec latency 21.308 ms 3 11295 4.92 MB/sec execute 19 sec latency 21.759 ms 3 11612 4.79 MB/sec execute 20 sec latency 55.317 ms 3 11880 4.74 MB/sec execute 21 sec latency 24.644 ms 3 12141 4.93 MB/sec execute 22 sec latency 32.980 ms 3 12336 4.74 MB/sec execute 23 sec latency 30.982 ms 3 12525 4.56 MB/sec execute 24 sec latency 25.322 ms 3 12780 4.42 MB/sec execute 25 sec latency 20.564 ms 3 13082 4.41 MB/sec execute 26 sec latency 25.530 ms 3 13364 4.57 MB/sec execute 27 sec latency 31.317 ms 3 13792 4.73 MB/sec execute 28 sec latency 25.928 ms 3 14135 4.92 MB/sec execute 29 sec latency 24.890 ms 3 14353 4.90 MB/sec execute 30 sec latency 19.792 ms 3 14537 4.85 MB/sec execute 31 sec latency 24.122 ms 3 14692 4.71 MB/sec execute 32 sec latency 21.461 ms 3 14909 4.59 MB/sec execute 33 sec latency 57.859 ms 3 15200 4.54 MB/sec execute 34 sec latency 23.203 ms 3 15409 4.48 MB/sec execute 35 sec latency 24.061 ms 3 15690 4.61 MB/sec execute 36 sec latency 27.037 ms 3 15883 4.50 MB/sec execute 37 sec latency 30.124 ms 3 16066 4.39 MB/sec execute 38 sec latency 24.904 ms 3 16337 4.30 MB/sec execute 39 sec latency 19.576 ms 3 16652 4.33 MB/sec execute 40 sec latency 23.368 ms 3 17048 4.52 MB/sec execute 41 sec latency 22.928 ms 3 17436 4.59 MB/sec execute 42 sec latency 23.949 ms 3 17752 4.72 MB/sec execute 43 sec latency 19.196 ms 3 17976 4.71 MB/sec execute 44 sec latency 20.935 ms 3 18134 4.62 MB/sec execute 45 sec latency 21.557 ms 3 18302 4.52 MB/sec execute 46 sec latency 20.602 ms 3 18599 4.46 MB/sec execute 47 sec latency 68.546 ms 3 18832 4.46 MB/sec execute 48 sec latency 27.676 ms 3 19059 4.44 MB/sec execute 49 sec latency 25.836 ms 3 19296 4.47 MB/sec execute 50 sec latency 25.559 ms 3 19501 4.39 MB/sec execute 51 sec latency 26.984 ms 3 19719 4.32 MB/sec execute 52 sec latency 20.949 ms 3 19936 4.25 MB/sec execute 53 sec latency 19.391 ms 3 20168 4.24 MB/sec execute 54 sec latency 25.916 ms 3 20374 4.22 MB/sec execute 55 sec latency 24.831 ms 3 20757 4.39 MB/sec execute 56 sec latency 23.764 ms 3 21315 4.57 MB/sec execute 57 sec latency 24.048 ms 3 21602 4.56 MB/sec execute 58 sec latency 20.451 ms 3 21757 4.50 MB/sec execute 59 sec latency 21.219 ms 3 21927 4.43 MB/sec execute 60 sec latency 23.394 ms 3 22136 4.38 MB/sec execute 61 sec latency 44.428 ms 3 22384 4.36 MB/sec execute 62 sec latency 22.938 ms 3 22778 4.46 MB/sec execute 63 sec latency 20.187 ms 3 23160 4.40 MB/sec execute 64 sec latency 13.106 ms 3 23650 4.39 MB/sec execute 65 sec latency 11.342 ms 3 24193 4.57 MB/sec execute 66 sec latency 13.993 ms 3 24581 4.59 MB/sec execute 67 sec latency 23.995 ms 3 24886 4.67 MB/sec execute 68 sec latency 23.698 ms 3 25087 4.65 MB/sec execute 69 sec latency 20.798 ms 3 25262 4.61 MB/sec execute 70 sec latency 21.716 ms 3 25428 4.55 MB/sec execute 71 sec latency 23.315 ms 3 25661 4.50 MB/sec execute 72 sec latency 37.210 ms 3 25861 4.46 MB/sec execute 73 sec latency 23.783 ms 3 26074 4.45 MB/sec execute 74 sec latency 25.659 ms 3 26335 4.51 MB/sec execute 75 sec latency 24.908 ms 3 26534 4.46 MB/sec execute 76 sec latency 25.861 ms 3 26714 4.40 MB/sec execute 77 sec latency 27.352 ms 3 26947 4.36 MB/sec execute 78 sec latency 22.894 ms 3 27199 4.34 MB/sec execute 79 sec latency 18.495 ms 3 27415 4.33 MB/sec execute 80 sec latency 21.279 ms 3 27796 4.43 MB/sec execute 81 sec latency 24.060 ms 3 28169 4.45 MB/sec execute 82 sec latency 24.319 ms 3 28461 4.52 MB/sec execute 83 sec latency 18.134 ms 3 28675 4.52 MB/sec execute 84 sec latency 20.446 ms 3 28840 4.47 MB/sec execute 85 sec latency 21.782 ms 3 29012 4.42 MB/sec execute 86 sec latency 21.342 ms 3 29204 4.39 MB/sec execute 87 sec latency 37.561 ms 3 29379 4.35 MB/sec execute 88 sec latency 34.058 ms 3 29578 4.34 MB/sec execute 89 sec latency 26.115 ms 3 29817 4.36 MB/sec execute 90 sec latency 25.642 ms 3 30035 4.35 MB/sec execute 91 sec latency 26.055 ms 3 30214 4.31 MB/sec execute 92 sec latency 27.762 ms 3 30442 4.27 MB/sec execute 93 sec latency 18.965 ms 3 30702 4.25 MB/sec execute 94 sec latency 19.638 ms 3 30891 4.24 MB/sec execute 95 sec latency 28.784 ms 3 31252 4.32 MB/sec execute 96 sec latency 28.656 ms 3 31679 4.35 MB/sec execute 97 sec latency 25.949 ms 3 31975 4.41 MB/sec execute 98 sec latency 26.185 ms 3 32260 4.41 MB/sec execute 99 sec latency 21.675 ms 3 32477 4.38 MB/sec execute 100 sec latency 20.417 ms 3 32719 4.34 MB/sec execute 101 sec latency 46.715 ms 3 33050 4.34 MB/sec execute 102 sec latency 17.973 ms 3 33298 4.37 MB/sec execute 103 sec latency 25.046 ms 3 33512 4.36 MB/sec execute 104 sec latency 30.247 ms 3 33711 4.32 MB/sec execute 105 sec latency 29.941 ms 3 33914 4.28 MB/sec execute 106 sec latency 23.895 ms 3 34164 4.25 MB/sec execute 107 sec latency 20.247 ms 3 34386 4.25 MB/sec execute 108 sec latency 25.135 ms 3 34668 4.29 MB/sec execute 109 sec latency 27.366 ms 3 35080 4.33 MB/sec execute 110 sec latency 25.612 ms 3 35514 4.41 MB/sec execute 111 sec latency 24.258 ms 3 35794 4.41 MB/sec execute 112 sec latency 21.245 ms 3 35956 4.38 MB/sec execute 113 sec latency 21.778 ms 3 36154 4.35 MB/sec execute 114 sec latency 36.909 ms 3 36512 4.34 MB/sec execute 115 sec latency 26.448 ms 3 36971 4.40 MB/sec execute 116 sec latency 25.444 ms 3 37316 4.37 MB/sec execute 117 sec latency 18.154 ms 3 37806 4.36 MB/sec execute 118 sec latency 13.164 ms 3 38031 4.35 MB/sec execute 119 sec latency 26.447 ms 3 cleanup 120 sec 0 cleanup 121 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 16727 10.878 42.242 Close 12287 1.929 15.976 Rename 715 16.711 30.965 Unlink 3375 5.897 33.056 Qpathinfo 15134 2.911 25.283 Qfileinfo 2667 0.567 4.249 Qfsinfo 2797 7.966 46.164 Sfileinfo 1368 9.820 32.425 Find 5862 1.105 37.256 WriteX 8308 2.483 16.376 ReadX 26639 0.089 1.218 LockX 58 1.783 2.320 UnlockX 58 1.963 3.207 Flush 1168 9.091 68.536 Throughput 4.3526 MB/sec 3 clients 3 procs max_latency=68.546 ms stopping dbench on /mnt/lustre/d8.sanity-quota at Sun Aug 4 14:36:20 EDT 2024 with return code 0 clean dbench files on /mnt/lustre/d8.sanity-quota /mnt/lustre/d8.sanity-quota /mnt/lustre/d8.sanity-quota removed directory: 'clients/client1/~dmtmp/PARADOX' removed directory: 'clients/client1/~dmtmp/PM' removed directory: 'clients/client1/~dmtmp/WORDPRO' removed directory: 'clients/client1/~dmtmp/SEED' removed directory: 'clients/client1/~dmtmp/WORD' removed directory: 'clients/client1/~dmtmp/PWRPNT' removed directory: 'clients/client1/~dmtmp/EXCEL' removed directory: 'clients/client1/~dmtmp/COREL' removed directory: 'clients/client1/~dmtmp/ACCESS' removed directory: 'clients/client1/~dmtmp' removed directory: 'clients/client1' removed directory: 'clients/client0/~dmtmp/PARADOX' removed directory: 'clients/client0/~dmtmp/PM' removed directory: 'clients/client0/~dmtmp/WORDPRO' removed directory: 'clients/client0/~dmtmp/SEED' removed directory: 'clients/client0/~dmtmp/WORD' removed directory: 'clients/client0/~dmtmp/PWRPNT' removed directory: 'clients/client0/~dmtmp/EXCEL' removed directory: 'clients/client0/~dmtmp/COREL' removed directory: 'clients/client0/~dmtmp/ACCESS' removed directory: 'clients/client0/~dmtmp' removed directory: 'clients/client0' removed directory: 'clients/client2/~dmtmp/PARADOX' removed directory: 'clients/client2/~dmtmp/PM' removed directory: 'clients/client2/~dmtmp/WORDPRO' removed directory: 'clients/client2/~dmtmp/SEED' removed directory: 'clients/client2/~dmtmp/WORD' removed directory: 'clients/client2/~dmtmp/PWRPNT' removed directory: 'clients/client2/~dmtmp/EXCEL' removed directory: 'clients/client2/~dmtmp/COREL' removed directory: 'clients/client2/~dmtmp/ACCESS' removed directory: 'clients/client2/~dmtmp' removed directory: 'clients/client2' removed directory: 'clients' removed 'client.txt' /mnt/lustre/d8.sanity-quota dbench successfully finished lfs project -C /mnt/lustre/d8.sanity-quota Delete files... Wait for unlink objects finished... Waiting for MDT destroys to complete oleg141-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported oleg141-server: fstrim: /mnt/lustre-ost1: the discard operation is not supported pdsh@oleg141-client: oleg141-server: ssh exited with exit code 1 pdsh@oleg141-client: oleg141-server: ssh exited with exit code 1