mirror of https://github.com/OpenIdentityPlatform/OpenDJ.git

Valery Kharseko
yesterday c4a377747777bf18869c389df8b4f9a099541ac6
refs
author Valery Kharseko <vharseko@3a-systems.ru>
Friday, October 2, 2026 10:56 +0200
committer GitHub <noreply@github.com>
Friday, October 2, 2026 10:56 +0200
commitc4a377747777bf18869c389df8b4f9a099541ac6
tree d32a39191b098c5a5a948c25b72a11f183ac86c4 tree | zip | gz
parent cdecf531161639f2f9fe98b5a8cc3facd43bb116 view | diff
[#1150] Read the benchmark JTL with a CSV parser, so that an error whose message holds a comma reaches the step log (#1151)

Fixes #1150

### Problem

`compare-opendj.sh` listed the errors of each benchmark side with `awk
-F','`. JMeter quotes a `responseMessage`
that contains a comma, and every message naming a DN does, so the
columns after it shifted and such rows were
never matched as failures. On the PDB side this hid the result-80 and
result-32 rows of #1149 and left only
`BIND | 49`, the last consequence of a failed ADD.

### Change

The error listing in `bench_one` now reads the JTL with `python3`'s
`csv` module, which is already used by the
same Docker jobs in `build.yml`.

- Failures are grouped by operation, result code and message, with
digits in the message replaced by `N`. The
messages carry the entry DN or the elapsed milliseconds, and without
this each failed row would be a kind of its
own.
- The step log shows the number of failed rows and kinds of error, the
ten most frequent kinds and, when there are
more, how many are left out.
- The listing still goes to stderr, because stdout of `bench_one`
carries the server version. `|| true` keeps a
failure of the listing from stopping the benchmark.

The `err` columns of the step summary come from `statistics.json`
through `summary.sh` and are unchanged.

### Verification

The block, cut verbatim from the script, run on the
`benchmark-pdb-vs-je` artifacts named in the issue:

| JTL | Before (awk) | After |
|---|---|---|
|
[36845050319](https://github.com/OpenIdentityPlatform/OpenDJ/actions/runs/36845050319),
PDB | `2 BIND \| 49` | 12 failed rows, 6 kinds of error:
DELETE/ADD/READD 80, COMPARE/MODIFY 32, BIND 49 |
|
[36738515054](https://github.com/OpenIdentityPlatform/OpenDJ/actions/runs/36738515054),
PDB | `5131 BIND \| 800`, `2 BIND \| 49` | 5145 failed rows, 7 kinds of
error, with the 80 and 32 rows present |
| JE side and both build-vs-release runs | nothing | nothing (no failed
rows) |

Output for run 36845050319, PDB:

```
[b] 12 failed rows, 6 kinds of error (count | op | code | message):
3 DELETE | 80 | pdb: backend 'userRoot' did not apply the transaction after N attempts in N ms, the attempt cap being spent; the last conflict was com.persistit.exception.RollbackException
2 ADD | 80 | pdb: backend 'userRoot' did not apply the transaction after N attempts in N ms, the attempt cap being spent; the last conflict was com.persistit.exception.RollbackException
2 COMPARE | 32 | The specified entry mail=u_N_N@test.com,ou=People,dc=example,dc=com does not exist in the Directory Server
2 MODIFY | 32 | Entry mail=u_N_N@test.com,ou=People,dc=example,dc=com cannot be modified because no such entry exists in the server
2 BIND | 49 | Invalid Credentials
1 READD | 80 | pdb: backend 'userRoot' did not apply the transaction after N attempts in N ms, the attempt cap being spent; the last conflict was com.persistit.exception.RollbackException
```

A synthetic JTL with 13 kinds of error, messages holding commas and one
message spanning two lines printed ten
kinds and `... 3 more kinds not shown`. When the JTL is missing or
malformed, the block exits with 0. In every case
nothing was written to stdout. `bash -n` passes.

#1145 edits the same file in a different place (the `sysctl` near the
top), so the two do not conflict.
1 files modified
26 ■■■■■ changed files
.github/benchmark/compare-opendj.sh 26 ●●●●● diff | view | raw | blame | history