Skip to main content

mErCS 1.3.0 and 1.2.0

How mErCS 1.3.0 compares with the previous release, 1.2.0: on the reference workload that the releases are accepted on, and on the matrix of single operations of the jecs page. 1.3.0 fixes the for-in loops of queries that pass over the bitsets when their body deletes entities ahead of them (1.0.0–1.2.0 returned the mark of the free slot, and in a run of full words did not end), at the cost of a comparison per returned entity, and gives OnChange hooks and changed listeners the value that the change replaced (see the changelog). Roblox Studio and the Roblox player were open on the machine during these measurements, which widens the noise of every row.

The reference workload​

The reference workload is the model of the work that the library is released against: P1-P9 at the small and the target scale, both retained-agent variants, the client and the buffers profiles, in the interpreter and in native code (tools/workload.py, Luau 0.740, -O2). Each build and its second run are a process of their own, five times in each mode; the noise of a row is measured on the two runs of 1.2.0 alone. The release report has all 232 rows with their ranges, and the raw samples.

  • Time: no row is slower than the noise. The release run flagged the stored queries at their periods (P4) at the target scale without owned entities in the interpreter (1.049×); P4 and the frame model (P7) were measured 15 more times at the four scales of the server in the interpreter, and with all repeats that row takes 1.041×, within the noise. These are the loops whose step 1.3.0 changes: in the interpreter P4 takes 1.025–1.041× and P7 1.015–1.025× of the time of 1.2.0, within the noise of each row; in native code the frame model takes 1.003–1.027× (1.014× at the target scale), and the client profile 1.00–1.03× in both modes.
  • Memory: the same. A world takes 56 586 bytes in both versions, the target world 137.63 MB after it is built and 138.57 MB after 100 lifecycle rounds, a distinct pair 1 527.8 bytes with its holder, and nothing accumulates over the lifecycle rounds and the long sessions. One row of P8 is flagged by its resolution of whole KiB: the bytes per distinct pair of the small scale in native code, one step of 1 KiB over its 594 pairs (+1.72 bytes; the same row moved one step down in the interpreter).

A frame of the model (P7, µs) and the heap it leaves per frame (P9, bytes), medians [min–max] of both runs of each version:

ModeScaleP7, µs/frame: 1.2.0 → 1.3.0P9, bytes/frame: 1.2.0 → 1.3.0
nativesmall99.34 [97.25–101.30] → 102.03 [98.46–111.24]4266.7 [4078.9–4386.1] → 4266.7 [4078.9–4386.1]
nativetarget338.89 [331.14–342.11] → 343.71 [338.49–361.42]7321.6 [6502.4–7714.1] → 7321.6 [6502.4–7714.1]
nativesmall-empty-retained100.18 [96.29–104.14] → 101.16 [99.27–106.58]4437.3 [4266.7–4522.7] → 4437.3 [4266.7–4522.7]
nativetarget-empty-retained324.88 [316.72–383.50] → 325.89 [316.38–343.18]6126.9 [5751.5–6963.2] → 6126.9 [5751.5–6963.2]
interpretersmall158.06 [155.05–179.14] → 160.41 [156.68–182.35]4266.7 [4078.9–4386.1] → 4266.7 [4078.9–4386.1]
interpretertarget496.31 [485.18–529.37] → 508.52 [489.84–541.46]7321.6 [6502.4–7714.1] → 7321.6 [6502.4–7714.1]
interpretersmall-empty-retained157.12 [154.62–161.79] → 159.71 [155.88–229.24]4437.3 [4266.7–4522.7] → 4437.3 [4249.6–4522.7]
interpretertarget-empty-retained485.89 [474.93–509.91] → 496.10 [484.60–742.41]6126.9 [5751.5–6963.2] → 6126.9 [5751.5–6963.2]

The lifecycle events at the target scale (P6, µs per event):

ModeP6 target, µs per event: 1.2.0 → 1.3.0
nativea new agent with its owned entities893.6 [880.3–965.2] → 904.4 [881.7–930.9]
nativethe end of the activity of an agent52.8 [50.9–57.1] → 52.2 [51.3–54.2]
nativethe removal of a retained agent (cascade)1042.7 [1020.8–1148.6] → 1048.6 [1029.7–1073.7]
nativethe cascade of the removal of an agent, isolated263.0 [257.7–277.3] → 267.9 [256.1–273.2]
interpretera new agent with its owned entities1531.9 [1505.9–1601.4] → 1534.3 [1499.0–1565.0]
interpreterthe end of the activity of an agent87.6 [85.2–93.0] → 86.7 [85.9–90.7]
interpreterthe removal of a retained agent (cascade)1570.1 [1541.4–1648.4] → 1563.2 [1515.2–1646.0]
interpreterthe cascade of the removal of an agent, isolated444.0 [430.4–470.0] → 437.7 [422.7–446.0]

The same at the small scale:

ModeP6 small, µs per event: 1.2.0 → 1.3.0
nativea new agent with its owned entities43.62 [42.55–48.00] → 44.12 [42.70–47.95]
nativethe end of the activity of an agent2.93 [2.70–3.45] → 3.10 [2.80–3.65]
nativethe removal of a retained agent (cascade)48.57 [44.00–56.45] → 52.15 [49.60–56.80]
nativethe cascade of the removal of an agent, isolated10.10 [10.00–10.90] → 10.20 [10.00–12.10]
interpretera new agent with its owned entities66.10 [64.35–67.80] → 67.30 [63.50–68.95]
interpreterthe end of the activity of an agent4.20 [3.90–5.35] → 4.80 [3.70–5.85]
interpreterthe removal of a retained agent (cascade)73.25 [70.55–79.10] → 75.47 [67.70–82.65]
interpreterthe cascade of the removal of an agent, isolated16.85 [16.50–19.80] → 17.05 [16.20–23.50]

The target world (P8):

ModeP8 target1.2.0 → 1.3.0
nativethe heap of the world after it is built137.63 [137.62–145.61] → 137.63 [137.62–145.61]
nativethe heap after 100 lifecycle rounds138.57 [138.56–146.56] → 138.57 [138.56–146.56]
nativebytes per distinct pair (and its holder)1527.808 [1527.808–1528.832] → 1527.808 [1527.808–1528.832]
nativebytes per holder of a pair37.139 [36.882–37.395] → 37.139 [37.139–37.395]
interpreterthe heap of the world after it is built137.63 [137.62–145.61] → 137.63 [137.62–145.61]
interpreterthe heap after 100 lifecycle rounds138.57 [138.56–146.56] → 138.57 [138.56–146.56]
interpreterbytes per distinct pair (and its holder)1527.808 [1527.808–1528.832] → 1528.320 [1527.808–1528.832]
interpreterbytes per holder of a pair37.139 [36.882–37.395] → 37.139 [36.882–37.395]

Single operations​

bench/run.luau and the probes of the jecs page, measured in the same runs as that page (mErCS 1.2.0 from its git tag): every library in a process of its own, fifteen times in each mode for the matrix and the synthetic frame and five for the other profiles, the order rotated; medians. Cells read "interpreter / native"; the ratio is the time of 1.3.0 divided by that of 1.2.0, lower is better. The complete measurements include the ranges.

The median ratio of the 95 rows that have one is 1.00× in both modes. The for-in loops of queries over dense matches pay the comparison per returned entity: a query of one value of an entity with 4 takes 40.1 / 29.2 instead of 36.9 / 27.9 ns per match (1.09× / 1.05×), four values over 8192, 256 and 16 entities per archetype 1.04–1.06× in the interpreter and 0.98–1.04× in native code, a query created and iterated once 1.06× / 1.06×, a query with 10 terms 1.04× / 0.99×; query:each and the loops over a match list or a sparse match run the code of 1.2.0 (0.95–1.01×). The other rows that move beyond a few percent move both ways within their ranges (pair(R, T) in native code, 0.86×). The cost of an OnRemove hook on a delete is 176 / 104 ns in 1.2.0 and 172 / 108 ns in 1.3.0 (the probe of the jecs page).

A synthetic game frame​

Mode / queriesmErCS 1.2.0mErCS 1.3.0ratio
interpreter, inline queries2.53 ms2.54 ms1.01×
interpreter, cached queries2.49 ms2.54 ms1.02×
native, inline queries1.74 ms1.76 ms1.01×
native, cached queries1.75 ms1.78 ms1.02×

A long session​

ModeFramemErCS 1.2.0mErCS 1.3.0ratio
interpreter1000+0.33 MiB, 0.094 ms+0.33 MiB, 0.094 ms—
interpreter3000+0.32 MiB, 0.095 ms+0.32 MiB, 0.094 ms—
interpreter5000+0.32 MiB, 0.094 ms+0.32 MiB, 0.093 ms—
native1000+0.33 MiB, 0.062 ms+0.33 MiB, 0.064 ms—
native3000+0.32 MiB, 0.062 ms+0.32 MiB, 0.062 ms—
native5000+0.32 MiB, 0.063 ms+0.32 MiB, 0.063 ms—

Query cases​

CasemErCS 1.2.0mErCS 1.3.0ratio
Scattered after recycling93.3 / 65.8 µs93 / 66.1 µs1.00× / 1.01×
Churn on a queried tag, 1 toggle per pass296 / 213 µs297 / 214 µs1.00× / 1.01×
Churn on a queried tag, 100 toggles per pass280 / 198 µs277 / 196 µs0.99× / 0.99×
Read 4 components1.34 ms / 678 µs1.35 ms / 679 µs1.00× / 1.00×
(*, T) with churn545 / 339 µs543 / 335 µs1.00× / 0.99×
Usually empty43.9 / 28.4 ns43.3 / 28.3 ns0.99× / 1.00×

A sparse world​

PassmErCS 1.2.0mErCS 1.3.0ratiomErCS 1.2.0, poolmErCS 1.3.0, poolratio, pool
Alive creatures, nothing changed5.75 / 4.31 µs5.76 / 4.31 µs1.00× / 1.00×6.93 / 4.11 µs6.94 / 4.16 µs1.00× / 1.01×
Alive creatures, Dead added to one and removed7.62 / 5.46 µs7.53 / 5.46 µs0.99× / 1.00×7.48 / 4.38 µs7.43 / 4.42 µs0.99× / 1.01×
Running creatures, Run toggled on one4.92 / 3.61 µs4.88 / 3.62 µs0.99× / 1.00×2.97 / 1.84 µs2.93 / 1.89 µs0.99× / 1.03×
A tag added to one creature, a pass, removed, a pass2.17 / 1.39 µs2.09 / 1.42 µs0.97× / 1.02×1.73 / 1.07 µs1.71 / 1.05 µs0.99× / 0.98×
A tag nobody has45.1 / 30.4 ns45.1 / 30.1 ns1.00× / 0.99×44.8 / 30.3 ns45.5 / 30.5 ns1.02× / 1.01×
All with Model (creatures and corpses)19.9 / 14.9 µs20 / 14.9 µs1.00× / 1.00×16 / 9.11 µs15.9 / 9.13 µs1.00× / 1.00×
A skill changes state, 10 queries over skills in Cast110 / 69.9 µs112 / 71.7 µs1.02× / 1.03×112 / 71.4 µs118 / 71.6 µs1.06× / 1.00×

State tags​

Frame of 1200 skills, 5 inline queriesmErCS 1.2.0mErCS 1.3.0ratio
10 skills change state41.9 / 24.4 µs40.7 / 24.7 µs0.97× / 1.01×
200 skills change state168 / 97.7 µs165 / 97.8 µs0.98× / 1.00×
800 skills change state371 / 222 µs369 / 221 µs0.99× / 0.99×

The strengths of jecs​

#CasemErCS 1.2.0mErCS 1.3.0ratio
1(*, T) values, 32 data relations, a pass over 20 000559 / 319 µs552 / 318 µs0.99× / 1.00×
2(R, *) values, 8 targets, a pass over 20 000552 / 319 µs546 / 317 µs0.99× / 1.00×
315 cached queries over scattered entities, per query740 / 329 µs754 / 334 µs1.02× / 1.01×
the same, memory per query263 KiB263 KiB1.00×
4usually empty, 20 swaps per pass (100 000 entities)13.7 / 8.12 µs14.1 / 7.62 µs1.03× / 0.94×
5 queries, 100 matches each, 40 toggles per frame32.9 / 17.6 µs33.6 / 17.8 µs1.02× / 1.01×
520 small queries, a shared tag toggled on 30 per frame23.2 / 14.8 µs22.7 / 14.9 µs0.98× / 1.01×
6query(A, pair(ChildOf, parent)) per parent (1000 × 5)590 / 488 ns594 / 498 ns1.01× / 1.02×
2 inline queries per new caster4.37 / 3.73 µs4.43 / 3.74 µs1.01× / 1.00×
the same, memory per caster377 B377 B1.00×
7world:each over 100 000, per entity31.6 / 25.2 ns31.7 / 24.8 ns1.00× / 0.98×
world:children, 1000 children, per child32.9 / 26.3 ns32.9 / 25.8 ns1.00× / 0.98×
88 kinds in turn, 3 values, for-in, per match90.4 / 85 ns87.7 / 83 ns0.97× / 0.98×
the same, each66.8 / 54.3 ns64.8 / 54.7 ns0.97× / 1.01×
the same, each, the kind from a slot pool36.9 / 21.6 ns37.3 / 21.2 ns1.01× / 0.98×
9delete entities with 100 targets, per pair64.8 / 46.4 ns64.8 / 46.4 ns1.00× / 1.00×
108 components on 1 entity in 64 of 131 072, memory per entity661 B661 B1.00×
the same, time per entity2.49 / 1.58 µs2.49 / 1.56 µs1.00× / 0.99×

Query monitors​

CasemErCS 1.2.0mErCS 1.3.0ratio
a tag of the query added and removed, per cycle (an entry and an exit)803 / 540 ns803 / 541 ns1.00× / 1.00×
members of the query deleted, per entity671 / 398 ns675 / 395 ns1.01× / 0.99×
children moved to another parent, (ChildOf, *) monitored, per move743 / 453 ns738 / 453 ns0.99× / 1.00×

Entities​

ScenariomErCS 1.2.0mErCS 1.3.0ratio
create an entity43.6 / 28.6 ns43.3 / 28.6 ns0.99× / 1.00×
delete an entity with 4 components455 / 240 ns451 / 240 ns0.99× / 1.00×

Components​

ScenariomErCS 1.2.0mErCS 1.3.0ratio
add a first component177 / 109 ns172 / 109 ns0.97× / 1.00×
set an existing component61.4 / 36.7 ns61.8 / 36.7 ns1.01× / 1.00×
remove a component118 / 59.5 ns118 / 59.7 ns1.00× / 1.00×
add when the entity has 5128 / 62.2 ns127 / 61.8 ns0.99× / 0.99×
remove when the entity has 6115 / 58.6 ns116 / 59 ns1.01× / 1.01×
add when the entity has 20124 / 61.9 ns125 / 63.1 ns1.00× / 1.02×
remove when the entity has 21115 / 59.3 ns116 / 59.4 ns1.01× / 1.00×
add when the entity has 40123 / 59.4 ns123 / 60.2 ns1.00× / 1.01×
remove when the entity has 41116 / 59.1 ns116 / 58.9 ns1.01× / 1.00×

Tags​

ScenariomErCS 1.2.0mErCS 1.3.0ratio
add a tag95.2 / 52.7 ns95 / 52.7 ns1.00× / 1.00×
remove a tag108 / 64.1 ns109 / 64.2 ns1.00× / 1.00×
toggle a tag on 10 % of the entities per frame176 / 95.5 ns178 / 96.4 ns1.01× / 1.01×

Access​

ScenariomErCS 1.2.0mErCS 1.3.0ratio
get 1 id47 / 28 ns47.1 / 28.2 ns1.00× / 1.01×
get 2 ids58.7 / 33.7 ns59.1 / 33.7 ns1.01× / 1.00×
get 4 ids89.1 / 45.8 ns89.7 / 44.9 ns1.01× / 0.98×
get 8 ids (jecs: at most 4)243 / 112 ns244 / 112 ns1.00× / 1.00×
has 1 id51.1 / 31.8 ns51.4 / 31.7 ns1.01× / 1.00×
has 4 ids102 / 44.3 ns101 / 44.9 ns0.99× / 1.01×
has 8 ids (jecs: at most 4)186 / 72.6 ns186 / 72.5 ns1.00× / 1.00×

Pairs​

ScenariomErCS 1.2.0mErCS 1.3.0ratio
pair(R, T)23.1 / 10.4 ns22.9 / 8.95 ns0.99× / 0.86×
add a pair275 / 171 ns278 / 172 ns1.01× / 1.00×
set the value of an existing pair68.3 / 45.7 ns68.3 / 45.3 ns1.00× / 0.99×
remove a pair200 / 112 ns201 / 112 ns1.00× / 1.00×
target (4 pairs)63.1 / 36.6 ns62.6 / 36.1 ns0.99× / 0.98×
ChildOf children, per child (10 000 parents × 5)140 / 104 ns140 / 101 ns1.00× / 0.98×
add a pair with a target of its own3.17 / 2.84 µs3.16 / 2.85 µs1.00× / 1.00×
delete a target: Remove (1000 × 100 sources, per source)156 / 82.5 ns154 / 83.2 ns0.99× / 1.01×
delete a parent: ChildOf cascade (per child)407 / 240 ns401 / 238 ns0.98× / 0.99×

Queries (per entity or match)​

ScenariomErCS 1.2.0mErCS 1.3.0ratio
for-in, 4 values, 8192 entities per archetype48.2 / 33.7 ns51.1 / 35.2 ns1.06× / 1.04×
query:each, the same (ratio: against the jecs for-in)32.3 / 16 ns32.1 / 16.1 ns0.99× / 1.01×
manual loop (spans / jecs archetypes), the same14.2 / 5.23 ns14.2 / 5.12 ns1.00× / 0.98×
for-in, 4 values, 256 entities per archetype52.1 / 35.8 ns54.5 / 36.7 ns1.05× / 1.03×
query:each, the same (against the jecs for-in)35.5 / 17.7 ns35.3 / 17.6 ns1.00× / 1.00×
manual loop, the same17.6 / 6.57 ns17.4 / 6.51 ns0.99× / 0.99×
for-in, 4 values, 16 entities per archetype68.1 / 42.5 ns70.8 / 41.9 ns1.04× / 0.98×
query:each, the same (against the jecs for-in)49.7 / 21.4 ns49.2 / 21.5 ns0.99× / 1.00×
manual loop, the same32 / 9.79 ns32 / 9.88 ns1.00× / 1.01×
for-in, 4 values, 1 entity per archetype74.3 / 47 ns76.6 / 46.1 ns1.03× / 0.98×
query:each, the same (against the jecs for-in)53.9 / 26.4 ns54.3 / 26.6 ns1.01× / 1.00×
manual loop, the same37.4 / 14.6 ns37.5 / 14.5 ns1.00× / 0.99×
a query created and iterated once48.8 / 33.8 ns51.8 / 35.8 ns1.06× / 1.06×
for-in, 1 value of an entity with 436.9 / 27.9 ns40.1 / 29.2 ns1.09× / 1.05×
for-in with without (half excluded)37.3 / 31.2 ns37.4 / 30.9 ns1.00× / 0.99×
query:each, the same (against the jecs for-in)22.1 / 13.3 ns22 / 13.2 ns0.99× / 1.00×
for-in over a sparse match (1 %)69 / 58.5 ns65.9 / 57.3 ns0.95× / 0.98×
query:each, the same (against the jecs for-in)45.2 / 31.1 ns45 / 30.8 ns0.99× / 0.99×
the first loop of a new query (2000 archetypes), per query430 / 338 ns421 / 339 ns0.98× / 1.00×
a small query created on every call (15 of 1000), per query1.2 / 1.07 µs1.21 / 1.06 µs1.00× / 1.00×
a query of a concrete pair created on every call (15 of 1000), per query1.05 µs / 890 ns1.05 µs / 881 ns1.00× / 0.99×
the same, nothing matches (0 of 1000)172 / 149 ns169 / 149 ns0.99× / 1.00×
an entity used as a term in 4 inline queries, then deleted, per entity5.87 / 5.21 µs5.82 / 5.27 µs0.99× / 1.01×
a query with 10 terms98.3 / 61.4 ns103 / 60.6 ns1.04× / 0.99×

Batch operations (per match; jecs: a collect-and-change loop)​

ScenariomErCS 1.2.0mErCS 1.3.0ratio
add a tag to the matches (half of the entities)37.1 / 13.4 ns37.7 / 13.5 ns1.01× / 1.01×
set a component on the matches72.5 / 27.3 ns72.2 / 27.5 ns1.00× / 1.01×
remove a component from the matches39.4 / 12.9 ns38.6 / 12.8 ns0.98× / 0.99×
delete the matches482 / 251 ns487 / 246 ns1.01× / 0.98×
count the matches7.18 / 2.52 ns7.16 / 2.49 ns1.00× / 0.99×

Churn (per cycle)​

ScenariomErCS 1.2.0mErCS 1.3.0ratio
spawn and despawn (10 components, 2 tags)2.4 / 1.21 µs2.43 / 1.19 µs1.01× / 0.98×
a hierarchy of 1 + 5 ChildOf, spawned and deleted10.9 / 7.29 µs11 / 7.31 µs1.01× / 1.00×

Memory and garbage​

Kept per unitmErCS 1.2.0mErCS 1.3.0ratio
an empty entity32 B32 B1.00×
an entity with 4 components131 B131 B1.00×
an entity with 10 components and 5 tags236 B236 B1.00×
a ChildOf child (10 000 parents × 5)344 B344 B1.00×
an entity with a tag of its own978 B978 B1.00×
a pair with a target of its own1457 B1457 B1.00×
a component of 1000 on 100 entities each, per add162 B162 B1.00×
growth per hierarchy spawn / despawn cycle0 B0 B—
an entity used in 4 inline queries, then deleted1 B1 B1.00×
LoopmErCS 1.2.0mErCS 1.3.0ratio
------------
Inline two-component query80 B80 B1.00×
Stored query0 B0 B—
world:each152 B152 B1.00×