Template:DropTest
Mission tested: Beyond the Call  [edit]
| |||
| By: CodeHydro Date(s): May 2016 Runs: 40 Cost/Run: 12
 | |||
| Item | Units | / unit |
Runs/Drop |
|---|---|---|---|
| Klingon K'Vort Bird-of-Prey Schematic (x3) | 33>> Klingon K'Vort Bird-of-Prey Schematic Average (mean) runs per x3 drop: 4 Based on test averages, to get one more x3 drop,
That is, 1 in 100 players may not see this item Also, a run dropping only this item is expected
|
14.5Statistical Strength: Somewhat unreliable Range of average cost per Klingon K'Vort Bird-of-Prey Schematic within 2 standard deviations (~95.5% confidence): 9.22 — 34.4>> | |
| Common Coffee ★ | 10>> Common Coffee ★ Average (mean) runs per drop: 4 Based on test averages, to get one more drop,
That is, 1 in 100 players may not see this item Also, a run dropping only this item is expected
|
48.0Statistical Strength: Somewhat unreliable Range of average cost per Common Coffee ★ within 2 standard deviations (~95.5% confidence): 29.85 — 122.47>> | |
| Basic Laser Scalpel | 10>> Basic Laser Scalpel Average (mean) runs per drop: 4 Based on test averages, to get one more drop,
That is, 1 in 100 players may not see this item Also, a run dropping only this item is expected
|
48.0Statistical Strength: Somewhat unreliable Range of average cost per Basic Laser Scalpel within 2 standard deviations (~95.5% confidence): 29.85 — 122.47>> | |
| Common Furs ★ | 36>> Common Furs ★ Average (mean) runs per drop: 2 Based on test averages, to get one more drop,
That is, 1 in 100 players may not see this item Also, a run dropping only this item is expected
|
13.3Statistical Strength: Fairly reliable Range of average cost per Common Furs ★ within 2 standard deviations (~95.5% confidence): 10.42 — 18.52>> | |
| Basic Alcohol |
53>> Basic Alcohol Average (mean) runs per drop: 1 Based on test averages, to get one more drop,
That is, 1 in 100 players may not see this item Also, a run dropping only this item is expected
|
9.1Statistical Strength: Fairly reliable Range of average cost per Basic Alcohol within 2 standard deviations (~95.5% confidence): 7.51 — 11.41>> | |
|
Average cost per unit assumes 3 standard rewards per run. Please Do NOT include results from "Double-Up" Adwarps. Double-Up results are not random and will skew results. This is a demo! Also note that item order has been rearranged and the multiplier removed so that it may be more easily compared against the current version. | |||
News
2016-7-26
- Hovering your pointer over the Units column now shows a tooltip that gives information about average runs per drop and streak tendency of each item.
- New #Single-item Form, allowing items from multiple DropTested missions to be summarized in a sortable table.
- Wrapping expressions in #expr tags is no longer necessary for the parameters that commonly use them. (i.e. may use
A + Binstead of{{{#expr: A + B}}) - Byline now links to contribution page instead of user talk.
- New parameter
|adjustments=(for experienced testers only, see #Adjustments for usage and rules). - Revised guidelines. Please re-read if you haven't yet ;)
2016-06-22
Tooltips and Color Indicator
There are two areas where you may hover your pointer to obtain additional information.
- Units column
- Cost / unit column
Hover your pointer over the number of units dropped of an item to see the average runs need per drop and some detailed "streak" information. Keep in mind those values are only as good as the reliability indicated in the next column.
You can hover over any average cost value for the range of costs at 2σ (~95.5%) confidence.
| Color | Meaning |
|---|---|
| none | Result is very reliable (average 2σ error < 20% of value, or error range < 25% of cost) |
| Yellow | Result is fairly reliable (average 2σ error < 45% of average, or error range < 50% of cost) |
| Orange | Result is somewhat unreliable (average 2σ error < 100% of value) |
| Red | Result is very unreliable (average 2σ error < 250% of value) |
| Dark Red | Result is nearly useless. (Seriously, don't expect the "true" average cost to be anything near the value shown.) |
Note that reliability merely indicates how close the test average cost is likely to be to the "true" average cost given an infinite number of runs; it does not indicate that you could reliably obtain a single unit after spending that amount of chronitons or tickets. Funny thing about averages is that roughly half of all cases will be above average, and half will be below... almost none will be exactly average.
Usage
This template provides a standard form for presenting test results and performs many calculations automatically. It also has some built-in warnings to ensure the validity of test results.
It is strongly recommended that you list items in order shown in the "travel" window ([Warp 1] [Warp Max] [Travel]). Mission rewards are always shown in that order in the results screen, making data entry easier.
New data template should always be created via the "Create one!" links generated by Template:DropTest/multi. The code below is provided as a reference (and may be used to reset a template, though may not be synced with the current default preloader)
<!--Use "Create" links. Do not modify this line:-->{{DropTest | mission = {{subst:#titleparts:{{subst:FULLPAGENAME}}|1|2}} | level = {{subst:#switch:{{subst:#titleparts:{{subst:FULLPAGENAME}}|1|3}}|1=normal|2=elite|3=epic}} | row = {{{row|}}}
<!-- chroniton/ticket cost/mission (enter 1 for cadet missions) -->
| cost =
<!-- username of tester(s); also by2, by3, etc. -->
| by = User1 <!--| by2/by3/etc. = {{subst:REVISIONUSER}} (substitutes your username on save)-->
<!-- total number of missions completed -->
| runs = 0 + 0
<!-- Date or date range of test. Just month and year. -->
| date = {{subst:CURRENTMONTHNAME}} {{subst:CURRENTYEAR}} {{--}} {{subst:CURRENTMONTHNAME}} {{subst:CURRENTYEAR}}
<!--//
Note: Do not list alphabetically. Please read Template:DropTest documentation and guidelines before using.
| item (use Template:Item) | total amount obtained | quantity obtainable per reward node (defaults to 1)
//-->
| {{item|item name|basic}} | 0 + 0 | 2
| | |
| | |
| | |
| | |
| | |
<!-- any remarks you want to add below the table -->
| note =
}}<!--//
If you wish to share raw data for individual runs, you may do so in this area.
//-->
Template Naming Convention
New DropTest templates should be named Template:DropTest/[mission]/[difficulty] where [mission] is the name of the mission and [level] is replaced by a number 1, 2, or 3 which represent normal, elite, and epic, respectively. That is, Template:DropTest/The Wrong Crowd/2 is for Elite difficulty of The Wrong Crowd.
No need to do this manually. Use Template:DropTest/multi on any mission page and it will generate a "Create one!" link that will automatically create a template of the appropriate title and which has some parameters pre-filled.
Tips and guidelines
Be sure read the core guidelines before gathering any data.
Core standards
- Record items in the order shown by the travel window; this matches the item order of the Success screen (but not in-mission reward crates). It just makes data entry easier and could help avoid/catch errors among similar-looking items.
- Note: Travel and success order parallelism may break down when missions drop distinct items of the same short name and rarity, as may Highwaymen
with ENT and VOY Starfleet Uniforms. For such cases, success window order may change from run to run. Continue using travel order.
- Note: Travel and success order parallelism may break down when missions drop distinct items of the same short name and rarity, as may Highwaymen
- Always resolve any Error 1 messages that show up after you edited a DropTest. Error 1 means there is incorrect data present (typos, failed nodes not giving a reward, etc.), so double-check everything, and if necessary, remove incorrect runs. You can ignore Error 2's, though, it's just a notification that a DropTest has only very few runs.
- Avoid sampling gaps–i.e., try not to have many unrecorded runs between the first and last recorded run. Also do NOT retroactively include results from memory. Both actions may introduce selection bias.
- Though DropTest takes only totals, you should record each run individually. That way, if you get #Error 1, you can just discard the anomalous run instead of the whole lot. (Just keep in mind each discard is a sampling gap!)
- Due to "luck", it is inevitable that even properly-conducted tests may yield off-the-mark results. That said, it is important never to manipulate data to"fit" your expectations or pre-existing results. It is not unusual for one tester to have results that are wildly different from that of another even if both testers have done a large number of runs. Both are important to share.
- If you have any doubts as to whether you entered a run correctly, delete the run from your data set. Do NOT delete a run just because it felt "too lucky".
- Do not use the adjustments parameter unless your data was gathered pursuant to the higher standards required.
Combining data
- When combining results from different testers, keep sources separate using expression notation
A + B + (C1 + C2). The order of the data sets (separated by +) should match that of the byline–that is, B is from by2 and C1 and C2 are both from by3. Example here. - Even though this template only displays the names of up to 4 testers, continue to use
|by5=|by6=.... Ideally the byline should be sorted by most runs to least, but simply arranging it so that the top 4 most runs are among the first 4 bys in any order is adequate. If by5+ has contributed less than 5 percent of the runs, you may discard that tester's data. (Larger, more contiguous data sets tend to be more reliable; thus, discarding weak sources may fortify results against selection bias.) - Please don't be a hero who refuses to take credit. The byline is not just for bragging rights but is an important tool for quickly identifying the source of each data set. (Makes adding and verifying data much easier).
- Some wikis have a policy of adding data only if your drop rate differ by, say, 10% or more. However, our in-house Vulcan has deemed this illogical since, when published results are "normal", it selectively prevents other normal results from inclusion (and ensures only abnormal data is shared). Instead, if your experience is drastically different, consider doing additional runs before publishing but publish anyhow and mention it on this template's talk page. Live long and prosper!
Best practices
The following are voluntary (optional) guidelines:
- To avoid selection bias, decide that you are going to submit data to the wiki for a run *before* actually doing the run. Never do a run first and then decide that it would be good to submit that run to the wiki
- Avoid "micro-publishing". Contributing very few runs may be more disruptive than helpful. Small data sets are highly susceptible to selection bias and clutter the data entry area, making it harder for others to contribute.
- Minimums depend on mission costs. As a rule of thumb, the sum of your total chronitons spent and your total runs should be at least 60–e.g. 12 run min for 4 runs (4*12 + 12 = 60) or a 3 run min for 20 runs (20 * 3 + 3 = 63 > 60). If less, just save your data elsewhere for now until you have a chance to do more runs. Disregard this minimum for limited-access missions such as cadet challenges or those locked by a trait.
- If you expect to share additional runs in the future after publishing initial runs, try to record every run of that mission to avoid sampling gaps. If you were unable to document some runs since your first contribution, hold off publishing until you have X runs without omitted runs such that X is the minimum described in the previous bullet point.
- Tip: If inconvenient to document some runs, take screenshots of the success screen so you may review them later.
- When testing a mission that drops an item you want, we ask–to reduce selection bias–that you not end your test the moment you obtain it; consider deciding beforehand "I will do X additional runs after getting Y" and stick to it. (Personally, I shoot for at least 3 extra runs. This suggestion is less important once a mission has over 100 runs or so.)
Adjustments
The |adjustments= parameter allows the inclusion of valid runs that normally trigger #Error 1 and avoids creating sampling gaps. Because Error 1 is the first line of defense against typos, this parameter is not included in the standard preloaded form to discourage the inappropriate suppression of that warning.
There are strict ground rules for using it:
- Only experienced users who have contributed to 10+ different DropTest data templates may use adjustments.
- If you do not meet this requirement, do NOT modify the adjustments parameter (except as needed for keeping data in the same order as the byline); simply exclude runs which cause Error 1 from your published values. (Keep the runs in your raw data for now; you may include them in your published values once you qualify if they were recorded in accordance with every rule in this section.)
- When gathering data, record adjustments as a separate pseudo-item (example) and be sure to take clear notes which explain why the standard reward count for that run is different. (In lieu of notes, you may have multiple pseudo-items such as "rares" and "failed/missed"). Valid reasons include:
- A) Rare reward recieved instead of standard reward
- B) Failed/missed a reward node due to locked trait or because getting a rare reward requires a sub-optimal path
- Note: Ignorance of a better path is NOT a valid reason because one cannot distinguish runs where one received max rewards by accidentally stumbling across the best path from runs where one accidentally typed 2 instead of 1. (Remember: If in doubt, delete the run.)
- C) A node gave double rewards. (Extremely rare, but possible.) If this happens, you may record receiving a "-1" adjustment pseudo-item in your raw data.
- Note: Do NOT give yourself the benefit of the doubt if you find excess rewards in runs not accompanied by a note or other proof that the extra reward is not the result of a typo.
- Devise some procedure to protect against the possibility that you may, for example, accidentally type "1" in the box for Rares instead of Clothing Pattern (or whatever item is next to your adjustment pseudo-item). Safeguards include putting an empty column/row between adjustments and standard rewards and/or redundancy (adding 1 to the reward box AND adding a separate note acknowledging your awareness of a rare reward).
- The adjustments parameter should be placed below the runs parameter with comments as shown here:
| runs = 152 + 79 + 13 + 5 <!-- Do not modify adjustments param without fully reading rules specific to adjustments at Template:DropTest --> | adjustments = (3-1) + -1 + 0 + 0
- If you find anyone using this parameter who might not be following the above rules, remove ALL data contributed by the user (or, if applicable, revert that user's data set to the version before he/she used adjustments).
As a final note, if only a handful of runs require adjustments and such runs are at the very beginning of your data set, consider simply omitting those runs from your published values instead of invoking the adjustments parameter. Remember. the main purpose of this parameter is to avoid sampling gaps between the first and last published run; thus, simply changing your starting point may be preferred for inexpensive missions.
Messages
Mission tested: An Illogical Result  [edit]
| |||
| By: anonymous-unreliable Date(s): [Missing] Runs: 3
| |||
| Item | Units | / unit |
Runs/Drop |
|---|---|---|---|
| how'd I get 7 if it drops 9 at a time?[note 1] (x9) | 7>>Unable to display streak info |
1.3Statistical Strength: Nearly useless Range of average cost per how'd I get 7 if it drops 9 at a time?[note 1] within 2 standard deviations (~95.5% confidence): 0.38 — Infinity>> |
0.3 |
| Common item ★ | 3>>Unable to display streak info |
3Statistical Strength: Nearly useless Range of average cost per Common item ★ within 2 standard deviations (~95.5% confidence): 1.58 — 24.86>> |
0.7 |
| Rare item ★★★ | 2>>Unable to display streak info |
4.5Statistical Strength: Nearly useless Range of average cost per Rare item ★★★ within 2 standard deviations (~95.5% confidence): 1.97 — Infinity>> |
1.1 |
|
Error 1: Totals do not match per node amounts. Results may be invalid. Error 2: Above results flagged as preliminary due to small sample size. * Units do not match runs. Averages calculated from 2.23 runs instead. Error 3: Adjustments exceed guidelines. Test flagged for manipulation risk. Of course, is it really an error if it was intentional?
| |||
Error 1
Basically, you see this message when your numbers don't add up. There are multiple ways this error may occur:
- A) You failed a reward node in one of your runs.
- B) You made a typo somewhere.
- C) One or more runs was sub-optimal (e.g. you took a path that hit only 3 reward nodes in one run but you looted 4 times in another run.)
- D) You forgot to enter or entered incorrectly the per-reward quantity of a plural drop.
- E) Disrupter Beam changed the quantity rewarded per drop.
- This happened on 1 June 2016, when missions suddenly stopped dropping more than one component at a time. Despite the change in amount, currently there is no evidence that drop frequency has changed; that is, if a mission dropped x3 Alcohol 50 times per hundred runs, it probably still drops alcohol 50 times per 100 runs, just only x1 instead. For now, using
{{#expr:}}tags to separate pre-update from new results and dividing old results by the former multiplier is acceptable. See example.
- This happened on 1 June 2016, when missions suddenly stopped dropping more than one component at a time. Despite the change in amount, currently there is no evidence that drop frequency has changed; that is, if a mission dropped x3 Alcohol 50 times per hundred runs, it probably still drops alcohol 50 times per 100 runs, just only x1 instead. For now, using
Warning 2
This message warns readers when the sample size is extremely small. It may be avoided by doing either:
- A) 40 runs or more.
- B) enough runs for all items to have been rewarded at least 4 times (a plural drop counts as one time).
If the above minimums seem lax, it is because this template is relatively new and not yet widely deployed. Thus, at the moment, increasing the number of missions with test results is more important than quality for single missions.
Error 3
This message appears when the total number of adjustments exceed 1/6th of the expected total number of rewards (drops). Because the adjustment parameter has high risk of distorting results, testers are STRONGLY discouraged from adding data requiring an adjustment value that could trigger this warning for missions costing less than 14 chronitons. If you spot this message on an inexpensive mission, please examine the page history to determine which editor triggered this message. Unless the editor left a good reason (as listed in the guidelines, please remove all data associated with that editor. Then kindly leave a message on that editor's talk page explaining this policy.
Note that "I did 100 runs of 3 rewards before realizing that 4 rewards were possible" is NOT a valid reason. (Someone careless enough to do that probably was not particularly careful in recording data.) A valid exception could be if a reward node were locked by a trait such that the editor did not have a crewman of sufficient skill for that node (for example, node 2.A in A Father Figure); for this reason to be acceptable, the editor must share raw data and list "locked" as a separate item.
Standardized disclosures
Sometimes DropTests get adjusted by DB, which will either modify the items that can be dropped, will adjust the drop rates, or will not influence the DropTest data at all. For these cases, specific tags can be added to each DropTest using Template:DropTest/note:
| First param | # of additional params |
Source example | Preview |
|---|---|---|---|
prejune
|
1 | |note={{dropTest/note|prejune
|basic Clothing Pattern
}}
|
|
update1.2
|
2 | |note={{dropTest/note|update1.2
|{{item|Starfleet Uniform (VOY)|0}} and {{item|Power Cell|0}}
|Starfleet Uniform (TNG) and Clothing Pattern, respectively
}}
|
|
dropAdjustAug
|
1 | |note={{dropTest/note|dropAdjustAug
|15.4, 23.6, 2.9, and 7.0
}}
|
|
dropAdjustMar17
|
1 | |note={{dropTest/note|dropAdjustMar17
|15.4, 23.6, 2.9, and 7.0
}}
|
|
| 2 | |note={{dropTest/note|dropAdjustMar17|itemsChanged=yes
|15.4, 23.6, 2.9, and 7.0
}}
|
Single-item Form
Short
Pass |row= to a DropTest data template, where row is the relevant row. Place each additional data template on a new line and then pass them all as the first parameter of Template:DropTest/rows (no spaces in front of each template):
==DropTests==
===Basic===
{{DropTest/rows|
{{DropTest/Beyond the Call/3| row = 1}}
{{DropTest/Putting the Free in Freedom/1| row = 1}}
}}
| Item | Units | Cost/Unit | Runs/Unit | Runs | From |
|---|---|---|---|---|---|
| Klingon K'Vort Bird-of-Prey Schematic (x3) | 555>> Klingon K'Vort Bird-of-Prey Schematic Average (mean) runs per x3 drop: 4 Based on test averages, to get one more x3 drop,
That is, 1 in 100 players may not see this item Also, a run dropping only this item is expected
|
12.6Statistical Strength: Very reliable Range of average cost per Klingon K'Vort Bird-of-Prey Schematic within 2 standard deviations (~95.5% confidence): 11.05 — 14.62>> |
1 | 582 |   Beyond the Call [edit]
|
| Klingon K'Vort Bird-of-Prey Schematic (x2) | 732>> Klingon K'Vort Bird-of-Prey Schematic Average (mean) runs per x2 drop: 4 Based on test averages, to get one more x2 drop,
That is, 1 in 100 players may not see this item Also, a run dropping only this item is expected
|
16.2Statistical Strength: Very reliable Range of average cost per Klingon K'Vort Bird-of-Prey Schematic within 2 standard deviations (~95.5% confidence): 14.72 — 17.95>> |
1.6 | 1184 |   Putting the Free in Freedom [edit]
|
The short form is preferred for item pages (example)
Long
Just set the first param as "+" of Template:DropTest/rows and in each DropTest data template insert "+" in front of the row value (|row=+#) to include [Expand] for full DropTest and last updated info. You may even add customized columns to further extend the table as much as you want using wikitable markup (though pipe characters will have to be escaped using {{{!}} or {{{!!}} and equal signs with {{{=}}).
{{DropTest/rows|+| ! My Runs !! class {{=}} "unsortable" {{!}} Remarks
{{DropTest/Beyond the Call/3| row = +1}} {{!!}} 318 {{!!}} something
{{DropTest/Putting the Free in Freedom/1| row = +1}} {{!!}} 0 {{!!}} I should share my data when I feel less lazy
}}
| Item | Units | Cost/Unit | Runs/Unit | Runs | From | Last Updated | My Runs | Remarks |
|---|---|---|---|---|---|---|---|---|
| Klingon K'Vort Bird-of-Prey Schematic (x3) | 555>> Klingon K'Vort Bird-of-Prey Schematic Average (mean) runs per x3 drop: 4 Based on test averages, to get one more x3 drop,
That is, 1 in 100 players may not see this item Also, a run dropping only this item is expected
|
12.6Statistical Strength: Very reliable Range of average cost per Klingon K'Vort Bird-of-Prey Schematic within 2 standard deviations (~95.5% confidence): 11.05 — 14.62>> |
1 | 582 | Beyond the Call [edit] Template loop detected: Template:DropTest/Beyond the Call/3
|
2019-12-5 imported>Joker41NAM | 318 | something |
| Klingon K'Vort Bird-of-Prey Schematic (x2) | 732>> Klingon K'Vort Bird-of-Prey Schematic Average (mean) runs per x2 drop: 4 Based on test averages, to get one more x2 drop,
That is, 1 in 100 players may not see this item Also, a run dropping only this item is expected
|
16.2Statistical Strength: Very reliable Range of average cost per Klingon K'Vort Bird-of-Prey Schematic within 2 standard deviations (~95.5% confidence): 14.72 — 17.95>> |
1.6 | 1184 | Putting the Free in Freedom [edit] Template loop detected: Template:DropTest/Putting the Free in Freedom/1
|
2019-07-9 imported>Joker41NAM | 0 | I should share my data when I feel less lazy |
Math mode
Add a "-" in front of the row value of each DropTest data template and it output only the average unit cost without tooltips or HTML. This also increases the output precision to 3 decimal points. The output may then be used in expressions.
Better: {{#ifexpr:{{DropTest/Beyond the Call/3|row=-1}}<{{DropTest/Putting the Free in Freedom/1|row=-1}}
|Beyond the Call ({{DropTest/Beyond the Call/3|row=-1}})
|Putting the Free in Freedom ({{DropTest/Putting the Free in Freedom/1|row=-1}})
}}
Better: Expression error: Unrecognized punctuation character "[".
See also
- Template:DropTest/multi - Automatic multiple templates
- Template:ItemX - Another item template by CodeHydro