Methods and calculations
How job postings become occupation-task profiles, related-occupation rankings and comparisons with observed transitions.
Worked USAJOBS notebooks: profiles → relatedness → Figure 2 · Paper code and released scores · Benchmark calculations in Python or Stata
The worked example uses another corpus. The sampling, review and benchmark rules below document the paper’s NLx analysis.
- Sample postings. Target 10% of each occupation’s October 2024–September 2025 job-months, with an annual minimum of 200 where available. Allocate the sample across months and weight each observation by the inverse sampling probability.
- Retrieve task IDs. TaskMatch compares text embeddings with O*NET task statements. Its matches identify existing tasks; they are not newly generated task descriptions.
- Filter and review assignments. Require five sampled job-months of support. Review selected additional task–occupation links with two generative models. Retain eligible unreviewed links and O*NET assignments; exclude rejected or incomplete non-O*NET reviews.
- Compare occupations. Weight tasks by their concentration in an occupation relative to all occupations (RCA), take square roots, normalize to unit length, and calculate cosine similarity.
- Check against observed transitions. Give both models the same number of recommendations for each starting occupation, up to 20, then measure how much of the observed movement they recover. Eligibility, occupational mappings and denominators differ by benchmark.
These are historical comparisons, not estimates of individual placement success. Detailed rules, population counts, sources and audit findings are available below.
Sample, weights, and scope
Sample, weights, and scope
The source is the monthly EXT posting frame for October 2024–September 2025, including January. The sampling unit is a job-month: a job present in two monthly tables can contribute twice. These are not counts of unique annual vacancies, employed workers, or all jobs in the economy.
For each occupation, the annual draw targets 10% of its frame, with a floor of 200 sampled job-months, capped at the available frame. Occupations with fewer than 200 frame job-months are taken in full. The annual target is allocated across nonempty months, first reserving one draw per month and then distributing the remainder in proportion to remaining monthly capacity, with integer rounding. The 200 floor is annual, not monthly; industry does not determine eligibility.
Occupation-month inclusion probability π = n / N
Design weight w = 1 / π = N / n
Here annual N is the occupation’s total frame count; in the last two formulas, N and n are the frame and draw counts for that occupation-month. Within each cell, records are selected by a seeded hash ordering. Weighted task incidence sums these weights over sampled job-months containing the task; raw counts are retained for support thresholds. Posting denominators include sampled records with no valid task IDs.
The weights undo the unequal sampling fractions within the observed posting frame. They do not correct which employers or jobs enter that frame, nor turn postings into a representative sample of workers or vacancies. Pooling all twelve months preserves the frame’s seasonal composition: months with more posting activity contribute more weight. This is not a seasonally adjusted estimate, and an occupation’s annual profile can reflect seasonal hiring.
Task support and priority review
Task support and priority review
The relatedness model includes an occupation–task assignment only if it has at least five sampled job-months of raw support and survives the recorded review exclusions. This support-five rule applies before weighting; it is distinct from the annual occupation sample floor and from the public disclosure threshold.
Tasks already linked to an occupation by O*NET are retained automatically. For other assignments, the review prompt gives the occupation title and description and the task statement, then asks whether the task plausibly belongs to that occupation. Two LLMs—Claude Sonnet 4.6 and GPT-4o (returned version 2024-08-06)—answer YES or NO at temperature zero. Acceptance requires both usable answers to be YES. A completed pair of answers with either NO is excluded. Attempted but incomplete reviews, including unusable responses or model-version mismatches, are quarantined and excluded; a failed review is not a substantive NO.
Which occupations are being counted?
Which occupations are being counted?
The counts below describe different layers of this release. Navigator entries combine two levels of the same classification; adding them does not create additional independent occupations or observations.
| Count | What it counts | How it is used |
|---|---|---|
| 1,016 specialties | Detailed eight-digit construction profiles. | All enter the pooled retained task-incidence reference for RCA, including specialties outside the fixed comparison population. This is a count of specialties across occupations, not the construction industry. |
| 912 specialties | The frozen common eligible detailed comparison population. | Established using annual sample support and availability of the compared representations, then held fixed during review corrections. It is narrower than the construction frame; eligibility is not a guarantee of assignment accuracy. |
| 790 SOC6 groups | Distinct six-digit parents of the 912 comparison specialties. | The broad six-digit illustration population. Individual validation benchmarks further restrict or remap eligible origins and destinations. |
| 1,883 navigator entries | 1,016 detailed specialties plus 867 six-digit parent entries. | The browsing catalog, which also offers entries outside the comparison population. A catalog entry does not imply an eligible ranking or a posting profile. |
| 1,880 posting-profile files | 1,013 detailed files plus all 867 pooled parent files. | Three detailed catalog entries have no posting-profile file: Entertainment Attendants and Related Workers, All Other (39-3099.00); First-Line Supervisors of Transportation Workers, All Other (53-1049.00); and Surgeons, All Other (29-1249.00). File presence does not guarantee released pair links. |
SST evaluation uses its principal matched transition universe and a bijective SOC crosswalk; EMSI evaluation uses its own principal destination coverage. Survey benchmarks use broader Census occupation categories (2010 for CPS/SIPP and 2002 for NLSY97/PSID), source-specific time horizons, and the saved minimum-40-observation origin rule. Their eligible origins and candidates therefore differ from the broad illustration and from one another. A missing benchmark value means unavailable evidence, not zero recovery. See validation definitions and results for those comparisons.
Benchmark construction and interpretation
Benchmark construction and interpretation
The audit traced released benchmark inputs through occupational crosswalks, pooling, ranking and equal-length evaluation. It reproduced all six headline comparisons in Python and Stata and reconstructed all 2,720 model-by-origin recommendation lists without differences from the frozen inputs. It found no headline arithmetic discrepancy in the checked release. It could not reconstruct proprietary resume records or the original survey preprocessing, imputation and weight calibration. The audit did not rerun extraction-model training or independently annotate its validation sets. The later reproduction package records additional source-release and model-matrix reconstruction checks.
Benchmark construction and reproduction · Reports, checking code and sensitivity results · Run the comparison in Python or Stata
Populations, outcomes and survey combinations
- SST: Resume-derived occupational flows based on adjacent-calendar-year co-occurrences, with the source's age weighting. Concurrent jobs may contribute; these are not necessarily direct job departures. One-to-one occupational crosswalks and model coverage retain 604 origins.
- EMSI: Recovery of the source's top 20 ranked destinations for 429 origins. Original flow counts are unavailable. Recovery divides the number of matched destinations by 20, even when the recommendation budget is shorter; it is neither precision nor a transition-weight share. The analysis uses the full released top-20 lists. Their destination sets differ from the separate source dashboard extract for 33 origins; the reason remains unresolved.
- CPS + SIPP: The source selects ages 18–44 and mid-level starting occupations (Job Zones 2–3.8). CPS compares the prior-year longest job with the current job; SIPP records monthly primary-job changes. Pooling does not make these a common-duration outcome.
- NLSY97 + PSID: The source selects ages 18–34 at occupational entry and mid-level starting occupations. Endpoints at months 35, 59 and 119 may follow several intervening moves. Nonworking/student endpoints and unchanged occupations are excluded. Results therefore concern observed moves to a different employed occupation, not employment success. The three horizons overlap and are not independent replications.
- Pooling: Append CPS and SIPP, or NLSY97 and PSID within each horizon, preserving the supplied combined-release weights. Do not normalize each survey to equal weight. NLSY97 contributes approximately 87%, 86% and 90% of retained transition weight at the three horizons; CPS contributes approximately 43% of CPS/SIPP weight. Equal-source sensitivity checks change the estimand and are not corrections to the source design.
Crosswalks, aggregation and comparison rules
- Review-state exception: Canonical O*NET membership takes precedence over judge rejection. The ledger contains 20 canonical pairs with at least one NO verdict that remain canonical-retained, subject to support filtering. Eligible never-reviewed links also remain. The released exclusions therefore do not mean every retained link passed both judges.
- Eligibility: Benchmarks use a fixed set of 912 eligible O*NET specialties, historical-code mappings and source-specific coverage restrictions. Ambiguous SST mappings are excluded; EMSI requires exact harmonized codes. Survey mappings require complete eligible specialty coverage and apply Census-code scope rules. Counts therefore differ across construction, navigation and evaluation.
- Survey support: An origin needs at least 40 retained changed-occupation records, not 40 distinct people or an effective sample size of 40. This yields 109 CPS/SIPP origins and 77, 76 and 65 trajectory origins. These restrictions further limit population coverage.
- Published O*NET comparator: Published rank r is converted to (21 − r)/20, with unlisted pairs assigned zero. Scores are averaged across all origin–destination specialty pairs within the evaluation groups, including zeros. This can rerank and combine related-occupation lists; it is more than deduplication. Self-links are removed and positive group scores are ranked.
- Posting-task groups: Group scores average specialty-pair cosine similarities with equal specialty weights, not posting-volume weights. Averaged group vectors are not renormalized: a grouped score is not the cosine of two renormalized group centroids.
- Equal length: For each origin, both models receive k = min(20, available posting-task destinations, available O*NET destinations). Outcomes do not choose k. Positive scores only are ranked, self-links are excluded, and ties in rounded ranking scores use ascending occupational codes. Recommendation budgets can therefore be below 20.
- Denominators: SST and survey capture retain observed transition weight to unmapped or otherwise ineligible destinations in the denominator. They do not renormalize to candidate-covered destinations. EMSI retains the original 20-destination denominator. Headline means weight origins equally; tied origins remain in the denominator of the share with greater recovery (difference tolerance 10⁻¹²).
Interpretation and uncertainty
These comparisons measure alignment with historical transitions, which precede the 2024–2025 posting sample. They are not untouched holdout tests. The RN example illustrates the calculation; it is not a randomly sampled validation case.
Reproduction scope
The public Python/Stata package recalculates recovery from released rankings, outcomes and denominators. Rebuilding national occupation-task profiles requires authorized NLx inputs. USAJOBS notebooks supply a public workflow using federal postings.
Source documentation: SST occupational mobility data; DOL CTOT public-use documentation; DOL methods report; O*NET related-occupations methodology. The detailed reports identify the local source files and code inspected.
How occupational relatedness is calculated
How occupational relatedness is calculated
Let mot be retained design-weighted task incidence for occupation o and task t. Model inputs require at least five supporting sampled job-months and apply the recorded review exclusions. The pooled reference uses all 1,016 specialties in the model-construction frame.
pt = Σo mot / Σo,u mou (pooled task share)
RCAot = qot / pt
vot = √RCAot / √(Σu RCAou)
S(o,d) = Σt votvdt
Square-root RCA coordinates are normalized to unit length; their dot product is cosine similarity. Six-digit group scores average the constituent specialty-pair cosines: group vectors are averaged without renormalizing them. The fixed comparison population comprises 912 specialties and 790 six-digit groups. These comparisons assess alignment with historical transitions; they are not tests on an untouched holdout sample.
What the displayed percentages mean
What the displayed percentages mean
Posting prevalence (%) = 100 × weighted task incidence / occupation posting weight
Transition-share recovery = Σ observed shares for recommended destinations
Top-20 destination recovery = number of observed top-20 destinations recovered / 20
Task contributions use the full pair score, even when only a few explanatory tasks are shown. Posting percentages can sum above 100%. Benchmark averages weight eligible origins equally; candidate sets and coverage differ by source. Missing evidence is not zero. Validation definitions and results →
What is public?
What is public?
“O*NET-linked” means O*NET assigns the task to the occupation. Other public task assignments have accepted review or retained support above 200 sampled job-months; six-digit identities also require permission in every contributing specialty. Public explanations are partial, while rankings retain the full research model. Statement tables and cloud exploration show up to 20 permitted posting tasks.
Technical definitions, coverage, disclosure, and attribution
What task information is public?
“O*NET-linked” means O*NET assigns that task to the selected occupation. In the downloadable data, the legacy status canonical_retained denotes a retained assignment covered by that O*NET link; “canonical” has this same meaning in historical documentation.
For each specialty, the public dataset contains tasks in its O*NET-linked list plus retained posting assignments accepted in completed review (reviewed_yes), or retained unreviewed assignments supported by more than 200 sampled job-months in that occupation. Raw support exactly 200 does not qualify through the support rule alone. The same rule applies to task tables, network endpoints, task explanations, cross-occupation lookups, and SVG downloads.
O*NET-linked tasks remain available regardless of posting frequency. Reviewed status applies to the task assignment in the focal occupation. A task being public in its O*NET source occupation does not reveal its occurrence in another occupation where it falls outside that occupation’s public allowance.
The additional-task cloud
Each dot represents an additional task linked by a positive released pair in the full research data. Other assignments have not been reviewed and are labeled “Not reviewed—task details not released.” They are represented only by counts grouped into coarse incidence-size categories. Dot positions are synthesized independently of task IDs. No original task identity, text, exact individual frequency, private position, or edge is supplied for anonymous dots. Aggregate cloud counts are retained. Exact cloud-incidence totals, unlinked shares, and rarefied detection statistics are withheld to prevent reconstructing an individual frequency when a cloud has few anonymous tasks. These aggregates are descriptive disclosure controls, not a formal differential-privacy guarantee.
The cloud allows exploration only of tasks among the selected occupation’s top 20 public posting tasks, ranked by posting percentage. Tasks already in the displayed network are omitted from the cloud. Other public linked tasks are gray, non-clickable dots of uniform size; withheld tasks are also gray but retain coarse size bands. Separate counts distinguish these groups. The cloud is a subset with retained co-occurrence links, not a count of all model tasks. Zooming, panning, equal sizing, and SVG export preserve the cloud representation. Positions do not measure semantic similarity.
Rankings and task networks
Related-occupation rankings retain the original full-data scores. Posting comparisons use design-weighted task counts, normalized shares, square-root transformation, inverse pooled retained-task-share weights (RCA), and cosine similarity. The reference task shares pool design-weighted retained support-five incidence across all 1,016 construction specialties (912 in the common eligible comparison population); each coordinate is sqrt(task share / pooled task share), normalized to unit length. The detailed comparison population contains 912 occupations; O*NET official lists may remain available for other sources. The public explanation shows only shared tasks permitted in both occupations. Its contribution percentage uses the full task cosine as its denominator.
The public network is rebuilt using only permitted endpoints of positive released pairs. Its components and compact core may differ from the unrestricted research version. Edge widths reflect sampled pair support, with a small minimum stroke for visibility; widths are scaled within the displayed network. The compact illustration offers a complementary view of tasks that co-occur within an occupation. It starts at the strongest pair in a selected connected component, grows to at most six tasks, and retains a maximum-support spanning tree plus additional strong links up to ten. This does not define an occupation’s essential or exhaustive core tasks. The principal between-occupation model compares RCA-weighted marginal task-frequency profiles by cosine similarity; neither these task-to-task edges nor the compact task selection enters that calculation. Missing links do not establish that tasks never occur together. Networks combine observations across jobs and are not individual job bundles.
Coverage and interpretation
The occupation-first posting sample covers October 2024–September 2025, including January. Counts are sampled job-months, not unique annual jobs. Completed priority-review corrections from September 2026 are applied; unreviewed not O*NET-linked assignments with support of 200 or fewer sampled job-months remain identifiable only in the private edition. Occupation rankings retain the full corrected data, including undisclosed tasks.
Teal network nodes and outlined teal cloud circles can be clicked to read task statements. Pale gray cloud dots are outside the top-20 exploration limit or have withheld details. A ring in the network marks tasks O*NET assigns to the selected occupation. These displays do not assess an individual’s abilities, credentials, or readiness for a move.
Sources and attribution
O*NET task statements and related-occupation information are used under CC BY 4.0. See the O*NET Database and O*NET 31.0 related-occupations documentation. The ranking comparisons, posting analysis, public filtering, and visualization are modifications for this research dashboard, not an official O*NET product or endorsement.
Six- and eight-digit occupation views
A clean page opens to Registered Nurses at six-digit SOC level, matching the paper illustration. Eight-digit O*NET view treats specialties separately. SOC6 frequencies pool specialty counts and denominators; task similarities are dot products of mean normalized specialty vectors, equivalently mean specialty-pair task cosines. Group vectors are not renormalized. Upper rankings use the full illustrative SOC6 population, while validation charts retain each benchmark’s eligible candidates. Group review acceptance requires every contributing retained specialty assignment to be accepted; O*NET task membership uses the SOC6 parent.
Why links may be unavailable
General pair links require at least 50 supporting sampled job-months within a specialty. Review exclusions remove pairs with excluded endpoints. An earlier extraction saved counts for every pair among 30 selected tasks in each of eight case occupations. Those files can supply positive pairs below the general 50-job-month threshold. This is an exception in the available source data, not a rule that forces the current illustration to display 30 tasks. SOC6 links sum available specialty pairs and cannot reconstruct suppressed pairs: counts are lower bounds on pooled co-occurrence. Missing links are unknown, not zero. “Task frequencies only” describes released evidence, not assignment validity.
Reading task frequencies
The statement table shows at most 20 tasks per tab, ordered by posting percentage, with task ID breaking ties. Search is limited to those 20. When posting frequencies are unavailable, O*NET tasks are ordered by task ID. The downloadable public data retain all permitted tasks. The statement table leads with the percentage of postings containing each task: design-weighted task incidence divided by all posting weight in the selected occupation. Percentages are not shares of task incidence and need not sum to 100%. Hover a percentage for raw support in sampled job-months. A support threshold does not establish substantive correctness.
Six-digit task identities additionally require permission in every contributing specialty. This complementary suppression prevents parent-minus-specialty reconstruction of hidden assignments. Rankings remain full research-model rankings; some explanatory tasks are withheld.
Sample units are job-months, not annual unique vacancies. Recorded .00 base profiles may include mechanically suffixed unspecified six-digit source labels; released aggregates cannot separate them. Missing released links are unknown. Six-digit maps sum only available released specialty links, leaving below-threshold pairs unobserved.
Worked example: individual tasks and tasks found together
Choices in representing occupational task bundles
Software Developers (15-1252.00) · 60,568 sampled job-months. The same six tasks illustrate two different representations: occupation-to-task model coordinates on the left and within-posting task pairs on the right. Labels are shortened; full statements and counts are below.
Occupational task bundle
How does the main model represent an occupation?
Occupation → task links. Width represents the normalized square-root RCA coordinate v, with a minimum visible stroke. Six entries from the full model profile are shown; they are not renormalized as a six-task bundle.
vot = √RCAot / √(Σu RCAou)
S(o,d) = Σt votvdt
q is the task’s share of retained task incidence in this occupation; p is its pooled share across the construction population. The sums use all model tasks. For task A, v = 0.025176, so v² = 0.000634: 0.063% of this occupation’s squared profile length. This is not a posting percentage or a contribution to an occupation-pair score; the latter requires a second occupation’s coordinate.
Occupational job-task bundle
Which duties are combined within the same postings?
Task → task links. Thicker lines mean more sampled job-months containing both tasks; node area represents individual task frequency. Ten strongest of 15 observed pairs shown. Widths use a different measure from the left panel and are not comparable across panels.
A promising direction for matching. Preserving combinations of duties could help distinguish jobs within the same occupation and identify experience with those combinations. This is a motivation for further analysis, not evidence of improved matching: pair-based profiles have not been evaluated against transitions or placement outcomes here.
Full task statements and all pair counts
Across the occupation
How often is each task detected? These individual frequencies describe an occupational task profile; they do not establish that the tasks appear together.
- A. Collaborate with system architects, software architects, design analysts, and others to understand business or industry requirements.
5,054 job-months (8.3%) - B. Assign duties or responsibilities to project personnel.
4,421 job-months (7.3%) - C. Design or develop automated testing tools.
3,691 job-months (6.1%) - D. Supervise, train, or mentor junior team members.
3,199 job-months (5.3%) - E. Assign duties to other staff and give instructions regarding work methods and routines.
2,633 job-months (4.3%) - F. Review designs, codes, test plans, or documentation to ensure quality.
2,018 job-months (3.3%)
Together within postings
How often are two tasks detected in the same sampled posting? Each row counts a pair within postings, then pools those counts across job-months.
| Task pair | Job-months with both | % of sample |
|---|---|---|
| A + B | 232 | 0.38% |
| A + C | 332 | 0.55% |
| A + D | 280 | 0.46% |
| A + E | 154 | 0.25% |
| A + F | 423 | 0.70% |
| B + C | 323 | 0.53% |
| B + D | 315 | 0.52% |
| B + E | 83 | 0.14% |
| B + F | 175 | 0.29% |
| C + D | 178 | 0.29% |
| C + E | 123 | 0.20% |
| C + F | 182 | 0.30% |
| D + E | 176 | 0.29% |
| D + F | 139 | 0.23% |
| E + F | 39 | 0.06% |
Pairwise evidence is not a complete job bundle. Even if every pair occurs, these counts do not tell us how often all six tasks occur together. Complete task-identity bundles for individual jobs are not retained in the local aggregate files. These are detected task assignments, not verified descriptions of every duty.
Relationship to the principal analysis. The between-occupation model uses design-weighted individual task frequencies transformed with square-root RCA and compared by cosine similarity. It does not use the pair counts or this six-task selection. The left panel uses the already released coordinates from the full production RCA profile. The right panel uses raw within-posting pair counts; its task percentages use the sampled job-month denominator. These are different measures, not two estimates of the same link strength.
Example selection and provenance
One of eight occupations with a complete pair-count grid among 30 originally selected tasks. We show the six tasks with the highest current design-weighted posting frequency among publicly permitted, retained tasks in that original grid. All 15 pairs are reported, including zeros if present. This selection is illustrative, not representative. Counts are sampled job-months across October 2024–September 2025, not unique annual vacancies.
The extraction iterates over each sampled job’s distinct task set, counts every unordered pair once, then pools counts across months. The example checks positive counts against the corrected public pair data and checks each pair against both task marginals. Source: case_core_pairs_complete.csv; extraction: run_occupation_first_extract.py, aggregate_month processing. Left-panel coordinates are read from the production profile at build time; only the six displayed, permitted coordinates are included on this page. No restricted task denominators or identities are added. These checks verify consistency with the available aggregates; they are not a fresh database extraction.
Model construction: code, data and review decisions
Worked notebooks · Paper code and released scores · Excluded task–occupation links (CSV) · Methods and audit
Figure F: what the percentage counts
Tasks allocated through detailed work activities (Figure 1F)
Figure 1F maps posting-derived task incidences through O*NET 27.2 detailed work activities (DWAs). Incidence for a task linked to multiple DWAs is divided equally across those activities. At the six-digit SOC level, 87.1% of the allocated design-weighted incidence falls outside O*NET’s occupational activity assignments. The calculation precedes minimum-support filtering and occupation-task review exclusions. This is a difference in assignments, not an error rate.
How AI is used
- Retrieval: TitleMatch and TaskMatch use embedding similarity to match input text to existing occupation codes and O*NET task IDs. Retrieved task statements come from the taxonomy.
- Generative-model review: selected posting task–occupation assignments and the OSHA task bundle use model judgments. Their review procedures and decisions are documented with each analysis; those judgments are not independent expert validation.
- Website and audit: Codex helped create the layout, code and explanatory text, and performed the disclosed computational audits. Tables and charts report calculations from the linked inputs. Audit checks do not replace researcher review.
Use the linked code, source versions and review records to inspect each step. Some construction inputs require authorized access.
Website layout and explanatory text were prepared with generative AI (Codex). Data sources, model methods and code.
The National Labor Exchange (NLx) Data Trust bears no responsibility for the analyses or interpretations of the data presented here. The opinions expressed herein, including any implications for policy, are those of the authors and not of the NLx Data Trust members or others.