External validation: shared work and observed mobility
← Occupation navigator · Skills and abilities
Updated 16 October 2026 · manuscript and SI comparisons
What this comparison asks
How well do related-occupation recommendations recover observed occupational moves? Mean recovery averages recovery equally across starting occupations. SST uses career-history transition shares; EMSI uses overlap with its ranked list of twenty destinations. Their percentages answer different questions; compare models within each benchmark.
What this comparison can and cannot establish
Do posting-derived task profiles identify related occupations that also appear in independently sourced records of occupational movement? Observed transitions are one external check on shared work, not a complete measure of opportunity. Hiring practices, credentials, geography, wages, and other skills-related constraints—including required knowledge and abilities—can affect which occupational moves occur. Absence from a transition record is not evidence that a transition is impossible. Similarity is not a transition probability or a claim of readiness for a job.
Tasks are a form of skills data: they describe concrete activities performed at work. Task descriptions, broader skill categories, knowledge, and abilities capture distinct aspects of work requirements. This analysis distinguishes these representations and tests how well each identifies related occupations. Task vectors use the normalized square root of RCA (occupation task share divided by pooled task share), without an RCA cutoff. This page compares task and O*NET representations; it does not use transition frequency to train or weight the navigator's recommendations. Results do not establish better placements or the causal effectiveness of employment services.
Main results · Up to 20 recommendations · USAJOBS replication · Survey detail · Methods · Sources
Benchmark results
Both models use the same number of destinations for every origin and benchmark: the smaller available model list, capped at 20. Each list retains its original ranking. The common length is determined without consulting observed transition outcomes. Zero-length pairs are excluded from equal-length summaries (none in the six primary benchmarks).
| Benchmark | Origins | Posting tasks: mean recovery | O*NET lists: mean recovery | Higher recovery | Equal recovery | Lower recovery |
|---|---|---|---|---|---|---|
| SST | 604 | 15.68% | 11.22% | 467/604 (77.3%) | 0/604 (0.0%) | 137/604 (22.7%) |
| EMSI | 429 | 27.41% | 17.75% | 319/429 (74.4%) | 55/429 (12.8%) | 55/429 (12.8%) |
Higher, equal and lower compare the posting-task model with O*NET for each origin. These counts are descriptive, not significance tests.
Across SST and EMSI: 668 unique occupations. Posting tasks are favored for 495 (74.1%); O*NET for 130 (19.5%); 8 (1.2%) are tied and 35 (5.2%) have mixed evidence. An occupation favors a model if that model wins in at least one benchmark and loses in neither. Mixed means opposite directions across the two benchmarks.
Repeat the workflow with public USAJOBS data
The public federal-posting database, three starter notebooks, figure and full benchmark results are available together. This repeats the workflow with a different corpus; it does not numerically reproduce NLx.
All six benchmarks: mean recovery and recommendation lengths
| Benchmark | Origins | O*NET available list length | Posting tasks (%) | O*NET published (%) | More recovery: count (%) | Ties |
|---|---|---|---|---|---|---|
| SST | 604 | 15.73 | 15.68 | 11.22 | 467/604 (77.3%) | 0 |
| EMSI | 429 | 17.67 | 27.41 | 17.75 | 319/429 (74.4%) | 55 |
| CPS + SIPP | 109 | 14.31 | 21.15 | 19.22 | 70/109 (64.2%) | 1 |
| NLSY97 + PSID · 3 years | 77 | 12.87 | 14.79 | 12.31 | 52/77 (67.5%) | 3 |
| NLSY97 + PSID · 5 years | 76 | 12.87 | 13.10 | 10.55 | 52/76 (68.4%) | 2 |
| NLSY97 + PSID · 10 years | 65 | 12.80 | 10.47 | 8.26 | 44/65 (67.7%) | 3 |
The denominator remains the full observed transition mass for SST and surveys, and the original twenty ranked destinations for EMSI. We do not replace EMSI's denominator with the shortened recommendation count. Origins receive equal weight. “More recovery” counts strictly positive paired differences above 1e-12; ties remain in the denominator. This compares ranking performance at equal recommendation lengths; it does not remove crosswalk or candidate-population limitations.
Worked RN example: destinations, mapping and arithmetic
Worked example: Registered Nurses
| Benchmark | Original lengths: tasks / O*NET | Equal length | Original scores: tasks / O*NET | Equal-length scores: tasks / O*NET |
|---|---|---|---|---|
| SST | 20 / 19 | 19 each | 27.5% / 9.5% | 27.4% / 9.5% |
| EMSI | 20 / 18 | 18 each | 30.0% / 20.0% | 30.0% / 20.0% |
| CPS + SIPP | 20 / 9 | 9 each | 38.9% / 27.5% | 27.9% / 27.5% |
Why SST uses 19 rather than 20. The broader SOC6 illustration and the SST evaluation use different candidate populations. Four destinations in the broader published-O*NET top 20 cannot pass our one-to-one SOC2010–SOC2018 crosswalk: Paramedics, Emergency Medical Technicians, Pediatric Surgeons, and Neurologists. Three destinations below that broader top-20 cutoff remain eligible—Psychologists, All Other; Obstetricians and Gynecologists; and Physical Therapists—so the full mapped O*NET ranking supplies 19 distinct destinations. The ranking combines published lists for the five RN specialties; this is not simply deleting one entry from the generic RN occupation's original list.
To match that length, we retain the first 19 posting-task recommendations and drop its rank-20 destination, Respiratory Therapists (29-1126). Respiratory Therapists are still an eligible SST destination and remain in O*NET's list. We remove them only from the posting-task recommendation list for this equal-length comparison—not from observed outcomes or the candidate pool. That changes RN posting-task capture from 27.50% to 27.42%. The rule is based on model ranks and available list lengths, not on whether that last recommendation helps the score.
Why CPS + SIPP uses nine. Its Census occupation groups are broader than SST's SOC occupations. Tracking the 19 eligible SST O*NET destinations into the CPS + SIPP candidate mapping gives:
| Mapping step | Effect on the 19 destinations |
|---|---|
| Six physician occupations fall in “Physicians and surgeons,” a Census group excluded by our conservative SOC-vintage crosswalk rule | −6 |
| “Psychologists, All Other” and “Psychiatric Technicians” fall in other excluded Census groups with non-one-to-one SOC-vintage mappings | −2 |
| “Mental Health Counselors” falls in a broader counselors group not fully represented in the fixed model population | −1 |
| “Nursing Assistants” and “Psychiatric Aides” merge into one nursing, psychiatric, and home health aides category | −1 |
| Distinct eligible destinations remaining | 9 |
These are classification and candidate-population restrictions, not a low destination-frequency threshold or evidence that those occupations are absent from the surveys. The mapping rule requires unambiguous SOC-vintage correspondences and complete representation of each retained Census group. It is a conservative analysis choice; nine is not an inherent limit of O*NET. We then apply the equal-length rule separately within this benchmark.
For CPS + SIPP, start with 400 eligible candidate destinations and the same 262 observed RN occupation-change records. After occupation mapping, the posting-task ranking supplies 20 recommendations and published O*NET supplies nine. We take the first nine destinations from each ranking, without using transition outcomes to choose them.
For each list, add the supplied survey weights of RN transitions into its recommended destinations, then divide by the weight of all 262 RN changes, including transitions to destinations outside the candidate pool. These percentages are weighted shares, not counts divided by 262.
- Posting tasks, first 9: capture 27.85% of observed transition weight.
- Published O*NET, all 9: capture 27.45%.
- Difference: 0.40 percentage points, compared with 11.49 points under the original 20-versus-9 comparison.
The additional posting-task recommendations at ranks 10–20 capture another 11.09 percentage points. They explain almost all of the larger original RN CPS + SIPP gap. Equal lengths reveal a small descriptive difference here, not evidence of a statistically significant advantage for RN. SST and EMSI retain larger gaps in the table above.
See the nine destinations and their contributions
| Rank | Posting-task destination | Captured share (tasks) | O*NET destination | Captured share (O*NET) |
|---|---|---|---|---|
| 1 | Nurse practitioners | 4.22% | Nurse practitioners | 4.22% |
| 2 | Licensed practical and licensed vocational nurses | 3.68% | Nurse midwives | 0.00% |
| 3 | Medical and health services managers | 3.88% | Licensed practical and licensed vocational nurses | 3.68% |
| 4 | Medical assistants | 0.64% | Respiratory therapists | 3.62% |
| 5 | Pharmacists | 0.00% | Nurse anesthetists | 0.50% |
| 6 | Nursing, psychiatric, and home health aides | 9.67% | Physician assistants | 1.01% |
| 7 | Physician assistants | 1.01% | Nursing, psychiatric, and home health aides | 9.67% |
| 8 | Physical therapists | 4.76% | Computer systems analysts | 0.00% |
| 9 | Nurse midwives | 0.00% | Physical therapists | 4.76% |
Each entry is that destination's share of all observed RN transition weight. Columns sum to 27.85% and 27.45%, subject to rounding. A zero indicates no observed RN transition weight to that destination in this sample. Occupation names are the broader Census categories used for the survey comparison.
How to read these comparisons
Mean capture / recovery asks how much observed mobility the recommendations recover, averaged equally across origins. Share of origins with more recovery asks how widespread a positive difference is; a tiny positive difference and a large one each count as one origin. Ties stay in the denominator. The two panels complement each other: one shows the average amount recovered, the other the prevalence of greater recovery.
Alternative rule: up to 20 recommendations
Original maximum of 20 recommendations
| Benchmark | Origins | O*NET available list length | Posting tasks (%) | O*NET published (%) | More recovery: count (%) | Ties |
|---|---|---|---|---|---|---|
| SST | 604 | 15.73 | 19.00 | 11.22 | 538/604 (89.1%) | 0 |
| EMSI | 429 | 17.67 | 30.29 | 17.75 | 347/429 (80.9%) | 43 |
| CPS + SIPP | 109 | 14.31 | 25.45 | 19.22 | 86/109 (78.9%) | 0 |
| NLSY97 + PSID · 3 years | 77 | 12.87 | 18.69 | 12.31 | 68/77 (88.3%) | 1 |
| NLSY97 + PSID · 5 years | 76 | 12.87 | 17.33 | 10.55 | 69/76 (90.8%) | 0 |
| NLSY97 + PSID · 10 years | 65 | 12.80 | 14.69 | 8.26 | 60/65 (92.3%) | 0 |
Here each model may return up to twenty destinations. Counts can differ even though the maximum is the same. Results are reported separately for each benchmark.
Results for both recommendation rules · Occupation-level paired results, including separate surveys
Sensitivity to occupation-task filtering
How does filtering affect the results?
This selector compares benchmark recovery. Occupation recommendations and task examples elsewhere on this page retain the paper’s profiles. Candidate occupations and recommendation lengths are fixed across variants; RCA is recomputed for each profile.
| Benchmark | Occupations / groups | Posting profiles · Mean recovery | O*NET · Mean recovery |
|---|
Filter comparison. “No exclusions” restores occupation-task profile links; it does not remove the OSHA evidence review. The paper’s filters retain eligible unreviewed links and canonical assignments. An observation is a sampled job-month.
Comparison results · CSV · Definitions and scope · Analysis code
Which source is which?
| Source | Observations and score | Interpretation |
|---|---|---|
| SST — Schubert, Stansbury and Taska | Career-history transitions; captured transition share | Separate historical mobility benchmark |
| CTOT–EMSI | Career-profile occupation-pair ranks; recovery of the original top 20 destinations | Rank recovery, not a worker-flow percentage; the DOL dashboard transition summary is derivative |
| CPS | CTOT occupation-change records, supplied weights | Distinct survey source; released ID cannot identify respondents |
| SIPP | CTOT job-change records, supplied weights | Same-occupation records excluded; repeated records possible |
| NLSY97 | Start-to-end occupations at 3, 5 and 10 years | Same survey across overlapping horizons |
| PSID | Start-to-end occupations at 3, 5 and 10 years | Same survey across overlapping horizons |
CTOT is the Department of Labor's Career Trajectories and Occupational Transitions study. EMSI is not a combination of the surveys. Four distinct survey sources do not become eight independent replications when horizons are repeated. All share CTOT processing and harmonization; observations and people can recur.
Survey benchmarks: coverage and pooling
Survey detail and pooling
CPS and SIPP are pooled by stacking their records and preserving supplied weights. NLSY97 and PSID are pooled separately for each horizon. This is not an average of separate-source headline percentages. Pooling increases eligible origins but does not resolve crosswalk exclusions, unequal source contributions or repeated observations. Trajectory weights can be dominated by NLSY97; they are not an equal-source consensus.
| Pool | Origins | Change records | Mapped destination weight (%) |
|---|---|---|---|
| CPS_SIPP | 109 | 26812 | 72.18 |
| NLSY97_PSID_3y | 77 | 15984 | 56.97 |
| NLSY97_PSID_5y | 76 | 16742 | 54.55 |
| NLSY97_PSID_10y | 65 | 13688 | 51.04 |
The table reports retained change records, not unique people. Mapping coverage is the share of evaluated destination weight represented in the model candidate pool. Unmapped destinations remain in the overall capture denominator. These are CTOT's supplied-weight analyses, not newly recalibrated national survey estimates.
Separate-source sensitivity
| dataset | Origins | O*NET published (%) | O*NET quantitative (%) | Posting tasks (%) |
|---|---|---|---|---|
| CPS | 62 | 23.42 | 26.58 | 28.97 |
| NLSY97_10y | 55 | 7.70 | 10.35 | 14.39 |
| NLSY97_3y | 65 | 11.76 | 14.00 | 18.22 |
| NLSY97_5y | 66 | 9.81 | 12.39 | 16.98 |
| PSID_10y | 10 | 12.82 | 16.15 | 17.44 |
| PSID_3y | 14 | 18.26 | 21.47 | 22.90 |
| PSID_5y | 16 | 16.81 | 18.93 | 21.76 |
| SIPP | 82 | 15.70 | 17.41 | 23.99 |
Keeping the separate sources visible prevents pooling from hiding heterogeneity. The primary survey comparison requires at least 40 observed changes per origin; this is our analysis threshold, not a claim that every survey record or occupation is represented. Different survey designs, horizons, occupations and retained samples can change absolute recovery. These results do not justify a blanket conclusion that surveys are poor benchmarks.
Code and worked example
Three worked USAJOBS notebooks · Paper figure and workbook. USAJOBS figure, benchmark results and downloads. The USAJOBS example uses a different corpus; the results on this page are the paper’s NLx analysis.
Methods and limits
Posting profiles retain the corrected research task assignments with a support floor of five sampled job-months and the frozen representation. Similarities are aggregated to benchmark occupation codes using the audited mappings. The original task/DWA crosswalks are labeled O*NET 27.2; the official 2025 relatedness algorithm used O*NET 30.0 inputs, and the published comparator uses released 31.0 lists. Do not interpret this as a synchronized latest-vintage experiment.
CPS/SIPP use Census2010 codes. The trajectories use Census2002, conservatively mapped through accepted unchanged-scope correspondences. Self-transitions are excluded, ties are deterministic, and outcomes outside the candidate pool remain in overall denominators. For example, RN has 262 eligible CPS–SIPP changes (Census2010 3255), but its older Census2002 3130 code is excluded by the audited correspondence, so the three-year focal RN result is unavailable—not zero.
RN's published O*NET list contains 20 detailed occupations, but Census aggregation absorbs four RN specialties into the origin itself. Conservative SOC-vintage crosswalk rules exclude physician and EMT/paramedic groups with non-one-to-one mappings. Combining the lists of all five RN specialties leaves nine eligible Census destinations. This is a mapping restriction, not a low destination-frequency threshold. CPS–SIPP comparisons use Census groups; for example, nursing assistants and orderlies share the broader nursing, psychiatric, and home health aides category. A missing badge can reflect mapping or a rank outside the observed top twenty.
Descriptive intervals condition on fixed profiles, outcomes, weights and crosswalks; they are not full survey-design intervals and omit several sources of measurement uncertainty. Overlapping horizons are not independent evidence. Posting data describe advertised demand, which differs from all work actually performed. Shared tasks can suggest a connection while licensing, training and other requirements still matter.
Skills and abilities
Compare tasks, work activities, skills and abilities using the representations listed in SI Tables S17 and S19.
Sources and downloads
- Aggregate main results, separate survey results, and . Downloads contain occupation-level summary statistics, no individual survey records or restricted task assignments.
- SST occupational-mobility data.
- DOL CTOT public-use documentation and Building Better Pathways methods report.
- Original CTOT data landing page. Some downloads required archive recovery. The original DOL study and datasets are the sources; the Internet Archive supplied recovered downloads. File horizons and recovered snapshots are recorded in the research provenance. Supplied CPS/SIPP and three-year files matched recovered copies; the supplied five-year file matched the recovered analytical fields.
- Official Census occupation crosswalk.
- O*NET Related Occupations 2025 methodology. O*NET data are developed by the U.S. Department of Labor/Employment and Training Administration and used under CC BY 4.0; these transformed research comparisons are not an O*NET endorsement.
The public edition presents aggregate evaluation results from the full research model. Its task explorer reveals O*NET-linked, reviewed, or retained support-above-200 assignments, with additional parent/specialty complementary suppression; other assignments remain anonymous. This page adds no task-level disclosure.
Benchmark methods and audit
Construction choices and audit
Read the benchmark construction and audit disclosure: populations, survey pooling, crosswalks, denominators and reproduction limits. OpenAI Codex performed the 6 October 2026 audit and generated its reports; it is not independent human review.
Model construction: code, data and review decisions
Worked notebooks · Paper code and released scores · Excluded task–occupation links (CSV) · Methods and audit