C. Finding Emerging Skills

When standard taxonomies do not contain task or skill items for emerging technologies, labor market signals are missed. This finding asks whether postings can supply emerging task and skill statements missing from standard taxonomies, and whether those statements can support informative measures of the technological frontier.

AIMatch starts from posting sentences, places them in a task and skill architecture, and scores each statement for AI relatedness. The current experimental taxonomy contains 19,660 base-level statements. It is not presented as a finished ontology. It is a provisional map: a structured set of posting-derived statements that can be validated, revised, and used to measure where AI-related work is appearing.

This construction uses the TaxonomyBuilder approach for organizing posting-derived task and skill sentences into inspectable maps. See Meisenbacher and Norlander (2026), “Building a Custom Taxonomy of AI Skills and Tasks from the Ground Up with Job Postings”, Proceedings of the 2nd Workshop on Customizable NLP (CustomNLP4U), forthcoming, arXiv:2605.21029.

Figure BNM-1. Taxonomy Builder Seeded With Posting Sentences

What Was Tested

For each statement, AI class-vector scores are built from cosine similarity to AI keyword classes and averaged across three embedding models. Strict and lenient AI classes then use agreement across models to separate clearer AI statements from AI-adjacent language.

The validation compares randomly selected pairs of USAJOBS posting text inside two strata:

• Strict AI: postings with at least one strict AI code.
• Lenient / AI-adjacent: postings with at least three lenient codes and no strict codes.

Within each stratum, pairs are grouped by AI-score gap: large gaps (>=0.10), medium gaps (0.05 to 0.10), and very small gaps below 0.03. Claude Sonnet 4.6 and OpenAI GPT-4o ranked which posting was more AI-related, or in the lenient stratum, which was closer to the AI / ML frontier.

Validation Results

The sign tests reject random ranking for large and medium score gaps, and fail to reject for very small gaps. That is the useful measurement claim: the AI score distinguishes postings that differ by about 0.05 or more, but not reliably below about 0.03.

The p-values in the summary are exact one-sided binomial sign-test p-values. They are not identical in the source files; very small values can look similar if rounded too aggressively. The audit tables below keep scientific notation and expose the full gatekeeping tests.

Table BNM-1. Posting-Level AI Score Validation by Judge

Table BNM-2. AI Score Resolution Checks

Aggregation Reliability

The pairwise validation is about document-level resolution. For group comparisons, reliability improves with sample size. The aggregation checks compare groups of postings by occupation code and by agency, not individual occupations or agencies. The estimates differ between occupation and agency groups because the same posting-level score has different within-group variation, between-group variation, and typical group size under each grouping. Occupation groups separate AI intensity more strongly than agencies for the strict-share measure, so the occupation-group reliability estimate is higher.

The compact formula is: reliability at group size n = n * ICC / (1 + (n - 1) * ICC), where ICC = between variance / (between variance + within variance). The minimum detectable two-group difference is 1.96 * within SD * sqrt(2 / n). With 300 observations in each group, the strict-share detectable difference is about 0.014, or 1.4 percentage points, for both occupation and agency comparisons. The displayed reliability and MDD use duplicate-content-adjusted effective n.

Table BNM-4. Aggregate AI Score Reliability Summary

External Check

As a convergent benchmark, industry-level AIMatch signals are compared with Census Business Trends and Outlook Survey AI-use rates. The rank correlation is positive for mean AI scores and stronger for strict AI counts.

Table BNM-6. BTOS Correlation Summary

Figure BNM-2. BTOS Firm AI Use vs. NLx Strict AI Intensity

Point labels show abbreviated NAICS-2 industry codes, such as 51 Info, 52 Fin, and 54 Prof/Sci. This is a cross-sectional comparison between the latest Census BTOS sector AI-use wave and NLx NAICS-2 AIMatch averages from the public aggregate ending in September 2025. A time-series validation is not possible in this public package because nonmissing BTOS responses to the current AI-use question begin in November 2025, after the NLx public aggregate ends. The fitted line is computed from the same 17 NAICS-2 sector points with ordinary least squares; it is a visual guide, not a separate validation test.

Table BNM-7. BTOS Sector Benchmark Rows

What This Supports

The evidence supports four conservative uses. First, posting sentences can reveal emerging task and skill language that standard taxonomies miss. Second, validated AI scores can distinguish meaningful posting-level differences above the observed resolution floor. Third, group means can support meaningful occupation, agency, or industry comparisons when each group has enough postings, especially above a few hundred observations. Fourth, the same statement-level map can be used as a substrate for future human-reviewed taxonomy development, where statements are retained, merged, relabeled, or rejected as the frontier changes.

Evidence Path

The validation tables use the public JAAT AI-intensity aggregate sources shown on this page. Release checks and public artifact paths are listed in Agreement And Audit Trail; broader limits of posting-derived taxonomies are summarized in What Postings And Taxonomies Can And Cannot Say.

Audit Downloads

The blocks below show the page-level SQL used for each table or figure on this page. They are collapsed so the main evidence stays readable.

You will be given two job postings, each identified by a unique integer ID.

Rank them by how AI-related the work is - the degree to which the role involves
artificial intelligence, machine learning, data science, advanced analytics, or
intelligent automation, including the software and data engineering such systems
are built on, and soft skills specifically required for AI/ML roles. Do NOT count
general use of computers or office software, or operating or repairing technical
or electronic equipment that has no AI or machine-learning component.

This is always a relative judgment: even if neither posting is strongly
AI-related, decide which one is closer to AI/ML work, based on the posting text.

Answer with a comma-separated Python list of the two posting IDs, most AI-related
first. Output only that list.

You will be given two job postings, each identified by a unique integer ID.

Both involve technology to some degree. Rank them by how close the work is to the
frontier of artificial intelligence and machine learning - i.e., which posting's
tasks, skills, and requirements are nearer to building, applying, or supporting
AI/ML, data science, advanced analytics, or intelligent automation, as opposed to
more general or conventional technology work.

This is always a relative judgment: pick the posting whose work is closer to AI/ML,
even if neither is strongly AI. Base your decision on the posting text.

Answer with a comma-separated Python list of the two posting IDs, closest to the
AI frontier first. Output only that list.