Lightcast is a labour market data modelling company based in the United States. We provide our customers with accurate and up-to-date labour market data to help them make informed business and marketing decisions. In this document, we will give a basic overview of where our core structural labour market data comes from, how we assemble it, and the checks we use to make sure that our data is the clearest and most reliable source of LMI available.

The implicit question for any data company is always “Why should I trust your data?” Lightcast data is created from a collection of government sources that are combined to provide multi-layered cross-checking. These sources are collected and combined to create Lightcast’ industry and occupation data, as well as staffing patterns used to connect the two. Twice a year, Lightcast updates its data with more than 20 million data points describing labour market conditions across the United Kingdom. It is the most accurate and reliable source of labour market information available today.

Our sources include the following datasets:

Lightcast provides industry, staffing, and occupation data for two types, or classes, of worker—employees and proprietors. Employees are workers who are on the payroll of a business. We also provide employment figures for proprietors, a subset of self-employment jobs that are registered for VAT or PAYE taxation. This data is derived from the BRES survey, and help fill out the landscape of employment in the UK. Proprietors cover about one million additional jobs, and a quarter of all self-employment in the UK.

Lightcast industry data comprise 660 industries (at the lowest level) in 385 detailed geographies (Local Administrative Units). We provide job count data by industry from 2003 to 9 years forward from the current year.

The process for creating industry data, for both employees and proprietors, can be divided into three sections:

Lightcast gathers historic industry employment data primarily from the Annual Business Inquiry (ABI) and the Business Register Employment Survey (BRES). These two are supplemented by other sources but together form the backbone of Lightcast’ industry data. Both ABI and BRES are suppressed datasets, so the first step in creating Lightcast data is to fill in the gaps left by the suppressed data points. We use year-to-year and cross-set comparisons to come up with initial estimates, which we then adjust for rounding error. ABI does not cover employment numbers in agriculture, financial activities, or public-sector employment in health, education and public administration, so we supplement ABI with other government sets to make up for the lacking data. These sets include data from the Census, the former Department for Children, Schools and Families (DCSF), and DEFRA. Together with these supplemental datasets, ABI forms our historic industry data from 2003 to 2007.

The primary source of our historic data from 2008 to the present is BRES. While more complete in its coverage of employment sectors than ABI, BRES does require supplemental agricultural data, which is gathered from Workforce Job Series levels which have been expanded geographically. This expansion makes use of data from DEFRA, a Welsh agricultural dataset called Agricultural Small Area Statistics, and the Annual Population Survey. Together, these datasets fill in the agricultural gaps in the BRES data. Because the results of BRES are not released until a year after the actual survey takes place, we report the current year data by projecting our historic data forward one year in full detail, using the projection methodology outlined in the next section. Workforce Job Series data is released three months after it is collected, as opposed to a year after collection for BRES. Because of this, we adjust our projection levels to match percent change in the WJS between the time BRES was released and the most recent WJS release. This completes our historic industry employment process.

Lightcast projects industry employment totals using the average of three linear regressions. These linear regressions are based on three segments of data that correspond to the last ten years, five years, and three years, respectively. All of our industries are projected at their most granular level. We average the regressions, forming a single trend which we damp over time to curb excessive increases or decreases in employment totals

Lightcast derives earnings data for industries from the Annual Survey of Hours and Earnings (ASHE). Figures from this survey include average annual industry earnings per worker.

Lightcast staffing data comprises 660 industries and 412 occupations across 12 geographies, and contains percentages of occupation employment by industry from 2003 and 9 years forward from the current year.

To construct staffing data, Lightcast uses three main datasets:

Lightcast Staffing data is created in three steps:

We create occupation job counts for each NUTS 1 region by combining the LFS staffing data with Lightcast industry data. These occupation job counts are projected for all future periods. The occupation job count is scaled to our Lightcast industry job count for each region, and the two together form the margins of our staffing matrix. For each region, we create the regional staffing by adjusting our national industry by occupation percentages, derived LFS microdata, and Lightcast industry data to match the regional margin totals.

Lightcast provides business counts by industry and size band. Business data is produced by suppressing rounded Business Counts data.

Lightcast occupation data comprises 412 occupations across 385 detailed geographies (Local Administrative Units). The jobs data covers the years from 2003 to 9 years from the current year, and the earnings data is from the latest available year.

To construct the occupation data, Lightcast uses three datasets:

To create occupation data, we first distribute the most granular regional industry data according to the staffing patterns described above. This distribution forms 2003 to 9 years from the current year employment levels across the United Kingdom. Next, we take NUTS 1 level earnings by occupation from the ASHE survey and apply them to all of the regional data. This completes the occupation data.

Once the industry, staffing, and occupation data are complete, they are incorporated into Analyst, Lightcast’s labour market tool, which gives users unprecedented access to detailed labour market information. This information is used to build strategies for businesses and to aid in course planning for FE and HE colleges.

Lightcast combines 2001 and 2011 Census population data and midyear population estimates from 2001 to the most recent year to create population counts by area, age, gender, and ethnicity. The combined data is then projected and adjusted to published population projections. The population estimates and projections sources are published by the Office of National Statistics, Welsh Government, and National Records Scotland.

Lightcast uses data from the Labour Force Survey for both the employed and self-employed workforce to create industry and occupation employment data by area, age, and gender. The data is then adjusted to match Lightcast’s industry and occupation data.