Variable Agreement Deutsch- December 20, 2020
There are several operational definitions of “inter-rated reliability” that reflect different views on what a reliable agreement between advisors is.  There are three operational definitions of the agreement: “explanatory notes” provide detailed information on the variables examined in the labour force survey. For each variable, it informs the user of the periodicity, indicates a brief description and purpose. Definition and implementation rules are also presented. Krippendorffs Alpha is a versatile statistic that evaluates the agreement between observers who categorize, evaluate or measure a certain number of objects against the values of a variable. It generalizes several specialized agreement coefficients by accepting any number of observers applicable to nominal, ordinal, interval and proportional levels of measurement, capable of processing missing and corrected data for small sample sizes. The International Model Occupational Classification (CITP) developed by the International Labour Organization is used to measure the professional status of workers. The AKE uses four-digit ISCO for the main work and three-digit for the previous occupation (the last figure is voluntary in both cases). The classification was last revised in 2008 (CITP 08). The Labour Force Survey has been using the revised classification (CITP-08) since 2011; ISCO-88 (COM) was used until 2010.
For more information on classification, see: Metadata Eurostat (CITP) and explanation of the changes: comparability between CITP-08 and CITP-88. More detailed information on the coding of FDI variables can be found using the CITP over time in the basic variables (use manual) and explanatory notes. The ESeG is a derivative classification that brings together people with similar economic, social and cultural characteristics throughout the European Union, based solely on fundamental social variables, in order to ensure comfortable use in all social surveys providing comparable results. The main social variables are the ILO`s labour status, employment status, employment in employment (according to CITP-08) and self-reported labour status. The classification and detailed explanations are available on the EseG page on Eurostat`s classification server, LE RAMON. Therefore, the common probability of an agreement will remain high, even in the absence of an “intrinsic” agreement between the councillors. A useful interrater reliability coefficient (a) is expected to be close to 0 if there is no “intrinsic” agreement and (b) increased if the “intrinsic” agreement rate improves. Most probability-adjusted match coefficients achieve the first objective. However, the second objective is not achieved by many well-known measures that correct the odds.
 There are a number of statistics that can be used to determine the reliability of inter-advisors. Different statistics are adapted to different types of measurement. Some options are the common probability of an agreement, Cohens Kappa, Scott`s pi and the Fleiss`Kappa associated with it, inter-rate correlation, correlation coefficient, intra-class correlation and Krippendorff alpha. The country codes are based on ISO 3166 (international Organisation of Standardisation – alpha-2 format), with two main exceptions for EU countries, Greece and the United Kingdom, which are coded in EL and the UNITED Kingdom respectively.